Remove pointless "vile hack" that can cause the build step to fail when some extension modules can't be imported.

See issue #5309 for the build failures, issue #458343 for the original motivation.