Implemented PEP 405 (Python virtual environments).

Doc/library/development.rst

@@ -23,3 +23,4 @@ The list of modules described in this chapter is:
    unittest.mock-examples.rst
    2to3.rst
    test.rst
+   venv.rst

Doc/library/sys.rst

@@ -29,6 +29,26 @@ always available.
    command line, see the :mod:`fileinput` module.
 
 
+.. data:: base_exec_prefix
+
+   Set during Python startup, before ``site.py`` is run, to the same value as
+   :data:`exec_prefix`. If not running in a virtual environment, the values
+   will stay the same; if ``site.py`` finds that a virtual environment is in
+   use, the values of :data:`prefix` and :data:`exec_prefix` will be changed to
+   point to the virtual environment, whereas :data:`base_prefix` and
+   :data:`base_exec_prefix` will remain pointing to the base Python
+   installation (the one which the virtual environment was created from).
+
+.. data:: base_prefix
+
+   Set during Python startup, before ``site.py`` is run, to the same value as
+   :data:`prefix`. If not running in a virtual environment, the values
+   will stay the same; if ``site.py`` finds that a virtual environment is in
+   use, the values of :data:`prefix` and :data:`exec_prefix` will be changed to
+   point to the virtual environment, whereas :data:`base_prefix` and
+   :data:`base_exec_prefix` will remain pointing to the base Python
+   installation (the one which the virtual environment was created from).
+
 .. data:: byteorder
 
    An indicator of the native byte order.  This will have the value ``'big'`` on
@@ -199,6 +219,10 @@ always available.
    installed in :file:`{exec_prefix}/lib/python{X.Y}/lib-dynload`, where *X.Y*
    is the version number of Python, for example ``3.2``.
 
+   .. note:: If a virtual environment is in effect, this value will be changed
+      in ``site.py`` to point to the virtual environment. The value for the
+      Python installation will still be available, via :data:`base_exec_prefix`.
+
 
 .. data:: executable
 
@@ -775,6 +799,10 @@ always available.
    stored in :file:`{prefix}/include/python{X.Y}`, where *X.Y* is the version
    number of Python, for example ``3.2``.
 
+   .. note:: If a virtual environment is in effect, this value will be changed
+      in ``site.py`` to point to the virtual environment. The value for the
+      Python installation will still be available, via :data:`base_prefix`.
+
 
 .. data:: ps1
           ps2

Doc/library/venv.rst

+:mod:`venv` --- Creation of virtual environments
+================================================
+
+.. module:: venv
+   :synopsis: Creation of virtual environments.
+.. moduleauthor:: Vinay Sajip <vinay_sajip@yahoo.co.uk>
+.. sectionauthor:: Vinay Sajip <vinay_sajip@yahoo.co.uk>
+
+
+.. index:: pair: Environments; virtual
+
+.. versionadded:: 3.3
+
+**Source code:** :source:`Lib/venv.py`
+
+--------------
+
+The :mod:`venv` module provides support for creating lightweight
+"virtual environments" with their own site directories, optionally
+isolated from system site directories.  Each virtual environment has
+its own Python binary (allowing creation of environments with various
+Python versions) and can have its own independent set of installed
+Python packages in its site directories.
+
+Creating virtual environments
+-----------------------------
+
+Creation of virtual environments is simplest executing the ``pyvenv``
+script::
+
+    pyvenv /path/to/new/virtual/environment
+
+Running this command creates the target directory (creating any parent
+directories that don't exist already) and places a ``pyvenv.cfg`` file
+in it with a ``home`` key pointing to the Python installation the
+command was run from.  It also creates a ``bin`` (or ``Scripts`` on
+Windows) subdirectory containing a copy of the ``python`` binary (or
+binaries, in the case of Windows) and the ``pysetup3`` script (to
+facilitate easy installation of packages from PyPI into the new virtualenv).
+It also creates an (initially empty) ``lib/pythonX.Y/site-packages``
+subdirectory (on Windows, this is ``Lib\site-packages``).
+
+.. highlight:: none
+
+On Windows, you may have to invoke the ``pyvenv`` script as follows, if you
+don't have the relevant PATH and PATHEXT settings::
+
+    c:\Temp>c:\Python33\python c:\Python33\Tools\Scripts\pyvenv.py myenv
+
+or equivalently::
+
+    c:\Temp>c:\Python33\python -m venv myenv
+
+The command, if run with ``-h``, will show the available options::
+
+    usage: pyvenv [-h] [--system-site-packages] [--symlink] [--clear]
+                  ENV_DIR [ENV_DIR ...]
+
+    Creates virtual Python environments in one or more target directories.
+
+    positional arguments:
+      ENV_DIR             A directory to create the environment in.
+
+    optional arguments:
+      -h, --help             show this help message and exit
+      --system-site-packages Give access to the global site-packages dir to the
+                             virtual environment.
+      --symlink              Attempt to symlink rather than copy.
+      --clear                Delete the environment directory if it already exists.
+                             If not specified and the directory exists, an error is
+                             raised.
+
+
+If the target directory already exists an error will be raised, unless
+the ``--clear`` option was provided, in which case the target
+directory will be deleted and virtual environment creation will
+proceed as usual.
+
+The created ``pyvenv.cfg`` file also includes the
+``include-system-site-packages`` key, set to ``true`` if ``venv`` is
+run with the ``--system-site-packages`` option, ``false`` otherwise.
+
+Multiple paths can be given to ``pyvenv``, in which case an identical
+virtualenv will be created, according to the given options, at each
+provided path.
+
+
+API
+---
+
+The high-level method described above makes use of a simple API which provides
+mechanisms for third-party virtual environment creators to customize
+environment creation according to their needs.
+
+The :class:`EnvBuilder` class accepts the following keyword arguments on
+instantiation:
+
+   * ``system_site_packages`` - A Boolean value indicating that the
+     system Python site-packages should be available to the
+     environment (defaults to ``False``).
+
+   * ``clear`` - A Boolean value which, if True, will delete any
+     existing target directory instead of raising an exception
+     (defaults to ``False``).
+
+   * ``symlinks`` - A Boolean value indicating whether to attempt
+     to symlink the Python binary (and any necessary DLLs or other
+     binaries, e.g. ``pythonw.exe``), rather than copying. Defaults to
+     ``True`` on Linux and Unix systems, but ``False`` on Windows and
+     Mac OS X.
+
+The returned env-builder is an object which has a method, ``create``,
+which takes as required argument the path (absolute or relative to the current
+directory) of the target directory which is to contain the virtual environment.
+The ``create`` method will either create the environment in the specified
+directory, or raise an appropriate exception.
+
+Creators of third-party virtual environment tools will be free to use
+the provided ``EnvBuilder`` class as a base class.
+
+.. highlight:: python
+
+The ``venv`` module will also provide a module-level function as a
+convenience::
+
+    def create(env_dir,
+               system_site_packages=False, clear=False, symlinks=False):
+        builder = EnvBuilder(
+            system_site_packages=system_site_packages,
+            clear=clear,
+            symlinks=symlinks)
+        builder.create(env_dir)
+
+The ``create`` method of the ``EnvBuilder`` class illustrates the
+hooks available for subclass customization::
+
+    def create(self, env_dir):
+        """
+        Create a virtualized Python environment in a directory.
+
+        :param env_dir: The target directory to create an environment in.
+
+        """
+        env_dir = os.path.abspath(env_dir)
+        context = self.create_directories(env_dir)
+        self.create_configuration(context)
+        self.setup_python(context)
+        self.setup_scripts(context)
+        self.post_setup(context)
+
+Each of the methods ``create_directories``, ``create_configuration``,
+``setup_python``, ``setup_scripts`` and ``post_setup`` can be
+overridden.  The functions of these methods are:
+
+   * ``create_directories`` - creates the environment directory and
+     all necessary directories, and returns a context object. This is
+     just a holder for attributes (such as paths), for use by the
+     other methods.
+
+   * ``create_configuration`` - creates the ``pyvenv.cfg``
+     configuration file in the environment.
+
+   * ``setup_python`` - creates a copy of the Python executable (and,
+     under Windows, DLLs) in the environment.
+
+   * ``setup_scripts`` - Installs activation scripts appropriate to the
+     platform into the virtual environment.
+
+   * ``post_setup`` - A placeholder method which can be overridden
+     in third party implementations to pre-install packages in the
+     virtual environment or perform other post-creation steps.
+
+In addition, ``EnvBuilder`` provides an ``install_scripts`` utility
+method that can be called from ``setup_scripts`` or ``post_setup`` in
+subclasses to assist in installing custom scripts into the virtual
+environment. The method accepts as arguments the context object (see
+above) and a path to a directory. The directory should contain
+subdirectories "common", "posix", "nt", each containing scripts
+destined for the bin directory in the environment. The contents of
+"common" and the directory corresponding to ``os.name`` are copied
+after some text replacement of placeholders:
+
+* ``__VENV_DIR__`` is replaced with the absolute path of the
+  environment directory.
+
+* ``__VENV_NAME__`` is replaced with the environment name (final path
+  segment of environment directory).
+
+* ``__VENV_BIN_NAME__`` is replaced with the name of the bin directory
+  (either ``bin`` or ``Scripts``).
+
+* ``__VENV_PYTHON__`` is replaced with the absolute path of the
+  environment's executable.

Lib/distutils/sysconfig.py

@@ -18,6 +18,8 @@ from .errors import DistutilsPlatformError
 # These are needed in a couple of spots, so just compute them once.
 PREFIX = os.path.normpath(sys.prefix)
 EXEC_PREFIX = os.path.normpath(sys.exec_prefix)
+BASE_PREFIX = os.path.normpath(sys.base_prefix)
+BASE_EXEC_PREFIX = os.path.normpath(sys.base_exec_prefix)
 
 # Path to the base directory of the project. On Windows the binary may
 # live in project/PCBuild9.  If we're dealing with an x64 Windows build,
@@ -39,11 +41,18 @@ if os.name == "nt" and "\\pcbuild\\amd64" in project_base[-14:].lower():
 # different (hard-wired) directories.
 # Setup.local is available for Makefile builds including VPATH builds,
 # Setup.dist is available on Windows
-def _python_build():
+def _is_python_source_dir(d):
     for fn in ("Setup.dist", "Setup.local"):
-        if os.path.isfile(os.path.join(project_base, "Modules", fn)):
+        if os.path.isfile(os.path.join(d, "Modules", fn)):
             return True
     return False
+_sys_home = getattr(sys, '_home', None)
+if _sys_home and os.name == 'nt' and _sys_home.lower().endswith('pcbuild'):
+    _sys_home = os.path.dirname(_sys_home)
+def _python_build():
+    if _sys_home:
+        return _is_python_source_dir(_sys_home)
+    return _is_python_source_dir(project_base)
 python_build = _python_build()
 
 # Calculate the build qualifier flags if they are defined.  Adding the flags
@@ -74,11 +83,11 @@ def get_python_inc(plat_specific=0, prefix=None):
     otherwise, this is the path to platform-specific header files
     (namely pyconfig.h).
 
-    If 'prefix' is supplied, use it instead of sys.prefix or
-    sys.exec_prefix -- i.e., ignore 'plat_specific'.
+    If 'prefix' is supplied, use it instead of sys.base_prefix or
+    sys.base_exec_prefix -- i.e., ignore 'plat_specific'.
     """
     if prefix is None:
-        prefix = plat_specific and EXEC_PREFIX or PREFIX
+        prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX
     if os.name == "posix":
         if python_build:
             # Assume the executable is in the build directory.  The
@@ -86,11 +95,12 @@ def get_python_inc(plat_specific=0, prefix=None):
             # the build directory may not be the source directory, we
             # must use "srcdir" from the makefile to find the "Include"
             # directory.
-            base = os.path.dirname(os.path.abspath(sys.executable))
+            base = _sys_home or os.path.dirname(os.path.abspath(sys.executable))
             if plat_specific:
                 return base
             else:
-                incdir = os.path.join(get_config_var('srcdir'), 'Include')
+                incdir = os.path.join(_sys_home or get_config_var('srcdir'),
+                                      'Include')
                 return os.path.normpath(incdir)
         python_dir = 'python' + get_python_version() + build_flags
         return os.path.join(prefix, "include", python_dir)
@@ -115,11 +125,14 @@ def get_python_lib(plat_specific=0, standard_lib=0, prefix=None):
     containing standard Python library modules; otherwise, return the
     directory for site-specific modules.
 
-    If 'prefix' is supplied, use it instead of sys.prefix or
-    sys.exec_prefix -- i.e., ignore 'plat_specific'.
+    If 'prefix' is supplied, use it instead of sys.base_prefix or
+    sys.base_exec_prefix -- i.e., ignore 'plat_specific'.
     """
     if prefix is None:
-        prefix = plat_specific and EXEC_PREFIX or PREFIX
+        if standard_lib:
+            prefix = plat_specific and BASE_EXEC_PREFIX or BASE_PREFIX
+        else:
+            prefix = plat_specific and EXEC_PREFIX or PREFIX
 
     if os.name == "posix":
         libpython = os.path.join(prefix,
@@ -232,9 +245,9 @@ def get_config_h_filename():
     """Return full pathname of installed pyconfig.h file."""
     if python_build:
         if os.name == "nt":
-            inc_dir = os.path.join(project_base, "PC")
+            inc_dir = os.path.join(_sys_home or project_base, "PC")
         else:
-            inc_dir = project_base
+            inc_dir = _sys_home or project_base
     else:
         inc_dir = get_python_inc(plat_specific=1)
     if get_python_version() < '2.2':
@@ -248,7 +261,8 @@ def get_config_h_filename():
 def get_makefile_filename():
     """Return full pathname of installed Makefile from the Python build."""
     if python_build:
-        return os.path.join(os.path.dirname(sys.executable), "Makefile")
+        return os.path.join(_sys_home or os.path.dirname(sys.executable),
+                                                         "Makefile")
     lib_dir = get_python_lib(plat_specific=0, standard_lib=1)
     config_file = 'config-{}{}'.format(get_python_version(), build_flags)
     return os.path.join(lib_dir, config_file, 'Makefile')

Lib/gettext.py

@@ -55,7 +55,7 @@ __all__ = ['NullTranslations', 'GNUTranslations', 'Catalog',
            'dgettext', 'dngettext', 'gettext', 'ngettext',
            ]
 
-_default_localedir = os.path.join(sys.prefix, 'share', 'locale')
+_default_localedir = os.path.join(sys.base_prefix, 'share', 'locale')
 
 
 def c2py(plural):

Lib/idlelib/EditorWindow.py

@@ -120,7 +120,7 @@ class EditorWindow(object):
 
     def __init__(self, flist=None, filename=None, key=None, root=None):
         if EditorWindow.help_url is None:
-            dochome =  os.path.join(sys.prefix, 'Doc', 'index.html')
+            dochome =  os.path.join(sys.base_prefix, 'Doc', 'index.html')
             if sys.platform.count('linux'):
                 # look for html docs in a couple of standard places
                 pyver = 'python-docs-' + '%s.%s.%s' % sys.version_info[:3]
@@ -131,13 +131,13 @@ class EditorWindow(object):
                     dochome = os.path.join(basepath, pyver,
                                            'Doc', 'index.html')
             elif sys.platform[:3] == 'win':
-                chmfile = os.path.join(sys.prefix, 'Doc',
+                chmfile = os.path.join(sys.base_prefix, 'Doc',
                                        'Python%s.chm' % _sphinx_version())
                 if os.path.isfile(chmfile):
                     dochome = chmfile
             elif macosxSupport.runningAsOSXApp():
                 # documentation is stored inside the python framework
-                dochome = os.path.join(sys.prefix,
+                dochome = os.path.join(sys.base_prefix,
                         'Resources/English.lproj/Documentation/index.html')
             dochome = os.path.normpath(dochome)
             if os.path.isfile(dochome):

Lib/packaging/command/build_ext.py

@@ -182,7 +182,10 @@ class build_ext(Command):
             # the 'libs' directory is for binary installs - we assume that
             # must be the *native* platform.  But we don't really support
             # cross-compiling via a binary install anyway, so we let it go.
-            self.library_dirs.append(os.path.join(sys.exec_prefix, 'libs'))
+            # Note that we must use sys.base_exec_prefix here rather than
+            # exec_prefix, since the Python libs are not copied to a virtual
+            # environment.
+            self.library_dirs.append(os.path.join(sys.base_exec_prefix, 'libs'))
             if self.debug:
                 self.build_temp = os.path.join(self.build_temp, "Debug")
             else:

Lib/pydoc.py

@@ -369,7 +369,7 @@ class Doc:
 
         docloc = os.environ.get("PYTHONDOCS", self.PYTHONDOCS)
 
-        basedir = os.path.join(sys.exec_prefix, "lib",
+        basedir = os.path.join(sys.base_exec_prefix, "lib",
                                "python%d.%d" %  sys.version_info[:2])
         if (isinstance(object, type(os)) and
             (object.__name__ in ('errno', 'exceptions', 'gc', 'imp',

Lib/site.py

@@ -13,6 +13,19 @@ prefixes directly, as well as with lib/site-packages appended.  The
 resulting directories, if they exist, are appended to sys.path, and
 also inspected for path configuration files.
 
+If a file named "pyvenv.cfg" exists one directory above sys.executable,
+sys.prefix and sys.exec_prefix are set to that directory and
+it is also checked for site-packages and site-python (sys.prefix and
+sys.exec_prefix will always be the "real" prefixes of the Python
+installation). If "pyvenv.cfg" (a bootstrap configuration file) contains
+the key "include-system-site-packages" set to anything other than "false"
+(case-insensitive), the system-level prefixes will still also be
+searched for site-packages; otherwise they won't.
+
+All of the resulting site-specific directories, if they exist, are
+appended to sys.path, and also inspected for path configuration
+files.
+
 A path configuration file is a file whose name has the form
 <package>.pth; its contents are additional directories (one per line)
 to be added to sys.path.  Non-existing directories (or
@@ -54,6 +67,7 @@ ImportError exception, it is silently ignored.
 
 import sys
 import os
+import re
 import builtins
 
 # Prefixes for site-packages; add additional prefixes like /usr/local here
@@ -179,6 +193,7 @@ def addsitedir(sitedir, known_paths=None):
     sitedir, sitedircase = makepath(sitedir)
     if not sitedircase in known_paths:
         sys.path.append(sitedir)        # Add path component
+        known_paths.add(sitedircase)
     try:
         names = os.listdir(sitedir)
     except os.error:
@@ -266,18 +281,21 @@ def addusersitepackages(known_paths):
         addsitedir(user_site, known_paths)
     return known_paths
 
-def getsitepackages():
+def getsitepackages(prefixes=None):
     """Returns a list containing all global site-packages directories
     (and possibly site-python).
 
-    For each directory present in the global ``PREFIXES``, this function
-    will find its `site-packages` subdirectory depending on the system
-    environment, and will return a list of full paths.
+    For each directory present in ``prefixes`` (or the global ``PREFIXES``),
+    this function will find its `site-packages` subdirectory depending on the
+    system environment, and will return a list of full paths.
     """
     sitepackages = []
     seen = set()
 
-    for prefix in PREFIXES:
+    if prefixes is None:
+        prefixes = PREFIXES
+
+    for prefix in prefixes:
         if not prefix or prefix in seen:
             continue
         seen.add(prefix)
@@ -303,9 +321,9 @@ def getsitepackages():
                             sys.version[:3], "site-packages"))
     return sitepackages
 
-def addsitepackages(known_paths):
+def addsitepackages(known_paths, prefixes=None):
     """Add site-packages (and possibly site-python) to sys.path"""
-    for sitedir in getsitepackages():
+    for sitedir in getsitepackages(prefixes):
         if os.path.isdir(sitedir):
             addsitedir(sitedir, known_paths)
 
@@ -475,6 +493,61 @@ def aliasmbcs():
                 encodings.aliases.aliases[enc] = 'mbcs'
 
 
+CONFIG_LINE = re.compile(r'^(?P<key>(\w|[-_])+)\s*=\s*(?P<value>.*)\s*$')
+
+def venv(known_paths):
+    global PREFIXES, ENABLE_USER_SITE
+
+    env = os.environ
+    if sys.platform == 'darwin' and '__PYTHONV_LAUNCHER__' in env:
+        executable = os.environ['__PYTHONV_LAUNCHER__']
+    else:
+        executable = sys.executable
+    executable_dir, executable_name = os.path.split(executable)
+    site_prefix = os.path.dirname(executable_dir)
+    sys._home = None
+    if sys.platform == 'win32':
+        executable_name = os.path.splitext(executable_name)[0]
+    conf_basename = 'pyvenv.cfg'
+    candidate_confs = [
+        conffile for conffile in (
+            os.path.join(executable_dir, conf_basename),
+            os.path.join(site_prefix, conf_basename)
+            )
+        if os.path.isfile(conffile)
+        ]
+
+    if candidate_confs:
+        virtual_conf = candidate_confs[0]
+        system_site = "true"
+        with open(virtual_conf) as f:
+            for line in f:
+                line = line.strip()
+                m = CONFIG_LINE.match(line)
+                if m:
+                    d = m.groupdict()
+                    key, value = d['key'].lower(), d['value']
+                    if key == 'include-system-site-packages':
+                        system_site = value.lower()
+                    elif key == 'home':
+                        sys._home = value
+
+        sys.prefix = sys.exec_prefix = site_prefix
+
+        # Doing this here ensures venv takes precedence over user-site
+        addsitepackages(known_paths, [sys.prefix])
+
+        # addsitepackages will process site_prefix again if its in PREFIXES,
+        # but that's ok; known_paths will prevent anything being added twice
+        if system_site == "true":
+            PREFIXES.insert(0, sys.prefix)
+        else:
+            PREFIXES = [sys.prefix]
+            ENABLE_USER_SITE = False
+
+    return known_paths
+
+
 def execsitecustomize():
     """Run custom site specific code, if available."""
     try:
@@ -517,6 +590,7 @@ def main():
 
     abs_paths()
     known_paths = removeduppaths()
+    known_paths = venv(known_paths)
     if ENABLE_USER_SITE is None:
         ENABLE_USER_SITE = check_enableusersite()
     known_paths = addusersitepackages(known_paths)

Lib/subprocess.py

@@ -1021,7 +1021,7 @@ class Popen(object):
             if not os.path.exists(w9xpopen):
                 # Eeek - file-not-found - possibly an embedding
                 # situation - see if we can locate it in sys.exec_prefix
-                w9xpopen = os.path.join(os.path.dirname(sys.exec_prefix),
+                w9xpopen = os.path.join(os.path.dirname(sys.base_exec_prefix),
                                         "w9xpopen.exe")
                 if not os.path.exists(w9xpopen):
                     raise RuntimeError("Cannot locate w9xpopen.exe, which is "

Lib/sysconfig.cfg

@@ -36,41 +36,41 @@ statedir = /var
 # User resource directory
 local = ~/.local/{distribution.name}
 
-stdlib = {base}/lib/python{py_version_short}
+stdlib = {installed_base}/lib/python{py_version_short}
 platstdlib = {platbase}/lib/python{py_version_short}
 purelib = {base}/lib/python{py_version_short}/site-packages
 platlib = {platbase}/lib/python{py_version_short}/site-packages
-include = {base}/include/python{py_version_short}{abiflags}
-platinclude = {platbase}/include/python{py_version_short}{abiflags}
+include = {installed_base}/include/python{py_version_short}{abiflags}
+platinclude = {installed_platbase}/include/python{py_version_short}{abiflags}
 data = {base}
 
 [posix_home]
-stdlib = {base}/lib/python
+stdlib = {installed_base}/lib/python
 platstdlib = {base}/lib/python
 purelib = {base}/lib/python
 platlib = {base}/lib/python
-include = {base}/include/python
-platinclude = {base}/include/python
+include = {installed_base}/include/python
+platinclude = {installed_base}/include/python
 scripts = {base}/bin
 data = {base}
 
 [nt]
-stdlib = {base}/Lib
+stdlib = {installed_base}/Lib
 platstdlib = {base}/Lib
 purelib = {base}/Lib/site-packages
 platlib = {base}/Lib/site-packages
-include = {base}/Include
-platinclude = {base}/Include
+include = {installed_base}/Include
+platinclude = {installed_base}/Include
 scripts = {base}/Scripts
 data = {base}
 
 [os2]
-stdlib = {base}/Lib
+stdlib = {installed_base}/Lib
 platstdlib = {base}/Lib
 purelib = {base}/Lib/site-packages
 platlib = {base}/Lib/site-packages
-include = {base}/Include
-platinclude = {base}/Include
+include = {installed_base}/Include
+platinclude = {installed_base}/Include
 scripts = {base}/Scripts
 data = {base}
 

Lib/sysconfig.py

@@ -3,6 +3,7 @@
 import os
 import re
 import sys
+import os
 from os.path import pardir, realpath
 from configparser import RawConfigParser
 
@@ -61,13 +62,15 @@ def _expand_globals(config):
 
 _expand_globals(_SCHEMES)
 
- # FIXME don't rely on sys.version here, its format is an implementatin detail
+ # FIXME don't rely on sys.version here, its format is an implementation detail
  # of CPython, use sys.version_info or sys.hexversion
 _PY_VERSION = sys.version.split()[0]
 _PY_VERSION_SHORT = sys.version[:3]
 _PY_VERSION_SHORT_NO_DOT = _PY_VERSION[0] + _PY_VERSION[2]
 _PREFIX = os.path.normpath(sys.prefix)
+_BASE_PREFIX = os.path.normpath(sys.base_prefix)
 _EXEC_PREFIX = os.path.normpath(sys.exec_prefix)
+_BASE_EXEC_PREFIX = os.path.normpath(sys.base_exec_prefix)
 _CONFIG_VARS = None
 _USER_BASE = None
 
@@ -94,14 +97,22 @@ if os.name == "nt" and "\\pc\\v" in _PROJECT_BASE[-10:].lower():
 if os.name == "nt" and "\\pcbuild\\amd64" in _PROJECT_BASE[-14:].lower():
     _PROJECT_BASE = _safe_realpath(os.path.join(_PROJECT_BASE, pardir, pardir))
 
-
-def is_python_build():
+def _is_python_source_dir(d):
     for fn in ("Setup.dist", "Setup.local"):
-        if os.path.isfile(os.path.join(_PROJECT_BASE, "Modules", fn)):
+        if os.path.isfile(os.path.join(d, "Modules", fn)):
             return True
     return False
 
-_PYTHON_BUILD = is_python_build()
+_sys_home = getattr(sys, '_home', None)
+if _sys_home and os.name == 'nt' and _sys_home.lower().endswith('pcbuild'):
+    _sys_home = os.path.dirname(_sys_home)
+
+def is_python_build(check_home=False):
+    if check_home and _sys_home:
+        return _is_python_source_dir(_sys_home)
+    return _is_python_source_dir(_PROJECT_BASE)
+
+_PYTHON_BUILD = is_python_build(True)
 
 if _PYTHON_BUILD:
     for scheme in ('posix_prefix', 'posix_home'):
@@ -312,7 +323,7 @@ def _parse_makefile(filename, vars=None):
 def get_makefile_filename():
     """Return the path of the Makefile."""
     if _PYTHON_BUILD:
-        return os.path.join(_PROJECT_BASE, "Makefile")
+        return os.path.join(_sys_home or _PROJECT_BASE, "Makefile")
     if hasattr(sys, 'abiflags'):
         config_dir_name = 'config-%s%s' % (_PY_VERSION_SHORT, sys.abiflags)
     else:
@@ -412,9 +423,9 @@ def get_config_h_filename():
     """Return the path of pyconfig.h."""
     if _PYTHON_BUILD:
         if os.name == "nt":
-            inc_dir = os.path.join(_PROJECT_BASE, "PC")
+            inc_dir = os.path.join(_sys_home or _PROJECT_BASE, "PC")
         else:
-            inc_dir = _PROJECT_BASE
+            inc_dir = _sys_home or _PROJECT_BASE
     else:
         inc_dir = get_path('platinclude')
     return os.path.join(inc_dir, 'pyconfig.h')
@@ -472,7 +483,9 @@ def get_config_vars(*args):
         _CONFIG_VARS['py_version'] = _PY_VERSION
         _CONFIG_VARS['py_version_short'] = _PY_VERSION_SHORT
         _CONFIG_VARS['py_version_nodot'] = _PY_VERSION[0] + _PY_VERSION[2]
+        _CONFIG_VARS['installed_base'] = _BASE_PREFIX
         _CONFIG_VARS['base'] = _PREFIX
+        _CONFIG_VARS['installed_platbase'] = _BASE_EXEC_PREFIX
         _CONFIG_VARS['platbase'] = _EXEC_PREFIX
         _CONFIG_VARS['projectbase'] = _PROJECT_BASE
         try:

Lib/test/regrtest.py

@@ -564,7 +564,7 @@ def main(tests=None, testdir=None, verbose=0, quiet=False,
         random.shuffle(selected)
     if trace:
         import trace, tempfile
-        tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix,
+        tracer = trace.Trace(ignoredirs=[sys.base_prefix, sys.base_exec_prefix,
                                          tempfile.gettempdir()],
                              trace=False, count=True)
 

Lib/test/test_cmd.py

@@ -228,7 +228,7 @@ def test_main(verbose=None):
 
 def test_coverage(coverdir):
     trace = support.import_module('trace')
-    tracer=trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix,],
+    tracer=trace.Trace(ignoredirs=[sys.base_prefix, sys.base_exec_prefix,],
                         trace=0, count=1)
     tracer.run('reload(cmd);test_main()')
     r=tracer.results()

Lib/test/test_doctest.py

@@ -2543,7 +2543,7 @@ import sys, re, io
 
 def test_coverage(coverdir):
     trace = support.import_module('trace')
-    tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix,],
+    tracer = trace.Trace(ignoredirs=[sys.base_prefix, sys.base_exec_prefix,],
                          trace=0, count=1)
     tracer.run('test_main()')
     r = tracer.results()

Lib/test/test_subprocess.py

@@ -189,6 +189,8 @@ class ProcessTestCase(BaseTestCase):
         p.wait()
         self.assertEqual(p.stderr, None)
 
+    @unittest.skipIf(sys.base_prefix != sys.prefix,
+                     'Test is not venv-compatible')
     def test_executable_with_cwd(self):
         python_dir = os.path.dirname(os.path.realpath(sys.executable))
         p = subprocess.Popen(["somethingyoudonthave", "-c",
@@ -197,6 +199,8 @@ class ProcessTestCase(BaseTestCase):
         p.wait()
         self.assertEqual(p.returncode, 47)
 
+    @unittest.skipIf(sys.base_prefix != sys.prefix,
+                     'Test is not venv-compatible')
     @unittest.skipIf(sysconfig.is_python_build(),
                      "need an installed Python. See #7774")
     def test_executable_without_cwd(self):

Lib/test/test_sys.py

@@ -419,6 +419,7 @@ class SysModuleTest(unittest.TestCase):
         self.assertIsInstance(sys.builtin_module_names, tuple)
         self.assertIsInstance(sys.copyright, str)
         self.assertIsInstance(sys.exec_prefix, str)
+        self.assertIsInstance(sys.base_exec_prefix, str)
         self.assertIsInstance(sys.executable, str)
         self.assertEqual(len(sys.float_info), 11)
         self.assertEqual(sys.float_info.radix, 2)
@@ -450,6 +451,7 @@ class SysModuleTest(unittest.TestCase):
         self.assertEqual(sys.maxunicode, 0x10FFFF)
         self.assertIsInstance(sys.platform, str)
         self.assertIsInstance(sys.prefix, str)
+        self.assertIsInstance(sys.base_prefix, str)
         self.assertIsInstance(sys.version, str)
         vi = sys.version_info
         self.assertIsInstance(vi[:], tuple)
@@ -541,6 +543,8 @@ class SysModuleTest(unittest.TestCase):
         out = p.communicate()[0].strip()
         self.assertEqual(out, b'?')
 
+    @unittest.skipIf(sys.base_prefix != sys.prefix,
+                     'Test is not venv-compatible')
     def test_executable(self):
         # sys.executable should be absolute
         self.assertEqual(os.path.abspath(sys.executable), sys.executable)

Lib/test/test_sysconfig.py

@@ -260,12 +260,17 @@ class TestSysConfig(unittest.TestCase):
         # the global scheme mirrors the distinction between prefix and
         # exec-prefix but not the user scheme, so we have to adapt the paths
         # before comparing (issue #9100)
-        adapt = sys.prefix != sys.exec_prefix
+        adapt = sys.base_prefix != sys.base_exec_prefix
         for name in ('stdlib', 'platstdlib', 'purelib', 'platlib'):
             global_path = get_path(name, 'posix_prefix')
             if adapt:
-                global_path = global_path.replace(sys.exec_prefix, sys.prefix)
-                base = base.replace(sys.exec_prefix, sys.prefix)
+                global_path = global_path.replace(sys.exec_prefix, sys.base_prefix)
+                base = base.replace(sys.exec_prefix, sys.base_prefix)
+            elif sys.base_prefix != sys.prefix:
+                # virtual environment? Likewise, we have to adapt the paths
+                # before comparing
+                global_path = global_path.replace(sys.base_prefix, sys.prefix)
+                base = base.replace(sys.base_prefix, sys.prefix)
             user_path = get_path(name, 'posix_user')
             self.assertEqual(user_path, global_path.replace(base, user, 1))
 

Lib/test/test_trace.py

@@ -316,8 +316,8 @@ class TestCoverage(unittest.TestCase):
         # Ignore all files, nothing should be traced nor printed
         libpath = os.path.normpath(os.path.dirname(os.__file__))
         # sys.prefix does not work when running from a checkout
-        tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix, libpath],
-                             trace=0, count=1)
+        tracer = trace.Trace(ignoredirs=[sys.base_prefix, sys.base_exec_prefix,
+                             libpath], trace=0, count=1)
         with captured_stdout() as stdout:
             self._coverage(tracer)
         if os.path.exists(TESTFN):

Lib/test/test_venv.py

+#!/usr/bin/env python
+#
+# Copyright 2011 by Vinay Sajip. All Rights Reserved.
+#
+# Permission to use, copy, modify, and distribute this software and its
+# documentation for any purpose and without fee is hereby granted,
+# provided that the above copyright notice appear in all copies and that
+# both that copyright notice and this permission notice appear in
+# supporting documentation, and that the name of Vinay Sajip
+# not be used in advertising or publicity pertaining to distribution
+# of the software without specific, written prior permission.
+# VINAY SAJIP DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE, INCLUDING
+# ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL
+# VINAY SAJIP BE LIABLE FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR
+# ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER
+# IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
+# OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+
+"""Test harness for the venv module. Run all tests.
+
+Copyright (C) 2011 Vinay Sajip. All Rights Reserved.
+"""
+
+import os
+import os.path
+import shutil
+import sys
+import tempfile
+from test.support import (captured_stdout, captured_stderr, run_unittest,
+                          can_symlink)
+import unittest
+import venv
+
+class BaseTest(unittest.TestCase):
+    """Base class for venv tests."""
+
+    def setUp(self):
+        self.env_dir = tempfile.mkdtemp()
+        if os.name == 'nt':
+            self.bindir = 'Scripts'
+            self.ps3name = 'pysetup3-script.py'
+            self.lib = ('Lib',)
+            self.include = 'Include'
+            self.exe = 'python.exe'
+        else:
+            self.bindir = 'bin'
+            self.ps3name = 'pysetup3'
+            self.lib = ('lib', 'python%s' % sys.version[:3])
+            self.include = 'include'
+            self.exe = 'python'
+
+    def tearDown(self):
+        shutil.rmtree(self.env_dir)
+
+    def run_with_capture(self, func, *args, **kwargs):
+        with captured_stdout() as output:
+            with captured_stderr() as error:
+                func(*args, **kwargs)
+        return output.getvalue(), error.getvalue()
+
+    def get_env_file(self, *args):
+        return os.path.join(self.env_dir, *args)
+
+    def get_text_file_contents(self, *args):
+        with open(self.get_env_file(*args), 'r') as f:
+            result = f.read()
+        return result
+
+class BasicTest(BaseTest):
+    """Test venv module functionality."""
+
+    def test_defaults(self):
+        """
+        Test the create function with default arguments.
+        """
+        def isdir(*args):
+            fn = self.get_env_file(*args)
+            self.assertTrue(os.path.isdir(fn))
+
+        shutil.rmtree(self.env_dir)
+        self.run_with_capture(venv.create, self.env_dir)
+        isdir(self.bindir)
+        isdir(self.include)
+        isdir(*self.lib)
+        data = self.get_text_file_contents('pyvenv.cfg')
+        if sys.platform == 'darwin' and ('__PYTHONV_LAUNCHER__'
+                                         in os.environ):
+            executable =  os.environ['__PYTHONV_LAUNCHER__']
+        else:
+            executable = sys.executable
+        path = os.path.dirname(executable)
+        self.assertIn('home = %s' % path, data)
+        data = self.get_text_file_contents(self.bindir, self.ps3name)
+        self.assertTrue(data.startswith('#!%s%s' % (self.env_dir, os.sep)))
+        fn = self.get_env_file(self.bindir, self.exe)
+        self.assertTrue(os.path.exists(fn))
+
+    def test_overwrite_existing(self):
+        """
+        Test control of overwriting an existing environment directory.
+        """
+        self.assertRaises(ValueError, venv.create, self.env_dir)
+        builder = venv.EnvBuilder(clear=True)
+        builder.create(self.env_dir)
+
+    def test_isolation(self):
+        """
+        Test isolation from system site-packages
+        """
+        for ssp, s in ((True, 'true'), (False, 'false')):
+            builder = venv.EnvBuilder(clear=True, system_site_packages=ssp)
+            builder.create(self.env_dir)
+            data = self.get_text_file_contents('pyvenv.cfg')
+            self.assertIn('include-system-site-packages = %s\n' % s, data)
+
+    @unittest.skipUnless(can_symlink(), 'Needs symlinks')
+    def test_symlinking(self):
+        """
+        Test symlinking works as expected
+        """
+        for usl in (False, True):
+            builder = venv.EnvBuilder(clear=True, symlinks=usl)
+            if (usl and sys.platform == 'darwin' and
+                '__PYTHONV_LAUNCHER__' in os.environ):
+                self.assertRaises(ValueError, builder.create, self.env_dir)
+            else:
+                builder.create(self.env_dir)
+                fn = self.get_env_file(self.bindir, self.exe)
+                # Don't test when False, because e.g. 'python' is always
+                # symlinked to 'python3.3' in the env, even when symlinking in
+                # general isn't wanted.
+                if usl:
+                    self.assertTrue(os.path.islink(fn))
+
+def test_main():
+    run_unittest(BasicTest)
+
+if __name__ == "__main__":
+    test_main()

Lib/tkinter/_fix.py

@@ -46,10 +46,10 @@ else:
             s = "\\" + s[3:]
         return s
 
-prefix = os.path.join(sys.prefix,"tcl")
+prefix = os.path.join(sys.base_prefix,"tcl")
 if not os.path.exists(prefix):
     # devdir/../tcltk/lib
-    prefix = os.path.join(sys.prefix, os.path.pardir, "tcltk", "lib")
+    prefix = os.path.join(sys.base_prefix, os.path.pardir, "tcltk", "lib")
     prefix = os.path.abspath(prefix)
 # if this does not exist, no further search is needed
 if os.path.exists(prefix):

Lib/trace.py

@@ -39,8 +39,8 @@ Sample use, programmatically
 
   # create a Trace object, telling it what to ignore, and whether to
   # do tracing or line-counting or both.
-  tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix,], trace=0,
-                    count=1)
+  tracer = trace.Trace(ignoredirs=[sys.base_prefix, sys.base_exec_prefix,],
+                       trace=0, count=1)
   # run the new command using the given tracer
   tracer.run('main()')
   # make a report, placing output in /tmp
@@ -749,10 +749,10 @@ def main(argv=None):
                 # should I also call expanduser? (after all, could use $HOME)
 
                 s = s.replace("$prefix",
-                              os.path.join(sys.prefix, "lib",
+                              os.path.join(sys.base_prefix, "lib",
                                            "python" + sys.version[:3]))
                 s = s.replace("$exec_prefix",
-                              os.path.join(sys.exec_prefix, "lib",
+                              os.path.join(sys.base_exec_prefix, "lib",
                                            "python" + sys.version[:3]))
                 s = os.path.normpath(s)
                 ignore_dirs.append(s)

Lib/venv/__main__.py

+import sys
+from . import main
+
+rc = 1
+try:
+    main()
+    rc = 0
+except Exception as e:
+    print('Error: %s' % e, file=sys.stderr)
+sys.exit(rc)

Lib/venv/scripts/nt/Activate.ps1

+$env:VIRTUAL_ENV="__VENV_DIR__"
+
+# Revert to original values
+if (Test-Path function:_OLD_VIRTUAL_PROMPT) {
+    copy-item function:_OLD_VIRTUAL_PROMPT function:prompt
+    remove-item function:_OLD_VIRTUAL_PROMPT
+}
+
+if (Test-Path env:_OLD_VIRTUAL_PYTHONHOME) {
+    copy-item env:_OLD_VIRTUAL_PYTHONHOME env:PYTHONHOME
+    remove-item env:_OLD_VIRTUAL_PYTHONHOME
+}
+
+if (Test-Path env:_OLD_VIRTUAL_PATH) {
+    copy-item env:_OLD_VIRTUAL_PATH env:PATH
+    remove-item env:_OLD_VIRTUAL_PATH
+}
+
+# Set the prompt to include the env name
+copy-item function:prompt function:_OLD_VIRTUAL_PROMPT
+function prompt {
+    Write-Host -NoNewline -ForegroundColor Green [__VENV_NAME__]
+    _OLD_VIRTUAL_PROMPT
+}
+
+# Clear PYTHONHOME
+if (Test-Path env:PYTHONHOME) {
+    copy-item env:PYTHONHOME env:_OLD_VIRTUAL_PYTHONHOME
+    remove-item env:PYTHONHOME
+}
+
+# Add the venv to the PATH
+copy-item env:PATH env:_OLD_VIRTUAL_PATH
+$env:PATH = "$env:VIRTUAL_ENV\__VENV_BIN_NAME__;$env:PATH"

Lib/venv/scripts/nt/Deactivate.ps1

+# Revert to original values
+if (Test-Path function:_OLD_VIRTUAL_PROMPT) {
+    copy-item function:_OLD_VIRTUAL_PROMPT function:prompt
+    remove-item function:_OLD_VIRTUAL_PROMPT
+}
+
+if (Test-Path env:_OLD_VIRTUAL_PYTHONHOME) {
+    copy-item env:_OLD_VIRTUAL_PYTHONHOME env:PYTHONHOME
+    remove-item env:_OLD_VIRTUAL_PYTHONHOME
+}
+
+if (Test-Path env:_OLD_VIRTUAL_PATH) {
+    copy-item env:_OLD_VIRTUAL_PATH env:PATH
+    remove-item env:_OLD_VIRTUAL_PATH
+}
+
+if (Test-Path env:VIRTUAL_ENV) {
+    remove-item env:VIRTUAL_ENV
+}

Lib/venv/scripts/nt/activate.bat

+@echo off
+set VIRTUAL_ENV=__VENV_DIR__
+
+if not defined PROMPT (
+    set PROMPT=$P$G
+)
+
+if defined _OLD_VIRTUAL_PROMPT (
+    set PROMPT=%_OLD_VIRTUAL_PROMPT%
+)
+
+if defined _OLD_VIRTUAL_PYTHONHOME (
+     set PYTHONHOME=%_OLD_VIRTUAL_PYTHONHOME%
+)
+
+set _OLD_VIRTUAL_PROMPT=%PROMPT%
+set PROMPT=__VENV_NAME__%PROMPT%
+
+if defined PYTHONHOME (
+     set _OLD_VIRTUAL_PYTHONHOME=%PYTHONHOME%
+     set PYTHONHOME=
+)
+
+if defined _OLD_VIRTUAL_PATH set PATH=%_OLD_VIRTUAL_PATH%; goto SKIPPATH
+
+set _OLD_VIRTUAL_PATH=%PATH%
+
+:SKIPPATH
+set PATH=%VIRTUAL_ENV%\__VENV_BIN_NAME__;%PATH%
+
+:END

Lib/venv/scripts/nt/deactivate.bat

+@echo off
+
+if defined _OLD_VIRTUAL_PROMPT (
+    set PROMPT=%_OLD_VIRTUAL_PROMPT%
+)
+set _OLD_VIRTUAL_PROMPT=
+
+if defined _OLD_VIRTUAL_PYTHONHOME (
+     set PYTHONHOME=%_OLD_VIRTUAL_PYTHONHOME%
+     set _OLD_VIRTUAL_PYTHONHOME=
+)
+
+if defined _OLD_VIRTUAL_PATH set PATH=%_OLD_VIRTUAL_PATH%
+
+set _OLD_VIRTUAL_PATH=
+
+:END

Lib/venv/scripts/nt/pysetup3-script.py

+#!__VENV_PYTHON__
+if __name__ == '__main__':
+    rc = 1
+    try:
+        import sys, re, packaging.run
+        sys.argv[0] = re.sub('-script.pyw?$', '', sys.argv[0])
+        rc = packaging.run.main() # None interpreted as 0
+    except Exception:
+        # use syntax which works with either 2.x or 3.x
+        sys.stderr.write('%s\n' % sys.exc_info()[1])
+    sys.exit(rc)

Lib/venv/scripts/posix/activate

+# This file must be used with "source bin/activate" *from bash*
+# you cannot run it directly
+
+deactivate () {
+    # reset old environment variables
+    if [ -n "$_OLD_VIRTUAL_PATH" ] ; then
+        PATH="$_OLD_VIRTUAL_PATH"
+        export PATH
+        unset _OLD_VIRTUAL_PATH
+    fi
+    if [ -n "$_OLD_VIRTUAL_PYTHONHOME" ] ; then
+        PYTHONHOME="$_OLD_VIRTUAL_PYTHONHOME"
+        export PYTHONHOME
+        unset _OLD_VIRTUAL_PYTHONHOME
+    fi
+
+    # This should detect bash and zsh, which have a hash command that must
+    # be called to get it to forget past commands.  Without forgetting
+    # past commands the $PATH changes we made may not be respected
+    if [ -n "$BASH" -o -n "$ZSH_VERSION" ] ; then
+        hash -r
+    fi
+
+    if [ -n "$_OLD_VIRTUAL_PS1" ] ; then
+        PS1="$_OLD_VIRTUAL_PS1"
+        export PS1
+        unset _OLD_VIRTUAL_PS1
+    fi
+
+    unset VIRTUAL_ENV
+    if [ ! "$1" = "nondestructive" ] ; then
+    # Self destruct!
+        unset -f deactivate
+    fi
+}
+
+# unset irrelavent variables
+deactivate nondestructive
+
+VIRTUAL_ENV="__VENV_DIR__"
+export VIRTUAL_ENV
+
+_OLD_VIRTUAL_PATH="$PATH"
+PATH="$VIRTUAL_ENV/__VENV_BIN_NAME__:$PATH"
+export PATH
+
+# unset PYTHONHOME if set
+# this will fail if PYTHONHOME is set to the empty string (which is bad anyway)
+# could use `if (set -u; : $PYTHONHOME) ;` in bash
+if [ -n "$PYTHONHOME" ] ; then
+    _OLD_VIRTUAL_PYTHONHOME="$PYTHONHOME"
+    unset PYTHONHOME
+fi
+
+if [ -z "$VIRTUAL_ENV_DISABLE_PROMPT" ] ; then
+    _OLD_VIRTUAL_PS1="$PS1"
+    if [ "x__VENV_NAME__" != x ] ; then
+	PS1="__VENV_NAME__$PS1"
+    else
+    if [ "`basename \"$VIRTUAL_ENV\"`" = "__" ] ; then
+        # special case for Aspen magic directories
+        # see http://www.zetadev.com/software/aspen/
+        PS1="[`basename \`dirname \"$VIRTUAL_ENV\"\``] $PS1"
+    else
+        PS1="(`basename \"$VIRTUAL_ENV\"`)$PS1"
+    fi
+    fi
+    export PS1
+fi
+
+# This should detect bash and zsh, which have a hash command that must
+# be called to get it to forget past commands.  Without forgetting
+# past commands the $PATH changes we made may not be respected
+if [ -n "$BASH" -o -n "$ZSH_VERSION" ] ; then
+    hash -r
+fi

Lib/venv/scripts/posix/pysetup3

+#!__VENV_PYTHON__
+if __name__ == '__main__':
+    rc = 1
+    try:
+        import sys, re, packaging.run
+        sys.argv[0] = re.sub('-script.pyw?$', '', sys.argv[0])
+        rc = packaging.run.main() # None interpreted as 0
+    except Exception:
+        # use syntax which works with either 2.x or 3.x
+        sys.stderr.write('%s\n' % sys.exc_info()[1])
+    sys.exit(rc)

Mac/Makefile.in

@@ -72,7 +72,7 @@ installunixtools:
 	for fn in python3 pythonw3 idle3 pydoc3 python3-config \
 		  python$(VERSION) pythonw$(VERSION) idle$(VERSION) \
 		  pydoc$(VERSION) python$(VERSION)-config 2to3 \
-		  2to3-$(VERSION) ;\
+		  2to3-$(VERSION) pyvenv pyvenv-$(VERSION) ;\
 	do \
 		ln -fs "$(prefix)/bin/$${fn}" "$(DESTDIR)$(FRAMEWORKUNIXTOOLSPREFIX)/bin/$${fn}" ;\
 	done
@@ -93,7 +93,7 @@ altinstallunixtools:
 		$(INSTALL) -d -m $(DIRMODE) "$(DESTDIR)$(FRAMEWORKUNIXTOOLSPREFIX)/bin" ;\
 	fi
 	for fn in python$(VERSION) pythonw$(VERSION) idle$(VERSION) \
-		  pydoc$(VERSION) python$(VERSION)-config 2to3-$(VERSION);\
+		  pydoc$(VERSION) python$(VERSION)-config 2to3-$(VERSION) pyvenv-$(VERSION) ;\
 	do \
 		ln -fs "$(prefix)/bin/$${fn}" "$(DESTDIR)$(FRAMEWORKUNIXTOOLSPREFIX)/bin/$${fn}" ;\
 	done

Mac/Tools/pythonw.c

@@ -150,6 +150,18 @@ setup_spawnattr(posix_spawnattr_t* spawnattr)
 int
 main(int argc, char **argv) {
     char* exec_path = get_python_path();
+    static char path[PATH_MAX * 2];
+    static char real_path[PATH_MAX * 2];
+    int status;
+    uint32_t size = PATH_MAX * 2;
+
+    /* Set the original executable path in the environment. */
+    status = _NSGetExecutablePath(path, &size);
+    if (status == 0) {
+        if (realpath(path, real_path) != NULL) {
+            setenv("__PYTHONV_LAUNCHER__", real_path, 1);
+        }
+    }
 
     /*
      * Let argv[0] refer to the new interpreter. This is needed to

Makefile.pre.in

@@ -947,6 +947,8 @@ bininstall: altbininstall
 	(cd $(DESTDIR)$(BINDIR); $(LN) -s 2to3-$(VERSION) 2to3)
 	-rm -f $(DESTDIR)$(BINDIR)/pysetup3
 	(cd $(DESTDIR)$(BINDIR); $(LN) -s pysetup$(VERSION) pysetup3)
+	-rm -f $(DESTDIR)$(BINDIR)/pyvenv
+	(cd $(DESTDIR)$(BINDIR); $(LN) -s pyvenv-$(VERSION) pyvenv)
 
 # Install the manual page
 maninstall:
@@ -1038,6 +1040,7 @@ LIBSUBDIRS=	tkinter tkinter/test tkinter/test/test_tkinter \
 		turtledemo \
 		multiprocessing multiprocessing/dummy \
 		unittest unittest/test unittest/test/testmock \
+		venv venv/scripts venv/scripts/posix \
 		curses pydoc_data $(MACHDEPS)
 libinstall:	build_all $(srcdir)/Lib/$(PLATDIR) $(srcdir)/Modules/xxmodule.c
 	@for i in $(SCRIPTDIR) $(LIBDEST); \

Modules/getpath.c

@@ -260,6 +260,59 @@ absolutize(wchar_t *path)
     wcscpy(path, buffer);
 }
 
+/* search for a prefix value in an environment file. If found, copy it
+   to the provided buffer, which is expected to be no more than MAXPATHLEN
+   bytes long.
+*/
+
+static int
+find_env_config_value(FILE * env_file, const wchar_t * key, wchar_t * value)
+{
+    int result = 0; /* meaning not found */
+    char buffer[MAXPATHLEN*2+1];  /* allow extra for key, '=', etc. */
+
+    fseek(env_file, 0, SEEK_SET);
+    while (!feof(env_file)) {
+        char * p = fgets(buffer, MAXPATHLEN*2, env_file);
+        wchar_t tmpbuffer[MAXPATHLEN*2+1];
+        PyObject * decoded;
+        int n;
+
+        if (p == NULL)
+            break;
+        n = strlen(p);
+        if (p[n - 1] != '\n') {
+            /* line has overflowed - bail */
+            break;
+        }
+        if (p[0] == '#')    /* Comment - skip */
+            continue;
+        decoded = PyUnicode_DecodeUTF8(buffer, n, "surrogateescape");
+        if (decoded != NULL) {
+            Py_ssize_t k;
+            wchar_t * state;
+            k = PyUnicode_AsWideChar(decoded,
+                                     tmpbuffer, MAXPATHLEN * 2);
+            Py_DECREF(decoded);
+            if (k >= 0) {
+                wchar_t * tok = wcstok(tmpbuffer, L" \t\r\n", &state);
+                if ((tok != NULL) && !wcscmp(tok, key)) {
+                    tok = wcstok(NULL, L" \t", &state);
+                    if ((tok != NULL) && !wcscmp(tok, L"=")) {
+                        tok = wcstok(NULL, L"\r\n", &state);
+                        if (tok != NULL) {
+                            wcsncpy(value, tok, MAXPATHLEN);
+                            result = 1;
+                            break;
+                        }
+                    }
+                }
+            }
+        }
+    }
+    return result;
+}
+
 /* search_for_prefix requires that argv0_path be no more than MAXPATHLEN
    bytes long.
 */
@@ -565,6 +618,39 @@ calculate_path(void)
        MAXPATHLEN bytes long.
     */
 
+    /* Search for an environment configuration file, first in the
+       executable's directory and then in the parent directory.
+       If found, open it for use when searching for prefixes.
+    */
+
+    {
+        wchar_t tmpbuffer[MAXPATHLEN+1];
+        wchar_t *env_cfg = L"pyvenv.cfg";
+        FILE * env_file = NULL;
+
+        wcscpy(tmpbuffer, argv0_path);
+        joinpath(tmpbuffer, env_cfg);
+        env_file = _Py_wfopen(tmpbuffer, L"r");
+        if (env_file == NULL) {
+            errno = 0;
+            reduce(tmpbuffer);
+            reduce(tmpbuffer);
+            joinpath(tmpbuffer, env_cfg);
+            env_file = _Py_wfopen(tmpbuffer, L"r");
+            if (env_file == NULL) {
+                errno = 0;
+            }
+        }
+        if (env_file != NULL) {
+            /* Look for a 'home' variable and set argv0_path to it, if found */
+            if (find_env_config_value(env_file, L"home", tmpbuffer)) {
+                wcscpy(argv0_path, tmpbuffer);
+            }
+            fclose(env_file);
+            env_file = NULL;
+        }
+    }
+
     if (!(pfound = search_for_prefix(argv0_path, home, _prefix))) {
         if (!Py_FrozenFlag)
             fprintf(stderr,

PC/getpathp.c

@@ -423,6 +423,53 @@ get_progpath(void)
         progpath[0] = '\0';
 }
 
+static int
+find_env_config_value(FILE * env_file, const wchar_t * key, wchar_t * value)
+{
+    int result = 0; /* meaning not found */
+    char buffer[MAXPATHLEN*2+1];  /* allow extra for key, '=', etc. */
+
+    fseek(env_file, 0, SEEK_SET);
+    while (!feof(env_file)) {
+        char * p = fgets(buffer, MAXPATHLEN*2, env_file);
+        wchar_t tmpbuffer[MAXPATHLEN*2+1];
+        PyObject * decoded;
+        int n;
+
+        if (p == NULL)
+            break;
+        n = strlen(p);
+        if (p[n - 1] != '\n') {
+            /* line has overflowed - bail */
+            break;
+        }
+        if (p[0] == '#')    /* Comment - skip */
+            continue;
+        decoded = PyUnicode_DecodeUTF8(buffer, n, "surrogateescape");
+        if (decoded != NULL) {
+            Py_ssize_t k;
+            k = PyUnicode_AsWideChar(decoded,
+                                     tmpbuffer, MAXPATHLEN * 2);
+            Py_DECREF(decoded);
+            if (k >= 0) {
+                wchar_t * tok = wcstok(tmpbuffer, L" \t\r\n");
+                if ((tok != NULL) && !wcscmp(tok, key)) {
+                    tok = wcstok(NULL, L" \t");
+                    if ((tok != NULL) && !wcscmp(tok, L"=")) {
+                        tok = wcstok(NULL, L"\r\n");
+                        if (tok != NULL) {
+                            wcsncpy(value, tok, MAXPATHLEN);
+                            result = 1;
+                            break;
+                        }
+                    }
+                }
+            }
+        }
+    }
+    return result;
+}
+
 static void
 calculate_path(void)
 {
@@ -457,6 +504,40 @@ calculate_path(void)
     /* progpath guaranteed \0 terminated in MAXPATH+1 bytes. */
     wcscpy(argv0_path, progpath);
     reduce(argv0_path);
+
+    /* Search for an environment configuration file, first in the
+       executable's directory and then in the parent directory.
+       If found, open it for use when searching for prefixes.
+    */
+
+    {
+        wchar_t tmpbuffer[MAXPATHLEN+1];
+        wchar_t *env_cfg = L"pyvenv.cfg";
+        FILE * env_file = NULL;
+
+        wcscpy(tmpbuffer, argv0_path);
+        join(tmpbuffer, env_cfg);
+        env_file = _Py_wfopen(tmpbuffer, L"r");
+        if (env_file == NULL) {
+            errno = 0;
+            reduce(tmpbuffer);
+            reduce(tmpbuffer);
+            join(tmpbuffer, env_cfg);
+            env_file = _Py_wfopen(tmpbuffer, L"r");
+            if (env_file == NULL) {
+                errno = 0;
+            }
+        }
+        if (env_file != NULL) {
+            /* Look for a 'home' variable and set argv0_path to it, if found */
+            if (find_env_config_value(env_file, L"home", tmpbuffer)) {
+                wcscpy(argv0_path, tmpbuffer);
+            }
+            fclose(env_file);
+            env_file = NULL;
+        }
+    }
+
     if (pythonhome == NULL || *pythonhome == '\0') {
         if (search_for_prefix(argv0_path, LANDMARK))
             pythonhome = prefix;

Python/sysmodule.c

@@ -1528,6 +1528,10 @@ _PySys_Init(void)
                         PyUnicode_FromWideChar(Py_GetPrefix(), -1));
     SET_SYS_FROM_STRING("exec_prefix",
                         PyUnicode_FromWideChar(Py_GetExecPrefix(), -1));
+    SET_SYS_FROM_STRING("base_prefix",
+                        PyUnicode_FromWideChar(Py_GetPrefix(), -1));
+    SET_SYS_FROM_STRING("base_exec_prefix",
+                        PyUnicode_FromWideChar(Py_GetExecPrefix(), -1));
     SET_SYS_FROM_STRING("maxsize",
                         PyLong_FromSsize_t(PY_SSIZE_T_MAX));
     SET_SYS_FROM_STRING("float_info",

Tools/msi/msi.py

@@ -1122,6 +1122,7 @@ def add_files(db):
             lib.add_file("2to3.py", src="2to3")
             lib.add_file("pydoc3.py", src="pydoc3")
             lib.add_file("pysetup3.py", src="pysetup3")
+            lib.add_file("pyvenv.py", src="pyvenv")
             if have_tcl:
                 lib.start_component("pydocgui.pyw", tcltk, keyfile="pydocgui.pyw")
                 lib.add_file("pydocgui.pyw")

Tools/scripts/pyvenv

+#!/usr/bin/env python3
+if __name__ == '__main__':
+    import sys
+    rc = 1
+    try:
+        import venv
+        venv.main()
+        rc = 0
+    except Exception as e:
+        print('Error: %s' % e, file=sys.stderr)
+    sys.exit(rc)

setup.py

@@ -431,7 +431,7 @@ class PyBuildExt(build_ext):
                     for directory in reversed(options.dirs):
                         add_dir_to_list(dir_list, directory)
 
-        if os.path.normpath(sys.prefix) != '/usr' \
+        if os.path.normpath(sys.base_prefix) != '/usr' \
                 and not sysconfig.get_config_var('PYTHONFRAMEWORK'):
             # OSX note: Don't add LIBDIR and INCLUDEDIR to building a framework
             # (PYTHONFRAMEWORK is set) to avoid # linking problems when
@@ -1978,7 +1978,7 @@ class PyBuildScripts(build_scripts):
         newoutfiles = []
         newupdated_files = []
         for filename in outfiles:
-            if filename.endswith('2to3'):
+            if filename.endswith(('2to3', 'pyvenv')):
                 newfilename = filename + fullversion
             else:
                 newfilename = filename + minoronly
@@ -2046,7 +2046,8 @@ def main():
           # check the PyBuildScripts command above, and change the links
           # created by the bininstall target in Makefile.pre.in
           scripts = ["Tools/scripts/pydoc3", "Tools/scripts/idle3",
-                     "Tools/scripts/2to3", "Tools/scripts/pysetup3"]
+                     "Tools/scripts/2to3", "Tools/scripts/pysetup3",
+                     "Tools/scripts/pyvenv"]
         )
 
 # --install-platlib