Merged latest changes from setuptools-0.6 branch

--HG--
rename : doc/formats.txt => docs/formats.txt

CHANGES.txt

@@ -2,6 +2,21 @@
 CHANGES
 =======
 
+---
+0.7
+---
+
+* Merged Setuptools and Distribute. See MERGE.txt for details.
+
+Added several features that were slated for setuptools 0.6c12:
+
+* Index URL now defaults to HTTPS.
+* Added environment marker support. Now clients may designate a PEP-426
+  environment marker for "extra" dependencies. For an example, see the
+  Setuptools ``setup.py`` script.
+* Added support for SSL certificate validation when installing packages from
+  an HTTPS service.
+
 ------
 0.6.39
 ------

docs/formats.txt

+=====================================
+The Internal Structure of Python Eggs
+=====================================
+
+STOP! This is not the first document you should read!
+
+This document assumes you have at least some passing familiarity with
+*using* setuptools, the ``pkg_resources`` module, and EasyInstall.  It
+does not attempt to explain basic concepts like inter-project
+dependencies, nor does it contain detailed lexical syntax for most
+file formats.  Neither does it explain concepts like "namespace
+packages" or "resources" in any detail, as all of these subjects are
+covered at length in the setuptools developer's guide and the
+``pkg_resources`` reference manual.
+
+Instead, this is **internal** documentation for how those concepts and
+features are *implemented* in concrete terms.  It is intended for people
+who are working on the setuptools code base, who want to be able to
+troubleshoot setuptools problems, want to write code that reads the file
+formats involved, or want to otherwise tinker with setuptools-generated
+files and directories.
+
+Note, however, that these are all internal implementation details and
+are therefore subject to change; stick to the published API if you don't
+want to be responsible for keeping your code from breaking when
+setuptools changes.  You have been warned.
+
+
+.. contents:: **Table of Contents**
+
+
+----------------------
+Eggs and their Formats
+----------------------
+
+A "Python egg" is a logical structure embodying the release of a
+specific version of a Python project, comprising its code, resources,
+and metadata. There are multiple formats that can be used to physically
+encode a Python egg, and others can be developed. However, a key
+principle of Python eggs is that they should be discoverable and
+importable. That is, it should be possible for a Python application to
+easily and efficiently find out what eggs are present on a system, and
+to ensure that the desired eggs' contents are importable.
+
+There are two basic formats currently implemented for Python eggs:
+
+1. ``.egg`` format: a directory or zipfile *containing* the project's
+   code and resources, along with an ``EGG-INFO`` subdirectory that
+   contains the project's metadata
+
+2. ``.egg-info`` format: a file or directory placed *adjacent* to the
+   project's code and resources, that directly contains the project's
+   metadata.
+
+Both formats can include arbitrary Python code and resources, including
+static data files, package and non-package directories, Python
+modules, C extension modules, and so on.  But each format is optimized
+for different purposes.
+
+The ``.egg`` format is well-suited to distribution and the easy
+uninstallation or upgrades of code, since the project is essentially
+self-contained within a single directory or file, unmingled with any
+other projects' code or resources.  It also makes it possible to have
+multiple versions of a project simultaneously installed, such that
+individual programs can select the versions they wish to use.
+
+The ``.egg-info`` format, on the other hand, was created to support
+backward-compatibility, performance, and ease of installation for system
+packaging tools that expect to install all projects' code and resources
+to a single directory (e.g. ``site-packages``).  Placing the metadata
+in that same directory simplifies the installation process, since it
+isn't necessary to create ``.pth`` files or otherwise modify
+``sys.path`` to include each installed egg.
+
+Its disadvantage, however, is that it provides no support for clean
+uninstallation or upgrades, and of course only a single version of a
+project can be installed to a given directory. Thus, support from a
+package management tool is required. (This is why setuptools' "install"
+command refers to this type of egg installation as "single-version,
+externally managed".)  Also, they lack sufficient data to allow them to
+be copied from their installation source.  easy_install can "ship" an
+application by copying ``.egg`` files or directories to a target
+location, but it cannot do this for ``.egg-info`` installs, because
+there is no way to tell what code and resources belong to a particular
+egg -- there may be several eggs "scrambled" together in a single
+installation location, and the ``.egg-info`` format does not currently
+include a way to list the files that were installed.  (This may change
+in a future version.)
+
+
+Code and Resources
+==================
+
+The layout of the code and resources is dictated by Python's normal
+import layout, relative to the egg's "base location".
+
+For the ``.egg`` format, the base location is the ``.egg`` itself. That
+is, adding the ``.egg`` filename or directory name to ``sys.path``
+makes its contents importable.
+
+For the ``.egg-info`` format, however, the base location is the
+directory that *contains* the ``.egg-info``, and thus it is the
+directory that must be added to ``sys.path`` to make the egg importable.
+(Note that this means that the "normal" installation of a package to a
+``sys.path`` directory is sufficient to make it an "egg" if it has an
+``.egg-info`` file or directory installed alongside of it.)
+
+
+Project Metadata
+=================
+
+If eggs contained only code and resources, there would of course be
+no difference between them and any other directory or zip file on
+``sys.path``.  Thus, metadata must also be included, using a metadata
+file or directory.
+
+For the ``.egg`` format, the metadata is placed in an ``EGG-INFO``
+subdirectory, directly within the ``.egg`` file or directory.  For the
+``.egg-info`` format, metadata is stored directly within the
+``.egg-info`` directory itself.
+
+The minimum project metadata that all eggs must have is a standard
+Python ``PKG-INFO`` file, named ``PKG-INFO`` and placed within the
+metadata directory appropriate to the format.  Because it's possible for
+this to be the only metadata file included, ``.egg-info`` format eggs
+are not required to be a directory; they can just be a ``.egg-info``
+file that directly contains the ``PKG-INFO`` metadata.  This eliminates
+the need to create a directory just to store one file.  This option is
+*not* available for ``.egg`` formats, since setuptools always includes
+other metadata.  (In fact, setuptools itself never generates
+``.egg-info`` files, either; the support for using files was added so
+that the requirement could easily be satisfied by other tools, such
+as the distutils in Python 2.5).
+
+In addition to the ``PKG-INFO`` file, an egg's metadata directory may
+also include files and directories representing various forms of
+optional standard metadata (see the section on `Standard Metadata`_,
+below) or user-defined metadata required by the project.  For example,
+some projects may define a metadata format to describe their application
+plugins, and metadata in this format would then be included by plugin
+creators in their projects' metadata directories.
+
+
+Filename-Embedded Metadata
+==========================
+
+To allow introspection of installed projects and runtime resolution of
+inter-project dependencies, a certain amount of information is embedded
+in egg filenames.  At a minimum, this includes the project name, and
+ideally will also include the project version number.  Optionally, it
+can also include the target Python version and required runtime
+platform if platform-specific C code is included.  The syntax of an
+egg filename is as follows::
+
+    name ["-" version ["-py" pyver ["-" required_platform]]] "." ext
+
+The "name" and "version" should be escaped using the ``to_filename()``
+function provided by ``pkg_resources``, after first processing them with
+``safe_name()`` and ``safe_version()`` respectively.  These latter two
+functions can also be used to later "unescape" these parts of the
+filename.  (For a detailed description of these transformations, please
+see the "Parsing Utilities" section of the ``pkg_resources`` manual.)
+
+The "pyver" string is the Python major version, as found in the first
+3 characters of ``sys.version``.  "required_platform" is essentially
+a distutils ``get_platform()`` string, but with enhancements to properly
+distinguish Mac OS versions.  (See the ``get_build_platform()``
+documentation in the "Platform Utilities" section of the
+``pkg_resources`` manual for more details.)
+
+Finally, the "ext" is either ``.egg`` or ``.egg-info``, as appropriate
+for the egg's format.
+
+Normally, an egg's filename should include at least the project name and
+version, as this allows the runtime system to find desired project
+versions without having to read the egg's PKG-INFO to determine its
+version number.
+
+Setuptools, however, only includes the version number in the filename
+when an ``.egg`` file is built using the ``bdist_egg`` command, or when
+an ``.egg-info`` directory is being installed by the
+``install_egg_info`` command. When generating metadata for use with the
+original source tree, it only includes the project name, so that the
+directory will not have to be renamed each time the project's version
+changes.
+
+This is especially important when version numbers change frequently, and
+the source metadata directory is kept under version control with the
+rest of the project.  (As would be the case when the project's source
+includes project-defined metadata that is not generated from by
+setuptools from data in the setup script.)
+
+
+Egg Links
+=========
+
+In addition to the ``.egg`` and ``.egg-info`` formats, there is a third
+egg-related extension that you may encounter on occasion: ``.egg-link``
+files.
+
+These files are not eggs, strictly speaking. They simply provide a way
+to reference an egg that is not physically installed in the desired
+location. They exist primarily as a cross-platform alternative to
+symbolic links, to support "installing" code that is being developed in
+a different location than the desired installation location. For
+example, if a user is developing an application plugin in their home
+directory, but the plugin needs to be "installed" in an application
+plugin directory, running "setup.py develop -md /path/to/app/plugins"
+will install an ``.egg-link`` file in ``/path/to/app/plugins``, that
+tells the egg runtime system where to find the actual egg (the user's
+project source directory and its ``.egg-info`` subdirectory).
+
+``.egg-link`` files are named following the format for ``.egg`` and
+``.egg-info`` names, but only the project name is included; no version,
+Python version, or platform information is included.  When the runtime
+searches for available eggs, ``.egg-link`` files are opened and the
+actual egg file/directory name is read from them.
+
+Each ``.egg-link`` file should contain a single file or directory name,
+with no newlines.  This filename should be the base location of one or
+more eggs.  That is, the name must either end in ``.egg``, or else it
+should be the parent directory of one or more ``.egg-info`` format eggs.
+
+As of setuptools 0.6c6, the path may be specified as a platform-independent
+(i.e. ``/``-separated) relative path from the directory containing the
+``.egg-link`` file, and a second line may appear in the file, specifying a
+platform-independent relative path from the egg's base directory to its
+setup script directory.  This allows installation tools such as EasyInstall
+to find the project's setup directory and build eggs or perform other setup
+commands on it.
+
+
+-----------------
+Standard Metadata
+-----------------
+
+In addition to the minimum required ``PKG-INFO`` metadata, projects can
+include a variety of standard metadata files or directories, as
+described below.  Except as otherwise noted, these files and directories
+are automatically generated by setuptools, based on information supplied
+in the setup script or through analysis of the project's code and
+resources.
+
+Most of these files and directories are generated via "egg-info
+writers" during execution of the setuptools ``egg_info`` command, and
+are listed in the ``egg_info.writers`` entry point group defined by
+setuptools' own ``setup.py`` file.
+
+Project authors can register their own metadata writers as entry points
+in this group (as described in the setuptools manual under "Adding new
+EGG-INFO Files") to cause setuptools to generate project-specific
+metadata files or directories during execution of the ``egg_info``
+command.  It is up to project authors to document these new metadata
+formats, if they create any.
+
+
+``.txt`` File Formats
+=====================
+
+Files described in this section that have ``.txt`` extensions have a
+simple lexical format consisting of a sequence of text lines, each line
+terminated by a linefeed character (regardless of platform).  Leading
+and trailing whitespace on each line is ignored, as are blank lines and
+lines whose first nonblank character is a ``#`` (comment symbol).  (This
+is the parsing format defined by the ``yield_lines()`` function of
+the ``pkg_resources`` module.)
+
+All ``.txt`` files defined by this section follow this format, but some
+are also "sectioned" files, meaning that their contents are divided into
+sections, using square-bracketed section headers akin to Windows
+``.ini`` format.  Note that this does *not* imply that the lines within
+the sections follow an ``.ini`` format, however.  Please see an
+individual metadata file's documentation for a description of what the
+lines and section names mean in that particular file.
+
+Sectioned files can be parsed using the ``split_sections()`` function;
+see the "Parsing Utilities" section of the ``pkg_resources`` manual for
+for details.
+
+
+Dependency Metadata
+===================
+
+
+``requires.txt``
+----------------
+
+This is a "sectioned" text file.  Each section is a sequence of
+"requirements", as parsed by the ``parse_requirements()`` function;
+please see the ``pkg_resources`` manual for the complete requirement
+parsing syntax.
+
+The first, unnamed section (i.e., before the first section header) in
+this file is the project's core requirements, which must be installed
+for the project to function.  (Specified using the ``install_requires``
+keyword to ``setup()``).
+
+The remaining (named) sections describe the project's "extra"
+requirements, as specified using the ``extras_require`` keyword to
+``setup()``.  The section name is the name of the optional feature, and
+the section body lists that feature's dependencies.
+
+Note that it is not normally necessary to inspect this file directly;
+``pkg_resources.Distribution`` objects have a ``requires()`` method
+that can be used to obtain ``Requirement`` objects describing the
+project's core and optional dependencies.
+
+
+
+``dependency_links.txt``
+------------------------
+
+A list of dependency URLs, one per line, as specified using the
+``dependency_links`` keyword to ``setup()``.  These may be direct
+download URLs, or the URLs of web pages containing direct download
+links, and will be used by EasyInstall to find dependencies, as though
+the user had manually provided them via the ``--find-links`` command
+line option.  Please see the setuptools manual and EasyInstall manual
+for more information on specifying this option, and for information on
+how EasyInstall processes ``--find-links`` URLs.
+
+
+``depends.txt`` -- Obsolete, do not create!
+-------------------------------------------
+
+This file follows an identical format to ``requires.txt``, but is
+obsolete and should not be used.  The earliest versions of setuptools
+required users to manually create and maintain this file, so the runtime
+still supports reading it, if it exists.  The new filename was created
+so that it could be automatically generated from ``setup()`` information
+without overwriting an existing hand-created ``depends.txt``, if one
+was already present in the project's source ``.egg-info`` directory.
+
+
+``namespace_packages.txt`` -- Namespace Package Metadata
+========================================================
+
+A list of namespace package names, one per line, as supplied to the
+``namespace_packages`` keyword to ``setup()``.  Please see the manuals
+for setuptools and ``pkg_resources`` for more information about
+namespace packages.
+
+
+``entry_points.txt`` -- "Entry Point"/Plugin Metadata
+=====================================================
+
+This is a "sectioned" text file, whose contents encode the
+``entry_points`` keyword supplied to ``setup()``.  All sections are
+named, as the section names specify the entry point groups in which the
+corresponding section's entry points are registered.
+
+Each section is a sequence of "entry point" lines, each parseable using
+the ``EntryPoint.parse`` classmethod; please see the ``pkg_resources``
+manual for the complete entry point parsing syntax.
+
+Note that it is not necessary to parse this file directly; the
+``pkg_resources`` module provides a variety of APIs to locate and load
+entry points automatically.  Please see the setuptools and
+``pkg_resources`` manuals for details on the nature and uses of entry
+points.
+
+
+The ``scripts`` Subdirectory
+============================
+
+This directory is currently only created for ``.egg`` files built by
+the setuptools ``bdist_egg`` command.  It will contain copies of all
+of the project's "traditional" scripts (i.e., those specified using the
+``scripts`` keyword to ``setup()``).  This is so that they can be
+reconstituted when an ``.egg`` file is installed.
+
+The scripts are placed here using the disutils' standard
+``install_scripts`` command, so any ``#!`` lines reflect the Python
+installation where the egg was built.  But instead of copying the
+scripts to the local script installation directory, EasyInstall writes
+short wrapper scripts that invoke the original scripts from inside the
+egg, after ensuring that sys.path includes the egg and any eggs it
+depends on.  For more about `script wrappers`_, see the section below on
+`Installation and Path Management Issues`_.
+
+
+Zip Support Metadata
+====================
+
+
+``native_libs.txt``
+-------------------
+
+A list of C extensions and other dynamic link libraries contained in
+the egg, one per line.  Paths are ``/``-separated and relative to the
+egg's base location.
+
+This file is generated as part of ``bdist_egg`` processing, and as such
+only appears in ``.egg`` files (and ``.egg`` directories created by
+unpacking them).  It is used to ensure that all libraries are extracted
+from a zipped egg at the same time, in case there is any direct linkage
+between them.  Please see the `Zip File Issues`_ section below for more
+information on library and resource extraction from ``.egg`` files.
+
+
+``eager_resources.txt``
+-----------------------
+
+A list of resource files and/or directories, one per line, as specified
+via the ``eager_resources`` keyword to ``setup()``.  Paths are
+``/``-separated and relative to the egg's base location.
+
+Resource files or directories listed here will be extracted
+simultaneously, if any of the named resources are extracted, or if any
+native libraries listed in ``native_libs.txt`` are extracted.  Please
+see the setuptools manual for details on what this feature is used for
+and how it works, as well as the `Zip File Issues`_ section below.
+
+
+``zip-safe`` and ``not-zip-safe``
+---------------------------------
+
+These are zero-length files, and either one or the other should exist.
+If ``zip-safe`` exists, it means that the project will work properly
+when installedas an ``.egg`` zipfile, and conversely the existence of
+``not-zip-safe`` means the project should not be installed as an
+``.egg`` file.  The ``zip_safe`` option to setuptools' ``setup()``
+determines which file will be written. If the option isn't provided,
+setuptools attempts to make its own assessment of whether the package
+can work, based on code and content analysis.
+
+If neither file is present at installation time, EasyInstall defaults
+to assuming that the project should be unzipped.  (Command-line options
+to EasyInstall, however, take precedence even over an existing
+``zip-safe`` or ``not-zip-safe`` file.)
+
+Note that these flag files appear only in ``.egg`` files generated by
+``bdist_egg``, and in ``.egg`` directories created by unpacking such an
+``.egg`` file.
+
+
+
+``top_level.txt`` -- Conflict Management Metadata
+=================================================
+
+This file is a list of the top-level module or package names provided
+by the project, one Python identifier per line.
+
+Subpackages are not included; a project containing both a ``foo.bar``
+and a ``foo.baz`` would include only one line, ``foo``, in its
+``top_level.txt``.
+
+This data is used by ``pkg_resources`` at runtime to issue a warning if
+an egg is added to ``sys.path`` when its contained packages may have
+already been imported.
+
+(It was also once used to detect conflicts with non-egg packages at
+installation time, but in more recent versions, setuptools installs eggs
+in such a way that they always override non-egg packages, thus
+preventing a problem from arising.)
+
+
+``SOURCES.txt`` -- Source Files Manifest
+========================================
+
+This file is roughly equivalent to the distutils' ``MANIFEST`` file.
+The differences are as follows:
+
+* The filenames always use ``/`` as a path separator, which must be
+  converted back to a platform-specific path whenever they are read.
+
+* The file is automatically generated by setuptools whenever the
+  ``egg_info`` or ``sdist`` commands are run, and it is *not*
+  user-editable.
+
+Although this metadata is included with distributed eggs, it is not
+actually used at runtime for any purpose.  Its function is to ensure
+that setuptools-built *source* distributions can correctly discover
+what files are part of the project's source, even if the list had been
+generated using revision control metadata on the original author's
+system.
+
+In other words, ``SOURCES.txt`` has little or no runtime value for being
+included in distributed eggs, and it is possible that future versions of
+the ``bdist_egg`` and ``install_egg_info`` commands will strip it before
+installation or distribution.  Therefore, do not rely on its being
+available outside of an original source directory or source
+distribution.
+
+
+------------------------------
+Other Technical Considerations
+------------------------------
+
+
+Zip File Issues
+===============
+
+Although zip files resemble directories, they are not fully
+substitutable for them.  Most platforms do not support loading dynamic
+link libraries contained in zipfiles, so it is not possible to directly
+import C extensions from ``.egg`` zipfiles.  Similarly, there are many
+existing libraries -- whether in Python or C -- that require actual
+operating system filenames, and do not work with arbitrary "file-like"
+objects or in-memory strings, and thus cannot operate directly on the
+contents of zip files.
+
+To address these issues, the ``pkg_resources`` module provides a
+"resource API" to support obtaining either the contents of a resource,
+or a true operating system filename for the resource.  If the egg
+containing the resource is a directory, the resource's real filename
+is simply returned.  However, if the egg is a zipfile, then the
+resource is first extracted to a cache directory, and the filename
+within the cache is returned.
+
+The cache directory is determined by the ``pkg_resources`` API; please
+see the ``set_cache_path()`` and ``get_default_cache()`` documentation
+for details.
+
+
+The Extraction Process
+----------------------
+
+Resources are extracted to a cache subdirectory whose name is based
+on the enclosing ``.egg`` filename and the path to the resource.  If
+there is already a file of the correct name, size, and timestamp, its
+filename is returned to the requester.  Otherwise, the desired file is
+extracted first to a temporary name generated using
+``mkstemp(".$extract",target_dir)``, and then its timestamp is set to
+match the one in the zip file, before renaming it to its final name.
+(Some collision detection and resolution code is used to handle the
+fact that Windows doesn't overwrite files when renaming.)
+
+If a resource directory is requested, all of its contents are
+recursively extracted in this fashion, to ensure that the directory
+name can be used as if it were valid all along.
+
+If the resource requested for extraction is listed in the
+``native_libs.txt`` or ``eager_resources.txt`` metadata files, then
+*all* resources listed in *either* file will be extracted before the
+requested resource's filename is returned, thus ensuring that all
+C extensions and data used by them will be simultaneously available.
+
+
+Extension Import Wrappers
+-------------------------
+
+Since Python's built-in zip import feature does not support loading
+C extension modules from zipfiles, the setuptools ``bdist_egg`` command
+generates special import wrappers to make it work.
+
+The wrappers are ``.py`` files (along with corresponding ``.pyc``
+and/or ``.pyo`` files) that have the same module name as the
+corresponding C extension.  These wrappers are located in the same
+package directory (or top-level directory) within the zipfile, so that
+say, ``foomodule.so`` will get a corresponding ``foo.py``, while
+``bar/baz.pyd`` will get a corresponding ``bar/baz.py``.
+
+These wrapper files contain a short stanza of Python code that asks
+``pkg_resources`` for the filename of the corresponding C extension,
+then reloads the module using the obtained filename.  This will cause
+``pkg_resources`` to first ensure that all of the egg's C extensions
+(and any accompanying "eager resources") are extracted to the cache
+before attempting to link to the C library.
+
+Note, by the way, that ``.egg`` directories will also contain these
+wrapper files.  However, Python's default import priority is such that
+C extensions take precedence over same-named Python modules, so the
+import wrappers are ignored unless the egg is a zipfile.
+
+
+Installation and Path Management Issues
+=======================================
+
+Python's initial setup of ``sys.path`` is very dependent on the Python
+version and installation platform, as well as how Python was started
+(i.e., script vs. ``-c`` vs. ``-m`` vs. interactive interpreter).
+In fact, Python also provides only two relatively robust ways to affect
+``sys.path`` outside of direct manipulation in code: the ``PYTHONPATH``
+environment variable, and ``.pth`` files.
+
+However, with no cross-platform way to safely and persistently change
+environment variables, this leaves ``.pth`` files as EasyInstall's only
+real option for persistent configuration of ``sys.path``.
+
+But ``.pth`` files are rather strictly limited in what they are allowed
+to do normally.  They add directories only to the *end* of ``sys.path``,
+after any locally-installed ``site-packages`` directory, and they are
+only processed *in* the ``site-packages`` directory to start with.
+
+This is a double whammy for users who lack write access to that
+directory, because they can't create a ``.pth`` file that Python will
+read, and even if a sympathetic system administrator adds one for them
+that calls ``site.addsitedir()`` to allow some other directory to
+contain ``.pth`` files, they won't be able to install newer versions of
+anything that's installed in the systemwide ``site-packages``, because
+their paths will still be added *after* ``site-packages``.
+
+So EasyInstall applies two workarounds to solve these problems.
+
+The first is that EasyInstall leverages ``.pth`` files' "import" feature
+to manipulate ``sys.path`` and ensure that anything EasyInstall adds
+to a ``.pth`` file will always appear before both the standard library
+and the local ``site-packages`` directories.  Thus, it is always
+possible for a user who can write a Python-read ``.pth`` file to ensure
+that their packages come first in their own environment.
+
+Second, when installing to a ``PYTHONPATH`` directory (as opposed to
+a "site" directory like ``site-packages``) EasyInstall will also install
+a special version of the ``site`` module.  Because it's in a
+``PYTHONPATH`` directory, this module will get control before the
+standard library version of ``site`` does.  It will record the state of
+``sys.path`` before invoking the "real" ``site`` module, and then
+afterwards it processes any ``.pth`` files found in ``PYTHONPATH``
+directories, including all the fixups needed to ensure that eggs always
+appear before the standard library in sys.path, but are in a relative
+order to one another that is defined by their ``PYTHONPATH`` and
+``.pth``-prescribed sequence.
+
+The net result of these changes is that ``sys.path`` order will be
+as follows at runtime:
+
+1. The ``sys.argv[0]`` directory, or an emtpy string if no script
+   is being executed.
+
+2. All eggs installed by EasyInstall in any ``.pth`` file in each
+   ``PYTHONPATH`` directory, in order first by ``PYTHONPATH`` order,
+   then normal ``.pth`` processing order (which is to say alphabetical
+   by ``.pth`` filename, then by the order of listing within each
+   ``.pth`` file).
+
+3. All eggs installed by EasyInstall in any ``.pth`` file in each "site"
+   directory (such as ``site-packages``), following the same ordering
+   rules as for the ones on ``PYTHONPATH``.
+
+4. The ``PYTHONPATH`` directories themselves, in their original order
+
+5. Any paths from ``.pth`` files found on ``PYTHONPATH`` that were *not*
+   eggs installed by EasyInstall, again following the same relative
+   ordering rules.
+
+6. The standard library and "site" directories, along with the contents
+   of any ``.pth`` files found in the "site" directories.
+
+Notice that sections 1, 4, and 6 comprise the "normal" Python setup for
+``sys.path``.  Sections 2 and 3 are inserted to support eggs, and
+section 5 emulates what the "normal" semantics of ``.pth`` files on
+``PYTHONPATH`` would be if Python natively supported them.
+
+For further discussion of the tradeoffs that went into this design, as
+well as notes on the actual magic inserted into ``.pth`` files to make
+them do these things, please see also the following messages to the
+distutils-SIG mailing list:
+
+* http://mail.python.org/pipermail/distutils-sig/2006-February/006026.html
+* http://mail.python.org/pipermail/distutils-sig/2006-March/006123.html
+
+
+Script Wrappers
+---------------
+
+EasyInstall never directly installs a project's original scripts to
+a script installation directory.  Instead, it writes short wrapper
+scripts that first ensure that the project's dependencies are active
+on sys.path, before invoking the original script.  These wrappers
+have a #! line that points to the version of Python that was used to
+install them, and their second line is always a comment that indicates
+the type of script wrapper, the project version required for the script
+to run, and information identifying the script to be invoked.
+
+The format of this marker line is::
+
+    "# EASY-INSTALL-" script_type ": " tuple_of_strings "\n"
+
+The ``script_type`` is one of ``SCRIPT``, ``DEV-SCRIPT``, or
+``ENTRY-SCRIPT``.  The ``tuple_of_strings`` is a comma-separated
+sequence of Python string constants.  For ``SCRIPT`` and ``DEV-SCRIPT``
+wrappers, there are two strings: the project version requirement, and
+the script name (as a filename within the ``scripts`` metadata
+directory).  For ``ENTRY-SCRIPT`` wrappers, there are three:
+the project version requirement, the entry point group name, and the
+entry point name.  (See the "Automatic Script Creation" section in the
+setuptools manual for more information about entry point scripts.)
+
+In each case, the project version requirement string will be a string
+parseable with the ``pkg_resources`` modules' ``Requirement.parse()``
+classmethod.  The only difference between a ``SCRIPT`` wrapper and a
+``DEV-SCRIPT`` is that a ``DEV-SCRIPT`` actually executes the original
+source script in the project's source tree, and is created when the
+"setup.py develop" command is run.  A ``SCRIPT`` wrapper, on the other
+hand, uses the "installed" script written to the ``EGG-INFO/scripts``
+subdirectory of the corresponding ``.egg`` zipfile or directory.
+(``.egg-info`` eggs do not have script wrappers associated with them,
+except in the "setup.py develop" case.)
+
+The purpose of including the marker line in generated script wrappers is
+to facilitate introspection of installed scripts, and their relationship
+to installed eggs.  For example, an uninstallation tool could use this
+data to identify what scripts can safely be removed, and/or identify
+what scripts would stop working if a particular egg is uninstalled.
+

pkg_resources.py

@@ -152,7 +152,7 @@ __all__ = [
     # Parsing functions and string utilities
     'parse_requirements', 'parse_version', 'safe_name', 'safe_version',
     'get_platform', 'compatible_platforms', 'yield_lines', 'split_sections',
-    'safe_extra', 'to_filename',
+    'safe_extra', 'to_filename', 'invalid_marker', 'evaluate_marker',
 
     # filesystem utilities
     'ensure_directory', 'normalize_path',
@@ -1170,6 +1170,129 @@ def to_filename(name):
 
 
 
+_marker_names = {
+    'os': ['name'], 'sys': ['platform'],
+    'platform': ['version','machine','python_implementation'],
+    'python_version': [], 'python_full_version': [], 'extra':[],
+}
+
+_marker_values = {
+    'os_name': lambda: os.name,
+    'sys_platform': lambda: sys.platform,
+    'python_full_version': lambda: sys.version.split()[0],
+    'python_version': lambda:'%s.%s' % (sys.version_info[0], sys.version_info[1]),
+    'platform_version': lambda: _platinfo('version'),
+    'platform_machine': lambda: _platinfo('machine'),
+    'python_implementation': lambda: _platinfo('python_implementation') or _pyimp(),
+}
+
+def _platinfo(attr):
+    try:
+        import platform
+    except ImportError:
+        return ''
+    return getattr(platform, attr, lambda:'')()
+
+def _pyimp():
+    if sys.platform=='cli':
+        return 'IronPython'
+    elif sys.platform.startswith('java'):
+        return 'Jython'
+    elif '__pypy__' in sys.builtin_module_names:
+        return 'PyPy'
+    else:
+        return 'CPython'                
+
+def invalid_marker(text):
+    """Validate text as a PEP 426 environment marker; return exception or False"""
+    try:
+        evaluate_marker(text)
+    except SyntaxError:
+        return sys.exc_info()[1]            
+    return False
+
+def evaluate_marker(text, extra=None, _ops={}):
+    """Evaluate a PEP 426 environment marker; SyntaxError if marker is invalid"""
+
+    if not _ops:
+
+        from token import NAME, STRING
+        import token, symbol, operator
+        
+        def and_test(nodelist):
+            # MUST NOT short-circuit evaluation, or invalid syntax can be skipped!
+            return reduce(operator.and_, [interpret(nodelist[i]) for i in range(1,len(nodelist),2)])
+        
+        def test(nodelist):
+            # MUST NOT short-circuit evaluation, or invalid syntax can be skipped!
+            return reduce(operator.or_, [interpret(nodelist[i]) for i in range(1,len(nodelist),2)])
+        
+        def atom(nodelist):
+            t = nodelist[1][0]
+            if t == token.LPAR:
+                if nodelist[2][0] == token.RPAR:
+                    raise SyntaxError("Empty parentheses")
+                return interpret(nodelist[2])
+            raise SyntaxError("Language feature not supported in environment markers")
+        
+        def comparison(nodelist):
+            if len(nodelist)>4:
+                raise SyntaxError("Chained comparison not allowed in environment markers")
+            comp = nodelist[2][1]
+            cop = comp[1]
+            if comp[0] == NAME:
+                if len(nodelist[2]) == 3:
+                    if cop == 'not':
+                        cop = 'not in'
+                    else:
+                        cop = 'is not'
+            try:
+                cop = _ops[cop]
+            except KeyError:
+                raise SyntaxError(repr(cop)+" operator not allowed in environment markers")
+            return cop(evaluate(nodelist[1]), evaluate(nodelist[3]))
+        
+        _ops.update({
+            symbol.test: test, symbol.and_test: and_test, symbol.atom: atom,
+            symbol.comparison: comparison, 'not in': lambda x,y: x not in y,
+            'in': lambda x,y: x in y, '==': operator.eq, '!=': operator.ne,
+        })
+        if hasattr(symbol,'or_test'):
+            _ops[symbol.or_test] = test
+
+    def interpret(nodelist):
+        while len(nodelist)==2: nodelist = nodelist[1]
+        try:
+            op = _ops[nodelist[0]]
+        except KeyError:
+            raise SyntaxError("Comparison or logical expression expected")
+            raise SyntaxError("Language feature not supported in environment markers: "+symbol.sym_name[nodelist[0]])
+        return op(nodelist)
+        
+    def evaluate(nodelist):
+        while len(nodelist)==2: nodelist = nodelist[1]
+        kind = nodelist[0]
+        name = nodelist[1]
+        #while len(name)==2: name = name[1]
+        if kind==NAME:
+            try:
+                op = _marker_values[name]
+            except KeyError:
+                raise SyntaxError("Unknown name %r" % name)
+            return op()
+        if kind==STRING:
+            s = nodelist[1]
+            if s[:1] not in "'\"" or s.startswith('"""') or s.startswith("'''") \
+            or '\\' in s:
+                raise SyntaxError(
+                    "Only plain strings allowed in environment markers")
+            return s[1:-1]
+        raise SyntaxError("Language feature not supported in environment markers")
+
+    import parser        
+    return interpret(parser.expr(text).totuple(1)[1])
+
+
 class NullProvider:
     """Try to implement resources and metadata for arbitrary PEP 302 loaders"""
 
@@ -1994,7 +2117,6 @@ def parse_version(s):
                 parts.pop()
         parts.append(part)
     return tuple(parts)
-
 class EntryPoint(object):
     """Object representing an advertised importable object"""
 
@@ -2235,7 +2357,14 @@ class Distribution(object):
             dm = self.__dep_map = {None: []}
             for name in 'requires.txt', 'depends.txt':
                 for extra,reqs in split_sections(self._get_metadata(name)):
-                    if extra: extra = safe_extra(extra)
+                    if extra:
+                        if ':' in extra:
+                            extra, marker = extra.split(':',1)
+                            if invalid_marker(marker):
+                                reqs=[] # XXX warn
+                            elif not evaluate_marker(marker):
+                                reqs=[]
+                        extra = safe_extra(extra) or None                                                            
                     dm.setdefault(extra,[]).extend(parse_requirements(reqs))
             return dm
     _dep_map = property(_dep_map)
@@ -2259,6 +2388,8 @@ class Distribution(object):
             for line in self.get_metadata_lines(name):
                 yield line
 
+
+
     def activate(self,path=None):
         """Ensure distribution is importable on `path` (default=sys.path)"""
         if path is None: path = sys.path
@@ -2297,6 +2428,9 @@ class Distribution(object):
             raise AttributeError,attr
         return getattr(self._provider, attr)
 
+
+
+
     #@classmethod
     def from_filename(cls,filename,metadata=None, **kw):
         return cls.from_location(
@@ -2338,18 +2472,6 @@ class Distribution(object):
 
 
 
-
-
-
-
-
-
-
-
-
-
-
-
     def insert_on(self, path, loc = None):
         """Insert self.location in path before its nearest parent directory"""
 

setup.py

@@ -207,5 +207,22 @@ dist = setup(
         Topic :: System :: Systems Administration
         Topic :: Utilities
         """).strip().splitlines(),
-    scripts = scripts,
+    extras_require = {
+        "ssl:sys_platform=='win32'": "wincertstore==0.1",
+        "ssl:sys_platform=='win32' and python_version in '2.3, 2.4'": "ctypes==1.0.2",
+        "ssl:python_version in '2.3, 2.4, 2.5'":"ssl==1.16",
+        "certs": "certifi==0.0.8",
+    },
+    dependency_links = [
+        'http://pypi.python.org/packages/source/c/certifi/certifi-0.0.8.tar.gz#md5=dc5f5e7f0b5fc08d27654b17daa6ecec',
+        'http://pypi.python.org/packages/source/s/ssl/ssl-1.16.tar.gz#md5=fb12d335d56f3c8c7c1fefc1c06c4bfb',
+        'http://pypi.python.org/packages/source/w/wincertstore/wincertstore-0.1.zip#md5=2f9accbebe8f7b4c06ac7aa83879b81c',
+        'http://sourceforge.net/projects/ctypes/files/ctypes/1.0.2/ctypes-1.0.2.win32-py2.3.exe/download#md5=9afe4b75240a8808a24df7a76b6081e3',
+        'http://sourceforge.net/projects/ctypes/files/ctypes/1.0.2/ctypes-1.0.2.win32-py2.4.exe/download#md5=9092a0ad5a3d79fa2d980f1ddc5e9dbc',
+        'http://peak.telecommunity.com/dist/ssl-1.16-py2.3-win32.egg#md5=658f74b3eb6f32050e8531bb73de8e74',
+        'http://peak.telecommunity.com/dist/ssl-1.16-py2.4-win32.egg#md5=3cfa2c526dc66e318e8520b6f1aadce5',
+        'http://peak.telecommunity.com/dist/ssl-1.16-py2.5-win32.egg#md5=85ad1cda806d639743121c0bbcb5f39b',
+    ],
+    scripts = [],
+    # tests_require = "setuptools[ssl]",
 )

setuptools.egg-info/dependency_links.txt

+http://pypi.python.org/packages/source/c/certifi/certifi-0.0.8.tar.gz#md5=dc5f5e7f0b5fc08d27654b17daa6ecec
+http://pypi.python.org/packages/source/s/ssl/ssl-1.16.tar.gz#md5=fb12d335d56f3c8c7c1fefc1c06c4bfb
+http://pypi.python.org/packages/source/w/wincertstore/wincertstore-0.1.zip#md5=2f9accbebe8f7b4c06ac7aa83879b81c
+http://sourceforge.net/projects/ctypes/files/ctypes/1.0.2/ctypes-1.0.2.win32-py2.3.exe/download#md5=9afe4b75240a8808a24df7a76b6081e3
+http://sourceforge.net/projects/ctypes/files/ctypes/1.0.2/ctypes-1.0.2.win32-py2.4.exe/download#md5=9092a0ad5a3d79fa2d980f1ddc5e9dbc
+http://peak.telecommunity.com/dist/ssl-1.16-py2.3-win32.egg#md5=658f74b3eb6f32050e8531bb73de8e74
+http://peak.telecommunity.com/dist/ssl-1.16-py2.4-win32.egg#md5=3cfa2c526dc66e318e8520b6f1aadce5
+http://peak.telecommunity.com/dist/ssl-1.16-py2.5-win32.egg#md5=85ad1cda806d639743121c0bbcb5f39b

setuptools.egg-info/entry_points.txt

@@ -7,6 +7,7 @@ build_py = setuptools.command.build_py:build_py
 saveopts = setuptools.command.saveopts:saveopts
 egg_info = setuptools.command.egg_info:egg_info
 register = setuptools.command.register:register
+upload = setuptools.command.upload:upload
 upload_docs = setuptools.command.upload_docs:upload_docs
 install_egg_info = setuptools.command.install_egg_info:install_egg_info
 alias = setuptools.command.alias:alias

setuptools.egg-info/requires.txt

+
+
+[ssl:sys_platform=='win32']
+wincertstore==0.1
+
+[certs]
+certifi==0.0.8
+
+[ssl:sys_platform=='win32' and python_version in '2.3, 2.4']
+ctypes==1.0.2
+
+[ssl:python_version in '2.3, 2.4, 2.5']
+ssl==1.16
\ No newline at end of file

setuptools/command/easy_install.py

@@ -283,7 +283,7 @@ class easy_install(Command):
                 else:
                     self.all_site_dirs.append(normalize_path(d))
         if not self.editable: self.check_site_dir()
-        self.index_url = self.index_url or "http://pypi.python.org/simple"
+        self.index_url = self.index_url or "https://pypi.python.org/simple"
         self.shadow_path = self.all_site_dirs[:]
         for path_item in self.install_dir, normalize_path(self.script_dir):
             if path_item not in self.shadow_path:

setuptools/dist.py

@@ -48,7 +48,6 @@ def assert_string_list(dist, attr, value):
         raise DistutilsSetupError(
             "%r must be a list of strings (got %r)" % (attr,value)
         )
-
 def check_nsp(dist, attr, value):
     """Verify that namespace packages are valid"""
     assert_string_list(dist,attr,value)
@@ -70,6 +69,10 @@ def check_extras(dist, attr, value):
     """Verify that extras_require mapping is valid"""
     try:
         for k,v in value.items():
+            if ':' in k:
+                k,m = k.split(':',1)
+                if pkg_resources.invalid_marker(m):
+                    raise DistutilsSetupError("Invalid environment marker: "+m)
             list(pkg_resources.parse_requirements(v))
     except (TypeError,ValueError,AttributeError):
         raise DistutilsSetupError(
@@ -78,9 +81,6 @@ def check_extras(dist, attr, value):
             "requirement specifiers."
         )
 
-
-
-
 def assert_bool(dist, attr, value):
     """Verify that value is True, False, 0, or 1"""
     if bool(value) != value:

setuptools/package_index.py

@@ -2,6 +2,7 @@
 import sys, os.path, re, urlparse, urllib2, shutil, random, socket, cStringIO
 import base64
 import httplib, urllib
+from setuptools import ssl_support
 from pkg_resources import *
 from distutils import log
 from distutils.errors import DistutilsError
@@ -157,12 +158,11 @@ user_agent = "Python-urllib/%s setuptools/%s" % (
     sys.version[:3], require('setuptools')[0].version
 )
 
-
 class PackageIndex(Environment):
     """A distribution index that scans web pages for download URLs"""
 
-    def __init__(self, index_url="http://pypi.python.org/simple", hosts=('*',),
-        *args, **kw
+    def __init__(self, index_url="https://pypi.python.org/simple", hosts=('*',),
+        ca_bundle=None, verify_ssl=True, *args, **kw
     ):
         Environment.__init__(self,*args,**kw)
         self.index_url = index_url + "/"[:not index_url.endswith('/')]
@@ -171,8 +171,9 @@ class PackageIndex(Environment):
         self.package_pages = {}
         self.allows = re.compile('|'.join(map(translate,hosts))).match
         self.to_scan = []
-
-
+        if verify_ssl and ssl_support.is_available and (ca_bundle or ssl_support.find_ca_bundle()):
+            self.opener = ssl_support.opener_for(ca_bundle)
+        else: self.opener = urllib2.urlopen
 
     def process_url(self, url, retrieve=False):
         """Evaluate a URL as a possible download, and maybe retrieve it"""
@@ -601,7 +602,7 @@ class PackageIndex(Environment):
         if url.startswith('file:'):
             return local_open(url)
         try:
-            return open_with_auth(url)
+            return open_with_auth(url, self.opener)
         except (ValueError, httplib.InvalidURL), v:
             msg = ' '.join([str(arg) for arg in v.args])
             if warning:
@@ -659,7 +660,6 @@ class PackageIndex(Environment):
             self.url_ok(url, True)   # raises error if not allowed
             return self._attempt_download(url, filename)
 
-
     def scan_url(self, url):
         self.process_url(url, True)
 
@@ -859,7 +859,7 @@ def _encode_auth(auth):
     # strip the trailing carriage return
     return encoded.rstrip()
 
-def open_with_auth(url):
+def open_with_auth(url, opener=urllib2.urlopen):
     """Open a urllib2 request, handling HTTP authentication"""
 
     scheme, netloc, path, params, query, frag = urlparse.urlparse(url)
@@ -883,7 +883,7 @@ def open_with_auth(url):
         request = urllib2.Request(url)
 
     request.add_header('User-Agent', user_agent)
-    fp = urllib2.urlopen(request)
+    fp = opener(request)
 
     if auth:
         # Put authentication info back into request URL if same host,

setuptools/ssl_support.py

+import sys, os, socket, urllib2, atexit, re
+from pkg_resources import ResolutionError, ExtractionError
+
+try:
+    import ssl
+except ImportError:
+    ssl = None
+
+__all__ = [
+    'VerifyingHTTPSHandler', 'find_ca_bundle', 'is_available', 'cert_paths',
+    'opener_for'
+]
+
+cert_paths = """
+/etc/pki/tls/certs/ca-bundle.crt
+/etc/ssl/certs/ca-certificates.crt
+/usr/share/ssl/certs/ca-bundle.crt
+/usr/local/share/certs/ca-root.crt
+/etc/ssl/cert.pem
+/System/Library/OpenSSL/certs/cert.pem
+""".strip().split()
+
+
+HTTPSHandler = HTTPSConnection = object
+
+for what, where in (
+    ('HTTPSHandler', ['urllib2','urllib.request']),
+    ('HTTPSConnection', ['httplib', 'http.client']),
+):
+    for module in where:
+        try:
+            exec("from %s import %s" % (module, what))
+        except ImportError:
+            pass
+
+is_available = ssl is not None and object not in (HTTPSHandler, HTTPSConnection)
+
+
+
+
+
+try:
+    from socket import create_connection
+except ImportError:
+    _GLOBAL_DEFAULT_TIMEOUT = getattr(socket, '_GLOBAL_DEFAULT_TIMEOUT', object())
+    def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
+                          source_address=None):
+        """Connect to *address* and return the socket object.
+    
+        Convenience function.  Connect to *address* (a 2-tuple ``(host,
+        port)``) and return the socket object.  Passing the optional
+        *timeout* parameter will set the timeout on the socket instance
+        before attempting to connect.  If no *timeout* is supplied, the
+        global default timeout setting returned by :func:`getdefaulttimeout`
+        is used.  If *source_address* is set it must be a tuple of (host, port)
+        for the socket to bind as a source address before making the connection.
+        An host of '' or port 0 tells the OS to use the default.
+        """   
+        host, port = address
+        err = None
+        for res in socket.getaddrinfo(host, port, 0, socket.SOCK_STREAM):
+            af, socktype, proto, canonname, sa = res
+            sock = None
+            try:
+                sock = socket.socket(af, socktype, proto)
+                if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
+                    sock.settimeout(timeout)
+                if source_address:
+                    sock.bind(source_address)
+                sock.connect(sa)
+                return sock
+
+            except error:
+                err = True
+                if sock is not None:
+                    sock.close()    
+        if err:
+            raise
+        else:
+            raise error("getaddrinfo returns an empty list")
+
+
+try:
+    from ssl import CertificateError, match_hostname
+except ImportError:
+    class CertificateError(ValueError):
+        pass
+       
+    def _dnsname_to_pat(dn):
+        pats = []
+        for frag in dn.split(r'.'):
+            if frag == '*':
+                # When '*' is a fragment by itself, it matches a non-empty dotless
+                # fragment.
+                pats.append('[^.]+')
+            else:
+                # Otherwise, '*' matches any dotless fragment.
+                frag = re.escape(frag)
+                pats.append(frag.replace(r'\*', '[^.]*'))
+        return re.compile(r'\A' + r'\.'.join(pats) + r'\Z', re.IGNORECASE)
+       
+    def match_hostname(cert, hostname):
+        """Verify that *cert* (in decoded format as returned by
+        SSLSocket.getpeercert()) matches the *hostname*.  RFC 2818 rules
+        are mostly followed, but IP addresses are not accepted for *hostname*.
+    
+        CertificateError is raised on failure. On success, the function
+        returns nothing.
+        """
+        if not cert:
+            raise ValueError("empty or no certificate")
+        dnsnames = []
+        san = cert.get('subjectAltName', ())
+        for key, value in san:
+            if key == 'DNS':
+                if _dnsname_to_pat(value).match(hostname):
+                    return
+                dnsnames.append(value)
+        if not dnsnames:
+            # The subject is only checked when there is no dNSName entry
+            # in subjectAltName
+            for sub in cert.get('subject', ()):
+                for key, value in sub:
+                    # XXX according to RFC 2818, the most specific Common Name
+                    # must be used.
+                    if key == 'commonName':
+                        if _dnsname_to_pat(value).match(hostname):
+                            return
+                        dnsnames.append(value)
+        if len(dnsnames) > 1:
+            raise CertificateError("hostname %r "
+                "doesn't match either of %s"
+                % (hostname, ', '.join(map(repr, dnsnames))))
+        elif len(dnsnames) == 1:
+            raise CertificateError("hostname %r "
+                "doesn't match %r"
+                % (hostname, dnsnames[0]))
+        else:
+            raise CertificateError("no appropriate commonName or "
+                "subjectAltName fields were found")
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+class VerifyingHTTPSHandler(HTTPSHandler):
+    """Simple verifying handler: no auth, subclasses, timeouts, etc."""
+
+    def __init__(self, ca_bundle):
+        self.ca_bundle = ca_bundle
+        HTTPSHandler.__init__(self)
+
+    def https_open(self, req):
+        return self.do_open(
+            lambda host, **kw: VerifyingHTTPSConn(host, self.ca_bundle, **kw), req
+        )
+
+
+class VerifyingHTTPSConn(HTTPSConnection):
+    """Simple verifying connection: no auth, subclasses, timeouts, etc."""
+    def __init__(self, host, ca_bundle, **kw):         
+        HTTPSConnection.__init__(self, host, **kw)
+        self.ca_bundle = ca_bundle
+
+    def connect(self):
+        sock = create_connection(
+            (self.host, self.port), getattr(self,'source_address',None)
+        )
+        self.sock = ssl.wrap_socket(
+            sock, cert_reqs=ssl.CERT_REQUIRED, ca_certs=self.ca_bundle
+        )               
+        try:
+            match_hostname(self.sock.getpeercert(), self.host)
+        except CertificateError:
+            self.sock.shutdown(socket.SHUT_RDWR)
+            self.sock.close()
+            raise
+
+def opener_for(ca_bundle=None):
+    """Get a urlopen() replacement that uses ca_bundle for verification"""
+    return urllib2.build_opener(
+        VerifyingHTTPSHandler(ca_bundle or find_ca_bundle())
+    ).open
+
+        
+
+_wincerts = None
+
+def get_win_certfile():
+    global _wincerts
+    if _wincerts is not None:
+        return _wincerts.name
+
+    try:        
+        from wincertstore import CertFile
+    except ImportError:
+        return None
+
+    class MyCertFile(CertFile):
+        def __init__(self, stores=(), certs=()):
+            CertFile.__init__(self)
+            for store in stores:
+                self.addstore(store)
+            self.addcerts(certs)
+            atexit.register(self.close)            
+
+    _wincerts = MyCertFile(stores=['CA', 'ROOT'])
+    return _wincerts.name
+
+
+def find_ca_bundle():
+    """Return an existing CA bundle path, or None"""
+    if os.name=='nt':
+        return get_win_certfile()
+    else:
+        for cert_path in cert_paths:
+            if os.path.isfile(cert_path):
+                return cert_path
+    try:
+        return pkg_resources.resource_filename('certifi', 'cacert.pem')
+    except (ImportError, ResolutionError, ExtractionError):
+        return None
+
+
+
+
+

setuptools/tests/test_resources.py

@@ -527,7 +527,7 @@ class ScriptHeaderTests(TestCase):
 
         platform = sys.platform
         sys.platform = 'java1.5.0_13'
-        stdout = sys.stdout
+        stdout, stderr = sys.stdout, sys.stderr
         try:
             # A mock sys.executable that uses a shebang line (this file)
             exe = os.path.normpath(os.path.splitext(__file__)[0] + '.py')
@@ -550,7 +550,7 @@ class ScriptHeaderTests(TestCase):
         finally:
             del sys.modules["java"]
             sys.platform = platform
-            sys.stdout = stdout
+            sys.stdout, sys.stderr = stdout, stderr
 
 
 

tests/api_tests.txt

@@ -119,7 +119,7 @@ editing are also a Distribution.  (And, with a little attention to the
 directory names used,  and including some additional metadata, such a
 "development distribution" can be made pluggable as well.)
 
-    >>> from pkg_resources import WorkingSet, VersionConflict
+    >>> from pkg_resources import WorkingSet
 
 A working set's entries are the sys.path entries that correspond to the active
 distributions.  By default, the working set's entries are the items on
@@ -170,8 +170,8 @@ You can append a path entry to a working set using ``add_entry()``::
     >>> ws.entries
     ['http://example.com/something']
     >>> ws.add_entry(pkg_resources.__file__)
-    >>> ws.entries == ['http://example.com/something', pkg_resources.__file__]
-    True
+    >>> ws.entries
+    ['http://example.com/something', '...pkg_resources.py...']
 
 Multiple additions result in multiple entries, even if the entry is already in
 the working set (because ``sys.path`` can contain the same entry more than
@@ -208,11 +208,11 @@ You can ask a WorkingSet to ``find()`` a distribution matching a requirement::
 Note that asking for a conflicting version of a distribution already in a
 working set triggers a ``pkg_resources.VersionConflict`` error:
 
-    >>> try:
-    ...     ws.find(Requirement.parse("Bar==1.0"))
-    ... except VersionConflict:
-    ...     print 'ok'
-    ok
+    >>> ws.find(Requirement.parse("Bar==1.0")) # doctest: +NORMALIZE_WHITESPACE
+    Traceback (most recent call last):
+      ...
+    VersionConflict: (Bar 0.9 (http://example.com/something),
+                      Requirement.parse('Bar==1.0'))
 
 You can subscribe a callback function to receive notifications whenever a new
 distribution is added to a working set.  The callback is immediately invoked
@@ -328,3 +328,94 @@ setuptools is provided as well::
     >>> cp("darwin-8.2.0-Power_Macintosh", "macosx-10.3-ppc")
     False
 
+
+Environment Markers
+-------------------
+
+    >>> from pkg_resources import invalid_marker as im, evaluate_marker as em
+    >>> import os
+
+    >>> print(im("sys_platform"))
+    Comparison or logical expression expected
+
+    >>> print(im("sys_platform=="))
+    unexpected EOF while parsing (line 1)
+    
+    >>> print(im("sys_platform=='win32'"))
+    False
+
+    >>> print(im("sys=='x'"))
+    Unknown name 'sys'
+
+    >>> print(im("(extra)"))
+    Comparison or logical expression expected
+
+    >>> print(im("(extra"))
+    unexpected EOF while parsing (line 1)
+
+    >>> print(im("os.open('foo')=='y'"))
+    Language feature not supported in environment markers
+    
+    >>> print(im("'x'=='y' and os.open('foo')=='y'"))   # no short-circuit!
+    Language feature not supported in environment markers
+
+    >>> print(im("'x'=='x' or os.open('foo')=='y'"))   # no short-circuit!
+    Language feature not supported in environment markers
+
+    >>> print(im("'x' < 'y'"))
+    '<' operator not allowed in environment markers
+    
+    >>> print(im("'x' < 'y' < 'z'"))
+    Chained comparison not allowed in environment markers
+
+    >>> print(im("r'x'=='x'"))
+    Only plain strings allowed in environment markers
+    
+    >>> print(im("'''x'''=='x'"))
+    Only plain strings allowed in environment markers
+    
+    >>> print(im('"""x"""=="x"'))
+    Only plain strings allowed in environment markers
+    
+    >>> print(im(r"'x\n'=='x'"))
+    Only plain strings allowed in environment markers
+
+    >>> print(im("os.open=='y'"))
+    Language feature not supported in environment markers
+    
+    >>> em('"x"=="x"')
+    True
+
+    >>> em('"x"=="y"')
+    False
+
+    >>> em('"x"=="y" and "x"=="x"')
+    False
+
+    >>> em('"x"=="y" or "x"=="x"')
+    True
+
+    >>> em('"x"=="y" and "x"=="q" or "z"=="z"')
+    True
+
+    >>> em('"x"=="y" and ("x"=="q" or "z"=="z")')
+    False
+
+    >>> em('"x"=="y" and "z"=="z" or "x"=="q"')
+    False
+
+    >>> em('"x"=="x" and "z"=="z" or "x"=="q"')
+    True
+
+    >>> em("sys_platform=='win32'") == (sys.platform=='win32')
+    True
+
+    >>> em("'x' in 'yx'")
+    True
+
+    >>> em("'yx' in 'x'")
+    False
+
+
+
+