Re-working docs to make space to document the new testrunner options.

Also documented the extras, and added a simpler one.

dev-requirements.txt

@@ -9,10 +9,8 @@ pylint>=1.8.0 ; python_version < "3.4"
 pylint >= 2.3.1 ; python_version >= "3.4"  and platform_python_implementation == "CPython"
 astroid >= 2.2.5 ; python_version >= "3.4" and platform_python_implementation == "CPython"
 
-# We need this at runtime to use the libev-CFFI and libuv backends
-cffi >= 1.12.2 ; platform_python_implementation == 'CPython'
 
 # benchmarks use this
 perf >= 1.6.0
 
--e .[test,events,dnspython,docs]
+-e .[test,docs]

docs/_about.rst

@@ -51,5 +51,5 @@ Check out the detailed changelog_ for this version.
 .. _inspired by eventlet: http://blog.gevent.org/2010/02/27/why-gevent/
 .. _use gevent: http://groups.google.com/group/gevent/browse_thread/thread/4de9703e5dca8271
 .. _open source projects based on gevent: https://github.com/gevent/gevent/wiki/Projects
-.. _what's new: http://www.gevent.org/whatsnew_1_4.html
+.. _what's new: http://www.gevent.org/whatsnew_1_5.html
 .. _changelog: http://www.gevent.org/changelog.html

docs/contents.rst

@@ -10,6 +10,7 @@ Introduction and Basics
 
    install
    intro
+   whatsnew_1_5
    whatsnew_1_4
    whatsnew_1_3
    api/gevent
@@ -20,6 +21,8 @@ Introduction and Basics
    loop_impls
    configuration
    changelog
+   installing_from_source
+   development
 
 API Details
 ===========

docs/development.rst

+=============
+ Development
+=============
+
+To install the latest development version::
+
+  pip install git+git://github.com/gevent/gevent.git#egg=gevent
+
+.. note::
+
+   You will not be able to run gevent's test suite using that method.
+
+To hack on gevent (using a virtualenv)::
+
+  $ git clone https://github.com/gevent/gevent.git
+  $ cd gevent
+  $ virtualenv env
+  $ source env/bin/activate
+  (env) $ pip install -r dev-requirements.txt
+
+.. note::
+
+   The notes above about installing from source apply here as well.
+   The ``dev-requirements.txt`` file takes care of the library
+   prerequisites (CFFI, Cython), but having a working C compiler that
+   can create Python extensions is up to you.
+
+.. warning::
+
+   This pip command does not work with pip 19.1. Either use pip 19.0
+   or below, or use pip 19.1.1 with ``--no-use-pep517``. See `issue
+   1412 <https://github.com/gevent/gevent/issues/1412>`_.
+
+
+Running Tests
+-------------
+
+There are a few different ways to run the tests. To simply run the
+tests on one version of Python during development, begin with the
+above instructions to install gevent in a virtual environment and then
+run::
+
+  (env) $ python -mgevent.tests
+
+Before submitting a pull request, it's a good idea to run the tests
+across all supported versions of Python, and to check the code quality
+using prospector. This is what is done on Travis CI. Locally it
+can be done using tox::
+
+  pip install tox
+  tox
+
+The testrunner accepts a ``--coverage`` argument to enable code
+coverage metrics through the `coverage.py`_ package. That would go
+something like this::
+
+  python -m gevent.tests --coverage
+  coverage combine
+  coverage html -i
+  <open htmlcov/index.html>
+
+Continuous integration
+----------------------
+
+A test suite is run for every push and pull request submitted. Travis
+CI is used to test on Linux, and `AppVeyor`_ runs the builds on
+Windows.
+
+.. image:: https://travis-ci.org/gevent/gevent.svg?branch=master
+   :target: https://travis-ci.org/gevent/gevent
+
+.. image:: https://ci.appveyor.com/api/projects/status/q4kl21ng2yo2ixur?svg=true
+   :target: https://ci.appveyor.com/project/denik/gevent
+
+
+Builds on Travis CI automatically submit updates to `coveralls.io`_ to
+monitor test coverage.
+
+.. image:: https://coveralls.io/repos/gevent/gevent/badge.svg?branch=master&service=github
+   :target: https://coveralls.io/github/gevent/gevent?branch=master
+
+.. note:: On Debian, you will probably need ``libpythonX.Y-testsuite``
+          installed to run all the tests.
+
+
+.. _coverage.py: https://pypi.python.org/pypi/coverage/
+.. _coveralls.io: https://coveralls.io/github/gevent/gevent
+.. _AppVeyor: https://ci.appveyor.com/project/denik/gevent

docs/index.rst

@@ -4,8 +4,6 @@
 
 .. include:: _about.rst
 
-
-
 :ref:`Continue reading <installation>` »
 
 If you like gevent, :doc:`donate <sfc>` to support the development.

docs/installing_from_source.rst

+========================
+ Installing From Source
+========================
+
+If you are unable to use the binary wheels (for platforms where no
+pre-built wheels are available or if wheel installation is disabled),
+you can build gevent from source. A normal ``pip install`` will
+fallback to doing this if no binary wheel is available. See
+`Installing From Source <installing-from-source>`_ for more.
+
+
+- You can force installation of gevent from source with ``pip
+  install --no-binary gevent gevent``. This is useful if there is a
+  binary wheel available, but you want to change the compile-time
+  options, such as to use a system version of libuv instead of the
+  embedded version. See ``
+
+- You'll need a working C compiler that can build Python extensions.
+  On some platforms, you may need to install Python development
+  packages.
+
+- Installing from source requires ``setuptools``. This is installed
+  automatically in virtual environments and by buildout. However,
+  gevent uses :pep:`496` environment markers in ``setup.py``.
+  Consequently, you'll need a version of setuptools newer than 25
+  (mid 2016) to install gevent from source; a version that's too old
+  will produce a ``ValueError``. Older versions of pipenv may also
+  `have issues installing gevent for this reason
+  <https://github.com/pypa/pipenv/issues/2113>`_.
+
+- gevent comes with a ``pyproject.toml`` file that installs the build
+  dependencies, including CFFI (needed for libuv support). pip 18 or
+  above is required for this support.
+
+
+Common Installation Issues
+==========================
+
+The following are some common installation problems and solutions for
+those compiling gevent from source.
+
+- Some Linux distributions are now mounting their temporary
+  directories with the ``noexec`` option. This can cause a standard
+  ``pip install gevent`` to fail with an error like ``cannot run C
+  compiled programs``. One fix is to mount the temporary directory
+  without that option. Another may be to use the ``--build`` option to
+  ``pip install`` to specify another directory. See `issue #570
+  <https://github.com/gevent/gevent/issues/570>`_ and `issue #612
+  <https://github.com/gevent/gevent/issues/612>`_ for examples.
+
+- Also check for conflicts with environment variables like ``CFLAGS``. For
+  example, see `Library Updates <http://www.gevent.org/whatsnew_1_1.html#library-updates-label>`_.
+
+- Users of a recent SmartOS release may need to customize the
+  ``CPPFLAGS`` (the environment variable containing the default
+  options for the C preprocessor) if they are using the libev shipped
+  with gevent. See `Operating Systems
+  <http://www.gevent.org/whatsnew_1_1.html#operating-systems-label>`_
+  for more information.
+
+- If you see ``ValueError: ("Expected ',' or end-of-list in", "cffi >=
+  1.11.5 ; sys_platform == 'win32' and platform_python_implementation
+  == 'CPython'", 'at', " ; sys_platform == 'win32' and
+  platform_python_implementation == 'CPython'")``, the version of
+  setuptools is too old. Install a more recent version of setuptools.

docs/whatsnew_1_4.rst

@@ -22,3 +22,13 @@ Platform Support
 gevent 1.4 supports the platforms that gevent 1.3 supported, with the
 exception that for users of Python 3.4, Python 3.4.3 is the minimum
 supported version.
+
+Test Changes
+============
+
+gevent's own test suite is now packaged as part of the gevent install,
+and the ``greentest/testrunner.py`` script is now gone from a source
+distribution or checkout. Instead, tests can be run with ``python -m
+gevent.tests``. Many tests can be run given an installed version of
+gevent, although the test dependencies, including cffi, must be
+installed for all of them to run.

docs/whatsnew_1_5.rst

+==========================
+ What's new in gevent 1.5
+==========================
+
+.. currentmodule:: gevent
+
+.. toctree::
+   :maxdepth: 2
+
+   changelog
+
+.. caution::
+
+   This document is currently being written.
+
+Detailed information on what has changed is available in the
+:doc:`changelog`. This document summarizes the most important changes
+since :doc:`gevent 1.4 <whatsnew_1_4>`.
+
+gevent 1.5 is a small maintenance release featuring bug fixes and a
+small number of API improvements.
+
+Platform Support
+================
+
+gevent 1.5 drops support for Python 3.4, and drops support for PyPy
+< 7.
+
+Packaging Changes
+=================
+
+gevent now distributes `manylinux2010
+<https://www.python.org/dev/peps/pep-0571/>`_ binary wheels for Linux,
+instead of the older ``manylinux1`` standard. This updated
+platform tag allows gevent to distribute libuv support by default.
+CentOS 6 is the baseline for this tag.
+
+Library Updates
+===============
+
+Test Updates
+============
+
+gevent's test suite has adopted the standard library's notion of "test
+resources," allowing users to disable
+certain tests based on their resource usage. This is primarily
+intended to support downstream packagers. For example, to disable
+tests that require Internet access, one could disable the ``network``
+resource using ``python -m gevent.tests -u-network`` or
+``GEVENTTEST_USE_RESOURCES=-network python -m gevent.tests``. See
+TODO: WRITE ME for more information.

setup.py

@@ -211,12 +211,13 @@ greenlet_requires = [
 # The exception is on Windows, where we want the libuv backend we distribute
 # to be the default, and that requires cffi; but don't try to install it
 # on PyPy or it messes up the build
-cffi_requires = [
-    "cffi >= 1.12.2 ; sys_platform == 'win32' and platform_python_implementation == 'CPython'",
+CFFI_DEP = "cffi >= 1.12.2 ; platform_python_implementation == 'CPython'"
+CFFI_REQUIRES = [
+    CFFI_DEP + " and sys_platform == 'win32'"
 ]
 
 
-install_requires = greenlet_requires + cffi_requires
+install_requires = greenlet_requires + CFFI_REQUIRES
 
 # We use headers from greenlet, so it needs to be installed before we
 # can compile. If it isn't already installed before we start
@@ -230,7 +231,7 @@ install_requires = greenlet_requires + cffi_requires
 # to install the headers at all, AFAICS, we don't need to bother with
 # the buggy setup_requires.)
 
-setup_requires = cffi_requires + []
+setup_requires = CFFI_REQUIRES + []
 
 if PYPY:
     # These use greenlet/greenlet.h, which doesn't exist on PyPy
@@ -288,6 +289,29 @@ if IGNORE_CFFI and not PYPY:
     # our copy of libev/libuv and they may not want that.
     del cffi_modules[:]
 
+## Extras
+
+EXTRA_DNSPYTHON = [
+    'dnspython >= 1.16.0',
+    'idna',
+]
+EXTRA_EVENTS = [
+    'zope.event',
+    'zope.interface',
+]
+# Fails to build on PyPy on Windows.
+# TODO: Is that still the case?
+EXTRA_PSUTIL_DEP = 'psutil >= 5.6.1 ; platform_python_implementation == "CPython" or sys_platform != "win32"'
+
+EXTRA_MONITOR = [
+    EXTRA_PSUTIL_DEP,
+]
+
+EXTRA_RECOMMENDED = [
+    # We need this at runtime to use the libev-CFFI and libuv backends
+    CFFI_DEP,
+] + EXTRA_DNSPYTHON + EXTRA_EVENTS + EXTRA_MONITOR
+
 # If we are running info / help commands, or we're being imported by
 # tools like pyroma, we don't need to build anything
 _BUILDING = True
@@ -362,28 +386,20 @@ def run_setup(ext_modules, run_make):
         install_requires=install_requires,
         setup_requires=setup_requires,
         extras_require={
-            'dnspython': [
-                'dnspython >= 1.16.0',
-                'idna',
-            ],
-            'events': [
-                'zope.event',
-                'zope.interface',
-            ],
+            # Each extra intended for end users must be documented in install.rst
+            'dnspython': EXTRA_DNSPYTHON,
+            'events': EXTRA_EVENTS,
+            'monitor': EXTRA_MONITOR,
+            'recommended': EXTRA_RECOMMENDED,
+            # End end-user extras
             'docs': [
                 'repoze.sphinx.autointerface',
             ],
-            'test': [
-                # To the extent possible, we should work to make sure
-                # our tests run, at least a basic set, without any of
-                # these extra dependencies (i.e., skip things when they are
-                # missing). This helps serve as a smoketest for users.
-                'zope.interface',
-                'zope.event',
-
-                # Makes tests faster
-                # Fails to build on PyPy on Windows.
-                'psutil >= 5.6.1 ; platform_python_implementation == "CPython" or sys_platform != "win32"',
+            # To the extent possible, we should work to make sure
+            # our tests run, at least a basic set, without any of
+            # these extra dependencies (i.e., skip things when they are
+            # missing). This helps serve as a smoketest for users.
+            'test': EXTRA_RECOMMENDED + [
                 # examples, called from tests, use this
                 'requests',
 
@@ -397,7 +413,7 @@ def run_setup(ext_modules, run_make):
 
                 # leak checks. previously we had a hand-rolled version.
                 'objgraph',
-            ]
+            ],
         },
         # It's always safe to pass the CFFI keyword, even if
         # cffi is not installed: it's just ignored in that case.