Refactor the Travis build to make the matrix explicit and faster. Eliminate Makefile

This should eliminate cache thrashing when we upgrade python versions. And also eventually might do more caching here.

Also simplify the paths at which pythons are installed, so we only have to change one place to update a version.

The cache is only shared between stages that match things like environment variables.

Ah, we need CFFI at runtime; build isolation means pyproject.toml doesn't deliver it to us. We must have had it cached.

A DNSPython test we didn't run before sometimes fails. Must be multiple answers.

Tweak test coverage and test_ssl for PyPy3

Refactoring the testrunner; more needed, but this gets us simple travis folding.

Reorganize the test matrix for clarity, and shrink it back to what we used to test.

Remove now-unused entries from the makefile.

Remove the makefile, port the posix specific rules to Python for 'setup.py clean'

.gitignore

@@ -91,8 +91,3 @@ deps/libuv/.libs
 deps/libuv/*.lo
 deps/libuv/*.la
 deps/libuv/*.o
-
-
-# running setup.py on PyPy
-config.h
-configure-output.txt

.travis.yml

-# .travis.yml based on https://github.com/DRMacIver/hypothesis/blob/master/.travis.yml
-language: python
+# .travis.yml based on
+# https://github.com/DRMacIver/hypothesis/blob/master/.travis.yml
+# Setting the language to C overrides our CC environment variable
+# and disables ccache.
+# Setting it to python overrides our TRAVIS_PYTHON_VERSION (which
+# we're only using in hopes of being able to go *back* to using
+# python: one day so making the migration simpler)
+language: minimal
 dist: xenial
 group: travis_latest
-python:
-  - 3.7
 
 env:
     global:
@@ -13,24 +17,247 @@ env:
         - CCACHE_NOCPP2=true
         - CCACHE_SLOPPINESS=file_macro,time_macros,include_file_ctime,include_file_mtime
         - CCACHE_NOHASHDIR=true
+        - BUILD_LIBS=$HOME/.libs
         - CFLAGS="-g -pipe"
+        - CPPFLAGS="-I$BUILD_LIBS/include"
+        - LDFLAGS="-L$BUILD_LIBS/lib"
+        - LD_LIBRARY_PATH="$BUILD_LIBS/lib"
 
+    # Note that this list is again *manually* expanded
+    # for the 'cache' stage. The benefit of doing this (instead of
+    # only listing a single version here, or using a different 'language')
+    # is that we get separate caches per version. This keeps them small,
+    # and prevents stepping on each other when we change Python minor
+    # versions.
+
+    # We do it here, rather than with the python: keyword
+    # because the way travis handles pypy is pretty messed up.
+
+    # Recall that unless otherwise specified, jobs inherit
+    # the first entry in this list.
     matrix:
-      # These are ordered to get as much diversity in the
-      # first group of parallel runs (4) as possible
-      - TASK=test-py27-noembed
-      - TASK=test-py37
-      - TASK=test-py27
-      - TASK=test-pypy
-      - TASK=test-py36
-      - TASK=test-pypy3
-      - TASK=test-py35
+      - TRAVIS_PYTHON_VERSION=2.7
+      - TRAVIS_PYTHON_VERSION=3.5
+      - TRAVIS_PYTHON_VERSION=3.6
+      - TRAVIS_PYTHON_VERSION=3.7
+      - TRAVIS_PYTHON_VERSION=pypy2.7
+      - TRAVIS_PYTHON_VERSION=pypy3.6
+      - TRAVIS_PYTHON_VERSION=2.7 EMBED=0
 
 matrix:
   fast_finish: true
 
+
+before_install:
+  - export PATH=$BUILD_RUNTIMES/snakepit/$TRAVIS_PYTHON_VERSION.d/bin:$PATH
+
+before_script:
+  # Show some details of interest
+  - python --version
+  - >
+    python -c "import greenlet; print(greenlet, greenlet.__version__)"
+  - >
+    python -c "import gevent.core; print(gevent.core.loop)"
+  - >
+    python -c "import gevent.ares; print(gevent.ares)"
+
+# Installing is taken care of by the first stage.
+install:
+  - ls -l $BUILD_RUNTIMES/snakepit/
+  - echo $BUILD_RUNTIMES/snakepit/$TRAVIS_PYTHON_VERSION.d
+  - ls -l $BUILD_RUNTIMES/snakepit/$TRAVIS_PYTHON_VERSION.d
+  - python -c 'import gevent; print(gevent.__version__)'
 script:
-    - make $TASK
+  # The generic script for the matrix expansion is to
+  # test the defaults.
+  - &test-basic python -m gevent.tests
+
+  # Now, the non-default threaded file object.
+  # In the past, we included all test files that had a reference to 'subprocess'' somewhere in their
+  # text. The monkey-patched stdlib tests were specifically included here.
+  # However, we now always also test on AppVeyor (Windows) which only has GEVENT_FILE=thread,
+  # so we can save a lot of CI time by reducing the set and excluding the stdlib tests without
+  # losing any coverage. Which is good, because coverage fails when run this way.
+  - (cd src/gevent/tests && GEVENT_FILE=thread python -mgevent.tests test__*subprocess*.py)
+
+
+# Submit coverage info
+after_success:
+  - python -m coverage combine
+  - python -m coverage report -i
+  - python -m coveralls
+
+
+stages:
+  - build-gevent
+  - test
+  - test-ares
+
+
+jobs:
+  include:
+    - &build-gevent
+      stage: build-gevent
+      install:
+        # Install the Python runtime
+        - &build-gevent-python time ./scripts/install.sh $TRAVIS_PYTHON_VERSION
+        # Install gevent. Yes, this will create different files each time,
+        # leading to a fresh cache. But because of CCache stats, we had already been doing
+        # that (before we learned about CCACHE_NOSTATS)
+        # First, the build dependencies (see setup.cfg)
+        # so that we don't have to use build isolation and can better use the cache;
+        # Note that we can't use -U for cffi and greenlet on PyPy.
+        - &build-gevent-deps pip install -U setuptools wheel && pip install cffi cython greenlet
+        - GEVENTSETUP_EV_VERIFY=3 pip install --no-build-isolation .[test,events,dnspython]
+      script: ccache -s
+      before_script: true
+      after_success: true
+    - <<: *build-gevent
+      env: TRAVIS_PYTHON_VERSION=3.5
+    - <<: *build-gevent
+      env: TRAVIS_PYTHON_VERSION=3.6
+    - <<: *build-gevent
+      env: TRAVIS_PYTHON_VERSION=3.7
+    - <<: *build-gevent
+      env: TRAVIS_PYTHON_VERSION=pypy2.7
+    - <<: *build-gevent
+      env: TRAVIS_PYTHON_VERSION=pypy3.6
+    - <<: *build-gevent
+      env: TRAVIS_PYTHON_VERSION=2.7 EMBED=0
+      install:
+        # Install the Python runtime
+        - *build-gevent-python
+        - *build-gevent-deps
+        # Install the C dependencies to a known location. This is used
+        # to test "no embed" cases. It might seem like it would prime
+        # the CCache for when we *do* embed if we did it as part of the generic build stage,
+        # but overall it just seems to slow things down. The Travis caching and CCache is not working as
+        # well as we would hope.
+        - pushd deps/libev && ./configure --prefix=$BUILD_LIBS && make install && popd
+        - pushd deps/c-ares && ./configure --prefix=$BUILD_LIBS && make -j4 install && popd
+        - pushd deps/libuv && ./autogen.sh && ./configure --disable-static --prefix=$BUILD_LIBS && make -j4 install && popd
+        # libev builds a manpage each time, and it includes today's date, so it frequently changes.
+        # delete to avoid repacking the archive
+        - rm -rf $BUILD_LIBS/share/man/
+        - ls -l $BUILD_LIBS $BUILD_LIBS/lib
+        - EMBED=0 pip install --no-build-isolation .[test,events,dnspython]
+
+    # Ok, now we switch to the test stage. These are all in addition
+    # to the jobs created by the matrix (and should override the `script` command).
+
+    # Lint the code. Because this is a separate job, even if it fails fast
+    # the tests will still run. Put it at the top for fast feedback.
+    # We only need to do this on one version, and it should be Python 3, because
+    # pylint has stopped updating for Python 2.
+    - stage: test
+      script: python -m pylint --rcfile=.pylintrc gevent
+      env: TRAVIS_PYTHON_VERSION=3.7
+      name: lint37
+
+    # Now the various functional test groups.
+    # We organize these by interpreter, rather than functional test group,
+    # so that it's easy to see which interpreters are in which group.
+
+    # First, default (cpython 2.7). This tests essentially all the groups.
+
+    # Test the c-ares resolver. It's implemented in C, so don't bother with coverage.
+    # Also, we don't support it on PyPy for production, so don't bother to test PyPy.
+    - &test-ares-jobs
+      stage: test
+      script: GEVENT_RESOLVER=ares python -mgevent.tests --ignore tests_that_dont_use_resolver.txt
+      name: ares27
+    # Run dnspython with coverage enabled, it's implemented in python whereas ares is C.
+    # PyPy is supported.
+    - &test-dnspython-jobs
+      script: GEVENT_RESOLVER=dnspython python -mgevent.tests --coverage --ignore tests_that_dont_use_resolver.txt
+      name: dnspython27
+    # Now test the alternate backends, starting with libuv-cffi, which should be present everywhere
+    - &test-libuv-jobs
+      script: GEVENT_LOOP=libuv python -mgevent.tests --coverage
+      name: libuv27
+      # Next the libev-cffi backend, which is only needed on CPython (default on PyPy)
+    - &test-libev-jobs
+      script: GEVENT_LOOP=libev-cffi python -mgevent.tests --coverage
+      name: libev-cffi27
+    # No compiled cython modules on CPython, using the default backend. Get coverage here.
+    # We should only need to run this for a single Python 2 and a Python 3
+    - &test-pure-jobs
+      script: PURE_PYTHON=1 python -mgevent.tests --coverage
+      name: pure27
+    # Run the leaktests; this seems to be extremely slow on Python 3.7
+    # XXX: Figure out why. Can we reproduce locally?
+    - &test-leak-jobs
+      script: GEVENTTEST_LEAKCHECK=1 python -m gevent.tests --ignore tests_that_dont_do_leakchecks.txt
+      name: leak27
+
+    # Now, in order of the matrix, tests for particular versions.
+
+    # For the CPython interpreters, unless we have reason to expect
+    # different behaviour across the versions (e.g., as measured by coverage)
+    # it's sufficient to test the alternate backends for non-current versions;
+    # run the full suite on the current version.
+
+    # 3.5.
+    - <<: *test-libuv-jobs
+      env: TRAVIS_PYTHON_VERSION=3.5
+      name: libuv35
+    - <<: *test-libev-jobs
+      env: TRAVIS_PYTHON_VERSION=3.5
+      name: libev-cffi35
+
+
+    # 3.6
+    - <<: *test-libuv-jobs
+      env: TRAVIS_PYTHON_VERSION=3.6
+      name: libuv36
+    - <<: *test-libev-jobs
+      env: TRAVIS_PYTHON_VERSION=3.6
+      name: libev-cffi36
+
+    # 3.7
+    - <<: *test-ares-jobs
+      env: TRAVIS_PYTHON_VERSION=3.7
+      name: ares37
+    - <<: *test-dnspython-jobs
+      env: TRAVIS_PYTHON_VERSION=3.7
+      name: dnspython37
+    - <<: *test-libuv-jobs
+      env: TRAVIS_PYTHON_VERSION=3.7
+      name: libuv37
+    - <<: *test-libev-jobs
+      env: TRAVIS_PYTHON_VERSION=3.7
+      name: libev-cffi37
+    - <<: *test-pure-jobs
+      env: TRAVIS_PYTHON_VERSION=3.7
+      name: pure37
+
+
+    # 2.7, no-embed. Run the tests that exercise the libraries we linked to.
+    - <<: *test-ares-jobs
+      env: TRAVIS_PYTHON_VERSION=2.7 EMBED=0
+      name: ares27-noembed
+    - <<: *test-libuv-jobs
+      env: TRAVIS_PYTHON_VERSION=2.7 EMBED=0
+      name: libuv27-noembed
+    - <<: *test-libev-jobs
+      env: TRAVIS_PYTHON_VERSION=2.7 EMBED=0
+      name: libev27-noembed
+
+    # PyPy 2.7
+    - <<: *test-dnspython-jobs
+      env: TRAVIS_PYTHON_VERSION=pypy2.7
+      name: dnspythonpypy27
+    - <<: *test-libuv-jobs
+      env: TRAVIS_PYTHON_VERSION=pypy2.7
+      name: libuvpypy27
+
+    # PyPy 3.6
+    # We don't do particular extra tests on it, it's not
+    # considered a production (still alpha at this writing).
+    # - <<: *test-dnspython-jobs
+    #   env: TRAVIS_PYTHON_VERSION=pypy3.6
+    #   name: dnspythonpypy36
+
 
 notifications:
   email: false
@@ -43,6 +270,7 @@ cache:
     - $HOME/.runtimes
     - $HOME/.wheelhouse
     - $HOME/.ccache
+    - $HOME/.libs
 
 before_cache:
     - rm -f $HOME/.cache/pip/log/debug.log

CHANGES.rst

@@ -78,6 +78,12 @@
 
 - Spawning greenlets can be up to 10% faster.
 
+- Remove the ``Makefile``. Its most useful commands, ``make clean``
+  and ``make distclean``, can now be accomplished in a cross-platform
+  way using ``python setup.py clean`` and ``python setup.py clean
+  -a``, respectively. The remainder of the ``Makefile`` contained
+  Travis CI commands that have been moved to ``.travis.yml``.
+
 1.4.0 (2019-01-04)
 ==================
 

Makefile

-# This file is renamed to "Makefile.ext" in release tarballs so that setup.py won't try to
-# run it.  If you want setup.py to run "make" automatically, rename it back to "Makefile".
-
-# The pyvenv multiple runtime support is based on https://github.com/DRMacIver/hypothesis/blob/master/Makefile
-
-PYTHON?=python${TRAVIS_PYTHON_VERSION}
-CYTHON?=cython
-
-
-
-export PATH:=$(BUILD_RUNTIMES)/snakepit:$(PATH)
-export LC_ALL=C.UTF-8
-export GEVENT_RESOLVER_NAMESERVERS=8.8.8.8
-
-
-clean:
-	rm -f src/gevent/libev/corecext.c src/gevent/libev/corecext.h
-	rm -f src/gevent/resolver/cares.c src/gevent/resolver/cares.h
-	rm -f src/gevent/_semaphore.c src/gevent/_semaphore.h
-	rm -f src/gevent/local.c src/gevent/local.h
-	rm -f src/gevent/*.so src/gevent/*.pyd src/gevent/libev/*.so src/gevent/libuv/*.so src/gevent/libev/*.pyd src/gevent/libuv/*.pyd
-	rm -rf src/gevent/libev/*.o src/gevent/libuv/*.o src/gevent/*.o
-	rm -rf src/gevent/__pycache__ src/greentest/__pycache__ src/greentest/greentest/__pycache__ src/gevent/libev/__pycache__
-	rm -rf src/gevent/*.pyc src/greentest/*.pyc src/gevent/libev/*.pyc
-	rm -rf htmlcov .coverage
-	rm -rf build
-
-distclean: clean
-	rm -rf dist
-	rm -rf deps/libev/config.h deps/libev/config.log deps/libev/config.status deps/libev/.deps deps/libev/.libs
-	rm -rf deps/c-ares/config.h deps/c-ares/config.log deps/c-ares/config.status deps/c-ares/.deps deps/c-ares/.libs
-
-doc:
-	cd doc && PYTHONPATH=.. make html
-
-
-prospector:
-	which pylint
-	pylint --rcfile=.pylintrc gevent
-# debugging
-#	pylint --rcfile=.pylintrc --init-hook="import sys, code; sys.excepthook = lambda exc, exc_type, tb: print(tb.tb_next.tb_next.tb_next.tb_next.tb_next.tb_next.tb_next.tb_next.tb_next.tb_next.tb_frame.f_locals['self'])" gevent src/greentest/* || true
-# XXX: prospector is failing right now. I can't reproduce locally:
-# https://travis-ci.org/gevent/gevent/jobs/345474139
-#	${PYTHON} scripts/gprospector.py -X
-
-lint: prospector
-
-test_prelim:
-	@which ${PYTHON}
-	@${PYTHON} --version
-	@${PYTHON} -c 'import greenlet; print(greenlet, greenlet.__version__)'
-	@${PYTHON} -c 'import gevent.core; print(gevent.core.loop)'
-	@${PYTHON} -c 'import gevent.ares; print(gevent.ares)'
-	@make bench
-
-# Folding from https://github.com/travis-ci/travis-rubies/blob/9f7962a881c55d32da7c76baefc58b89e3941d91/build.sh#L38-L44
-
-basictest: test_prelim
-	@${PYTHON} scripts/travis.py fold_start basictest "Running basic tests"
-	GEVENT_RESOLVER=thread ${PYTHON} -mgevent.tests --config known_failures.py --quiet
-	@${PYTHON} scripts/travis.py fold_end basictest
-
-alltest: basictest
-	@${PYTHON} scripts/travis.py fold_start ares "Running c-ares tests"
-	GEVENT_RESOLVER=ares ${PYTHON} -mgevent.tests --config known_failures.py --ignore tests_that_dont_use_resolver.txt --quiet
-	@${PYTHON} scripts/travis.py fold_end ares
-	@${PYTHON} scripts/travis.py fold_start dnspython "Running dnspython tests"
-	GEVENT_RESOLVER=dnspython ${PYTHON} -mgevent.tests --config known_failures.py --ignore tests_that_dont_use_resolver.txt --quiet
-	@${PYTHON} scripts/travis.py fold_end dnspython
-# In the past, we included all test files that had a reference to 'subprocess'' somewhere in their
-# text. The monkey-patched stdlib tests were specifically included here.
-# However, we now always also test on AppVeyor (Windows) which only has GEVENT_FILE=thread,
-# so we can save a lot of CI time by reducing the set and excluding the stdlib tests without
-# losing any coverage.
-	@${PYTHON} scripts/travis.py fold_start thread "Running GEVENT_FILE=thread tests"
-	cd src/gevent/tests && GEVENT_FILE=thread ${PYTHON} -mgevent.tests --config known_failures.py test__*subprocess*.py --quiet
-	@${PYTHON} scripts/travis.py fold_end thread
-
-allbackendtest:
-	@${PYTHON} scripts/travis.py fold_start default "Testing default backend"
-	GEVENTTEST_COVERAGE=1 make alltest
-	@${PYTHON} scripts/travis.py fold_end default
-	GEVENTTEST_COVERAGE=1 make cffibackendtest
-# because we set parallel=true, each run produces new and different coverage files; they all need
-# to be combined
-	make coverage_combine
-
-
-cffibackendtest:
-	@${PYTHON} scripts/travis.py fold_start libuv "Testing libuv backend"
-	GEVENT_LOOP=libuv make alltest
-	@${PYTHON} scripts/travis.py fold_end libuv
-	@${PYTHON} scripts/travis.py fold_start libev "Testing libev CFFI backend"
-	GEVENT_LOOP=libev-cffi make alltest
-	@${PYTHON} scripts/travis.py fold_end libev
-
-leaktest: test_prelim
-	@${PYTHON} scripts/travis.py fold_start leaktest "Running leak tests"
-	GEVENT_RESOLVER=thread GEVENTTEST_LEAKCHECK=1 ${PYTHON} -mgevent.tests --config known_failures.py --quiet --ignore tests_that_dont_do_leakchecks.txt
-	@${PYTHON} scripts/travis.py fold_end leaktest
-	@${PYTHON} scripts/travis.py fold_start default "Testing default backend pure python"
-	PURE_PYTHON=1 GEVENTTEST_COVERAGE=1 make basictest
-	@${PYTHON} scripts/travis.py fold_end default
-
-bench:
-	time ${PYTHON} benchmarks/bench_sendall.py --loops 3 --processes 2 --values 2 --warmups 2 --quiet
-
-travis_test_linters:
-	make lint
-	make leaktest
-	make cffibackendtest
-
-coverage_combine:
-	coverage combine .
-	coverage report -i
-	-coveralls
-
-
-.PHONY: clean doc prospector lint travistest travis
-
-# Managing runtimes
-
-BUILD_RUNTIMES?=$(PWD)/.runtimes
-
-PY27=$(BUILD_RUNTIMES)/snakepit/python2.7.16
-PY35=$(BUILD_RUNTIMES)/snakepit/python3.5.6
-PY36=$(BUILD_RUNTIMES)/snakepit/python3.6.8
-PY37=$(BUILD_RUNTIMES)/snakepit/python3.7.2
-PYPY=$(BUILD_RUNTIMES)/snakepit/pypy710
-PYPY3=$(BUILD_RUNTIMES)/snakepit/pypy3.6_710
-
-
-$(PY27):
-	scripts/install.sh 2.7
-
-$(PY35):
-	scripts/install.sh 3.5
-
-$(PY36):
-	scripts/install.sh 3.6
-
-$(PY37):
-	scripts/install.sh 3.7
-
-$(PYPY):
-	scripts/install.sh pypy
-
-$(PYPY3):
-	scripts/install.sh pypy3
-
-
-develop:
-	@${PYTHON} scripts/travis.py fold_start install "Installing gevent"
-	@echo python is at `which $(PYTHON)`
-# First install a newer pip so that it can use the wheel cache
-# (only needed until travis upgrades pip to 7.x; note that the 3.5
-# environment uses pip 7.1 by default)
-	${PYTHON} -m pip install -U pip setuptools
-# Then start installing our deps so they can be cached. Note that use of --build-options / --global-options / --install-options
-# disables the cache.
-# We need wheel>=0.26 on Python 3.5. See previous revisions.
-	GEVENTSETUP_EV_VERIFY=3 time ${PYTHON} -m pip install -U --upgrade-strategy=eager -r dev-requirements.txt
-	${PYTHON} -m pip freeze
-	ccache -s
-	@${PYTHON} scripts/travis.py fold_end install
-
-test-py27: $(PY27)
-	PYTHON=python2.7.16 PATH=$(BUILD_RUNTIMES)/versions/python2.7.16/bin:$(PATH) make develop leaktest cffibackendtest coverage_combine
-
-test-py35: $(PY35)
-	PYTHON=python3.5.6 PATH=$(BUILD_RUNTIMES)/versions/python3.5.6/bin:$(PATH) make develop basictest
-
-test-py36: $(PY36)
-	PYTHON=python3.6.8 PATH=$(BUILD_RUNTIMES)/versions/python3.6.8/bin:$(PATH) make develop lint basictest
-
-test-py37: $(PY37)
-	PYTHON=python3.7.2 PATH=$(BUILD_RUNTIMES)/versions/python3.7.2/bin:$(PATH) make develop leaktest cffibackendtest coverage_combine
-
-test-pypy: $(PYPY)
-	PYTHON=$(PYPY) PATH=$(BUILD_RUNTIMES)/versions/pypy710/bin:$(PATH) make develop cffibackendtest
-
-test-pypy3: $(PYPY3)
-	PYTHON=$(PYPY3) PATH=$(BUILD_RUNTIMES)/versions/pypy3.6_710/bin:$(PATH) make develop basictest
-
-test-py27-noembed: $(PY27)
-	@python2.7.16 scripts/travis.py fold_start conf_libev "Configuring libev"
-	cd deps/libev && ./configure --disable-dependency-tracking && make
-	@python2.7.16 scripts/travis.py fold_end conf_libev
-	@python2.7.16 scripts/travis.py fold_start conf_cares "Configuring cares"
-	cd deps/c-ares && ./configure --disable-dependency-tracking && make
-	@python2.7.16 scripts/travis.py fold_end conf_cares
-	@python2.7.16 scripts/travis.py fold_start conf_libuv "Configuring libuv"
-	cd deps/libuv && ./autogen.sh && ./configure --disable-static && make
-	@python2.7.16 scripts/travis.py fold_end conf_libuv
-	CPPFLAGS="-Ideps/libev -Ideps/c-ares -Ideps/libuv/include" LDFLAGS="-Ldeps/libev/.libs -Ldeps/c-ares/.libs -Ldeps/libuv/.libs" LD_LIBRARY_PATH="$(PWD)/deps/libev/.libs:$(PWD)/deps/c-ares/.libs:$(PWD)/deps/libuv/.libs" EMBED=0 PYTHON=python2.7.16 PATH=$(BUILD_RUNTIMES)/versions/python2.7.16/bin:$(PATH) make develop alltest cffibackendtest

_setuputils.py

@@ -221,15 +221,11 @@ class ConfiguringBuildExt(build_ext):
     def build_extension(self, ext):
         self.gevent_prepare(ext)
         try:
-            result = build_ext.build_extension(self, ext)
+            return build_ext.build_extension(self, ext)
         except ext_errors:
             if getattr(ext, 'optional', False):
                 raise BuildFailed()
-            else:
-                raise
-        return result
-
-
+            raise
 
 
 class Extension(_Extension):
@@ -242,3 +238,137 @@ class Extension(_Extension):
         # Python 2 has this as an old-style class for some reason
         # so super() doesn't work.
         _Extension.__init__(self, *args, **kwargs) # pylint:disable=no-member,non-parent-init-called
+
+
+from distutils.command.clean import clean # pylint:disable=no-name-in-module,import-error
+from distutils import log # pylint:disable=no-name-in-module
+from distutils.dir_util import remove_tree # pylint:disable=no-name-in-module,import-error
+
+class GeventClean(clean):
+
+    BASE_GEVENT_SRC = os.path.join('src', 'gevent')
+
+    def __find_directories_in(self, top, named=None):
+        """
+        Iterate directories, beneath and including *top* ignoring '.'
+        entries.
+        """
+        for dirpath, dirnames, _ in os.walk(top):
+            # Modify dirnames in place to prevent walk from
+            # recursing into hidden directories.
+            dirnames[:] = [x for x in dirnames if not x.startswith('.')]
+            for dirname in dirnames:
+                if named is None or named == dirname:
+                    yield os.path.join(dirpath, dirname)
+
+    def __glob_under(self, base, file_pat):
+        return glob_many(
+            os.path.join(base, file_pat),
+            *(os.path.join(x, file_pat)
+              for x in
+              self.__find_directories_in(base)))
+
+    def __remove_dirs(self, remove_file):
+
+        dirs_to_remove = [
+            'htmlcov',
+            '.eggs',
+        ]
+        if self.all:
+            dirs_to_remove += [
+                # tox
+                '.tox',
+                # instal.sh for pyenv
+                '.runtimes',
+                # Built wheels from manylinux
+                'wheelhouse',
+                # Doc build
+                os.path.join('.', 'doc', '_build'),
+            ]
+        dir_finders = [
+            # All python cache dirs
+            (self.__find_directories_in, '.', '__pycache__'),
+        ]
+
+        for finder in dir_finders:
+            func = finder[0]
+            args = finder[1:]
+            dirs_to_remove.extend(func(*args))
+
+        for f in sorted(dirs_to_remove):
+            remove_file(f)
+
+    def run(self):
+        clean.run(self)
+
+        if self.dry_run:
+            def remove_file(f):
+                if os.path.isdir(f):
+                    remove_tree(f, dry_run=self.dry_run)
+                elif os.path.exists(f):
+                    log.info("Would remove '%s'", f)
+        else:
+            def remove_file(f):
+                if os.path.isdir(f):
+                    remove_tree(f, dry_run=self.dry_run)
+                elif os.path.exists(f):
+                    log.info("Removing '%s'", f)
+                    os.remove(f)
+
+        # Remove directories first before searching for individual files
+        self.__remove_dirs(remove_file)
+
+        def glob_gevent(file_path):
+            return glob(os.path.join(self.BASE_GEVENT_SRC, file_path))
+
+        def glob_gevent_and_under(file_pat):
+            return self.__glob_under(self.BASE_GEVENT_SRC, file_pat)
+
+        def glob_root_and_under(file_pat):
+            return self.__glob_under('.', file_pat)
+
+        files_to_remove = [
+            '.coverage',
+            # One-off cython-generated code that doesn't
+            # follow a globbale-pattern
+            os.path.join(self.BASE_GEVENT_SRC, 'libev', 'corecext.c'),
+            os.path.join(self.BASE_GEVENT_SRC, 'libev', 'corecext.h'),
+            os.path.join(self.BASE_GEVENT_SRC, 'resolver', 'cares.c'),
+            os.path.join(self.BASE_GEVENT_SRC, 'resolver', 'cares.c'),
+        ]
+
+        def dep_configure_artifacts(dep):
+            for f in (
+                    'config.h',
+                    'config.log',
+                    'config.status',
+                    '.libs'
+            ):
+                yield os.path.join('deps', dep, f)
+
+        file_finders = [
+            # The base gevent directory contains
+            # only generated .c code. Remove it.
+            (glob_gevent, "*.c"),
+            # Any .html files found in the gevent directory
+            # are the result of Cython annotations. Remove them.
+            (glob_gevent_and_under, "*.html"),
+            # Any compiled binaries have to go
+            (glob_gevent_and_under, "*.so"),
+            (glob_gevent_and_under, "*.pyd"),
+            (glob_root_and_under, "*.o"),
+            # Compiled python files too
+            (glob_gevent_and_under, "*.pyc"),
+            (glob_gevent_and_under, "*.pyo"),
+
+            # Artifacts of building dependencies in place
+            (dep_configure_artifacts, 'libev'),
+            (dep_configure_artifacts, 'libuv'),
+            (dep_configure_artifacts, 'c-ares'),
+        ]
+
+        for func, pat in file_finders:
+            files_to_remove.extend(func(pat))
+
+        for f in sorted(files_to_remove):
+            remove_file(f)

deps/README.rst

@@ -2,10 +2,6 @@
  Managing Embedded Dependencies
 ================================
 
-- Modify the c-ares Makefile.in[c] to empty out the MANPAGES variables
-  so that we don't have to ship those in the sdist.
-
-  XXX: We need a patch for that.
 
 Updating libev
 ==============
@@ -30,6 +26,17 @@ Check if 'config.guess' and/or 'config.sub' went backwards in time
 from the latest source
 http://git.savannah.gnu.org/gitweb/?p=config.git;a=tree )
 
+Updating c-ares
+===============
+
+- Modify the c-ares Makefile.in[c] to empty out the MANPAGES variables
+  so that we don't have to ship those in the sdist.
+
+  XXX: We need a patch for that.
+
+- Follow the same 'config.guess' and 'config.sub' steps as libev.
+
+
 Updating libuv
 ==============
 
@@ -57,3 +64,5 @@ and the build process. Evaluate those and add them to git and to
 ``src/gevent/libuv/_corecffi_build.py`` as needed. Then check if there
 are changes to the build system (e.g., the .gyp files) that need to be
 accounted for in our build file.
+
+- Follow the same 'config.guess' and 'config.sub' steps as libev.

deps/c-ares/config.guess

 #! /bin/sh
 # Attempt to guess a canonical system name.
-#   Copyright 1992-2018 Free Software Foundation, Inc.
+#   Copyright 1992-2019 Free Software Foundation, Inc.
 
-timestamp='2018-03-08'
+timestamp='2019-03-04'
 
 # This file is free software; you can redistribute it and/or modify it
 # under the terms of the GNU General Public License as published by
@@ -50,7 +50,7 @@ version="\
 GNU config.guess ($timestamp)
 
 Originally written by Per Bothner.
-Copyright 1992-2018 Free Software Foundation, Inc.
+Copyright 1992-2019 Free Software Foundation, Inc.
 
 This is free software; see the source for copying conditions.  There is NO
 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE."
@@ -84,8 +84,6 @@ if test $# != 0; then
   exit 1
 fi
 
-trap 'exit 1' 1 2 15
-
 # CC_FOR_BUILD -- compiler used by this script. Note that the use of a
 # compiler to aid in system detection is discouraged as it requires
 # temporary files to be created and, as you can see below, it is a
@@ -96,34 +94,38 @@ trap 'exit 1' 1 2 15
 
 # Portable tmp directory creation inspired by the Autoconf team.
 
-set_cc_for_build='
-trap "exitcode=\$?; (rm -f \$tmpfiles 2>/dev/null; rmdir \$tmp 2>/dev/null) && exit \$exitcode" 0 ;
-trap "rm -f \$tmpfiles 2>/dev/null; rmdir \$tmp 2>/dev/null; exit 1" 1 2 13 15 ;
-: ${TMPDIR=/tmp} ;
- { tmp=`(umask 077 && mktemp -d "$TMPDIR/cgXXXXXX") 2>/dev/null` && test -n "$tmp" && test -d "$tmp" ; } ||
- { test -n "$RANDOM" && tmp=$TMPDIR/cg$$-$RANDOM && (umask 077 && mkdir $tmp) ; } ||
- { tmp=$TMPDIR/cg-$$ && (umask 077 && mkdir $tmp) && echo "Warning: creating insecure temp directory" >&2 ; } ||
- { echo "$me: cannot create a temporary directory in $TMPDIR" >&2 ; exit 1 ; } ;
-dummy=$tmp/dummy ;
-tmpfiles="$dummy.c $dummy.o $dummy.rel $dummy" ;
-case $CC_FOR_BUILD,$HOST_CC,$CC in
- ,,)    echo "int x;" > "$dummy.c" ;
-	for c in cc gcc c89 c99 ; do
-	  if ($c -c -o "$dummy.o" "$dummy.c") >/dev/null 2>&1 ; then
-	     CC_FOR_BUILD="$c"; break ;
-	  fi ;
-	done ;
-	if test x"$CC_FOR_BUILD" = x ; then
-	  CC_FOR_BUILD=no_compiler_found ;
-	fi
-	;;
- ,,*)   CC_FOR_BUILD=$CC ;;
- ,*,*)  CC_FOR_BUILD=$HOST_CC ;;
-esac ; set_cc_for_build= ;'
+tmp=
+# shellcheck disable=SC2172
+trap 'test -z "$tmp" || rm -fr "$tmp"' 0 1 2 13 15
+
+set_cc_for_build() {
+    : "${TMPDIR=/tmp}"
+    # shellcheck disable=SC2039
+    { tmp=`(umask 077 && mktemp -d "$TMPDIR/cgXXXXXX") 2>/dev/null` && test -n "$tmp" && test -d "$tmp" ; } ||
+	{ test -n "$RANDOM" && tmp=$TMPDIR/cg$$-$RANDOM && (umask 077 && mkdir "$tmp" 2>/dev/null) ; } ||
+	{ tmp=$TMPDIR/cg-$$ && (umask 077 && mkdir "$tmp" 2>/dev/null) && echo "Warning: creating insecure temp directory" >&2 ; } ||
+	{ echo "$me: cannot create a temporary directory in $TMPDIR" >&2 ; exit 1 ; }
+    dummy=$tmp/dummy
+    case ${CC_FOR_BUILD-},${HOST_CC-},${CC-} in
+	,,)    echo "int x;" > "$dummy.c"
+	       for driver in cc gcc c89 c99 ; do
+		   if ($driver -c -o "$dummy.o" "$dummy.c") >/dev/null 2>&1 ; then
+		       CC_FOR_BUILD="$driver"
+		       break
+		   fi
+	       done
+	       if test x"$CC_FOR_BUILD" = x ; then
+		   CC_FOR_BUILD=no_compiler_found
+	       fi
+	       ;;
+	,,*)   CC_FOR_BUILD=$CC ;;
+	,*,*)  CC_FOR_BUILD=$HOST_CC ;;
+    esac
+}
 
 # This is needed to find uname on a Pyramid OSx when run in the BSD universe.
 # (ghazi@noc.rutgers.edu 1994-08-24)
-if (test -f /.attbin/uname) >/dev/null 2>&1 ; then
+if test -f /.attbin/uname ; then
 	PATH=$PATH:/.attbin ; export PATH
 fi
 
@@ -138,7 +140,7 @@ Linux|GNU|GNU/*)
 	# We could probably try harder.
 	LIBC=gnu
 
-	eval "$set_cc_for_build"
+	set_cc_for_build
 	cat <<-EOF > "$dummy.c"
 	#include <features.h>
 	#if defined(__UCLIBC__)
@@ -199,7 +201,7 @@ case "$UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION" in
 		os=netbsdelf
 		;;
 	    arm*|i386|m68k|ns32k|sh3*|sparc|vax)
-		eval "$set_cc_for_build"
+		set_cc_for_build
 		if echo __ELF__ | $CC_FOR_BUILD -E - 2>/dev/null \
 			| grep -q __ELF__
 		then
@@ -237,7 +239,7 @@ case "$UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION" in
 	# Since CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM:
 	# contains redundant information, the shorter form:
 	# CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM is used.
-	echo "$machine-${os}${release}${abi}"
+	echo "$machine-${os}${release}${abi-}"
 	exit ;;
     *:Bitrig:*:*)
 	UNAME_MACHINE_ARCH=`arch | sed 's/Bitrig.//'`
@@ -389,7 +391,7 @@ case "$UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION" in
 	echo i386-pc-auroraux"$UNAME_RELEASE"
 	exit ;;
     i86pc:SunOS:5.*:* | i86xen:SunOS:5.*:*)
-	eval "$set_cc_for_build"
+	set_cc_for_build
 	SUN_ARCH=i386
 	# If there is a compiler, see if it is configured for 64-bit objects.
 	# Note that the Sun cc does not turn __LP64__ into 1 like gcc does.
@@ -482,7 +484,7 @@ case "$UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION" in
 	echo clipper-intergraph-clix"$UNAME_RELEASE"
 	exit ;;
     mips:*:*:UMIPS | mips:*:*:RISCos)
-	eval "$set_cc_for_build"
+	set_cc_for_build
 	sed 's/^	//' << EOF > "$dummy.c"
 #ifdef __cplusplus
 #include <stdio.h>  /* for printf() prototype */
@@ -579,7 +581,7 @@ EOF
 	exit ;;
     *:AIX:2:3)
 	if grep bos325 /usr/include/stdio.h >/dev/null 2>&1; then
-		eval "$set_cc_for_build"
+		set_cc_for_build
 		sed 's/^		//' << EOF > "$dummy.c"
 		#include <sys/systemcfg.h>
 
@@ -660,7 +662,7 @@ EOF
 		    esac
 		fi
 		if [ "$HP_ARCH" = "" ]; then
-		    eval "$set_cc_for_build"
+		    set_cc_for_build
 		    sed 's/^		//' << EOF > "$dummy.c"
 
 		#define _HPUX_SOURCE
@@ -700,7 +702,7 @@ EOF
 	esac
 	if [ "$HP_ARCH" = hppa2.0w ]
 	then
-	    eval "$set_cc_for_build"
+	    set_cc_for_build
 
 	    # hppa2.0w-hp-hpux* has a 64-bit kernel and a compiler generating
 	    # 32-bit code.  hppa64-hp-hpux* has the same kernel and a compiler
@@ -726,7 +728,7 @@ EOF
 	echo ia64-hp-hpux"$HPUX_REV"
 	exit ;;
     3050*:HI-UX:*:*)
-	eval "$set_cc_for_build"
+	set_cc_for_build
 	sed 's/^	//' << EOF > "$dummy.c"
 	#include <unistd.h>
 	int
@@ -840,6 +842,17 @@ EOF
     *:BSD/OS:*:*)
 	echo "$UNAME_MACHINE"-unknown-bsdi"$UNAME_RELEASE"
 	exit ;;
+    arm:FreeBSD:*:*)
+	UNAME_PROCESSOR=`uname -p`
+	set_cc_for_build
+	if echo __ARM_PCS_VFP | $CC_FOR_BUILD -E - 2>/dev/null \
+	    | grep -q __ARM_PCS_VFP
+	then
+	    echo "${UNAME_PROCESSOR}"-unknown-freebsd"`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`"-gnueabi
+	else
+	    echo "${UNAME_PROCESSOR}"-unknown-freebsd"`echo ${UNAME_RELEASE}|sed -e 's/[-(].*//'`"-gnueabihf
+	fi
+	exit ;;
     *:FreeBSD:*:*)
 	UNAME_PROCESSOR=`/usr/bin/uname -p`
 	case "$UNAME_PROCESSOR" in
@@ -881,7 +894,7 @@ EOF
 	echo "$UNAME_MACHINE"-pc-uwin
 	exit ;;
     amd64:CYGWIN*:*:* | x86_64:CYGWIN*:*:*)
-	echo x86_64-unknown-cygwin
+	echo x86_64-pc-cygwin
 	exit ;;
     prep*:SunOS:5.*:*)
 	echo powerpcle-unknown-solaris2"`echo "$UNAME_RELEASE"|sed -e 's/[^.]*//'`"
@@ -894,8 +907,8 @@ EOF
 	# other systems with GNU libc and userland
 	echo "$UNAME_MACHINE-unknown-`echo "$UNAME_SYSTEM" | sed 's,^[^/]*/,,' | tr "[:upper:]" "[:lower:]"``echo "$UNAME_RELEASE"|sed -e 's/[-(].*//'`-$LIBC"
 	exit ;;
-    i*86:Minix:*:*)
-	echo "$UNAME_MACHINE"-pc-minix
+    *:Minix:*:*)
+	echo "$UNAME_MACHINE"-unknown-minix
 	exit ;;
     aarch64:Linux:*:*)
 	echo "$UNAME_MACHINE"-unknown-linux-"$LIBC"
@@ -922,7 +935,7 @@ EOF
 	echo "$UNAME_MACHINE"-unknown-linux-"$LIBC"
 	exit ;;
     arm*:Linux:*:*)
-	eval "$set_cc_for_build"
+	set_cc_for_build
 	if echo __ARM_EABI__ | $CC_FOR_BUILD -E - 2>/dev/null \
 	    | grep -q __ARM_EABI__
 	then
@@ -971,23 +984,51 @@ EOF
 	echo "$UNAME_MACHINE"-unknown-linux-"$LIBC"
 	exit ;;
     mips:Linux:*:* | mips64:Linux:*:*)
-	eval "$set_cc_for_build"
+	set_cc_for_build
+	IS_GLIBC=0
+	test x"${LIBC}" = xgnu && IS_GLIBC=1
 	sed 's/^	//' << EOF > "$dummy.c"
 	#undef CPU
-	#undef ${UNAME_MACHINE}
-	#undef ${UNAME_MACHINE}el
+	#undef mips
+	#undef mipsel
+	#undef mips64
+	#undef mips64el
+	#if ${IS_GLIBC} && defined(_ABI64)
+	LIBCABI=gnuabi64
+	#else
+	#if ${IS_GLIBC} && defined(_ABIN32)
+	LIBCABI=gnuabin32
+	#else
+	LIBCABI=${LIBC}
+	#endif
+	#endif
+
+	#if ${IS_GLIBC} && defined(__mips64) && defined(__mips_isa_rev) && __mips_isa_rev>=6
+	CPU=mipsisa64r6
+	#else
+	#if ${IS_GLIBC} && !defined(__mips64) && defined(__mips_isa_rev) && __mips_isa_rev>=6
+	CPU=mipsisa32r6
+	#else
+	#if defined(__mips64)
+	CPU=mips64
+	#else
+	CPU=mips
+	#endif
+	#endif
+	#endif
+
 	#if defined(__MIPSEL__) || defined(__MIPSEL) || defined(_MIPSEL) || defined(MIPSEL)
-	CPU=${UNAME_MACHINE}el
+	MIPS_ENDIAN=el
 	#else
 	#if defined(__MIPSEB__) || defined(__MIPSEB) || defined(_MIPSEB) || defined(MIPSEB)
-	CPU=${UNAME_MACHINE}
+	MIPS_ENDIAN=
 	#else
-	CPU=
+	MIPS_ENDIAN=
 	#endif
 	#endif
 EOF
-	eval "`$CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^CPU'`"
-	test "x$CPU" != x && { echo "$CPU-unknown-linux-$LIBC"; exit; }
+	eval "`$CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^CPU\|^MIPS_ENDIAN\|^LIBCABI'`"
+	test "x$CPU" != x && { echo "$CPU${MIPS_ENDIAN}-unknown-linux-$LIBCABI"; exit; }
 	;;
     mips64el:Linux:*:*)
 	echo "$UNAME_MACHINE"-unknown-linux-"$LIBC"
@@ -1100,7 +1141,7 @@ EOF
 	    *Pentium)	     UNAME_MACHINE=i586 ;;
 	    *Pent*|*Celeron) UNAME_MACHINE=i686 ;;
 	esac
-	echo "$UNAME_MACHINE-unknown-sysv${UNAME_RELEASE}${UNAME_SYSTEM}{$UNAME_VERSION}"
+	echo "$UNAME_MACHINE-unknown-sysv${UNAME_RELEASE}${UNAME_SYSTEM}${UNAME_VERSION}"
 	exit ;;
     i*86:*:3.2:*)
 	if test -f /usr/options/cb.name; then
@@ -1285,7 +1326,7 @@ EOF
 	exit ;;
     *:Darwin:*:*)
 	UNAME_PROCESSOR=`uname -p` || UNAME_PROCESSOR=unknown
-	eval "$set_cc_for_build"
+	set_cc_for_build
 	if test "$UNAME_PROCESSOR" = unknown ; then
 	    UNAME_PROCESSOR=powerpc
 	fi
@@ -1358,6 +1399,7 @@ EOF
 	# "uname -m" is not consistent, so use $cputype instead. 386
 	# is converted to i386 for consistency with other x86
 	# operating systems.
+	# shellcheck disable=SC2154
 	if test "$cputype" = 386; then
 	    UNAME_MACHINE=i386
 	else
@@ -1414,8 +1456,135 @@ EOF
     amd64:Isilon\ OneFS:*:*)
 	echo x86_64-unknown-onefs
 	exit ;;
+    *:Unleashed:*:*)
+	echo "$UNAME_MACHINE"-unknown-unleashed"$UNAME_RELEASE"
+	exit ;;
 esac
 
+# No uname command or uname output not recognized.
+set_cc_for_build
+cat > "$dummy.c" <<EOF
+#ifdef _SEQUENT_
+#include <sys/types.h>
+#include <sys/utsname.h>
+#endif
+main ()
+{
+#if defined (sony)
+#if defined (MIPSEB)
+  /* BFD wants "bsd" instead of "newsos".  Perhaps BFD should be changed,
+     I don't know....  */
+  printf ("mips-sony-bsd\n"); exit (0);
+#else
+#include <sys/param.h>
+  printf ("m68k-sony-newsos%s\n",
+#ifdef NEWSOS4
+  "4"
+#else
+  ""
+#endif
+  ); exit (0);
+#endif
+#endif
+
+#if defined (NeXT)
+#if !defined (__ARCHITECTURE__)
+#define __ARCHITECTURE__ "m68k"
+#endif
+  int version;
+  version=`(hostinfo | sed -n 's/.*NeXT Mach \([0-9]*\).*/\1/p') 2>/dev/null`;
+  if (version < 4)
+    printf ("%s-next-nextstep%d\n", __ARCHITECTURE__, version);
+  else
+    printf ("%s-next-openstep%d\n", __ARCHITECTURE__, version);
+  exit (0);
+#endif
+
+#if defined (MULTIMAX) || defined (n16)
+#if defined (UMAXV)
+  printf ("ns32k-encore-sysv\n"); exit (0);
+#else
+#if defined (CMU)
+  printf ("ns32k-encore-mach\n"); exit (0);
+#else
+  printf ("ns32k-encore-bsd\n"); exit (0);
+#endif
+#endif
+#endif
+
+#if defined (__386BSD__)
+  printf ("i386-pc-bsd\n"); exit (0);
+#endif
+
+#if defined (sequent)
+#if defined (i386)
+  printf ("i386-sequent-dynix\n"); exit (0);
+#endif
+#if defined (ns32000)
+  printf ("ns32k-sequent-dynix\n"); exit (0);
+#endif
+#endif
+
+#if defined (_SEQUENT_)
+  struct utsname un;
+
+  uname(&un);
+  if (strncmp(un.version, "V2", 2) == 0) {
+    printf ("i386-sequent-ptx2\n"); exit (0);
+  }
+  if (strncmp(un.version, "V1", 2) == 0) { /* XXX is V1 correct? */
+    printf ("i386-sequent-ptx1\n"); exit (0);
+  }
+  printf ("i386-sequent-ptx\n"); exit (0);
+#endif
+
+#if defined (vax)
+#if !defined (ultrix)
+#include <sys/param.h>
+#if defined (BSD)
+#if BSD == 43
+  printf ("vax-dec-bsd4.3\n"); exit (0);
+#else
+#if BSD == 199006
+  printf ("vax-dec-bsd4.3reno\n"); exit (0);
+#else
+  printf ("vax-dec-bsd\n"); exit (0);
+#endif
+#endif
+#else
+  printf ("vax-dec-bsd\n"); exit (0);
+#endif
+#else
+  printf ("vax-dec-ultrix\n"); exit (0);
+#endif
+#endif
+#if defined(ultrix) || defined(_ultrix) || defined(__ultrix) || defined(__ultrix__)
+#if defined(mips) || defined(__mips) || defined(__mips__) || defined(MIPS) || defined(__MIPS__)
+#include <signal.h>
+#if defined(_SIZE_T_) /* >= ULTRIX4 */
+  printf ("mips-dec-ultrix4\n"); exit (0);
+#else
+#if defined(ULTRIX3) || defined(ultrix3) || defined(SIGLOST)
+  printf ("mips-dec-ultrix3\n"); exit (0);
+#endif
+#endif
+#endif
+#endif
+
+#if defined (alliant) && defined (i860)
+  printf ("i860-alliant-bsd\n"); exit (0);
+#endif
+
+  exit (1);
+}
+EOF
+
+$CC_FOR_BUILD -o "$dummy" "$dummy.c" 2>/dev/null && SYSTEM_NAME=`$dummy` &&
+	{ echo "$SYSTEM_NAME"; exit; }
+
+# Apollos put the system type in the environment.
+test -d /usr/apollo && { echo "$ISP-apollo-$SYSTYPE"; exit; }
+
 echo "$0: unable to guess system type" >&2
 
 case "$UNAME_MACHINE:$UNAME_SYSTEM" in

deps/c-ares/config.sub

 #! /bin/sh
 # Configuration validation subroutine script.
-#   Copyright 1992-2018 Free Software Foundation, Inc.
+#   Copyright 1992-2019 Free Software Foundation, Inc.
 
-timestamp='2018-05-03'
+timestamp='2019-01-05'
 
 # This file is free software; you can redistribute it and/or modify it
 # under the terms of the GNU General Public License as published by
@@ -67,7 +67,7 @@ Report bugs and patches to <config-patches@gnu.org>."
 version="\
 GNU config.sub ($timestamp)
 
-Copyright 1992-2018 Free Software Foundation, Inc.
+Copyright 1992-2019 Free Software Foundation, Inc.
 
 This is free software; see the source for copying conditions.  There is NO
 warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE."
@@ -89,7 +89,7 @@ while test $# -gt 0 ; do
     - )	# Use stdin as input.
        break ;;
     -* )
-       echo "$me: invalid option $1$help"
+       echo "$me: invalid option $1$help" >&2
        exit 1 ;;
 
     *local*)
@@ -110,1234 +110,1167 @@ case $# in
     exit 1;;
 esac
 
-# Separate what the user gave into CPU-COMPANY and OS or KERNEL-OS (if any).
-# Here we must recognize all the valid KERNEL-OS combinations.
-maybe_os=`echo "$1" | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\2/'`
-case $maybe_os in
-	nto-qnx* | linux-gnu* | linux-android* | linux-dietlibc | linux-newlib* | \
-	linux-musl* | linux-uclibc* | uclinux-uclibc* | uclinux-gnu* | kfreebsd*-gnu* | \
-	knetbsd*-gnu* | netbsd*-gnu* | netbsd*-eabi* | \
-	kopensolaris*-gnu* | cloudabi*-eabi* | \
-	storm-chaos* | os2-emx* | rtmk-nova*)
-	os=-$maybe_os
-	basic_machine=`echo "$1" | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\1/'`
-	;;
-    android-linux)
-	os=-linux-android
-	basic_machine=`echo "$1" | sed 's/^\(.*\)-\([^-]*-[^-]*\)$/\1/'`-unknown
-	;;
-    *)
-	basic_machine=`echo "$1" | sed 's/-[^-]*$//'`
-	case $1 in
-	    *-*)
-		os=`echo "$1" | sed 's/.*-/-/'`
-		;;
-	    *)
-		os=
-		;;
-	esac
-	;;
-esac
+# Split fields of configuration type
+# shellcheck disable=SC2162
+IFS="-" read field1 field2 field3 field4 <<EOF
+$1
+EOF
 
-### Let's recognize common machines as not being operating systems so
-### that things like config.sub decstation-3100 work.  We also
-### recognize some manufacturers as not being operating systems, so we
-### can provide default operating systems below.
-case $os in
-	-sun*os*)
-		# Prevent following clause from handling this invalid input.
-		;;
-	-dec* | -mips* | -sequent* | -encore* | -pc532* | -sgi* | -sony* | \
-	-att* | -7300* | -3300* | -delta* | -motorola* | -sun[234]* | \
-	-unicom* | -ibm* | -next | -hp | -isi* | -apollo | -altos* | \
-	-convergent* | -ncr* | -news | -32* | -3600* | -3100* | -hitachi* |\
-	-c[123]* | -convex* | -sun | -crds | -omron* | -dg | -ultra | -tti* | \
-	-harris | -dolphin | -highlevel | -gould | -cbm | -ns | -masscomp | \
-	-apple | -axis | -knuth | -cray | -microblaze*)
-		os=
-		basic_machine=$1
-		;;
-	-bluegene*)
-		os=-cnk
-		;;
-	-sim | -cisco | -oki | -wec | -winbond)
-		os=
-		basic_machine=$1
-		;;
-	-scout)
-		;;
-	-wrs)
-		os=-vxworks
-		basic_machine=$1
-		;;
-	-chorusos*)
-		os=-chorusos
-		basic_machine=$1
-		;;
-	-chorusrdb)
-		os=-chorusrdb
-		basic_machine=$1
-		;;
-	-hiux*)
-		os=-hiuxwe2
-		;;
-	-sco6)
-		os=-sco5v6
-		basic_machine=`echo "$1" | sed -e 's/86-.*/86-pc/'`
-		;;
-	-sco5)
-		os=-sco3.2v5
-		basic_machine=`echo "$1" | sed -e 's/86-.*/86-pc/'`
-		;;
-	-sco4)
-		os=-sco3.2v4
-		basic_machine=`echo "$1" | sed -e 's/86-.*/86-pc/'`
-		;;
-	-sco3.2.[4-9]*)
-		os=`echo $os | sed -e 's/sco3.2./sco3.2v/'`
-		basic_machine=`echo "$1" | sed -e 's/86-.*/86-pc/'`
-		;;
-	-sco3.2v[4-9]*)
-		# Don't forget version if it is 3.2v4 or newer.
-		basic_machine=`echo "$1" | sed -e 's/86-.*/86-pc/'`
-		;;
-	-sco5v6*)
-		# Don't forget version if it is 3.2v4 or newer.
-		basic_machine=`echo "$1" | sed -e 's/86-.*/86-pc/'`
-		;;
-	-sco*)
-		os=-sco3.2v2
-		basic_machine=`echo "$1" | sed -e 's/86-.*/86-pc/'`
-		;;
-	-udk*)
-		basic_machine=`echo "$1" | sed -e 's/86-.*/86-pc/'`
-		;;
-	-isc)
-		os=-isc2.2
-		basic_machine=`echo "$1" | sed -e 's/86-.*/86-pc/'`
-		;;
-	-clix*)
-		basic_machine=clipper-intergraph
-		;;
-	-isc*)
-		basic_machine=`echo "$1" | sed -e 's/86-.*/86-pc/'`
-		;;
-	-lynx*178)
-		os=-lynxos178
-		;;
-	-lynx*5)
-		os=-lynxos5
+# Separate into logical components for further validation
+case $1 in
+	*-*-*-*-*)
+		echo Invalid configuration \`"$1"\': more than four components >&2
+		exit 1
 		;;
-	-lynx*)
-		os=-lynxos
+	*-*-*-*)
+		basic_machine=$field1-$field2
+		os=$field3-$field4
 		;;
-	-ptx*)
-		basic_machine=`echo "$1" | sed -e 's/86-.*/86-sequent/'`
+	*-*-*)
+		# Ambiguous whether COMPANY is present, or skipped and KERNEL-OS is two
+		# parts
+		maybe_os=$field2-$field3
+		case $maybe_os in
+			nto-qnx* | linux-gnu* | linux-android* | linux-dietlibc \
+			| linux-newlib* | linux-musl* | linux-uclibc* | uclinux-uclibc* \
+			| uclinux-gnu* | kfreebsd*-gnu* | knetbsd*-gnu* | netbsd*-gnu* \
+			| netbsd*-eabi* | kopensolaris*-gnu* | cloudabi*-eabi* \
+			| storm-chaos* | os2-emx* | rtmk-nova*)
+				basic_machine=$field1
+				os=$maybe_os
+				;;
+			android-linux)
+				basic_machine=$field1-unknown
+				os=linux-android
+				;;
+			*)
+				basic_machine=$field1-$field2
+				os=$field3
+				;;
+		esac
 		;;
-	-psos*)
-		os=-psos
+	*-*)
+		# A lone config we happen to match not fitting any pattern
+		case $field1-$field2 in
+			decstation-3100)
+				basic_machine=mips-dec
+				os=
+				;;
+			*-*)
+				# Second component is usually, but not always the OS
+				case $field2 in
+					# Prevent following clause from handling this valid os
+					sun*os*)
+						basic_machine=$field1
+						os=$field2
+						;;
+					# Manufacturers
+					dec* | mips* | sequent* | encore* | pc533* | sgi* | sony* \
+					| att* | 7300* | 3300* | delta* | motorola* | sun[234]* \
+					| unicom* | ibm* | next | hp | isi* | apollo | altos* \
+					| convergent* | ncr* | news | 32* | 3600* | 3100* \
+					| hitachi* | c[123]* | convex* | sun | crds | omron* | dg \
+					| ultra | tti* | harris | dolphin | highlevel | gould \
+					| cbm | ns | masscomp | apple | axis | knuth | cray \
+					| microblaze* | sim | cisco \
+					| oki | wec | wrs | winbond)
+						basic_machine=$field1-$field2
+						os=
+						;;
+					*)
+						basic_machine=$field1
+						os=$field2
+						;;
+				esac
+			;;
+		esac
 		;;
-	-mint | -mint[0-9]*)
-		basic_machine=m68k-atari
-		os=-mint
+	*)
+		# Convert single-component short-hands not valid as part of
+		# multi-component configurations.
+		case $field1 in
+			386bsd)
+				basic_machine=i386-pc
+				os=bsd
+				;;
+			a29khif)
+				basic_machine=a29k-amd
+				os=udi
+				;;
+			adobe68k)
+				basic_machine=m68010-adobe
+				os=scout
+				;;
+			alliant)
+				basic_machine=fx80-alliant
+				os=
+				;;
+			altos | altos3068)
+				basic_machine=m68k-altos
+				os=
+				;;
+			am29k)
+				basic_machine=a29k-none
+				os=bsd
+				;;
+			amdahl)
+				basic_machine=580-amdahl
+				os=sysv
+				;;
+			amiga)
+				basic_machine=m68k-unknown
+				os=
+				;;
+			amigaos | amigados)
+				basic_machine=m68k-unknown
+				os=amigaos
+				;;
+			amigaunix | amix)
+				basic_machine=m68k-unknown
+				os=sysv4
+				;;
+			apollo68)
+				basic_machine=m68k-apollo
+				os=sysv
+				;;
+			apollo68bsd)
+				basic_machine=m68k-apollo
+				os=bsd
+				;;
+			aros)
+				basic_machine=i386-pc
+				os=aros
+				;;
+			aux)
+				basic_machine=m68k-apple
+				os=aux
+				;;
+			balance)
+				basic_machine=ns32k-sequent
+				os=dynix
+				;;
+			blackfin)
+				basic_machine=bfin-unknown
+				os=linux
+				;;
+			cegcc)
+				basic_machine=arm-unknown
+				os=cegcc
+				;;
+			convex-c1)
+				basic_machine=c1-convex
+				os=bsd
+				;;
+			convex-c2)
+				basic_machine=c2-convex
+				os=bsd
+				;;
+			convex-c32)
+				basic_machine=c32-convex
+				os=bsd
+				;;
+			convex-c34)
+				basic_machine=c34-convex
+				os=bsd
+				;;
+			convex-c38)
+				basic_machine=c38-convex
+				os=bsd
+				;;
+			cray)
+				basic_machine=j90-cray
+				os=unicos
+				;;
+			crds | unos)
+				basic_machine=m68k-crds
+				os=
+				;;
+			da30)
+				basic_machine=m68k-da30
+				os=
+				;;
+			decstation | pmax | pmin | dec3100 | decstatn)
+				basic_machine=mips-dec
+				os=
+				;;
+			delta88)
+				basic_machine=m88k-motorola
+				os=sysv3
+				;;
+			dicos)
+				basic_machine=i686-pc
+				os=dicos
+				;;
+			djgpp)
+				basic_machine=i586-pc
+				os=msdosdjgpp
+				;;
+			ebmon29k)
+				basic_machine=a29k-amd
+				os=ebmon
+				;;
+			es1800 | OSE68k | ose68k | ose | OSE)
+				basic_machine=m68k-ericsson
+				os=ose
+				;;
+			gmicro)
+				basic_machine=tron-gmicro
+				os=sysv
+				;;
+			go32)
+				basic_machine=i386-pc
+				os=go32
+				;;
+			h8300hms)
+				basic_machine=h8300-hitachi
+				os=hms
+				;;
+			h8300xray)
+				basic_machine=h8300-hitachi
+				os=xray
+				;;
+			h8500hms)
+				basic_machine=h8500-hitachi
+				os=hms
+				;;
+			harris)
+				basic_machine=m88k-harris
+				os=sysv3
+				;;
+			hp300)
+				basic_machine=m68k-hp
+				;;
+			hp300bsd)
+				basic_machine=m68k-hp
+				os=bsd
+				;;
+			hp300hpux)
+				basic_machine=m68k-hp
+				os=hpux
+				;;
+			hppaosf)
+				basic_machine=hppa1.1-hp
+				os=osf
+				;;
+			hppro)
+				basic_machine=hppa1.1-hp
+				os=proelf
+				;;
+			i386mach)
+				basic_machine=i386-mach
+				os=mach
+				;;
+			vsta)
+				basic_machine=i386-pc
+				os=vsta
+				;;
+			isi68 | isi)
+				basic_machine=m68k-isi
+				os=sysv
+				;;
+			m68knommu)
+				basic_machine=m68k-unknown
+				os=linux
+				;;
+			magnum | m3230)
+				basic_machine=mips-mips
+				os=sysv
+				;;
+			merlin)
+				basic_machine=ns32k-utek
+				os=sysv
+				;;
+			mingw64)
+				basic_machine=x86_64-pc
+				os=mingw64
+				;;
+			mingw32)
+				basic_machine=i686-pc
+				os=mingw32
+				;;
+			mingw32ce)
+				basic_machine=arm-unknown
+				os=mingw32ce
+				;;
+			monitor)
+				basic_machine=m68k-rom68k
+				os=coff
+				;;
+			morphos)
+				basic_machine=powerpc-unknown
+				os=morphos
+				;;
+			moxiebox)
+				basic_machine=moxie-unknown
+				os=moxiebox
+				;;
+			msdos)
+				basic_machine=i386-pc
+				os=msdos
+				;;
+			msys)
+				basic_machine=i686-pc
+				os=msys
+				;;
+			mvs)
+				basic_machine=i370-ibm
+				os=mvs
+				;;
+			nacl)
+				basic_machine=le32-unknown
+				os=nacl
+				;;
+			ncr3000)
+				basic_machine=i486-ncr
+				os=sysv4
+				;;
+			netbsd386)
+				basic_machine=i386-pc
+				os=netbsd
+				;;
+			netwinder)
+				basic_machine=armv4l-rebel
+				os=linux
+				;;
+			news | news700 | news800 | news900)
+				basic_machine=m68k-sony
+				os=newsos
+				;;
+			news1000)
+				basic_machine=m68030-sony
+				os=newsos
+				;;
+			necv70)
+				basic_machine=v70-nec
+				os=sysv
+				;;
+			nh3000)
+				basic_machine=m68k-harris
+				os=cxux
+				;;
+			nh[45]000)
+				basic_machine=m88k-harris
+				os=cxux
+				;;
+			nindy960)
+				basic_machine=i960-intel
+				os=nindy
+				;;
+			mon960)
+				basic_machine=i960-intel
+				os=mon960
+				;;
+			nonstopux)
+				basic_machine=mips-compaq
+				os=nonstopux
+				;;
+			os400)
+				basic_machine=powerpc-ibm
+				os=os400
+				;;
+			OSE68000 | ose68000)
+				basic_machine=m68000-ericsson
+				os=ose
+				;;
+			os68k)
+				basic_machine=m68k-none
+				os=os68k
+				;;
+			paragon)
+				basic_machine=i860-intel
+				os=osf
+				;;
+			parisc)
+				basic_machine=hppa-unknown
+				os=linux
+				;;
+			pw32)
+				basic_machine=i586-unknown
+				os=pw32
+				;;
+			rdos | rdos64)
+				basic_machine=x86_64-pc
+				os=rdos
+				;;
+			rdos32)
+				basic_machine=i386-pc
+				os=rdos
+				;;
+			rom68k)
+				basic_machine=m68k-rom68k
+				os=coff
+				;;
+			sa29200)
+				basic_machine=a29k-amd
+				os=udi
+				;;
+			sei)
+				basic_machine=mips-sei
+				os=seiux
+				;;
+			sequent)
+				basic_machine=i386-sequent
+				os=
+				;;
+			sps7)
+				basic_machine=m68k-bull
+				os=sysv2
+				;;
+			st2000)
+				basic_machine=m68k-tandem
+				os=
+				;;
+			stratus)
+				basic_machine=i860-stratus
+				os=sysv4
+				;;
+			sun2)
+				basic_machine=m68000-sun
+				os=
+				;;
+			sun2os3)
+				basic_machine=m68000-sun
+				os=sunos3
+				;;
+			sun2os4)
+				basic_machine=m68000-sun
+				os=sunos4
+				;;
+			sun3)
+				basic_machine=m68k-sun
+				os=
+				;;
+			sun3os3)
+				basic_machine=m68k-sun
+				os=sunos3
+				;;
+			sun3os4)
+				basic_machine=m68k-sun
+				os=sunos4
+				;;
+			sun4)
+				basic_machine=sparc-sun
+				os=
+				;;
+			sun4os3)
+				basic_machine=sparc-sun
+				os=sunos3
+				;;
+			sun4os4)
+				basic_machine=sparc-sun
+				os=sunos4
+				;;
+			sun4sol2)
+				basic_machine=sparc-sun
+				os=solaris2
+				;;
+			sun386 | sun386i | roadrunner)
+				basic_machine=i386-sun
+				os=
+				;;
+			sv1)
+				basic_machine=sv1-cray
+				os=unicos
+				;;
+			symmetry)
+				basic_machine=i386-sequent
+				os=dynix
+				;;
+			t3e)
+				basic_machine=alphaev5-cray
+				os=unicos
+				;;
+			t90)
+				basic_machine=t90-cray
+				os=unicos
+				;;
+			toad1)
+				basic_machine=pdp10-xkl
+				os=tops20
+				;;
+			tpf)
+				basic_machine=s390x-ibm
+				os=tpf
+				;;
+			udi29k)
+				basic_machine=a29k-amd
+				os=udi
+				;;
+			ultra3)
+				basic_machine=a29k-nyu
+				os=sym1
+				;;
+			v810 | necv810)
+				basic_machine=v810-nec
+				os=none
+				;;
+			vaxv)
+				basic_machine=vax-dec
+				os=sysv
+				;;
+			vms)
+				basic_machine=vax-dec
+				os=vms
+				;;
+			vxworks960)
+				basic_machine=i960-wrs
+				os=vxworks
+				;;
+			vxworks68)
+				basic_machine=m68k-wrs
+				os=vxworks
+				;;
+			vxworks29k)
+				basic_machine=a29k-wrs
+				os=vxworks
+				;;
+			xbox)
+				basic_machine=i686-pc
+				os=mingw32
+				;;
+			ymp)
+				basic_machine=ymp-cray
+				os=unicos
+				;;
+			*)
+				basic_machine=$1
+				os=
+				;;
+		esac
 		;;
 esac
 
-# Decode aliases for certain CPU-COMPANY combinations.
+# Decode 1-component or ad-hoc basic machines
 case $basic_machine in
-	# Recognize the basic CPU types without company name.
-	# Some are omitted here because they have special meanings below.
-	1750a | 580 \
-	| a29k \
-	| aarch64 | aarch64_be \
-	| alpha | alphaev[4-8] | alphaev56 | alphaev6[78] | alphapca5[67] \
-	| alpha64 | alpha64ev[4-8] | alpha64ev56 | alpha64ev6[78] | alpha64pca5[67] \
-	| am33_2.0 \
-	| arc | arceb \
-	| arm | arm[bl]e | arme[lb] | armv[2-8] | armv[3-8][lb] | armv6m | armv[78][arm] \
-	| avr | avr32 \
-	| ba \
-	| be32 | be64 \
-	| bfin \
-	| c4x | c8051 | clipper | csky \
-	| d10v | d30v | dlx | dsp16xx \
-	| e2k | epiphany \
-	| fido | fr30 | frv | ft32 \
-	| h8300 | h8500 | hppa | hppa1.[01] | hppa2.0 | hppa2.0[nw] | hppa64 \
-	| hexagon \
-	| i370 | i860 | i960 | ia16 | ia64 \
-	| ip2k | iq2000 \
-	| k1om \
-	| le32 | le64 \
-	| lm32 \
-	| m32c | m32r | m32rle | m68000 | m68k | m88k \
-	| maxq | mb | microblaze | microblazeel | mcore | mep | metag \
-	| mips | mipsbe | mipseb | mipsel | mipsle \
-	| mips16 \
-	| mips64 | mips64el \
-	| mips64octeon | mips64octeonel \
-	| mips64orion | mips64orionel \
-	| mips64r5900 | mips64r5900el \
-	| mips64vr | mips64vrel \
-	| mips64vr4100 | mips64vr4100el \
-	| mips64vr4300 | mips64vr4300el \
-	| mips64vr5000 | mips64vr5000el \
-	| mips64vr5900 | mips64vr5900el \
-	| mipsisa32 | mipsisa32el \
-	| mipsisa32r2 | mipsisa32r2el \
-	| mipsisa32r6 | mipsisa32r6el \
-	| mipsisa64 | mipsisa64el \
-	| mipsisa64r2 | mipsisa64r2el \
-	| mipsisa64r6 | mipsisa64r6el \
-	| mipsisa64sb1 | mipsisa64sb1el \
-	| mipsisa64sr71k | mipsisa64sr71kel \
-	| mipsr5900 | mipsr5900el \
-	| mipstx39 | mipstx39el \
-	| mn10200 | mn10300 \
-	| moxie \
-	| mt \
-	| msp430 \
-	| nds32 | nds32le | nds32be \
-	| nfp \
-	| nios | nios2 | nios2eb | nios2el \
-	| ns16k | ns32k \
-	| open8 | or1k | or1knd | or32 \
-	| pdp10 | pj | pjl \
-	| powerpc | powerpc64 | powerpc64le | powerpcle \
-	| pru \
-	| pyramid \
-	| riscv32 | riscv64 \
-	| rl78 | rx \
-	| score \
-	| sh | sh[1234] | sh[24]a | sh[24]aeb | sh[23]e | sh[234]eb | sheb | shbe | shle | sh[1234]le | sh3ele \
-	| sh64 | sh64le \
-	| sparc | sparc64 | sparc64b | sparc64v | sparc86x | sparclet | sparclite \
-	| sparcv8 | sparcv9 | sparcv9b | sparcv9v \
-	| spu \
-	| tahoe | tic4x | tic54x | tic55x | tic6x | tic80 | tron \
-	| ubicom32 \
-	| v850 | v850e | v850e1 | v850e2 | v850es | v850e2v3 \
-	| visium \
-	| wasm32 \
-	| x86 | xc16x | xstormy16 | xtensa \
-	| z8k | z80)
-		basic_machine=$basic_machine-unknown
-		;;
-	c54x)
-		basic_machine=tic54x-unknown
-		;;
-	c55x)
-		basic_machine=tic55x-unknown
-		;;
-	c6x)
-		basic_machine=tic6x-unknown
-		;;
-	leon|leon[3-9])
-		basic_machine=sparc-$basic_machine
-		;;
-	m6811 | m68hc11 | m6812 | m68hc12 | m68hcs12x | nvptx | picochip)
-		basic_machine=$basic_machine-unknown
-		os=-none
-		;;
-	m88110 | m680[12346]0 | m683?2 | m68360 | m5200 | v70 | w65)
+	# Here we handle the default manufacturer of certain CPU types.  It is in
+	# some cases the only manufacturer, in others, it is the most popular.
+	w89k)
+		cpu=hppa1.1
+		vendor=winbond
 		;;
-	m9s12z | m68hcs12z | hcs12z | s12z)
-		basic_machine=s12z-unknown
-		os=-none
+	op50n)
+		cpu=hppa1.1
+		vendor=oki
 		;;
-	ms1)
-		basic_machine=mt-unknown
+	op60c)
+		cpu=hppa1.1
+		vendor=oki
 		;;
-
-	strongarm | thumb | xscale)
-		basic_machine=arm-unknown
+	ibm*)
+		cpu=i370
+		vendor=ibm
 		;;
-	xgate)
-		basic_machine=$basic_machine-unknown
-		os=-none
+	orion105)
+		cpu=clipper
+		vendor=highlevel
 		;;
-	xscaleeb)
-		basic_machine=armeb-unknown
+	mac | mpw | mac-mpw)
+		cpu=m68k
+		vendor=apple
 		;;
-
-	xscaleel)
-		basic_machine=armel-unknown
+	pmac | pmac-mpw)
+		cpu=powerpc
+		vendor=apple
 		;;
 
-	# We use `pc' rather than `unknown'
-	# because (1) that's what they normally are, and
-	# (2) the word "unknown" tends to confuse beginning users.
-	i*86 | x86_64)
-	  basic_machine=$basic_machine-pc
-	  ;;
-	# Object if more than one company name word.
-	*-*-*)
-		echo Invalid configuration \`"$1"\': machine \`"$basic_machine"\' not recognized 1>&2
-		exit 1
-		;;
-	# Recognize the basic CPU types with company name.
-	580-* \
-	| a29k-* \
-	| aarch64-* | aarch64_be-* \
-	| alpha-* | alphaev[4-8]-* | alphaev56-* | alphaev6[78]-* \
-	| alpha64-* | alpha64ev[4-8]-* | alpha64ev56-* | alpha64ev6[78]-* \
-	| alphapca5[67]-* | alpha64pca5[67]-* | arc-* | arceb-* \
-	| arm-*  | armbe-* | armle-* | armeb-* | armv*-* \
-	| avr-* | avr32-* \
-	| ba-* \
-	| be32-* | be64-* \
-	| bfin-* | bs2000-* \
-	| c[123]* | c30-* | [cjt]90-* | c4x-* \
-	| c8051-* | clipper-* | craynv-* | csky-* | cydra-* \
-	| d10v-* | d30v-* | dlx-* \
-	| e2k-* | elxsi-* \
-	| f30[01]-* | f700-* | fido-* | fr30-* | frv-* | fx80-* \
-	| h8300-* | h8500-* \
-	| hppa-* | hppa1.[01]-* | hppa2.0-* | hppa2.0[nw]-* | hppa64-* \
-	| hexagon-* \
-	| i*86-* | i860-* | i960-* | ia16-* | ia64-* \
-	| ip2k-* | iq2000-* \
-	| k1om-* \
-	| le32-* | le64-* \
-	| lm32-* \
-	| m32c-* | m32r-* | m32rle-* \
-	| m68000-* | m680[012346]0-* | m68360-* | m683?2-* | m68k-* \
-	| m88110-* | m88k-* | maxq-* | mcore-* | metag-* \
-	| microblaze-* | microblazeel-* \
-	| mips-* | mipsbe-* | mipseb-* | mipsel-* | mipsle-* \
-	| mips16-* \
-	| mips64-* | mips64el-* \
-	| mips64octeon-* | mips64octeonel-* \
-	| mips64orion-* | mips64orionel-* \
-	| mips64r5900-* | mips64r5900el-* \
-	| mips64vr-* | mips64vrel-* \
-	| mips64vr4100-* | mips64vr4100el-* \
-	| mips64vr4300-* | mips64vr4300el-* \
-	| mips64vr5000-* | mips64vr5000el-* \
-	| mips64vr5900-* | mips64vr5900el-* \
-	| mipsisa32-* | mipsisa32el-* \
-	| mipsisa32r2-* | mipsisa32r2el-* \
-	| mipsisa32r6-* | mipsisa32r6el-* \
-	| mipsisa64-* | mipsisa64el-* \
-	| mipsisa64r2-* | mipsisa64r2el-* \
-	| mipsisa64r6-* | mipsisa64r6el-* \
-	| mipsisa64sb1-* | mipsisa64sb1el-* \
-	| mipsisa64sr71k-* | mipsisa64sr71kel-* \
-	| mipsr5900-* | mipsr5900el-* \
-	| mipstx39-* | mipstx39el-* \
-	| mmix-* \
-	| mt-* \
-	| msp430-* \
-	| nds32-* | nds32le-* | nds32be-* \
-	| nfp-* \
-	| nios-* | nios2-* | nios2eb-* | nios2el-* \
-	| none-* | np1-* | ns16k-* | ns32k-* \
-	| open8-* \
-	| or1k*-* \
-	| orion-* \
-	| pdp10-* | pdp11-* | pj-* | pjl-* | pn-* | power-* \
-	| powerpc-* | powerpc64-* | powerpc64le-* | powerpcle-* \
-	| pru-* \
-	| pyramid-* \
-	| riscv32-* | riscv64-* \
-	| rl78-* | romp-* | rs6000-* | rx-* \
-	| sh-* | sh[1234]-* | sh[24]a-* | sh[24]aeb-* | sh[23]e-* | sh[34]eb-* | sheb-* | shbe-* \
-	| shle-* | sh[1234]le-* | sh3ele-* | sh64-* | sh64le-* \
-	| sparc-* | sparc64-* | sparc64b-* | sparc64v-* | sparc86x-* | sparclet-* \
-	| sparclite-* \
-	| sparcv8-* | sparcv9-* | sparcv9b-* | sparcv9v-* | sv1-* | sx*-* \
-	| tahoe-* \
-	| tic30-* | tic4x-* | tic54x-* | tic55x-* | tic6x-* | tic80-* \
-	| tile*-* \
-	| tron-* \
-	| ubicom32-* \
-	| v850-* | v850e-* | v850e1-* | v850es-* | v850e2-* | v850e2v3-* \
-	| vax-* \
-	| visium-* \
-	| wasm32-* \
-	| we32k-* \
-	| x86-* | x86_64-* | xc16x-* | xps100-* \
-	| xstormy16-* | xtensa*-* \
-	| ymp-* \
-	| z8k-* | z80-*)
-		;;
-	# Recognize the basic CPU types without company name, with glob match.
-	xtensa*)
-		basic_machine=$basic_machine-unknown
-		;;
 	# Recognize the various machine names and aliases which stand
 	# for a CPU type and a company and sometimes even an OS.
-	386bsd)
-		basic_machine=i386-pc
-		os=-bsd
-		;;
 	3b1 | 7300 | 7300-att | att-7300 | pc7300 | safari | unixpc)
-		basic_machine=m68000-att
+		cpu=m68000
+		vendor=att
 		;;
 	3b*)
-		basic_machine=we32k-att
-		;;
-	a29khif)
-		basic_machine=a29k-amd
-		os=-udi
-		;;
-	abacus)
-		basic_machine=abacus-unknown
-		;;
-	adobe68k)
-		basic_machine=m68010-adobe
-		os=-scout
-		;;
-	alliant | fx80)
-		basic_machine=fx80-alliant
-		;;
-	altos | altos3068)
-		basic_machine=m68k-altos
-		;;
-	am29k)
-		basic_machine=a29k-none
-		os=-bsd
-		;;
-	amd64)
-		basic_machine=x86_64-pc
-		;;
-	amd64-*)
-		basic_machine=x86_64-`echo "$basic_machine" | sed 's/^[^-]*-//'`
-		;;
-	amdahl)
-		basic_machine=580-amdahl
-		os=-sysv
-		;;
-	amiga | amiga-*)
-		basic_machine=m68k-unknown
-		;;
-	amigaos | amigados)
-		basic_machine=m68k-unknown
-		os=-amigaos
-		;;
-	amigaunix | amix)
-		basic_machine=m68k-unknown
-		os=-sysv4
-		;;
-	apollo68)
-		basic_machine=m68k-apollo
-		os=-sysv
-		;;
-	apollo68bsd)
-		basic_machine=m68k-apollo
-		os=-bsd
-		;;
-	aros)
-		basic_machine=i386-pc
-		os=-aros
-		;;
-	asmjs)
-		basic_machine=asmjs-unknown
-		;;
-	aux)
-		basic_machine=m68k-apple
-		os=-aux
-		;;
-	balance)
-		basic_machine=ns32k-sequent
-		os=-dynix
-		;;
-	blackfin)
-		basic_machine=bfin-unknown
-		os=-linux
-		;;
-	blackfin-*)
-		basic_machine=bfin-`echo "$basic_machine" | sed 's/^[^-]*-//'`
-		os=-linux
+		cpu=we32k
+		vendor=att
 		;;
 	bluegene*)
-		basic_machine=powerpc-ibm
-		os=-cnk
-		;;
-	c54x-*)
-		basic_machine=tic54x-`echo "$basic_machine" | sed 's/^[^-]*-//'`
-		;;
-	c55x-*)
-		basic_machine=tic55x-`echo "$basic_machine" | sed 's/^[^-]*-//'`
-		;;
-	c6x-*)
-		basic_machine=tic6x-`echo "$basic_machine" | sed 's/^[^-]*-//'`
-		;;
-	c90)
-		basic_machine=c90-cray
-		os=-unicos
-		;;
-	cegcc)
-		basic_machine=arm-unknown
-		os=-cegcc
-		;;
-	convex-c1)
-		basic_machine=c1-convex
-		os=-bsd
-		;;
-	convex-c2)
-		basic_machine=c2-convex
-		os=-bsd
-		;;
-	convex-c32)
-		basic_machine=c32-convex
-		os=-bsd
-		;;
-	convex-c34)
-		basic_machine=c34-convex
-		os=-bsd
-		;;
-	convex-c38)
-		basic_machine=c38-convex
-		os=-bsd
-		;;
-	cray | j90)
-		basic_machine=j90-cray
-		os=-unicos
-		;;
-	craynv)
-		basic_machine=craynv-cray
-		os=-unicosmp
-		;;
-	cr16 | cr16-*)
-		basic_machine=cr16-unknown
-		os=-elf
-		;;
-	crds | unos)
-		basic_machine=m68k-crds
-		;;
-	crisv32 | crisv32-* | etraxfs*)
-		basic_machine=crisv32-axis
-		;;
-	cris | cris-* | etrax*)
-		basic_machine=cris-axis
-		;;
-	crx)
-		basic_machine=crx-unknown
-		os=-elf
-		;;
-	da30 | da30-*)
-		basic_machine=m68k-da30
-		;;
-	decstation | decstation-3100 | pmax | pmax-* | pmin | dec3100 | decstatn)
-		basic_machine=mips-dec
+		cpu=powerpc
+		vendor=ibm
+		os=cnk
 		;;
 	decsystem10* | dec10*)
-		basic_machine=pdp10-dec
-		os=-tops10
+		cpu=pdp10
+		vendor=dec
+		os=tops10
 		;;
 	decsystem20* | dec20*)
-		basic_machine=pdp10-dec
-		os=-tops20
+		cpu=pdp10
+		vendor=dec
+		os=tops20
 		;;
 	delta | 3300 | motorola-3300 | motorola-delta \
 	      | 3300-motorola | delta-motorola)
-		basic_machine=m68k-motorola
-		;;
-	delta88)
-		basic_machine=m88k-motorola
-		os=-sysv3
-		;;
-	dicos)
-		basic_machine=i686-pc
-		os=-dicos
-		;;
-	djgpp)
-		basic_machine=i586-pc
-		os=-msdosdjgpp
-		;;
-	dpx20 | dpx20-*)
-		basic_machine=rs6000-bull
-		os=-bosx
+		cpu=m68k
+		vendor=motorola
 		;;
 	dpx2*)
-		basic_machine=m68k-bull
-		os=-sysv3
-		;;
-	e500v[12])
-		basic_machine=powerpc-unknown
-		os=$os"spe"
-		;;
-	e500v[12]-*)
-		basic_machine=powerpc-`echo "$basic_machine" | sed 's/^[^-]*-//'`
-		os=$os"spe"
-		;;
-	ebmon29k)
-		basic_machine=a29k-amd
-		os=-ebmon
-		;;
-	elxsi)
-		basic_machine=elxsi-elxsi
-		os=-bsd
+		cpu=m68k
+		vendor=bull
+		os=sysv3
 		;;
 	encore | umax | mmax)
-		basic_machine=ns32k-encore
+		cpu=ns32k
+		vendor=encore
 		;;
-	es1800 | OSE68k | ose68k | ose | OSE)
-		basic_machine=m68k-ericsson
-		os=-ose
+	elxsi)
+		cpu=elxsi
+		vendor=elxsi
+		os=${os:-bsd}
 		;;
 	fx2800)
-		basic_machine=i860-alliant
+		cpu=i860
+		vendor=alliant
 		;;
 	genix)
-		basic_machine=ns32k-ns
-		;;
-	gmicro)
-		basic_machine=tron-gmicro
-		os=-sysv
-		;;
-	go32)
-		basic_machine=i386-pc
-		os=-go32
+		cpu=ns32k
+		vendor=ns
 		;;
 	h3050r* | hiux*)
-		basic_machine=hppa1.1-hitachi
-		os=-hiuxwe2
-		;;
-	h8300hms)
-		basic_machine=h8300-hitachi
-		os=-hms
-		;;
-	h8300xray)
-		basic_machine=h8300-hitachi
-		os=-xray
-		;;
-	h8500hms)
-		basic_machine=h8500-hitachi
-		os=-hms
-		;;
-	harris)
-		basic_machine=m88k-harris
-		os=-sysv3
-		;;
-	hp300-*)
-		basic_machine=m68k-hp
-		;;
-	hp300bsd)
-		basic_machine=m68k-hp
-		os=-bsd
-		;;
-	hp300hpux)
-		basic_machine=m68k-hp
-		os=-hpux
+		cpu=hppa1.1
+		vendor=hitachi
+		os=hiuxwe2
 		;;
 	hp3k9[0-9][0-9] | hp9[0-9][0-9])
-		basic_machine=hppa1.0-hp
+		cpu=hppa1.0
+		vendor=hp
 		;;
 	hp9k2[0-9][0-9] | hp9k31[0-9])
-		basic_machine=m68000-hp
+		cpu=m68000
+		vendor=hp
 		;;
 	hp9k3[2-9][0-9])
-		basic_machine=m68k-hp
+		cpu=m68k
+		vendor=hp
 		;;
 	hp9k6[0-9][0-9] | hp6[0-9][0-9])
-		basic_machine=hppa1.0-hp
+		cpu=hppa1.0
+		vendor=hp
 		;;
 	hp9k7[0-79][0-9] | hp7[0-79][0-9])
-		basic_machine=hppa1.1-hp
+		cpu=hppa1.1
+		vendor=hp
 		;;
 	hp9k78[0-9] | hp78[0-9])
 		# FIXME: really hppa2.0-hp
-		basic_machine=hppa1.1-hp
+		cpu=hppa1.1
+		vendor=hp
 		;;
 	hp9k8[67]1 | hp8[67]1 | hp9k80[24] | hp80[24] | hp9k8[78]9 | hp8[78]9 | hp9k893 | hp893)
 		# FIXME: really hppa2.0-hp
-		basic_machine=hppa1.1-hp
+		cpu=hppa1.1
+		vendor=hp
 		;;
 	hp9k8[0-9][13679] | hp8[0-9][13679])
-		basic_machine=hppa1.1-hp
+		cpu=hppa1.1
+		vendor=hp
 		;;
 	hp9k8[0-9][0-9] | hp8[0-9][0-9])
-		basic_machine=hppa1.0-hp
-		;;
-	hppaosf)
-		basic_machine=hppa1.1-hp
-		os=-osf
-		;;
-	hppro)
-		basic_machine=hppa1.1-hp
-		os=-proelf
-		;;
-	i370-ibm* | ibm*)
-		basic_machine=i370-ibm
+		cpu=hppa1.0
+		vendor=hp
 		;;
 	i*86v32)
-		basic_machine=`echo "$1" | sed -e 's/86.*/86-pc/'`
-		os=-sysv32
+		cpu=`echo "$1" | sed -e 's/86.*/86/'`
+		vendor=pc
+		os=sysv32
 		;;
 	i*86v4*)
-		basic_machine=`echo "$1" | sed -e 's/86.*/86-pc/'`
-		os=-sysv4
+		cpu=`echo "$1" | sed -e 's/86.*/86/'`
+		vendor=pc
+		os=sysv4
 		;;
 	i*86v)
-		basic_machine=`echo "$1" | sed -e 's/86.*/86-pc/'`
-		os=-sysv
+		cpu=`echo "$1" | sed -e 's/86.*/86/'`
+		vendor=pc
+		os=sysv
 		;;
 	i*86sol2)
-		basic_machine=`echo "$1" | sed -e 's/86.*/86-pc/'`
-		os=-solaris2
-		;;
-	i386mach)
-		basic_machine=i386-mach
-		os=-mach
+		cpu=`echo "$1" | sed -e 's/86.*/86/'`
+		vendor=pc
+		os=solaris2
 		;;
-	vsta)
-		basic_machine=i386-unknown
-		os=-vsta
+	j90 | j90-cray)
+		cpu=j90
+		vendor=cray
+		os=${os:-unicos}
 		;;
 	iris | iris4d)
-		basic_machine=mips-sgi
+		cpu=mips
+		vendor=sgi
 		case $os in
-		    -irix*)
+		    irix*)
 			;;
 		    *)
-			os=-irix4
+			os=irix4
 			;;
 		esac
 		;;
-	isi68 | isi)
-		basic_machine=m68k-isi
-		os=-sysv
-		;;
-	leon-*|leon[3-9]-*)
-		basic_machine=sparc-`echo "$basic_machine" | sed 's/-.*//'`
-		;;
-	m68knommu)
-		basic_machine=m68k-unknown
-		os=-linux
-		;;
-	m68knommu-*)
-		basic_machine=m68k-`echo "$basic_machine" | sed 's/^[^-]*-//'`
-		os=-linux
-		;;
-	magnum | m3230)
-		basic_machine=mips-mips
-		os=-sysv
-		;;
-	merlin)
-		basic_machine=ns32k-utek
-		os=-sysv
-		;;
-	microblaze*)
-		basic_machine=microblaze-xilinx
-		;;
-	mingw64)
-		basic_machine=x86_64-pc
-		os=-mingw64
-		;;
-	mingw32)
-		basic_machine=i686-pc
-		os=-mingw32
-		;;
-	mingw32ce)
-		basic_machine=arm-unknown
-		os=-mingw32ce
-		;;
 	miniframe)
-		basic_machine=m68000-convergent
-		;;
-	*mint | -mint[0-9]* | *MiNT | *MiNT[0-9]*)
-		basic_machine=m68k-atari
-		os=-mint
-		;;
-	mips3*-*)
-		basic_machine=`echo "$basic_machine" | sed -e 's/mips3/mips64/'`
-		;;
-	mips3*)
-		basic_machine=`echo "$basic_machine" | sed -e 's/mips3/mips64/'`-unknown
-		;;
-	monitor)
-		basic_machine=m68k-rom68k
-		os=-coff
-		;;
-	morphos)
-		basic_machine=powerpc-unknown
-		os=-morphos
-		;;
-	moxiebox)
-		basic_machine=moxie-unknown
-		os=-moxiebox
-		;;
-	msdos)
-		basic_machine=i386-pc
-		os=-msdos
-		;;
-	ms1-*)
-		basic_machine=`echo "$basic_machine" | sed -e 's/ms1-/mt-/'`
-		;;
-	msys)
-		basic_machine=i686-pc
-		os=-msys
-		;;
-	mvs)
-		basic_machine=i370-ibm
-		os=-mvs
-		;;
-	nacl)
-		basic_machine=le32-unknown
-		os=-nacl
+		cpu=m68000
+		vendor=convergent
 		;;
-	ncr3000)
-		basic_machine=i486-ncr
-		os=-sysv4
-		;;
-	netbsd386)
-		basic_machine=i386-unknown
-		os=-netbsd
-		;;
-	netwinder)
-		basic_machine=armv4l-rebel
-		os=-linux
-		;;
-	news | news700 | news800 | news900)
-		basic_machine=m68k-sony
-		os=-newsos
-		;;
-	news1000)
-		basic_machine=m68030-sony
-		os=-newsos
+	*mint | mint[0-9]* | *MiNT | *MiNT[0-9]*)
+		cpu=m68k
+		vendor=atari
+		os=mint
 		;;
 	news-3600 | risc-news)
-		basic_machine=mips-sony
-		os=-newsos
-		;;
-	necv70)
-		basic_machine=v70-nec
-		os=-sysv
+		cpu=mips
+		vendor=sony
+		os=newsos
 		;;
 	next | m*-next)
-		basic_machine=m68k-next
+		cpu=m68k
+		vendor=next
 		case $os in
-		    -nextstep* )
+		    openstep*)
+		        ;;
+		    nextstep*)
 			;;
-		    -ns2*)
-		      os=-nextstep2
+		    ns2*)
+		      os=nextstep2
 			;;
 		    *)
-		      os=-nextstep3
+		      os=nextstep3
 			;;
 		esac
 		;;
-	nh3000)
-		basic_machine=m68k-harris
-		os=-cxux
-		;;
-	nh[45]000)
-		basic_machine=m88k-harris
-		os=-cxux
-		;;
-	nindy960)
-		basic_machine=i960-intel
-		os=-nindy
-		;;
-	mon960)
-		basic_machine=i960-intel
-		os=-mon960
-		;;
-	nonstopux)
-		basic_machine=mips-compaq
-		os=-nonstopux
-		;;
 	np1)
-		basic_machine=np1-gould
-		;;
-	neo-tandem)
-		basic_machine=neo-tandem
-		;;
-	nse-tandem)
-		basic_machine=nse-tandem
-		;;
-	nsr-tandem)
-		basic_machine=nsr-tandem
-		;;
-	nsv-tandem)
-		basic_machine=nsv-tandem
-		;;
-	nsx-tandem)
-		basic_machine=nsx-tandem
+		cpu=np1
+		vendor=gould
 		;;
 	op50n-* | op60c-*)
-		basic_machine=hppa1.1-oki
-		os=-proelf
-		;;
-	openrisc | openrisc-*)
-		basic_machine=or32-unknown
-		;;
-	os400)
-		basic_machine=powerpc-ibm
-		os=-os400
-		;;
-	OSE68000 | ose68000)
-		basic_machine=m68000-ericsson
-		os=-ose
-		;;
-	os68k)
-		basic_machine=m68k-none
-		os=-os68k
+		cpu=hppa1.1
+		vendor=oki
+		os=proelf
 		;;
 	pa-hitachi)
-		basic_machine=hppa1.1-hitachi
-		os=-hiuxwe2
-		;;
-	paragon)
-		basic_machine=i860-intel
-		os=-osf
-		;;
-	parisc)
-		basic_machine=hppa-unknown
-		os=-linux
-		;;
-	parisc-*)
-		basic_machine=hppa-`echo "$basic_machine" | sed 's/^[^-]*-//'`
-		os=-linux
+		cpu=hppa1.1
+		vendor=hitachi
+		os=hiuxwe2
 		;;
 	pbd)
-		basic_machine=sparc-tti
+		cpu=sparc
+		vendor=tti
 		;;
 	pbb)
-		basic_machine=m68k-tti
-		;;
-	pc532 | pc532-*)
-		basic_machine=ns32k-pc532
-		;;
-	pc98)
-		basic_machine=i386-pc
-		;;
-	pc98-*)
-		basic_machine=i386-`echo "$basic_machine" | sed 's/^[^-]*-//'`
-		;;
-	pentium | p5 | k5 | k6 | nexgen | viac3)
-		basic_machine=i586-pc
-		;;
-	pentiumpro | p6 | 6x86 | athlon | athlon_*)
-		basic_machine=i686-pc
-		;;
-	pentiumii | pentium2 | pentiumiii | pentium3)
-		basic_machine=i686-pc
-		;;
-	pentium4)
-		basic_machine=i786-pc
-		;;
-	pentium-* | p5-* | k5-* | k6-* | nexgen-* | viac3-*)
-		basic_machine=i586-`echo "$basic_machine" | sed 's/^[^-]*-//'`
-		;;
-	pentiumpro-* | p6-* | 6x86-* | athlon-*)
-		basic_machine=i686-`echo "$basic_machine" | sed 's/^[^-]*-//'`
+		cpu=m68k
+		vendor=tti
 		;;
-	pentiumii-* | pentium2-* | pentiumiii-* | pentium3-*)
-		basic_machine=i686-`echo "$basic_machine" | sed 's/^[^-]*-//'`
-		;;
-	pentium4-*)
-		basic_machine=i786-`echo "$basic_machine" | sed 's/^[^-]*-//'`
+	pc532)
+		cpu=ns32k
+		vendor=pc532
 		;;
 	pn)
-		basic_machine=pn-gould
-		;;
-	power)	basic_machine=power-ibm
-		;;
-	ppc | ppcbe)	basic_machine=powerpc-unknown
-		;;
-	ppc-* | ppcbe-*)
-		basic_machine=powerpc-`echo "$basic_machine" | sed 's/^[^-]*-//'`
+		cpu=pn
+		vendor=gould
 		;;
-	ppcle | powerpclittle)
-		basic_machine=powerpcle-unknown
+	power)
+		cpu=power
+		vendor=ibm
 		;;
-	ppcle-* | powerpclittle-*)
-		basic_machine=powerpcle-`echo "$basic_machine" | sed 's/^[^-]*-//'`
-		;;
-	ppc64)	basic_machine=powerpc64-unknown
+	ps2)
+		cpu=i386
+		vendor=ibm
 		;;
-	ppc64-*) basic_machine=powerpc64-`echo "$basic_machine" | sed 's/^[^-]*-//'`
+	rm[46]00)
+		cpu=mips
+		vendor=siemens
 		;;
-	ppc64le | powerpc64little)
-		basic_machine=powerpc64le-unknown
+	rtpc | rtpc-*)
+		cpu=romp
+		vendor=ibm
 		;;
-	ppc64le-* | powerpc64little-*)
-		basic_machine=powerpc64le-`echo "$basic_machine" | sed 's/^[^-]*-//'`
+	sde)
+		cpu=mipsisa32
+		vendor=sde
+		os=${os:-elf}
 		;;
-	ps2)
-		basic_machine=i386-ibm
+	simso-wrs)
+		cpu=sparclite
+		vendor=wrs
+		os=vxworks
 		;;
-	pw32)
-		basic_machine=i586-unknown
-		os=-pw32
+	tower | tower-32)
+		cpu=m68k
+		vendor=ncr
 		;;
-	rdos | rdos64)
-		basic_machine=x86_64-pc
-		os=-rdos
+	vpp*|vx|vx-*)
+		cpu=f301
+		vendor=fujitsu
 		;;
-	rdos32)
-		basic_machine=i386-pc
-		os=-rdos
+	w65)
+		cpu=w65
+		vendor=wdc
 		;;
-	rom68k)
-		basic_machine=m68k-rom68k
-		os=-coff
+	w89k-*)
+		cpu=hppa1.1
+		vendor=winbond
+		os=proelf
 		;;
-	rm[46]00)
-		basic_machine=mips-siemens
+	none)
+		cpu=none
+		vendor=none
 		;;
-	rtpc | rtpc-*)
-		basic_machine=romp-ibm
+	leon|leon[3-9])
+		cpu=sparc
+		vendor=$basic_machine
 		;;
-	s390 | s390-*)
-		basic_machine=s390-ibm
+	leon-*|leon[3-9]-*)
+		cpu=sparc
+		vendor=`echo "$basic_machine" | sed 's/-.*//'`
 		;;
-	s390x | s390x-*)
-		basic_machine=s390x-ibm
+
+	*-*)
+		# shellcheck disable=SC2162
+		IFS="-" read cpu vendor <<EOF
+$basic_machine
+EOF
 		;;
-	sa29200)
-		basic_machine=a29k-amd
-		os=-udi
+	# We use `pc' rather than `unknown'
+	# because (1) that's what they normally are, and
+	# (2) the word "unknown" tends to confuse beginning users.
+	i*86 | x86_64)
+		cpu=$basic_machine
+		vendor=pc
 		;;
-	sb1)
-		basic_machine=mipsisa64sb1-unknown
+	# These rules are duplicated from below for sake of the special case above;
+	# i.e. things that normalized to x86 arches should also default to "pc"
+	pc98)
+		cpu=i386
+		vendor=pc
 		;;
-	sb1el)
-		basic_machine=mipsisa64sb1el-unknown
+	x64 | amd64)
+		cpu=x86_64
+		vendor=pc
 		;;
-	sde)
-		basic_machine=mipsisa32-sde
-		os=-elf
+	# Recognize the basic CPU types without company name.
+	*)
+		cpu=$basic_machine
+		vendor=unknown
 		;;
-	sei)
-		basic_machine=mips-sei
-		os=-seiux
+esac
+
+unset -v basic_machine
+
+# Decode basic machines in the full and proper CPU-Company form.
+case $cpu-$vendor in
+	# Here we handle the default manufacturer of certain CPU types in canonical form. It is in
+	# some cases the only manufacturer, in others, it is the most popular.
+	craynv-unknown)
+		vendor=cray
+		os=${os:-unicosmp}
 		;;
-	sequent)
-		basic_machine=i386-sequent
+	c90-unknown | c90-cray)
+		vendor=cray
+		os=${os:-unicos}
 		;;
-	sh5el)
-		basic_machine=sh5le-unknown
+	fx80-unknown)
+		vendor=alliant
 		;;
-	simso-wrs)
-		basic_machine=sparclite-wrs
-		os=-vxworks
+	romp-unknown)
+		vendor=ibm
 		;;
-	sps7)
-		basic_machine=m68k-bull
-		os=-sysv2
+	mmix-unknown)
+		vendor=knuth
 		;;
-	spur)
-		basic_machine=spur-unknown
+	microblaze-unknown | microblazeel-unknown)
+		vendor=xilinx
 		;;
-	st2000)
-		basic_machine=m68k-tandem
+	rs6000-unknown)
+		vendor=ibm
 		;;
-	stratus)
-		basic_machine=i860-stratus
-		os=-sysv4
+	vax-unknown)
+		vendor=dec
 		;;
-	strongarm-* | thumb-*)
-		basic_machine=arm-`echo "$basic_machine" | sed 's/^[^-]*-//'`
+	pdp11-unknown)
+		vendor=dec
 		;;
-	sun2)
-		basic_machine=m68000-sun
+	we32k-unknown)
+		vendor=att
 		;;
-	sun2os3)
-		basic_machine=m68000-sun
-		os=-sunos3
+	cydra-unknown)
+		vendor=cydrome
 		;;
-	sun2os4)
-		basic_machine=m68000-sun
-		os=-sunos4
+	i370-ibm*)
+		vendor=ibm
 		;;
-	sun3os3)
-		basic_machine=m68k-sun
-		os=-sunos3
+	orion-unknown)
+		vendor=highlevel
 		;;
-	sun3os4)
-		basic_machine=m68k-sun
-		os=-sunos4
+	xps-unknown | xps100-unknown)
+		cpu=xps100
+		vendor=honeywell
 		;;
-	sun4os3)
-		basic_machine=sparc-sun
-		os=-sunos3
+
+	# Here we normalize CPU types with a missing or matching vendor
+	dpx20-unknown | dpx20-bull)
+		cpu=rs6000
+		vendor=bull
+		os=${os:-bosx}
 		;;
-	sun4os4)
-		basic_machine=sparc-sun
-		os=-sunos4
+
+	# Here we normalize CPU types irrespective of the vendor
+	amd64-*)
+		cpu=x86_64
 		;;
-	sun4sol2)
-		basic_machine=sparc-sun
-		os=-solaris2
+	blackfin-*)
+		cpu=bfin
+		os=linux
 		;;
-	sun3 | sun3-*)
-		basic_machine=m68k-sun
+	c54x-*)
+		cpu=tic54x
 		;;
-	sun4)
-		basic_machine=sparc-sun
+	c55x-*)
+		cpu=tic55x
 		;;
-	sun386 | sun386i | roadrunner)
-		basic_machine=i386-sun
+	c6x-*)
+		cpu=tic6x
 		;;
-	sv1)
-		basic_machine=sv1-cray
-		os=-unicos
+	e500v[12]-*)
+		cpu=powerpc
+		os=$os"spe"
 		;;
-	symmetry)
-		basic_machine=i386-sequent
-		os=-dynix
+	mips3*-*)
+		cpu=mips64
 		;;
-	t3e)
-		basic_machine=alphaev5-cray
-		os=-unicos
+	ms1-*)
+		cpu=mt
 		;;
-	t90)
-		basic_machine=t90-cray
-		os=-unicos
+	m68knommu-*)
+		cpu=m68k
+		os=linux
 		;;
-	tile*)
-		basic_machine=$basic_machine-unknown
-		os=-linux-gnu
+	m9s12z-* | m68hcs12z-* | hcs12z-* | s12z-*)
+		cpu=s12z
 		;;
-	tx39)
-		basic_machine=mipstx39-unknown
+	openrisc-*)
+		cpu=or32
 		;;
-	tx39el)
-		basic_machine=mipstx39el-unknown
+	parisc-*)
+		cpu=hppa
+		os=linux
 		;;
-	toad1)
-		basic_machine=pdp10-xkl
-		os=-tops20
+	pentium-* | p5-* | k5-* | k6-* | nexgen-* | viac3-*)
+		cpu=i586
 		;;
-	tower | tower-32)
-		basic_machine=m68k-ncr
+	pentiumpro-* | p6-* | 6x86-* | athlon-* | athalon_*-*)
+		cpu=i686
 		;;
-	tpf)
-		basic_machine=s390x-ibm
-		os=-tpf
+	pentiumii-* | pentium2-* | pentiumiii-* | pentium3-*)
+		cpu=i686
 		;;
-	udi29k)
-		basic_machine=a29k-amd
-		os=-udi
+	pentium4-*)
+		cpu=i786
 		;;
-	ultra3)
-		basic_machine=a29k-nyu
-		os=-sym1
+	pc98-*)
+		cpu=i386
 		;;
-	v810 | necv810)
-		basic_machine=v810-nec
-		os=-none
+	ppc-* | ppcbe-*)
+		cpu=powerpc
 		;;
-	vaxv)
-		basic_machine=vax-dec
-		os=-sysv
+	ppcle-* | powerpclittle-*)
+		cpu=powerpcle
 		;;
-	vms)
-		basic_machine=vax-dec
-		os=-vms
+	ppc64-*)
+		cpu=powerpc64
 		;;
-	vpp*|vx|vx-*)
-		basic_machine=f301-fujitsu
+	ppc64le-* | powerpc64little-*)
+		cpu=powerpc64le
 		;;
-	vxworks960)
-		basic_machine=i960-wrs
-		os=-vxworks
+	sb1-*)
+		cpu=mipsisa64sb1
 		;;
-	vxworks68)
-		basic_machine=m68k-wrs
-		os=-vxworks
+	sb1el-*)
+		cpu=mipsisa64sb1el
 		;;
-	vxworks29k)
-		basic_machine=a29k-wrs
-		os=-vxworks
+	sh5e[lb]-*)
+		cpu=`echo "$cpu" | sed 's/^\(sh.\)e\(.\)$/\1\2e/'`
 		;;
-	w65*)
-		basic_machine=w65-wdc
-		os=-none
+	spur-*)
+		cpu=spur
 		;;
-	w89k-*)
-		basic_machine=hppa1.1-winbond
-		os=-proelf
+	strongarm-* | thumb-*)
+		cpu=arm
 		;;
-	x64)
-		basic_machine=x86_64-pc
+	tx39-*)
+		cpu=mipstx39
 		;;
-	xbox)
-		basic_machine=i686-pc
-		os=-mingw32
+	tx39el-*)
+		cpu=mipstx39el
 		;;
-	xps | xps100)
-		basic_machine=xps100-honeywell
+	x64-*)
+		cpu=x86_64
 		;;
 	xscale-* | xscalee[bl]-*)
-		basic_machine=`echo "$basic_machine" | sed 's/^xscale/arm/'`
-		;;
-	ymp)
-		basic_machine=ymp-cray
-		os=-unicos
-		;;
-	none)
-		basic_machine=none-none
-		os=-none
+		cpu=`echo "$cpu" | sed 's/^xscale/arm/'`
 		;;
 
-# Here we handle the default manufacturer of certain CPU types.  It is in
-# some cases the only manufacturer, in others, it is the most popular.
-	w89k)
-		basic_machine=hppa1.1-winbond
+	# Recognize the canonical CPU Types that limit and/or modify the
+	# company names they are paired with.
+	cr16-*)
+		os=${os:-elf}
 		;;
-	op50n)
-		basic_machine=hppa1.1-oki
-		;;
-	op60c)
-		basic_machine=hppa1.1-oki
-		;;
-	romp)
-		basic_machine=romp-ibm
+	crisv32-* | etraxfs*-*)
+		cpu=crisv32
+		vendor=axis
 		;;
-	mmix)
-		basic_machine=mmix-knuth
+	cris-* | etrax*-*)
+		cpu=cris
+		vendor=axis
 		;;
-	rs6000)
-		basic_machine=rs6000-ibm
+	crx-*)
+		os=${os:-elf}
 		;;
-	vax)
-		basic_machine=vax-dec
-		;;
-	pdp11)
-		basic_machine=pdp11-dec
-		;;
-	we32k)
-		basic_machine=we32k-att
+	neo-tandem)
+		cpu=neo
+		vendor=tandem
 		;;
-	sh[1234] | sh[24]a | sh[24]aeb | sh[34]eb | sh[1234]le | sh[23]ele)
-		basic_machine=sh-unknown
+	nse-tandem)
+		cpu=nse
+		vendor=tandem
 		;;
-	cydra)
-		basic_machine=cydra-cydrome
+	nsr-tandem)
+		cpu=nsr
+		vendor=tandem
 		;;
-	orion)
-		basic_machine=orion-highlevel
+	nsv-tandem)
+		cpu=nsv
+		vendor=tandem
 		;;
-	orion105)
-		basic_machine=clipper-highlevel
+	nsx-tandem)
+		cpu=nsx
+		vendor=tandem
 		;;
-	mac | mpw | mac-mpw)
-		basic_machine=m68k-apple
+	s390-*)
+		cpu=s390
+		vendor=ibm
 		;;
-	pmac | pmac-mpw)
-		basic_machine=powerpc-apple
+	s390x-*)
+		cpu=s390x
+		vendor=ibm
 		;;
-	*-unknown)
-		# Make sure to match an already-canonicalized machine name.
+	tile*-*)
+		os=${os:-linux-gnu}
 		;;
+
 	*)
-		echo Invalid configuration \`"$1"\': machine \`"$basic_machine"\' not recognized 1>&2
-		exit 1
+		# Recognize the canonical CPU types that are allowed with any
+		# company name.
+		case $cpu in
+			1750a | 580 \
+			| a29k \
+			| aarch64 | aarch64_be \
+			| abacus \
+			| alpha | alphaev[4-8] | alphaev56 | alphaev6[78] \
+			| alpha64 | alpha64ev[4-8] | alpha64ev56 | alpha64ev6[78] \
+			| alphapca5[67] | alpha64pca5[67] \
+			| am33_2.0 \
+			| amdgcn \
+			| arc | arceb \
+			| arm  | arm[lb]e | arme[lb] | armv* \
+			| avr | avr32 \
+			| asmjs \
+			| ba \
+			| be32 | be64 \
+			| bfin | bs2000 \
+			| c[123]* | c30 | [cjt]90 | c4x \
+			| c8051 | clipper | craynv | csky | cydra \
+			| d10v | d30v | dlx | dsp16xx \
+			| e2k | elxsi | epiphany \
+			| f30[01] | f700 | fido | fr30 | frv | ft32 | fx80 \
+			| h8300 | h8500 \
+			| hppa | hppa1.[01] | hppa2.0 | hppa2.0[nw] | hppa64 \
+			| hexagon \
+			| i370 | i*86 | i860 | i960 | ia16 | ia64 \
+			| ip2k | iq2000 \
+			| k1om \
+			| le32 | le64 \
+			| lm32 \
+			| m32c | m32r | m32rle \
+			| m5200 | m68000 | m680[012346]0 | m68360 | m683?2 | m68k \
+			| m6811 | m68hc11 | m6812 | m68hc12 | m68hcs12x \
+			| m88110 | m88k | maxq | mb | mcore | mep | metag \
+			| microblaze | microblazeel \
+			| mips | mipsbe | mipseb | mipsel | mipsle \
+			| mips16 \
+			| mips64 | mips64eb | mips64el \
+			| mips64octeon | mips64octeonel \
+			| mips64orion | mips64orionel \
+			| mips64r5900 | mips64r5900el \
+			| mips64vr | mips64vrel \
+			| mips64vr4100 | mips64vr4100el \
+			| mips64vr4300 | mips64vr4300el \
+			| mips64vr5000 | mips64vr5000el \
+			| mips64vr5900 | mips64vr5900el \
+			| mipsisa32 | mipsisa32el \
+			| mipsisa32r2 | mipsisa32r2el \
+			| mipsisa32r6 | mipsisa32r6el \
+			| mipsisa64 | mipsisa64el \
+			| mipsisa64r2 | mipsisa64r2el \
+			| mipsisa64r6 | mipsisa64r6el \
+			| mipsisa64sb1 | mipsisa64sb1el \
+			| mipsisa64sr71k | mipsisa64sr71kel \
+			| mipsr5900 | mipsr5900el \
+			| mipstx39 | mipstx39el \
+			| mmix \
+			| mn10200 | mn10300 \
+			| moxie \
+			| mt \
+			| msp430 \
+			| nds32 | nds32le | nds32be \
+			| nfp \
+			| nios | nios2 | nios2eb | nios2el \
+			| none | np1 | ns16k | ns32k | nvptx \
+			| open8 \
+			| or1k* \
+			| or32 \
+			| orion \
+			| picochip \
+			| pdp10 | pdp11 | pj | pjl | pn | power \
+			| powerpc | powerpc64 | powerpc64le | powerpcle | powerpcspe \
+			| pru \
+			| pyramid \
+			| riscv | riscv32 | riscv64 \
+			| rl78 | romp | rs6000 | rx \
+			| score \
+			| sh | shl \
+			| sh[1234] | sh[24]a | sh[24]ae[lb] | sh[23]e | she[lb] | sh[lb]e \
+			| sh[1234]e[lb] |  sh[12345][lb]e | sh[23]ele | sh64 | sh64le \
+			| sparc | sparc64 | sparc64b | sparc64v | sparc86x | sparclet \
+			| sparclite \
+			| sparcv8 | sparcv9 | sparcv9b | sparcv9v | sv1 | sx* \
+			| spu \
+			| tahoe \
+			| tic30 | tic4x | tic54x | tic55x | tic6x | tic80 \
+			| tron \
+			| ubicom32 \
+			| v70 | v850 | v850e | v850e1 | v850es | v850e2 | v850e2v3 \
+			| vax \
+			| visium \
+			| w65 \
+			| wasm32 | wasm64 \
+			| we32k \
+			| x86 | x86_64 | xc16x | xgate | xps100 \
+			| xstormy16 | xtensa* \
+			| ymp \
+			| z8k | z80)
+				;;
+
+			*)
+				echo Invalid configuration \`"$1"\': machine \`"$cpu-$vendor"\' not recognized 1>&2
+				exit 1
+				;;
+		esac
 		;;
 esac
 
 # Here we canonicalize certain aliases for manufacturers.
-case $basic_machine in
-	*-digital*)
-		basic_machine=`echo "$basic_machine" | sed 's/digital.*/dec/'`
+case $vendor in
+	digital*)
+		vendor=dec
 		;;
-	*-commodore*)
-		basic_machine=`echo "$basic_machine" | sed 's/commodore.*/cbm/'`
+	commodore*)
+		vendor=cbm
 		;;
 	*)
 		;;
@@ -1345,199 +1278,245 @@ esac
 
 # Decode manufacturer-specific aliases for certain operating systems.
 
-if [ x"$os" != x"" ]
+if [ x$os != x ]
 then
 case $os in
 	# First match some system type aliases that might get confused
 	# with valid system types.
-	# -solaris* is a basic system type, with this one exception.
-	-auroraux)
-		os=-auroraux
+	# solaris* is a basic system type, with this one exception.
+	auroraux)
+		os=auroraux
 		;;
-	-solaris1 | -solaris1.*)
+	bluegene*)
+		os=cnk
+		;;
+	solaris1 | solaris1.*)
 		os=`echo $os | sed -e 's|solaris1|sunos4|'`
 		;;
-	-solaris)
-		os=-solaris2
+	solaris)
+		os=solaris2
 		;;
-	-unixware*)
-		os=-sysv4.2uw
+	unixware*)
+		os=sysv4.2uw
 		;;
-	-gnu/linux*)
+	gnu/linux*)
 		os=`echo $os | sed -e 's|gnu/linux|linux-gnu|'`
 		;;
 	# es1800 is here to avoid being matched by es* (a different OS)
-	-es1800*)
-		os=-ose
+	es1800*)
+		os=ose
+		;;
+	# Some version numbers need modification
+	chorusos*)
+		os=chorusos
+		;;
+	isc)
+		os=isc2.2
+		;;
+	sco6)
+		os=sco5v6
+		;;
+	sco5)
+		os=sco3.2v5
+		;;
+	sco4)
+		os=sco3.2v4
+		;;
+	sco3.2.[4-9]*)
+		os=`echo $os | sed -e 's/sco3.2./sco3.2v/'`
+		;;
+	sco3.2v[4-9]* | sco5v6*)
+		# Don't forget version if it is 3.2v4 or newer.
+		;;
+	scout)
+		# Don't match below
+		;;
+	sco*)
+		os=sco3.2v2
+		;;
+	psos*)
+		os=psos
 		;;
 	# Now accept the basic system types.
 	# The portable systems comes first.
 	# Each alternative MUST end in a * to match a version number.
-	# -sysv* is not here because it comes later, after sysvr4.
-	-gnu* | -bsd* | -mach* | -minix* | -genix* | -ultrix* | -irix* \
-	      | -*vms* | -sco* | -esix* | -isc* | -aix* | -cnk* | -sunos | -sunos[34]*\
-	      | -hpux* | -unos* | -osf* | -luna* | -dgux* | -auroraux* | -solaris* \
-	      | -sym* | -kopensolaris* | -plan9* \
-	      | -amigaos* | -amigados* | -msdos* | -newsos* | -unicos* | -aof* \
-	      | -aos* | -aros* | -cloudabi* | -sortix* \
-	      | -nindy* | -vxsim* | -vxworks* | -ebmon* | -hms* | -mvs* \
-	      | -clix* | -riscos* | -uniplus* | -iris* | -rtu* | -xenix* \
-	      | -hiux* | -knetbsd* | -mirbsd* | -netbsd* \
-	      | -bitrig* | -openbsd* | -solidbsd* | -libertybsd* \
-	      | -ekkobsd* | -kfreebsd* | -freebsd* | -riscix* | -lynxos* \
-	      | -bosx* | -nextstep* | -cxux* | -aout* | -elf* | -oabi* \
-	      | -ptx* | -coff* | -ecoff* | -winnt* | -domain* | -vsta* \
-	      | -udi* | -eabi* | -lites* | -ieee* | -go32* | -aux* | -hcos* \
-	      | -chorusos* | -chorusrdb* | -cegcc* | -glidix* \
-	      | -cygwin* | -msys* | -pe* | -psos* | -moss* | -proelf* | -rtems* \
-	      | -midipix* | -mingw32* | -mingw64* | -linux-gnu* | -linux-android* \
-	      | -linux-newlib* | -linux-musl* | -linux-uclibc* \
-	      | -uxpv* | -beos* | -mpeix* | -udk* | -moxiebox* \
-	      | -interix* | -uwin* | -mks* | -rhapsody* | -darwin* \
-	      | -openstep* | -oskit* | -conix* | -pw32* | -nonstopux* \
-	      | -storm-chaos* | -tops10* | -tenex* | -tops20* | -its* \
-	      | -os2* | -vos* | -palmos* | -uclinux* | -nucleus* \
-	      | -morphos* | -superux* | -rtmk* | -windiss* \
-	      | -powermax* | -dnix* | -nx6 | -nx7 | -sei* | -dragonfly* \
-	      | -skyos* | -haiku* | -rdos* | -toppers* | -drops* | -es* \
-	      | -onefs* | -tirtos* | -phoenix* | -fuchsia* | -redox* | -bme* \
-	      | -midnightbsd*)
+	# sysv* is not here because it comes later, after sysvr4.
+	gnu* | bsd* | mach* | minix* | genix* | ultrix* | irix* \
+	     | *vms* | esix* | aix* | cnk* | sunos | sunos[34]*\
+	     | hpux* | unos* | osf* | luna* | dgux* | auroraux* | solaris* \
+	     | sym* | kopensolaris* | plan9* \
+	     | amigaos* | amigados* | msdos* | newsos* | unicos* | aof* \
+	     | aos* | aros* | cloudabi* | sortix* \
+	     | nindy* | vxsim* | vxworks* | ebmon* | hms* | mvs* \
+	     | clix* | riscos* | uniplus* | iris* | isc* | rtu* | xenix* \
+	     | knetbsd* | mirbsd* | netbsd* \
+	     | bitrig* | openbsd* | solidbsd* | libertybsd* \
+	     | ekkobsd* | kfreebsd* | freebsd* | riscix* | lynxos* \
+	     | bosx* | nextstep* | cxux* | aout* | elf* | oabi* \
+	     | ptx* | coff* | ecoff* | winnt* | domain* | vsta* \
+	     | udi* | eabi* | lites* | ieee* | go32* | aux* | hcos* \
+	     | chorusrdb* | cegcc* | glidix* \
+	     | cygwin* | msys* | pe* | moss* | proelf* | rtems* \
+	     | midipix* | mingw32* | mingw64* | linux-gnu* | linux-android* \
+	     | linux-newlib* | linux-musl* | linux-uclibc* \
+	     | uxpv* | beos* | mpeix* | udk* | moxiebox* \
+	     | interix* | uwin* | mks* | rhapsody* | darwin* \
+	     | openstep* | oskit* | conix* | pw32* | nonstopux* \
+	     | storm-chaos* | tops10* | tenex* | tops20* | its* \
+	     | os2* | vos* | palmos* | uclinux* | nucleus* \
+	     | morphos* | superux* | rtmk* | windiss* \
+	     | powermax* | dnix* | nx6 | nx7 | sei* | dragonfly* \
+	     | skyos* | haiku* | rdos* | toppers* | drops* | es* \
+	     | onefs* | tirtos* | phoenix* | fuchsia* | redox* | bme* \
+	     | midnightbsd* | amdhsa* | unleashed* | emscripten* | wasi*)
 	# Remember, each alternative MUST END IN *, to match a version number.
 		;;
-	-qnx*)
-		case $basic_machine in
-		    x86-* | i*86-*)
+	qnx*)
+		case $cpu in
+		    x86 | i*86)
 			;;
 		    *)
-			os=-nto$os
+			os=nto-$os
 			;;
 		esac
 		;;
-	-nto-qnx*)
+	hiux*)
+		os=hiuxwe2
+		;;
+	nto-qnx*)
 		;;
-	-nto*)
+	nto*)
 		os=`echo $os | sed -e 's|nto|nto-qnx|'`
 		;;
-	-sim | -xray | -os68k* | -v88r* \
-	      | -windows* | -osx | -abug | -netware* | -os9* \
-	      | -macos* | -mpw* | -magic* | -mmixware* | -mon960* | -lnews*)
+	sim | xray | os68k* | v88r* \
+	    | windows* | osx | abug | netware* | os9* \
+	    | macos* | mpw* | magic* | mmixware* | mon960* | lnews*)
+		;;
+	linux-dietlibc)
+		os=linux-dietlibc
+		;;
+	linux*)
+		os=`echo $os | sed -e 's|linux|linux-gnu|'`
+		;;
+	lynx*178)
+		os=lynxos178
 		;;
-	-mac*)
+	lynx*5)
+		os=lynxos5
+		;;
+	lynx*)
+		os=lynxos
+		;;
+	mac*)
 		os=`echo "$os" | sed -e 's|mac|macos|'`
 		;;
-	-linux-dietlibc)
-		os=-linux-dietlibc
+	opened*)
+		os=openedition
 		;;
-	-linux*)
-		os=`echo $os | sed -e 's|linux|linux-gnu|'`
+	os400*)
+		os=os400
 		;;
-	-sunos5*)
+	sunos5*)
 		os=`echo "$os" | sed -e 's|sunos5|solaris2|'`
 		;;
-	-sunos6*)
+	sunos6*)
 		os=`echo "$os" | sed -e 's|sunos6|solaris3|'`
 		;;
-	-opened*)
-		os=-openedition
-		;;
-	-os400*)
-		os=-os400
+	wince*)
+		os=wince
 		;;
-	-wince*)
-		os=-wince
+	utek*)
+		os=bsd
 		;;
-	-utek*)
-		os=-bsd
+	dynix*)
+		os=bsd
 		;;
-	-dynix*)
-		os=-bsd
+	acis*)
+		os=aos
 		;;
-	-acis*)
-		os=-aos
+	atheos*)
+		os=atheos
 		;;
-	-atheos*)
-		os=-atheos
+	syllable*)
+		os=syllable
 		;;
-	-syllable*)
-		os=-syllable
-		;;
-	-386bsd)
-		os=-bsd
+	386bsd)
+		os=bsd
 		;;
-	-ctix* | -uts*)
-		os=-sysv
+	ctix* | uts*)
+		os=sysv
 		;;
-	-nova*)
-		os=-rtmk-nova
+	nova*)
+		os=rtmk-nova
 		;;
-	-ns2)
-		os=-nextstep2
+	ns2)
+		os=nextstep2
 		;;
-	-nsk*)
-		os=-nsk
+	nsk*)
+		os=nsk
 		;;
 	# Preserve the version number of sinix5.
-	-sinix5.*)
+	sinix5.*)
 		os=`echo $os | sed -e 's|sinix|sysv|'`
 		;;
-	-sinix*)
-		os=-sysv4
+	sinix*)
+		os=sysv4
 		;;
-	-tpf*)
-		os=-tpf
+	tpf*)
+		os=tpf
 		;;
-	-triton*)
-		os=-sysv3
+	triton*)
+		os=sysv3
 		;;
-	-oss*)
-		os=-sysv3
+	oss*)
+		os=sysv3
 		;;
-	-svr4*)
-		os=-sysv4
+	svr4*)
+		os=sysv4
 		;;
-	-svr3)
-		os=-sysv3
+	svr3)
+		os=sysv3
 		;;
-	-sysvr4)
-		os=-sysv4
+	sysvr4)
+		os=sysv4
 		;;
-	# This must come after -sysvr4.
-	-sysv*)
+	# This must come after sysvr4.
+	sysv*)
 		;;
-	-ose*)
-		os=-ose
+	ose*)
+		os=ose
 		;;
-	-*mint | -mint[0-9]* | -*MiNT | -MiNT[0-9]*)
-		os=-mint
+	*mint | mint[0-9]* | *MiNT | MiNT[0-9]*)
+		os=mint
 		;;
-	-zvmoe)
-		os=-zvmoe
+	zvmoe)
+		os=zvmoe
 		;;
-	-dicos*)
-		os=-dicos
+	dicos*)
+		os=dicos
 		;;
-	-pikeos*)
+	pikeos*)
 		# Until real need of OS specific support for
 		# particular features comes up, bare metal
 		# configurations are quite functional.
-		case $basic_machine in
+		case $cpu in
 		    arm*)
-			os=-eabi
+			os=eabi
 			;;
 		    *)
-			os=-elf
+			os=elf
 			;;
 		esac
 		;;
-	-nacl*)
+	nacl*)
 		;;
-	-ios)
+	ios)
 		;;
-	-none)
+	none)
+		;;
+	*-eabi)
 		;;
 	*)
-		# Get rid of the `-' at the beginning of $os.
-		os=`echo $os | sed 's/[^-]*-//'`
 		echo Invalid configuration \`"$1"\': system \`"$os"\' not recognized 1>&2
 		exit 1
 		;;
@@ -1554,254 +1533,261 @@ else
 # will signal an error saying that MANUFACTURER isn't an operating
 # system, and we'll never get to this point.
 
-case $basic_machine in
+case $cpu-$vendor in
 	score-*)
-		os=-elf
+		os=elf
 		;;
 	spu-*)
-		os=-elf
+		os=elf
 		;;
 	*-acorn)
-		os=-riscix1.2
+		os=riscix1.2
 		;;
 	arm*-rebel)
-		os=-linux
+		os=linux
 		;;
 	arm*-semi)
-		os=-aout
+		os=aout
 		;;
 	c4x-* | tic4x-*)
-		os=-coff
+		os=coff
 		;;
 	c8051-*)
-		os=-elf
+		os=elf
+		;;
+	clipper-intergraph)
+		os=clix
 		;;
 	hexagon-*)
-		os=-elf
+		os=elf
 		;;
 	tic54x-*)
-		os=-coff
+		os=coff
 		;;
 	tic55x-*)
-		os=-coff
+		os=coff
 		;;
 	tic6x-*)
-		os=-coff
+		os=coff
 		;;
 	# This must come before the *-dec entry.
 	pdp10-*)
-		os=-tops20
+		os=tops20
 		;;
 	pdp11-*)
-		os=-none
+		os=none
 		;;
 	*-dec | vax-*)
-		os=-ultrix4.2
+		os=ultrix4.2
 		;;
 	m68*-apollo)
-		os=-domain
+		os=domain
 		;;
 	i386-sun)
-		os=-sunos4.0.2
+		os=sunos4.0.2
 		;;
 	m68000-sun)
-		os=-sunos3
+		os=sunos3
 		;;
 	m68*-cisco)
-		os=-aout
+		os=aout
 		;;
 	mep-*)
-		os=-elf
+		os=elf
 		;;
 	mips*-cisco)
-		os=-elf
+		os=elf
 		;;
 	mips*-*)
-		os=-elf
+		os=elf
 		;;
 	or32-*)
-		os=-coff
+		os=coff
 		;;
 	*-tti)	# must be before sparc entry or we get the wrong os.
-		os=-sysv3
+		os=sysv3
 		;;
 	sparc-* | *-sun)
-		os=-sunos4.1.1
+		os=sunos4.1.1
 		;;
 	pru-*)
-		os=-elf
+		os=elf
 		;;
 	*-be)
-		os=-beos
+		os=beos
 		;;
 	*-ibm)
-		os=-aix
+		os=aix
 		;;
 	*-knuth)
-		os=-mmixware
+		os=mmixware
 		;;
 	*-wec)
-		os=-proelf
+		os=proelf
 		;;
 	*-winbond)
-		os=-proelf
+		os=proelf
 		;;
 	*-oki)
-		os=-proelf
+		os=proelf
 		;;
 	*-hp)
-		os=-hpux
+		os=hpux
 		;;
 	*-hitachi)
-		os=-hiux
+		os=hiux
 		;;
 	i860-* | *-att | *-ncr | *-altos | *-motorola | *-convergent)
-		os=-sysv
+		os=sysv
 		;;
 	*-cbm)
-		os=-amigaos
+		os=amigaos
 		;;
 	*-dg)
-		os=-dgux
+		os=dgux
 		;;
 	*-dolphin)
-		os=-sysv3
+		os=sysv3
 		;;
 	m68k-ccur)
-		os=-rtu
+		os=rtu
 		;;
 	m88k-omron*)
-		os=-luna
+		os=luna
 		;;
 	*-next)
-		os=-nextstep
+		os=nextstep
 		;;
 	*-sequent)
-		os=-ptx
+		os=ptx
 		;;
 	*-crds)
-		os=-unos
+		os=unos
 		;;
 	*-ns)
-		os=-genix
+		os=genix
 		;;
 	i370-*)
-		os=-mvs
+		os=mvs
 		;;
 	*-gould)
-		os=-sysv
+		os=sysv
 		;;
 	*-highlevel)
-		os=-bsd
+		os=bsd
 		;;
 	*-encore)
-		os=-bsd
+		os=bsd
 		;;
 	*-sgi)
-		os=-irix
+		os=irix
 		;;
 	*-siemens)
-		os=-sysv4
+		os=sysv4
 		;;
 	*-masscomp)
-		os=-rtu
+		os=rtu
 		;;
 	f30[01]-fujitsu | f700-fujitsu)
-		os=-uxpv
+		os=uxpv
 		;;
 	*-rom68k)
-		os=-coff
+		os=coff
 		;;
 	*-*bug)
-		os=-coff
+		os=coff
 		;;
 	*-apple)
-		os=-macos
+		os=macos
 		;;
 	*-atari*)
-		os=-mint
+		os=mint
+		;;
+	*-wrs)
+		os=vxworks
 		;;
 	*)
-		os=-none
+		os=none
 		;;
 esac
 fi
 
 # Here we handle the case where we know the os, and the CPU type, but not the
 # manufacturer.  We pick the logical manufacturer.
-vendor=unknown
-case $basic_machine in
-	*-unknown)
+case $vendor in
+	unknown)
 		case $os in
-			-riscix*)
+			riscix*)
 				vendor=acorn
 				;;
-			-sunos*)
+			sunos*)
 				vendor=sun
 				;;
-			-cnk*|-aix*)
+			cnk*|-aix*)
 				vendor=ibm
 				;;
-			-beos*)
+			beos*)
 				vendor=be
 				;;
-			-hpux*)
+			hpux*)
 				vendor=hp
 				;;
-			-mpeix*)
+			mpeix*)
 				vendor=hp
 				;;
-			-hiux*)
+			hiux*)
 				vendor=hitachi
 				;;
-			-unos*)
+			unos*)
 				vendor=crds
 				;;
-			-dgux*)
+			dgux*)
 				vendor=dg
 				;;
-			-luna*)
+			luna*)
 				vendor=omron
 				;;
-			-genix*)
+			genix*)
 				vendor=ns
 				;;
-			-mvs* | -opened*)
+			clix*)
+				vendor=intergraph
+				;;
+			mvs* | opened*)
 				vendor=ibm
 				;;
-			-os400*)
+			os400*)
 				vendor=ibm
 				;;
-			-ptx*)
+			ptx*)
 				vendor=sequent
 				;;
-			-tpf*)
+			tpf*)
 				vendor=ibm
 				;;
-			-vxsim* | -vxworks* | -windiss*)
+			vxsim* | vxworks* | windiss*)
 				vendor=wrs
 				;;
-			-aux*)
+			aux*)
 				vendor=apple
 				;;
-			-hms*)
+			hms*)
 				vendor=hitachi
 				;;
-			-mpw* | -macos*)
+			mpw* | macos*)
 				vendor=apple
 				;;
-			-*mint | -mint[0-9]* | -*MiNT | -MiNT[0-9]*)
+			*mint | mint[0-9]* | *MiNT | MiNT[0-9]*)
 				vendor=atari
 				;;
-			-vos*)
+			vos*)
 				vendor=stratus
 				;;
 		esac
-		basic_machine=`echo "$basic_machine" | sed "s/unknown/$vendor/"`
 		;;
 esac
 
-echo "$basic_machine$os"
+echo "$cpu-$vendor-$os"
 exit
 
 # Local variables:

dev-requirements.txt

@@ -9,6 +9,9 @@ pylint>=1.8.0 ; python_version < "3.4"
 pylint >= 2.3.1 ; python_version >= "3.4"  and platform_python_implementation == "CPython"
 astroid >= 2.2.5 ; python_version >= "3.4" and platform_python_implementation == "CPython"
 
+# We need this at runtime to use the libev-CFFI and libuv backends
+cffi >= 1.12.2 ; platform_python_implementation == 'CPython'
+
 # benchmarks use this
 perf >= 1.6.0
 

scripts/install.sh

@@ -10,98 +10,100 @@
 set -e
 set -x
 
-# This is to guard against multiple builds in parallel. The various installers will tend
-# to stomp all over eachother if you do this and they haven't previously successfully
-# succeeded. We use a lock file to block progress so only one install runs at a time.
-# This script should be pretty fast once files are cached, so the lost of concurrency
-# is not a major problem.
-# This should be using the lockfile command, but that's not available on the
-# containerized travis and we can't install it without sudo.
-# Is is unclear if this is actually useful. I was seeing behaviour that suggested
-# concurrent runs of the installer, but I can't seem to find any evidence of this lock
-# ever not being acquired.
 
+# Where installations go
 BASE=${BUILD_RUNTIMES-$PWD/.runtimes}
+PYENV=$BASE/pyenv
 echo $BASE
 mkdir -p $BASE
 
-LOCKFILE="$BASE/.install-lockfile"
-while true; do
-  if mkdir $LOCKFILE 2>/dev/null; then
-    echo "Successfully acquired installer."
-    break
-  else
-    echo "Failed to acquire lock. Is another installer running? Waiting a bit."
-  fi
-
-  sleep $[ ( $RANDOM % 10)  + 1 ].$[ ( $RANDOM % 100) ]s
-
-  if (( $(date '+%s') > 300 + $(stat --format=%X $LOCKFILE) )); then
-    echo "We've waited long enough"
-    rm -rf $LOCKFILE
-  fi
-done
-trap "rm -rf $LOCKFILE" EXIT
-
-
-PYENV=$BASE/pyenv
 
 if [ ! -d "$PYENV/.git" ]; then
-  rm -rf $PYENV
-  git clone https://github.com/pyenv/pyenv.git $BASE/pyenv
+    rm -rf $PYENV
+    git clone https://github.com/pyenv/pyenv.git $BASE/pyenv
 else
-  back=$PWD
-  cd $PYENV
-  git fetch || echo "Fetch failed to complete. Ignoring"
-  git reset --hard origin/master
-  cd $back
+    back=$PWD
+    cd $PYENV
+    # We don't fetch or reset after the initial creation;
+    # doing so causes the Travis cache to need re-packed and uploaded,
+    # and it's pretty large.
+    # So if we need to incorporate changes from pyenv, either temporarily
+    # turn this back on, or remove the Travis caches.
+    # git fetch || echo "Fetch failed to complete. Ignoring"
+    # git reset --hard origin/master
+    cd $back
 fi
 
 
 SNAKEPIT=$BASE/snakepit
 
+##
+# install(exact-version, bin-alias, dir-alias)
+#
+# Produce a python executable at $SNAKEPIT/bin-alias
+# having the exact version given as exact-version.
+#
+# Also produces a $SNAKEPIT/dir-alias/ pointing to the root
+# of the python install.
+##
 install () {
 
-  VERSION="$1"
-  ALIAS="$2"
-  mkdir -p $BASE/versions
-  SOURCE=$BASE/versions/$ALIAS
+    VERSION="$1"
+    ALIAS="$2"
+    DIR_ALIAS="$3"
+
+    DESTINATION=$BASE/versions/$VERSION
 
-  if [ ! -e "$SOURCE" ]; then
-    mkdir -p $SNAKEPIT
     mkdir -p $BASE/versions
-    $BASE/pyenv/plugins/python-build/bin/python-build $VERSION $SOURCE
-  fi
- rm -f $SNAKEPIT/$ALIAS
- mkdir -p $SNAKEPIT
- ls -l $SNAKEPIT
- ls -l $BASE/versions
- ls -l $SOURCE/
- ls -l $SOURCE/bin
- ln -s $SOURCE/bin/python $SNAKEPIT/$ALIAS
- $SOURCE/bin/python -m pip.__main__ install --upgrade pip wheel virtualenv
+    mkdir -p $SNAKEPIT
+
+    if [ ! -e "$DESTINATION" ]; then
+        mkdir -p $SNAKEPIT
+        mkdir -p $BASE/versions
+        $BASE/pyenv/plugins/python-build/bin/python-build $VERSION $DESTINATION
+    fi
+
+    # Travis CI doesn't take symlink changes (or creation!) into
+    # account on its caching, So we need to write an actual file if we
+    # actually changed something. For python version upgrades, this is
+    # usually handled automatically (obviously) because we installed
+    # python. But if we make changes *just* to symlink locations above,
+    # nothing happens. So for every symlink, write a file...with identical contents,
+    # so that we don't get *spurious* caching. (Travis doesn't check for mod times,
+    # just contents, so echoing each time doesn't cause it to re-cache.)
+
+    # Overwrite an existing alias
+    ln -sf $DESTINATION/bin/python $SNAKEPIT/$ALIAS
+    ln -sf $DESTINATION $SNAKEPIT/$DIR_ALIAS
+    echo $VERSION $ALIAS $DIR_ALIAS > $SNAKEPIT/$ALIAS.installed
+    $SNAKEPIT/$ALIAS --version
+    # Set the PATH to include the install's bin directory so pip
+    # doesn't nag.
+    PATH="$DESTINATION/bin/:$PATH" $SNAKEPIT/$ALIAS -m pip install --upgrade pip wheel virtualenv
+    ls -l $SNAKEPIT
+
 }
 
 
 for var in "$@"; do
-  case "${var}" in
-    2.7)
-      install 2.7.16 python2.7.16
-      ;;
-    3.5)
-      install 3.5.6 python3.5.6
-      ;;
-    3.6)
-      install 3.6.8 python3.6.8
-      ;;
-    3.7)
-      install 3.7.2 python3.7.2
-      ;;
-    pypy)
-      install pypy2.7-7.1.0 pypy710
-      ;;
-    pypy3)
-      install pypy3.6-7.1.0 pypy3.6_710
-      ;;
-  esac
+    case "${var}" in
+        2.7)
+            install 2.7.16 python2.7 2.7.d
+            ;;
+        3.5)
+            install 3.5.6 python3.5 3.5.d
+            ;;
+        3.6)
+            install 3.6.8 python3.6 3.6.d
+            ;;
+        3.7)
+            install 3.7.2 python3.7 3.7.d
+            ;;
+        pypy2.7)
+            install pypy2.7-7.1.0 pypy2.7 pypy2.7.d
+            ;;
+        pypy3.6)
+            install pypy3.6-7.1.0 pypy3.6 pypy3.6.d
+            ;;
+    esac
 done

setup.py

@@ -20,6 +20,7 @@ from _setuputils import PYPY, WIN
 from _setuputils import IGNORE_CFFI
 from _setuputils import SKIP_LIBUV
 from _setuputils import ConfiguringBuildExt
+from _setuputils import GeventClean
 from _setuputils import BuildFailed
 from _setuputils import cythonize1
 
@@ -325,6 +326,17 @@ def run_setup(ext_modules, run_make):
             # TODO: Generalize this.
             if LIBEV_CFFI_MODULE in cffi_modules and not WIN:
                 system(libev_configure_command)
+                # This changed to the libev directory, and ran configure .
+                # It then copied the generated config.h back to the previous
+                # directory, which happened to be beside us. In the embedded case,
+                # we're building in a different directory, so it copied it back to build
+                # directory, but here, we're building in the embedded directory, so
+                # it gave us useless files.
+                bad_file = None
+                for bad_file in ('config.h', 'configure-output.txt'):
+                    if os.path.exists(bad_file):
+                        os.remove(bad_file)
+                del bad_file
 
     setup(
         name='gevent',
@@ -347,7 +359,10 @@ def run_setup(ext_modules, run_make):
         packages=find_packages('src'),
         include_package_data=True,
         ext_modules=ext_modules,
-        cmdclass=dict(build_ext=ConfiguringBuildExt),
+        cmdclass={
+            'build_ext': ConfiguringBuildExt,
+            'clean': GeventClean,
+        },
         install_requires=install_requires,
         setup_requires=setup_requires,
         extras_require={

src/gevent/testing/testrunner.py

@@ -18,6 +18,7 @@ from .sysinfo import PY2
 from .sysinfo import RESOLVER_ARES
 from .sysinfo import RUN_LEAKCHECKS
 from . import six
+from . import travis
 
 # Import this while we're probably single-threaded/single-processed
 # to try to avoid issues with PyPy 5.10.
@@ -63,103 +64,172 @@ def _dir_from_package_name(package):
     return package_dir
 
 
-def run_many(tests,
-             configured_failing_tests=(),
-             failfast=False,
-             quiet=False,
-             configured_run_alone_tests=()):
-    # pylint:disable=too-many-locals,too-many-statements
-    global NWORKERS
-    start = time.time()
-    total = 0
-    failed = {}
-    passed = {}
-    total_cases = [0]
-    total_skipped = [0]
-
-    NWORKERS = min(len(tests), NWORKERS) or 1
+class ResultCollector(object):
 
-    pool = ThreadPool(NWORKERS)
-    util.BUFFER_OUTPUT = NWORKERS > 1 or quiet
+    def __init__(self):
+        self.total = 0
+        self.failed = {}
+        self.passed = {}
+        self.total_cases = 0
+        self.total_skipped = 0
 
-    def run_one(cmd, **kwargs):
-        kwargs['quiet'] = quiet
-        result = util.run(cmd, **kwargs)
-        if result:
-            if failfast:
-                sys.exit(1)
-            failed[result.name] = [cmd, kwargs]
+    def __iadd__(self, result):
+        if not result:
+            self.failed[result.name] = result #[cmd, kwargs]
         else:
-            passed[result.name] = True
-        total_cases[0] += result.run_count
-        total_skipped[0] += result.skipped_count
+            self.passed[result.name] = True
+        self.total_cases += result.run_count
+        self.total_skipped += result.skipped_count
+        return self
+
+
+class Runner(object):
+
+    TIME_WAIT_REAP = 0.1
+    TIME_WAIT_SPAWN = 0.05
+
+    def __init__(self,
+                 tests,
+                 configured_failing_tests=(),
+                 failfast=False,
+                 quiet=False,
+                 configured_run_alone_tests=()):
+        self._tests = tests
+        self._configured_failing_tests = configured_failing_tests
+        self._failfast = failfast
+        self._quiet = quiet
+        self._configured_run_alone_tests = configured_run_alone_tests
 
-    results = []
+        self.results = ResultCollector()
+        self.results.total = len(self._tests)
+        self._running_jobs = []
 
-    def reap():
-        for r in results[:]:
+        self._worker_count = min(len(tests), NWORKERS) or 1
+
+    def _run_one(self, cmd, **kwargs):
+        kwargs['quiet'] = self._quiet
+        result = util.run(cmd, **kwargs)
+        if not result and self._failfast:
+            sys.exit(1)
+        self.results += result
+
+    def _reap(self):
+        "Clean up the list of running jobs, returning how many are still outstanding."
+        for r in self._running_jobs[:]:
             if not r.ready():
                 continue
             if r.successful():
-                results.remove(r)
+                self._running_jobs.remove(r)
             else:
                 r.get()
                 sys.exit('Internal error in testrunner.py: %r' % (r, ))
-        return len(results)
+        return len(self._running_jobs)
 
-    def reap_all():
-        while reap() > 0:
-            time.sleep(0.1)
+    def _reap_all(self):
+        while self._reap() > 0:
+            time.sleep(self.TIME_WAIT_REAP)
 
-    def spawn(cmd, options):
+    def _spawn(self, pool, cmd, options):
         while True:
-            if reap() < NWORKERS:
-                r = pool.apply_async(run_one, (cmd, ), options or {})
-                results.append(r)
+            if self._reap() < self._worker_count:
+                job = pool.apply_async(self._run_one, (cmd, ), options or {})
+                self._running_jobs.append(job)
                 return
 
-            time.sleep(0.05)
+            time.sleep(self.TIME_WAIT_SPAWN)
 
-    run_alone = []
+    def __call__(self):
+        util.log("Running tests in parallel with concurrency %s" % (self._worker_count,),)
+        # Setting global state, in theory we can be used multiple times.
+        # This is fine as long as we are single threaded and call these
+        # sequentially.
+        util.BUFFER_OUTPUT = self._worker_count > 1 or self._quiet
 
-    try:
+        start = time.time()
+        try:
+            self._run_tests()
+        except KeyboardInterrupt:
+            self._report(time.time() - start, exit=False)
+            util.log('(partial results)\n')
+            raise
+        except:
+            traceback.print_exc()
+            raise
+
+        self._reap_all()
+        self._report(time.time() - start, exit=True)
+
+    def _run_tests(self):
+        "Runs the tests, produces no report."
+        run_alone = []
+
+        tests = self._tests
+        pool = ThreadPool(self._worker_count)
         try:
-            util.log("Running tests in parallel with concurrency %s" % (NWORKERS,),)
             for cmd, options in tests:
-                total += 1
                 options = options or {}
-                if matches(configured_run_alone_tests, cmd):
+                if matches(self._configured_run_alone_tests, cmd):
                     run_alone.append((cmd, options))
                 else:
-                    spawn(cmd, options)
+                    self._spawn(pool, cmd, options)
             pool.close()
             pool.join()
 
             if run_alone:
                 util.log("Running tests marked standalone")
                 for cmd, options in run_alone:
-                    run_one(cmd, **options)
-
+                    self._run_one(cmd, **options)
         except KeyboardInterrupt:
             try:
                 util.log('Waiting for currently running to finish...')
-                reap_all()
+                self._reap_all()
             except KeyboardInterrupt:
                 pool.terminate()
-                report(total, failed, passed, exit=False, took=time.time() - start,
-                       configured_failing_tests=configured_failing_tests,
-                       total_cases=total_cases[0], total_skipped=total_skipped[0])
-                util.log('(partial results)\n')
                 raise
-    except:
-        traceback.print_exc()
-        pool.terminate()
-        raise
+        except:
+            pool.terminate()
+            raise
+
+    def _report(self, elapsed_time, exit=False):
+        results = self.results
+        report(
+            results.total, results.failed, results.passed,
+            exit=exit,
+            took=elapsed_time,
+            configured_failing_tests=self._configured_failing_tests,
+            total_cases=results.total_cases,
+            total_skipped=results.total_skipped
+        )
+
+
+class TravisFoldingRunner(object):
+
+    def __init__(self, runner, travis_fold_msg):
+        self._runner = runner
+        self._travis_fold_msg = travis_fold_msg
+        self._travis_fold_name = str(int(time.time()))
+
+        # A zope-style acquisition proxy would be convenient here.
+        run_tests = runner._run_tests
+
+        def _run_tests():
+            self._begin_fold()
+            try:
+                run_tests()
+            finally:
+                self._end_fold()
 
-    reap_all()
-    report(total, failed, passed, took=time.time() - start,
-           configured_failing_tests=configured_failing_tests,
-           total_cases=total_cases[0], total_skipped=total_skipped[0])
+        runner._run_tests = _run_tests
+
+    def _begin_fold(self):
+        travis.fold_start(self._travis_fold_name,
+                          self._travis_fold_msg)
+
+    def _end_fold(self):
+        travis.fold_end(self._travis_fold_name)
+
+    def __call__(self):
+        return self._runner()
 
 def discover(
         tests=None, ignore_files=None,
@@ -296,7 +366,7 @@ def report(total, failed, passed, exit=True, took=None,
            configured_failing_tests=(),
            total_cases=0, total_skipped=0):
     # pylint:disable=redefined-builtin,too-many-branches,too-many-locals
-    runtimelog = util.runtimelog
+    runtimelog = util.runtimelog # XXX: Global state!
     if runtimelog:
         util.log('\nLongest-running tests:')
         runtimelog.sort()
@@ -422,6 +492,8 @@ def main():
     parser.add_argument("--verbose", action="store_false", dest='quiet')
     parser.add_argument("--debug", action="store_true", default=False)
     parser.add_argument("--package", default="gevent.tests")
+    parser.add_argument("--travis-fold", metavar="MSG",
+                        help="Emit Travis CI log fold markers around the output.")
     parser.add_argument('tests', nargs='*')
     options = parser.parse_args()
 
@@ -439,19 +511,22 @@ def main():
 
     coverage = False
     if options.coverage or os.environ.get("GEVENTTEST_COVERAGE"):
-        coverage = True
-        os.environ['COVERAGE_PROCESS_START'] = os.path.abspath(".coveragerc")
-        if PYPY:
-            os.environ['COVERAGE_PROCESS_START'] = os.path.abspath(".coveragerc-pypy")
-        this_dir = os.path.dirname(__file__)
-        site_dir = os.path.join(this_dir, 'coveragesite')
-        site_dir = os.path.abspath(site_dir)
-        os.environ['PYTHONPATH'] = site_dir + os.pathsep + os.environ.get("PYTHONPATH", "")
-        # We change directory often, use an absolute path to keep all the
-        # coverage files (which will have distinct suffixes because of parallel=true in .coveragerc
-        # in this directory; makes them easier to combine and use with coverage report)
-        os.environ['COVERAGE_FILE'] = os.path.abspath(".") + os.sep + ".coverage"
-        print("Enabling coverage to", os.environ['COVERAGE_FILE'], "with site", site_dir)
+        if PYPY and RUNNING_ON_CI:
+            print("Ignoring coverage option on PyPy on CI; slow")
+        else:
+            coverage = True
+            os.environ['COVERAGE_PROCESS_START'] = os.path.abspath(".coveragerc")
+            if PYPY:
+                os.environ['COVERAGE_PROCESS_START'] = os.path.abspath(".coveragerc-pypy")
+            this_dir = os.path.dirname(__file__)
+            site_dir = os.path.join(this_dir, 'coveragesite')
+            site_dir = os.path.abspath(site_dir)
+            os.environ['PYTHONPATH'] = site_dir + os.pathsep + os.environ.get("PYTHONPATH", "")
+            # We change directory often, use an absolute path to keep all the
+            # coverage files (which will have distinct suffixes because of parallel=true in .coveragerc
+            # in this directory; makes them easier to combine and use with coverage report)
+            os.environ['COVERAGE_FILE'] = os.path.abspath(".") + os.sep + ".coverage"
+            print("Enabling coverage to", os.environ['COVERAGE_FILE'], "with site", site_dir)
 
     _setup_environ(debug=options.debug)
 
@@ -490,7 +565,7 @@ def main():
             # Put this directory on the path so relative imports work.
             package_dir = _dir_from_package_name(options.package)
             os.environ['PYTHONPATH'] = os.environ.get('PYTHONPATH', "") + os.pathsep + package_dir
-        run_many(
+        runner = Runner(
             tests,
             configured_failing_tests=FAILING_TESTS,
             failfast=options.failfast,
@@ -498,6 +573,11 @@ def main():
             configured_run_alone_tests=RUN_ALONE,
         )
 
+        if options.travis_fold:
+            runner = TravisFoldingRunner(runner, options.travis_fold)
+
+        runner()
+
 
 if __name__ == '__main__':
     main()

scripts/travis.py → src/gevent/testing/travis.py

@@ -8,15 +8,15 @@ from __future__ import print_function
 
 import sys
 
+
 commands = {}
 
 def command(func):
-    commands[func.__name__] = func
+    commands[func.__name__] = lambda: func(*sys.argv[2:])
+    return func
 
 @command
-def fold_start():
-    name = sys.argv[2]
-    msg = sys.argv[3]
+def fold_start(name, msg):
     sys.stdout.write('travis_fold:start:')
     sys.stdout.write(name)
     sys.stdout.write(chr(0o33))
@@ -26,8 +26,7 @@ def fold_start():
     sys.stdout.write('[33;0m\n')
 
 @command
-def fold_end():
-    name = sys.argv[2]
+def fold_end(name):
     sys.stdout.write("\ntravis_fold:end:")
     sys.stdout.write(name)
     sys.stdout.write("\r\n")

src/gevent/testing/util.py

@@ -257,10 +257,24 @@ def start(command, quiet=False, **kwargs):
 
 
 class RunResult(object):
+    """
+    The results of running an external command.
+
+    If the command was successful, this has a boolean
+    value of True; otherwise, a boolean value of false.
+
+    The integer value of this object is the command's exit code.
+
+    """
 
-    def __init__(self, code,
+    def __init__(self,
+                 command,
+                 run_kwargs,
+                 code,
                  output=None, name=None,
                  run_count=0, skipped_count=0):
+        self.command = command
+        self.run_kwargs = run_kwargs
         self.code = code
         self.output = output
         self.name = name
@@ -269,7 +283,7 @@ class RunResult(object):
 
 
     def __bool__(self):
-        return bool(self.code)
+        return not bool(self.code)
 
     __nonzero__ = __bool__
 
@@ -322,6 +336,7 @@ def _find_test_status(took, out):
 
 
 def run(command, **kwargs): # pylint:disable=too-many-locals
+    "Execute *command*, returning a `RunResult`"
     buffer_output = kwargs.pop('buffer_output', BUFFER_OUTPUT)
     quiet = kwargs.pop('quiet', QUIET)
     verbose = not quiet
@@ -361,7 +376,8 @@ def run(command, **kwargs): # pylint:disable=too-many-locals
             log('- %s %s', name, status)
     if took >= MIN_RUNTIME:
         runtimelog.append((-took, name))
-    return RunResult(result, out, name, run_count, skipped_count)
+    return RunResult(command, kwargs, result,
+                     output=out, name=name, run_count=run_count, skipped_count=skipped_count)
 
 
 class NoSetupPyFound(Exception):

src/gevent/tests/known_failures.py

@@ -21,10 +21,6 @@ from gevent.testing.sysinfo import LIBUV
 IGNORED_TESTS = []
 
 FAILING_TESTS = [
-
-    # Sometimes fails with AssertionError: ...\nIOError: close() called during concurrent operation on the same file object.\n'
-    # Sometimes it contains "\nUnhandled exception in thread started by \nsys.excepthook is missing\nlost sys.stderr\n"
-    "FLAKY test__subprocess_interrupted.py",
     # test__issue6 (see comments in test file) is really flaky on both Travis and Appveyor;
     # on Travis we could just run the test again (but that gets old fast), but on appveyor
     # we don't have that option without a new commit---and sometimes we really need a build
@@ -233,6 +229,7 @@ if PYPY:
         'FLAKY test__backdoor.py',
     ]
 
+
     if RESOLVER_NOT_SYSTEM:
 
         FAILING_TESTS += [
@@ -243,7 +240,10 @@ if PYPY:
             # AssertionError: Lists differ:
             #     (10, 1, 6, '', ('2607:f8b0:4004:810::200e', 80, 0L, 0L))
             #     (10, 1, 6, '', ('2607:f8b0:4004:805::200e', 80, 0, 0))
-            'test__socket_dns.py',
+            #
+            # Somehow it seems most of these are fixed with PyPy3.6-7 under dnspython,
+            # (once we commented out TestHostname)?
+            'FLAKY test__socket_dns.py',
         ]
 
     if LIBUV:
@@ -260,10 +260,14 @@ if PYPY:
             # This fails to get the correct results, sometimes. I can't reproduce locally
             'FLAKY test__example_udp_server.py',
             'FLAKY test__example_udp_client.py',
+        ]
 
+        IGNORED_TESTS += [
             # PyPy 7.0 and 7.1 on Travis with Ubunto Xenial 16.04
             # can't allocate SSL Context objects, either in Python 2.7
             # or 3.6. There must be some library incompatibility.
+            # No point even running them.
+            # XXX: Remember to turn this back on.
             'test_ssl.py',
         ]
 

src/gevent/tests/test___monkey_patching.py

@@ -106,7 +106,7 @@ def TESTRUNNER(tests=None):
 
 def main():
     from gevent.testing import testrunner
-    return testrunner.run_many(list(TESTRUNNER(sys.argv[1:])))
+    return testrunner.Runner(list(TESTRUNNER(sys.argv[1:])))()
 
 
 if __name__ == '__main__':

src/gevent/tests/test__examples.py

@@ -48,13 +48,13 @@ class _AbstractTestMixin(util.ExampleMixin):
     def test_runs(self):
         start = time.time()
         min_time, max_time = self.time_range
-        if util.run([sys.executable, '-u', self.filename],
-                    timeout=max_time,
-                    cwd=self.cwd,
-                    quiet=True,
-                    buffer_output=True,
-                    nested=True,
-                    setenv={'GEVENT_DEBUG': 'error'}):
+        if not util.run([sys.executable, '-u', self.filename],
+                        timeout=max_time,
+                        cwd=self.cwd,
+                        quiet=True,
+                        buffer_output=True,
+                        nested=True,
+                        setenv={'GEVENT_DEBUG': 'error'}):
             self.fail("Failed example: " + self.filename)
         else:
             took = time.time() - start

src/gevent/tests/test__socket_dns.py

@@ -380,7 +380,12 @@ add(TestTypeError, 25)
 class TestHostname(TestCase):
     pass
 
-add(TestHostname, socket.gethostname)
+add(
+    TestHostname,
+    socket.gethostname,
+    skip=greentest.RUNNING_ON_TRAVIS and greentest.RESOLVER_DNSPYTHON,
+    skip_reason="Sometimes get a different result for getaddrinfo",
+)
 
 
 class TestLocalhost(TestCase):

src/gevent/tests/test__subprocess_interrupted.py

@@ -14,7 +14,21 @@ else:
         out, err = subprocess.Popen([sys.executable, '-W', 'ignore',
                                      __file__, 'runtestcase'],
                                     stderr=subprocess.PIPE).communicate()
-        if b'refs' in err: # Something to do with debug mode python builds?
+        # We've seen three. unexpected forms out output.
+        #
+        # The first involves 'refs'; I don't remember what that was
+        # about, but I think it had to do with debug builds of Python.
+        #
+        # The second is the classic "Unhandled exception in thread
+        # started by \nsys.excepthook is missing\nlost sys.stderr".
+        # This is a race condition between closing sys.stderr and
+        # writing buffered data to a pipe that hasn't been read. We
+        # only see this using GEVENT_FILE=thread (which makes sense).
+        #
+        # The third is similar to the second: "AssertionError:
+        # ...\nIOError: close() called during concurrent operation on
+        # the same file object.\n"
+        if b'refs' in err or b'sys.excepthook' in err or b'concurrent' in err:
             assert err.startswith(b'bye'), repr(err) # pragma: no cover
         else:
             assert err.strip() == b'bye', repr(err)

tox.ini

@@ -14,13 +14,6 @@ whitelist_externals =
 commands =
     python -m gevent.tests
 
-
-[testenv:py27-full]
-basepython = python2.7
-commands =
-    make allbackendtest
-
-
 [testenv:lint]
 skip_install = true
 skipsdist = true
@@ -49,4 +42,4 @@ setenv =
 basepython =
     python2.7
 commands =
-    make leaktest
+    GEVENTTEST_LEAKCHECK=1 python -mgevent.tests --config known_failures.py --quiet --ignore tests_that_dont_do_leakchecks.txt