(cherry picked from commit 8adc73c2)

(cherry picked from commit e503ca52)

The ssl and hashlib modules now call OPENSSL_add_all_algorithms_noconf() on
OpenSSL < 1.1.0. The function detects CPU features and enables optimizations
on some CPU architectures such as POWER8. Patch is based on research from
Gustavo Serra Scalet.
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit c941e623)

Include sys/sysmacros.h for major(), minor(), and makedev(). GNU C libray
plans to remove the functions from sys/types.h.

Signed-off-by: Christian Heimes <christian@python.org>.
(cherry picked from commit 75b96186)

* Change NPN detection:

Version breakdown, support disabled (pre-patch/post-patch):
- pre-1.0.1: OPENSSL_NPN_NEGOTIATED will not be defined -> False/False
- 1.0.1 and 1.0.2: OPENSSL_NPN_NEGOTIATED will not be defined ->
False/False
- 1.1.0+: OPENSSL_NPN_NEGOTIATED will be defined and
OPENSSL_NO_NEXTPROTONEG will be defined -> True/False

Version breakdown support enabled (pre-patch/post-patch):
- pre-1.0.1: OPENSSL_NPN_NEGOTIATED will not be defined -> False/False
- 1.0.1 and 1.0.2: OPENSSL_NPN_NEGOTIATED will be defined and
OPENSSL_NO_NEXTPROTONEG will not be defined -> True/True
- 1.1.0+: OPENSSL_NPN_NEGOTIATED will be defined and
OPENSSL_NO_NEXTPROTONEG will not be defined -> True/True

* Refine NPN guard:

- If NPN is disabled, but ALPN is available we need our callback
- Make clinic's ssl behave the same way

This created a working ssl module for me, with NPN disabled and ALPN
enabled for OpenSSL 1.1.0f.

Concerns to address:
The initial commit for NPN support into OpenSSL [1], had the
OPENSSL_NPN_* variables defined inside the OPENSSL_NO_NEXTPROTONEG
guard. The question is if that ever made it into a release.
This would need an ugly hack, something like:

	GH-if defined(OPENSSL_NO_NEXTPROTONEG) && \
		!defined(OPENSSL_NPN_NEGOTIATED)
	GH-	define OPENSSL_NPN_UNSUPPORTED 0
	GH-	define OPENSSL_NPN_NEGOTIATED 1
	GH-	define OPENSSL_NPN_NO_OVERLAP 2
	GH-endif

[1] https://github.com/openssl/openssl/commit/68b33cc5c7.
(cherry picked from commit b2d096bd)

The Windows build now depends on Python 3.6 to fetch externals, but it
will be downloaded via NuGet (which is downloaded via PowerShell) if it
is not available via `py -3.6`.  This means the only thing that must be
installed on a modern Windows box to do a full build of CPython with all
extensions is Visual Studio.

Cherry-picked from 51599e2b, parts of 40a23e88, parts of 68d663cf, d5cd21d7, and possibly others that I've missed.

Also:

* Rename db -> bsddb for disambiguity

* Update sqlite3 to 3.14.2.0 since it's the version we use on 3.x, and it's simpler to just use it than to also upload the old version to cpython-source-deps

* Add PCbuild/*.ilk to .gitignore

* bpo-25674: remove sha256.tbs-internet.com ssl test (#3297)
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit 002d6403)

* [2.7] bpo-25674: remove sha256.tbs-internet.com ssl test (GH-3297)

Signed-off-by: Christian Heimes <christian@python.org>.
(cherry picked from commit 002d6403)

(cherry picked from commit a6296d34)

[2.7] bpo-10746: Fix ctypes PEP 3118 type codes for c_long, c_bool, c_int (GH-31)

Ctypes currently produces wrong pep3118 type codes for several types.
E.g. memoryview(ctypes.c_long()).format gives "<l" on 64-bit platforms,
but it should be "<q" instead for sizeof(c_long) == 8

The problem is that the '<>' endian specification in the struct syntax
also turns on the "standard size" mode, which makes type characters have
a platform-independent meaning, which does not match with the codes used
internally in ctypes.  The struct module format syntax also does not
allow specifying native-size non-native-endian items.

This commit adds a converter function that maps the internal ctypes
codes to appropriate struct module standard-size codes in the pep3118
format strings. The tests are modified to check for this..
(cherry picked from commit 07f1658a)

[2.7] bpo-28261: Prevent raising SystemError where PyArg_ParseTuple is used to parse non-args. (#3213)

Use self.addCleanup(self.server.stop) to stop the HTTP server. Some
tests didn't stop the server like test_https().

* bpo-30871: Add test.pythoninfo (#3075)

* Add Lib/test/pythoninfo.py: script collecting various informations
  about Python to help debugging test failures.
* regrtest: remove sys.hash_info and sys.flags from header.
* Travis CI, Appveyor: run pythoninfo before tests
(cherry picked from commit b907abc8)

* bpo-30871: pythoninfo: add expat and _decimal (#3121)

* bpo-30871: pythoninfo: add expat and _decimal

* Remove _decimal.__version__

The string is hardcoded, not really interesting.

(cherry picked from commit f6ebd838)

* bpo-30871: Add "make pythoninfo" (#3120)

(cherry picked from commit a3a01a2f)

* bpo-30871: pythoninfo: more sys, os, time data (#3130)

* bpo-30871: pythoninfo: more sys, os, time data

PythonInfo now converts types other than intger to string by default.

* fix typo

(cherry picked from commit ad7eaed5)

* bpo-31231: Fix pythoninfo in Travis config (#3134)

bpo-31231, bpo-30871: Replace "./python -m test.pythoninfo" with
"make pythoninfo", since macOS uses ./python.exe.
(cherry picked from commit 92b1f901)

(cherry picked from commit 29d007bb670b486788f73c2d742b0ad0b679ff13)

Check "tests" before its value is replaced.

Add the CPU count in the header.

* bpo-30947: Update libexpat from 2.2.1 to 2.2.3

* Add NEWS entry

* Add new loadlibrary.c

* expat_external.h: restore include "pyexpatns.h"

* PCbuild: add expat/loadlibrary.c

* Define XML_POOR_ENTROPY to compile expat

Python 2.7 backport: add expat/loadlibrary.c to PC/VS9.0/ project
files (_elementtree and pyexpat).

(cherry picked from commit 93d0cb58)

The `Show Source` was broken because of a change made in sphinx 1.5.1
In Sphinx 1.4.9, the sourcename was "index.txt".
In Sphinx 1.5.1+, it is now "index.rst.txt".

(cherry picked from commit b9ff4987)

* bpo-31221: patchcheck ignores external libraries (#3109)

Tools/scripts/patchcheck.py now ignores changes in directories which
are copies of external libraries:

* Modules/_ctypes/libffi_msvc/
* Modules/_ctypes/libffi_osx/
* Modules/_decimal/libmpdec/
* Modules/expat/
* Modules/zlib/

Drop also support for Mercurial, since CPython migrated to Git.

Exclude also libmpdec

patchcheck: exclude also libffi_osx and libffi_msvc
(cherry picked from commit 4a347ce4)

* Exclude also Modules/_ctypes/libffi on Python 2.7

* Remove _decimal/libmpdec, not in Python 2.7

Change the option for `Français` into `French` to be consistent with the other language selections that are already in English.
(cherry picked from commit b616b972)

Fix a warning about dangling processes in test_rapid_restart() of
_test_multiprocessing: join the process.
(cherry picked from commit 17657bb9)

OpenSSL 1.1.0 to 1.1.0e aborted the handshake when server and client
could not agree on a protocol using ALPN. OpenSSL 1.1.0f changed that.
The most recent version now behaves like OpenSSL 1.0.2 again. The ALPN
callback can pretend to not been set.

See https://github.com/openssl/openssl/pull/3158 for more details
Signed-off-by: Christian Heimes <christian@python.org>
(cherry picked from commit a5c1bab352671e043645163ca50c5211aa657acd)

* bpo-31159: fix language switch regex on unknown yet built languages. (#3039)

This fix a regex issue (a missing non-matching group around an 'or'
list) and the specific possible case where a translation is built but
not yet in known by the picker, but not explicitly listing possible
languages in the regex.
(cherry picked from commit 122081de)

* bpo-31149: Doc: Add Japanese to the language switcher. (#3028)

(cherry picked from commit c82b7f33)

(cherry picked from commit e8e7fba0b24582959feca9c31f2a72fc0251f83d)

test_subprocess now also calls reap_children() in tearDown(), not
only on setUp().
(cherry picked from commit cc42c121)

The current test_child_terminated_in_stopped_state() function test
creates a child process which calls ptrace(PTRACE_TRACEME, 0, 0) and
then crash (SIGSEGV). The problem is that calling os.waitpid() in the
parent process is not enough to close the process: the child process
remains alive and so the unit test leaks a child process in a
strange state. Closing the child process requires non-trivial code,
maybe platform specific.

Remove the functional test and replaces it with an unit test which
mocks os.waitpid() using a new _testcapi.W_STOPCODE() function to
test the WIFSTOPPED() path.
(cherry picked from commit 7b7c6dcf)

* bpo-31160: regrtest now reaps child processes (#3044)

Add a post_test_cleanup() function which currently only calls
support.reap_children().
(cherry picked from commit e3510d74)

* bpo-31160: test_tempfile: Fix reap_children() warning (#3056)

TestRandomNameSequence.test_process_awareness() now calls
os.waitpid() to avoid leaking a zombie process.
(cherry picked from commit 6c8c2943)

bpo-31133: Downgrade PCbuild/pcbuild.sln from format version 12.00 to
11.00 to support Visual Studio 2010 and newer, not only VS 2013 and
newer.

[2.7] bpo-31150: Wait for child process in test_forkinthread to avoid thread reaped warnings (#3042)

bpo-31135: Call the parent destroy() method even if the used
attribute doesn't exist.

The LabeledScale.destroy() method now also explicitly clears label
and scale attributes to help the garbage collector to destroy all
widgets.

* Doc: Indicate the language

* Renaming version_switcher to switchers (to add language_switcher).

* Adding language switch.

* Doc switchers: Enhance readability of regex parsing versions.

* Doc switchers: Desambiguate the need of a replace(/\/+$/g, '') by proper naming.

* Doc switchers: py3k can't reach js, it's redirected server-side by nginx.

* Doc switchers: Examples matching actual regexes.

* Doc switchers: Better fallback on unexisting translated version.

(cherry picked from commit dff9b5f9)

(cherry picked from commit c4c98660)

Change the shadowing naming, 'value' (Python-ast.c:3814), to 'val'
to prevent the variables from being misused.