Band-aid for macOS: Some platforms seem to ignore unknown protocols.
Signed-off-by: Christian Heimes <christian@python.org>

Allow os.PathLike args in subprocess APIs.

Fix two (in my opinion) spurious failure conditions in the lib2to3.tests.test_parser.TestParserIdempotency test_parser test.

    Use the same encoding found in the initial file to write a temp file for a diff. This retains the BOM if the encoding was initially utf-8-sig.

    If the file cannot be parsed using the normal grammar, try again with no print statement which should succeed for valid files using future print_function

For case (1), the driver was correctly handling a BOM in a utf-8 file, but then the test was not writing a comparison file using 'utf-8-sig' to diff against, so the BOM got removed. I don't think that is the fault of the parser, and lib2to3 will retain the BOM.

For case (2), lib2to3 pre-detects the use of from __future__ import print_function or allows the user to force this interpretation with a -p flag, and then selects a different grammar with the print statement removed. That makes the test cases unfair to this test as the driver itself doesn't know which grammar to use. As a minimal fix, the test will try using a grammar with the print statement, and if that fails fall back on a grammar without it. A more thorough handling of the idempotency test would to be to parse all files using both grammars and ignore if one of the two failed but otherwise check both. I didn't think this was necessary but can change.

Add "capture_output=True" option to subprocess.run, this is equivalent to
setting stdout=PIPE, stderr=PIPE but is much more readable.

See https://bugs.python.org/issue32441 for where this was introduced.

This allows the compression level to be specified when writing zipfiles
(for the entire file *and* overridden on a per-file basis).

Contributed by Bo Bayles

Do not allow receiving a SIGINT to cause the subprocess module to trigger an
immediate SIGKILL of the child process.  SIGINT is normally sent to all child
processes by the OS at the same time already as was the established normal
behavior in 2.7 and 3.2.  This behavior change was introduced during the fix to https://bugs.python.org/issue12494 and is generally surprising to command line
tool users who expect other tools launched in child processes to get their own
SIGINT and do their own cleanup.

In Python 3.3-3.6 subprocess.call and subprocess.run would immediately
SIGKILL the child process upon receiving a SIGINT (which raises a
KeyboardInterrupt).  We now give the child a small amount of time to
exit gracefully before resorting to a SIGKILL.

This is also the case for subprocess.Popen.__exit__ which would
previously block indefinitely waiting for the child to die.  This was
hidden from many users by virtue of subprocess.call and subprocess.run
sending the signal immediately.

Behavior change: subprocess.Popen.__exit__ will not block indefinitely
when the exiting exception is a KeyboardInterrupt.  This is done for
user friendliness as people expect their ^C to actually happen.  This
could cause occasional orphaned Popen objects when not using `call` or
`run` with a child process that hasn't exited.

Refactoring involved: The Popen.wait method deals with the
KeyboardInterrupt second chance, existing platform specific internals
have been renamed to _wait().
Also fixes comment typos.

The module is primarily intended for internal use in the test suite.  Building the module under Windows will come in a follow-up PR.

Fix socket(fileno=fd) by auto-detecting the socket's family, type,
and proto from the file descriptor. The auto-detection can be overruled
by passing in family, type, and proto explicitly.

Without the fix, all socket except for TCP/IP over IPv4 are basically broken:

>>> s = socket.create_connection(('www.python.org', 443))
>>> s
<socket.socket fd=3, family=AddressFamily.AF_INET6, type=SocketKind.SOCK_STREAM, proto=6, laddr=('2003:58:bc4a:3b00:56ee:75ff:fe47:ca7b', 59730, 0, 0), raddr=('2a04:4e42:1b::223', 443, 0, 0)>
>>> socket.socket(fileno=s.fileno())
<socket.socket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('2003:58:bc4a:3b00::%2550471192', 59730, 0, 2550471192), raddr=('2a04:4e42:1b:0:700c:e70b:ff7f:0%2550471192', 443, 0, 2550471192)>
Signed-off-by: Christian Heimes <christian@python.org>

* Fix memory leaks and error handling in posix spawn
* Improve error handling when destroying the file_actions object
* Py_DECREF the result of PySequence_Fast on error
* Handle uninitialized pid
* Use OSError if file actions fails to initialize
* Move _file_actions to outer scope to avoid undefined behaviour
* Remove HAVE_POSIX_SPAWN define in Modules/posixmodule.c
* Unshadow exception and clean error message

When the C imp module became _imp in 6f44d66b, the initialization function should have been renamed from PyInit_imp to PyInit__imp.

The commit removes one unnecessary "if" clause in genobject.c.  That "if" clause was masking un-awaited coroutines warnings just to make writing unittests more convenient.

Correctly handle addresses whose username is an empty quoted string.

Add test cases for IDNA 2003 and 2008 host names. IDNA 2003
internationalized host names are working since bpo-31399 has landed. IDNA
2008 deviations are still broken and will be fixed in another patch.
Signed-off-by: Christian Heimes <christian@python.org>

Until now Python used a hard coded white list of default TLS cipher
suites. The old approach has multiple downsides. OpenSSL's default
selection was completely overruled. Python did neither benefit from new
cipher suites (ChaCha20, TLS 1.3 suites) nor blacklisted cipher suites.
For example we used to re-enable 3DES.

Python now defaults to OpenSSL DEFAULT cipher suite selection and black
lists all unwanted ciphers. Downstream vendors can override the default
cipher list with --with-ssl-default-suites.
Signed-off-by: Christian Heimes <christian@python.org>

This test is unstable and currently prevents to make any new change
since the test always fails on Travis CI.

Skip the test to get more time to fix it.

socketserver.ForkingMixIn.server_close() and
socketserver.ThreadingMixIn.server_close() now waits until all child
processes and non-daemonic threads complete.

The GIL is no longer created "on demand" to fix a race condition when
PyGILState_Ensure() is called in a non-Python thread.

To mitigate the situation when the buildbot is under load
and is unable to send/receive data fast enough:

* reduce the size of the payload
* set a generous timeout for socket ops

Add os.posix_spawn to wrap the low level POSIX API of the same name.

Contributed by Pablo Galindo.

Document the error return of PyLong_As* APIs.

A frequent Python C API usage error is neglecting to check the return
value and/or PyErr_Occurred().