* Add _PyPreConfig structure
* Move 'ignored' and 'use_environment' fields from _PyCoreConfig
  to _PyPreConfig
* Add a new "_PyPreConfig preconfig;" field to _PyCoreConfig

Fix two unlikely reference leaks in _hashopenssl. The leaks only occur in
out-of-memory cases. Thanks to Charalampos Stratakis.
Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue36179

For C++ extensions, distutils tries to replace the C compiler with the
C++ compiler, but it assumes that C compiler is the first element after
any environment variables set. On AIX, linking goes through ld_so_aix,
so it is the first element and the compiler is the next element. Thus
the replacement is faulty:

ld_so_aix gcc ... -> g++ gcc ...

Also, it assumed that self.compiler_cxx had only 1 element or that
there were the same number of elements as the linker has and in the
same order. This might not be the case, so instead concatenate
everything together.

Revert: bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617) (GH-12159)

* Revert "bpo-36097: Use only public C-API in the_xxsubinterpreters module (adding as necessary). (#12003)"

This reverts commit bcfa450f.

* Revert "bpo-33608: Simplify ceval's DISPATCH by hoisting eval_breaker ahead of time. (gh-12062)"

This reverts commit bda918bf.

* Revert "bpo-33608: Use _Py_AddPendingCall() in _PyCrossInterpreterData_Release(). (gh-12024)"

This reverts commit b05b711a.

* Revert "bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). (GH-11617)"

This reverts commit ef4ac967.

Use UTF-8 as the system encoding on VxWorks.

The main reason are:

1. The locale is frequently misconfigured.
2. Missing some functions to deal with locale in VxWorks C library.

* bpo-20906: Corrections & revisions to Unicode HOWTO

* bpo-34484: don't describe range as a Private Use Area

Co-authored-by: Maxwell <maxwellpxt@gmail.com>
Co-authored-by: Stéphane Wirtel <stephane@wirtel.be>





https://bugs.python.org/issue35899

The 3.8 docs claim that `os.posix_spawn` was introduced in 3.7, but it wasn't; it will be introduced in 3.8.

Bug introduced by 848037c1.

The previous code hardcoded `SEEK_SET`, etc. While it's very unlikely
that these values will change, it's best to use the definitions to avoid
there being mismatches in behavior with the code in the future.
Signed-off-by: Enji Cooper <yaneurabeya@gmail.com>

It is changed from 16KiB to 64KiB.  The previous default value
is used since 1990.

coreutils chose 128 KiB as minimum buffer size for block device I/O.

But shutil.copyfileobj() can be used for non block devices.
So I choose more conservative value.

As my quick benchmark, performance difference between 64KiB and
128 KiB is up to ~5%.  On the other hand, performance difference
between 32 KiB and 64 KiB can be more than 10% when file is fully
buffered.

This is why 64 KiB is rational value.

Pgen is the oldest piece of technology in the CPython repository, building it requires various #if[n]def PGEN hacks in other parts of the code and it also depends more and more on CPython internals. This commit removes the old pgen C code and replaces it for a new version implemented in pure Python. This is a modified and adapted version of lib2to3/pgen2 that can generate grammar files compatibles with the current parser.

This commit also eliminates all the #ifdef and code branches related to pgen, simplifying the code and making it more maintainable. The regen-grammar step now uses $(PYTHON_FOR_REGEN) that can be any version of the interpreter, so the new pgen code maintains compatibility with older versions of the interpreter (this also allows regenerating the grammar with the current CI solution that uses Python3.5). The new pgen Python module also makes use of the Grammar/Tokens file that holds the token specification, so is always kept in sync and avoids having to maintain duplicate token definitions.

 Patch by Kevin Walzer.

https://bugs.python.org/issue36043

This includes fixes to various _Py_atomic_* macros.

Add TEST_EXTENSIONS constant to setup.py to allow to not build test
extensions like _testcapi.

Changes:

* Add add_ldflags_cppflags() subfunction
* Rename add_compiler_directories() to configure_compiler().
* Remove unused COMPILED_WITH_PYDEBUG constant.
* Use self.add() rather than accessing directly self.extensions.
* Remove module_enabled() function: check differently if curses
  extension is built or not.

Ensure custom formatwarning function can receive line as positional argument.
Co-Authored-By: Tashrif Billah <tashrifbillah@gmail.com>

* Add _PyArgv_Decode() function
* Move _Py_ClearFileSystemEncoding() and _Py_SetFileSystemEncoding()
  to preconfig.c.

* Move set_compiler_flags() calls and concurrent.future hack from
  module top-level to main()
* Remove unused variables 'macros' and 'libraries' from
  detect_multiprocessing().
* Move SUMMARY and CLASSIFIERS constants at the top, move
  set_compiler_flags() function below these constants.
* Add some empty new lines to respect PEP 8.

* Split PyBuildExt.detect_modules() huge function into subfunctions.
* Move curses, hashlib and some other code to reorganize the code.
* detect_tkinter() now returns False if the extension is missing.
* Add PyBuildExt.config_h_vars attribute

* Add _PyCoreConfig_ReadFromArgv() function which parses command line
  options: move code from main.c to coreconfig.c.
* Add _PyCoreConfig_Write() to write the new configuration: coerce
  the LC_CTYPE locale, set Py_xxx global configuration variables,
  etc.
* _PyCoreConfig_ReadFromArgv() now only changes the LC_CTYPE locale
  temporarily. _PyCoreConfig_Write() becomes responsible to set the
  LC_CTYPE locale.
* Add _Py_SetArgcArgv() and _Py_ClearArgcArgv() functions
* Rename many "pymain_xxx()" functions
* Add "const" to some function parameters
* Reorganize main.c to declare functions in the order in which they
  are called.

* Add PyBuildExt.srcdir atribute in setup.py: the source directory is
  now always absolute.
* Add PyBuildExt.inc_dirs and PyBuildExt.lib_dirs attributes:
  replace 'inc_dirs' and 'lib_dirs' local variables of
  detect_modules().
* Replace "from distutils.errors import *"
  with "from distutils.errors import CCompilerError, DistutilsError"
  to be able to use static analyzers like pyflakes
* Reorder imports.

* Add PyBuildExt.add() which adds the extension directly to
  self.extensions, rather than using a temporary 'exts' local
  variable in detect_modules() and then add 'exts' to self.extensions
* Convert 'missing' local variable from detect_modules()
  into PyBuildExt.missing attribute
* _detect_openssl(), _decimal_ext() and _detect_nis() now call
  directly self.add(), rather than returning an extension
  (or None if not found).
* Rename _decimal_ext() to _detect_decimal() for consistency with
  other methods.

Fix setup.py on macOS: only add /usr/include/ffi to include
directories of _ctypes, not for all extensions.

* Move fields from _PyMain to _PyCoreConfig:

  * skip_first_line
  * run_command
  * run_module
  * run_filename

* Replace _PyMain.stdin_is_interactive with a new
  stdin_is_interactive(config) function
* Rename _PyMain to _PyArgv. Add "const _PyArgv *args" field
  to _PyCmdline.

Add a new _Py_INIT_EXIT() macro to be able to exit Python with an
exitcode using _PyInitError API. Rewrite function calls by
pymain_main() to use _PyInitError.

Changes:

* Remove _PyMain.err and _PyMain.status field
* Add _Py_INIT_EXIT() macro and _PyInitError.exitcode field.
* Rename _Py_FatalInitError() to _Py_ExitInitError().

Remove colorizer.ColorDelegator.close_when_done and the corresponding argument of .close().  In IDLE, both have always been None or False since 2007.

https://bugs.python.org/issue36018

The whole coreconfig.h header is now excluded from Py_LIMITED_API.
Move functions definitions into a new internal pycore_coreconfig.h
header.

* Move Include/coreconfig.h to Include/cpython/coreconfig.h
* coreconfig.h header is now excluded from Py_LIMITED_API
* Move functions to pycore_coreconfig.h

Use locale.getpreferredencoding() rather than locale.getlocale() to
get the locale encoding. With some locales, locale.getlocale()
returns the wrong encoding.

For example, on Fedora 29, locale.getlocale() returns ISO-8859-1
encoding for the "en_IN" locale, whereas
locale.getpreferredencoding() reports the correct encoding: UTF-8.

https://bugs.python.org/issue36018