Issue #14018: Backport OS X installer updates from 3.3.

Mac/BuildScript/README.txt

@@ -8,70 +8,83 @@ $DESTROOT, massages that installation to remove .pyc files and such, creates
 an Installer package from the installation plus other files in ``resources`` 
 and ``scripts`` and placed that on a ``.dmg`` disk image.
 
-As of Python 2.7.x and 3.2, PSF practice is to build two installer variants
-for each release:
+For Python 2.7.x and 3.2.x, PSF practice is to build two installer variants
+for each release.
 
 1.  32-bit-only, i386 and PPC universal, capable on running on all machines
-    supported by Mac OS X 10.3.9 through (at least) 10.6::
+    supported by Mac OS X 10.3.9 through (at least) 10.8::
 
-        python build-installer.py \
+        /usr/bin/python build-installer.py \
             --sdk-path=/Developer/SDKs/MacOSX10.4u.sdk \
             --universal-archs=32-bit \
             --dep-target=10.3 
-            # These are the current default options
 
     - builds the following third-party libraries
 
         * Bzip2
-        * Zlib 1.2.3
-        * GNU Readline (GPL)
-        * SQLite 3
         * NCurses
+        * GNU Readline (GPL)
+        * SQLite 3.7.13
+        * Zlib 1.2.3
         * Oracle Sleepycat DB 4.8 (Python 2.x only)
 
     - requires ActiveState ``Tcl/Tk 8.4`` (currently 8.4.19) to be installed for building
 
-    - current target build environment:
+    - recommended build environment:
         
         * Mac OS X 10.5.8 PPC or Intel
-        * Xcode 3.1.4 (or later)
+        * Xcode 3.1.4
         * ``MacOSX10.4u`` SDK (later SDKs do not support PPC G3 processors)
         * ``MACOSX_DEPLOYMENT_TARGET=10.3``
         * Apple ``gcc-4.0``
-        * Python 2.n (n >= 4) for documentation build with Sphinx
+        * system Python 2.5 for documentation build with Sphinx
 
     - alternate build environments:
 
-        * Mac OS X 10.4.11 with Xcode 2.5
-        * Mac OS X 10.6.6 with Xcode 3.2.5
+        * Mac OS X 10.6.8 with Xcode 3.2.6
             - need to change ``/System/Library/Frameworks/{Tcl,Tk}.framework/Version/Current`` to ``8.4``
+        * Note Xcode 4.* does not support building for PPC so cannot be used for this build
+
 
 2.  64-bit / 32-bit, x86_64 and i386 universal, for OS X 10.6 (and later)::
 
-        python build-installer.py \
+        /usr/bin/python build-installer.py \
             --sdk-path=/Developer/SDKs/MacOSX10.6.sdk \
             --universal-archs=intel \
             --dep-target=10.6
 
+    - builds the following third-party libraries
+
+        * NCurses 5.9 (http://bugs.python.org/issue15037)
+        * SQLite 3.7.13
+
     - uses system-supplied versions of third-party libraries
-    
+
         * readline module links with Apple BSD editline (libedit)
-        * builds Oracle Sleepycat DB 4.8 (Python 2.x only)
 
     - requires ActiveState Tcl/Tk 8.5.9 (or later) to be installed for building
 
-    - current target build environment:
-        
-        * Mac OS X 10.6.6 (or later)
-        * Xcode 3.2.5 (or later)
+    - recommended build environment:
+
+        * Mac OS X 10.6.8 (or later)
+        * Xcode 3.2.6
         * ``MacOSX10.6`` SDK
         * ``MACOSX_DEPLOYMENT_TARGET=10.6``
         * Apple ``gcc-4.2``
-        * Python 2.n (n >= 4) for documentation build with Sphinx
+        * system Python 2.6 for documentation build with Sphinx
 
     - alternate build environments:
 
-        * none
+        * none.  Xcode 4.x currently supplies two C compilers.
+          ``llvm-gcc-4.2.1`` has been found to miscompile Python 3.3.x and
+          produce a non-functional Python executable.  As it appears to be
+          considered a migration aid by Apple and is not likely to be fixed,
+          its use should be avoided.  The other compiler, ``clang``, has been
+          undergoing rapid development.  While it appears to have become
+          production-ready in the most recent Xcode 4 releases (Xcode 4.5.x
+          as of this writing), there are still some open issues when
+          building Python and there has not yet been the level of exposure in
+          production environments that the Xcode 3 gcc-4.2 compiler has had.
 
 
 General Prerequisites
@@ -87,6 +100,11 @@ General Prerequisites
 * It is safest to start each variant build with an empty source directory
   populated with a fresh copy of the untarred source.
 
+* It is recommended that you remove any existing installed version of the
+  Python being built::
+
+      sudo rm -rf /Library/Frameworks/Python.framework/Versions/n.n
+
 
 The Recipe
 ----------
@@ -107,9 +125,9 @@ Building other universal installers
 ...................................
 
 It is also possible to build a 4-way universal installer that runs on 
-OS X Leopard or later::
+OS X 10.5 Leopard or later::
 
-  python 2.6 /build-installer.py \
+    /usr/bin/python /build-installer.py \
         --dep-target=10.5
         --universal-archs=all
         --sdk-path=/Developer/SDKs/MacOSX10.5.sdk
@@ -120,7 +138,8 @@ also that you are building on at least OS X 10.5.  4-way includes
 variants can only be run on G5 machines running 10.5.  Note that,
 while OS X 10.6 is only supported on Intel-based machines, it is possible
 to run ``ppc`` (32-bit) executables unmodified thanks to the Rosetta ppc
-emulation in OS X 10.5 and 10.6.
+emulation in OS X 10.5 and 10.6.  The 4-way installer variant must be
+built with Xcode 3.  It is not regularly built or tested.
 
 Other ``--universal-archs`` options are ``64-bit`` (``x86_64``, ``ppc64``),
 and ``3-way`` (``ppc``, ``i386``, ``x86_64``).  None of these options
@@ -133,15 +152,21 @@ Testing
 Ideally, the resulting binaries should be installed and the test suite run
 on all supported OS X releases and architectures.  As a practical matter,
 that is generally not possible.  At a minimum, variant 1 should be run on
-at least one Intel, one PPC G4, and one PPC G3 system and one each of 
-OS X 10.6, 10.5, 10.4, and 10.3.9.  Not all tests run on 10.3.9.
-Variant 2 should be run on 10.6 in both 32-bit and 64-bit modes.::
+a PPC G4 system with OS X 10.5 and at least one Intel system running OS X
+10.8, 10.7, 10.6, or 10.5.  Variant 2 should be run on 10.8, 10.7, and 10.6
+systems in both 32-bit and 64-bit modes.::
 
-    arch -i386 /usr/local/bin/pythonn.n -m test.regrtest -w -u all 
-    arch -X86_64 /usr/local/bin/pythonn.n -m test.regrtest -w -u all
+    /usr/local/bin/pythonn.n -m test -w -u all,-largefile
+    /usr/local/bin/pythonn.n-32 -m test -w -u all
     
 Certain tests will be skipped and some cause the interpreter to fail
 which will likely generate ``Python quit unexpectedly`` alert messages
-to be generated at several points during a test run.  These can
-be ignored.
+to be generated at several points during a test run.  These are normal
+during testing and can be ignored.
+
+It is also recommend to launch IDLE and verify that it is at least
+functional.  Double-click on the IDLE app icon in ``/Applications/Pythonn.n``.
+It should also be tested from the command line::
+
+    /usr/local/bin/idlen.n
 

Mac/BuildScript/ncurses-5.5.patch

-diff -r -u ncurses-5.5-orig/test/Makefile.in ncurses-5.5/test/Makefile.in
---- ncurses-5.5-orig/test/Makefile.in	2006-03-24 12:47:40.000000000 +0100
-+++ ncurses-5.5/test/Makefile.in	2006-03-24 12:47:50.000000000 +0100
-@@ -75,7 +75,7 @@
- MATH_LIB	= @MATH_LIB@
- 
- LD		= @LD@
--LINK		= @LINK_TESTS@ $(LIBTOOL_LINK) $(CC) $(CFLAGS)
-+LINK		= @LINK_TESTS@ $(LIBTOOL_LINK) $(CC) 
- 
- usFLAGS		= @LD_MODEL@ @LOCAL_LDFLAGS@ @LDFLAGS@
- 
-diff -ru ncurses-5.5-orig/ncurses/tinfo/read_entry.c ncurses-5.5/ncurses/tinfo/read_entry.c
---- ncurses-5.5-orig/ncurses/tinfo/read_entry.c 2004-01-11 02:57:05.000000000 +0100
-+++ ncurses-5.5/ncurses/tinfo/read_entry.c      2006-03-25 22:49:39.000000000 +0100
-@@ -474,7 +474,7 @@
-     }
- 
-     /* truncate the terminal name to prevent buffer overflow */
--    (void) sprintf(ttn, "%c/%.*s", *tn, (int) sizeof(ttn) - 3, tn);
-+    (void) sprintf(ttn, "%x/%.*s", *tn, (int) sizeof(ttn) - 3, tn);
- 
-     /* This is System V behavior, in conjunction with our requirements for
-      * writing terminfo entries.
-diff -ru ncurses-5.5-orig/configure ncurses-5.5/configure
---- ncurses-5.5-orig/configure	2005-09-24 23:50:50.000000000 +0200
-+++ ncurses-5.5/configure	2006-03-26 22:24:59.000000000 +0200
-@@ -5027,7 +5027,7 @@
- 	darwin*)
- 		EXTRA_CFLAGS="-no-cpp-precomp"
- 		CC_SHARED_OPTS="-dynamic"
--		MK_SHARED_LIB='$(CC) -dynamiclib -install_name $(DESTDIR)$(libdir)/`basename $@` -compatibility_version $(ABI_VERSION) -current_version $(ABI_VERSION) -o $@'
-+		MK_SHARED_LIB='$(CC) $(CFLAGS) -dynamiclib -install_name $(DESTDIR)$(libdir)/`basename $@` -compatibility_version $(ABI_VERSION) -current_version $(ABI_VERSION) -o $@'
- 		test "$cf_cv_shlib_version" = auto && cf_cv_shlib_version=abi
- 		cf_cv_shlib_version_infix=yes
- 		;;

Mac/BuildScript/resources/ReadMe.txt

@@ -5,8 +5,15 @@ $ARCHITECTURES.
 Installation requires approximately $INSTALL_SIZE MB of disk space,
 ignore the message that it will take zero bytes.
 
-You must install onto your current boot disk, even though the
-installer does not enforce this, otherwise things will not work.
+If you are attempting to install on an OS X 10.8 system, you may
+see a message that Python can't be installed because it is from an
+unidentified developer.  This is because this Python installer
+package is not yet compatible with the Gatekeeper security feature
+introduced in OS X 10.8.  To allow Python to be installed, you
+can override the Gatekeeper policy for this install.  In the Finder,
+instead of double-clicking, control-click or right click the "Python"
+installer package icon.  Then select "Open using ... Installer" from
+the contextual menu that appears.
 
 Python consists of the Python programming language interpreter, plus
 a set of programs to allow easy access to it for Mac users including
@@ -16,10 +23,11 @@ Python programs.
 
                     **** IMPORTANT ****
                     
-Before using IDLE or other programs using the tkinter graphical user
-interface toolkit, visit http://www.python.org/download/mac/tcltk/
-for current information about supported and recommended versions
-of Tcl/Tk for this version of Python and Mac OS X.
+To use IDLE or other programs that use the tkinter graphical user
+interface toolkit, you may need to install a third-party version of
+the Tcl/Tk frameworks.  Visit http://www.python.org/download/mac/tcltk/
+for current information about supported and recommended versions of
+Tcl/Tk for this version of Python and of Mac OS X.
 
                     *******************
 
@@ -32,5 +40,13 @@ optionally place links to the command-line tools in /usr/local/bin as
 well. Double-click on the "Update Shell Profile" command to add the
 "bin" directory inside the framework to your shell's search path.
 
+You must install onto your current boot disk, even though the
+installer does not enforce this, otherwise things will not work.
+
+You can verify the integrity of the disk image file containing the
+installer package and this ReadMe file by comparing its md5 checksum
+and size with the values published on the release page linked at
+http://www.python.org/download/
+
 More information on Python in general can be found at
 http://www.python.org.

Mac/BuildScript/resources/Welcome.rtf

-{\rtf1\ansi\ansicpg1252\cocoartf1038\cocoasubrtf350
+{\rtf1\ansi\ansicpg1252\cocoartf1138\cocoasubrtf470
 {\fonttbl\f0\fswiss\fcharset0 Helvetica;}
 {\colortbl;\red255\green255\blue255;}
 \paperw11904\paperh16836\margl1440\margr1440\vieww9640\viewh10620\viewkind0
-\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640\ql\qnatural
+\pard\tx720\tx1440\tx2160\tx2880\tx3600\tx4320\tx5040\tx5760\tx6480\tx7200\tx7920\tx8640
 
 \f0\fs24 \cf0 This package will install 
 \b Python $FULL_VERSION
@@ -20,9 +20,9 @@ See the ReadMe file and the Python documentation for more information.\
 \
 
 \b NOTE: 
-\b0 This package will by default not update your shell profile and will also not install files in /usr/local.  Double-click 
+\b0 This package will not update your shell profile by default.  Double-click 
 \b Update Shell Profile
-\b0  at any time to make $FULL_VERSION the default Python.\
+\b0  at any time to make $FULL_VERSION the default Python 3 version.  This version can co-exist with other installed versions of Python 3 and Python 2.\
 \
 
 \b IMPORTANT:

Mac/BuildScript/scripts/postflight.documentation

@@ -5,19 +5,10 @@ FWK="/Library/Frameworks/Python.framework/Versions/${PYVER}"
 FWK_DOCDIR_SUBPATH="Resources/English.lproj/Documentation"
 FWK_DOCDIR="${FWK}/${FWK_DOCDIR_SUBPATH}"
 APPDIR="/Applications/Python ${PYVER}"
-DEV_DOCDIR="/Developer/Documentation"
 SHARE_DIR="${FWK}/share"
 SHARE_DOCDIR="${SHARE_DIR}/doc/python${PYVER}"
 SHARE_DOCDIR_TO_FWK="../../.."
 
-# make link in /Developer/Documentation/ for Xcode users
-if [ -d "${DEV_DOCDIR}" ]; then
-    if [ ! -d "${DEV_DOCDIR}/Python" ]; then
-        mkdir -p "${DEV_DOCDIR}/Python"
-    fi
-    ln -fhs "${FWK_DOCDIR}" "${DEV_DOCDIR}/Python/Reference Documentation ${PYVER}"
-fi
-
 # make link in /Applications/Python m.n/ for Finder users
 if [ -d "${APPDIR}" ]; then
     ln -fhs "${FWK_DOCDIR}/index.html" "${APPDIR}/Python Documentation.html"

Mac/BuildScript/scripts/postflight.framework

@@ -8,14 +8,24 @@ FWK="/Library/Frameworks/Python.framework/Versions/@PYVER@"
 
 "${FWK}/bin/python@PYVER@" -Wi \
     "${FWK}/lib/python${PYVER}/compileall.py" \
-    -x badsyntax -x site-packages \
+    -f -x 'bad_coding|badsyntax|site-packages|lib2to3/tests/data' \
     "${FWK}/lib/python${PYVER}"
 
 "${FWK}/bin/python@PYVER@" -Wi -O \
     "${FWK}/lib/python${PYVER}/compileall.py" \
-    -x badsyntax -x site-packages \
+    -f -x 'bad_coding|badsyntax|site-packages|lib2to3/tests/data' \
     "${FWK}/lib/python${PYVER}"
 
+"${FWK}/bin/python@PYVER@" -Wi \
+    "${FWK}/lib/python${PYVER}/compileall.py" \
+    -f -x badsyntax \
+    "${FWK}/lib/python${PYVER}/site-packages"
+
+"${FWK}/bin/python@PYVER@" -Wi -O \
+    "${FWK}/lib/python${PYVER}/compileall.py" \
+    -f -x badsyntax \
+    "${FWK}/lib/python${PYVER}/site-packages"
+
 chgrp -R admin "${FWK}"
 chmod -R g+w "${FWK}"
 

Mac/Makefile.in

@@ -40,14 +40,13 @@ INSTALL_SCRIPT= @INSTALL_SCRIPT@
 INSTALL_DATA=@INSTALL_DATA@
 LN=@LN@
 STRIPFLAG=-s
-CPMAC=/Developer/Tools/CpMac
+CPMAC=CpMac
 
 APPTEMPLATE=$(srcdir)/Resources/app
-APPSUBDIRS=MacOS Resources 
+APPSUBDIRS=MacOS Resources
 compileall=$(srcdir)/../Lib/compileall.py
 
-installapps: install_Python install_pythonw install_PythonLauncher install_IDLE \
-	checkapplepython
+installapps: install_Python install_pythonw install_PythonLauncher install_IDLE
 
 install_pythonw: pythonw
 	$(INSTALL_PROGRAM) $(STRIPFLAG) pythonw "$(DESTDIR)$(prefix)/bin/pythonw$(VERSION)"
@@ -196,14 +195,6 @@ installextras: $(srcdir)/Extras.install.py
 		"$(DESTDIR)$(prefix)/share/doc/python$(VERSION)/examples/Tools" ; \
 		chmod -R ugo+rX,go-w "$(DESTDIR)$(prefix)/share/doc/python$(VERSION)/examples/Tools"
 
-
-checkapplepython: $(srcdir)/Tools/fixapplepython23.py
-	@if ! $(RUNSHARED) $(BUILDPYTHON) $(srcdir)/Tools/fixapplepython23.py -n; then \
-		echo "* WARNING: Apple-installed Python 2.3 will have trouble building extensions from now on."; \
-		echo "* WARNING: Run $(srcdir)/Tools/fixapplepython23.py with \"sudo\" to fix this."; \
-	fi
-
-
 clean:
 	rm pythonw
 	cd PythonLauncher && make clean

Mac/Tools/fixapplepython23.py

-#!/usr/bin/python
-"""fixapplepython23 - Fix Apple-installed Python 2.3 (on Mac OS X 10.3)
-
-Python 2.3 (and 2.3.X for X<5) have the problem that building an extension
-for a framework installation may accidentally pick up the framework
-of a newer Python, in stead of the one that was used to build the extension.
-
-This script modifies the Makefile (in .../lib/python2.3/config) to use
-the newer method of linking extensions with "-undefined dynamic_lookup"
-which fixes this problem.
-
-The script will first check all prerequisites, and return a zero exit
-status also when nothing needs to be fixed.
-"""
-import sys
-import os
-import platform
-
-MAKEFILE='/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/config/Makefile'
-CHANGES=((
-    'LDSHARED=\t$(CC) $(LDFLAGS) -bundle -framework $(PYTHONFRAMEWORK)\n',
-    'LDSHARED=\t$(CC) $(LDFLAGS) -bundle -undefined dynamic_lookup\n'
-    ),(
-    'BLDSHARED=\t$(CC) $(LDFLAGS) -bundle -framework $(PYTHONFRAMEWORK)\n',
-    'BLDSHARED=\t$(CC) $(LDFLAGS) -bundle -undefined dynamic_lookup\n'
-    ),(
-    'CC=\t\tgcc\n',
-    'CC=\t\t/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/config/PantherPythonFix/run-gcc\n'
-    ),(
-    'CXX=\t\tc++\n',
-    'CXX=\t\t/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/config/PantherPythonFix/run-g++\n'
-))
-
-GCC_SCRIPT='/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/config/PantherPythonFix/run-gcc'
-GXX_SCRIPT='/System/Library/Frameworks/Python.framework/Versions/2.3/lib/python2.3/config/PantherPythonFix/run-g++'
-SCRIPT="""#!/bin/sh
-export MACOSX_DEPLOYMENT_TARGET=10.3
-exec %s "${@}"
-"""
-
-def findline(lines, start):
-    """return line starting with given string or -1"""
-    for i in range(len(lines)):
-        if lines[i][:len(start)] == start:
-            return i
-    return -1
-
-def fix(makefile, do_apply):
-    """Fix the Makefile, if required."""
-    fixed = False
-    lines = open(makefile).readlines()
-
-    for old, new in CHANGES:
-        i = findline(lines, new)
-        if i >= 0:
-            # Already fixed
-            continue
-        i = findline(lines, old)
-        if i < 0:
-            print('fixapplepython23: Python installation not fixed (appears broken)')
-            print('fixapplepython23: missing line:', old)
-            return 2
-        lines[i] = new
-        fixed = True
-
-    if fixed:
-        if do_apply:
-            print('fixapplepython23: Fix to Apple-installed Python 2.3 applied')
-            os.rename(makefile, makefile + '~')
-            open(makefile, 'w').writelines(lines)
-            return 0
-        else:
-            print('fixapplepython23: Fix to Apple-installed Python 2.3 should be applied')
-            return 1
-    else:
-        print('fixapplepython23: No fix needed, appears to have been applied before')
-        return 0
-
-def makescript(filename, compiler):
-    """Create a wrapper script for a compiler"""
-    dirname = os.path.split(filename)[0]
-    if not os.access(dirname, os.X_OK):
-        os.mkdir(dirname, 0o755)
-    fp = open(filename, 'w')
-    fp.write(SCRIPT % compiler)
-    fp.close()
-    os.chmod(filename, 0o755)
-    print('fixapplepython23: Created', filename)
-
-def main():
-    # Check for -n option
-    if len(sys.argv) > 1 and sys.argv[1] == '-n':
-        do_apply = False
-    else:
-        do_apply = True
-    # First check OS version
-    if sys.byteorder == 'little':
-        # All intel macs are fine
-        print("fixapplypython23: no fix is needed on MacOSX on Intel")
-        sys.exit(0)
-
-    osver =  platform.mac_ver()
-    if osver != '10.3' and os.ver < '10.3.':
-        print('fixapplepython23: no fix needed on MacOSX < 10.3')
-        sys.exit(0)
-
-    if osver >= '10.4':
-        print('fixapplepython23: no fix needed on MacOSX >= 10.4')
-        sys.exit(0)
-
-    # Test that a framework Python is indeed installed
-    if not os.path.exists(MAKEFILE):
-        print('fixapplepython23: Python framework does not appear to be installed (?), nothing fixed')
-        sys.exit(0)
-    # Check that we can actually write the file
-    if do_apply and not os.access(MAKEFILE, os.W_OK):
-        print('fixapplepython23: No write permission, please run with "sudo"')
-        sys.exit(2)
-    # Create the shell scripts
-    if do_apply:
-        if not os.access(GCC_SCRIPT, os.X_OK):
-            makescript(GCC_SCRIPT, "gcc")
-        if not os.access(GXX_SCRIPT, os.X_OK):
-            makescript(GXX_SCRIPT, "g++")
-    #  Finally fix the makefile
-    rv = fix(MAKEFILE, do_apply)
-    #sys.exit(rv)
-    sys.exit(0)
-
-if __name__ == '__main__':
-    main()