Split "Extending & Embedding" into separate files, one per chapter.

Doc/Makefile.deps

@@ -26,6 +26,11 @@ DOCFILES= $(HOWTOSTYLES) \
 	doc/doc.tex
 
 EXTFILES= ext/ext.tex $(MANSTYLES) $(INDEXSTYLES) $(COMMONTEX) \
+	ext/extending.tex \
+	ext/newtypes.tex \
+	ext/unix.tex \
+	ext/windows.tex \
+	ext/embedding.tex \
 	texinputs/reportingbugs.tex
 
 TUTFILES= tut/tut.tex $(MANSTYLES) $(COMMONTEX)

Doc/ext/unix.tex

+\chapter{Building C and \Cpp{} Extensions on \UNIX{}
+     \label{building-on-unix}}
+
+\sectionauthor{Jim Fulton}{jim@zope.com}
+
+
+%The make file make file, building C extensions on Unix
+
+
+Starting in Python 1.4, Python provides a special make file for
+building make files for building dynamically-linked extensions and
+custom interpreters.  The make file make file builds a make file
+that reflects various system variables determined by configure when
+the Python interpreter was built, so people building module's don't
+have to resupply these settings.  This vastly simplifies the process
+of building extensions and custom interpreters on Unix systems.
+
+The make file make file is distributed as the file
+\file{Misc/Makefile.pre.in} in the Python source distribution.  The
+first step in building extensions or custom interpreters is to copy
+this make file to a development directory containing extension module
+source.
+
+The make file make file, \file{Makefile.pre.in} uses metadata
+provided in a file named \file{Setup}.  The format of the \file{Setup}
+file is the same as the \file{Setup} (or \file{Setup.dist}) file
+provided in the \file{Modules/} directory of the Python source
+distribution.  The \file{Setup} file contains variable definitions:
+
+\begin{verbatim}
+EC=/projects/ExtensionClass
+\end{verbatim}
+
+and module description lines.  It can also contain blank lines and
+comment lines that start with \character{\#}.
+
+A module description line includes a module name, source files,
+options, variable references, and other input files, such
+as libraries or object files.  Consider a simple example:
+
+\begin{verbatim}
+ExtensionClass ExtensionClass.c
+\end{verbatim}
+
+This is the simplest form of a module definition line.  It defines a
+module, \module{ExtensionClass}, which has a single source file,
+\file{ExtensionClass.c}.
+
+This slightly more complex example uses an \strong{-I} option to
+specify an include directory:
+
+\begin{verbatim}
+EC=/projects/ExtensionClass
+cPersistence cPersistence.c -I$(EC)
+\end{verbatim} % $ <-- bow to font lock
+
+This example also illustrates the format for variable references.
+
+For systems that support dynamic linking, the \file{Setup} file should 
+begin:
+
+\begin{verbatim}
+*shared*
+\end{verbatim}
+
+to indicate that the modules defined in \file{Setup} are to be built
+as dynamically linked modules.  A line containing only \samp{*static*}
+can be used to indicate the subsequently listed modules should be
+statically linked.
+
+Here is a complete \file{Setup} file for building a
+\module{cPersistent} module:
+
+\begin{verbatim}
+# Set-up file to build the cPersistence module. 
+# Note that the text should begin in the first column.
+*shared*
+
+# We need the path to the directory containing the ExtensionClass
+# include file.
+EC=/projects/ExtensionClass
+cPersistence cPersistence.c -I$(EC)
+\end{verbatim} % $ <-- bow to font lock
+
+After the \file{Setup} file has been created, \file{Makefile.pre.in}
+is run with the \samp{boot} target to create a make file:
+
+\begin{verbatim}
+make -f Makefile.pre.in boot
+\end{verbatim}
+
+This creates the file, Makefile.  To build the extensions, simply
+run the created make file:
+
+\begin{verbatim}
+make
+\end{verbatim}
+
+It's not necessary to re-run \file{Makefile.pre.in} if the
+\file{Setup} file is changed.  The make file automatically rebuilds
+itself if the \file{Setup} file changes.
+
+
+\section{Building Custom Interpreters \label{custom-interps}}
+
+The make file built by \file{Makefile.pre.in} can be run with the
+\samp{static} target to build an interpreter:
+
+\begin{verbatim}
+make static
+\end{verbatim}
+
+Any modules defined in the \file{Setup} file before the
+\samp{*shared*} line will be statically linked into the interpreter.
+Typically, a \samp{*shared*} line is omitted from the
+\file{Setup} file when a custom interpreter is desired.
+
+
+\section{Module Definition Options \label{module-defn-options}}
+
+Several compiler options are supported:
+
+\begin{tableii}{l|l}{programopt}{Option}{Meaning}
+  \lineii{-C}{Tell the C pre-processor not to discard comments}
+  \lineii{-D\var{name}=\var{value}}{Define a macro}
+  \lineii{-I\var{dir}}{Specify an include directory, \var{dir}}
+  \lineii{-L\var{dir}}{Specify a link-time library directory, \var{dir}}
+  \lineii{-R\var{dir}}{Specify a run-time library directory, \var{dir}}
+  \lineii{-l\var{lib}}{Link a library, \var{lib}}
+  \lineii{-U\var{name}}{Undefine a macro}
+\end{tableii}
+
+Other compiler options can be included (snuck in) by putting them
+in variables.
+
+Source files can include files with \file{.c}, \file{.C}, \file{.cc},
+\file{.cpp}, \file{.cxx}, and \file{.c++} extensions. 
+
+Other input files include files with \file{.a}, \file{.o}, \file{.sl}, 
+and \file{.so} extensions.
+
+
+\section{Example \label{module-defn-example}}
+
+Here is a more complicated example from \file{Modules/Setup.dist}:
+
+\begin{verbatim}
+GMP=/ufs/guido/src/gmp
+mpz mpzmodule.c -I$(GMP) $(GMP)/libgmp.a
+\end{verbatim}
+
+which could also be written as:
+
+\begin{verbatim}
+mpz mpzmodule.c -I$(GMP) -L$(GMP) -lgmp
+\end{verbatim}
+
+
+\section{Distributing your extension modules
+     \label{distributing}}
+
+There are two ways to distribute extension modules for others to use.
+The way that allows the easiest cross-platform support is to use the
+\module{distutils}\refstmodindex{distutils} package.  The manual
+\citetitle[../dist/dist.html]{Distributing Python Modules} contains
+information on this approach.  It is recommended that all new
+extensions be distributed using this approach to allow easy building
+and installation across platforms.  Older extensions should migrate to
+this approach as well.
+
+What follows describes the older approach; there are still many
+extensions which use this.
+
+When distributing your extension modules in source form, make sure to
+include a \file{Setup} file.  The \file{Setup} file should be named
+\file{Setup.in} in the distribution.  The make file make file,
+\file{Makefile.pre.in}, will copy \file{Setup.in} to \file{Setup} if
+the person installing the extension doesn't do so manually.
+Distributing a \file{Setup.in} file makes it easy for people to
+customize the \file{Setup} file while keeping the original in
+\file{Setup.in}.
+
+It is a good idea to include a copy of \file{Makefile.pre.in} for
+people who do not have a source distribution of Python.
+
+Do not distribute a make file.  People building your modules
+should use \file{Makefile.pre.in} to build their own make file.  A
+\file{README} file included in the package should provide simple
+instructions to perform the build.

Doc/ext/windows.tex

+\chapter{Building C and \Cpp{} Extensions on Windows
+     \label{building-on-windows}}
+
+
+This chapter briefly explains how to create a Windows extension module
+for Python using Microsoft Visual \Cpp{}, and follows with more
+detailed background information on how it works.  The explanatory
+material is useful for both the Windows programmer learning to build
+Python extensions and the \UNIX{} programmer interested in producing
+software which can be successfully built on both \UNIX{} and Windows.
+
+
+\section{A Cookbook Approach \label{win-cookbook}}
+
+\sectionauthor{Neil Schemenauer}{neil_schemenauer@transcanada.com}
+
+This section provides a recipe for building a Python extension on
+Windows.
+
+Grab the binary installer from \url{http://www.python.org/} and
+install Python.  The binary installer has all of the required header
+files except for \file{pyconfig.h}.
+
+Get the source distribution and extract it into a convenient location.
+Copy the \file{pyconfig.h} from the \file{PC/} directory into the
+\file{include/} directory created by the installer.
+
+Create a \file{Setup} file for your extension module, as described in
+chapter \ref{building-on-unix}.
+
+Get David Ascher's \file{compile.py} script from
+\url{http://starship.python.net/crew/da/compile/}.  Run the script to
+create Microsoft Visual \Cpp{} project files.
+
+Open the DSW file in Visual \Cpp{} and select \strong{Build}.
+
+If your module creates a new type, you may have trouble with this line:
+
+\begin{verbatim}
+    PyObject_HEAD_INIT(&PyType_Type)
+\end{verbatim}
+
+Change it to:
+
+\begin{verbatim}
+    PyObject_HEAD_INIT(NULL)
+\end{verbatim}
+
+and add the following to the module initialization function:
+
+\begin{verbatim}
+    MyObject_Type.ob_type = &PyType_Type;
+\end{verbatim}
+
+Refer to section 3 of the
+\citetitle[http://www.python.org/doc/FAQ.html]{Python FAQ} for details
+on why you must do this.
+
+
+\section{Differences Between \UNIX{} and Windows
+     \label{dynamic-linking}}
+\sectionauthor{Chris Phoenix}{cphoenix@best.com}
+
+
+\UNIX{} and Windows use completely different paradigms for run-time
+loading of code.  Before you try to build a module that can be
+dynamically loaded, be aware of how your system works.
+
+In \UNIX{}, a shared object (\file{.so}) file contains code to be used by the
+program, and also the names of functions and data that it expects to
+find in the program.  When the file is joined to the program, all
+references to those functions and data in the file's code are changed
+to point to the actual locations in the program where the functions
+and data are placed in memory.  This is basically a link operation.
+
+In Windows, a dynamic-link library (\file{.dll}) file has no dangling
+references.  Instead, an access to functions or data goes through a
+lookup table.  So the DLL code does not have to be fixed up at runtime
+to refer to the program's memory; instead, the code already uses the
+DLL's lookup table, and the lookup table is modified at runtime to
+point to the functions and data.
+
+In \UNIX{}, there is only one type of library file (\file{.a}) which
+contains code from several object files (\file{.o}).  During the link
+step to create a shared object file (\file{.so}), the linker may find
+that it doesn't know where an identifier is defined.  The linker will
+look for it in the object files in the libraries; if it finds it, it
+will include all the code from that object file.
+
+In Windows, there are two types of library, a static library and an
+import library (both called \file{.lib}).  A static library is like a
+\UNIX{} \file{.a} file; it contains code to be included as necessary.
+An import library is basically used only to reassure the linker that a
+certain identifier is legal, and will be present in the program when
+the DLL is loaded.  So the linker uses the information from the
+import library to build the lookup table for using identifiers that
+are not included in the DLL.  When an application or a DLL is linked,
+an import library may be generated, which will need to be used for all
+future DLLs that depend on the symbols in the application or DLL.
+
+Suppose you are building two dynamic-load modules, B and C, which should
+share another block of code A.  On \UNIX{}, you would \emph{not} pass
+\file{A.a} to the linker for \file{B.so} and \file{C.so}; that would
+cause it to be included twice, so that B and C would each have their
+own copy.  In Windows, building \file{A.dll} will also build
+\file{A.lib}.  You \emph{do} pass \file{A.lib} to the linker for B and
+C.  \file{A.lib} does not contain code; it just contains information
+which will be used at runtime to access A's code.  
+
+In Windows, using an import library is sort of like using \samp{import
+spam}; it gives you access to spam's names, but does not create a
+separate copy.  On \UNIX{}, linking with a library is more like
+\samp{from spam import *}; it does create a separate copy.
+
+
+\section{Using DLLs in Practice \label{win-dlls}}
+\sectionauthor{Chris Phoenix}{cphoenix@best.com}
+
+Windows Python is built in Microsoft Visual \Cpp{}; using other
+compilers may or may not work (though Borland seems to).  The rest of
+this section is MSV\Cpp{} specific.
+
+When creating DLLs in Windows, you must pass \file{python15.lib} to
+the linker.  To build two DLLs, spam and ni (which uses C functions
+found in spam), you could use these commands:
+
+\begin{verbatim}
+cl /LD /I/python/include spam.c ../libs/python15.lib
+cl /LD /I/python/include ni.c spam.lib ../libs/python15.lib
+\end{verbatim}
+
+The first command created three files: \file{spam.obj},
+\file{spam.dll} and \file{spam.lib}.  \file{Spam.dll} does not contain
+any Python functions (such as \cfunction{PyArg_ParseTuple()}), but it
+does know how to find the Python code thanks to \file{python15.lib}.
+
+The second command created \file{ni.dll} (and \file{.obj} and
+\file{.lib}), which knows how to find the necessary functions from
+spam, and also from the Python executable.
+
+Not every identifier is exported to the lookup table.  If you want any
+other modules (including Python) to be able to see your identifiers,
+you have to say \samp{_declspec(dllexport)}, as in \samp{void
+_declspec(dllexport) initspam(void)} or \samp{PyObject
+_declspec(dllexport) *NiGetSpamData(void)}.
+
+Developer Studio will throw in a lot of import libraries that you do
+not really need, adding about 100K to your executable.  To get rid of
+them, use the Project Settings dialog, Link tab, to specify
+\emph{ignore default libraries}.  Add the correct
+\file{msvcrt\var{xx}.lib} to the list of libraries.