Merged revisions...

Merged revisions 62260-62261,62266,62271,62277-62279,62289-62290,62293-62298,62302-62306,62308,62311,62313-62315,62319-62321 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk

........
  r62260 | gregory.p.smith | 2008-04-10 01:11:56 +0200 (Thu, 10 Apr 2008) | 2 lines

  better diagnostics
........
  r62261 | gregory.p.smith | 2008-04-10 01:16:37 +0200 (Thu, 10 Apr 2008) | 3 lines

  Raise SystemError when size < 0 is passed into PyString_FromStringAndSize,
  PyBytes_FromStringAndSize or PyUnicode_FromStringAndSize.  [issue2587]
........
  r62266 | neal.norwitz | 2008-04-10 07:46:39 +0200 (Thu, 10 Apr 2008) | 5 lines

  Remove the test file before writing it in case there is no write permission.
  This might help fix some of the failures on Windows box(es).  It doesn't hurt
  either way and ensure the tests are a little more self contained (ie have
  less assumptions).
........
  r62271 | gregory.p.smith | 2008-04-10 21:50:36 +0200 (Thu, 10 Apr 2008) | 2 lines

  get rid of assert (size >= 0) now that an explicit if (size < 0) is in the code.
........
  r62277 | andrew.kuchling | 2008-04-10 23:27:10 +0200 (Thu, 10 Apr 2008) | 1 line

  Remove forward-looking statement
........
  r62278 | andrew.kuchling | 2008-04-10 23:28:51 +0200 (Thu, 10 Apr 2008) | 1 line

  Add punctuation
........
  r62279 | andrew.kuchling | 2008-04-10 23:29:01 +0200 (Thu, 10 Apr 2008) | 1 line

  Use issue directive
........
  r62289 | thomas.heller | 2008-04-11 15:05:38 +0200 (Fri, 11 Apr 2008) | 3 lines

  Move backwards compatibility macro to the correct place;
  PyIndex_Check() was introduced in Python 2.5.
........
  r62290 | thomas.heller | 2008-04-11 16:20:26 +0200 (Fri, 11 Apr 2008) | 2 lines

  Performance improvements.
........
  r62293 | christian.heimes | 2008-04-12 15:03:03 +0200 (Sat, 12 Apr 2008) | 2 lines

  Applied patch #2617 from Frank Wierzbicki wit some extras from me
  -J and -X are now reserved for Jython and non-standard arguments (e.g. IronPython). I've added some extra comments to make sure the reservation don't get missed in the future.
........
  r62294 | georg.brandl | 2008-04-12 20:11:18 +0200 (Sat, 12 Apr 2008) | 2 lines

  Use absolute path in sys.path.
........
  r62295 | georg.brandl | 2008-04-12 20:36:09 +0200 (Sat, 12 Apr 2008) | 2 lines

  #2615: small consistency update by Jeroen Ruigrok van der Werven.
........
  r62296 | georg.brandl | 2008-04-12 21:00:20 +0200 (Sat, 12 Apr 2008) | 2 lines

  Add Jeroen.
........
  r62297 | georg.brandl | 2008-04-12 21:05:37 +0200 (Sat, 12 Apr 2008) | 2 lines

  Don't offend snake lovers.
........
  r62298 | gregory.p.smith | 2008-04-12 22:37:48 +0200 (Sat, 12 Apr 2008) | 2 lines

  fix compiler warnings
........
  r62302 | gregory.p.smith | 2008-04-13 00:24:04 +0200 (Sun, 13 Apr 2008) | 3 lines

  socket.error inherits from IOError, it no longer needs listing in
  the all_errors tuple.
........
  r62303 | brett.cannon | 2008-04-13 01:44:07 +0200 (Sun, 13 Apr 2008) | 8 lines

  Re-implement the 'warnings' module in C. This allows for usage of the
  'warnings' code in places where it was previously not possible (e.g., the
  parser). It could also potentially lead to a speed-up in interpreter start-up
  if the C version of the code (_warnings) is imported over the use of the
  Python version in key places.

  Closes issue #1631171.
........
  r62304 | gregory.p.smith | 2008-04-13 02:03:25 +0200 (Sun, 13 Apr 2008) | 3 lines

  Adds a profile-opt target for easy compilation of a python binary using
  gcc's profile guided optimization.
........
  r62305 | brett.cannon | 2008-04-13 02:18:44 +0200 (Sun, 13 Apr 2008) | 3 lines

  Fix a bug in PySys_HasWarnOption() where it was not properly checking the
  length of the list storing the warning options.
........
  r62306 | brett.cannon | 2008-04-13 02:25:15 +0200 (Sun, 13 Apr 2008) | 2 lines

  Fix an accidental bug of an non-existent init function.
........
  r62308 | andrew.kuchling | 2008-04-13 03:05:59 +0200 (Sun, 13 Apr 2008) | 1 line

  Mention -J, -X
........
  r62311 | benjamin.peterson | 2008-04-13 04:20:05 +0200 (Sun, 13 Apr 2008) | 2 lines

  Give the "Interactive Interpreter Changes" section in 2.6 whatsnew a unique link name
........
  r62313 | brett.cannon | 2008-04-13 04:42:36 +0200 (Sun, 13 Apr 2008) | 3 lines

  Fix test_warnings by making the state of things more consistent for each test
  when it is run.
........
  r62314 | skip.montanaro | 2008-04-13 05:17:30 +0200 (Sun, 13 Apr 2008) | 2 lines

  spelling
........
  r62315 | georg.brandl | 2008-04-13 09:07:44 +0200 (Sun, 13 Apr 2008) | 2 lines

  Fix markup.
........
  r62319 | christian.heimes | 2008-04-13 11:30:17 +0200 (Sun, 13 Apr 2008) | 1 line

  Fix compiler warning Include/warnings.h:19:28: warning: no newline at end of file
........
  r62320 | christian.heimes | 2008-04-13 11:33:24 +0200 (Sun, 13 Apr 2008) | 1 line

  Use PyString_InternFromString instead of PyString_FromString for static vars
........
  r62321 | christian.heimes | 2008-04-13 11:37:05 +0200 (Sun, 13 Apr 2008) | 1 line

  Added new files to the pcbuild files
........

Doc/conf.py

@@ -8,7 +8,7 @@
 # that aren't pickleable (module imports are okay, they're removed automatically).
 
 import sys, os, time
-sys.path.append('tools/sphinxext')
+sys.path.append(os.path.abspath('tools/sphinxext'))
 
 # General configuration
 # ---------------------

Doc/library/warnings.rst

@@ -202,16 +202,16 @@ Available Functions
    sources).
 
 
-.. function:: showwarning(message, category, filename, lineno[, file])
+.. function:: showwarning(message, category, filename, lineno[, file[, line]])
 
    Write a warning to a file.  The default implementation calls
-   ``formatwarning(message, category, filename, lineno)`` and writes the resulting
-   string to *file*, which defaults to ``sys.stderr``.  You may replace this
-   function with an alternative implementation by assigning to
+   ``formatwarning(message, category, filename, lineno, line)`` and writes the
+   resulting string to *file*, which defaults to ``sys.stderr``.  You may replace
+   this function with an alternative implementation by assigning to
    ``warnings.showwarning``.
 
 
-.. function:: formatwarning(message, category, filename, lineno)
+.. function:: formatwarning(message, category, filename, lineno[, line])
 
    Format a warning the standard way.  This returns a string  which may contain
    embedded newlines and ends in a newline.

Doc/library/xml.dom.minidom.rst

@@ -126,7 +126,7 @@ module documentation.  This section lists the differences between the API and
    to discard children of that node.
 
 
-.. method:: Node.writexml(writer[,indent=""[,addindent=""[,newl=""]]])
+.. method:: Node.writexml(writer[, indent=""[, addindent=""[, newl=""[, encoding=""]]]])
 
    Write XML to the writer object.  The writer should have a :meth:`write` method
    which matches that of the file object interface.  The *indent* parameter is the
@@ -153,7 +153,7 @@ module documentation.  This section lists the differences between the API and
    encoding argument should be specified as "utf-8".
 
 
-.. method:: Node.toprettyxml([indent[, newl[, encoding]]])
+.. method:: Node.toprettyxml([indent=""[, newl=""[, encoding=""]]])
 
    Return a pretty-printed version of the document. *indent* specifies the
    indentation string and defaults to a tabulator; *newl* specifies the string

Doc/reference/simple_stmts.rst

@@ -438,7 +438,36 @@ The :keyword:`yield` statement is only used when defining a generator function,
 and is only used in the body of the generator function. Using a :keyword:`yield`
 statement in a function definition is sufficient to cause that definition to
 create a generator function instead of a normal function.
->>>>>>> .merge-right.r59773
+When a generator function is called, it returns an iterator known as a generator
+iterator, or more commonly, a generator.  The body of the generator function is
+executed by calling the generator's :meth:`next` method repeatedly until it
+raises an exception.
+
+When a :keyword:`yield` statement is executed, the state of the generator is
+frozen and the value of :token:`expression_list` is returned to :meth:`next`'s
+caller.  By "frozen" we mean that all local state is retained, including the
+current bindings of local variables, the instruction pointer, and the internal
+evaluation stack: enough information is saved so that the next time :meth:`next`
+is invoked, the function can proceed exactly as if the :keyword:`yield`
+statement were just another external call.
+
+As of Python version 2.5, the :keyword:`yield` statement is now allowed in the
+:keyword:`try` clause of a :keyword:`try` ...  :keyword:`finally` construct.  If
+the generator is not resumed before it is finalized (by reaching a zero
+reference count or by being garbage collected), the generator-iterator's
+:meth:`close` method will be called, allowing any pending :keyword:`finally`
+clauses to execute.
+
+
+.. seealso::
+
+   :pep:`0255` - Simple Generators
+      The proposal for adding generators and the :keyword:`yield` statement to Python.
+
+   :pep:`0342` - Coroutines via Enhanced Generators
+      The proposal that, among other generator enhancements, proposed allowing
+      :keyword:`yield` to appear inside a :keyword:`try` ... :keyword:`finally` block.
+
 
 .. _raise:
 

Doc/tutorial/appetite.rst

@@ -68,7 +68,7 @@ application written in C and use it as an extension or command language for that
 application.
 
 By the way, the language is named after the BBC show "Monty Python's Flying
-Circus" and has nothing to do with nasty reptiles.  Making references to Monty
+Circus" and has nothing to do with reptiles.  Making references to Monty
 Python skits in documentation is not only allowed, it is encouraged!
 
 Now that you are all excited about Python, you'll want to examine it in some

Include/Python.h

@@ -93,6 +93,7 @@
 #include "iterobject.h"
 #include "genobject.h"
 #include "descrobject.h"
+#include "warnings.h"
 #include "weakrefobject.h"
 
 #include "codecs.h"

Include/pyerrors.h

@@ -218,13 +218,6 @@ PyAPI_FUNC(PyObject *) PyErr_NewException(const char *name, PyObject *base,
                                          PyObject *dict);
 PyAPI_FUNC(void) PyErr_WriteUnraisable(PyObject *);
 
-/* Issue a warning or exception */
-PyAPI_FUNC(int) PyErr_WarnEx(PyObject *category, const char *msg,
-			     Py_ssize_t stack_level);
-PyAPI_FUNC(int) PyErr_WarnExplicit(PyObject *, const char *,
-				   const char *, int,
-				   const char *, PyObject *);
-
 /* In sigcheck.c or signalmodule.c */
 PyAPI_FUNC(int) PyErr_CheckSignals(void);
 PyAPI_FUNC(void) PyErr_SetInterrupt(void);

Include/sysmodule.h

@@ -22,6 +22,7 @@ PyAPI_DATA(int) _PySys_CheckInterval;
 
 PyAPI_FUNC(void) PySys_ResetWarnOptions(void);
 PyAPI_FUNC(void) PySys_AddWarnOption(const wchar_t *);
+PyAPI_FUNC(int) PySys_HasWarnOptions(void);
 
 #ifdef __cplusplus
 }

Include/traceback.h

@@ -19,6 +19,7 @@ typedef struct _traceback {
 
 PyAPI_FUNC(int) PyTraceBack_Here(struct _frame *);
 PyAPI_FUNC(int) PyTraceBack_Print(PyObject *, PyObject *);
+PyAPI_FUNC(int) Py_DisplaySourceLine(PyObject *, const char *, int);
 
 /* Reveal traceback type so we can typecheck traceback objects */
 PyAPI_DATA(PyTypeObject) PyTraceBack_Type;

Include/warnings.h

+#ifndef Py_WARNINGS_H
+#define Py_WARNINGS_H
+#ifdef __cplusplus
+extern "C" {
+#endif
+
+PyAPI_FUNC(void) _PyWarnings_Init(void);
+
+PyAPI_FUNC(int) PyErr_WarnEx(PyObject *, const char *, Py_ssize_t);
+PyAPI_FUNC(int) PyErr_WarnExplicit(PyObject *, const char *, const char *, int,
+                                    const char *, PyObject *);
+
+/* DEPRECATED: Use PyErr_WarnEx() instead. */
+#define PyErr_Warn(category, msg) PyErr_WarnEx(category, msg, 1)
+
+#ifdef __cplusplus
+}
+#endif
+#endif /* !Py_WARNINGS_H */
+

Lib/ftplib.py

@@ -65,7 +65,7 @@ class error_proto(Error): pass          # response does not begin with [1-5]
 
 # All exceptions (hopefully) that may be raised here and that aren't
 # (always) programming errors on our side
-all_errors = (Error, socket.error, IOError, EOFError)
+all_errors = (Error, IOError, EOFError)
 
 
 # Line terminators (we always output CRLF, but accept any of CRLF, CR, LF)

Lib/test/test_array.py

@@ -162,6 +162,7 @@ class BaseTest(unittest.TestCase):
     def test_tofromfile(self):
         a = array.array(self.typecode, 2*self.example)
         self.assertRaises(TypeError, a.tofile)
+        test_support.unlink(test_support.TESTFN)
         f = open(test_support.TESTFN, 'wb')
         try:
             a.tofile(f)

Lib/test/test_deque.py

@@ -64,6 +64,7 @@ class TestBasic(unittest.TestCase):
         self.assertEqual(list(d), [7, 8, 9])
         d = deque(range(200), maxlen=10)
         d.append(d)
+        test_support.unlink(test_support.TESTFN)
         fo = open(test_support.TESTFN, "w")
         try:
             fo.write(str(d))
@@ -283,6 +284,7 @@ class TestBasic(unittest.TestCase):
         d = deque(range(200))
         d.append(d)
         try:
+            test_support.unlink(test_support.TESTFN)
             fo = open(test_support.TESTFN, "w")
             print(d, file=fo, end='')
             fo.close()

Lib/test/test_support.py

@@ -354,14 +354,26 @@ class WarningMessage(object):
         self.filename = None
         self.lineno = None
 
-    def _showwarning(self, message, category, filename, lineno, file=None):
+    def _showwarning(self, message, category, filename, lineno, file=None,
+                        line=None):
         self.message = message
         self.category = category
         self.filename = filename
         self.lineno = lineno
+        self.line = line
+
+    def reset(self):
+        self._showwarning(*((None,)*6))
+
+    def __str__(self):
+        return ("{message : %r, category : %r, filename : %r, lineno : %s, "
+                    "line : %r}" % (self.message,
+                            self.category.__name__ if self.category else None,
+                            self.filename, self.lineno, self.line))
+
 
 @contextlib.contextmanager
-def catch_warning():
+def catch_warning(module=warnings):
     """
     Guard the warnings filter from being permanently changed and record the
     data of the last warning that has been issued.
@@ -372,15 +384,15 @@ def catch_warning():
             warnings.warn("foo")
             assert str(w.message) == "foo"
     """
-    warning = WarningMessage()
-    original_filters = warnings.filters[:]
-    original_showwarning = warnings.showwarning
-    warnings.showwarning = warning._showwarning
+    warning_obj = WarningMessage()
+    original_filters = module.filters[:]
+    original_showwarning = module.showwarning
+    module.showwarning = warning_obj._showwarning
     try:
-        yield warning
+        yield warning_obj
     finally:
-        warnings.showwarning = original_showwarning
-        warnings.filters = original_filters
+        module.showwarning = original_showwarning
+        module.filters = original_filters
 
 class EnvironmentVarGuard(object):
 
@@ -542,7 +554,7 @@ def bigmemtest(minsize, memuse, overhead=5*_1M):
     'minsize' is the minimum useful size for the test (in arbitrary,
     test-interpreted units.) 'memuse' is the number of 'bytes per size' for
     the test, or a good estimate of it. 'overhead' specifies fixed overhead,
-    independant of the testsize, and defaults to 5Mb.
+    independent of the testsize, and defaults to 5Mb.
 
     The decorator tries to guess a good value for 'size' and passes it to
     the decorated test function. If minsize * memuse is more than the

Lib/test/test_uu.py

@@ -121,6 +121,7 @@ class UUFileTest(unittest.TestCase):
     def test_encode(self):
         fin = fout = None
         try:
+            test_support.unlink(self.tmpin)
             fin = open(self.tmpin, 'wb')
             fin.write(plaintext)
             fin.close()
@@ -150,6 +151,7 @@ class UUFileTest(unittest.TestCase):
     def test_decode(self):
         f = None
         try:
+            test_support.unlink(self.tmpin)
             f = open(self.tmpin, 'wb')
             f.write(encodedtextwrapped(0o644, self.tmpout))
             f.close()

Makefile.pre.in

@@ -175,6 +175,10 @@ LIBOBJS=	@LIBOBJS@
 PYTHON=		python$(EXE)
 BUILDPYTHON=	python$(BUILDEXE)
 
+# The task to run while instrument when building the profile-opt target
+PROFILE_TASK=	Tools/pybench/pybench.py -n 2 --with-gc --with-syscheck
+#PROFILE_TASK=	Lib/test/regrtest.py
+
 # === Definitions added by makesetup ===
 
 
@@ -244,6 +248,7 @@ ASDLGEN=	$(srcdir)/Parser/asdl_c.py
 ##########################################################################
 # Python
 PYTHON_OBJS=	\
+		Python/_warnings.o \
 		Python/Python-ast.o \
 		Python/asdl.o \
 		Python/ast.o \
@@ -346,6 +351,27 @@ LIBRARY_OBJS=	\
 all:		build_all
 build_all:	$(BUILDPYTHON) oldsharedmods sharedmods
 
+# Compile a binary with gcc profile guided optimization.
+profile-opt:
+	@echo "Building with support for profile generation:"
+	$(MAKE) clean
+	$(MAKE) build_all_generate_profile
+	@echo "Running benchmark to generate profile data:"
+	$(MAKE) profile-removal
+	$(MAKE) run_profile_task
+	@echo "Rebuilding with profile guided optimizations:"
+	$(MAKE) clean
+	$(MAKE) build_all_use_profile
+
+build_all_generate_profile:
+	$(MAKE) all CFLAGS="$(CFLAGS) -fprofile-generate" LIBS="$(LIBS) -lgcov"
+
+run_profile_task:
+	./$(BUILDPYTHON) $(PROFILE_TASK)
+
+build_all_use_profile:
+	$(MAKE) all CFLAGS="$(CFLAGS) -fprofile-use"
+
 # Build the interpreter
 $(BUILDPYTHON):	Modules/python.o $(LIBRARY) $(LDLIBRARY)
 		$(LINKCC) $(LDFLAGS) $(LINKFORSHARED) -o $@ \
@@ -616,6 +642,7 @@ PYTHON_HEADERS= \
 		Include/tupleobject.h \
 		Include/ucnhash.h \
 		Include/unicodeobject.h \
+		Include/warnings.h \
 		Include/weakrefobject.h \
 		pyconfig.h \
 		$(PARSER_HEADERS)
@@ -1100,7 +1127,10 @@ clean: pycremoval
 	find $(srcdir)/build -name 'fficonfig.h' -exec rm -f {} ';' || true
 	find $(srcdir)/build -name 'fficonfig.py' -exec rm -f {} ';' || true
 
-clobber: clean rmtestturds docclean
+profile-removal:
+	find . -name '*.gc??' -exec rm -f {} ';'
+
+clobber: clean profile-removal
 	-rm -f $(BUILDPYTHON) $(PGEN) $(LIBRARY) $(LDLIBRARY) $(DLLLIBRARY) \
 		tags TAGS \
 		config.cache config.log pyconfig.h Modules/config.c

Misc/developers.txt

@@ -17,6 +17,9 @@ the format to accommodate documentation needs as they arise.
 Permissions History
 -------------------
 
+- Jeroen Ruigrok van der Werven was given SVN access on 12 April 2008
+  by Georg Brandl, for documentation work.
+
 - Josiah Carlson was given SVN access on 26 March 2008 by Georg Brandl,
   for work on asyncore/asynchat.
 

Modules/_ctypes/_ctypes.c

@@ -2171,7 +2171,7 @@ KeepRef(CDataObject *target, Py_ssize_t index, PyObject *keep)
 		return 0;
 	}
 	ob = CData_GetContainer(target);
-	if (ob->b_objects == NULL || !PyDict_Check(ob->b_objects)) {
+	if (ob->b_objects == NULL || !PyDict_CheckExact(ob->b_objects)) {
 		Py_XDECREF(ob->b_objects);
 		ob->b_objects = keep; /* refcount consumed */
 		return 0;
@@ -4544,7 +4544,7 @@ Pointer_init(CDataObject *self, PyObject *args, PyObject *kw)
 {
 	PyObject *value = NULL;
 
-	if (!PyArg_ParseTuple(args, "|O:POINTER", &value))
+	if (!PyArg_UnpackTuple(args, "POINTER", 0, 1, &value))
 		return -1;
 	if (value == NULL)
 		return 0;
@@ -4946,7 +4946,7 @@ cast(void *ptr, PyObject *src, PyObject *ctype)
 		}
 		Py_XINCREF(obj->b_objects);
 		result->b_objects = obj->b_objects;
-		if (result->b_objects && PyDict_Check(result->b_objects)) {
+		if (result->b_objects && PyDict_CheckExact(result->b_objects)) {
 			PyObject *index;
 			int rc;
 			index = PyLong_FromVoidPtr((void *)src);

Modules/config.c.in

@@ -28,6 +28,7 @@ extern void PyMarshal_Init(void);
 extern void initimp(void);
 extern void initgc(void);
 extern void init_ast(void);
+extern void _PyWarnings_Init(void);
 
 struct _inittab _PyImport_Inittab[] = {
 
@@ -50,6 +51,9 @@ struct _inittab _PyImport_Inittab[] = {
 	/* This lives in gcmodule.c */
 	{"gc", initgc},
 
+    /* This lives in _warnings.c */
+    {"_warnings", _PyWarnings_Init},
+
 	/* Sentinel */
 	{0, 0}
 };

Modules/main.c

@@ -45,7 +45,7 @@ static wchar_t **orig_argv;
 static int  orig_argc;
 
 /* command line options */
-#define BASE_OPTS L"bBc:dEhim:OStuvVW:xX?"
+#define BASE_OPTS L"bBc:dEhiJm:OStuvVW:xX?"
 
 #define PROGRAM_OPTS BASE_OPTS
 
@@ -345,6 +345,8 @@ Py_Main(int argc, wchar_t **argv)
 			Py_InteractiveFlag++;
 			break;
 
+		/* case 'J': reserved for Jython */
+
 		case 'O':
 			Py_OptimizeFlag++;
 			break;
@@ -378,6 +380,8 @@ Py_Main(int argc, wchar_t **argv)
 			skipfirstline = 1;
 			break;
 
+		/* case 'X': reserved for non-standard arguments */
+
 		case 'h':
 		case '?':
 			help++;

Objects/bytesobject.c

@@ -108,7 +108,11 @@ PyBytes_FromStringAndSize(const char *bytes, Py_ssize_t size)
     PyBytesObject *new;
     Py_ssize_t alloc;
 
-    assert(size >= 0);
+    if (size < 0) {
+        PyErr_SetString(PyExc_SystemError,
+            "Negative size passed to PyBytes_FromStringAndSize");
+        return NULL;
+    }
 
     new = PyObject_New(PyBytesObject, &PyBytes_Type);
     if (new == NULL)

Objects/stringobject.c

@@ -64,7 +64,11 @@ PyObject *
 PyString_FromStringAndSize(const char *str, Py_ssize_t size)
 {
 	register PyStringObject *op;
-	assert(size >= 0);
+	if (size < 0) {
+		PyErr_SetString(PyExc_SystemError,
+		    "Negative size passed to PyString_FromStringAndSize");
+		return NULL;
+	}
 	if (size == 0 && (op = nullstring) != NULL) {
 #ifdef COUNT_ALLOCS
 		null_strings++;

Objects/unicodeobject.c

@@ -497,6 +497,13 @@ PyObject *PyUnicode_FromUnicode(const Py_UNICODE *u,
 PyObject *PyUnicode_FromStringAndSize(const char *u, Py_ssize_t size)
 {
     PyUnicodeObject *unicode;
+
+	if (size < 0) {
+		PyErr_SetString(PyExc_SystemError,
+		    "Negative size passed to PyUnicode_FromStringAndSize");
+		return NULL;
+	}
+
     /* If the Unicode data is known at construction time, we can apply
        some optimizations which share commonly used objects.
        Also, this means the input must be UTF-8, so fall back to the

PC/config.c

@@ -60,6 +60,7 @@ extern void init_lsprof(void);
 extern void init_ast(void);
 extern void init_fileio(void);
 extern void initatexit(void);
+extern void _PyWarnings_Init(void);
 
 /* tools/freeze/makeconfig.py marker for additional "extern" */
 /* -- ADDMODULE MARKER 1 -- */
@@ -144,7 +145,8 @@ struct _inittab _PyImport_Inittab[] = {
         {"__main__", NULL},
         {"builtins", NULL},
         {"sys", NULL},
-        
+        {"_warnings", _PyWarnings_Init},
+
         {"_fileio", init_fileio},
         {"atexit", initatexit},
 

PCbuild/pythoncore.vcproj

@@ -1746,6 +1746,10 @@
 				RelativePath="..\Python\traceback.c"
 				>
 			</File>
+                        <File
+                                RelativePath="..\Python\_warnings.c"
+                                >
+                        </File>
 		</Filter>
 		<Filter
 			Name="Resource Files"

Python/errors.c

@@ -673,70 +673,6 @@ PyErr_WriteUnraisable(PyObject *obj)
 
 extern PyObject *PyModule_GetWarningsModule(void);
 
-/* Function to issue a warning message; may raise an exception. */
-int
-PyErr_WarnEx(PyObject *category, const char *message, Py_ssize_t stack_level)
-{
-	PyObject *dict, *func = NULL;
-	PyObject *warnings_module = PyModule_GetWarningsModule();
-
-	if (warnings_module != NULL) {
-		dict = PyModule_GetDict(warnings_module);
-		if (dict != NULL)
-			func = PyDict_GetItemString(dict, "warn");
-	}
-	if (func == NULL) {
-		PySys_WriteStderr("warning: %s\n", message);
-		return 0;
-	}
-	else {
-		PyObject *res;
-
-		if (category == NULL)
-			category = PyExc_RuntimeWarning;
-		res = PyObject_CallFunction(func, "sOn",
-					    message, category, stack_level);
-		if (res == NULL)
-			return -1;
-		Py_DECREF(res);
-		return 0;
-	}
-}
-
-/* Warning with explicit origin */
-int
-PyErr_WarnExplicit(PyObject *category, const char *message,
-		   const char *filename, int lineno,
-		   const char *module, PyObject *registry)
-{
-	PyObject *mod, *dict, *func = NULL;
-
-	mod = PyImport_ImportModuleNoBlock("warnings");
-	if (mod != NULL) {
-		dict = PyModule_GetDict(mod);
-		func = PyDict_GetItemString(dict, "warn_explicit");
-		Py_DECREF(mod);
-	}
-	if (func == NULL) {
-		PySys_WriteStderr("warning: %s\n", message);
-		return 0;
-	}
-	else {
-		PyObject *res;
-
-		if (category == NULL)
-			category = PyExc_RuntimeWarning;
-		if (registry == NULL)
-			registry = Py_None;
-		res = PyObject_CallFunction(func, "sOsizO", message, category,
-					    filename, lineno, module, registry);
-		if (res == NULL)
-			return -1;
-		Py_DECREF(res);
-		return 0;
-	}
-}
-
 
 /* Set file and line information for the current exception.
    If the exception is not a SyntaxError, also sets additional attributes

Python/getopt.c

@@ -83,7 +83,18 @@ int _PyOS_GetOpt(int argc, wchar_t **argv, wchar_t *optstring)
 
 	if ( (option = *opt_ptr++) == L'\0')
 		return -1;
-	
+
+	if (option == 'J') {
+		fprintf(stderr, "-J is reserved for Jython\n");
+		return '_';
+	}
+
+	if (option == 'X') {
+		fprintf(stderr,
+			"-X is reserved for non-standard arguments\n");
+		return '_';
+	}
+
 	if ((ptr = wcschr(optstring, option)) == NULL) {
 		if (_PyOS_opterr)
 		  fprintf(stderr, "Unknown option: -%c\n", (char)option);

Python/pythonrun.c

@@ -84,38 +84,12 @@ int Py_UseClassExceptionsFlag = 1; /* Needed by bltinmodule.c: deprecated */
 int Py_FrozenFlag; /* Needed by getpath.c */
 int Py_IgnoreEnvironmentFlag; /* e.g. PYTHONPATH, PYTHONHOME */
 
-/* Reference to 'warnings' module, to avoid importing it
-   on the fly when the import lock may be held.  See 683658/771097
-*/
-static PyObject *warnings_module = NULL;
-
-/* Returns a borrowed reference to the 'warnings' module, or NULL.
-   If the module is returned, it is guaranteed to have been obtained
-   without acquiring the import lock
-*/
-PyObject *PyModule_GetWarningsModule(void)
-{
-	PyObject *typ, *val, *tb;
-	PyObject *all_modules;
-	/* If we managed to get the module at init time, just use it */
-	if (warnings_module)
-		return warnings_module;
-	/* If it wasn't available at init time, it may be available
-	   now in sys.modules (common scenario is frozen apps: import
-	   at init time fails, but the frozen init code sets up sys.path
-	   correctly, then does an implicit import of warnings for us
-	*/
-	/* Save and restore any exceptions */
-	PyErr_Fetch(&typ, &val, &tb);
-
-	all_modules = PySys_GetObject("modules");
-	if (all_modules) {
-		warnings_module = PyDict_GetItemString(all_modules, "warnings");
-		/* We keep a ref in the global */
-		Py_XINCREF(warnings_module);
-	}
-	PyErr_Restore(typ, val, tb);
-	return warnings_module;
+/* PyModule_GetWarningsModule is no longer necessary as of 2.6
+since _warnings is builtin.  This API should not be used. */
+PyObject *
+PyModule_GetWarningsModule(void)
+{
+	return PyImport_ImportModule("warnings");
 }
 
 static int initialized = 0;
@@ -255,6 +229,15 @@ Py_InitializeEx(int install_sigs)
 
 	if (install_sigs)
 		initsigs(); /* Signal handling stuff, including initintr() */
+		
+    /* Initialize warnings. */
+    _PyWarnings_Init();
+    if (PySys_HasWarnOptions()) {
+        PyObject *warnings_module = PyImport_ImportModule("warnings");
+        if (!warnings_module)
+            PyErr_Clear();
+        Py_XDECREF(warnings_module);
+    }
 
 	initmain(); /* Module __main__ */
 	if (initstdio() < 0)
@@ -268,30 +251,6 @@ Py_InitializeEx(int install_sigs)
 	_PyGILState_Init(interp, tstate);
 #endif /* WITH_THREAD */
 
-	warnings_module = PyImport_ImportModule("warnings");
-	if (!warnings_module) {
-		PyErr_Clear();
-	}
-	else {
-		PyObject *o;
-		char *action[8];
-
-		if (Py_BytesWarningFlag > 1)
-			*action = "error";
-		else if (Py_BytesWarningFlag)
-			*action = "default";
-		else
-			*action = "ignore";
-
-		o = PyObject_CallMethod(warnings_module,
-					"simplefilter", "sO",
-					*action, PyExc_BytesWarning);
-		if (o == NULL)
-			Py_FatalError("Py_Initialize: can't initialize"
-				      "warning filter for BytesWarning.");
-		Py_DECREF(o);
-        }
-
 #if defined(HAVE_LANGINFO_H) && defined(CODESET)
 	/* On Unix, set the file system encoding according to the
 	   user's preference, if the CODESET names a well-known
@@ -403,10 +362,6 @@ Py_Finalize(void)
 	/* Disable signal handling */
 	PyOS_FiniInterrupts();
 
-	/* drop module references we saved */
-	Py_XDECREF(warnings_module);
-	warnings_module = NULL;
-
 	/* Clear type lookup cache */
 	PyType_ClearCache();
 

Python/sysmodule.c

@@ -899,6 +899,12 @@ PySys_AddWarnOption(const wchar_t *s)
 	}
 }
 
+int
+PySys_HasWarnOptions(void)
+{
+    return (warnoptions != NULL && (PyList_Size(warnoptions) > 0)) ? 1 : 0;
+}
+
 /* XXX This doc string is too long to be a single string literal in VC++ 5.0.
    Two literals concatenated works just fine.  If you have a K&R compiler
    or other abomination that however *does* understand longer strings,
@@ -1101,6 +1107,7 @@ static PyStructSequence_Field flags_fields[] = {
 #endif
 	/* {"unbuffered",		"-u"}, */
 	/* {"skip_first",		"-x"}, */
+	{"bytes_warning", "-b"},
 	{0}
 };
 
@@ -1144,6 +1151,7 @@ make_flags(void)
 #endif
 	/* SetFlag(saw_unbuffered_flag); */
 	/* SetFlag(skipfirstline); */
+    SetFlag(Py_BytesWarningFlag);
 #undef SetFlag
 
 	if (PyErr_Occurred()) {

Python/traceback.c

@@ -128,16 +128,16 @@ PyTraceBack_Here(PyFrameObject *frame)
 	return 0;
 }
 
-static int
-tb_displayline(PyObject *f, char *filename, int lineno, char *name)
+int
+Py_DisplaySourceLine(PyObject *f, const char *filename, int lineno)
 {
 	int err = 0;
-	FILE *xfp;
+	FILE *xfp = NULL;
 	char linebuf[2000];
 	int i;
 	char namebuf[MAXPATHLEN+1];
 
-	if (filename == NULL || name == NULL)
+	if (filename == NULL)
 		return -1;
 	/* This is needed by Emacs' compile command */
 #define FMT "  File \"%.500s\", line %d, in %.500s\n"
@@ -145,7 +145,7 @@ tb_displayline(PyObject *f, char *filename, int lineno, char *name)
 	if (xfp == NULL) {
 		/* Search tail of filename in sys.path before giving up */
 		PyObject *path;
-		char *tail = strrchr(filename, SEP);
+		const char *tail = strrchr(filename, SEP);
 		if (tail == NULL)
 			tail = filename;
 		else
@@ -181,14 +181,14 @@ tb_displayline(PyObject *f, char *filename, int lineno, char *name)
 			}
 		}
 	}
-	PyOS_snprintf(linebuf, sizeof(linebuf), FMT, filename, lineno, name);
-	err = PyFile_WriteString(linebuf, f);
-	if (xfp == NULL)
-		return err;
-	else if (err != 0) {
-		fclose(xfp);
-		return err;
-	}
+
+        if (xfp == NULL)
+            return err;
+        if (err != 0) {
+            fclose(xfp);
+            return err;
+        }
+
 	for (i = 0; i < lineno; i++) {
 		char* pLastChar = &linebuf[sizeof(linebuf)-2];
 		do {
@@ -206,22 +206,38 @@ tb_displayline(PyObject *f, char *filename, int lineno, char *name)
 		char *p = linebuf;
 		while (*p == ' ' || *p == '\t' || *p == '\014')
 			p++;
-		err = PyFile_WriteString("    ", f);
-		if (err == 0) {
-			err = PyFile_WriteString(p, f);
-			if (err == 0 && strchr(p, '\n') == NULL)
-				err = PyFile_WriteString("\n", f);
-		}
+                    err = PyFile_WriteString(p, f);
+                    if (err == 0 && strchr(p, '\n') == NULL)
+                            err = PyFile_WriteString("\n", f);
 	}
 	fclose(xfp);
 	return err;
 }
 
 static int
-tb_printinternal(PyTracebackObject *tb, PyObject *f, int limit)
+tb_displayline(PyObject *f, const char *filename, int lineno, const char *name)
+{
+	int err = 0;
+        char linebuf[2000];
+
+	if (filename == NULL || name == NULL)
+		return -1;
+	/* This is needed by Emacs' compile command */
+#define FMT "  File \"%.500s\", line %d, in %.500s\n"
+	PyOS_snprintf(linebuf, sizeof(linebuf), FMT, filename, lineno, name);
+	err = PyFile_WriteString(linebuf, f);
+	if (err != 0)
+		return err;
+
+        err = PyFile_WriteString("    ", f);
+        return Py_DisplaySourceLine(f, filename, lineno);
+}
+
+static int
+tb_printinternal(PyTracebackObject *tb, PyObject *f, long limit)
 {
 	int err = 0;
-	int depth = 0;
+	long depth = 0;
 	PyTracebackObject *tb1 = tb;
 	while (tb1 != NULL) {
 		depth++;
@@ -250,7 +266,7 @@ PyTraceBack_Print(PyObject *v, PyObject *f)
 {
 	int err;
 	PyObject *limitv;
-	int limit = PyTraceBack_LIMIT;
+	long limit = PyTraceBack_LIMIT;
 
 	if (v == NULL)
 		return 0;