NEWS 36 KB
Newer Older
1 2 3 4 5
What's New in Python 2.1 alpha 1?
=================================

Core language, builtins, and interpreter

6
- %[xXo] formats of negative Python longs now produce a sign
7 8 9 10 11 12 13 14
  character.  In 1.6 and earlier, they never produced a sign,
  and raised an error if the value of the long was too large
  to fit in a Python int.  In 2.0, they produced a sign if and
  only if too large to fit in an int.  This was inconsistent
  across platforms (because the size of an int varies across
  platforms), and inconsistent with hex() and oct().  Example:

  >>> "%x" % -0x42L
15
  '-42'      # in 2.1
16 17 18 19
  'ffffffbe' # in 2.0 and before, on 32-bit machines
  >>> hex(-0x42L)
  '-0x42L'   # in all versions of Python

20 21 22 23 24 25 26 27 28 29
  The behavior of %d formats for negative Python longs remains
  the same as in 2.0 (although in 1.6 and before, they raised
  an error if the long didn't fit in a Python int).

  %u formats don't make sense for Python longs, but are allowed
  and treated the same as %d in 2.1.  In 2.0, a negative long
  formatted via %u produced a sign if and only if too large to
  fit in an int.  In 1.6 and earlier, a negative long formatted
  via %u raised an error if it was too big to fit in an int.

Guido van Rossum's avatar
Guido van Rossum committed
30 31 32 33 34 35 36
- Dictionary objects have an odd new method, popitem().  This removes
  an arbitrary item from the dictionary and returns it (in the form of
  a (key, value) pair).  This can be useful for algorithms that use a
  dictionary as a bag of "to do" items and repeatedly need to pick one
  item.  Such algorithms normally end up running in quadratic time;
  using popitem() they can usually be made to run in linear time.

37 38 39 40 41 42 43 44
Windows changes

- select module:  By default under Windows, a select() call
  can specify no more than 64 sockets.  Python now boosts
  this Microsoft default to 512.  If you need even more than
  that, see the MS docs (you'll need to #define FD_SETSIZE
  and recompile Python from source).

45

46
What's New in Python 2.0?
47
=========================
48

49
Below is a list of all relevant changes since release 1.6.  Older
Guido van Rossum's avatar
Guido van Rossum committed
50 51 52
changes are in the file HISTORY.  If you are making the jump directly
from Python 1.5.2 to 2.0, make sure to read the section for 1.6 in the
HISTORY file!  Many important changes listed there.
53

Guido van Rossum's avatar
Guido van Rossum committed
54 55 56
Alternatively, a good overview of the changes between 1.5.2 and 2.0 is
the document "What's New in Python 2.0" by Kuchling and Moshe Zadka:
http://starship.python.net/crew/amk/python/writing/new-python/.
57

Guido van Rossum's avatar
Guido van Rossum committed
58
--Guido van Rossum (home page: http://www.pythonlabs.com/~guido/)
59 60

======================================================================
61

62 63 64 65 66 67 68 69 70 71 72 73 74 75
What's new in 2.0 (since release candidate 1)?
==============================================

Standard library

- The copy_reg module was modified to clarify its intended use: to
  register pickle support for extension types, not for classes.
  pickle() will raise a TypeError if it is passed a class.

- Fixed a bug in gettext's "normalize and expand" code that prevented
  it from finding an existing .mo file.

- Restored support for HTTP/0.9 servers in httplib.

76 77 78 79 80
- The math module was changed to stop raising OverflowError in case of
  underflow, and return 0 instead in underflow cases.  Whether Python
  used to raise OverflowError in case of underflow was platform-
  dependent (it did when the platform math library set errno to ERANGE
  on underflow).
81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

- Fixed a bug in StringIO that occurred when the file position was not
  at the end of the file and write() was called with enough data to
  extend past the end of the file.

- Fixed a bug that caused Tkinter error messages to get lost on
  Windows.  The bug was fixed by replacing direct use of
  interp->result with Tcl_GetStringResult(interp).

- Fixed bug in urllib2 that caused it to fail when it received an HTTP
  redirect response.

- Several changes were made to distutils: Some debugging code was
  removed from util.  Fixed the installer used when an external zip
  program (like WinZip) is not found; the source code for this
  installer is in Misc/distutils.  check_lib() was modified to behave
  more like AC_CHECK_LIB by add other_libraries() as a parameter.  The
  test for whether installed modules are on sys.path was changed to
  use both normcase() and normpath().

Jeremy Hylton's avatar
Jeremy Hylton committed
101 102
- Several minor bugs were fixed in the xml package (the minidom,
  pulldom, expatreader, and saxutils modules).
103 104 105 106 107

- The regression test driver (regrtest.py) behavior when invoked with
  -l changed: It now reports a count of objects that are recognized as
  garbage but not freed by the garbage collector.

108 109 110 111 112
- The regression test for the math module was changed to test
  exceptional behavior when the test is run in verbose mode.  Python
  cannot yet guarantee consistent exception behavior across platforms,
  so the exception part of test_math is run only in verbose mode, and
  may fail on your platform.
113 114 115 116 117 118 119 120 121 122 123

Internals

- PyOS_CheckStack() has been disabled on Win64, where it caused
  test_sre to fail.

Build issues

- Changed compiler flags, so that gcc is always invoked with -Wall and
  -Wstrict-prototypes.  Users compiling Python with GCC should see
  exactly one warning, except if they have passed configure the
124
  --with-pydebug flag.  The expected warning is for getopt() in
Tim Peters's avatar
Tim Peters committed
125
  Modules/main.c.  This warning will be fixed for Python 2.1.
126

127
- Fixed configure to add -threads argument during linking on OSF1.
128 129 130 131 132 133 134 135

Tools and other miscellany

- The compiler in Tools/compiler was updated to support the new
  language features introduced in 2.0: extended print statement, list
  comprehensions, and augmented assignments.  The new compiler should
  also be backwards compatible with Python 1.5.2; the compiler will
  always generate code for the version of the interpreter it runs
136
  under.
137

138 139 140
What's new in 2.0 release candidate 1 (since beta 2)?
=====================================================

141 142 143 144 145 146 147 148 149 150
What is release candidate 1?

We believe that release candidate 1 will fix all known bugs that we
intend to fix for the 2.0 final release.  This release should be a bit
more stable than the previous betas.  We would like to see even more
widespread testing before the final release, so we are producing this
release candidate.  The final release will be exactly the same unless
any show-stopping (or brown bag) bugs are found by testers of the
release candidate.

151
All the changes since the last beta release are bug fixes or changes
152
to support building Python for specific platforms.
153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178

Core language, builtins, and interpreter

- A bug that caused crashes when __coerce__ was used with augmented
  assignment, e.g. +=, was fixed.

- Raise ZeroDivisionError when raising zero to a negative number,
  e.g. 0.0 ** -2.0.  Note that math.pow is unrelated to the builtin
  power operator and the result of math.pow(0.0, -2.0) will vary by
  platform.  On Linux, it raises a ValueError.

- A bug in Unicode string interpolation was fixed that occasionally
  caused errors with formats including "%%".  For example, the
  following expression "%% %s" % u"abc" no longer raises a TypeError.

- Compilation of deeply nested expressions raises MemoryError instead
  of SyntaxError, e.g. eval("[" * 50 + "]" * 50).

- In 2.0b2 on Windows, the interpreter wrote .pyc files in text mode,
  rendering them useless.  They are now written in binary mode again.

Standard library

- Keyword arguments are now accepted for most pattern and match object
  methods in SRE, the standard regular expression engine.

179
- In SRE, fixed error with negative lookahead and lookbehind that
Jeremy Hylton's avatar
typo  
Jeremy Hylton committed
180
  manifested itself as a runtime error in patterns like "(?<!abc)(def)".
181

182 183
- Several bugs in the Unicode handling and error handling in _tkinter
  were fixed.
184 185 186 187 188 189 190

- Fix memory management errors in Merge() and Tkapp_Call() routines.

- Several changes were made to cStringIO to make it compatible with
  the file-like object interface and with StringIO.  If operations are
  performed on a closed object, an exception is raised.  The truncate
  method now accepts a position argument and readline accepts a size
191
  argument.
192 193 194

- There were many changes made to the linuxaudiodev module and its
  test suite; as a result, a short, unexpected audio sample should now
195
  play when the regression test is run.
196 197 198

  Note that this module is named poorly, because it should work
  correctly on any platform that supports the Open Sound System
199
  (OSS).
200 201 202 203 204 205 206 207 208 209 210 211 212

  The module now raises exceptions when errors occur instead of
  crashing.  It also defines the AFMT_A_LAW format (logarithmic A-law
  audio) and defines a getptr() method that calls the
  SNDCTL_DSP_GETxPTR ioctl defined in the OSS Programmer's Guide.

- The library_version attribute, introduced in an earlier beta, was
  removed because it can not be supported with early versions of the C
  readline library, which provides no way to determine the version at
  compile-time.

- The binascii module is now enabled on Win64.

213 214 215
- tokenize.py no longer suffers "recursion depth" errors when parsing
  programs with very long string literals.

216 217
Internals

218
- Fixed several buffer overflow vulnerabilities in calculate_path(),
219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246
  which is called when the interpreter starts up to determine where
  the standard library is installed.  These vulnerabilities affect all
  previous versions of Python and can be exploited by setting very
  long values for PYTHONHOME or argv[0].  The risk is greatest for a
  setuid Python script, although use of the wrapper in
  Misc/setuid-prog.c will eliminate the vulnerability.

- Fixed garbage collection bugs in instance creation that were
  triggered when errors occurred during initialization.  The solution,
  applied in cPickle and in PyInstance_New(), is to call
  PyObject_GC_Init() after the initialization of the object's
  container attributes is complete.

- pyexpat adds definitions of PyModule_AddStringConstant and
  PyModule_AddObject if the Python version is less than 2.0, which
  provides compatibility with PyXML on Python 1.5.2.

- If the platform has a bogus definition for LONG_BIT (the number of
  bits in a long), an error will be reported at compile time.

- Fix bugs in _PyTuple_Resize() which caused hard-to-interpret garbage
  collection crashes and possibly other, unreported crashes.

- Fixed a memory leak in _PyUnicode_Fini().

Build issues

- configure now accepts a --with-suffix option that specifies the
247
  executable suffix.  This is useful for builds on Cygwin and Mac OS
248
  X, for example.
249 250 251 252 253 254 255 256 257 258

- The mmap.PAGESIZE constant is now initialized using sysconf when
  possible, which eliminates a dependency on -lucb for Reliant UNIX.

- The md5 file should now compile on all platforms.

- The select module now compiles on platforms that do not define
  POLLRDNORM and related constants.

- Darwin (Mac OS X):  Initial support for static builds on this
259
  platform.
260

261 262 263 264 265
- BeOS: A number of changes were made to the build and installation
  process.  ar-fake now operates on a directory of object files.
  dl_export.h is gone, and its macros now appear on the mwcc command
  line during build on PPC BeOS.

266
- Platform directory in lib/python2.0 is "plat-beos5" (or
267
  "plat-beos4", if building on BeOS 4.5), rather than "plat-beos".
268 269 270 271 272 273 274 275 276

- Cygwin: Support for shared libraries, Tkinter, and sockets.

- SunOS 4.1.4_JL: Fix test for directory existence in configure.

Tools and other miscellany

- Removed debugging prints from main used with freeze.

277 278
- IDLE auto-indent no longer crashes when it encounters Unicode
  characters.
279 280 281 282 283 284

What's new in 2.0 beta 2 (since beta 1)?
========================================

Core language, builtins, and interpreter

Tim Peters's avatar
Tim Peters committed
285
- Add support for unbounded ints in %d,i,u,x,X,o formats; for example
286 287 288 289
  "%d" % 2L**64 == "18446744073709551616".

- Add -h and -V command line options to print the usage message and
  Python version number and exit immediately.
290

291 292 293 294 295 296 297 298 299 300 301
- eval() and exec accept Unicode objects as code parameters.

- getattr() and setattr() now also accept Unicode objects for the
  attribute name, which are converted to strings using the default
  encoding before lookup.

- Multiplication on string and Unicode now does proper bounds
  checking; e.g. 'a' * 65536 * 65536 will raise ValueError, "repeated
  string is too long."

- Better error message when continue is found in try statement in a
302
  loop.
303

304 305 306

Standard library and extensions

307
- array: reverse() method of array now works.  buffer_info() now does
308 309
  argument checking; it still takes no arguments.

310
- asyncore/asynchat: Included most recent version from Sam Rushing.
311

312
- cgi: Accept '&' or ';' as separator characters when parsing form data.
313

314
- CGIHTTPServer: Now works on Windows (and perhaps even Mac).
315 316

- ConfigParser: When reading the file, options spelled in upper case
317
  letters are now correctly converted to lowercase.
318 319 320

- copy: Copy Unicode objects atomically.

321
- cPickle: Fail gracefully when copy_reg can't be imported.
322

323
- cStringIO: Implemented readlines() method.
324

325 326 327 328 329
- dbm: Add get() and setdefault() methods to dbm object.  Add constant
  `library' to module that names the library used.  Added doc strings
  and method names to error messages.  Uses configure to determine
  which ndbm.h file to include; Berkeley DB's nbdm and GDBM's ndbm is
  now available options.
330 331 332 333 334 335 336 337 338 339 340

- distutils: Update to version 0.9.3.

- dl: Add several dl.RTLD_ constants.

- fpectl: Now supported on FreeBSD.

- gc: Add DEBUG_SAVEALL option.  When enabled all garbage objects
  found by the collector will be saved in gc.garbage.  This is useful
  for debugging a program that creates reference cycles.

341
- httplib: Three changes: Restore support for set_debuglevel feature
342 343 344
  of HTTP class.  Do not close socket on zero-length response.  Do not
  crash when server sends invalid content-length header.

345
- mailbox: Mailbox class conforms better to qmail specifications.
346

347 348 349 350 351
- marshal: When reading a short, sign-extend on platforms where shorts
  are bigger than 16 bits.  When reading a long, repair the unportable
  sign extension that was being done for 64-bit machines.  (It assumed
  that signed right shift sign-extends.)

352 353
- operator: Add contains(), invert(), __invert__() as aliases for
  __contains__(), inv(), and __inv__() respectively.
354

355 356
- os: Add support for popen2() and popen3() on all platforms where
  fork() exists.  (popen4() is still in the works.)
357

358
- os: (Windows only:) Add startfile() function that acts like double-
Tim Peters's avatar
Tim Peters committed
359 360
  clicking on a file in Explorer (or passing the file name to the
  DOS "start" command).
361

362
- os.path: (Windows, DOS:) Treat trailing colon correctly in
Tim Peters's avatar
Tim Peters committed
363
  os.path.join.  os.path.join("a:", "b") yields "a:b".
364 365 366 367 368

- pickle: Now raises ValueError when an invalid pickle that contains
  a non-string repr where a string repr was expected.  This behavior
  matches cPickle.

369
- posixfile: Remove broken __del__() method.
370

371
- py_compile: support CR+LF line terminators in source file.
372 373

- readline: Does not immediately exit when ^C is hit when readline and
374
  threads are configured.  Adds definition of rl_library_version.  (The
375
  latter addition requires GNU readline 2.2 or later.)
376 377

- rfc822: Domain literals returned by AddrlistClass method
378
  getdomainliteral() are now properly wrapped in brackets.
379 380

- site: sys.setdefaultencoding() should only be called in case the
Tim Peters's avatar
Tim Peters committed
381
  standard default encoding ("ascii") is changed. This saves quite a
382 383 384 385
  few cycles during startup since the first call to
  setdefaultencoding() will initialize the codec registry and the
  encodings package.

386 387
- socket: Support for size hint in readlines() method of object returned
  by makefile().
388

389
- sre: Added experimental expand() method to match objects.  Does not
Jeremy Hylton's avatar
typo  
Jeremy Hylton committed
390
  use buffer interface on Unicode strings.  Does not hang if group id
391 392
  is followed by whitespace.

393
- StringIO: Size hint in readlines() is now supported as documented.
394 395 396 397

- struct: Check ranges for bytes and shorts.

- urllib: Improved handling of win32 proxy settings. Fixed quote and
398
  quote_plus functions so that the always encode a comma.
399 400 401 402 403

- Tkinter: Image objects are now guaranteed to have unique ids.  Set
  event.delta to zero if Tk version doesn't support mousewheel.
  Removed some debugging prints.

404
- UserList: now implements __contains__().
405

406
- webbrowser: On Windows, use os.startfile() instead of os.popen(),
407 408
  which works around a bug in Norton AntiVirus 2000 that leads directly
  to a Blue Screen freeze.
409 410 411 412

- xml: New version detection code allows PyXML to override standard
  XML package if PyXML version is greater than 0.6.1.

413 414 415 416 417
- xml.dom: DOM level 1 support for basic XML.  Includes xml.dom.minidom
  (conventional DOM), and xml.dom.pulldom, which allows building the DOM
  tree only for nodes which are sufficiently interesting to a specific
  application.  Does not provide the HTML-specific extensions.  Still
  undocumented.
418

419 420 421
- xml.sax: SAX 2 support for Python, including all the handler
  interfaces needed to process XML 1.0 compliant XML.  Some
  documentation is already available.
422

423 424
- pyexpat: Renamed to xml.parsers.expat since this is part of the new,
  packagized XML support.
425

426

427 428 429 430 431 432
C API

- Add three new convenience functions for module initialization --
  PyModule_AddObject(), PyModule_AddIntConstant(), and
  PyModule_AddStringConstant().

433
- Cleaned up definition of NULL in C source code; all definitions were
434 435 436
  removed and add #error to Python.h if NULL isn't defined after
  #include of stdio.h.

437
- Py_PROTO() macros that were removed in 2.0b1 have been restored for
438 439
  backwards compatibility (at the source level) with old extensions.

440 441 442 443
- A wrapper API was added for signal() and sigaction().  Instead of
  either function, always use PyOS_getsig() to get a signal handler
  and PyOS_setsig() to set one.  A new convenience typedef
  PyOS_sighandler_t is defined for the type of signal handlers.
444

445
- Add PyString_AsStringAndSize() function that provides access to the
446 447 448
  internal data buffer and size of a string object -- or the default
  encoded version of a Unicode object.

449 450
- PyString_Size() and PyString_AsString() accept Unicode objects.

451
- The standard header <limits.h> is now included by Python.h (if it
452 453
  exists).  INT_MAX and LONG_MAX will always be defined, even if
  <limits.h> is not available.
454

455 456 457 458 459 460 461 462 463 464 465 466
- PyFloat_FromString takes a second argument, pend, that was
  effectively useless.  It is now officially useless but preserved for
  backwards compatibility.  If the pend argument is not NULL, *pend is
  set to NULL.

- PyObject_GetAttr() and PyObject_SetAttr() now accept Unicode objects
  for the attribute name.  See note on getattr() above.

- A few bug fixes to argument processing for Unicode.
  PyArg_ParseTupleAndKeywords() now accepts "es#" and "es".
  PyArg_Parse() special cases "s#" for Unicode objects; it returns a
  pointer to the default encoded string data instead of to the raw
467
  UTF-16.
468 469 470

- Py_BuildValue accepts B format (for bgen-generated code).

471

472 473 474 475 476
Internals

- On Unix, fix code for finding Python installation directory so that
  it works when argv[0] is a relative path.

477
- Added a true tnicode_internal_encode() function and fixed the
478
  unicode_internal_decode function() to support Unicode objects directly
479 480
  rather than by generating a copy of the object.

Tim Peters's avatar
Tim Peters committed
481 482
- Several of the internal Unicode tables are much smaller now, and
  the source code should be much friendlier to weaker compilers.
483

Jeremy Hylton's avatar
Jeremy Hylton committed
484 485 486 487
- In the garbage collector: Fixed bug in collection of tuples.  Fixed
  bug that caused some instances to be removed from the container set
  while they were still live.  Fixed parsing in gc.set_debug() for
  platforms where sizeof(long) > sizeof(int).
488 489 490 491 492 493 494 495

- Fixed refcount problem in instance deallocation that only occurred
  when Py_REF_DEBUG was defined and Py_TRACE_REFS was not.

- On Windows, getpythonregpath is now protected against null data in
  registry key.

- On Unix, create .pyc/.pyo files with O_EXCL flag to avoid a race
496
  condition.
497

498

499 500 501 502
Build and platform-specific issues

- Better support of GNU Pth via --with-pth configure option.

503 504
- Python/C API now properly exposed to dynamically-loaded extension
  modules on Reliant UNIX.
505 506 507 508 509

- Changes for the benefit of SunOS 4.1.4 (really!).  mmapmodule.c:
  Don't define MS_SYNC to be zero when it is undefined.  Added missing
  prototypes in posixmodule.c.

510
- Improved support for HP-UX build.  Threads should now be correctly
511
  configured (on HP-UX 10.20 and 11.00).
512

513 514 515 516 517 518 519 520 521 522
- Fix largefile support on older NetBSD systems and OpenBSD by adding
  define for TELL64.


Tools and other miscellany

- ftpmirror: Call to main() is wrapped in if __name__ == "__main__".

- freeze: The modulefinder now works with 2.0 opcodes.

523
- IDLE:
524 525 526 527
  Move hackery of sys.argv until after the Tk instance has been
  created, which allows the application-specific Tkinter
  initialization to be executed if present; also pass an explicit
  className parameter to the Tk() constructor.
528

529 530 531

What's new in 2.0 beta 1?
=========================
532

Guido van Rossum's avatar
Guido van Rossum committed
533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562
Source Incompatibilities
------------------------

None.  Note that 1.6 introduced several incompatibilities with 1.5.2,
such as single-argument append(), connect() and bind(), and changes to
str(long) and repr(float).


Binary Incompatibilities
------------------------

- Third party extensions built for Python 1.5.x or 1.6 cannot be used
with Python 2.0; these extensions will have to be rebuilt for Python
2.0.

- On Windows, attempting to import a third party extension built for
Python 1.5.x or 1.6 results in an immediate crash; there's not much we
can do about this.  Check your PYTHONPATH environment variable!

- Python bytecode files (*.pyc and *.pyo) are not compatible between
releases.


Overview of Changes Since 1.6
-----------------------------

There are many new modules (including brand new XML support through
the xml package, and i18n support through the gettext module); a list
of all new modules is included below.  Lots of bugs have been fixed.

563 564 565 566
The process for making major new changes to the language has changed
since Python 1.6.  Enhancements must now be documented by a Python
Enhancement Proposal (PEP) before they can be accepted.

Guido van Rossum's avatar
Guido van Rossum committed
567 568 569 570 571 572 573 574 575 576 577 578 579 580 581
There are several important syntax enhancements, described in more
detail below:

  - Augmented assignment, e.g. x += 1

  - List comprehensions, e.g. [x**2 for x in range(10)]

  - Extended import statement, e.g. import Module as Name

  - Extended print statement, e.g. print >> file, "Hello"

Other important changes:

  - Optional collection of cyclical garbage

582 583 584 585 586 587 588 589 590 591 592 593 594 595 596
Python Enhancement Proposal (PEP)
---------------------------------

PEP stands for Python Enhancement Proposal.  A PEP is a design
document providing information to the Python community, or describing
a new feature for Python.  The PEP should provide a concise technical
specification of the feature and a rationale for the feature.

We intend PEPs to be the primary mechanisms for proposing new
features, for collecting community input on an issue, and for
documenting the design decisions that have gone into Python.  The PEP
author is responsible for building consensus within the community and
documenting dissenting opinions.

The PEPs are available at http://python.sourceforge.net/peps/.
Guido van Rossum's avatar
Guido van Rossum committed
597 598 599 600 601 602 603

Augmented Assignment
--------------------

This must have been the most-requested feature of the past years!
Eleven new assignment operators were added:

604
    += -= *= /= %= **= <<= >>= &= ^= |=
Guido van Rossum's avatar
Guido van Rossum committed
605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640

For example,

    A += B

is similar to

    A = A + B

except that A is evaluated only once (relevant when A is something
like dict[index].attr).

However, if A is a mutable object, A may be modified in place.  Thus,
if A is a number or a string, A += B has the same effect as A = A+B
(except A is only evaluated once); but if a is a list, A += B has the
same effect as A.extend(B)!

Classes and built-in object types can override the new operators in
order to implement the in-place behavior; the not-in-place behavior is
used automatically as a fallback when an object doesn't implement the
in-place behavior.  For classes, the method name is derived from the
method name for the corresponding not-in-place operator by inserting
an 'i' in front of the name, e.g. __iadd__ implements in-place
__add__.

Augmented assignment was implemented by Thomas Wouters.


List Comprehensions
-------------------

This is a flexible new notation for lists whose elements are computed
from another list (or lists).  The simplest form is:

    [<expression> for <variable> in <sequence>]

641
For example, [i**2 for i in range(4)] yields the list [0, 1, 4, 9].
642
This is more efficient than a for loop with a list.append() call.
Guido van Rossum's avatar
Guido van Rossum committed
643 644 645 646 647 648 649

You can also add a condition:

    [<expression> for <variable> in <sequence> if <condition>]

For example, [w for w in words if w == w.lower()] would yield the list
of words that contain no uppercase characters.  This is more efficient
650
than a for loop with an if statement and a list.append() call.
Guido van Rossum's avatar
Guido van Rossum committed
651 652 653 654 655 656 657 658 659 660 661 662 663 664

You can also have nested for loops and more than one 'if' clause.  For
example, here's a function that flattens a sequence of sequences::

    def flatten(seq):
        return [x for subseq in seq for x in subseq]

    flatten([[0], [1,2,3], [4,5], [6,7,8,9], []])

This prints

    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

List comprehensions originated as a patch set from Greg Ewing; Skip
665
Montanaro and Thomas Wouters also contributed.  Described by PEP 202.
Guido van Rossum's avatar
Guido van Rossum committed
666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694


Extended Import Statement
-------------------------

Many people have asked for a way to import a module under a different
name.  This can be accomplished like this:

    import foo
    bar = foo
    del foo

but this common idiom gets old quickly.  A simple extension of the
import statement now allows this to be written as follows:

    import foo as bar

There's also a variant for 'from ... import':

    from foo import bar as spam

This also works with packages; e.g. you can write this:

    import test.regrtest as regrtest

Note that 'as' is not a new keyword -- it is recognized only in this
context (this is only possible because the syntax for the import
statement doesn't involve expressions).

695
Implemented by Thomas Wouters.  Described by PEP 221.
Guido van Rossum's avatar
Guido van Rossum committed
696 697 698 699 700 701 702 703 704 705 706 707 708 709 710


Extended Print Statement
------------------------

Easily the most controversial new feature, this extension to the print
statement adds an option to make the output go to a different file
than the default sys.stdout.

For example, to write an error message to sys.stderr, you can now
write:

    print >> sys.stderr, "Error: bad dog!"

As a special feature, if the expression used to indicate the file
711
evaluates to None, the current value of sys.stdout is used.  Thus:
Guido van Rossum's avatar
Guido van Rossum committed
712 713 714 715 716 717 718

    print >> None, "Hello world"

is equivalent to

    print "Hello world"

719
Design and implementation by Barry Warsaw.  Described by PEP 214.
Guido van Rossum's avatar
Guido van Rossum committed
720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737


Optional Collection of Cyclical Garbage
---------------------------------------

Python is now equipped with a garbage collector that can hunt down
cyclical references between Python objects.  It's no replacement for
reference counting; in fact, it depends on the reference counts being
correct, and decides that a set of objects belong to a cycle if all
their reference counts can be accounted for from their references to
each other.  This devious scheme was first proposed by Eric Tiedemann,
and brought to implementation by Neil Schemenauer.

There's a module "gc" that lets you control some parameters of the
garbage collection.  There's also an option to the configure script
that lets you enable or disable the garbage collection.  In 2.0b1,
it's on by default, so that we (hopefully) can collect decent user
experience with this new feature.  There are some questions about its
738
performance.  If it proves to be too much of a problem, we'll turn it
Guido van Rossum's avatar
Guido van Rossum committed
739 740 741 742 743 744 745 746 747 748
off by default in the final 2.0 release.


Smaller Changes
---------------

A new function zip() was added.  zip(seq1, seq2, ...) is equivalent to
map(None, seq1, seq2, ...) when the sequences have the same length;
i.e. zip([1,2,3], [10,20,30]) returns [(1,10), (2,20), (3,30)].  When
the lists are not all the same length, the shortest list wins:
749
zip([1,2,3], [10,20]) returns [(1,10), (2,20)].  See PEP 201.
Guido van Rossum's avatar
Guido van Rossum committed
750 751 752 753 754 755 756 757 758 759 760 761 762 763 764

sys.version_info is a tuple (major, minor, micro, level, serial).

Dictionaries have an odd new method, setdefault(key, default).
dict.setdefault(key, default) returns dict[key] if it exists; if not,
it sets dict[key] to default and returns that value.  Thus:

    dict.setdefault(key, []).append(item)

does the same work as this common idiom:

    if not dict.has_key(key):
        dict[key] = []
    dict[key].append(item)

765 766 767 768 769
There are two new variants of SyntaxError that are raised for
indentation-related errors: IndentationError and TabError.

Changed \x to consume exactly two hex digits; see PEP 223.  Added \U
escape that consumes exactly eight hex digits.
Jeremy Hylton's avatar
Jeremy Hylton committed
770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786

The limits on the size of expressions and file in Python source code
have been raised from 2**16 to 2**32.  Previous versions of Python
were limited because the maximum argument size the Python VM accepted
was 2**16.  This limited the size of object constructor expressions,
e.g. [1,2,3] or {'a':1, 'b':2}, and the size of source files.  This
limit was raised thanks to a patch by Charles Waldman that effectively
fixes the problem.  It is now much more likely that you will be
limited by available memory than by an arbitrary limit in Python.

The interpreter's maximum recursion depth can be modified by Python
programs using sys.getrecursionlimit and sys.setrecursionlimit.  This
limit is the maximum number of recursive calls that can be made by
Python code.  The limit exists to prevent infinite recursion from
overflowing the C stack and causing a core dump.  The default value is
1000.  The maximum safe value for a particular platform can be found
by running Misc/find_recursionlimit.py.
Guido van Rossum's avatar
Guido van Rossum committed
787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808

New Modules and Packages
------------------------

atexit - for registering functions to be called when Python exits.

imputil - Greg Stein's alternative API for writing custom import
hooks.

pyexpat - an interface to the Expat XML parser, contributed by Paul
Prescod.

xml - a new package with XML support code organized (so far) in three
subpackages: xml.dom, xml.sax, and xml.parsers.  Describing these
would fill a volume.  There's a special feature whereby a
user-installed package named _xmlplus overrides the standard
xmlpackage; this is intended to give the XML SIG a hook to distribute
backwards-compatible updates to the standard xml package.

webbrowser - a platform-independent API to launch a web browser.


809 810 811
Changed Modules
---------------

812 813 814 815 816 817
array -- new methods for array objects: count, extend, index, pop, and
remove

binascii -- new functions b2a_hex and a2b_hex that convert between
binary data and its hex representation

Jeremy Hylton's avatar
Jeremy Hylton committed
818 819 820 821 822 823 824 825 826 827 828 829 830
calendar -- Many new functions that support features including control
over which day of the week is the first day, returning strings instead
of printing them.  Also new symbolic constants for days of week,
e.g. MONDAY, ..., SUNDAY.

cgi -- FieldStorage objects have a getvalue method that works like a
dictionary's get method and returns the value attribute of the object.

ConfigParser -- The parser object has new methods has_option,
remove_section, remove_option, set, and write.  They allow the module
to be used for writing config files as well as reading them.

ftplib -- ntransfercmd(), transfercmd(), and retrbinary() all now
831 832
optionally support the RFC 959 REST command.

Jeremy Hylton's avatar
Jeremy Hylton committed
833 834 835 836 837
gzip -- readline and readlines now accept optional size arguments

httplib -- New interfaces and support for HTTP/1.1 by Greg Stein.  See
the module doc strings for details.

838 839 840 841 842 843 844
locale -- implement getdefaultlocale for Win32 and Macintosh

marshal -- no longer dumps core when marshaling deeply nested or
recursive data structures

os -- new functions isatty, seteuid, setegid, setreuid, setregid

Jeremy Hylton's avatar
Jeremy Hylton committed
845 846 847
os/popen2 -- popen2/popen3/popen4 support under Windows.  popen2/popen3
support under Unix.

848
os/pty -- support for openpty and forkpty
Jeremy Hylton's avatar
Jeremy Hylton committed
849 850 851 852 853 854

os.path -- fix semantics of os.path.commonprefix

smtplib -- support for sending very long messages

socket -- new function getfqdn()
855

Jeremy Hylton's avatar
Jeremy Hylton committed
856 857 858
readline -- new functions to read, write and truncate history files.
The readline section of the library reference manual contains an
example.
859

860 861
select -- add interface to poll system call

Jeremy Hylton's avatar
Jeremy Hylton committed
862 863 864 865 866
shutil -- new copyfileobj function

SimpleHTTPServer, CGIHTTPServer -- Fix problems with buffering in the
HTTP server.

867
Tkinter -- optimization of function flatten
Jeremy Hylton's avatar
Jeremy Hylton committed
868 869

urllib -- scans environment variables for proxy configuration,
870
e.g. http_proxy.
Jeremy Hylton's avatar
Jeremy Hylton committed
871 872

whichdb -- recognizes dumbdbm format
873 874 875 876 877 878 879 880 881 882 883 884 885


Obsolete Modules
----------------

None.  However note that 1.6 made a whole slew of modules obsolete:
stdwin, soundex, cml, cmpcache, dircache, dump, find, grep, packmail,
poly, zmod, strop, util, whatsound.


Changed, New, Obsolete Tools
----------------------------

886
None.
887 888


Guido van Rossum's avatar
Guido van Rossum committed
889 890 891 892 893 894 895 896 897 898 899 900 901 902 903
C-level Changes
---------------

Several cleanup jobs were carried out throughout the source code.

All C code was converted to ANSI C; we got rid of all uses of the
Py_PROTO() macro, which makes the header files a lot more readable.

Most of the portability hacks were moved to a new header file,
pyport.h; several other new header files were added and some old
header files were removed, in an attempt to create a more rational set
of header files.  (Few of these ever need to be included explicitly;
they are all included by Python.h.)

Trent Mick ensured portability to 64-bit platforms, under both Linux
904 905
and Win64, especially for the new Intel Itanium processor.  Mick also
added large file support for Linux64 and Win64.
Guido van Rossum's avatar
Guido van Rossum committed
906

Jeremy Hylton's avatar
Jeremy Hylton committed
907 908 909 910 911 912 913
The C APIs to return an object's size have been update to consistently
use the form PyXXX_Size, e.g. PySequence_Size and PyDict_Size.  In
previous versions, the abstract interfaces used PyXXX_Length and the
concrete interfaces used PyXXX_Size.  The old names,
e.g. PyObject_Length, are still available for backwards compatibility
at the API level, but are deprecated.

914 915 916
The PyOS_CheckStack function has been implemented on Windows by
Fredrik Lundh.  It prevents Python from failing with a stack overflow
on Windows.
Jeremy Hylton's avatar
Jeremy Hylton committed
917 918 919

The GC changes resulted in creation of two new slots on object,
tp_traverse and tp_clear.  The augmented assignment changes result in
Guido van Rossum's avatar
Guido van Rossum committed
920
the creation of a new slot for each in-place operator.
Jeremy Hylton's avatar
Jeremy Hylton committed
921 922

The GC API creates new requirements for container types implemented in
Guido van Rossum's avatar
Guido van Rossum committed
923
C extension modules.  See Include/objimpl.h for details.
Jeremy Hylton's avatar
Jeremy Hylton committed
924

925 926 927 928 929
PyErr_Format has been updated to automatically calculate the size of
the buffer needed to hold the formatted result string.  This change
prevents crashes caused by programmer error.

New C API calls: PyObject_AsFileDescriptor, PyErr_WriteUnraisable.
Jeremy Hylton's avatar
Jeremy Hylton committed
930

931 932 933
PyRun_AnyFileEx, PyRun_SimpleFileEx, PyRun_FileEx -- New functions
that are the same as their non-Ex counterparts except they take an
extra flag argument that tells them to close the file when done.
934

935
XXX There were other API changes that should be fleshed out here.
936

937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952

Windows Changes
---------------

New popen2/popen3/peopen4 in os module (see Changed Modules above).

os.popen is much more usable on Windows 95 and 98.  See Microsoft
Knowledge Base article Q150956.  The Win9x workaround described there
is implemented by the new w9xpopen.exe helper in the root of your
Python installation.  Note that Python uses this internally; it is not
a standalone program.

Administrator privileges are no longer required to install Python
on Windows NT or Windows 2000.  If you have administrator privileges,
Python's registry info will be written under HKEY_LOCAL_MACHINE.
Otherwise the installer backs off to writing Python's registry info
Guido van Rossum's avatar
Guido van Rossum committed
953
under HKEY_CURRENT_USER.  The latter is sufficient for all "normal"
954 955 956 957 958 959 960 961 962 963 964 965 966 967
uses of Python, but will prevent some advanced uses from working
(for example, running a Python script as an NT service, or possibly
from CGI).

[This was new in 1.6] The installer no longer runs a separate Tcl/Tk
installer; instead, it installs the needed Tcl/Tk files directly in the
Python directory.  If you already have a Tcl/Tk installation, this
wastes some disk space (about 4 Megs) but avoids problems with
conflicting Tcl/Tk installations, and makes it much easier for Python
to ensure that Tcl/Tk can find all its files.

[This was new in 1.6] The Windows installer now installs by default in
\Python20\ on the default volume, instead of \Program Files\Python-2.0\.

968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988

Updates to the changes between 1.5.2 and 1.6
--------------------------------------------

The 1.6 NEWS file can't be changed after the release is done, so here
is some late-breaking news:

New APIs in locale.py: normalize(), getdefaultlocale(), resetlocale(),
and changes to getlocale() and setlocale().

The new module is now enabled per default.

It is not true that the encodings codecs cannot be used for normal
strings: the string.encode() (which is also present on 8-bit strings
!) allows using them for 8-bit strings too, e.g. to convert files from
cp1252 (Windows) to latin-1 or vice-versa.

Japanese codecs are available from Tamito KAJIYAMA:
http://pseudo.grad.sccs.chukyo-u.ac.jp/~kajiyama/python/


989
======================================================================