merge the io-c branch: C implementation of the io module

The main io module now uses the C implementation. The Python one still exists in Lib/_pyio.py for ease of testing new features and usefulness to other implementers. The rewrite was done by Antoine Pitrou and Amaury Forgeot d'Arc. I was slightly helpful at the end. :) Following are the log messages from the io-c branch: Merged revisions 68683-68685,68687-68689,68693,68704,68741-68743,68745,68747,68752-68754,68756,68758,68812,68816-68817,68820-68822,68824-68825,68828,68876-68877,69037,69044,69104,69115,69194,69626-69629,69636,69638,69641-69642,69644-69654,69656-69661,69671,69677,69812-69815,69817,69827-69830,69839,69841-69845,69848,69850,69852,69854,69860,69865-69866,69868,69872-69873,69885,69888,69891-69893,69911,69913-69916,69963,70033,70035,70038,70041-70048,70067-70070,70075,70112,70133,70135,70140 via svnmerge from svn+ssh://pythondev@svn.python.org/python/branches/io-c ........ r68683 | antoine.pitrou | 2009-01-17 17:13:48 -0600 (Sat, 17 Jan 2009) | 3 lines Merge in changes from the io-c sandbox. Tests will follow in separate commits. ........ r68684 | antoine.pitrou | 2009-01-17 17:17:26 -0600 (Sat, 17 Jan 2009) | 3 lines Fixes and additions to test_io.py ........ r68685 | antoine.pitrou | 2009-01-17 17:22:04 -0600 (Sat, 17 Jan 2009) | 1 line Fix test_fileio ........ r68687 | antoine.pitrou | 2009-01-17 17:35:11 -0600 (Sat, 17 Jan 2009) | 3 lines Add dependency to _iomodule.h for the various C sources ........ r68688 | antoine.pitrou | 2009-01-17 17:38:18 -0600 (Sat, 17 Jan 2009) | 3 lines These precautions are not needed anymore! ........ r68689 | antoine.pitrou | 2009-01-17 17:41:48 -0600 (Sat, 17 Jan 2009) | 3 lines Fix another test ........ r68693 | antoine.pitrou | 2009-01-17 17:49:58 -0600 (Sat, 17 Jan 2009) | 3 lines Fix test_uu (which was using private attributes of TextIOWrapper) ........ r68704 | antoine.pitrou | 2009-01-17 18:45:29 -0600 (Sat, 17 Jan 2009) | 3 lines Most io sources are Py_ssize_t-clean (I don't know about bytesio and stringio) ........ r68741 | antoine.pitrou | 2009-01-18 15:20:30 -0600 (Sun, 18 Jan 2009) | 3 lines Check return type in TextIOWrapper.__next__ ........ r68742 | antoine.pitrou | 2009-01-18 15:28:48 -0600 (Sun, 18 Jan 2009) | 4 lines Make binary buffered readline and iteration much faster (8x as fast as the IOBase generic implementation) ........ r68743 | antoine.pitrou | 2009-01-18 15:47:47 -0600 (Sun, 18 Jan 2009) | 3 lines Reinsert test_io_after_close (was removed by mistake) ........ r68745 | antoine.pitrou | 2009-01-18 16:16:06 -0600 (Sun, 18 Jan 2009) | 3 lines Add read, read1 and write methods to BufferedIOBase ........ r68747 | antoine.pitrou | 2009-01-18 16:35:58 -0600 (Sun, 18 Jan 2009) | 3 lines Kill test failure ........ r68752 | amaury.forgeotdarc | 2009-01-18 17:05:43 -0600 (Sun, 18 Jan 2009) | 3 lines Fix a segfault when e.g a BufferedReader is created with a FileIO in read mode. ........ r68753 | antoine.pitrou | 2009-01-18 17:13:09 -0600 (Sun, 18 Jan 2009) | 3 lines Add truncate() to text IO objects ........ r68754 | antoine.pitrou | 2009-01-18 17:51:08 -0600 (Sun, 18 Jan 2009) | 3 lines Remove IOBase.__del__ and replace it with custom code with tp_dealloc ........ r68756 | antoine.pitrou | 2009-01-18 18:10:16 -0600 (Sun, 18 Jan 2009) | 3 lines Remove irrelevant comment. ........ r68758 | antoine.pitrou | 2009-01-18 18:36:16 -0600 (Sun, 18 Jan 2009) | 3 lines in importlib:_fileio._FileIO -> _io.FileIO ........ r68812 | antoine.pitrou | 2009-01-20 14:15:51 -0600 (Tue, 20 Jan 2009) | 3 lines Add garbage collection support to FileIO objects ........ r68816 | antoine.pitrou | 2009-01-20 14:56:28 -0600 (Tue, 20 Jan 2009) | 3 lines Add GC support to Buffered and Text IO objects ........ r68817 | antoine.pitrou | 2009-01-20 15:19:45 -0600 (Tue, 20 Jan 2009) | 3 lines Add some file headers ........ r68820 | antoine.pitrou | 2009-01-20 15:29:59 -0600 (Tue, 20 Jan 2009) | 3 lines Add class TextIOBase ........ r68821 | antoine.pitrou | 2009-01-20 15:36:16 -0600 (Tue, 20 Jan 2009) | 3 lines Add properties to TextIOBase ........ r68822 | antoine.pitrou | 2009-01-20 15:41:19 -0600 (Tue, 20 Jan 2009) | 3 lines Disable the pure Python TextIOBase class, and inject C the implementation instead ........ r68824 | antoine.pitrou | 2009-01-20 16:36:28 -0600 (Tue, 20 Jan 2009) | 3 lines Fix two leaks ........ r68825 | antoine.pitrou | 2009-01-20 16:38:29 -0600 (Tue, 20 Jan 2009) | 3 lines FileIO.name is just a plain attribute, we can set it directly ........ r68828 | antoine.pitrou | 2009-01-20 17:06:33 -0600 (Tue, 20 Jan 2009) | 3 lines Speed up closed checks on text IO objects. Good for a 25% speedup on small ops. ........ r68876 | antoine.pitrou | 2009-01-23 17:01:25 -0600 (Fri, 23 Jan 2009) | 3 lines Two typos ........ r68877 | antoine.pitrou | 2009-01-23 18:13:20 -0600 (Fri, 23 Jan 2009) | 3 lines Remove two unused functions ........ r69037 | amaury.forgeotdarc | 2009-01-27 17:10:25 -0600 (Tue, 27 Jan 2009) | 2 lines Update the win32 project files ........ r69044 | antoine.pitrou | 2009-01-27 18:51:07 -0600 (Tue, 27 Jan 2009) | 3 lines Improve heuristic in IncrementalNewlineDecoder + some micro-optimizations ........ r69104 | antoine.pitrou | 2009-01-29 15:23:42 -0600 (Thu, 29 Jan 2009) | 3 lines Fix some crashers found by Victor ........ r69115 | hirokazu.yamamoto | 2009-01-29 20:36:28 -0600 (Thu, 29 Jan 2009) | 1 line Updated VC6 project file. ........ r69194 | antoine.pitrou | 2009-02-01 16:57:18 -0600 (Sun, 01 Feb 2009) | 3 lines Fix downcasting warnings in 32-bit mode with 64-bit offsets (Windows) ........ r69626 | benjamin.peterson | 2009-02-14 17:33:34 -0600 (Sat, 14 Feb 2009) | 1 line only catch AttributeError and UnsupportedOperation ........ r69627 | benjamin.peterson | 2009-02-14 21:35:28 -0600 (Sat, 14 Feb 2009) | 1 line give the IO module its own state and store the os and locale modules in it ........ r69628 | benjamin.peterson | 2009-02-14 22:08:32 -0600 (Sat, 14 Feb 2009) | 1 line put interned strings in the module state structure ........ r69629 | benjamin.peterson | 2009-02-14 22:15:29 -0600 (Sat, 14 Feb 2009) | 1 line put UnsupportedOperation in the module state ........ r69636 | benjamin.peterson | 2009-02-15 08:31:42 -0600 (Sun, 15 Feb 2009) | 1 line dealloc unsupported_operation ........ r69638 | benjamin.peterson | 2009-02-15 09:24:45 -0600 (Sun, 15 Feb 2009) | 1 line actually test the C implementation ........ r69641 | benjamin.peterson | 2009-02-15 10:12:37 -0600 (Sun, 15 Feb 2009) | 5 lines make interned strings globals again ;( putting them in the module state was asking for trouble when the module was dealloced before the classes in it were ........ r69642 | benjamin.peterson | 2009-02-15 10:19:45 -0600 (Sun, 15 Feb 2009) | 1 line actually test the python implementations ........ r69644 | antoine.pitrou | 2009-02-15 11:59:30 -0600 (Sun, 15 Feb 2009) | 3 lines Fix memory leak in destructor when a Python class inherits from IOBase (or an IOBase-derived type) ........ r69645 | antoine.pitrou | 2009-02-15 12:23:26 -0600 (Sun, 15 Feb 2009) | 3 lines Add a warning about the embarassing state of IOBase finalization ........ r69646 | antoine.pitrou | 2009-02-15 13:14:42 -0600 (Sun, 15 Feb 2009) | 3 lines Fix opening of 8-bit filenames with FileIO ........ r69647 | antoine.pitrou | 2009-02-15 13:20:22 -0600 (Sun, 15 Feb 2009) | 3 lines Fix leak in FileIO constructor ........ r69648 | antoine.pitrou | 2009-02-15 13:58:16 -0600 (Sun, 15 Feb 2009) | 3 lines Fix some refleaks ........ r69649 | antoine.pitrou | 2009-02-15 14:05:13 -0600 (Sun, 15 Feb 2009) | 3 lines Fix a leak in IOBase.writelines ........ r69650 | antoine.pitrou | 2009-02-15 14:11:56 -0600 (Sun, 15 Feb 2009) | 3 lines Fix memory leak in BufferedWriter.truncate ........ r69651 | antoine.pitrou | 2009-02-15 14:25:34 -0600 (Sun, 15 Feb 2009) | 3 lines Fix a leak in TextIOWrapper.seek ........ r69652 | antoine.pitrou | 2009-02-15 14:26:28 -0600 (Sun, 15 Feb 2009) | 3 lines Unify implementations of truncate for buffered objects ........ r69653 | antoine.pitrou | 2009-02-15 15:15:15 -0600 (Sun, 15 Feb 2009) | 3 lines Fix more leaks in TextIOWrapper ........ r69654 | antoine.pitrou | 2009-02-15 15:21:57 -0600 (Sun, 15 Feb 2009) | 3 lines Smaller chunk size for a faster test ........ r69656 | benjamin.peterson | 2009-02-15 17:29:48 -0600 (Sun, 15 Feb 2009) | 1 line braces make this much clearer ........ r69657 | benjamin.peterson | 2009-02-15 17:46:07 -0600 (Sun, 15 Feb 2009) | 1 line use the correct macro ........ r69658 | antoine.pitrou | 2009-02-15 19:38:59 -0600 (Sun, 15 Feb 2009) | 5 lines Fix crash in test_urllib2_localnet in debug mode. It was due to an HTTPResponse object being revived when calling its close() method in IOBase's tp_dealloc. _PyIOBase_finalize() starts looking scary... ........ r69659 | benjamin.peterson | 2009-02-15 20:55:48 -0600 (Sun, 15 Feb 2009) | 1 line fix segfault on initialization failing ........ r69660 | benjamin.peterson | 2009-02-15 21:09:31 -0600 (Sun, 15 Feb 2009) | 1 line apparently locale.getprefferedencoding() can raise a ImportError, too ........ r69661 | benjamin.peterson | 2009-02-15 21:54:15 -0600 (Sun, 15 Feb 2009) | 1 line it's amazing this worked at all; I was using the wrong structs! ........ r69671 | benjamin.peterson | 2009-02-16 08:38:27 -0600 (Mon, 16 Feb 2009) | 1 line add garbage collection support to bytesio ........ r69677 | benjamin.peterson | 2009-02-16 10:31:03 -0600 (Mon, 16 Feb 2009) | 5 lines reduce ImportError catching code duplication I'm not sure this makes the code clearer with its new gotos, but at least I added a big fat comment ........ r69812 | antoine.pitrou | 2009-02-20 13:50:16 -0600 (Fri, 20 Feb 2009) | 3 lines _StringIO now belongs to the _io modules, rather to its own _stringio module ........ r69813 | antoine.pitrou | 2009-02-20 13:58:22 -0600 (Fri, 20 Feb 2009) | 3 lines Add a test for StringIO properties ........ r69814 | antoine.pitrou | 2009-02-20 14:06:03 -0600 (Fri, 20 Feb 2009) | 3 lines Reimplement a few trivial StringIO functions and properties in C ........ r69815 | antoine.pitrou | 2009-02-20 14:13:11 -0600 (Fri, 20 Feb 2009) | 3 lines Add the line_buffering property to TextIOWrapper, and test for it ........ r69817 | antoine.pitrou | 2009-02-20 14:45:50 -0600 (Fri, 20 Feb 2009) | 4 lines Allow IncrementalNewlineDecoder to take unicode objects as decoding input if the decoder parameter is None This will help rewriting StringIO to C ........ r69827 | antoine.pitrou | 2009-02-20 19:00:30 -0600 (Fri, 20 Feb 2009) | 3 lines Rewrite most of StringIO in C. Some almost empty stubs remain to be converted. ........ r69828 | antoine.pitrou | 2009-02-20 19:09:25 -0600 (Fri, 20 Feb 2009) | 3 lines Plug a leak, and remove an unused string ........ r69829 | benjamin.peterson | 2009-02-20 20:02:28 -0600 (Fri, 20 Feb 2009) | 1 line this assertions makes more sense here ........ r69830 | benjamin.peterson | 2009-02-20 20:03:04 -0600 (Fri, 20 Feb 2009) | 1 line PyModule_AddObject can fail; simplify this code with a macro ........ r69839 | antoine.pitrou | 2009-02-21 12:54:01 -0600 (Sat, 21 Feb 2009) | 3 lines StringIO is now written entirely in C (and blazingly fast) ........ r69841 | benjamin.peterson | 2009-02-21 14:05:40 -0600 (Sat, 21 Feb 2009) | 1 line split the Python implementation of io into another module and rewrite the tests to test both implementations ........ r69842 | benjamin.peterson | 2009-02-21 14:10:00 -0600 (Sat, 21 Feb 2009) | 1 line closed is not a function ........ r69843 | benjamin.peterson | 2009-02-21 14:13:04 -0600 (Sat, 21 Feb 2009) | 1 line fix __all__ test ........ r69844 | benjamin.peterson | 2009-02-21 14:21:24 -0600 (Sat, 21 Feb 2009) | 1 line fix the rest of the Misc tests ........ r69845 | benjamin.peterson | 2009-02-21 14:26:59 -0600 (Sat, 21 Feb 2009) | 1 line RawIOBase is better for FileIO ........ r69848 | benjamin.peterson | 2009-02-21 15:33:53 -0600 (Sat, 21 Feb 2009) | 1 line fix some more tests broken by bag argument validation ........ r69850 | benjamin.peterson | 2009-02-21 16:16:42 -0600 (Sat, 21 Feb 2009) | 1 line make the python IncrementalNewineDecoder support a None decoder ........ r69852 | benjamin.peterson | 2009-02-21 16:36:09 -0600 (Sat, 21 Feb 2009) | 1 line fix a BlockingIOError.characters_written bug ........ r69854 | benjamin.peterson | 2009-02-21 16:49:02 -0600 (Sat, 21 Feb 2009) | 1 line check whence ........ r69860 | benjamin.peterson | 2009-02-21 17:42:50 -0600 (Sat, 21 Feb 2009) | 1 line fix some of these Misbehaving io tests ........ r69865 | benjamin.peterson | 2009-02-21 18:59:52 -0600 (Sat, 21 Feb 2009) | 1 line don't use super here() ........ r69866 | benjamin.peterson | 2009-02-21 19:05:28 -0600 (Sat, 21 Feb 2009) | 1 line use implementation specific classes ........ r69868 | benjamin.peterson | 2009-02-21 22:12:05 -0600 (Sat, 21 Feb 2009) | 1 line use a more DRY friendly approach to injecting module contents into test classes ........ r69872 | antoine.pitrou | 2009-02-22 13:39:45 -0600 (Sun, 22 Feb 2009) | 3 lines Sanitize destructor behaviour of IOBase. Now Python-defined attributes can be accessed from close(). ........ r69873 | antoine.pitrou | 2009-02-22 13:50:14 -0600 (Sun, 22 Feb 2009) | 4 lines Only set the internal fd after it has been checked to be valid (otherwise, the destructor will attempt to close it) ........ r69885 | benjamin.peterson | 2009-02-22 15:30:14 -0600 (Sun, 22 Feb 2009) | 1 line convert some other tests to use both io implementations ........ r69888 | antoine.pitrou | 2009-02-22 17:03:16 -0600 (Sun, 22 Feb 2009) | 3 lines Silence all exceptions when finalizing ........ r69891 | benjamin.peterson | 2009-02-22 17:27:24 -0600 (Sun, 22 Feb 2009) | 1 line convert another test to test both io implementations ........ r69892 | benjamin.peterson | 2009-02-22 17:32:15 -0600 (Sun, 22 Feb 2009) | 1 line help poor people like me to find their io tests (did I miss any?) ........ r69893 | benjamin.peterson | 2009-02-22 17:37:56 -0600 (Sun, 22 Feb 2009) | 1 line put a big note in the test telling people to write tests for both implementations now ........ r69911 | antoine.pitrou | 2009-02-23 13:57:18 -0600 (Mon, 23 Feb 2009) | 3 lines expose DEFAULT_BUFFER_SIZE again (fixes a bunch of test failures) ........ r69913 | antoine.pitrou | 2009-02-23 14:10:30 -0600 (Mon, 23 Feb 2009) | 4 lines Do the cyclic garbage collection tests only on the C version. The Python version is helpless as it uses __del__. ........ r69914 | antoine.pitrou | 2009-02-23 14:21:41 -0600 (Mon, 23 Feb 2009) | 3 lines Adapt test_largefile to test both implementations ........ r69915 | antoine.pitrou | 2009-02-23 14:25:14 -0600 (Mon, 23 Feb 2009) | 3 lines One small failure ........ r69916 | antoine.pitrou | 2009-02-23 14:28:33 -0600 (Mon, 23 Feb 2009) | 3 lines Add a comment, at BP's request ........ r69963 | antoine.pitrou | 2009-02-25 09:42:59 -0600 (Wed, 25 Feb 2009) | 3 lines Add a test of ABC inheritance ........ r70033 | antoine.pitrou | 2009-02-27 15:49:50 -0600 (Fri, 27 Feb 2009) | 3 lines The base classes now are ABCs. ........ r70035 | benjamin.peterson | 2009-02-27 15:57:41 -0600 (Fri, 27 Feb 2009) | 1 line good house keeping ........ r70038 | antoine.pitrou | 2009-02-27 17:05:23 -0600 (Fri, 27 Feb 2009) | 4 lines Make the buffer allocation overflow tests specific to the C implementation, since the Python implementation resizes its buffers when needed rather than allocating them up front. ........ r70041 | benjamin.peterson | 2009-02-27 18:26:12 -0600 (Fri, 27 Feb 2009) | 1 line kill java naming for sanity ........ r70042 | benjamin.peterson | 2009-02-27 18:28:53 -0600 (Fri, 27 Feb 2009) | 2 lines timingTest is superseded by iobench ........ r70043 | antoine.pitrou | 2009-02-27 19:13:50 -0600 (Fri, 27 Feb 2009) | 3 lines Remove the last traces of java naming in test_io ........ r70044 | antoine.pitrou | 2009-02-27 19:18:34 -0600 (Fri, 27 Feb 2009) | 3 lines Better resource cleanup ........ r70045 | antoine.pitrou | 2009-02-27 19:29:00 -0600 (Fri, 27 Feb 2009) | 3 lines Remove dubious uses of super(), and fix one test ........ r70046 | antoine.pitrou | 2009-02-27 19:31:00 -0600 (Fri, 27 Feb 2009) | 3 lines Bump up CHUNK_SIZE (no need to make the Python version look slower than it is) ........ r70047 | benjamin.peterson | 2009-02-27 20:03:26 -0600 (Fri, 27 Feb 2009) | 1 line fix typo ........ r70048 | benjamin.peterson | 2009-02-27 21:35:11 -0600 (Fri, 27 Feb 2009) | 1 line move code to a better place ........ r70067 | benjamin.peterson | 2009-02-28 10:43:20 -0600 (Sat, 28 Feb 2009) | 4 lines 1. make sure to undo buffered read aheads in BufferedRandom.seek() 2. refill the buffer if have <= 0 3. fix the last failing test_io test! ........ r70068 | benjamin.peterson | 2009-02-28 10:57:50 -0600 (Sat, 28 Feb 2009) | 1 line define read1() on the python implementation's BufferedIOBase ........ r70069 | benjamin.peterson | 2009-02-28 11:01:17 -0600 (Sat, 28 Feb 2009) | 1 line document read1() in BufferedIOBase ........ r70070 | benjamin.peterson | 2009-02-28 11:06:42 -0600 (Sat, 28 Feb 2009) | 1 line give credit where credit is due ........ r70075 | antoine.pitrou | 2009-02-28 13:34:59 -0600 (Sat, 28 Feb 2009) | 3 lines Amaury's name ........ r70112 | antoine.pitrou | 2009-03-02 17:11:55 -0600 (Mon, 02 Mar 2009) | 4 lines Looks like this is necessary in order to build cleanly under Windows (someone correct this if it's wrong, I'm no Windows user) ........ r70133 | benjamin.peterson | 2009-03-03 15:23:32 -0600 (Tue, 03 Mar 2009) | 1 line fix test_newline_property on _pyio.StringIO ........ r70135 | benjamin.peterson | 2009-03-03 15:47:30 -0600 (Tue, 03 Mar 2009) | 1 line fix typos and inconsistencies. thanks to Daniel Diniz ........ r70140 | benjamin.peterson | 2009-03-03 16:21:10 -0600 (Tue, 03 Mar 2009) | 1 line add the test from #5266 ........

merge the io-c branch: C implementation of the io module
The main io module now uses the C implementation. The Python one still exists in Lib/_pyio.py for ease of testing new features and usefulness to other implementers. The rewrite was done by Antoine Pitrou and Amaury Forgeot d'Arc. I was slightly helpful at the end. :) Following are the log messages from the io-c branch: Merged revisions 68683-68685,68687-68689,68693,68704,68741-68743,68745,68747,68752-68754,68756,68758,68812,68816-68817,68820-68822,68824-68825,68828,68876-68877,69037,69044,69104,69115,69194,69626-69629,69636,69638,69641-69642,69644-69654,69656-69661,69671,69677,69812-69815,69817,69827-69830,69839,69841-69845,69848,69850,69852,69854,69860,69865-69866,69868,69872-69873,69885,69888,69891-69893,69911,69913-69916,69963,70033,70035,70038,70041-70048,70067-70070,70075,70112,70133,70135,70140 via svnmerge from svn+ssh://pythondev@svn.python.org/python/branches/io-c ........ r68683 | antoine.pitrou | 2009-01-17 17:13:48 -0600 (Sat, 17 Jan 2009) | 3 lines Merge in changes from the io-c sandbox. Tests will follow in separate commits. ........ r68684 | antoine.pitrou | 2009-01-17 17:17:26 -0600 (Sat, 17 Jan 2009) | 3 lines Fixes and additions to test_io.py ........ r68685 | antoine.pitrou | 2009-01-17 17:22:04 -0600 (Sat, 17 Jan 2009) | 1 line Fix test_fileio ........ r68687 | antoine.pitrou | 2009-01-17 17:35:11 -0600 (Sat, 17 Jan 2009) | 3 lines Add dependency to _iomodule.h for the various C sources ........ r68688 | antoine.pitrou | 2009-01-17 17:38:18 -0600 (Sat, 17 Jan 2009) | 3 lines These precautions are not needed anymore! ........ r68689 | antoine.pitrou | 2009-01-17 17:41:48 -0600 (Sat, 17 Jan 2009) | 3 lines Fix another test ........ r68693 | antoine.pitrou | 2009-01-17 17:49:58 -0600 (Sat, 17 Jan 2009) | 3 lines Fix test_uu (which was using private attributes of TextIOWrapper) ........ r68704 | antoine.pitrou | 2009-01-17 18:45:29 -0600 (Sat, 17 Jan 2009) | 3 lines Most io sources are Py_ssize_t-clean (I don't know about bytesio and stringio) ........ r68741 | antoine.pitrou | 2009-01-18 15:20:30 -0600 (Sun, 18 Jan 2009) | 3 lines Check return type in TextIOWrapper.__next__ ........ r68742 | antoine.pitrou | 2009-01-18 15:28:48 -0600 (Sun, 18 Jan 2009) | 4 lines Make binary buffered readline and iteration much faster (8x as fast as the IOBase generic implementation) ........ r68743 | antoine.pitrou | 2009-01-18 15:47:47 -0600 (Sun, 18 Jan 2009) | 3 lines Reinsert test_io_after_close (was removed by mistake) ........ r68745 | antoine.pitrou | 2009-01-18 16:16:06 -0600 (Sun, 18 Jan 2009) | 3 lines Add read, read1 and write methods to BufferedIOBase ........ r68747 | antoine.pitrou | 2009-01-18 16:35:58 -0600 (Sun, 18 Jan 2009) | 3 lines Kill test failure ........ r68752 | amaury.forgeotdarc | 2009-01-18 17:05:43 -0600 (Sun, 18 Jan 2009) | 3 lines Fix a segfault when e.g a BufferedReader is created with a FileIO in read mode. ........ r68753 | antoine.pitrou | 2009-01-18 17:13:09 -0600 (Sun, 18 Jan 2009) | 3 lines Add truncate() to text IO objects ........ r68754 | antoine.pitrou | 2009-01-18 17:51:08 -0600 (Sun, 18 Jan 2009) | 3 lines Remove IOBase.__del__ and replace it with custom code with tp_dealloc ........ r68756 | antoine.pitrou | 2009-01-18 18:10:16 -0600 (Sun, 18 Jan 2009) | 3 lines Remove irrelevant comment. ........ r68758 | antoine.pitrou | 2009-01-18 18:36:16 -0600 (Sun, 18 Jan 2009) | 3 lines in importlib:_fileio._FileIO -> _io.FileIO ........ r68812 | antoine.pitrou | 2009-01-20 14:15:51 -0600 (Tue, 20 Jan 2009) | 3 lines Add garbage collection support to FileIO objects ........ r68816 | antoine.pitrou | 2009-01-20 14:56:28 -0600 (Tue, 20 Jan 2009) | 3 lines Add GC support to Buffered and Text IO objects ........ r68817 | antoine.pitrou | 2009-01-20 15:19:45 -0600 (Tue, 20 Jan 2009) | 3 lines Add some file headers ........ r68820 | antoine.pitrou | 2009-01-20 15:29:59 -0600 (Tue, 20 Jan 2009) | 3 lines Add class TextIOBase ........ r68821 | antoine.pitrou | 2009-01-20 15:36:16 -0600 (Tue, 20 Jan 2009) | 3 lines Add properties to TextIOBase ........ r68822 | antoine.pitrou | 2009-01-20 15:41:19 -0600 (Tue, 20 Jan 2009) | 3 lines Disable the pure Python TextIOBase class, and inject C the implementation instead ........ r68824 | antoine.pitrou | 2009-01-20 16:36:28 -0600 (Tue, 20 Jan 2009) | 3 lines Fix two leaks ........ r68825 | antoine.pitrou | 2009-01-20 16:38:29 -0600 (Tue, 20 Jan 2009) | 3 lines FileIO.name is just a plain attribute, we can set it directly ........ r68828 | antoine.pitrou | 2009-01-20 17:06:33 -0600 (Tue, 20 Jan 2009) | 3 lines Speed up closed checks on text IO objects. Good for a 25% speedup on small ops. ........ r68876 | antoine.pitrou | 2009-01-23 17:01:25 -0600 (Fri, 23 Jan 2009) | 3 lines Two typos ........ r68877 | antoine.pitrou | 2009-01-23 18:13:20 -0600 (Fri, 23 Jan 2009) | 3 lines Remove two unused functions ........ r69037 | amaury.forgeotdarc | 2009-01-27 17:10:25 -0600 (Tue, 27 Jan 2009) | 2 lines Update the win32 project files ........ r69044 | antoine.pitrou | 2009-01-27 18:51:07 -0600 (Tue, 27 Jan 2009) | 3 lines Improve heuristic in IncrementalNewlineDecoder + some micro-optimizations ........ r69104 | antoine.pitrou | 2009-01-29 15:23:42 -0600 (Thu, 29 Jan 2009) | 3 lines Fix some crashers found by Victor ........ r69115 | hirokazu.yamamoto | 2009-01-29 20:36:28 -0600 (Thu, 29 Jan 2009) | 1 line Updated VC6 project file. ........ r69194 | antoine.pitrou | 2009-02-01 16:57:18 -0600 (Sun, 01 Feb 2009) | 3 lines Fix downcasting warnings in 32-bit mode with 64-bit offsets (Windows) ........ r69626 | benjamin.peterson | 2009-02-14 17:33:34 -0600 (Sat, 14 Feb 2009) | 1 line only catch AttributeError and UnsupportedOperation ........ r69627 | benjamin.peterson | 2009-02-14 21:35:28 -0600 (Sat, 14 Feb 2009) | 1 line give the IO module its own state and store the os and locale modules in it ........ r69628 | benjamin.peterson | 2009-02-14 22:08:32 -0600 (Sat, 14 Feb 2009) | 1 line put interned strings in the module state structure ........ r69629 | benjamin.peterson | 2009-02-14 22:15:29 -0600 (Sat, 14 Feb 2009) | 1 line put UnsupportedOperation in the module state ........ r69636 | benjamin.peterson | 2009-02-15 08:31:42 -0600 (Sun, 15 Feb 2009) | 1 line dealloc unsupported_operation ........ r69638 | benjamin.peterson | 2009-02-15 09:24:45 -0600 (Sun, 15 Feb 2009) | 1 line actually test the C implementation ........ r69641 | benjamin.peterson | 2009-02-15 10:12:37 -0600 (Sun, 15 Feb 2009) | 5 lines make interned strings globals again ;( putting them in the module state was asking for trouble when the module was dealloced before the classes in it were ........ r69642 | benjamin.peterson | 2009-02-15 10:19:45 -0600 (Sun, 15 Feb 2009) | 1 line actually test the python implementations ........ r69644 | antoine.pitrou | 2009-02-15 11:59:30 -0600 (Sun, 15 Feb 2009) | 3 lines Fix memory leak in destructor when a Python class inherits from IOBase (or an IOBase-derived type) ........ r69645 | antoine.pitrou | 2009-02-15 12:23:26 -0600 (Sun, 15 Feb 2009) | 3 lines Add a warning about the embarassing state of IOBase finalization ........ r69646 | antoine.pitrou | 2009-02-15 13:14:42 -0600 (Sun, 15 Feb 2009) | 3 lines Fix opening of 8-bit filenames with FileIO ........ r69647 | antoine.pitrou | 2009-02-15 13:20:22 -0600 (Sun, 15 Feb 2009) | 3 lines Fix leak in FileIO constructor ........ r69648 | antoine.pitrou | 2009-02-15 13:58:16 -0600 (Sun, 15 Feb 2009) | 3 lines Fix some refleaks ........ r69649 | antoine.pitrou | 2009-02-15 14:05:13 -0600 (Sun, 15 Feb 2009) | 3 lines Fix a leak in IOBase.writelines ........ r69650 | antoine.pitrou | 2009-02-15 14:11:56 -0600 (Sun, 15 Feb 2009) | 3 lines Fix memory leak in BufferedWriter.truncate ........ r69651 | antoine.pitrou | 2009-02-15 14:25:34 -0600 (Sun, 15 Feb 2009) | 3 lines Fix a leak in TextIOWrapper.seek ........ r69652 | antoine.pitrou | 2009-02-15 14:26:28 -0600 (Sun, 15 Feb 2009) | 3 lines Unify implementations of truncate for buffered objects ........ r69653 | antoine.pitrou | 2009-02-15 15:15:15 -0600 (Sun, 15 Feb 2009) | 3 lines Fix more leaks in TextIOWrapper ........ r69654 | antoine.pitrou | 2009-02-15 15:21:57 -0600 (Sun, 15 Feb 2009) | 3 lines Smaller chunk size for a faster test ........ r69656 | benjamin.peterson | 2009-02-15 17:29:48 -0600 (Sun, 15 Feb 2009) | 1 line braces make this much clearer ........ r69657 | benjamin.peterson | 2009-02-15 17:46:07 -0600 (Sun, 15 Feb 2009) | 1 line use the correct macro ........ r69658 | antoine.pitrou | 2009-02-15 19:38:59 -0600 (Sun, 15 Feb 2009) | 5 lines Fix crash in test_urllib2_localnet in debug mode. It was due to an HTTPResponse object being revived when calling its close() method in IOBase's tp_dealloc. _PyIOBase_finalize() starts looking scary... ........ r69659 | benjamin.peterson | 2009-02-15 20:55:48 -0600 (Sun, 15 Feb 2009) | 1 line fix segfault on initialization failing ........ r69660 | benjamin.peterson | 2009-02-15 21:09:31 -0600 (Sun, 15 Feb 2009) | 1 line apparently locale.getprefferedencoding() can raise a ImportError, too ........ r69661 | benjamin.peterson | 2009-02-15 21:54:15 -0600 (Sun, 15 Feb 2009) | 1 line it's amazing this worked at all; I was using the wrong structs! ........ r69671 | benjamin.peterson | 2009-02-16 08:38:27 -0600 (Mon, 16 Feb 2009) | 1 line add garbage collection support to bytesio ........ r69677 | benjamin.peterson | 2009-02-16 10:31:03 -0600 (Mon, 16 Feb 2009) | 5 lines reduce ImportError catching code duplication I'm not sure this makes the code clearer with its new gotos, but at least I added a big fat comment ........ r69812 | antoine.pitrou | 2009-02-20 13:50:16 -0600 (Fri, 20 Feb 2009) | 3 lines _StringIO now belongs to the _io modules, rather to its own _stringio module ........ r69813 | antoine.pitrou | 2009-02-20 13:58:22 -0600 (Fri, 20 Feb 2009) | 3 lines Add a test for StringIO properties ........ r69814 | antoine.pitrou | 2009-02-20 14:06:03 -0600 (Fri, 20 Feb 2009) | 3 lines Reimplement a few trivial StringIO functions and properties in C ........ r69815 | antoine.pitrou | 2009-02-20 14:13:11 -0600 (Fri, 20 Feb 2009) | 3 lines Add the line_buffering property to TextIOWrapper, and test for it ........ r69817 | antoine.pitrou | 2009-02-20 14:45:50 -0600 (Fri, 20 Feb 2009) | 4 lines Allow IncrementalNewlineDecoder to take unicode objects as decoding input if the decoder parameter is None This will help rewriting StringIO to C ........ r69827 | antoine.pitrou | 2009-02-20 19:00:30 -0600 (Fri, 20 Feb 2009) | 3 lines Rewrite most of StringIO in C. Some almost empty stubs remain to be converted. ........ r69828 | antoine.pitrou | 2009-02-20 19:09:25 -0600 (Fri, 20 Feb 2009) | 3 lines Plug a leak, and remove an unused string ........ r69829 | benjamin.peterson | 2009-02-20 20:02:28 -0600 (Fri, 20 Feb 2009) | 1 line this assertions makes more sense here ........ r69830 | benjamin.peterson | 2009-02-20 20:03:04 -0600 (Fri, 20 Feb 2009) | 1 line PyModule_AddObject can fail; simplify this code with a macro ........ r69839 | antoine.pitrou | 2009-02-21 12:54:01 -0600 (Sat, 21 Feb 2009) | 3 lines StringIO is now written entirely in C (and blazingly fast) ........ r69841 | benjamin.peterson | 2009-02-21 14:05:40 -0600 (Sat, 21 Feb 2009) | 1 line split the Python implementation of io into another module and rewrite the tests to test both implementations ........ r69842 | benjamin.peterson | 2009-02-21 14:10:00 -0600 (Sat, 21 Feb 2009) | 1 line closed is not a function ........ r69843 | benjamin.peterson | 2009-02-21 14:13:04 -0600 (Sat, 21 Feb 2009) | 1 line fix __all__ test ........ r69844 | benjamin.peterson | 2009-02-21 14:21:24 -0600 (Sat, 21 Feb 2009) | 1 line fix the rest of the Misc tests ........ r69845 | benjamin.peterson | 2009-02-21 14:26:59 -0600 (Sat, 21 Feb 2009) | 1 line RawIOBase is better for FileIO ........ r69848 | benjamin.peterson | 2009-02-21 15:33:53 -0600 (Sat, 21 Feb 2009) | 1 line fix some more tests broken by bag argument validation ........ r69850 | benjamin.peterson | 2009-02-21 16:16:42 -0600 (Sat, 21 Feb 2009) | 1 line make the python IncrementalNewineDecoder support a None decoder ........ r69852 | benjamin.peterson | 2009-02-21 16:36:09 -0600 (Sat, 21 Feb 2009) | 1 line fix a BlockingIOError.characters_written bug ........ r69854 | benjamin.peterson | 2009-02-21 16:49:02 -0600 (Sat, 21 Feb 2009) | 1 line check whence ........ r69860 | benjamin.peterson | 2009-02-21 17:42:50 -0600 (Sat, 21 Feb 2009) | 1 line fix some of these Misbehaving io tests ........ r69865 | benjamin.peterson | 2009-02-21 18:59:52 -0600 (Sat, 21 Feb 2009) | 1 line don't use super here() ........ r69866 | benjamin.peterson | 2009-02-21 19:05:28 -0600 (Sat, 21 Feb 2009) | 1 line use implementation specific classes ........ r69868 | benjamin.peterson | 2009-02-21 22:12:05 -0600 (Sat, 21 Feb 2009) | 1 line use a more DRY friendly approach to injecting module contents into test classes ........ r69872 | antoine.pitrou | 2009-02-22 13:39:45 -0600 (Sun, 22 Feb 2009) | 3 lines Sanitize destructor behaviour of IOBase. Now Python-defined attributes can be accessed from close(). ........ r69873 | antoine.pitrou | 2009-02-22 13:50:14 -0600 (Sun, 22 Feb 2009) | 4 lines Only set the internal fd after it has been checked to be valid (otherwise, the destructor will attempt to close it) ........ r69885 | benjamin.peterson | 2009-02-22 15:30:14 -0600 (Sun, 22 Feb 2009) | 1 line convert some other tests to use both io implementations ........ r69888 | antoine.pitrou | 2009-02-22 17:03:16 -0600 (Sun, 22 Feb 2009) | 3 lines Silence all exceptions when finalizing ........ r69891 | benjamin.peterson | 2009-02-22 17:27:24 -0600 (Sun, 22 Feb 2009) | 1 line convert another test to test both io implementations ........ r69892 | benjamin.peterson | 2009-02-22 17:32:15 -0600 (Sun, 22 Feb 2009) | 1 line help poor people like me to find their io tests (did I miss any?) ........ r69893 | benjamin.peterson | 2009-02-22 17:37:56 -0600 (Sun, 22 Feb 2009) | 1 line put a big note in the test telling people to write tests for both implementations now ........ r69911 | antoine.pitrou | 2009-02-23 13:57:18 -0600 (Mon, 23 Feb 2009) | 3 lines expose DEFAULT_BUFFER_SIZE again (fixes a bunch of test failures) ........ r69913 | antoine.pitrou | 2009-02-23 14:10:30 -0600 (Mon, 23 Feb 2009) | 4 lines Do the cyclic garbage collection tests only on the C version. The Python version is helpless as it uses __del__. ........ r69914 | antoine.pitrou | 2009-02-23 14:21:41 -0600 (Mon, 23 Feb 2009) | 3 lines Adapt test_largefile to test both implementations ........ r69915 | antoine.pitrou | 2009-02-23 14:25:14 -0600 (Mon, 23 Feb 2009) | 3 lines One small failure ........ r69916 | antoine.pitrou | 2009-02-23 14:28:33 -0600 (Mon, 23 Feb 2009) | 3 lines Add a comment, at BP's request ........ r69963 | antoine.pitrou | 2009-02-25 09:42:59 -0600 (Wed, 25 Feb 2009) | 3 lines Add a test of ABC inheritance ........ r70033 | antoine.pitrou | 2009-02-27 15:49:50 -0600 (Fri, 27 Feb 2009) | 3 lines The base classes now are ABCs. ........ r70035 | benjamin.peterson | 2009-02-27 15:57:41 -0600 (Fri, 27 Feb 2009) | 1 line good house keeping ........ r70038 | antoine.pitrou | 2009-02-27 17:05:23 -0600 (Fri, 27 Feb 2009) | 4 lines Make the buffer allocation overflow tests specific to the C implementation, since the Python implementation resizes its buffers when needed rather than allocating them up front. ........ r70041 | benjamin.peterson | 2009-02-27 18:26:12 -0600 (Fri, 27 Feb 2009) | 1 line kill java naming for sanity ........ r70042 | benjamin.peterson | 2009-02-27 18:28:53 -0600 (Fri, 27 Feb 2009) | 2 lines timingTest is superseded by iobench ........ r70043 | antoine.pitrou | 2009-02-27 19:13:50 -0600 (Fri, 27 Feb 2009) | 3 lines Remove the last traces of java naming in test_io ........ r70044 | antoine.pitrou | 2009-02-27 19:18:34 -0600 (Fri, 27 Feb 2009) | 3 lines Better resource cleanup ........ r70045 | antoine.pitrou | 2009-02-27 19:29:00 -0600 (Fri, 27 Feb 2009) | 3 lines Remove dubious uses of super(), and fix one test ........ r70046 | antoine.pitrou | 2009-02-27 19:31:00 -0600 (Fri, 27 Feb 2009) | 3 lines Bump up CHUNK_SIZE (no need to make the Python version look slower than it is) ........ r70047 | benjamin.peterson | 2009-02-27 20:03:26 -0600 (Fri, 27 Feb 2009) | 1 line fix typo ........ r70048 | benjamin.peterson | 2009-02-27 21:35:11 -0600 (Fri, 27 Feb 2009) | 1 line move code to a better place ........ r70067 | benjamin.peterson | 2009-02-28 10:43:20 -0600 (Sat, 28 Feb 2009) | 4 lines 1. make sure to undo buffered read aheads in BufferedRandom.seek() 2. refill the buffer if have <= 0 3. fix the last failing test_io test! ........ r70068 | benjamin.peterson | 2009-02-28 10:57:50 -0600 (Sat, 28 Feb 2009) | 1 line define read1() on the python implementation's BufferedIOBase ........ r70069 | benjamin.peterson | 2009-02-28 11:01:17 -0600 (Sat, 28 Feb 2009) | 1 line document read1() in BufferedIOBase ........ r70070 | benjamin.peterson | 2009-02-28 11:06:42 -0600 (Sat, 28 Feb 2009) | 1 line give credit where credit is due ........ r70075 | antoine.pitrou | 2009-02-28 13:34:59 -0600 (Sat, 28 Feb 2009) | 3 lines Amaury's name ........ r70112 | antoine.pitrou | 2009-03-02 17:11:55 -0600 (Mon, 02 Mar 2009) | 4 lines Looks like this is necessary in order to build cleanly under Windows (someone correct this if it's wrong, I'm no Windows user) ........ r70133 | benjamin.peterson | 2009-03-03 15:23:32 -0600 (Tue, 03 Mar 2009) | 1 line fix test_newline_property on _pyio.StringIO ........ r70135 | benjamin.peterson | 2009-03-03 15:47:30 -0600 (Tue, 03 Mar 2009) | 1 line fix typos and inconsistencies. thanks to Daniel Diniz ........ r70140 | benjamin.peterson | 2009-03-03 16:21:10 -0600 (Tue, 03 Mar 2009) | 1 line add the test from #5266 ........
4fa88fa0 · Benjamin Peterson · 03ad8124 · 4fa88fa0 · 4fa88fa0 · 4fa88fa0
Commit 4fa88fa0 authored Mar 04, 2009 by Benjamin Peterson
30 changed files
--- a/Doc/library/io.rst
+++ b/Doc/library/io.rst
@@ -6,6 +6,8 @@
 .. moduleauthor:: Guido van Rossum <guido@python.org>
 .. moduleauthor:: Mike Verdone <mike.verdone@gmail.com>
 .. moduleauthor:: Mark Russell <mark.russell@zen.co.uk>
+.. moduleauthor:: Antoine Pitrou <solipsis@pitrou.net>
+.. moduleauthor:: Amaury Forgeot d'Arc <amauryfa@gmail.com>
 .. sectionauthor:: Benjamin Peterson <benjamin@python.org>
 The :mod:`io` module provides the Python interfaces to stream handling.  The
@@ -364,6 +366,11 @@ I/O Base Classes
      A :exc:`BlockingIOError` is raised if the underlying raw stream has no
      data at the moment.
+   .. method:: read1([n])
+      Read and return up to *n* bytes, with at most one call to the underlying
+      raw stream's :meth:`~RawIOBase.read` method.
   .. method:: readinto(b)
      Read up to len(b) bytes into bytearray *b* and return the number of bytes
@@ -501,7 +508,7 @@ Buffered Streams
   The constructor creates a :class:`BufferedWriter` for the given writeable
   *raw* stream.  If the *buffer_size* is not given, it defaults to
-   :data:`DEAFULT_BUFFER_SIZE`.  If *max_buffer_size* is omitted, it defaults to
+   :data:`DEFAULT_BUFFER_SIZE`.  If *max_buffer_size* is omitted, it defaults to
   twice the buffer size.
   :class:`BufferedWriter` provides or overrides these methods in addition to

--- a/Lib/_pyio.py
+++ b/Lib/_pyio.py
--- a/Lib/importlib/__init__.py
+++ b/Lib/importlib/__init__.py
@@ -93,12 +93,12 @@ except ImportError:
        except ImportError:
            raise ImportError('posix, nt, or os2 module required for importlib')
 _bootstrap._os = _os
-import imp, sys, marshal, errno, _fileio
+import imp, sys, marshal, errno, _io
 _bootstrap.imp = imp
 _bootstrap.sys = sys
 _bootstrap.marshal = marshal
 _bootstrap.errno = errno
-_bootstrap._fileio = _fileio
+_bootstrap._io = _io
 import _warnings
 _bootstrap._warnings = _warnings

--- a/Lib/importlib/_bootstrap.py
+++ b/Lib/importlib/_bootstrap.py
@@ -473,7 +473,7 @@ class PyFileLoader(PyLoader):
        if source_path is None:
            return None
        import tokenize
-        with closing(_fileio._FileIO(source_path, 'r')) as file:
+        with closing(_io.FileIO(source_path, 'r')) as file:
            encoding, lines = tokenize.detect_encoding(file.readline)
        # XXX Will fail when passed to compile() if the encoding is
        # anything other than UTF-8.
@@ -527,7 +527,7 @@ class PyPycFileLoader(PyPycLoader, PyFileLoader):
        bytecode_path = self.bytecode_path(name)
        if not bytecode_path:
            bytecode_path = self._base_path + suffix_list(imp.PY_COMPILED)[0]
-        file = _fileio._FileIO(bytecode_path, 'w')
+        file = _io.FileIO(bytecode_path, 'w')
        try:
            with closing(file) as bytecode_file:
                bytecode_file.write(data)

--- a/Lib/io.py
+++ b/Lib/io.py
--- a/Lib/test/test_bufio.py
+++ b/Lib/test/test_bufio.py
 import unittest
 from test import support
-# Simple test to ensure that optimizations in fileobject.c deliver
+import io # C implementation.
-# the expected results.  For best testing, run this under a debug-build
+import _pyio as pyio # Python implementation.
-# Python too (to exercise asserts in the C code).
+# Simple test to ensure that optimizations in the IO library deliver the
+# expected results.  For best testing, run this under a debug-build Python too
+# (to exercise asserts in the C code).
 lengths = list(range(1, 257)) + [512, 1000, 1024, 2048, 4096, 8192, 10000,
                                 16384, 32768, 65536, 1000000]
@@ -18,7 +21,7 @@ class BufferSizeTest(unittest.TestCase):
        # Since C doesn't guarantee we can write/read arbitrary bytes in text
        # files, use binary mode.
-        f = open(support.TESTFN, "wb")
+        f = self.open(support.TESTFN, "wb")
        try:
            # write once with \n and once without
            f.write(s)
@@ -58,8 +61,16 @@ class BufferSizeTest(unittest.TestCase):
    def test_nullpat(self):
        self.drive_one(bytes(1000))
+class CBufferSizeTest(BufferSizeTest):
+    open = io.open
+class PyBufferSizeTest(BufferSizeTest):
+    open = staticmethod(pyio.open)
 def test_main():
-    support.run_unittest(BufferSizeTest)
+    support.run_unittest(CBufferSizeTest, PyBufferSizeTest)
 if __name__ == "__main__":
    test_main()
--- a/Lib/test/test_descr.py
+++ b/Lib/test/test_descr.py
@@ -2605,10 +2605,10 @@ order (MRO) for bases """
    def test_descrdoc(self):
        # Testing descriptor doc strings...
-        from _fileio import _FileIO
+        from _io import FileIO
        def check(descr, what):
            self.assertEqual(descr.__doc__, what)
-        check(_FileIO.closed, "True if the file is closed") # getset descriptor
+        check(FileIO.closed, "True if the file is closed") # getset descriptor
        check(complex.real, "the real part of a complex number") # member descriptor
    def test_doc_descriptor(self):

--- a/Lib/test/test_file.py
+++ b/Lib/test/test_file.py
@@ -4,6 +4,9 @@ import unittest
 from array import array
 from weakref import proxy
+import io
+import _pyio as pyio
 from test.support import TESTFN, findfile, run_unittest
 from collections import UserList
@@ -11,7 +14,7 @@ class AutoFileTests(unittest.TestCase):
    # file tests for which a test file is automatically set up
    def setUp(self):
-        self.f = open(TESTFN, 'wb')
+        self.f = self.open(TESTFN, 'wb')
    def tearDown(self):
        if self.f:
@@ -39,7 +42,7 @@ class AutoFileTests(unittest.TestCase):
        self.f.write(b'12')
        self.f.close()
        a = array('b', b'x'*10)
-        self.f = open(TESTFN, 'rb')
+        self.f = self.open(TESTFN, 'rb')
        n = self.f.readinto(a)
        self.assertEquals(b'12', a.tostring()[:n])
@@ -47,7 +50,7 @@ class AutoFileTests(unittest.TestCase):
        # verify readinto refuses text files
        a = array('b', b'x'*10)
        self.f.close()
-        self.f = open(TESTFN, 'r')
+        self.f = self.open(TESTFN, 'r')
        if hasattr(self.f, "readinto"):
            self.assertRaises(TypeError, self.f.readinto, a)
@@ -56,7 +59,7 @@ class AutoFileTests(unittest.TestCase):
        l = UserList([b'1', b'2'])
        self.f.writelines(l)
        self.f.close()
-        self.f = open(TESTFN, 'rb')
+        self.f = self.open(TESTFN, 'rb')
        buf = self.f.read()
        self.assertEquals(buf, b'12')
@@ -126,13 +129,20 @@ class AutoFileTests(unittest.TestCase):
    def testReadWhenWriting(self):
        self.assertRaises(IOError, self.f.read)
+class CAutoFileTests(AutoFileTests):
+    open = io.open
+class PyAutoFileTests(AutoFileTests):
+    open = staticmethod(pyio.open)
 class OtherFileTests(unittest.TestCase):
    def testModeStrings(self):
        # check invalid mode strings
        for mode in ("", "aU", "wU+"):
            try:
-                f = open(TESTFN, mode)
+                f = self.open(TESTFN, mode)
            except ValueError:
                pass
            else:
@@ -153,7 +163,7 @@ class OtherFileTests(unittest.TestCase):
        # verify that we get a sensible error message for bad mode argument
        bad_mode = "qwerty"
        try:
-            f = open(TESTFN, bad_mode)
+            f = self.open(TESTFN, bad_mode)
        except ValueError as msg:
            if msg.args[0] != 0:
                s = str(msg)
@@ -170,11 +180,11 @@ class OtherFileTests(unittest.TestCase):
        # misbehaviour especially with repeated close() calls
        for s in (-1, 0, 1, 512):
            try:
-                f = open(TESTFN, 'wb', s)
+                f = self.open(TESTFN, 'wb', s)
                f.write(str(s).encode("ascii"))
                f.close()
                f.close()
-                f = open(TESTFN, 'rb', s)
+                f = self.open(TESTFN, 'rb', s)
                d = int(f.read().decode("ascii"))
                f.close()
                f.close()
@@ -187,13 +197,13 @@ class OtherFileTests(unittest.TestCase):
        # "file.truncate fault on windows"
        os.unlink(TESTFN)
-        f = open(TESTFN, 'wb')
+        f = self.open(TESTFN, 'wb')
        try:
            f.write(b'12345678901')   # 11 bytes
            f.close()
-            f = open(TESTFN,'rb+')
+            f = self.open(TESTFN,'rb+')
            data = f.read(5)
            if data != b'12345':
                self.fail("Read on file opened for update failed %r" % data)
@@ -233,13 +243,13 @@ class OtherFileTests(unittest.TestCase):
        try:
            # Prepare the testfile
-            bag = open(TESTFN, "wb")
+            bag = self.open(TESTFN, "wb")
            bag.write(filler * nchunks)
            bag.writelines(testlines)
            bag.close()
            # Test for appropriate errors mixing read* and iteration
            for methodname, args in methods:
-                f = open(TESTFN, 'rb')
+                f = self.open(TESTFN, 'rb')
                if next(f) != filler:
                    self.fail, "Broken testfile"
                meth = getattr(f, methodname)
@@ -253,7 +263,7 @@ class OtherFileTests(unittest.TestCase):
            # ("h", "a", "m", "\n"), so 4096 lines of that should get us
            # exactly on the buffer boundary for any power-of-2 buffersize
            # between 4 and 16384 (inclusive).
-            f = open(TESTFN, 'rb')
+            f = self.open(TESTFN, 'rb')
            for i in range(nchunks):
                next(f)
            testline = testlines.pop(0)
@@ -295,7 +305,7 @@ class OtherFileTests(unittest.TestCase):
                self.fail("readlines() after next() with empty buffer "
                          "failed. Got %r, expected %r" % (line, testline))
            # Reading after iteration hit EOF shouldn't hurt either
-            f = open(TESTFN, 'rb')
+            f = self.open(TESTFN, 'rb')
            try:
                for line in f:
                    pass
@@ -311,12 +321,19 @@ class OtherFileTests(unittest.TestCase):
        finally:
            os.unlink(TESTFN)
+class COtherFileTests(OtherFileTests):
+    open = io.open
+class PyOtherFileTests(OtherFileTests):
+    open = staticmethod(pyio.open)
 def test_main():
    # Historically, these tests have been sloppy about removing TESTFN.
    # So get rid of it no matter what.
    try:
-        run_unittest(AutoFileTests, OtherFileTests)
+        run_unittest(CAutoFileTests, PyAutoFileTests,
+                     COtherFileTests, PyOtherFileTests)
    finally:
        if os.path.exists(TESTFN):
            os.unlink(TESTFN)

--- a/Lib/test/test_fileio.py
+++ b/Lib/test/test_fileio.py
@@ -10,13 +10,13 @@ from test.support import (TESTFN, findfile, check_warnings, run_unittest,
                          make_bad_fd)
 from collections import UserList
-import _fileio
+from _io import FileIO as _FileIO
 class AutoFileTests(unittest.TestCase):
    # file tests for which a test file is automatically set up
    def setUp(self):
-        self.f = _fileio._FileIO(TESTFN, 'w')
+        self.f = _FileIO(TESTFN, 'w')
    def tearDown(self):
        if self.f:
@@ -63,13 +63,13 @@ class AutoFileTests(unittest.TestCase):
        self.f.write(bytes([1, 2]))
        self.f.close()
        a = array('b', b'x'*10)
-        self.f = _fileio._FileIO(TESTFN, 'r')
+        self.f = _FileIO(TESTFN, 'r')
        n = self.f.readinto(a)
        self.assertEquals(array('b', [1, 2]), a[:n])
    def testRepr(self):
        self.assertEquals(repr(self.f),
-                          "_fileio._FileIO(%d, %s)" % (self.f.fileno(),
+                          "io.FileIO(%d, %s)" % (self.f.fileno(),
                                                       repr(self.f.mode)))
    def testErrors(self):
@@ -80,7 +80,7 @@ class AutoFileTests(unittest.TestCase):
        self.assertRaises(ValueError, f.read, 10) # Open for reading
        f.close()
        self.assert_(f.closed)
-        f = _fileio._FileIO(TESTFN, 'r')
+        f = _FileIO(TESTFN, 'r')
        self.assertRaises(TypeError, f.readinto, "")
        self.assert_(not f.closed)
        f.close()
@@ -106,7 +106,7 @@ class AutoFileTests(unittest.TestCase):
        # Windows always returns "[Errno 13]: Permission denied
        # Unix calls dircheck() and returns "[Errno 21]: Is a directory"
        try:
-            _fileio._FileIO('.', 'r')
+            _FileIO('.', 'r')
        except IOError as e:
            self.assertNotEqual(e.errno, 0)
            self.assertEqual(e.filename, ".")
@@ -118,19 +118,19 @@ class OtherFileTests(unittest.TestCase):
    def testAbles(self):
        try:
-            f = _fileio._FileIO(TESTFN, "w")
+            f = _FileIO(TESTFN, "w")
            self.assertEquals(f.readable(), False)
            self.assertEquals(f.writable(), True)
            self.assertEquals(f.seekable(), True)
            f.close()
-            f = _fileio._FileIO(TESTFN, "r")
+            f = _FileIO(TESTFN, "r")
            self.assertEquals(f.readable(), True)
            self.assertEquals(f.writable(), False)
            self.assertEquals(f.seekable(), True)
            f.close()
-            f = _fileio._FileIO(TESTFN, "a+")
+            f = _FileIO(TESTFN, "a+")
            self.assertEquals(f.readable(), True)
            self.assertEquals(f.writable(), True)
            self.assertEquals(f.seekable(), True)
@@ -139,14 +139,14 @@ class OtherFileTests(unittest.TestCase):
            if sys.platform != "win32":
                try:
-                    f = _fileio._FileIO("/dev/tty", "a")
+                    f = _FileIO("/dev/tty", "a")
                except EnvironmentError:
                    # When run in a cron job there just aren't any
                    # ttys, so skip the test.  This also handles other
                    # OS'es that don't support /dev/tty.
                    pass
                else:
-                    f = _fileio._FileIO("/dev/tty", "a")
+                    f = _FileIO("/dev/tty", "a")
                    self.assertEquals(f.readable(), False)
                    self.assertEquals(f.writable(), True)
                    if sys.platform != "darwin" and \
@@ -163,7 +163,7 @@ class OtherFileTests(unittest.TestCase):
        # check invalid mode strings
        for mode in ("", "aU", "wU+", "rw", "rt"):
            try:
-                f = _fileio._FileIO(TESTFN, mode)
+                f = _FileIO(TESTFN, mode)
            except ValueError:
                pass
            else:
@@ -172,10 +172,26 @@ class OtherFileTests(unittest.TestCase):
    def testUnicodeOpen(self):
        # verify repr works for unicode too
-        f = _fileio._FileIO(str(TESTFN), "w")
+        f = _FileIO(str(TESTFN), "w")
        f.close()
        os.unlink(TESTFN)
+    def testBytesOpen(self):
+        # Opening a bytes filename
+        try:
+            fn = TESTFN.encode("ascii")
+        except UnicodeEncodeError:
+            # Skip test
+            return
+        f = _FileIO(fn, "w")
+        try:
+            f.write(b"abc")
+            f.close()
+            with open(TESTFN, "rb") as f:
+                self.assertEquals(f.read(), b"abc")
+        finally:
+            os.unlink(TESTFN)
    def testInvalidFd(self):
        self.assertRaises(ValueError, _fileio._FileIO, -10)
        self.assertRaises(OSError, _fileio._FileIO, make_bad_fd())
@@ -184,7 +200,7 @@ class OtherFileTests(unittest.TestCase):
        # verify that we get a sensible error message for bad mode argument
        bad_mode = "qwerty"
        try:
-            f = _fileio._FileIO(TESTFN, bad_mode)
+            f = _FileIO(TESTFN, bad_mode)
        except ValueError as msg:
            if msg.args[0] != 0:
                s = str(msg)
@@ -200,11 +216,11 @@ class OtherFileTests(unittest.TestCase):
        def bug801631():
            # SF bug <http://www.python.org/sf/801631>
            # "file.truncate fault on windows"
-            f = _fileio._FileIO(TESTFN, 'w')
+            f = _FileIO(TESTFN, 'w')
            f.write(bytes(range(11)))
            f.close()
-            f = _fileio._FileIO(TESTFN,'r+')
+            f = _FileIO(TESTFN,'r+')
            data = f.read(5)
            if data != bytes(range(5)):
                self.fail("Read on file opened for update failed %r" % data)
@@ -244,14 +260,14 @@ class OtherFileTests(unittest.TestCase):
                pass
    def testInvalidInit(self):
-        self.assertRaises(TypeError, _fileio._FileIO, "1", 0, 0)
+        self.assertRaises(TypeError, _FileIO, "1", 0, 0)
    def testWarnings(self):
        with check_warnings() as w:
            self.assertEqual(w.warnings, [])
-            self.assertRaises(TypeError, _fileio._FileIO, [])
+            self.assertRaises(TypeError, _FileIO, [])
            self.assertEqual(w.warnings, [])
-            self.assertRaises(ValueError, _fileio._FileIO, "/some/invalid/name", "rt")
+            self.assertRaises(ValueError, _FileIO, "/some/invalid/name", "rt")
            self.assertEqual(w.warnings, [])

--- a/Lib/test/test_io.py
+++ b/Lib/test/test_io.py
--- a/Lib/test/test_largefile.py
+++ b/Lib/test/test_largefile.py
@@ -7,6 +7,8 @@ import sys
 import unittest
 from test.support import run_unittest, TESTFN, verbose, requires, \
                              TestSkipped, unlink
+import io  # C implementation of io
+import _pyio as pyio # Python implementation of io
 try:
    import signal
@@ -21,7 +23,7 @@ except (ImportError, AttributeError):
 size = 2500000000
-class TestCase(unittest.TestCase):
+class LargeFileTest(unittest.TestCase):
    """Test that each file function works as expected for a large
    (i.e. > 2GB, do  we have to check > 4GB) files.
@@ -34,7 +36,7 @@ class TestCase(unittest.TestCase):
    def test_seek(self):
        if verbose:
            print('create large file via seek (may be sparse file) ...')
-        with open(TESTFN, 'wb') as f:
+        with self.open(TESTFN, 'wb') as f:
            f.write(b'z')
            f.seek(0)
            f.seek(size)
@@ -52,7 +54,7 @@ class TestCase(unittest.TestCase):
    def test_seek_read(self):
        if verbose:
            print('play around with seek() and read() with the built largefile')
-        with open(TESTFN, 'rb') as f:
+        with self.open(TESTFN, 'rb') as f:
            self.assertEqual(f.tell(), 0)
            self.assertEqual(f.read(1), b'z')
            self.assertEqual(f.tell(), 1)
@@ -85,7 +87,7 @@ class TestCase(unittest.TestCase):
    def test_lseek(self):
        if verbose:
            print('play around with os.lseek() with the built largefile')
-        with open(TESTFN, 'rb') as f:
+        with self.open(TESTFN, 'rb') as f:
            self.assertEqual(os.lseek(f.fileno(), 0, 0), 0)
            self.assertEqual(os.lseek(f.fileno(), 42, 0), 42)
            self.assertEqual(os.lseek(f.fileno(), 42, 1), 84)
@@ -100,7 +102,7 @@ class TestCase(unittest.TestCase):
    def test_truncate(self):
        if verbose:
            print('try truncate')
-        with open(TESTFN, 'r+b') as f:
+        with self.open(TESTFN, 'r+b') as f:
            # this is already decided before start running the test suite
            # but we do it anyway for extra protection
            if not hasattr(f, 'truncate'):
@@ -143,7 +145,7 @@ def test_main():
        # Only run if the current filesystem supports large files.
        # (Skip this test on Windows, since we now always support
        # large files.)
-        f = open(TESTFN, 'wb')
+        f = open(TESTFN, 'wb', buffering=0)
        try:
            # 2**31 == 2147483648
            f.seek(2147483649)
@@ -158,14 +160,19 @@ def test_main():
        else:
            f.close()
    suite = unittest.TestSuite()
-    suite.addTest(TestCase('test_seek'))
+    for _open, prefix in [(io.open, 'C'), (pyio.open, 'Py')]:
-    suite.addTest(TestCase('test_osstat'))
+        class TestCase(LargeFileTest):
-    suite.addTest(TestCase('test_seek_read'))
+            pass
-    suite.addTest(TestCase('test_lseek'))
+        TestCase.open = staticmethod(_open)
-    with open(TESTFN, 'w') as f:
+        TestCase.__name__ = prefix + LargeFileTest.__name__
-        if hasattr(f, 'truncate'):
+        suite.addTest(TestCase('test_seek'))
-            suite.addTest(TestCase('test_truncate'))
+        suite.addTest(TestCase('test_osstat'))
-    unlink(TESTFN)
+        suite.addTest(TestCase('test_seek_read'))
+        suite.addTest(TestCase('test_lseek'))
+        with _open(TESTFN, 'wb') as f:
+            if hasattr(f, 'truncate'):
+                suite.addTest(TestCase('test_truncate'))
+        unlink(TESTFN)
    try:
        run_unittest(suite)
    finally:

--- a/Lib/test/test_memoryio.py
+++ b/Lib/test/test_memoryio.py
@@ -7,13 +7,52 @@ import unittest
 from test import support
 import io
+import _pyio as pyio
 import sys
-try:
+class MemorySeekTestMixin:
-    import _bytesio, _stringio
-    has_c_implementation = True
+    def testInit(self):
-except ImportError:
+        buf = self.buftype("1234567890")
-    has_c_implementation = False
+        bytesIo = self.ioclass(buf)
+    def testRead(self):
+        buf = self.buftype("1234567890")
+        bytesIo = self.ioclass(buf)
+        self.assertEquals(buf[:1], bytesIo.read(1))
+        self.assertEquals(buf[1:5], bytesIo.read(4))
+        self.assertEquals(buf[5:], bytesIo.read(900))
+        self.assertEquals(self.EOF, bytesIo.read())
+    def testReadNoArgs(self):
+        buf = self.buftype("1234567890")
+        bytesIo = self.ioclass(buf)
+        self.assertEquals(buf, bytesIo.read())
+        self.assertEquals(self.EOF, bytesIo.read())
+    def testSeek(self):
+        buf = self.buftype("1234567890")
+        bytesIo = self.ioclass(buf)
+        bytesIo.read(5)
+        bytesIo.seek(0)
+        self.assertEquals(buf, bytesIo.read())
+        bytesIo.seek(3)
+        self.assertEquals(buf[3:], bytesIo.read())
+        self.assertRaises(TypeError, bytesIo.seek, 0.0)
+    def testTell(self):
+        buf = self.buftype("1234567890")
+        bytesIo = self.ioclass(buf)
+        self.assertEquals(0, bytesIo.tell())
+        bytesIo.seek(5)
+        self.assertEquals(5, bytesIo.tell())
+        bytesIo.seek(10000)
+        self.assertEquals(10000, bytesIo.tell())
 class MemoryTestMixin:
@@ -148,7 +187,7 @@ class MemoryTestMixin:
        self.assertEqual(memio.readline(), self.EOF)
        memio.seek(0)
        self.assertEqual(type(memio.readline()), type(buf))
-        self.assertEqual(memio.readline(None), buf)
+        self.assertEqual(memio.readline(), buf)
        self.assertRaises(TypeError, memio.readline, '')
        memio.close()
        self.assertRaises(ValueError,  memio.readline)
@@ -296,11 +335,11 @@ class MemoryTestMixin:
        self.assertEqual(test2(), buf)
-class PyBytesIOTest(MemoryTestMixin, unittest.TestCase):
+class PyBytesIOTest(MemoryTestMixin, MemorySeekTestMixin, unittest.TestCase):
    @staticmethod
    def buftype(s):
        return s.encode("ascii")
-    ioclass = io._BytesIO
+    ioclass = pyio.BytesIO
    EOF = b""
    def test_read1(self):
@@ -371,11 +410,32 @@ class PyBytesIOTest(MemoryTestMixin, unittest.TestCase):
        self.assertEqual(memio.getvalue(), buf)
-class PyStringIOTest(MemoryTestMixin, unittest.TestCase):
+class PyStringIOTest(MemoryTestMixin, MemorySeekTestMixin, unittest.TestCase):
    buftype = str
-    ioclass = io._StringIO
+    ioclass = pyio.StringIO
    EOF = ""
+    # TextIO-specific behaviour.
+    def test_newlines_property(self):
+        memio = self.ioclass(newline=None)
+        # The C StringIO decodes newlines in write() calls, but the Python
+        # implementation only does when reading.  This function forces them to
+        # be decoded for testing.
+        def force_decode():
+            memio.seek(0)
+            memio.read()
+        self.assertEqual(memio.newlines, None)
+        memio.write("a\n")
+        force_decode()
+        self.assertEqual(memio.newlines, "\n")
+        memio.write("b\r\n")
+        force_decode()
+        self.assertEqual(memio.newlines, ("\n", "\r\n"))
+        memio.write("c\rd")
+        force_decode()
+        self.assertEqual(memio.newlines, ("\r", "\n", "\r\n"))
    def test_relative_seek(self):
        memio = self.ioclass()
@@ -386,32 +446,99 @@ class PyStringIOTest(MemoryTestMixin, unittest.TestCase):
        self.assertRaises(IOError, memio.seek, 1, 1)
        self.assertRaises(IOError, memio.seek, 1, 2)
+    def test_textio_properties(self):
+        memio = self.ioclass()
+        # These are just dummy values but we nevertheless check them for fear
+        # of unexpected breakage.
+        self.assertEqual(memio.encoding, "utf-8")
+        self.assertEqual(memio.errors, "strict")
+        self.assertEqual(memio.line_buffering, False)
+    def test_newline_none(self):
+        # newline=None
+        memio = self.ioclass("a\nb\r\nc\rd", newline=None)
+        self.assertEqual(list(memio), ["a\n", "b\n", "c\n", "d"])
+        memio.seek(0)
+        self.assertEqual(memio.read(1), "a")
+        self.assertEqual(memio.read(2), "\nb")
+        self.assertEqual(memio.read(2), "\nc")
+        self.assertEqual(memio.read(1), "\n")
+        memio = self.ioclass(newline=None)
+        self.assertEqual(2, memio.write("a\n"))
+        self.assertEqual(3, memio.write("b\r\n"))
+        self.assertEqual(3, memio.write("c\rd"))
+        memio.seek(0)
+        self.assertEqual(memio.read(), "a\nb\nc\nd")
+        memio = self.ioclass("a\r\nb", newline=None)
+        self.assertEqual(memio.read(3), "a\nb")
+    def test_newline_empty(self):
+        # newline=""
+        memio = self.ioclass("a\nb\r\nc\rd", newline="")
+        self.assertEqual(list(memio), ["a\n", "b\r\n", "c\r", "d"])
+        memio.seek(0)
+        self.assertEqual(memio.read(4), "a\nb\r")
+        self.assertEqual(memio.read(2), "\nc")
+        self.assertEqual(memio.read(1), "\r")
+        memio = self.ioclass(newline="")
+        self.assertEqual(2, memio.write("a\n"))
+        self.assertEqual(2, memio.write("b\r"))
+        self.assertEqual(2, memio.write("\nc"))
+        self.assertEqual(2, memio.write("\rd"))
+        memio.seek(0)
+        self.assertEqual(list(memio), ["a\n", "b\r\n", "c\r", "d"])
+    def test_newline_lf(self):
+        # newline="\n"
+        memio = self.ioclass("a\nb\r\nc\rd")
+        self.assertEqual(list(memio), ["a\n", "b\r\n", "c\rd"])
+    def test_newline_cr(self):
+        # newline="\r"
+        memio = self.ioclass("a\nb\r\nc\rd", newline="\r")
+        memio.seek(0)
+        self.assertEqual(memio.read(), "a\rb\r\rc\rd")
+        memio.seek(0)
+        self.assertEqual(list(memio), ["a\r", "b\r", "\r", "c\r", "d"])
+    def test_newline_crlf(self):
+        # newline="\r\n"
+        memio = self.ioclass("a\nb\r\nc\rd", newline="\r\n")
+        memio.seek(0)
+        self.assertEqual(memio.read(), "a\r\nb\r\r\nc\rd")
+        memio.seek(0)
+        self.assertEqual(list(memio), ["a\r\n", "b\r\r\n", "c\rd"])
+    def test_issue5265(self):
+        # StringIO can duplicate newlines in universal newlines mode
+        memio = self.ioclass("a\r\nb\r\n", newline=None)
+        self.assertEqual(memio.read(5), "a\nb\n")
+class CBytesIOTest(PyBytesIOTest):
+    ioclass = io.BytesIO
+class CStringIOTest(PyStringIOTest):
+    ioclass = io.StringIO
    # XXX: For the Python version of io.StringIO, this is highly
    # dependent on the encoding used for the underlying buffer.
-    # def test_widechar(self):
+    def test_widechar(self):
-    #     buf = self.buftype("\U0002030a\U00020347")
+        buf = self.buftype("\U0002030a\U00020347")
-    #     memio = self.ioclass(buf)
+        memio = self.ioclass(buf)
-    #
-    #     self.assertEqual(memio.getvalue(), buf)
+        self.assertEqual(memio.getvalue(), buf)
-    #     self.assertEqual(memio.write(buf), len(buf))
+        self.assertEqual(memio.write(buf), len(buf))
-    #     self.assertEqual(memio.tell(), len(buf))
+        self.assertEqual(memio.tell(), len(buf))
-    #     self.assertEqual(memio.getvalue(), buf)
+        self.assertEqual(memio.getvalue(), buf)
-    #     self.assertEqual(memio.write(buf), len(buf))
+        self.assertEqual(memio.write(buf), len(buf))
-    #     self.assertEqual(memio.tell(), len(buf) * 2)
+        self.assertEqual(memio.tell(), len(buf) * 2)
-    #     self.assertEqual(memio.getvalue(), buf + buf)
+        self.assertEqual(memio.getvalue(), buf + buf)
-if has_c_implementation:
-    class CBytesIOTest(PyBytesIOTest):
-        ioclass = io.BytesIO
-    class CStringIOTest(PyStringIOTest):
-        ioclass = io.StringIO
 def test_main():
-    tests = [PyBytesIOTest, PyStringIOTest]
+    tests = [PyBytesIOTest, PyStringIOTest, CBytesIOTest, CStringIOTest]
-    if has_c_implementation:
-        tests.extend([CBytesIOTest, CStringIOTest])
    support.run_unittest(*tests)
 if __name__ == '__main__':

--- a/Lib/test/test_univnewlines.py
+++ b/Lib/test/test_univnewlines.py
 # Tests universal newline support for both reading and parsing files.
+import io
+import _pyio as pyio
 import unittest
 import os
 import sys
@@ -35,7 +37,7 @@ class TestGenericUnivNewlines(unittest.TestCase):
    WRITEMODE = 'wb'
    def setUp(self):
-        fp = open(support.TESTFN, self.WRITEMODE)
+        fp = self.open(support.TESTFN, self.WRITEMODE)
        data = self.DATA
        if "b" in self.WRITEMODE:
            data = data.encode("ascii")
@@ -49,19 +51,19 @@ class TestGenericUnivNewlines(unittest.TestCase):
            pass
    def test_read(self):
-        fp = open(support.TESTFN, self.READMODE)
+        fp = self.open(support.TESTFN, self.READMODE)
        data = fp.read()
        self.assertEqual(data, DATA_LF)
        self.assertEqual(repr(fp.newlines), repr(self.NEWLINE))
    def test_readlines(self):
-        fp = open(support.TESTFN, self.READMODE)
+        fp = self.open(support.TESTFN, self.READMODE)
        data = fp.readlines()
        self.assertEqual(data, DATA_SPLIT)
        self.assertEqual(repr(fp.newlines), repr(self.NEWLINE))
    def test_readline(self):
-        fp = open(support.TESTFN, self.READMODE)
+        fp = self.open(support.TESTFN, self.READMODE)
        data = []
        d = fp.readline()
        while d:
@@ -71,7 +73,7 @@ class TestGenericUnivNewlines(unittest.TestCase):
        self.assertEqual(repr(fp.newlines), repr(self.NEWLINE))
    def test_seek(self):
-        fp = open(support.TESTFN, self.READMODE)
+        fp = self.open(support.TESTFN, self.READMODE)
        fp.readline()
        pos = fp.tell()
        data = fp.readlines()
@@ -94,7 +96,7 @@ class TestCRLFNewlines(TestGenericUnivNewlines):
    DATA = DATA_CRLF
    def test_tell(self):
-        fp = open(support.TESTFN, self.READMODE)
+        fp = self.open(support.TESTFN, self.READMODE)
        self.assertEqual(repr(fp.newlines), repr(None))
        data = fp.readline()
        pos = fp.tell()
@@ -106,12 +108,22 @@ class TestMixedNewlines(TestGenericUnivNewlines):
 def test_main():
-    support.run_unittest(
+    base_tests = (TestCRNewlines,
-        TestCRNewlines,
+                  TestLFNewlines,
-        TestLFNewlines,
+                  TestCRLFNewlines,
-        TestCRLFNewlines,
+                  TestMixedNewlines)
-        TestMixedNewlines
+    tests = []
-     )
+    # Test the C and Python implementations.
+    for test in base_tests:
+        class CTest(test):
+            open = io.open
+        CTest.__name__ = "C" + test.__name__
+        class PyTest(test):
+            open = staticmethod(pyio.open)
+        PyTest.__name__ = "Py" + test.__name__
+        tests.append(CTest)
+        tests.append(PyTest)
+    support.run_unittest(*tests)
 if __name__ == '__main__':
    test_main()
--- a/Lib/test/test_uu.py
+++ b/Lib/test/test_uu.py
@@ -32,6 +32,8 @@ class FakeIO(io.TextIOWrapper):
                                     encoding=encoding,
                                     errors=errors,
                                     newline=newline)
+        self._encoding = encoding
+        self._errors = errors
        if initial_value:
            if not isinstance(initial_value, str):
                initial_value = str(initial_value)

--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -193,6 +193,15 @@ MODULE_OBJS=	\
 # Used of signalmodule.o is not available
 SIGNAL_OBJS=	@SIGNAL_OBJS@
+IO_H=		Modules/_iomodule.h
+IO_OBJS=	\
+		Modules/io.o \
+		Modules/_iobase.o \
+		Modules/_fileio.o \
+		Modules/_bufferedio.o \
+		Modules/_textio.o \
+		Modules/_bytesio.o
 ##########################################################################
 # Grammar
@@ -521,6 +530,7 @@ Modules/getpath.o: $(srcdir)/Modules/getpath.c Makefile
 Modules/python.o: $(srcdir)/Modules/python.c
 	$(MAINCC) -c $(PY_CFLAGS) -o $@ $(srcdir)/Modules/python.c
+$(IO_OBJS): $(IO_H)
 $(GRAMMAR_H) $(GRAMMAR_C): $(PGEN) $(GRAMMAR_INPUT)
 		-@$(INSTALL) -d Include

--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -14,6 +14,8 @@ Core and Builtins
 =======
+- The io module has been reimplemented in C for speed.
 - Give dict views an informative __repr__.
 - Issue #5247: Improve error message when unknown format codes are

--- a/Modules/Setup.dist
+++ b/Modules/Setup.dist
@@ -111,10 +111,10 @@ pwd pwdmodule.c			# this is needed to find out the user's home dir
 				# if $HOME is not set
 _sre _sre.c			# Fredrik Lundh's new regular expressions
 _codecs _codecsmodule.c		# access to the builtin codecs and codec registry
-_fileio _fileio.c		# Standard I/O baseline
 _weakref _weakref.c		# weak references
-_bytesio _bytesio.c		# For Lib/io.py
-_stringio _stringio.c		# For Lib/io.py
+# Standard I/O baseline
+_io io.c _iobase.c _fileio.c _bytesio.c _bufferedio.c _textio.c _stringio.c
 # The zipimport module is always imported at startup. Having it as a
 # builtin module avoids some bootstrapping problems and reduces overhead.
@@ -164,7 +164,6 @@ _symtable symtablemodule.c
 #_collections _collectionsmodule.c # Container types
 #itertools itertoolsmodule.c	# Functions creating iterators for efficient looping 
 #atexit atexitmodule.c      # Register functions to be run at interpreter-shutdown
-#_functools _functoolsmodule.c	# Tools for working with functions and callable objects
 #_elementtree -I$(srcdir)/Modules/expat -DHAVE_EXPAT_CONFIG_H -DUSE_PYEXPAT_CAPI _elementtree.c	# elementtree accelerator
 #_pickle _pickle.c	# pickle accelerator
 #datetime datetimemodule.c	# date/time type

--- a/Modules/_bufferedio.c
+++ b/Modules/_bufferedio.c
--- a/Modules/_bytesio.c
+++ b/Modules/_bytesio.c
 #include "Python.h"
+#include "structmember.h"       /* for offsetof() */
+#include "_iomodule.h"
 typedef struct {
    PyObject_HEAD
@@ -6,6 +8,8 @@ typedef struct {
    Py_ssize_t pos;
    Py_ssize_t string_size;
    size_t buf_size;
+    PyObject *dict;
+    PyObject *weakreflist;
 } BytesIOObject;
 #define CHECK_CLOSED(self)                                  \
@@ -144,10 +148,12 @@ write_bytes(BytesIOObject *self, const char *bytes, Py_ssize_t len)
 static PyObject *
 bytesio_get_closed(BytesIOObject *self)
 {
-    if (self->buf == NULL)
+    if (self->buf == NULL) {
        Py_RETURN_TRUE;
-    else
+    }
+    else {
        Py_RETURN_FALSE;
+    }
 }
 /* Generic getter for the writable, readable and seekable properties */
@@ -532,22 +538,22 @@ PyDoc_STRVAR(write_doc,
 static PyObject *
 bytesio_write(BytesIOObject *self, PyObject *obj)
 {
-    const char *bytes;
-    Py_ssize_t size;
    Py_ssize_t n = 0;
+    Py_buffer buf;
+    PyObject *result = NULL;
    CHECK_CLOSED(self);
-    if (PyObject_AsReadBuffer(obj, (void *)&bytes, &size) < 0)
+    if (PyObject_GetBuffer(obj, &buf, PyBUF_CONTIG_RO) < 0)
        return NULL;
-    if (size != 0) {
+    if (buf.len != 0)
-        n = write_bytes(self, bytes, size);
+        n = write_bytes(self, buf.buf, buf.len);
-        if (n < 0)
+    if (n >= 0)
-            return NULL;
+        result = PyLong_FromSsize_t(n);
-    }
-    return PyLong_FromSsize_t(n);
+    PyBuffer_Release(&buf);
+    return result;
 }
 PyDoc_STRVAR(writelines_doc,
@@ -607,6 +613,7 @@ bytesio_dealloc(BytesIOObject *self)
        PyMem_Free(self->buf);
        self->buf = NULL;
    }
+    Py_TYPE(self)->tp_clear((PyObject *)self);
    Py_TYPE(self)->tp_free(self);
 }
@@ -656,6 +663,24 @@ bytesio_init(BytesIOObject *self, PyObject *args, PyObject *kwds)
    return 0;
 }
+static int
+bytesio_traverse(BytesIOObject *self, visitproc visit, void *arg)
+{
+    Py_VISIT(self->dict);
+    Py_VISIT(self->weakreflist);
+    return 0;
+}
+static int
+bytesio_clear(BytesIOObject *self)
+{
+    Py_CLEAR(self->dict);
+    if (self->weakreflist != NULL)
+        PyObject_ClearWeakRefs((PyObject *)self);
+    return 0;
+}
 static PyGetSetDef bytesio_getsetlist[] = {
    {"closed",  (getter)bytesio_get_closed, NULL,
     "True if the file is closed."},
@@ -689,9 +714,9 @@ PyDoc_STRVAR(bytesio_doc,
 "Create a buffered I/O implementation using an in-memory bytes\n"
 "buffer, ready for reading and writing.");
-static PyTypeObject BytesIO_Type = {
+PyTypeObject PyBytesIO_Type = {
    PyVarObject_HEAD_INIT(NULL, 0)
-    "_bytesio._BytesIO",                       /*tp_name*/
+    "_io.BytesIO",                             /*tp_name*/
    sizeof(BytesIOObject),                     /*tp_basicsize*/
    0,                                         /*tp_itemsize*/
    (destructor)bytesio_dealloc,               /*tp_dealloc*/
@@ -709,12 +734,13 @@ static PyTypeObject BytesIO_Type = {
    0,                                         /*tp_getattro*/
    0,                                         /*tp_setattro*/
    0,                                         /*tp_as_buffer*/
-    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,  /*tp_flags*/
+    Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE |
+    Py_TPFLAGS_HAVE_GC,                        /*tp_flags*/
    bytesio_doc,                               /*tp_doc*/
-    0,                                         /*tp_traverse*/
+    (traverseproc)bytesio_traverse,            /*tp_traverse*/
-    0,                                         /*tp_clear*/
+    (inquiry)bytesio_clear,                    /*tp_clear*/
    0,                                         /*tp_richcompare*/
-    0,                                         /*tp_weaklistoffset*/
+    offsetof(BytesIOObject, weakreflist),      /*tp_weaklistoffset*/
    PyObject_SelfIter,                         /*tp_iter*/
    (iternextfunc)bytesio_iternext,            /*tp_iternext*/
    bytesio_methods,                           /*tp_methods*/
@@ -724,36 +750,8 @@ static PyTypeObject BytesIO_Type = {
    0,                                         /*tp_dict*/
    0,                                         /*tp_descr_get*/
    0,                                         /*tp_descr_set*/
-    0,                                         /*tp_dictoffset*/
+    offsetof(BytesIOObject, dict),             /*tp_dictoffset*/
    (initproc)bytesio_init,                    /*tp_init*/
    0,                                         /*tp_alloc*/
    bytesio_new,                               /*tp_new*/
 };
-static struct PyModuleDef _bytesiomodule = {
-	PyModuleDef_HEAD_INIT,
-	"_bytesio",
-	NULL,
-	-1,
-	NULL,
-	NULL,
-	NULL,
-	NULL,
-	NULL
-};
-PyMODINIT_FUNC
-PyInit__bytesio(void)
-{
-    PyObject *m;
-    if (PyType_Ready(&BytesIO_Type) < 0)
-        return NULL;
-    m = PyModule_Create(&_bytesiomodule);
-    if (m == NULL)
-        return NULL;
-    Py_INCREF(&BytesIO_Type);
-    PyModule_AddObject(m, "_BytesIO", (PyObject *)&BytesIO_Type);
-    return m;
-}
--- a/Modules/_fileio.c
+++ b/Modules/_fileio.c
--- a/Modules/_iobase.c
+++ b/Modules/_iobase.c
--- a/Modules/_iomodule.h
+++ b/Modules/_iomodule.h
--- a/Modules/_stringio.c
+++ b/Modules/_stringio.c
--- a/Modules/_textio.c
+++ b/Modules/_textio.c
--- a/Modules/io.c
+++ b/Modules/io.c
--- a/PC/VC6/pythoncore.dsp
+++ b/PC/VC6/pythoncore.dsp
--- a/PC/config.c
+++ b/PC/config.c
--- a/PCbuild/pythoncore.vcproj
+++ b/PCbuild/pythoncore.vcproj
--- a/Python/pythonrun.c
+++ b/Python/pythonrun.c
--- a/setup.py
+++ b/setup.py