CHANGES.rst 13.1 KB
Newer Older
1 2 3
===========
 Changelog
===========
Denis Bilenko's avatar
Denis Bilenko committed
4

5 6
.. currentmodule:: gevent

7 8 9
.. towncrier release notes start


Jason Madden's avatar
Jason Madden committed
10 11 12 13 14 15
20.0.0 (unreleased)
===================

- Nothing changed yet.


Jason Madden's avatar
Jason Madden committed
16
1.5.0 (2020-04-10)
Jason Madden's avatar
Jason Madden committed
17 18
==================

19
- Make `gevent.lock.RLock.acquire` accept the *timeout* parameter.
20 21 22
- Fix an ``AttributeError`` when wrapping gevent's ``FileObject``
  around an opened text stream. Reported in :issue:`1542` by
  dmrlawson.
23
- Fix compilation of libuv on AIX and Solaris 10. See :pr:`1549` and :pr:`1548`
24
  by Arnon Yaari.
Jason Madden's avatar
Jason Madden committed
25

Jason Madden's avatar
Jason Madden committed
26
1.5a4 (2020-03-23)
Jason Madden's avatar
Jason Madden committed
27 28
==================

29 30 31 32
Platform and Packaging Updates
------------------------------

- Python version updates: gevent is now tested with CPython 3.6.10,
33
  3.7.7 and 3.8.2. It is also tested with PyPy2 7.3 and PyPy 3.6
34
  7.3.
Jason Madden's avatar
Jason Madden committed
35

Jason Madden's avatar
Jason Madden committed
36 37 38 39
- The include directories used to compile the C extensions have been
  tweaked with the intent of making it easier to use older debug
  versions of Python. See :issue:`1461`.

40 41 42 43
- The binary wheels and installed package no longer include generated
  C source files and headers. Also excluded are Cython .pxd files,
  which were never documented. Please file an issue if you miss these.

Jason Madden's avatar
Jason Madden committed
44 45 46 47
Library and Dependency Updates
------------------------------

- Upgrade libev from 4.25 to 4.31 and update its embedded
48
  ``config.guess`` to the latest. See :issue:`1504`.
Jason Madden's avatar
Jason Madden committed
49

50 51 52 53 54 55 56 57 58 59 60 61 62 63 64
  .. important::

     libev, when built with ``EV_VERIFY >= 2``, now performs
     verification of file descriptors when IO watchers are started
     *and* when they are stopped. If you first close a file descriptor
     and only then stop an associated watcher, libev will abort the
     process.

     Using the standard gevent socket and file objects handles this
     automatically, but if you're using the IO watchers directly,
     you'll need to watch out for this.

     The binary wheels gevent distributes *do not* set ``EV_VERIFY``
     and don't have this issue.

65 66 67 68
- Make libuv and libev use the Python memory allocators. This assists
  with debugging. The event libraries allocate small amounts of memory
  at startup. The allocation functions have to take the GIL, but
  because of the limited amount of actual allocation that gets done
69 70
  this is not expected to be a bottleneck.

Jason Madden's avatar
Jason Madden committed
71 72 73 74
- Update the bundled ``tblib`` library to the unreleased 1.7.0
  version. The only change is to add more attributes to ``Frame`` and ``Code``
  objects for pytest compatibility. See :pr:`1541`.

75 76 77 78 79 80 81 82 83
Potentially Breaking Changes
----------------------------

- Remove the magic proxy object ``gevent.signal``. This served as both
  a deprecated alias of `gevent.signal_handler` *and* the module
  `gevent.signal`. This made it confusing to humans and static
  analysis tools alike. The alias was deprecated since gevent 1.1b4.
  See :issue:`1596`.

84 85 86 87 88 89 90 91 92 93 94 95 96
Other
-----

- Make `gevent.subprocess.Popen.communicate` raise exceptions raised
  by reading from the process, like the standard library. In
  particular, under Python 3, if the process output is being decoded
  as text, this can now raise ``UnicodeDecodeError``. Reported in
  :issue:`1510` by Ofer Koren.

- Make `gevent.subprocess.Popen.communicate` be more careful about
  closing files. Previously if a timeout error happened, a second call
  to ``communicate`` might not close the pipe.

97 98 99 100
- Add `gevent.contextvars`, a cooperative version of `contextvars`.
  This is available to all Python versions. On Python 3.7 and above,
  where `contextvars` is a standard library module, it is
  monkey-patched by default. See :issue:`1407`.
101

102 103 104 105 106 107 108
- Use `selectors.PollSelector` as the `selectors.DefaultSelector`
  after monkey-patching if `select.poll` was defined. Previously,
  gevent replaced it with `selectors.SelectSelector`, which has a
  different set of limitations (e.g., on certain platforms such as
  glibc Linux, it has a hardcoded limitation of only working with file
  descriptors < 1024). See :issue:`1466` reported by Sam Wong.

109 110 111 112
- Make the dnspython resolver work if dns python had been imported
  before the gevent resolver was initialized. Reported in
  :issue:`1526` by Chris Utz and Josh Zuech.

113

Jason Madden's avatar
Jason Madden committed
114
1.5a3 (2020-01-01)
Jason Madden's avatar
Jason Madden committed
115 116
==================

Jason Madden's avatar
Jason Madden committed
117 118 119
- Python version updates: gevent is now tested with CPython 2.7.17,
  3.5.9, 3.6.9, 3.7.5 and 3.8.0 (final). It is also tested with PyPy2
  7.2 and PyPy 3.6 7.2
Jason Madden's avatar
Jason Madden committed
120

121 122 123 124 125 126
- Fix using monkey-patched ``threading.Lock`` and ``threading.RLock``
  objects as spin locks by making them call ``sleep(0)`` if they
  failed to acquire the lock in a non-blocking call. This lets other
  callbacks run to release the lock, simulating preemptive threading.
  Using spin locks is not recommended, but may have been done in code
  written for threads, especially on Python 3. See :issue:`1464`.
Jason Madden's avatar
Jason Madden committed
127

Jason Madden's avatar
Jason Madden committed
128 129 130 131 132 133 134
- Fix Semaphore (and monkey-patched threading locks) to be fair. This
  eliminates the rare potential for starvation of greenlets. As part
  of this change, the low-level method ``rawlink`` of Semaphore,
  Event, and AsyncResult now always remove the link object when
  calling it, so ``unlink`` can sometimes be optimized out. See
  :issue:`1487`.

135 136 137
- Make ``gevent.pywsgi`` support ``Connection: keep-alive`` in
  HTTP/1.0. Based on :pr:`1331` by tanchuhan.

138 139 140 141 142
- Fix a potential crash using ``gevent.idle()`` when using libuv. See
  :issue:`1489`.

- Fix some potential crashes using libuv async watchers.

143 144 145 146 147 148 149 150 151
- Make ``ThreadPool`` consistently raise ``InvalidThreadUseError``
  when ``spawn`` is called from a thread different than the thread
  that created the threadpool. This has never been allowed, but was
  inconsistently enforced. On gevent 1.3 and before, this would always
  raise "greenlet error: invalid thread switch," or ``LoopExit``. On
  gevent 1.4, it *could* raise ``LoopExit``, depending on the number
  of tasks, but still, calling it from a different thread was likely
  to corrupt libev or libuv internals.

152
- Remove some undocumented, deprecated functions from the threadpool module.
153

154 155 156 157 158 159 160 161 162 163
- libuv: Fix a perceived slowness spawning many greenlets at the same time
  without yielding to the event loop while having no active IO
  watchers or timers. If the time spent launching greenlets exceeded
  the switch interval and there were no other active watchers, then
  the default IO poll time of about .3s would elapse between spawning
  batches. This could theoretically apply for any non-switching
  callbacks. This can be produced in synthetic benchmarks and other
  special circumstances, but real applications are unlikely to be
  affected. See :issue:`1493`.

164 165 166 167
- Fix using the threadpool inside a script or module run with ``python
  -m gevent.monkey``. Previously it would use greenlets instead of
  native threads. See :issue:`1484`.

168 169 170 171 172
- Fix potential crashes in the FFI backends if a watcher was closed
  and stopped in the middle of a callback from the event loop and then
  raised an exception. This could happen if the hub's ``handle_error``
  function was poorly customized, for example. See :issue:`1482`

173 174 175 176 177 178 179 180 181 182 183 184 185
- Make ``gevent.killall`` stop greenlets from running that hadn't been
  run yet. This make it consistent with ``Greenlet.kill()``. See
  :issue:`1473` reported by kochelmonster.

- Make ``gevent.spawn_raw`` set the ``loop`` attribute on returned
  greenlets. This lets them work with more gevent APIs, notably
  ``gevent.killall()``. They already had dictionaries, but this may
  make them slightly larger, depending on platform (on CPython 2.7
  through 3.6 there is no apparent difference for one attribute but on
  CPython 3.7 and 3.8 dictionaries are initially empty and only
  allocate space once an attribute is added; they're still smaller
  than on earlier versions though).

186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214

File Object Changes
-------------------

.. caution:: There may be breaking changes here for applications that
             relied on the old behaviour. The old behaviour was under
             specified and inconsistent and really only worked
             consistently with 'wb' and 'rb' modes, so most
             applications shouldn't be affected.

- The file objects (``FileObjectPosix``, ``FileObjectThread``) now
  consistently support text and binary modes. If neither 'b' nor 't' is given
  in the mode, they will read and write native strings. If 't' is
  given, they will always work with unicode strings, and 'b' will
  always work with byte strings. (``FileObjectPosix`` already worked this
  way.) See :issue:`1441`.

- The file objects accept *encoding*, *errors* and *newline*
  arguments. On Python 2, these are only used if 't' is in the mode.

- The default mode for ``FileObjectPosix`` changed from ``rb`` to simply
  ``r``, for consistency with the other file objects and the standard
  `open` and :func:`io.open` functions.

- Fix ``FileObjectPosix`` improperly being used from multiple
  greenlets. Previously this was hidden by forcing buffering, which
  raised ``RuntimeError``.


Jason Madden's avatar
Jason Madden committed
215
1.5a2 (2019-10-21)
Jason Madden's avatar
Jason Madden committed
216 217
==================

Jason Madden's avatar
Jason Madden committed
218 219 220
- Add support for CPython 3.8.0. (Windows wheels are not yet available.)

- Update to Cython 0.29.13 and cffi 1.12.3.
Jason Madden's avatar
Jason Madden committed
221

222
- Add an ``--module`` option to ``gevent.monkey`` allowing to run a Python
Jason Madden's avatar
Jason Madden committed
223
  module rather than a script. See :pr:`1440`.
224

225 226
- Improve the way joining the main thread works on Python 3.

227 228
- Implement ``SSLSocket.verify_client_post_handshake()`` when available.

229 230 231 232 233 234 235
- Fix tests when TLS1.3 is supported.

- Disable Nagle's algorithm in the backdoor server. This can improve
  interactive response time.

- Test on Python 3.7.4. There are important SSL test fixes.

Jason Madden's avatar
Jason Madden committed
236
1.5a1 (2019-05-02)
Jason Madden's avatar
Jason Madden committed
237 238
==================

239 240 241
Platform and Packaging Updates
------------------------------

242 243 244 245
- Python version updates: gevent is now tested with CPython 2.7.16,
  3.5.6, 3.6.8, and 3.7.2. It is also tested with PyPy2 7.1 and PyPy 3.6
  7.1 (PyPy 7.0 and 7.1 were not capable of running SSL tests on Travis
  CI).
246 247 248

- Support for Python 3.4 has been removed, as that version is no
  longer supported uptstream.
Jason Madden's avatar
Jason Madden committed
249

250 251 252
- gevent binary wheels are now manylinux2010 and include libuv
  support. pip 19 is needed to install them. See :issue:`1346`.

253 254
- gevent is now compiled with Cython 0.29.6 and cffi 1.12.2.

255 256 257 258
- gevent sources include a pyproject.toml file, specifying the build
  requirements and enabling build isolation. pip 18 or above is needed
  to take advantage of this. See :issue:`1180`.

259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282
- libev-cffi: Let the compiler fill in the definition of ``nlink_t`` for
  ``st_nlink`` in ``struct stat``, instead of trying to guess it
  ourself. Reported in :issue:`1372` by Andreas Schwab.

- Remove the ``Makefile``. Its most useful commands, ``make clean``
  and ``make distclean``, can now be accomplished in a cross-platform
  way using ``python setup.py clean`` and ``python setup.py clean
  -a``, respectively. The remainder of the ``Makefile`` contained
  Travis CI commands that have been moved to ``.travis.yml``.

- Deprecate the ``EMBED`` and ``LIBEV_EMBED``, etc, build-time environment
  variables. Instead, use ``GEVENTSETUP_EMBED`` and
  ``GEVENTSETUP_EMBED_LIBEV``. See :issue:`1402`.

- The CFFI backends now respect the embed build-time setting. This allows
  building the libuv backend without embedding libuv (except on Windows).

- Support test resources. This allows disabling tests that use the
  network. See :ref:`limiting-test-resource-usage` for more.


Library and Dependency Updates
------------------------------

Jason Madden's avatar
Jason Madden committed
283 284
- Upgrade libuv from 1.24.0 to 1.27.0.

Jason Madden's avatar
Jason Madden committed
285 286 287
- Upgrade libev from 4.23 to 4.25 and update its embedded
  ``config.guess`` to the latest.

288 289
- Upgrade c-ares from 1.14 to 1.15.

290 291
- dnspython >= 1.16.0 is now required for the dnspython resolver.

292 293 294
Bug fixes
---------

295 296 297
- Python 3.7 subprocess: Copy a ``STARTUPINFO`` passed as a parameter.
  Contributed by AndCycle in :pr:`1352`.

298
- subprocess: ``WIFSTOPPED`` and ``SIGCHLD`` are now handled for
299 300
  determining ``Popen.returncode``. See
  https://bugs.python.org/issue29335
301

302 303 304 305 306
- subprocess: No longer close redirected FDs if they are in
  ``pass_fds``. This is `a bugfix from Python 3.7
  <https://bugs.python.org/issue32270>`_ applied to all versions
  gevent runs on.

307 308 309 310 311 312 313 314 315
- Fix certain operations on a Greenlet in an invalid state (with an
  invalid parent) to raise a `TypeError` sooner rather than an
  `AttributeError` later. This is also slightly faster on CPython with
  Cython. Inspired by :issue:`1363` as reported by Carson Ip. This
  means that some extreme corner cases that might have passed by
  replacing a Greenlet's parent with something that's not a gevent hub
  now no longer will.

- Fix: The ``spawning_stack`` for Greenlets on CPython should now have
316
  correct line numbers in more cases. See :pr:`1379`.
317 318 319 320 321 322 323

Enhancements
------------

- The result of ``gevent.ssl.SSLSocket.makefile()`` can be used as a
  context manager on Python 2.

324 325 326 327 328 329 330
- Python 2: If the backport of the ``_thread_`` module from
  ``futures`` has already been imported at monkey-patch time, also
  patch this module to be consistent. The ``pkg_resources`` package
  imports this, and ``pkg_resources`` is often imported early on
  Python 2 for namespace packages, so if ``futures`` is installed this
  will likely be the case.

331 332
- Python 2: Avoid a memory leak when an `io.BufferedWriter` is wrapped
  around a socket. Reported by Damien Tournoud in :issue:`1318`.
333

334 335 336
- Avoid unbounded memory usage when creating very deep spawn trees.
  Reported in :issue:`1371` by dmrlawson.

Jason Madden's avatar
Jason Madden committed
337 338 339
- Win: Make ``examples/process.py`` do something useful. See
  :pr:`1378` by Robert Iannucci.

340
- Spawning greenlets can be up to 10% faster. See :pr:`1379`.