CHANGES.rst 35.7 KB
Newer Older
1 2 3
===========
 Changelog
===========
Denis Bilenko's avatar
Denis Bilenko committed
4

5 6
.. currentmodule:: gevent

Jason Madden's avatar
Jason Madden committed
7 8 9 10 11 12
1.5a2 (unreleased)
==================

- Nothing changed yet.


Jason Madden's avatar
Jason Madden committed
13
1.5a1 (2019-05-02)
Jason Madden's avatar
Jason Madden committed
14 15
==================

16 17 18 19
- Python version updates: gevent is now tested with CPython 2.7.16,
  3.5.6, 3.6.8, and 3.7.2. It is also tested with PyPy2 7.1 and PyPy 3.6
  7.1 (PyPy 7.0 and 7.1 were not capable of running SSL tests on Travis
  CI).
20 21 22

- Support for Python 3.4 has been removed, as that version is no
  longer supported uptstream.
Jason Madden's avatar
Jason Madden committed
23

24
- gevent is now compiled with Cython 0.29.6 and cffi 1.12.2.
Jason Madden's avatar
Jason Madden committed
25

26 27 28
- gevent binary wheels are now manylinux2010 and include libuv
  support. pip 19 is needed to install them. See :issue:`1346`.

29 30 31 32
- gevent sources include a pyproject.toml file, specifying the build
  requirements and enabling build isolation. pip 18 or above is needed
  to take advantage of this. See :issue:`1180`.

Jason Madden's avatar
Jason Madden committed
33 34
- Upgrade libuv from 1.24.0 to 1.27.0.

Jason Madden's avatar
Jason Madden committed
35 36 37
- Upgrade libev from 4.23 to 4.25 and update its embedded
  ``config.guess`` to the latest.

38 39
- Upgrade c-ares from 1.14 to 1.15.

40 41
- dnspython >= 1.16.0 is now required for the dnspython resolver.

42 43 44
- Python 3.7 subprocess: Copy a ``STARTUPINFO`` passed as a parameter.
  Contributed by AndCycle in :pr:`1352`.

45 46 47
- subprocess: ``WIFSTOPPED`` and ``SIGCHLD`` are now handled for
  determining ``Popen.returncode``. See https://bugs.python.org/issue29335

48 49 50
- The result of ``gevent.ssl.SSLSocket.makefile()`` can be used as a
  context manager on Python 2.

51 52 53 54 55
- subprocess: No longer close redirected FDs if they are in
  ``pass_fds``. This is `a bugfix from Python 3.7
  <https://bugs.python.org/issue32270>`_ applied to all versions
  gevent runs on.

56 57 58 59 60 61 62
- Python 2: If the backport of the ``_thread_`` module from
  ``futures`` has already been imported at monkey-patch time, also
  patch this module to be consistent. The ``pkg_resources`` package
  imports this, and ``pkg_resources`` is often imported early on
  Python 2 for namespace packages, so if ``futures`` is installed this
  will likely be the case.

63 64
- Python 2: Avoid a memory leak when an `io.BufferedWriter` is wrapped
  around a socket. Reported by Damien Tournoud in :issue:`1318`.
65

66 67 68
- Avoid unbounded memory usage when creating very deep spawn trees.
  Reported in :issue:`1371` by dmrlawson.

69 70 71 72
- libev-cffi: Let the compiler fill in the definition of ``nlink_t`` for
  ``st_nlink`` in ``struct stat``, instead of trying to guess it
  ourself. Reported in :issue:`1372` by Andreas Schwab.

Jason Madden's avatar
Jason Madden committed
73 74 75
- Win: Make ``examples/process.py`` do something useful. See
  :pr:`1378` by Robert Iannucci.

76 77 78 79 80 81 82 83 84 85 86 87 88
- Fix certain operations on a Greenlet in an invalid state (with an
  invalid parent) to raise a `TypeError` sooner rather than an
  `AttributeError` later. This is also slightly faster on CPython with
  Cython. Inspired by :issue:`1363` as reported by Carson Ip. This
  means that some extreme corner cases that might have passed by
  replacing a Greenlet's parent with something that's not a gevent hub
  now no longer will.

- Fix: The ``spawning_stack`` for Greenlets on CPython should now have
  correct line numbers in more cases.

- Spawning greenlets can be up to 10% faster.

89 90 91 92 93 94
- Remove the ``Makefile``. Its most useful commands, ``make clean``
  and ``make distclean``, can now be accomplished in a cross-platform
  way using ``python setup.py clean`` and ``python setup.py clean
  -a``, respectively. The remainder of the ``Makefile`` contained
  Travis CI commands that have been moved to ``.travis.yml``.

95 96 97 98
- Deprecate the ``EMBED`` and ``LIBEV_EMBED``, etc, build-time environment
  variables. Instead, use ``GEVENTSETUP_EMBED`` and
  ``GEVENTSETUP_EMBED_LIBEV``. See :issue:`1402`.

99 100 101
- The CFFI backends now respect the embed build-time setting. This allows
  building the libuv backend without embedding libuv (except on Windows).

102 103 104
- Support test resources. This allows disabling tests that use the
  network. See :ref:`limiting-test-resource-usage` for more.

Jason Madden's avatar
Jason Madden committed
105
1.4.0 (2019-01-04)
Jason Madden's avatar
Jason Madden committed
106 107
==================

108 109
- Build with Cython 0.29 in '3str' mode.

110 111
- Test with PyPy 6.0 on Windows.

Jason Madden's avatar
Jason Madden committed
112 113
- Add support for application-wide callbacks when ``Greenlet`` objects
  are started. See :pr:`1289`, provided by Yury Selivanov.
Jason Madden's avatar
Jason Madden committed
114

115 116 117
- Fix consuming a single ready object using
  ``next(gevent.iwait(objs))``. Previously such a construction would
  hang because `iter` was not called. See :pr:`1288`, provided by Josh
Jason Madden's avatar
Jason Madden committed
118
  Snyder. This is not recommended, though, as unwaited objects will
119
  have dangling links (but see next item).
Jason Madden's avatar
Jason Madden committed
120

121 122 123 124
- Make `gevent.iwait` return an iterator that can now also be used as
  a context manager. If you'll only be consuming part of the iterator,
  use it in a ``with`` block to avoid leaking resources. See
  :pr:`1290`, provided by Josh Snyder.
125

126 127
- Fix semaphores to immediately notify links if they are ready and
  ``rawlink()`` is called. This behaves like ``Event`` and
128 129
  ``AsyncEvent``. Note that the order in which semaphore links are
  called is not specified. See :issue:`1287`, reported by Dan Milon.
130

131 132 133
- Improve safety of handling exceptions during interpreter shutdown.
  See :issue:`1295` reported by BobDenar1212.

134 135 136 137 138
- Remove the deprecated ability to specify ``GEVENT_RESOLVER`` and
  other importable settings as a ``path/to/a/package.module.item``.
  This had race conditions and didn't work with complicated resolver
  implementations. Place the required package or module on `sys.path`
  first.
139

140 141 142 143 144
- Reduce the chances that using the blocking monitor functionality
  could result in apparently random ``SystemError:
  Objects/tupleobject.c: bad argument to internal function``. Reported
  in :issue:`1302` by Ulrich Petri.

145 146 147 148
- Refactored the gevent test runner and test suite to make them more
  reusable. In particular, the tests are now run with ``python -m
  gevent.tests``. See :issue:`1293`.

149 150 151 152
- Make a monkey-patched ``socket.getaddrinfo`` return socket module
  enums instead of plain integers for the socket type and address
  family on Python 3. See :issue:`1310` reported by TheYOSH.

153 154 155
  .. note:: This requires Python 3.4.3 or above due to an undocumented
            change in that version.

156 157 158
- Make gevent's pywsgi server set the non-standard environment value
  ``wsgi.input_terminated`` to True. See :issue:`1308`.

159 160 161
- Make `gevent.util.assert_switches` produce more informative messages
  when the assertion fails.

162 163 164 165 166
- Python 2: If a `gevent.socket` was closed asynchronously (in a
  different greenlet or a hub callback), `AttributeError` could result
  if the socket was already in use. Now the correct socket.error
  should be raised.

167 168 169 170
- Fix :meth:`gevent.threadpool.ThreadPool.join` raising a
  `UserWarning` when using the libuv backend. Reported in
  :issue:`1321` by ZeroNet.

171 172 173 174
- Fix ``FileObjectPosix.seek`` raising `OSError` when it should have
  been `IOError` on Python 2. Reported by, and PR by, Ricardo Kirkner.
  See :issue:`1323`.

Jason Madden's avatar
Jason Madden committed
175 176
- Upgrade libuv from 1.23.2 to 1.24.0.

Jason Madden's avatar
Jason Madden committed
177
1.3.7 (2018-10-12)
Jason Madden's avatar
Jason Madden committed
178 179
==================

180 181
- Formatting run info no longer includes ``gevent.local.local``
  objects that have no value in the greenlet. See :issue:`1275`.
182

183
- Fixed negative length in pywsgi's Input read functions for non chunked body.
184
  Reported in :issue:`1274` by tzickel.
185 186 187 188 189

- Upgrade libuv from 1.22.0 to 1.23.2.

- Fix opening files in text mode in CPython 2 on Windows by patching
  libuv. See :issue:`1282` reported by wiggin15.
Jason Madden's avatar
Jason Madden committed
190

Jason Madden's avatar
Jason Madden committed
191
1.3.6 (2018-08-17)
Jason Madden's avatar
Jason Madden committed
192 193
==================

194 195 196 197
- gevent now depends on greenlet 0.4.14 or above. gevent binary wheels
  for 1.3.5 and below must have greenlet 0.4.13 installed on Python
  3.7 or they will crash. Reported by Alexey Stepanov in :issue:`1260`
  and pkittenis in :issue:`1261`.
Jason Madden's avatar
Jason Madden committed
198

199 200 201 202
- :class:`gevent.local.local` subclasses correctly supports
  ``@staticmethod`` functions. Reported by Brendan Powers in
  :issue:`1266`.

Jason Madden's avatar
Jason Madden committed
203

Jason Madden's avatar
Jason Madden committed
204
1.3.5 (2018-07-16)
Jason Madden's avatar
Jason Madden committed
205 206
==================

Jason Madden's avatar
Jason Madden committed
207
- Update the bundled libuv from 1.20.1 to 1.22.0.
208

Jason Madden's avatar
Jason Madden committed
209 210 211
- Test Python 3.7 on Appveyor. Fix the handling of Popen's
  ``close_fds`` argument on 3.7.

212
- Update Python versions tested on Travis, including PyPy to 6.0. See :issue:`1195`.
Jason Madden's avatar
Jason Madden committed
213

214 215 216 217 218 219 220
- :mod:`gevent.queue` imports ``_PySimpleQueue`` instead of
  ``SimpleQueue`` so that it doesn't block the event loop.
  :func:`gevent.monkey.patch_all` makes this same substitution in
  :mod:`queue`. This fixes issues with
  :class:`concurrent.futures.ThreadPoolExecutor` as well. Reported in
  :issue:`1248` by wwqgtxx and :issue:`1251` by pyld.

221 222
- :meth:`gevent.socket.socket.connect` doesn't pass the port (service)
  to :func:`socket.getaddrinfo` when it resolves an ``AF_INET`` or
Jason Madden's avatar
Jason Madden committed
223 224
  ``AF_INET6`` address. (The standard library doesn't either.) This
  fixes an issue on Solaris. Reported in :issue:`1252` by wiggin15.
225 226 227 228

- :meth:`gevent.socket.socket.connect` works with more address
  families, notably AF_TIPC, AF_NETLINK, AF_BLUETOOTH, AF_ALG and AF_VSOCK.

229

Jason Madden's avatar
Jason Madden committed
230
1.3.4 (2018-06-20)
Jason Madden's avatar
Jason Madden committed
231 232
==================

233 234 235 236
- Be more careful about issuing ``MonkeyPatchWarning`` for ssl
  imports. Now, we only issue it if we detect the one specific
  condition that is known to lead to RecursionError. This may produce
  false negatives, but should reduce or eliminate false positives.
Jason Madden's avatar
Jason Madden committed
237

238 239 240
- Based on measurements and discussion in :issue:`1233`, adjust the
  way :mod:`gevent.pywsgi` generates HTTP chunks. This is intended to
  reduce network overhead, especially for smaller chunk sizes.
Jason Madden's avatar
Jason Madden committed
241

Jason Madden's avatar
Jason Madden committed
242 243 244
- Additional slight performance improvements in :mod:`gevent.pywsgi`.
  See :pr:`1241`.

Jason Madden's avatar
Jason Madden committed
245
1.3.3 (2018-06-08)
Jason Madden's avatar
Jason Madden committed
246 247
==================

248 249 250 251
- :func:`gevent.sleep` updates the loop's notion of the current time
  before sleeping so that sleep duration corresponds more closely to
  elapsed (wall clock) time. :class:`gevent.Timeout` does the same.
  Reported by champax and FoP in :issue:`1227`.
Jason Madden's avatar
Jason Madden committed
252

253 254
- Fix an ``UnboundLocalError`` in SSL servers when wrapping a socket
  throws an error. Reported in :issue:`1236` by kochelmonster.
Jason Madden's avatar
Jason Madden committed
255

256
1.3.2.post0 (2018-05-30)
257
========================
Jason Madden's avatar
Jason Madden committed
258

259 260
- Fix a packaging error in manylinux binary wheels that prevented some
  imports from working. See :issue:`1219`.
Jason Madden's avatar
Jason Madden committed
261 262


Jason Madden's avatar
Jason Madden committed
263
1.3.2 (2018-05-29)
Jason Madden's avatar
Jason Madden committed
264 265
==================

266 267
- Allow weak refeneces to :class:`gevent.queue.Queue`. Reported in
  :issue:`1217` by githrdw.
Jason Madden's avatar
Jason Madden committed
268 269


Jason Madden's avatar
Jason Madden committed
270
1.3.1 (2018-05-18)
Jason Madden's avatar
Jason Madden committed
271 272
==================

273
- Allow weak references to :class:`gevent.event.Event`. Reported in
274
  :issue:`1211` by Matias Guijarro.
Jason Madden's avatar
Jason Madden committed
275

276 277
- Fix embedded uses of :func:`gevent.Greenlet.spawn`, especially under
  uwsgi. Reported in :issue:`1212` by Kunal Gangakhedkar.
Jason Madden's avatar
Jason Madden committed
278

279 280 281 282
- Fix :func:`gevent.os.nb_write` and :func:`gevent.os.nb_read` not
  always closing the IO event they opened in the event of an
  exception. This would be a problem especially for libuv.

Jason Madden's avatar
Jason Madden committed
283 284
1.3.0 (2018-05-11)
==================
285

286
- Python 3.7 passes the automated memory leak checks. See :issue:`1197`.
287

288 289
- Update autoconf's config.guess and config.sub to the latest versions
  for c-ares and libev.
290

291 292 293
- :class:`gevent.local.local` subclasses that mix-in ABCs can be instantiated.
  Reported in :issue:`1201` by Bob Jordan.

Jason Madden's avatar
Jason Madden committed
294
1.3b2 (2018-05-03)
Jason Madden's avatar
Jason Madden committed
295 296
==================

297 298
- On Windows, CFFI is now a dependency so that the libuv backend
  really can be used by default.
Jason Madden's avatar
Jason Madden committed
299

300 301 302 303 304 305 306 307 308 309 310 311
- Fix a bug detecting whether we can use the memory monitoring
  features when psutil is not installed.

- `gevent.subprocess.Popen` uses ``/proc/self/fd`` (on Linux) or
  ``/dev/fd`` (on BSD, including macOS) to find the file descriptors
  to close when ``close_fds`` is true. This matches an optimization
  added to Python 3 (and backports it to Python 2.7), making process
  spawning up to 9 times faster. Also, on Python 3, since Python 3.3
  is no longer supported, we can also optimize the case where
  ``close_fds`` is false (not the default), making process spawning up
  to 38 times faster. Initially reported in :issue:`1172` by Ofer Koren.

Jason Madden's avatar
Jason Madden committed
312
- The bundled libuv is now 1.20.1, up from 1.19.2. See :issue:`1177`.
Jason Madden's avatar
Jason Madden committed
313

314 315
- The long-deprecated and undocumented module ``gevent.wsgi`` was removed.

316 317 318
- Add `gevent.util.assert_switches` to build on the monitoring
  functions. Fixes :issue:`1182`.

319 320 321
- A started monitor thread for the active hub now survives a fork. See
  :issue:`1185`.

322 323 324 325
- The greenlet tracer functions used for the various monitoring
  capabilities are now compiled with Cython for substantially lower
  overhead. See :pr:`1190`.

326 327
- libuv now collects all pending watchers and runs their callbacks at
  the end of the loop iteration using UV_RUN_ONCE. This eliminates the
328 329 330 331 332
  need to patch libuv to be greenlet-safe. It also means that
  zero-duration timer watchers are actual timer watchers again
  (instead of being turned into check watchers); newly added
  zero-duration timers cannot block the event loop because they won't
  be run until a safe time.
333

334 335 336
- Python 3.7.0b4 is now the tested and supported version of Python
  3.7. PyPy 6.0 has been tested, although CI continues to use 5.10.

Jason Madden's avatar
Jason Madden committed
337
1.3b1 (2018-04-13)
Jason Madden's avatar
Jason Madden committed
338 339
==================

340 341 342
Dependencies
------------

343
- Cython 0.28.2 is now used to build gevent from a source checkout.
344

345 346
- The bundled libuv is now 1.19.2, up from 1.18.0.

Jason Madden's avatar
Jason Madden committed
347 348 349 350 351
Platform Support
----------------

- Travis CI tests on Python 3.7.0b3.

352 353
- Windows now defaults to the libuv backend if CFFI is installed. See
  :issue:`1163`.
354

355 356
Bug Fixes
---------
Jason Madden's avatar
Jason Madden committed
357

358 359 360 361
- On Python 2, when monkey-patching `threading.Event`, also
  monkey-patch the underlying class, ``threading._Event``. Some code
  may be type-checking for that. See :issue:`1136`.

362 363 364 365 366 367
- Fix libuv io watchers polling for events that only stopped watchers
  are interested in, reducing CPU usage. Reported in :issue:`1144` by
  wwqgtxx.

Enhancements
------------
368 369 370

- Add additional optimizations for spawning greenlets, making it
  faster than 1.3a2.
Jason Madden's avatar
Jason Madden committed
371

372 373
- Use strongly typed watcher callbacks in the libuv CFFI extensions.
  This prevents dozens of compiler warnings.
374 375 376 377 378 379 380 381 382 383 384

- When gevent prints a timestamp as part of an error message, it is
  now in UTC format as specified by RFC3339.

- Threadpool threads that exit now always destroy their hub (if one
  was created). This prevents some forms of resource leaks (notably
  visible as blocking functions reported by the new monitoring abilities).

- Hub objects now include the value of their ``name`` attribute in
  their repr.

Jason Madden's avatar
Jason Madden committed
385 386 387
- Pools for greenlets and threads have lower overhead, especially for
  ``map``. See :pr:`1153`.

388 389 390 391
- The undocumented, internal implementation classes ``IMap`` and
  ``IMapUnordered`` classes are now compiled with Cython, further
  reducing the overhead of ``[Thread]Pool.imap``.

392
- The classes `gevent.event.Event` and `gevent.event.AsyncResult`
393
  are compiled with Cython for improved performance, as is the
394 395
  ``gevent.queue`` module and ``gevent.hub.Waiter`` and certain
  time-sensitive parts of the hub itself. Please report any
396 397
  compatibility issues.

398 399 400 401
- ``python -m gevent.monkey <script>`` accepts more values for
  ``<script>``, including paths to packages or compiled bytecode.
  Reported in :issue:`1157` by Eddie Linder.

402 403 404 405 406 407 408 409
- Add a simple event framework for decoupled communication. It uses
  :mod:`zope.event` if that is installed.

- :mod:`gevent.monkey` has support for plugins in the form of event
  subscribers and setuptools entry points. See :pr:`1158` and
  :issue:`1162`. setuptools must be installed at runtime for its entry
  points to function.

410 411 412 413 414 415 416 417 418 419 420 421 422 423 424
Monitoring and Debugging
------------------------

- Introduce the configuration variable
  `gevent.config.track_greenlet_tree` (aka
  ``GEVENT_TRACK_GREENLET_TREE``) to allow disabling the greenlet tree
  features for applications where greenlet spawning is performance
  critical. This restores spawning performance to 1.2 levels.

- Add an optional monitoring thread for each hub. When enabled, this
  thread (by default) looks for greenlets that block the event loop
  for more than 0.1s. You can add your own periodic monitoring
  functions to this thread. Set ``GEVENT_MONITOR_THREAD_ENABLE`` to
  use it, and ``GEVENT_MAX_BLOCKING_TIME`` to configure the blocking
  interval.
425

426 427
- The monitoring thread emits events when it detects certain
  conditions, like loop blocked or memory limits exceeded.
428

429 430 431
- Add settings for monitoring memory usage and emitting events when a
  threshold is exceeded and then corrected. gevent currently supplies
  no policy for what to do when memory exceeds the configured limit.
432 433
  ``psutil`` must be installed to use this. See :pr:`1150`.

434

Jason Madden's avatar
Jason Madden committed
435
1.3a2 (2018-03-06)
Jason Madden's avatar
Jason Madden committed
436 437
==================

438 439
Dependencies
------------
440 441

- Cython 0.28b1 or later is now required to build gevent from a source
442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461
  checkout (Cython is *not* required to build a source distribution
  from PyPI).

- Update c-ares to 1.14.0. See :issue:`1105`.

- gevent now **requires** the patched version of libuv it is
  distributed with. Building gevent with a non-embedded libuv, while
  not previously supported, is not possible now. See
  :issue:`1126`.


Platform Support
----------------

- Travis CI tests on Python 3.7.0b2 and PyPy 2.7 5.10.0 and PyPy 3.5
  5.10.1.

Build Changes
-------------

462 463 464
- Fix building from a source distribution (PyPI) without Cython
  installed.

465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515
Enhancements
------------

- Add the ``dnspython`` resolver as a lightweight alternative to
  c-ares. It is generally faster than c-ares and is supported on PyPy.
  c-ares may be deprecated in the future. See :pr:`1088` and
  :issue:`1103`.

- Add the module :mod:`gevent.time` that can be imported instead of
  :mod:`time`, much like :mod:`gevent.socket` can be imported instead
  of :mod:`socket`. It contains ``gevent.sleep``. This aids
  monkey-patching.

- Simple subclasses of `gevent.local.local` now have the same
  (substantially improved) performance characteristics of plain
  `gevent.local.local` itself, making them 2 to 3 times faster than
  before. See :pr:`1117`. If there are any compatibility
  problems, please open issues.

Monitoring and Debugging
~~~~~~~~~~~~~~~~~~~~~~~~

- Greenlet objects now keep track of their spawning parent greenlet
  and the code location that spawned them, in addition to maintaining
  a "spawn tree local" mapping. This adds some runtime overhead in
  relative terms, but absolute numbers are still relatively small.
  Based on a proposal from PayPal and comments by Mahmoud Hashemi and
  Kurt Rose. See :issue:`755` and :pr:`1115`. As always, feedback is
  appreciated.

- Greenlet objects now have a `minimal_ident
  <gevent.Greenlet.minimal_ident>` property. It functions
  similarly to ``Thread.ident`` or ``id`` by uniquely identifying the
  greenlet object while it remains alive, and it can be reused after
  the greenlet object is dead. It is different in that it is small and
  sequential. Based on a proposal from PayPal and comments by Mahmoud
  Hashemi and Kurt Rose. See :issue:`755`. As always, feedback is
  appreciated.

- `gevent.Greenlet` objects now have a `gevent.Greenlet.name`
  attribute that is included in the default repr.

- Include the values of `gevent.local.local` objects associated with
  each greenlet in `gevent.util.format_run_info`.

- Add `gevent.util.GreenletTree` to visualize the greenlet tree. This
  is used by `gevent.util.format_run_info`.


Subprocess
~~~~~~~~~~
516 517 518 519 520 521 522 523 524 525 526 527 528 529

- Make :class:`gevnt.subprocess.Popen` accept the ``restore_signals``
  keyword argument on all versions of Python, and on Python 2 have it
  default to false. It previously defaulted to true on all versions;
  now it only defaults to true on Python 3. The standard library in
  Python 2 does not have this argument and its behaviour with regards
  to signals is undocumented, but there is code known to rely on
  signals not being restored under Python 2. Initial report and patch
  in :pr:`1063` by Florian Margaine.

- Allow :class:`gevent.subprocess.Popen` to accept the keyword
  arguments ``pass_fds`` and ``start_new_session`` under Python 2.
  They have always had the same default as Python 3, namely an empty
  tuple and false, but now are accessible to Python 2.
Jason Madden's avatar
Jason Madden committed
530

531 532 533
- Support the ``capture_output`` argument added to Python 3.7 in
  :func:`gevent.subprocess.run`.

534 535
Configuration
~~~~~~~~~~~~~
536

537 538 539 540 541 542 543 544 545 546 547 548
- Centralize all gevent configuration in an object at
  ``gevent.config``, allowing for gevent to be configured through code
  and not *necessarily* environment variables, and also provide a
  centralized place for documentation. See :issue:`1090`.

  - The new ``GEVENT_CORE_CFFI_ONLY`` environment variable has been
    replaced with the pre-existing ``GEVENT_LOOP`` environment
    variable. That variable may take the values ``libev-cext``,
    ``libev-cffi``, or ``libuv-cffi``, (or be a list in preference
    order, or be a dotted name; it may also be assigned to an
    object in Python code at ``gevent.config.loop``).

549 550 551 552 553 554 555 556 557 558 559
  - The ``GEVENTARES_SERVERS`` environment variable is deprecated in
    favor of ``GEVENT_RESOLVER_SERVERS``. See :issue:`1103`.


Bug Fixes
---------

- Fix calling ``shutdown`` on a closed socket. It was raising
  ``AttributeError``, now it once again raises the correct
  ``socket.error``. Reported in :issue:`1089` by André Cimander.

560 561 562 563 564 565
- Fix an interpreter crash that could happen if two or more ``loop``
  objects referenced the default event loop and one of them was
  destroyed and then the other one destroyed or (in the libev C
  extension implementation only) deallocated (garbage collected). See
  :issue:`1098`.

566 567 568 569 570 571 572 573 574 575 576
- Fix a race condition in libuv child callbacks. See :issue:`1104`.

Other Changes
-------------

- The internal, undocumented module ``gevent._threading`` has been
  simplified.

- The internal, undocumented class ``gevent._socket3._fileobject`` has
  been removed. See :issue:`1084`.

577 578 579 580 581
- Simplify handling of the libev default loop and the ``destroy()``
  method. The default loop, when destroyed, can again be requested and
  it will regenerate itself. The default loop is the only one that can
  receive child events.

582 583 584
- Make :meth:`gevent.socket.socket.sendall` up to ten times faster on
  PyPy3, through the same change that was applied in gevent 1.1b3 for PyPy2.

585 586 587
- Be more careful about issuing a warning about patching SSL on
  Python 2. See :issue:`1108`.

588 589 590
- Signal handling under PyPy with libuv is more reliable. See
  :issue:`1112`.

591
- The :mod:`gevent.greenlet` module is now compiled with Cython to
592
  offset any performance decrease due to :issue:`755`. Please open
Jason Madden's avatar
Jason Madden committed
593
  issues for any compatibility concerns. See :pr:`1115` and :pr:`1120`.
594

595
- On CPython, allow the pure-Python implementations of
Jason Madden's avatar
Jason Madden committed
596
  `gevent.Greenlet`, `gevent.local` and `gevent.lock` to be
597 598 599
  used when the environment variable ``PURE_PYTHON`` is set. This is
  not recommended except for debugging and testing. See :issue:`1118`.

600 601 602 603 604 605
- :meth:`gevent.select.poll.poll` now interprets a *timeout* of -1 the
  same as a *timeout* of *None* as the standard requires. Previously,
  on libuv this was interpreted the same as a *timeout* of 0. In
  addition, all *timeout* values less than zero are interpreted like
  *None* (as they always were under libev). See :issue:`1127`.

606 607
- Monkey-patching now defaults to patching ``threading.Event``.

Jason Madden's avatar
Jason Madden committed
608
1.3a1 (2018-01-27)
Jason Madden's avatar
Jason Madden committed
609 610
==================

Jason Madden's avatar
Jason Madden committed
611 612 613 614 615 616 617 618 619 620 621 622
Dependencies
------------

- gevent is now built and tested with Cython 0.27. This is required
  for Python 3.7 support.

- Update c-ares to 1.13.0. See :issue:`990`.


Platform Support
----------------

Jason Madden's avatar
Jason Madden committed
623 624 625 626 627 628 629 630 631
- Add initial support for Python 3.7a3. It has the same level of
  support as Python 3.6.

  - Using unreleased Cython 0.28 and greenlet 0.4.13; requires Python 3.7a3.

  - The ``async`` functions and classes have been renamed to
    ``async_`` due to ``async`` becoming a keyword in Python 3.7.
    Aliases are still in place for older versions. See :issue:`1047`.

Jason Madden's avatar
Jason Madden committed
632 633 634 635 636 637
- gevent is now tested on Python 3.6.4. This includes the following
  fixes and changes:

  - Errors raised from :mod:`gevent.subprocess` will have a
    ``filename`` attribute set.
  - The :class:`threading.Timer` class is now monkey-patched and can
638 639
    be joined. Previously on Python 3.4 and above, joining a ``Timer``
    would hang the process.
Jason Madden's avatar
Jason Madden committed
640 641 642 643 644 645 646 647 648
  - :meth:`gevent.ssl.SSLSocket.unwrap` behaves more like the standard
    library, including returning a SSLSocket and allowing certain
    timeout-related SSL errors to propagate. The added standard
    library tests ``test_ftplib.py`` now passes.
  - :class:`gevent.subprocess.Popen` accepts a "path-like object" for
    the *cwd* parameter on all platforms. Previously this only worked
    on POSIX platforms under Python 3.6. Now it also works on Windows under
    Python 3.6 (as expected) and is backported to all previous versions.

Jason Madden's avatar
Jason Madden committed
649 650
- Linux CI now tests on PyPy3 3.5-5.9.0, updated from PyPy3 3.5-5.7.1.
  See :issue:`1001`. PyPy2 has been updated to 5.9.0 from 5.7.1,
Jason Madden's avatar
Jason Madden committed
651 652
  Python 2.7 has been updated to 2.7.14 from 2.7.13, Python 3.4 is
  updated to 3.4.7 from 3.4.5, Python 3.5 is now 3.5.4 from 3.5.3, and
Jason Madden's avatar
Jason Madden committed
653
  Python 3.6 is now 3.6.4 from 3.6.0.
Jason Madden's avatar
Jason Madden committed
654 655 656 657 658 659

- Drop support for Python 3.3. The documentation has only claimed
  support for 3.4+ since gevent 1.2 was released, and only 3.4+ has
  been tested. This merely removes the supporting Trove classifier and
  remaining test code. See :issue:`997`.

660 661 662
- PyPy is now known to run on Windows using the libuv backend, with
  caveats. See the section on libuv for more information.

Jason Madden's avatar
Jason Madden committed
663 664 665 666
- Due to security concerns, official support for Python 2.7.8 and
  earlier (without a modern SSL implementation) has been dropped.
  These versions are no longer tested with gevent, but gevent can
  still be installed on them. Supporting code will be removed in the
Jason Madden's avatar
Jason Madden committed
667
  next major version of gevent. See :issue:`1073`.
Jason Madden's avatar
Jason Madden committed
668

Jason Madden's avatar
Jason Madden committed
669 670 671 672 673 674 675
Build Changes
-------------

- When building gevent from a source checkout (*not* a distributed
  source distribution), ``make`` is no longer required and the
  ``Makefile`` is not used. Neither is an external ``cython`` command.
  Instead, the ``cythonize`` function is used, as recommended by
Jason Madden's avatar
Jason Madden committed
676 677
  Cython. (The external commands were never required by source
  distributions.) See :issue:`1076`.
Jason Madden's avatar
Jason Madden committed
678

Jason Madden's avatar
Jason Madden committed
679
- :class:`gevent.local.local` is compiled with Cython on CPython.
Jason Madden's avatar
Jason Madden committed
680

681 682 683 684
- The Cython ares 'channel' class is no longer declared to be publicly
  accessible from a named C structure. Doing so caused a conflict with
  the c-ares header files.

Jason Madden's avatar
Jason Madden committed
685 686
Bug Fixes
---------
Ron Rothman's avatar
Ron Rothman committed
687

688 689 690 691 692
- If a single greenlet created and destroyed many
  :class:`gevent.local.local` objects without ever exiting, there
  would be a leak of the function objects intended to clean up the
  locals after the greenlet exited. Introduce a weak reference to
  avoid that. Reported in :issue:`981` by Heungsub Lee.
Jason Madden's avatar
Jason Madden committed
693

694
- pywsgi also catches and ignores by default
695 696 697
  :const:`errno.WSAECONNABORTED` on Windows. Initial patch in
  :pr:`999` by Jan van Valburg.

698 699 700 701 702
- :meth:`gevent.subprocess.Popen.communicate` returns the correct type
  of str (not bytes) in universal newline mode under Python 3, or when
  an encoding has been specified. Initial patch in :pr:`939` by
  William Grzybowski.

703 704 705 706 707 708 709 710 711 712
- :meth:`gevent.subprocess.Popen.communicate` (and in general,
  accessing ``Popen.stdout`` and ``Popen.stderr``) returns the correct
  type of str (bytes) in universal newline mode under Python 2.
  Previously it always returned unicode strings. Reported in
  :issue:`1039` by Michal Petrucha.

- :class:`gevent.fileobject.FileObjectPosix` returns native strings in
  universal newline mode on Python 2. This is consistent with what
  :class:`.FileObjectThread` does. See :issue:`1039`.

Jason Madden's avatar
Jason Madden committed
713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748
- ``socket.send()`` now catches ``EPROTOTYPE`` on macOS to handle a race
  condition during shutdown. Fixed in :pr:`1035` by Jay Oster.

- :func:`gevent.socket.create_connection` now properly cleans up open
  sockets if connecting or binding raises a :exc:`BaseException` like
  :exc:`KeyboardInterrupt`, :exc:`greenlet.GreenletExit` or
  :exc:`gevent.timeout.Timeout`. Reported in :issue:`1044` by
  kochelmonster.


Other Changes
-------------

- ``Pool.add`` now accepts ``blocking`` and ``timeout`` parameters,
  which function similarly to their counterparts in ``Semaphore``.
  See :pr:`1032` by Ron Rothman.

- Defer adjusting the stdlib's list of active threads until
  ``threading`` is monkey patched. Previously this was done when
  :mod:`gevent.threading` was imported. That module is documented to
  be used as a helper for monkey patching, so this should generally
  function the same, but some applications ignore the documentation
  and directly import that module anyway.

  A positive consequence is that ``import gevent.threading, threading;
  threading.current_thread()`` will no longer return a DummyThread
  before monkey-patching. Another positive consequence is that PyPy
  will no longer print a ``KeyError`` on exit if
  :mod:`gevent.threading` was imported *without* monkey-patching.

  See :issue:`984`.

- Specify the Requires-Python metadata for improved installation
  support in certain tools (setuptools v24.2.1 or newer is required).
  See :issue:`995`.

749 750 751
- Monkey-patching after the :mod:`ssl` module has been imported now
  prints a warning because this can produce ``RecursionError``.

752 753
- :class:`gevent.local.local` objects are now approximately 3.5 times faster
  getting, setting and deleting attributes on PyPy. This involved
754 755
  implementing more of the attribute protocols directly. Please open
  an issue if you have any compatibility problems. See :issue:`1020`.
756

Jason Madden's avatar
Jason Madden committed
757 758 759 760 761 762 763 764
- :class:`gevent.local.local` is compiled with Cython on CPython. It
  was already 5 to 6 times faster due to the work on :issue:`1020`,
  and compiling it with Cython makes it another 5 to 6 times faster,
  for a total speed up of about 35 times. It is now in the same
  ballpark as the native :class:`threading.local` class. It also uses
  one pointer less memory per object, and one pointer less memory per
  greenlet. See :pr:`1024`.

765 766
- More safely terminate subprocesses on Windows with
  :meth:`gevent.subprocess.Popen.terminate`. Reported in :issue:`1023`
767 768
  by Giacomo Debidda.

769 770 771
- gevent now uses cffi's "extern 'Python'" callbacks. These should be
  faster and more stable. This requires at least cffi 1.4.0. See :issue:`1049`.

772 773 774 775 776 777 778 779 780 781
- gevent now approximately tries to stick to a scheduling interval
  when running callbacks, instead of simply running a count of
  callbacks. The interval is determined by
  :func:`gevent.getswitchinterval`. On Python 3, this is the same as
  the thread switch interval. On Python 2, this defaults to 0.005s and
  can be changed with :func:`gevent.setswitchinterval`. This should
  result in more fair "scheduling" of greenlets, especially when
  ``gevent.sleep(0)`` or other busy callbacks are in use. The interval
  is checked every 50 callbacks to keep overhead low. See
  :issue:`1072`. With thanks to Arcadiy Ivanov and Antonio Cuni.
782 783 784 785

libuv
-----

Jason Madden's avatar
Jason Madden committed
786 787
- Add initial *experimental* support for using libuv as a backend
  instead of libev, controlled by setting the environment variable
788
  ``GEVENT_CORE_CFFI_ONLY=libuv`` before importing gevent. This
789
  suffers a number of limitations compared to libev, notably:
Jason Madden's avatar
Jason Madden committed
790

Jason Madden's avatar
Jason Madden committed
791 792 793 794 795
  - libuv support is not available in the manylinux wheels uploaded to
    PyPI. The manylinux specification requires glibc 2.5, while libuv
    requires glibc 2.12. Install from source to access libuv on Linux
    (e.g., pip's ``--no-binary`` option).

Jason Madden's avatar
Jason Madden committed
796
  - Timers (such as ``gevent.sleep`` and ``gevent.Timeout``) only
797 798
    support a resolution of 1ms (in practice, it's closer to 1.5ms).
    Attempting to use something smaller will automatically increase it
799 800 801 802
    to 1ms and issue a warning. Because libuv only supports
    millisecond resolution by rounding a higher-precision clock to an
    integer number of milliseconds, timers apparently suffer from more
    jitter.
Jason Madden's avatar
Jason Madden committed
803 804 805 806

  - Using negative timeouts may behave differently from libev.

  - libuv blocks delivery of all signals, so signals are handled using
807
    an (arbitrary) 0.3 second timer. This means that signal handling
Jason Madden's avatar
Jason Madden committed
808 809
    will be delayed by up to that amount, and that the longest the
    event loop can sleep in the operating system's ``poll`` call is
810 811
    that amount. Note that this is what gevent does for libev on
    Windows too.
Jason Madden's avatar
Jason Madden committed
812 813 814 815 816 817 818 819 820 821 822 823

  - libuv only supports one io watcher per file descriptor, whereas
    libev and gevent have always supported many watchers using
    different settings. The libev behaviour is emulated at the Python
    level, but that adds overhead.

  - Looping multiple times and expecting events for the same file
    descriptor to be raised each time without any data being read or
    written (as works with libev) does not appear to work correctly on
    Linux when using ``gevent.select.poll`` or a monkey-patched
    ``selectors.PollSelector``.

824 825 826
  - The build system does not support using a system libuv; the
    embedded copy must be used. Using setuptools to compile libuv was
    the most portable method found.
Jason Madden's avatar
Jason Madden committed
827 828 829 830 831 832

  - If anything unexpected happens, libuv likes to ``abort()`` the
    entire process instead of reporting an error. For example, closing
    a file descriptor it is using in a watcher may cause the entire
    process to be exited.

833 834 835 836
  - There may be occasional otherwise unexplained and hard to
    duplicate crashes. If you can duplicate a crash, **please** submit
    an issue.

837 838 839 840 841 842 843 844
  - This is the only backend that PyPy can use on Windows. As of this
    alpha, there are many known issues with non-blocking sockets
    (e.g., as used by :mod:`asyncore`; see ``test_ftplib.py``) and
    sometimes sockets not getting closed in a timely fashion
    (apparently; see ``test_httpservers.py``) and communicating with
    subprocesses (it always hangs). Help tracking those down would be
    appreciated. Only PyPy2 is tested.

Jason Madden's avatar
Jason Madden committed
845
  Feedback and pull requests are welcome, especially to address the
846
  issues mentioned above.
Jason Madden's avatar
Jason Madden committed
847

848 849 850 851
  Other differences include:

  - The order in which timers and other callbacks are invoked may be
    different than in libev. In particular, timers and IO callbacks
852 853
    happen in a different order, and timers may easily be off by up to
    half of the supposed 1ms resolution. See :issue:`1057`.
854

855 856 857 858
  - Starting a ``timer`` watcher does not update the loop's time by
    default. This is because, unlike libev, a timer callback can cause
    other timer callbacks to be run if they expire because the loop's
    time updated, without cycling the event loop. See :issue:`1057`.
859

860
    libev has also been changed to follow this behaviour.
861

862 863
    Also see :issue:`1072`.

864 865 866 867 868 869
  - Timers of zero duration do not necessarily cause the event loop to
    cycle, as they do in libev. Instead, they may be called
    immediately. If zero duration timers are added from other zero
    duration timer callbacks, this can lead the loop to appear to
    hang, as no IO will actually be done.

870 871 872
    To mitigate this issue, ``loop.timer()`` detects attempts to use
    zero duration timers and turns them into a check watcher. check
    watchers do not support the ``again`` method.
873

874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889
  - All watchers (e.g., ``loop.io``) and the ``Timeout`` class have a
    ``close`` method that should be called when code is done using the
    object (they also function as context managers and a ``with``
    statement will automatically close them). gevent does this
    internally for sockets, file objects and internal timeouts.
    Neglecting to close an object may result in leaking native
    resources. To debug this, set the environment variables
    ``GEVENT_DEBUG=debug`` and ``PYTHONTRACEMALLOC=n`` before starting
    the process.

    The traditional cython-based libev backend will not leak if
    ``close`` is not called and will not produce warnings. The
    CFFI-based libev backend will not currently leak but will produce
    warnings. The CFFI-based libuv backend may leak and will produce
    warnings.

Jason Madden's avatar
Jason Madden committed
890 891 892 893 894
  Again, this is extremely experimental and all of it is subject to
  change.

  See :issue:`790` for history and more in-depth discussion.

895 896 897 898 899
libev
-----

- The C extension has been updated to use more modern Cython idioms
  and generate less code for simplicity, faster compilation and better
900
  cache usage. See :pr:`1077`.
901 902 903 904 905 906 907

  - Watcher objects may be slightly larger. On a 64-bit platform, a
    typical watcher may be 16 bytes (2 pointers) larger. This is
    offset by slight performance gains.

  - Cython is no longer preprocessed. Certain attributes that were
    previously only defined in certain compilation modes (notably
Jason Madden's avatar
Jason Madden committed
908 909 910 911
    LIBEV_EMBED) are now always defined, but will raise
    ``AttributeError`` or have a negative value when not available. In
    general these attributes are not portable or documented and are
    not implemented by libuv or the CFFI backend. See :issue:`1076`.
912

Jason Madden's avatar
Jason Madden committed
913
  - Certain private helper functions (``gevent_handle_error``, and part of
914 915
    ``gevent_call``) are now implemented in Cython instead of C. This
    reduces our reliance on internal undocumented implementation
Jason Madden's avatar
Jason Madden committed
916
    details of Cython and Python that could change. See :pr:`1080`.