Skip to content

C
cpython
Commit 7c7605ff authored by Yury Selivanov's avatar Yury Selivanov Committed by GitHub
Browse files
Options

bpo-33649: First asyncio docs improvement pass (GH-9142) 

Rewritten/updated sections:

* Event Loop APIs
* Transports & Protocols
* Streams
* Exceptions
* Policies
* Queues
* Subprocesses
* Platforms
parent 735171e3
Show whitespace changes
Inline Side-by-side
Showing with 2303 additions and 1737 deletions
Doc/library/asyncio-dev.rst
View file @ 7c7605ff
...@@ -23,7 +23,7 @@ To enable all debug checks for an application: ...@@ -23,7 +23,7 @@ To enable all debug checks for an application:
* Enable the asyncio debug mode globally by setting the environment variable * Enable the asyncio debug mode globally by setting the environment variable
:envvar:`PYTHONASYNCIODEBUG` to ``1``, using ``-X dev`` command line option :envvar:`PYTHONASYNCIODEBUG` to ``1``, using ``-X dev`` command line option
(see the :option:`-X` option), or by calling (see the :option:`-X` option), or by calling
:meth:`AbstractEventLoop.set_debug`. :meth:`loop.set_debug`.
* Set the log level of the :ref:`asyncio logger <asyncio-logger>` to * Set the log level of the :ref:`asyncio logger <asyncio-logger>` to
:py:data:`logging.DEBUG`. For example, call :py:data:`logging.DEBUG`. For example, call
``logging.basicConfig(level=logging.DEBUG)`` at startup. ``logging.basicConfig(level=logging.DEBUG)`` at startup.
...@@ -35,11 +35,11 @@ Examples debug checks: ...@@ -35,11 +35,11 @@ Examples debug checks:
* Log :ref:`coroutines defined but never "yielded from" * Log :ref:`coroutines defined but never "yielded from"
<asyncio-coroutine-not-scheduled>` <asyncio-coroutine-not-scheduled>`
* :meth:`~AbstractEventLoop.call_soon` and :meth:`~AbstractEventLoop.call_at` methods * :meth:`loop.call_soon` and :meth:`loop.call_at` methods
raise an exception if they are called from the wrong thread. raise an exception if they are called from the wrong thread.
* Log the execution time of the selector * Log the execution time of the selector
* Log callbacks taking more than 100 ms to be executed. The * Log callbacks taking more than 100 ms to be executed. The
:attr:`AbstractEventLoop.slow_callback_duration` attribute is the minimum :attr:`loop.slow_callback_duration` attribute is the minimum
duration in seconds of "slow" callbacks. duration in seconds of "slow" callbacks.
* :exc:`ResourceWarning` warnings are emitted when transports and event loops * :exc:`ResourceWarning` warnings are emitted when transports and event loops
are :ref:`not closed explicitly <asyncio-close-transports>`. are :ref:`not closed explicitly <asyncio-close-transports>`.
...@@ -51,7 +51,7 @@ Examples debug checks: ...@@ -51,7 +51,7 @@ Examples debug checks:
.. seealso:: .. seealso::
The :meth:`AbstractEventLoop.set_debug` method and the :ref:`asyncio logger The :meth:`loop.set_debug` method and the :ref:`asyncio logger
<asyncio-logger>`. <asyncio-logger>`.
...@@ -75,7 +75,7 @@ For example, write:: ...@@ -75,7 +75,7 @@ For example, write::
Don't schedule directly a call to the :meth:`~Future.set_result` or the Don't schedule directly a call to the :meth:`~Future.set_result` or the
:meth:`~Future.set_exception` method of a future with :meth:`~Future.set_exception` method of a future with
:meth:`AbstractEventLoop.call_soon`: the future can be cancelled before its method :meth:`loop.call_soon`: the future can be cancelled before its method
is called. is called.
If you wait for a future, you should check early if the future was cancelled to If you wait for a future, you should check early if the future was cancelled to
...@@ -96,13 +96,14 @@ The :func:`shield` function can also be used to ignore cancellation. ...@@ -96,13 +96,14 @@ The :func:`shield` function can also be used to ignore cancellation.
Concurrency and multithreading Concurrency and multithreading
------------------------------ ------------------------------
An event loop runs in a thread and executes all callbacks and tasks in the same An event loop runs in a thread (typically the main thread) and executes
thread. While a task is running in the event loop, no other task is running in all callbacks and tasks in its thread. While a task is running in the
the same thread. But when the task uses ``await``, the task is suspended event loop, no other task is running in the same thread. When a task
and the event loop executes the next task. executes an ``await`` expression, the task gets suspended and the
event loop executes the next task.
To schedule a callback from a different thread, the To schedule a callback from a different thread, the
:meth:`AbstractEventLoop.call_soon_threadsafe` method should be used. Example:: :meth:`loop.call_soon_threadsafe` method should be used. Example::
loop.call_soon_threadsafe(callback, *args) loop.call_soon_threadsafe(callback, *args)
...@@ -122,7 +123,7 @@ To schedule a coroutine object from a different thread, the ...@@ -122,7 +123,7 @@ To schedule a coroutine object from a different thread, the
future = asyncio.run_coroutine_threadsafe(coro_func(), loop) future = asyncio.run_coroutine_threadsafe(coro_func(), loop)
result = future.result(timeout) # Wait for the result with a timeout result = future.result(timeout) # Wait for the result with a timeout
The :meth:`AbstractEventLoop.run_in_executor` method can be used with a thread pool The :meth:`loop.run_in_executor` method can be used with a thread pool
executor to execute a callback in different thread to not block the thread of executor to execute a callback in different thread to not block the thread of
the event loop. the event loop.
...@@ -151,7 +152,7 @@ APIs like :ref:`protocols <asyncio-protocol>`. ...@@ -151,7 +152,7 @@ APIs like :ref:`protocols <asyncio-protocol>`.
An executor can be used to run a task in a different thread or even in a An executor can be used to run a task in a different thread or even in a
different process, to not block the thread of the event loop. See the different process, to not block the thread of the event loop. See the
:meth:`AbstractEventLoop.run_in_executor` method. :meth:`loop.run_in_executor` method.
.. seealso:: .. seealso::
...@@ -182,7 +183,7 @@ Detect coroutine objects never scheduled ...@@ -182,7 +183,7 @@ Detect coroutine objects never scheduled
---------------------------------------- ----------------------------------------
When a coroutine function is called and its result is not passed to When a coroutine function is called and its result is not passed to
:func:`ensure_future` or to the :meth:`AbstractEventLoop.create_task` method, :func:`ensure_future` or to the :meth:`loop.create_task` method,
the execution of the coroutine object will never be scheduled which is the execution of the coroutine object will never be scheduled which is
probably a bug. :ref:`Enable the debug mode of asyncio <asyncio-debug-mode>` probably a bug. :ref:`Enable the debug mode of asyncio <asyncio-debug-mode>`
to :ref:`log a warning <asyncio-logger>` to detect it. to :ref:`log a warning <asyncio-logger>` to detect it.
...@@ -204,7 +205,7 @@ Output in debug mode:: ...@@ -204,7 +205,7 @@ Output in debug mode::
test() test()
The fix is to call the :func:`ensure_future` function or the The fix is to call the :func:`ensure_future` function or the
:meth:`AbstractEventLoop.create_task` method with the coroutine object. :meth:`loop.create_task` method with the coroutine object.
.. seealso:: .. seealso::
...@@ -279,7 +280,7 @@ coroutine in another coroutine and use classic try/except:: ...@@ -279,7 +280,7 @@ coroutine in another coroutine and use classic try/except::
loop.run_forever() loop.run_forever()
loop.close() loop.close()
Another option is to use the :meth:`AbstractEventLoop.run_until_complete` Another option is to use the :meth:`loop.run_until_complete`
function:: function::
task = asyncio.ensure_future(bug()) task = asyncio.ensure_future(bug())
......
Doc/library/asyncio-eventloop.rst
View file @ 7c7605ff
.. currentmodule:: asyncio .. currentmodule:: asyncio
.. _asyncio-event-loop:
Base Event Loop ==========
=============== Event Loop
==========
**Source code:** :source:`Lib/asyncio/events.py`
The event loop is the central execution device provided by :mod:`asyncio`. .. rubric:: Preface
It provides multiple facilities, including:
* Registering, executing and cancelling delayed calls (timeouts). An event loop is the central component of every asyncio application.
Event loops run asynchronous tasks and callbacks, perform network
IO operations, run subprocesses, etc.
* Creating client and server :ref:`transports <asyncio-transport>` for various In general, it is *not recommended* to use event loops directly at
kinds of communication. the application-level asyncio code. They should only be accessed
in low-level code in libraries and frameworks.
* Launching subprocesses and the associated :ref:`transports High-level asyncio applications should not need to work with event
<asyncio-transport>` for communication with an external program. loops and should use the :func:`asyncio.run` function to initialize
and run asynchronous code.
* Delegating costly function calls to a pool of threads.
.. class:: BaseEventLoop .. rubric:: Accessing Event Loop
This class is an implementation detail. It is a subclass of The following low-level functions can be used to get, set, or create
:class:`AbstractEventLoop` and may be a base class of concrete an event loop:
event loop implementations found in :mod:`asyncio`. It should not
be used directly; use :class:`AbstractEventLoop` instead.
``BaseEventLoop`` should not be subclassed by third-party code; the
internal interface is not stable.
.. class:: AbstractEventLoop .. function:: get_running_loop()
Abstract base class of event loops. Return the running event loop in the current OS thread.
This class is :ref:`not thread safe <asyncio-multithreading>`. If there is no running event loop a :exc:`RuntimeError` is raised.
This function can only be called from a coroutine or a callback.
Run an event loop .. versionadded:: 3.7
-----------------
.. method:: AbstractEventLoop.run_forever() .. function:: get_event_loop()
Run until :meth:`stop` is called. If :meth:`stop` is called before Get the current event loop. If there is no current event loop set
:meth:`run_forever()` is called, this polls the I/O selector once in the current OS thread and :func:`set_event_loop` has not yet
with a timeout of zero, runs all callbacks scheduled in response to been called, asyncio will create a new event loop and set it as the
I/O events (and those that were already scheduled), and then exits. current one.
If :meth:`stop` is called while :meth:`run_forever` is running,
this will run the current batch of callbacks and then exit. Note
that callbacks scheduled by callbacks will not run in that case;
they will run the next time :meth:`run_forever` is called.
.. versionchanged:: 3.5.1 Because this function has a rather complex behavior (especially
when custom event loop policies are in use), it is recommended
to use the :func:`get_running_loop` function in coroutines and
callbacks instead.
.. method:: AbstractEventLoop.run_until_complete(future) Consider also using the :func:`asyncio.run` function instead of
manually creating and closing an event loop.
Run until the :class:`Future` is done. .. function:: set_event_loop(loop)
If the argument is a :ref:`coroutine object <coroutine>`, it is wrapped by Set *loop* as a current event loop for the current OS thread.
:func:`ensure_future`.
Return the Future's result, or raise its exception. .. function:: new_event_loop()
.. method:: AbstractEventLoop.is_running() Create a new event loop object.
Returns running status of event loop. Note that the behaviour of :func:`get_event_loop`, :func:`set_event_loop`,
and :func:`new_event_loop` functions can be altered by
:ref:`setting a custom event loop policy <asyncio-policies>`.
.. method:: AbstractEventLoop.stop()
Stop running the event loop. .. rubric:: Contents
This causes :meth:`run_forever` to exit at the next suitable This documentation page contains the following sections:
opportunity (see there for more details).
.. versionchanged:: 3.5.1 * The `Event Loop Methods`_ section is a reference documentation of
event loop APIs;
* The `Callback Handles`_ section documents the :class:`Handle` and
:class:`TimerHandle`, instances of which are returned from functions
:meth:`loop.call_soon`, :meth:`loop.call_later`, etc;
* The `Server Objects`_ sections documents types returned from
event loop methods like :meth:`loop.create_server`;
* The `Event Loops Implementations`_ section documents the
:class:`SelectorEventLoop` and :class:`ProactorEventLoop` classes;
* The `Examples`_ section showcases how to work with some event
loop APIs.
.. _asyncio-event-loop:
Event Loop Methods
==================
Event loops provide the following **low-level** APIs:
.. contents::
:depth: 1
:local:
Running and stopping the loop
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. method:: loop.run_until_complete(future)
Run until the *future* (an instance of :class:`Future`) is
completed.
If the argument is a :ref:`coroutine object <coroutine>` it
is implicitly wrapped into an :class:`asyncio.Task`.
Return the Future's result or raise its exception.
.. method:: loop.run_forever()
Run the event loop until :meth:`stop` is called.
If :meth:`stop` is called before :meth:`run_forever()` is called,
the loop will poll the I/O selector once with a timeout of zero,
run all callbacks scheduled in response to I/O events (and
those that were already scheduled), and then exit.
If :meth:`stop` is called while :meth:`run_forever` is running,
the loop will run the current batch of callbacks and then exit.
Note that callbacks scheduled by callbacks will not run in that
case; they will run the next time :meth:`run_forever` or
:meth:`run_until_complete` is called.
.. method:: loop.stop()
Stop the event loop.
.. method:: loop.is_running()
.. method:: AbstractEventLoop.is_closed() Return ``True`` if the event loop is currently running.
Returns ``True`` if the event loop was closed. .. method:: loop.is_closed()
.. versionadded:: 3.4.2 Return ``True`` if the event loop was closed.
.. method:: AbstractEventLoop.close() .. method:: loop.close()
Close the event loop. The loop must not be running. Pending Close the event loop.
callbacks will be lost.
This clears the queues and shuts down the executor, but does not wait for The loop cannot not be running when this function is called.
the executor to finish. Any pending callbacks will be discarded.
This is idempotent and irreversible. No other methods should be called after This method clears all queues and shuts down the executor, but does
this one. not wait for the executor to finish.
This method is idempotent and irreversible. No other methods
should be called after the event loop is closed.
.. coroutinemethod:: AbstractEventLoop.shutdown_asyncgens() .. coroutinemethod:: loop.shutdown_asyncgens()
Schedule all currently open :term:`asynchronous generator` objects to Schedule all currently open :term:`asynchronous generator` objects to
close with an :meth:`~agen.aclose()` call. After calling this method, close with an :meth:`~agen.aclose()` call. After calling this method,
the event loop will issue a warning whenever a new asynchronous generator the event loop will issue a warning if a new asynchronous generator
is iterated. Should be used to finalize all scheduled asynchronous is iterated. Should be used to reliably finalize all scheduled
generators reliably. Example:: asynchronous generators, e.g.:
try: try:
loop.run_forever() loop.run_forever()
...@@ -110,163 +168,147 @@ Run an event loop ...@@ -110,163 +168,147 @@ Run an event loop
.. _asyncio-pass-keywords: .. _asyncio-pass-keywords:
Calls Scheduling callbacks
----- ^^^^^^^^^^^^^^^^^^^^
Most :mod:`asyncio` functions don't accept keywords. If you want to pass
keywords to your callback, use :func:`functools.partial`. For example,
``loop.call_soon(functools.partial(print, "Hello", flush=True))`` will call
``print("Hello", flush=True)``.
.. note::
:func:`functools.partial` is better than ``lambda`` functions, because
:mod:`asyncio` can inspect :func:`functools.partial` object to display
parameters in debug mode, whereas ``lambda`` functions have a poor
representation.
.. method:: AbstractEventLoop.call_soon(callback, *args, context=None) .. method:: loop.call_soon(callback, *args, context=None)
Arrange for a callback to be called as soon as possible. The callback is Schedule *callback* to be called with *args* arguments at
called after :meth:`call_soon` returns, when control returns to the event the next iteration of the event loop.
loop.
This operates as a :abbr:`FIFO (first-in, first-out)` queue, callbacks Callbacks are called in the order in which they are registered.
are called in the order in which they are registered. Each callback Each callback will be called exactly once.
will be called exactly once.
Any positional arguments after the callback will be passed to the An optional keyword-only *context* argument allows specifying a
callback when it is called. custom :class:`contextvars.Context` for the *callback* to run in.
The current context is used when no *context* is provided.
An optional keyword-only *context* argument allows specifying a custom
:class:`contextvars.Context` for the *callback* to run in. The current
context is used when no *context* is provided.
An instance of :class:`asyncio.Handle` is returned, which can be An instance of :class:`asyncio.Handle` is returned, which can be
used to cancel the callback. used to cancel the callback.
:ref:`Use functools.partial to pass keywords to the callback .. method:: loop.call_soon_threadsafe(callback, *args, context=None)
<asyncio-pass-keywords>`.
.. versionchanged:: 3.7
The *context* keyword-only parameter was added. See :pep:`567`
for more details.
.. method:: AbstractEventLoop.call_soon_threadsafe(callback, *args, context=None)
Like :meth:`call_soon`, but thread safe. A thread-safe variant of :meth:`call_soon`. Must be used to
schedule callbacks *from another thread*.
See the :ref:`concurrency and multithreading <asyncio-multithreading>` See the :ref:`concurrency and multithreading <asyncio-multithreading>`
section of the documentation. section of the documentation.
.. versionchanged:: 3.7 .. versionchanged:: 3.7
The *context* keyword-only parameter was added. See :pep:`567` The *context* keyword-only parameter was added. See :pep:`567`
for more details. for more details.
.. note::
.. _asyncio-delayed-calls: Most :mod:`asyncio` scheduling functions don't allow to pass
keyword arguments. To do that, use :func:`functools.partial`,
e.g.::
Delayed calls # will schedule "print("Hello", flush=True)":
------------- loop.call_soon(
functools.partial(print, "Hello", flush=True))
The event loop has its own internal clock for computing timeouts. Using partial objects is usually more convenient than using lambdas,
Which clock is used depends on the (platform-specific) event loop as asyncio can better render partial objects in debug and error
implementation; ideally it is a monotonic clock. This will generally be messages.
a different clock than :func:`time.time`.
.. method:: AbstractEventLoop.call_later(delay, callback, *args, context=None) .. _asyncio-delayed-calls:
Arrange for the *callback* to be called after the given *delay* Scheduling delayed callbacks
seconds (either an int or float). ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
An instance of :class:`asyncio.TimerHandle` is returned, which can be Event loop provides mechanisms to schedule callback functions
used to cancel the callback. to be called at some point in the future. Event loop uses monotonic
clocks to track time.
*callback* will be called exactly once per call to :meth:`call_later`.
If two callbacks are scheduled for exactly the same time, it is
undefined which will be called first.
The optional positional *args* will be passed to the callback when it .. method:: loop.call_later(delay, callback, *args, context=None)
is called. If you want the callback to be called with some named
arguments, use a closure or :func:`functools.partial`.
An optional keyword-only *context* argument allows specifying a custom Schedule *callback* to be called after the given *delay*
:class:`contextvars.Context` for the *callback* to run in. The current number of seconds (can be either an int or a float).
context is used when no *context* is provided.
:ref:`Use functools.partial to pass keywords to the callback An instance of :class:`asyncio.TimerHandle` is returned which can
<asyncio-pass-keywords>`. be used to cancel the callback.
*callback* will be called exactly once. If two callbacks are
scheduled for exactly the same time, it is undefined which will
be called first.
The optional positional *args* will be passed to the callback when
it is called. If you want the callback to be called with keyword
arguments use :func:`functools.partial`.
An optional keyword-only *context* argument allows specifying a
custom :class:`contextvars.Context` for the *callback* to run in.
The current context is used when no *context* is provided.
.. versionchanged:: 3.7 .. versionchanged:: 3.7
The *context* keyword-only parameter was added. See :pep:`567` The *context* keyword-only parameter was added. See :pep:`567`
for more details. for more details.
.. method:: AbstractEventLoop.call_at(when, callback, *args, context=None) .. method:: loop.call_at(when, callback, *args, context=None)
Arrange for the *callback* to be called at the given absolute timestamp Schedule *callback* to be called at the given absolute timestamp
*when* (an int or float), using the same time reference as *when* (an int or a float), using the same time reference as
:meth:`AbstractEventLoop.time`. :meth:`loop.time`.
This method's behavior is the same as :meth:`call_later`. This method's behavior is the same as :meth:`call_later`.
An instance of :class:`asyncio.TimerHandle` is returned, which can be An instance of :class:`asyncio.TimerHandle` is returned which can
used to cancel the callback. be used to cancel the callback.
:ref:`Use functools.partial to pass keywords to the callback
<asyncio-pass-keywords>`.
.. versionchanged:: 3.7 .. versionchanged:: 3.7
The *context* keyword-only parameter was added. See :pep:`567` The *context* keyword-only parameter was added. See :pep:`567`
for more details. for more details.
.. method:: AbstractEventLoop.time() .. method:: loop.time()
Return the current time, as a :class:`float` value, according to the Return the current time, as a :class:`float` value, according to
event loop's internal clock. the event loop's internal monotonic clock.
.. note::
Timeouts (relative *delay* or absolute *when*) should not
exceed one day.
.. seealso:: .. seealso::
The :func:`asyncio.sleep` function. The :func:`asyncio.sleep` function.
Futures Creating Futures and Tasks
------- ^^^^^^^^^^^^^^^^^^^^^^^^^^
.. method:: AbstractEventLoop.create_future() .. method:: loop.create_future()
Create an :class:`asyncio.Future` object attached to the loop. Create an :class:`asyncio.Future` object attached to the event loop.
This is a preferred way to create futures in asyncio, as event This is the preferred way to create Futures in asyncio, that lets
loop implementations can provide alternative implementations third-party event loops to provide alternative implementations of
of the Future class (with better performance or instrumentation). the Future object (with better performance or instrumentation).
.. versionadded:: 3.5.2 .. versionadded:: 3.5.2
.. method:: loop.create_task(coro, \*, name=None)
Tasks Schedule the execution of a :ref:`coroutine`.
----- Return a :class:`Task` object.
.. method:: AbstractEventLoop.create_task(coro, \*, name=None)
Schedule the execution of a :ref:`coroutine object <coroutine>`: wrap it in
a future. Return a :class:`Task` object.
Third-party event loops can use their own subclass of :class:`Task` for Third-party event loops can use their own subclass of :class:`Task`
interoperability. In this case, the result type is a subclass of for interoperability. In this case, the result type is a subclass
:class:`Task`. of :class:`Task`.
If the *name* argument is provided and not ``None``, it is set as the name If the *name* argument is provided and not ``None``, it is set as
of the task using :meth:`Task.set_name`. the name of the task using :meth:`Task.set_name`.
.. versionadded:: 3.4.2
.. versionchanged:: 3.8 .. versionchanged:: 3.8
Added the ``name`` parameter. Added the ``name`` parameter.
.. method:: AbstractEventLoop.set_task_factory(factory) .. method:: loop.set_task_factory(factory)
Set a task factory that will be used by Set a task factory that will be used by
:meth:`AbstractEventLoop.create_task`. :meth:`loop.create_task`.
If *factory* is ``None`` the default task factory will be set. If *factory* is ``None`` the default task factory will be set.
...@@ -275,53 +317,59 @@ Tasks ...@@ -275,53 +317,59 @@ Tasks
event loop, *coro* will be a coroutine object. The callable event loop, *coro* will be a coroutine object. The callable
must return an :class:`asyncio.Future` compatible object. must return an :class:`asyncio.Future` compatible object.
.. versionadded:: 3.4.4 .. method:: loop.get_task_factory()
Return a task factory or ``None`` if the default one is in use.
.. method:: AbstractEventLoop.get_task_factory()
Return a task factory, or ``None`` if the default one is in use. Opening network connections
^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. versionadded:: 3.4.4 .. coroutinemethod:: loop.create_connection(protocol_factory, \
host=None, port=None, \*, ssl=None, \
family=0, proto=0, flags=0, sock=None, \
local_addr=None, server_hostname=None, \
ssl_handshake_timeout=None)
Open a streaming transport connection to a given
address specified by *host* and *port*.
Creating connections The socket family can be either :py:data:`~socket.AF_INET` or
-------------------- :py:data:`~socket.AF_INET6` depending on *host* (or the *family*
argument, if provided).
.. coroutinemethod:: AbstractEventLoop.create_connection(protocol_factory, host=None, port=None, \*, ssl=None, family=0, proto=0, flags=0, sock=None, local_addr=None, server_hostname=None, ssl_handshake_timeout=None) The socket type will be :py:data:`~socket.SOCK_STREAM`.
Create a streaming transport connection to a given Internet *host* and *protocol_factory* must be a callable returning an
*port*: socket family :py:data:`~socket.AF_INET` or :ref:`asyncio protocol <asyncio-protocol>` implementation.
:py:data:`~socket.AF_INET6` depending on *host* (or *family* if specified),
socket type :py:data:`~socket.SOCK_STREAM`. *protocol_factory* must be a
callable returning a :ref:`protocol <asyncio-protocol>` instance.
This method will try to establish the connection in the background. This method will try to establish the connection in the background.
When successful, it returns a ``(transport, protocol)`` pair. When successful, it returns a ``(transport, protocol)`` pair.
The chronological synopsis of the underlying operation is as follows: The chronological synopsis of the underlying operation is as follows:
#. The connection is established, and a :ref:`transport <asyncio-transport>` #. The connection is established and a :ref:`transport <asyncio-transport>`
is created to represent it. is created for it.
#. *protocol_factory* is called without arguments and must return a #. *protocol_factory* is called without arguments and is expected to
:ref:`protocol <asyncio-protocol>` instance. return a :ref:`protocol <asyncio-protocol>` instance.
#. The protocol instance is tied to the transport, and its #. The protocol instance is coupled with the transport by calling its
:meth:`connection_made` method is called. :meth:`~BaseProtocol.connection_made` method.
#. The coroutine returns successfully with the ``(transport, protocol)`` #. A ``(transport, protocol)`` tuple is returned on success.
pair.
The created transport is an implementation-dependent bidirectional stream. The created transport is an implementation-dependent bidirectional
stream.
.. note:: .. note::
*protocol_factory* can be any kind of callable, not necessarily *protocol_factory* can be any kind of callable, not necessarily
a class. For example, if you want to use a pre-created a class. For example, if you want to use a pre-created
protocol instance, you can pass ``lambda: my_protocol``. protocol instance, you can pass ``lambda: my_protocol``.
Options that change how the connection is created: Other arguments:
* *ssl*: if given and not false, a SSL/TLS transport is created * *ssl*: if given and not false, an SSL/TLS transport is created
(by default a plain TCP transport is created). If *ssl* is (by default a plain TCP transport is created). If *ssl* is
a :class:`ssl.SSLContext` object, this context is used to create a :class:`ssl.SSLContext` object, this context is used to create
the transport; if *ssl* is :const:`True`, a context with some the transport; if *ssl* is :const:`True`, a context with some
...@@ -359,28 +407,41 @@ Creating connections ...@@ -359,28 +407,41 @@ Creating connections
The *ssl_handshake_timeout* parameter. The *ssl_handshake_timeout* parameter.
.. versionchanged:: 3.6
The socket option :py:data:`~socket.TCP_NODELAY` is set by default
for all TCP connections.
.. versionchanged:: 3.5 .. versionchanged:: 3.5
On Windows with :class:`ProactorEventLoop`, SSL/TLS is now supported. Added support for SSL/TLS for :class:`ProactorEventLoop`.
.. seealso:: .. seealso::
The :func:`open_connection` function can be used to get a pair of The :func:`open_connection` function is a high-level alternative
(:class:`StreamReader`, :class:`StreamWriter`) instead of a protocol. API. It returns a pair of (:class:`StreamReader`, :class:`StreamWriter`)
that can be used directly in async/await code.
.. coroutinemethod:: loop.create_datagram_endpoint(protocol_factory, \
local_addr=None, remote_addr=None, \*, \
family=0, proto=0, flags=0, \
reuse_address=None, reuse_port=None, \
allow_broadcast=None, sock=None)
.. coroutinemethod:: AbstractEventLoop.create_datagram_endpoint(protocol_factory, local_addr=None, remote_addr=None, \*, family=0, proto=0, flags=0, reuse_address=None, reuse_port=None, allow_broadcast=None, sock=None) Create a datagram connection.
Create datagram connection: socket family :py:data:`~socket.AF_INET`, The socket family can be either :py:data:`~socket.AF_INET`,
:py:data:`~socket.AF_INET6` or :py:data:`~socket.AF_UNIX` depending on :py:data:`~socket.AF_INET6`, or :py:data:`~socket.AF_UNIX`,
*host* (or *family* if specified), socket type depending on *host* (or the *family* argument, if provided).
:py:data:`~socket.SOCK_DGRAM`. *protocol_factory* must be a
callable returning a :ref:`protocol <asyncio-protocol>` instance.
This method will try to establish the connection in the background. The socket type will be :py:data:`~socket.SOCK_DGRAM`.
When successful, it returns a ``(transport, protocol)`` pair.
Options changing how the connection is created: *protocol_factory* must be a callable returning a
:ref:`protocol <asyncio-protocol>` implementation.
A tuple of ``(transport, protocol)`` is returned on success.
Other arguments:
* *local_addr*, if given, is a ``(local_host, local_port)`` tuple used * *local_addr*, if given, is a ``(local_host, local_port)`` tuple used
to bind the socket to locally. The *local_host* and *local_port* to bind the socket to locally. The *local_host* and *local_port*
...@@ -396,7 +457,7 @@ Creating connections ...@@ -396,7 +457,7 @@ Creating connections
corresponding :mod:`socket` module constants. corresponding :mod:`socket` module constants.
* *reuse_address* tells the kernel to reuse a local socket in * *reuse_address* tells the kernel to reuse a local socket in
TIME_WAIT state, without waiting for its natural timeout to ``TIME_WAIT`` state, without waiting for its natural timeout to
expire. If not specified will automatically be set to ``True`` on expire. If not specified will automatically be set to ``True`` on
UNIX. UNIX.
...@@ -423,21 +484,24 @@ Creating connections ...@@ -423,21 +484,24 @@ Creating connections
The *family*, *proto*, *flags*, *reuse_address*, *reuse_port, The *family*, *proto*, *flags*, *reuse_address*, *reuse_port,
*allow_broadcast*, and *sock* parameters were added. *allow_broadcast*, and *sock* parameters were added.
.. coroutinemethod:: AbstractEventLoop.create_unix_connection(protocol_factory, path=None, \*, ssl=None, sock=None, server_hostname=None, ssl_handshake_timeout=None) .. coroutinemethod:: loop.create_unix_connection(protocol_factory, \
path=None, \*, ssl=None, sock=None, \
server_hostname=None, ssl_handshake_timeout=None)
Create UNIX connection: socket family :py:data:`~socket.AF_UNIX`, socket Create UNIX connection.
type :py:data:`~socket.SOCK_STREAM`. The :py:data:`~socket.AF_UNIX` socket
family is used to communicate between processes on the same machine
efficiently.
This method will try to establish the connection in the background. The socket family will be :py:data:`~socket.AF_UNIX`; socket
When successful, it returns a ``(transport, protocol)`` pair. type will be :py:data:`~socket.SOCK_STREAM`.
A tuple of ``(transport, protocol)`` is returned on success.
*path* is the name of a UNIX domain socket, and is required unless a *sock* *path* is the name of a UNIX domain socket and is required,
parameter is specified. Abstract UNIX sockets, :class:`str`, unless a *sock* parameter is specified. Abstract UNIX sockets,
:class:`bytes`, and :class:`~pathlib.Path` paths are supported. :class:`str`, :class:`bytes`, and :class:`~pathlib.Path` paths are
supported.
See the :meth:`AbstractEventLoop.create_connection` method for parameters. See the documentation of the :meth:`loop.create_connection` method
for information about arguments to this method.
Availability: UNIX. Availability: UNIX.
...@@ -450,19 +514,23 @@ Creating connections ...@@ -450,19 +514,23 @@ Creating connections
The *path* parameter can now be a :term:`path-like object`. The *path* parameter can now be a :term:`path-like object`.
Creating listening connections Creating network servers
------------------------------ ^^^^^^^^^^^^^^^^^^^^^^^^
.. coroutinemethod:: AbstractEventLoop.create_server(protocol_factory, host=None, port=None, \*, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None, reuse_port=None, ssl_handshake_timeout=None, start_serving=True) .. coroutinemethod:: loop.create_server(protocol_factory, \
host=None, port=None, \*, \
family=socket.AF_UNSPEC, \
flags=socket.AI_PASSIVE, \
sock=None, backlog=100, ssl=None, \
reuse_address=None, reuse_port=None, \
ssl_handshake_timeout=None, start_serving=True)
Create a TCP server (socket type :data:`~socket.SOCK_STREAM`) bound to Create a TCP server (socket type :data:`~socket.SOCK_STREAM`) listening
*host* and *port*. on the *host* and *port* address.
Return a :class:`Server` object, its :attr:`~Server.sockets` attribute Returns a :class:`Server` object.
contains created sockets. Use the :meth:`Server.close` method to stop the
server: close listening sockets.
Parameters: Arguments:
* The *host* parameter can be a string, in that case the TCP server is * The *host* parameter can be a string, in that case the TCP server is
bound to *host* and *port*. The *host* parameter can also be a sequence bound to *host* and *port*. The *host* parameter can also be a sequence
...@@ -472,8 +540,9 @@ Creating listening connections ...@@ -472,8 +540,9 @@ Creating listening connections
for IPv4 and another one for IPv6). for IPv4 and another one for IPv6).
* *family* can be set to either :data:`socket.AF_INET` or * *family* can be set to either :data:`socket.AF_INET` or
:data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6. If not set :data:`~socket.AF_INET6` to force the socket to use IPv4 or IPv6.
it will be determined from host (defaults to :data:`socket.AF_UNSPEC`). If not set it will be determined from host name
(defaults to :data:`~socket.AF_UNSPEC`).
* *flags* is a bitmask for :meth:`getaddrinfo`. * *flags* is a bitmask for :meth:`getaddrinfo`.
...@@ -488,7 +557,7 @@ Creating listening connections ...@@ -488,7 +557,7 @@ Creating listening connections
accepted connections. accepted connections.
* *reuse_address* tells the kernel to reuse a local socket in * *reuse_address* tells the kernel to reuse a local socket in
TIME_WAIT state, without waiting for its natural timeout to ``TIME_WAIT`` state, without waiting for its natural timeout to
expire. If not specified will automatically be set to ``True`` on expire. If not specified will automatically be set to ``True`` on
UNIX. UNIX.
...@@ -509,30 +578,40 @@ Creating listening connections ...@@ -509,30 +578,40 @@ Creating listening connections
.. versionadded:: 3.7 .. versionadded:: 3.7
*ssl_handshake_timeout* and *start_serving* parameters. Added *ssl_handshake_timeout* and *start_serving* parameters.
.. versionchanged:: 3.5 .. versionchanged:: 3.6
On Windows with :class:`ProactorEventLoop`, SSL/TLS is now supported. The socket option :py:data:`~socket.TCP_NODELAY` is set by default
for all TCP connections.
.. seealso:: .. versionchanged:: 3.5
The function :func:`start_server` creates a (:class:`StreamReader`, Added support for SSL/TLS on Windows with
:class:`StreamWriter`) pair and calls back a function with this pair. :class:`ProactorEventLoop`.
.. versionchanged:: 3.5.1 .. versionchanged:: 3.5.1
The *host* parameter can now be a sequence of strings. The *host* parameter can be a sequence of strings.
.. seealso::
.. coroutinemethod:: AbstractEventLoop.create_unix_server(protocol_factory, path=None, \*, sock=None, backlog=100, ssl=None, ssl_handshake_timeout=None, start_serving=True) The :func:`start_server` function is a higher-level alternative API
that returns a pair of :class:`StreamReader` and :class:`StreamWriter`
that can be used in an async/await code.
Similar to :meth:`AbstractEventLoop.create_server`, but specific to the
socket family :py:data:`~socket.AF_UNIX`.
*path* is the name of a UNIX domain socket, and is required unless a *sock* .. coroutinemethod:: loop.create_unix_server(protocol_factory, path=None, \
parameter is specified. Abstract UNIX sockets, :class:`str`, \*, sock=None, backlog=100, ssl=None, \
:class:`bytes`, and :class:`~pathlib.Path` paths are supported. ssl_handshake_timeout=None, start_serving=True)
Similar to :meth:`loop.create_server` but works with the
:py:data:`~socket.AF_UNIX` socket family.
*path* is the name of a UNIX domain socket, and is required,
unless a *sock* argument is provided. Abstract UNIX sockets,
:class:`str`, :class:`bytes`, and :class:`~pathlib.Path` paths
are supported.
Availability: UNIX. Availability: UNIX.
...@@ -544,26 +623,27 @@ Creating listening connections ...@@ -544,26 +623,27 @@ Creating listening connections
The *path* parameter can now be a :class:`~pathlib.Path` object. The *path* parameter can now be a :class:`~pathlib.Path` object.
.. coroutinemethod:: BaseEventLoop.connect_accepted_socket(protocol_factory, sock, \*, ssl=None, ssl_handshake_timeout=None) .. coroutinemethod:: loop.connect_accepted_socket(protocol_factory, \
sock, \*, ssl=None, ssl_handshake_timeout=None)
Handle an accepted connection. Wrap an already accepted connection into a transport/protocol pair.
This is used by servers that accept connections outside of This method can be used by servers that accept connections outside
asyncio but that use asyncio to handle them. of asyncio but that use asyncio to handle them.
Parameters: Parameters:
* *sock* is a preexisting socket object returned from an ``accept`` * *sock* is a preexisting socket object returned from
call. :meth:`socket.accept <socket.socket.accept>`.
* *ssl* can be set to an :class:`~ssl.SSLContext` to enable SSL over the * *ssl* can be set to an :class:`~ssl.SSLContext` to enable SSL over
accepted connections. the accepted connections.
* *ssl_handshake_timeout* is (for an SSL connection) the time in seconds to * *ssl_handshake_timeout* is (for an SSL connection) the time in seconds to
wait for the SSL handshake to complete before aborting the connection. wait for the SSL handshake to complete before aborting the connection.
``60.0`` seconds if ``None`` (default). ``60.0`` seconds if ``None`` (default).
When completed it returns a ``(transport, protocol)`` pair. Returns a ``(transport, protocol)`` pair.
.. versionadded:: 3.7 .. versionadded:: 3.7
...@@ -572,15 +652,14 @@ Creating listening connections ...@@ -572,15 +652,14 @@ Creating listening connections
.. versionadded:: 3.5.3 .. versionadded:: 3.5.3
File Transferring Transferring files
----------------- ^^^^^^^^^^^^^^^^^^
.. coroutinemethod:: AbstractEventLoop.sendfile(transport, file, \ .. coroutinemethod:: loop.sendfile(transport, file, \
offset=0, count=None, \ offset=0, count=None, *, fallback=True)
*, fallback=True)
Send a *file* to *transport*, return the total number of bytes Send a *file* over a *transport*. Return the total number of bytes
which were sent. sent.
The method uses high-performance :meth:`os.sendfile` if available. The method uses high-performance :meth:`os.sendfile` if available.
...@@ -594,7 +673,7 @@ File Transferring ...@@ -594,7 +673,7 @@ File Transferring
which were sent. which were sent.
*fallback* set to ``True`` makes asyncio to manually read and send *fallback* set to ``True`` makes asyncio to manually read and send
the file when the platform does not support the sendfile syscall the file when the platform does not support the sendfile system call
(e.g. Windows or SSL socket on Unix). (e.g. Windows or SSL socket on Unix).
Raise :exc:`SendfileNotAvailableError` if the system does not support Raise :exc:`SendfileNotAvailableError` if the system does not support
...@@ -604,26 +683,28 @@ File Transferring ...@@ -604,26 +683,28 @@ File Transferring
TLS Upgrade TLS Upgrade
----------- ^^^^^^^^^^^
.. coroutinemethod:: AbstractEventLoop.start_tls(transport, protocol, sslcontext, \*, server_side=False, server_hostname=None, ssl_handshake_timeout=None) .. coroutinemethod:: loop.start_tls(transport, protocol, \
sslcontext, \*, server_side=False, \
server_hostname=None, ssl_handshake_timeout=None)
Upgrades an existing connection to TLS. Upgrade an existing transport-based connection to TLS.
Returns a new transport instance, that the *protocol* must start using Return a new transport instance, that the *protocol* must start using
immediately after the *await*. The *transport* instance passed to immediately after the *await*. The *transport* instance passed to
the *start_tls* method should never be used again. the *start_tls* method should never be used again.
Parameters: Parameters:
* *transport* and *protocol* instances that methods like * *transport* and *protocol* instances that methods like
:meth:`~AbstractEventLoop.create_server` and :meth:`~loop.create_server` and
:meth:`~AbstractEventLoop.create_connection` return. :meth:`~loop.create_connection` return.
* *sslcontext*: a configured instance of :class:`~ssl.SSLContext`. * *sslcontext*: a configured instance of :class:`~ssl.SSLContext`.
* *server_side* pass ``True`` when a server-side connection is being * *server_side* pass ``True`` when a server-side connection is being
upgraded (like the one created by :meth:`~AbstractEventLoop.create_server`). upgraded (like the one created by :meth:`~loop.create_server`).
* *server_hostname*: sets or overrides the host name that the target * *server_hostname*: sets or overrides the host name that the target
server's certificate will be matched against. server's certificate will be matched against.
...@@ -635,120 +716,112 @@ TLS Upgrade ...@@ -635,120 +716,112 @@ TLS Upgrade
.. versionadded:: 3.7 .. versionadded:: 3.7
Watch file descriptors Watching file descriptors
---------------------- ^^^^^^^^^^^^^^^^^^^^^^^^^
On Windows with :class:`SelectorEventLoop`, only socket handles are supported .. method:: loop.add_reader(fd, callback, \*args)
(ex: pipe file descriptors are not supported).
On Windows with :class:`ProactorEventLoop`, these methods are not supported. Start watching the file descriptor for read availability and
call the *callback* with specified arguments.
.. method:: AbstractEventLoop.add_reader(fd, callback, \*args) .. method:: loop.remove_reader(fd)
Start watching the file descriptor for read availability and then call the
*callback* with specified arguments.
:ref:`Use functools.partial to pass keywords to the callback
<asyncio-pass-keywords>`.
.. method:: AbstractEventLoop.remove_reader(fd)
Stop watching the file descriptor for read availability. Stop watching the file descriptor for read availability.
.. method:: AbstractEventLoop.add_writer(fd, callback, \*args) .. method:: loop.add_writer(fd, callback, \*args)
Start watching the file descriptor for write availability and then call the Start watching the file descriptor for write availability and then
*callback* with specified arguments. call the *callback* with specified arguments.
:ref:`Use functools.partial to pass keywords to the callback Use :func:`functools.partial` :ref:`to pass keywords
<asyncio-pass-keywords>`. <asyncio-pass-keywords>` to *func*.
.. method:: AbstractEventLoop.remove_writer(fd) .. method:: loop.remove_writer(fd)
Stop watching the file descriptor for write availability. Stop watching the file descriptor for write availability.
The :ref:`watch a file descriptor for read events <asyncio-watch-read-event>` See also :ref:`Platform Support <asyncio-platform-support>` section
example uses the low-level :meth:`AbstractEventLoop.add_reader` method to register for some limitations of these methods.
the file descriptor of a socket.
Low-level socket operations Working with socket objects directly
--------------------------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
.. coroutinemethod:: AbstractEventLoop.sock_recv(sock, nbytes) In general, protocols implementations that use transport-based APIs
such as :meth:`loop.create_connection` and :meth:`loop.create_server`
are faster than implementations that work with sockets directly.
However, there are use cases when performance is not critical and
working with :class:`~socket.socket` objects directly is more
convenient.
Receive data from the socket. Modeled after blocking .. coroutinemethod:: loop.sock_recv(sock, nbytes)
:meth:`socket.socket.recv` method.
The return value is a bytes object Receive data. Asynchronous version of
representing the data received. The maximum amount of data to be received :meth:`socket.recv() <socket.socket.recv>`.
at once is specified by *nbytes*.
With :class:`SelectorEventLoop` event loop, the socket *sock* must be The received data is returned as a bytes object. The maximum amount
non-blocking. of data to be received is specified by the *nbytes* argument.
The socket *sock* must be non-blocking.
.. versionchanged:: 3.7 .. versionchanged:: 3.7
Even though the method was always documented as a coroutine Even though the method was always documented as a coroutine
method, before Python 3.7 it returned a :class:`Future`. method, before Python 3.7 it returned a :class:`Future`.
Since Python 3.7, this is an ``async def`` method. Since Python 3.7 this is an ``async def`` method.
.. coroutinemethod:: AbstractEventLoop.sock_recv_into(sock, buf) .. coroutinemethod:: loop.sock_recv_into(sock, buf)
Receive data from the socket. Modeled after blocking Receive data into a buffer. Modeled after the blocking
:meth:`socket.socket.recv_into` method. :meth:`socket.recv_into() <socket.socket.recv_into>` method.
The received data is written into *buf* (a writable buffer). Return the number of bytes written to the buffer.
The return value is the number of bytes written.
With :class:`SelectorEventLoop` event loop, the socket *sock* must be The socket *sock* must be non-blocking.
non-blocking.
.. versionadded:: 3.7 .. versionadded:: 3.7
.. coroutinemethod:: AbstractEventLoop.sock_sendall(sock, data) .. coroutinemethod:: loop.sock_sendall(sock, data)
Send data to the socket. Modeled after blocking Send data to the socket. Asynchronous version of
:meth:`socket.socket.sendall` method. :meth:`socket.sendall() <socket.socket.sendall>`.
The socket must be connected to a remote socket.
This method continues to send data from *data* until either all data has This method continues to send data from *data* until either all data has
been sent or an error occurs. ``None`` is returned on success. On error, been sent or an error occurs. ``None`` is returned on success. On error,
an exception is raised, and there is no way to determine how much data, if an exception is raised, and there is no way to determine how much data, if
any, was successfully processed by the receiving end of the connection. any, was successfully processed by the receiving end of the connection.
With :class:`SelectorEventLoop` event loop, the socket *sock* must be The socket *sock* must be non-blocking.
non-blocking.
.. versionchanged:: 3.7 .. versionchanged:: 3.7
Even though the method was always documented as a coroutine Even though the method was always documented as a coroutine
method, before Python 3.7 it returned an :class:`Future`. method, before Python 3.7 it returned an :class:`Future`.
Since Python 3.7, this is an ``async def`` method. Since Python 3.7, this is an ``async def`` method.
.. coroutinemethod:: AbstractEventLoop.sock_connect(sock, address) .. coroutinemethod:: loop.sock_connect(sock, address)
Connect to a remote socket at *address*.
Connect to a remote socket at *address*. Modeled after Asynchronous version of :meth:`socket.connect() <socket.socket.connect>`.
blocking :meth:`socket.socket.connect` method.
With :class:`SelectorEventLoop` event loop, the socket *sock* must be The socket *sock* must be non-blocking.
non-blocking.
.. versionchanged:: 3.5.2 .. versionchanged:: 3.5.2
``address`` no longer needs to be resolved. ``sock_connect`` ``address`` no longer needs to be resolved. ``sock_connect``
will try to check if the *address* is already resolved by calling will try to check if the *address* is already resolved by calling
:func:`socket.inet_pton`. If not, :func:`socket.inet_pton`. If not,
:meth:`AbstractEventLoop.getaddrinfo` will be used to resolve the :meth:`loop.getaddrinfo` will be used to resolve the
*address*. *address*.
.. seealso:: .. seealso::
:meth:`AbstractEventLoop.create_connection` :meth:`loop.create_connection`
and :func:`asyncio.open_connection() <open_connection>`. and :func:`asyncio.open_connection() <open_connection>`.
.. coroutinemethod:: AbstractEventLoop.sock_accept(sock) .. coroutinemethod:: loop.sock_accept(sock)
Accept a connection. Modeled after blocking Accept a connection. Modeled after the blocking
:meth:`socket.socket.accept`. :meth:`socket.accept() <socket.socket.accept>` method.
The socket must be bound to an address and listening The socket must be bound to an address and listening
for connections. The return value is a pair ``(conn, address)`` where *conn* for connections. The return value is a pair ``(conn, address)`` where *conn*
...@@ -765,16 +838,15 @@ Low-level socket operations ...@@ -765,16 +838,15 @@ Low-level socket operations
.. seealso:: .. seealso::
:meth:`AbstractEventLoop.create_server` and :func:`start_server`. :meth:`loop.create_server` and :func:`start_server`.
.. coroutinemethod:: AbstractEventLoop.sock_sendfile(sock, file, \ .. coroutinemethod:: loop.sock_sendfile(sock, file, offset=0, count=None, \
offset=0, count=None, \ \*, fallback=True)
*, fallback=True)
Send a file using high-performance :mod:`os.sendfile` if possible Send a file using high-performance :mod:`os.sendfile` if possible.
and return the total number of bytes which were sent. Return the total number of bytes which were sent.
Asynchronous version of :meth:`socket.socket.sendfile`. Asynchronous version of :meth:`socket.sendfile() <socket.socket.sendfile>`.
*sock* must be non-blocking :class:`~socket.socket` of *sock* must be non-blocking :class:`~socket.socket` of
:const:`socket.SOCK_STREAM` type. :const:`socket.SOCK_STREAM` type.
...@@ -795,21 +867,22 @@ Low-level socket operations ...@@ -795,21 +867,22 @@ Low-level socket operations
Raise :exc:`SendfileNotAvailableError` if the system does not support Raise :exc:`SendfileNotAvailableError` if the system does not support
*sendfile* syscall and *fallback* is ``False``. *sendfile* syscall and *fallback* is ``False``.
The socket *sock* must be non-blocking.
.. versionadded:: 3.7 .. versionadded:: 3.7
Resolve host name DNS
----------------- ^^^
.. coroutinemethod:: AbstractEventLoop.getaddrinfo(host, port, \*, family=0, type=0, proto=0, flags=0) .. coroutinemethod:: loop.getaddrinfo(host, port, \*, family=0, \
type=0, proto=0, flags=0)
This method is a :ref:`coroutine <coroutine>`, similar to Asynchronous version of :meth:`socket.getaddrinfo`.
:meth:`socket.getaddrinfo` function but non-blocking.
.. coroutinemethod:: AbstractEventLoop.getnameinfo(sockaddr, flags=0) .. coroutinemethod:: loop.getnameinfo(sockaddr, flags=0)
This method is a :ref:`coroutine <coroutine>`, similar to Asynchronous version of :meth:`socket.getnameinfo`.
:meth:`socket.getnameinfo` function but non-blocking.
.. versionchanged:: 3.7 .. versionchanged:: 3.7
Both *getaddrinfo* and *getnameinfo* methods were always documented Both *getaddrinfo* and *getnameinfo* methods were always documented
...@@ -818,95 +891,99 @@ Resolve host name ...@@ -818,95 +891,99 @@ Resolve host name
both methods are coroutines. both methods are coroutines.
Connect pipes Working with pipes
------------- ^^^^^^^^^^^^^^^^^^
On Windows with :class:`SelectorEventLoop`, these methods are not supported. .. coroutinemethod:: loop.connect_read_pipe(protocol_factory, pipe)
Use :class:`ProactorEventLoop` to support pipes on Windows.
.. coroutinemethod:: AbstractEventLoop.connect_read_pipe(protocol_factory, pipe) Register a read-pipe in the event loop.
Register read pipe in eventloop. *protocol_factory* must be a callable returning an
:ref:`asyncio protocol <asyncio-protocol>` implementation.
*protocol_factory* should instantiate object with :class:`Protocol` *pipe* is a :term:`file-like object <file object>`.
interface. *pipe* is a :term:`file-like object <file object>`.
Return pair ``(transport, protocol)``, where *transport* supports the Return pair ``(transport, protocol)``, where *transport* supports
:class:`ReadTransport` interface. the :class:`ReadTransport` interface.
With :class:`SelectorEventLoop` event loop, the *pipe* is set to With :class:`SelectorEventLoop` event loop, the *pipe* is set to
non-blocking mode. non-blocking mode.
.. coroutinemethod:: AbstractEventLoop.connect_write_pipe(protocol_factory, pipe) .. coroutinemethod:: loop.connect_write_pipe(protocol_factory, pipe)
Register a write-pipe in the event loop.
*protocol_factory* must be a callable returning an
:ref:`asyncio protocol <asyncio-protocol>` implementation.
Register write pipe in eventloop. *pipe* is :term:`file-like object <file object>`.
*protocol_factory* should instantiate object with :class:`BaseProtocol`
interface. *pipe* is :term:`file-like object <file object>`.
Return pair ``(transport, protocol)``, where *transport* supports Return pair ``(transport, protocol)``, where *transport* supports
:class:`WriteTransport` interface. :class:`WriteTransport` interface.
With :class:`SelectorEventLoop` event loop, the *pipe* is set to With :class:`SelectorEventLoop` event loop, the *pipe* is set to
non-blocking mode. non-blocking mode.
.. note::
:class:`SelectorEventLoop` does not support the above methods on
Windows. Use :class:`ProactorEventLoop` instead.
.. seealso:: .. seealso::
The :meth:`AbstractEventLoop.subprocess_exec` and The :meth:`loop.subprocess_exec` and
:meth:`AbstractEventLoop.subprocess_shell` methods. :meth:`loop.subprocess_shell` methods.
UNIX signals UNIX signals
------------ ^^^^^^^^^^^^
Availability: UNIX only. .. method:: loop.add_signal_handler(signum, callback, \*args)
.. method:: AbstractEventLoop.add_signal_handler(signum, callback, \*args)
Add a handler for a signal. Add a handler for a signal.
Raise :exc:`ValueError` if the signal number is invalid or uncatchable. Raise :exc:`ValueError` if the signal number is invalid or uncatchable.
Raise :exc:`RuntimeError` if there is a problem setting up the handler. Raise :exc:`RuntimeError` if there is a problem setting up the handler.
:ref:`Use functools.partial to pass keywords to the callback Use :func:`functools.partial` :ref:`to pass keywords
<asyncio-pass-keywords>`. <asyncio-pass-keywords>` to *func*.
.. method:: AbstractEventLoop.remove_signal_handler(sig) .. method:: loop.remove_signal_handler(sig)
Remove a handler for a signal. Remove a handler for a signal.
Return ``True`` if a signal handler was removed, ``False`` if not. Return ``True`` if a signal handler was removed, ``False`` if not.
Availability: UNIX.
.. seealso:: .. seealso::
The :mod:`signal` module. The :mod:`signal` module.
Executor Executing code in thread or process pools
-------- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Call a function in an :class:`~concurrent.futures.Executor` (pool of threads or
pool of processes). By default, an event loop uses a thread pool executor
(:class:`~concurrent.futures.ThreadPoolExecutor`).
.. method:: AbstractEventLoop.run_in_executor(executor, func, \*args) .. method:: loop.run_in_executor(executor, func, \*args)
Arrange for a *func* to be called in the specified executor. Arrange for a *func* to be called in the specified executor.
The *executor* argument should be an :class:`~concurrent.futures.Executor` The *executor* argument should be an :class:`concurrent.futures.Executor`
instance. The default executor is used if *executor* is ``None``. instance. The default executor is used if *executor* is ``None``.
:ref:`Use functools.partial to pass keywords to the *func* Use :func:`functools.partial` :ref:`to pass keywords
<asyncio-pass-keywords>`. <asyncio-pass-keywords>` to *func*.
This method returns a :class:`asyncio.Future` object. This method returns a :class:`asyncio.Future` object.
.. versionchanged:: 3.5.3 .. versionchanged:: 3.5.3
:meth:`BaseEventLoop.run_in_executor` no longer configures the :meth:`loop.run_in_executor` no longer configures the
``max_workers`` of the thread pool executor it creates, instead ``max_workers`` of the thread pool executor it creates, instead
leaving it up to the thread pool executor leaving it up to the thread pool executor
(:class:`~concurrent.futures.ThreadPoolExecutor`) to set the (:class:`~concurrent.futures.ThreadPoolExecutor`) to set the
default. default.
.. method:: AbstractEventLoop.set_default_executor(executor) .. method:: loop.set_default_executor(executor)
Set *executor* as the default executor used by :meth:`run_in_executor`. Set *executor* as the default executor used by :meth:`run_in_executor`.
*executor* should be an instance of *executor* should be an instance of
...@@ -917,13 +994,16 @@ pool of processes). By default, an event loop uses a thread pool executor ...@@ -917,13 +994,16 @@ pool of processes). By default, an event loop uses a thread pool executor
:class:`~concurrent.futures.ThreadPoolExecutor` is deprecated and :class:`~concurrent.futures.ThreadPoolExecutor` is deprecated and
will trigger an error in Python 3.9. will trigger an error in Python 3.9.
*executor* must be an instance of
:class:`concurrent.futures.ThreadPoolExecutor`.
Error Handling API Error Handling API
------------------ ^^^^^^^^^^^^^^^^^^
Allows customizing how exceptions are handled in the event loop. Allows customizing how exceptions are handled in the event loop.
.. method:: AbstractEventLoop.set_exception_handler(handler) .. method:: loop.set_exception_handler(handler)
Set *handler* as the new event loop exception handler. Set *handler* as the new event loop exception handler.
...@@ -936,14 +1016,14 @@ Allows customizing how exceptions are handled in the event loop. ...@@ -936,14 +1016,14 @@ Allows customizing how exceptions are handled in the event loop.
will be a ``dict`` object (see :meth:`call_exception_handler` will be a ``dict`` object (see :meth:`call_exception_handler`
documentation for details about context). documentation for details about context).
.. method:: AbstractEventLoop.get_exception_handler() .. method:: loop.get_exception_handler()
Return the exception handler, or ``None`` if the default one Return the exception handler, or ``None`` if the default one
is in use. is in use.
.. versionadded:: 3.5.2 .. versionadded:: 3.5.2
.. method:: AbstractEventLoop.default_exception_handler(context) .. method:: loop.default_exception_handler(context)
Default exception handler. Default exception handler.
...@@ -954,7 +1034,7 @@ Allows customizing how exceptions are handled in the event loop. ...@@ -954,7 +1034,7 @@ Allows customizing how exceptions are handled in the event loop.
*context* parameter has the same meaning as in *context* parameter has the same meaning as in
:meth:`call_exception_handler`. :meth:`call_exception_handler`.
.. method:: AbstractEventLoop.call_exception_handler(context) .. method:: loop.call_exception_handler(context)
Call the current event loop exception handler. Call the current event loop exception handler.
...@@ -975,10 +1055,10 @@ Allows customizing how exceptions are handled in the event loop. ...@@ -975,10 +1055,10 @@ Allows customizing how exceptions are handled in the event loop.
event loops. For any custom exception handling, use event loops. For any custom exception handling, use
:meth:`set_exception_handler()` method. :meth:`set_exception_handler()` method.
Debug mode Enabling debug mode
---------- ^^^^^^^^^^^^^^^^^^^
.. method:: AbstractEventLoop.get_debug() .. method:: loop.get_debug()
Get the debug mode (:class:`bool`) of the event loop. Get the debug mode (:class:`bool`) of the event loop.
...@@ -986,29 +1066,167 @@ Debug mode ...@@ -986,29 +1066,167 @@ Debug mode
:envvar:`PYTHONASYNCIODEBUG` is set to a non-empty string, ``False`` :envvar:`PYTHONASYNCIODEBUG` is set to a non-empty string, ``False``
otherwise. otherwise.
.. versionadded:: 3.4.2 .. method:: loop.set_debug(enabled: bool)
.. method:: AbstractEventLoop.set_debug(enabled: bool)
Set the debug mode of the event loop. Set the debug mode of the event loop.
.. versionadded:: 3.4.2
.. seealso:: .. seealso::
The :ref:`debug mode of asyncio <asyncio-debug-mode>`. The :ref:`debug mode of asyncio <asyncio-debug-mode>`.
Server
------
.. class:: Server Running Subprocesses
^^^^^^^^^^^^^^^^^^^^
Methods described in this subsections are low-level. In an
async/await code consider using high-level convenient
:func:`asyncio.create_subprocess_shell` and
:func:`asyncio.create_subprocess_exec` functions instead.
.. note::
The default event loop that asyncio is pre-configured
to use on **Windows** does not support subprocesses.
See :ref:`Subprocess Support on Windows <asyncio-windows-subprocess>`
for details.
.. coroutinemethod:: loop.subprocess_exec(protocol_factory, \*args, \
stdin=subprocess.PIPE, stdout=subprocess.PIPE, \
stderr=subprocess.PIPE, \*\*kwargs)
Create a subprocess from one or more string arguments specified by
*args*.
*args* must be a list of strings represented by:
* :class:`str`;
* or :class:`bytes`, encoded to the
:ref:`filesystem encoding <filesystem-encoding>`.
The first string specifies the program to execute,
and the remaining strings specify the arguments. Together string
arguments form the ``argv`` of the program.
This is similar to the standard library :class:`subprocess.Popen`
class called with ``shell=False`` and the list of strings passed as
the first argument; however, where :class:`~subprocess.Popen` takes
a single argument which is list of strings, *subprocess_exec*
takes multiple string arguments.
The *protocol_factory* must instantiate a subclass of the
:class:`asyncio.SubprocessProtocol` class.
Other parameters:
* *stdin*: either a file-like object representing a pipe to be
connected to the subprocess's standard input stream using
:meth:`~loop.connect_write_pipe`, or the
:const:`subprocess.PIPE` constant (default). By default a new
pipe will be created and connected.
* *stdout*: either a file-like object representing the pipe to be
connected to the subprocess's standard output stream using
:meth:`~loop.connect_read_pipe`, or the
:const:`subprocess.PIPE` constant (default). By default a new pipe
will be created and connected.
* *stderr*: either a file-like object representing the pipe to be
connected to the subprocess's standard error stream using
:meth:`~loop.connect_read_pipe`, or one of
:const:`subprocess.PIPE` (default) or :const:`subprocess.STDOUT`
constants.
By default a new pipe will be created and connected. When
:const:`subprocess.STDOUT` is specified, the subprocess' standard
error stream will be connected to the same pipe as the standard
output stream.
* All other keyword arguments are passed to :class:`subprocess.Popen`
without interpretation, except for *bufsize*, *universal_newlines*
and *shell*, which should not be specified at all.
See the constructor of the :class:`subprocess.Popen` class
for documentation on other arguments.
Returns a pair of ``(transport, protocol)``, where *transport*
conforms to the :class:`asyncio.SubprocessTransport` base class.
.. coroutinemethod:: loop.subprocess_shell(protocol_factory, cmd, \*, \
stdin=subprocess.PIPE, stdout=subprocess.PIPE, \
stderr=subprocess.PIPE, \*\*kwargs)
Create a subprocess from *cmd*, which can be a :class:`str` or a
:class:`bytes` string encoded to the
:ref:`filesystem encoding <filesystem-encoding>`,
using the platform's "shell" syntax.
This is similar to the standard library :class:`subprocess.Popen`
class called with ``shell=True``.
The *protocol_factory* must instantiate a subclass of the
:class:`SubprocessProtocol` class.
See :meth:`~loop.subprocess_exec` for more details about
the remaining arguments.
Returns a pair of ``(transport, protocol)``, where *transport*
conforms to the :class:`SubprocessTransport` base class.
.. note::
It is the application's responsibility to ensure that all whitespace
and metacharacters are quoted appropriately to avoid `shell injection
<https://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_
vulnerabilities. The :func:`shlex.quote` function can be used to
properly escape whitespace and shell metacharacters in strings that
are going to be used to construct shell commands.
Callback Handles
================
.. class:: Handle
A callback wrapper object returned by :meth:`loop.call_soon`,
:meth:`loop.call_soon_threadsafe`.
.. method:: cancel()
Cancel the call. If the callback is already canceled or executed,
this method has no effect.
.. method:: cancelled()
Return ``True`` if the call was cancelled.
.. versionadded:: 3.7
.. class:: TimerHandle
A callback wrapper object returned by :meth:`loop.call_later`,
and :meth:`loop.call_at`.
The class is inherited from :class:`Handle`.
.. method:: when()
Server listening on sockets. Return a scheduled callback time as :class:`float` seconds.
The time is an absolute timestamp, using the same time
reference as :meth:`loop.time`.
.. versionadded:: 3.7
Server Objects
==============
Object created by :meth:`AbstractEventLoop.create_server`, Server objects are created by :meth:`loop.create_server`,
:meth:`AbstractEventLoop.create_unix_server`, :func:`start_server`, :meth:`loop.create_unix_server`, :func:`start_server`,
and :func:`start_unix_server` functions. Don't instantiate the class and :func:`start_unix_server` functions.
directly.
Do not instantiate the class directly.
.. class:: Server
*Server* objects are asynchronous context managers. When used in an *Server* objects are asynchronous context managers. When used in an
``async with`` statement, it's guaranteed that the Server object is ``async with`` statement, it's guaranteed that the Server object is
...@@ -1020,7 +1238,8 @@ Server ...@@ -1020,7 +1238,8 @@ Server
async with srv: async with srv:
# some code # some code
# At this point, srv is closed and no longer accepts new connections. # At this point, srv is closed and no longer accepts new
connections.
.. versionchanged:: 3.7 .. versionchanged:: 3.7
...@@ -1031,8 +1250,8 @@ Server ...@@ -1031,8 +1250,8 @@ Server
Stop serving: close listening sockets and set the :attr:`sockets` Stop serving: close listening sockets and set the :attr:`sockets`
attribute to ``None``. attribute to ``None``.
The sockets that represent existing incoming client connections are left The sockets that represent existing incoming client connections
open. are left open.
The server is closed asynchronously, use the :meth:`wait_closed` The server is closed asynchronously, use the :meth:`wait_closed`
coroutine to wait until the server is closed. coroutine to wait until the server is closed.
...@@ -1051,7 +1270,7 @@ Server ...@@ -1051,7 +1270,7 @@ Server
the server is already being serving. the server is already being serving.
The new *start_serving* keyword-only parameter to The new *start_serving* keyword-only parameter to
:meth:`AbstractEventLoop.create_server` and :meth:`loop.create_server` and
:meth:`asyncio.start_server` allows to create a Server object :meth:`asyncio.start_server` allows to create a Server object
that is not accepting connections right away. In which case that is not accepting connections right away. In which case
this method, or :meth:`Server.serve_forever` can be used this method, or :meth:`Server.serve_forever` can be used
...@@ -1097,8 +1316,8 @@ Server ...@@ -1097,8 +1316,8 @@ Server
.. attribute:: sockets .. attribute:: sockets
List of :class:`socket.socket` objects the server is listening to, or List of :class:`socket.socket` objects the server is listening to,
``None`` if the server is closed. or ``None`` if the server is closed.
.. versionchanged:: 3.7 .. versionchanged:: 3.7
Prior to Python 3.7 ``Server.sockets`` used to return the Prior to Python 3.7 ``Server.sockets`` used to return the
...@@ -1106,65 +1325,85 @@ Server ...@@ -1106,65 +1325,85 @@ Server
of that list is returned. of that list is returned.
Handle .. _asyncio-event-loops:
------
.. class:: Handle Event Loops Implementations
===========================
A callback wrapper object returned by :func:`AbstractEventLoop.call_soon`, asyncio ships with two different event loop implementations:
:func:`AbstractEventLoop.call_soon_threadsafe`. :class:`SelectorEventLoop` and :class:`ProactorEventLoop`.
.. method:: cancel() By default asyncio is configured to use :class:`SelectorEventLoop`
on all platforms.
Cancel the call. If the callback is already canceled or executed,
this method has no effect.
.. method:: cancelled() .. class:: SelectorEventLoop
Return ``True`` if the call was cancelled. An event loop based on the :mod:`selectors` module.
.. versionadded:: 3.7 Uses the most efficient *selector* available for the given
platform. It is also possible to manually configure what
exact selector implementation should be used::
.. class:: TimerHandle import asyncio
import selectors
A callback wrapper object returned by :func:`AbstractEventLoop.call_later`, selector = selectors.SelectSelector()
and :func:`AbstractEventLoop.call_at`. loop = asyncio.SelectorEventLoop(selector)
asyncio.set_event_loop(loop)
The class is inherited from :class:`Handle`.
.. method:: when() Availability: UNIX, Windows.
Return a scheduled callback time as :class:`float` seconds.
The time is an absolute timestamp, using the same time .. class:: ProactorEventLoop
reference as :meth:`AbstractEventLoop.time`.
.. versionadded:: 3.7 An event loop for Windows that uses "I/O Completion Ports" (IOCP).
Availability: Windows.
SendfileNotAvailableError An example how to use :class:`ProactorEventLoop` on Windows::
-------------------------
import asyncio
import sys
.. exception:: SendfileNotAvailableError if sys.platform == 'win32':
loop = asyncio.ProactorEventLoop()
asyncio.set_event_loop(loop)
Sendfile syscall is not available, subclass of :exc:`RuntimeError`. .. seealso::
`MSDN documentation on I/O Completion Ports
<https://docs.microsoft.com/en-ca/windows/desktop/FileIO/i-o-completion-ports>`_.
.. class:: AbstractEventLoop
Abstract base class for asyncio-compliant event loops.
The :ref:`Event Loop Methods <asyncio-event-loop>` section lists all
methods that an alternative implementation of ``AbstractEventLoop``
should have defined.
Raised if the OS does not support sendfile syscall for
given socket or file type.
Examples
========
Note that all examples in this section **purposefully** show how
to use low-level event loop APIs such as :meth:`loop.run_forever`
and :meth:`loop.call_soon`. Modern asyncio applications rarely
need to be written this way; consider using high-level functions
like :func:`asyncio.run`.
Event loop examples
-------------------
.. _asyncio-hello-world-callback: .. _asyncio-hello-world-callback:
Hello World with call_soon() Hello World with call_soon()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Example using the :meth:`AbstractEventLoop.call_soon` method to schedule a An example using the :meth:`loop.call_soon` method to schedule a
callback. The callback displays ``"Hello World"`` and then stops the event callback. The callback displays ``"Hello World"`` and then stops the
loop:: event loop::
import asyncio import asyncio
...@@ -1178,13 +1417,15 @@ loop:: ...@@ -1178,13 +1417,15 @@ loop::
loop.call_soon(hello_world, loop) loop.call_soon(hello_world, loop)
# Blocking call interrupted by loop.stop() # Blocking call interrupted by loop.stop()
try:
loop.run_forever() loop.run_forever()
finally:
loop.close() loop.close()
.. seealso:: .. seealso::
The :ref:`Hello World coroutine <asyncio-hello-world-coroutine>` example A similar :ref:`Hello World <asyncio-hello-world-coroutine>`
uses a :ref:`coroutine <coroutine>`. example created with a coroutine and the :func:`run` function.
.. _asyncio-date-callback: .. _asyncio-date-callback:
...@@ -1192,9 +1433,9 @@ loop:: ...@@ -1192,9 +1433,9 @@ loop::
Display the current date with call_later() Display the current date with call_later()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Example of callback displaying the current date every second. The callback uses An example of callback displaying the current date every second. The
the :meth:`AbstractEventLoop.call_later` method to reschedule itself during 5 callback uses the :meth:`loop.call_later` method to reschedule itself
seconds, and then stops the event loop:: during 5 seconds, and then stops the event loop::
import asyncio import asyncio
import datetime import datetime
...@@ -1213,14 +1454,15 @@ seconds, and then stops the event loop:: ...@@ -1213,14 +1454,15 @@ seconds, and then stops the event loop::
loop.call_soon(display_date, end_time, loop) loop.call_soon(display_date, end_time, loop)
# Blocking call interrupted by loop.stop() # Blocking call interrupted by loop.stop()
try:
loop.run_forever() loop.run_forever()
finally:
loop.close() loop.close()
.. seealso:: .. seealso::
The :ref:`coroutine displaying the current date A similar :ref:`current date <asyncio-date-coroutine>` example
<asyncio-date-coroutine>` example uses a :ref:`coroutine created with a coroutine and the :func:`run` function.
<coroutine>`.
.. _asyncio-watch-read-event: .. _asyncio-watch-read-event:
...@@ -1229,7 +1471,7 @@ Watch a file descriptor for read events ...@@ -1229,7 +1471,7 @@ Watch a file descriptor for read events
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Wait until a file descriptor received some data using the Wait until a file descriptor received some data using the
:meth:`AbstractEventLoop.add_reader` method and then close the event loop:: :meth:`loop.add_reader` method and then close the event loop::
import asyncio import asyncio
from socket import socketpair from socket import socketpair
...@@ -1241,8 +1483,10 @@ Wait until a file descriptor received some data using the ...@@ -1241,8 +1483,10 @@ Wait until a file descriptor received some data using the
def reader(): def reader():
data = rsock.recv(100) data = rsock.recv(100)
print("Received:", data.decode()) print("Received:", data.decode())
# We are done: unregister the file descriptor # We are done: unregister the file descriptor
loop.remove_reader(rsock) loop.remove_reader(rsock)
# Stop the event loop # Stop the event loop
loop.stop() loop.stop()
...@@ -1252,9 +1496,10 @@ Wait until a file descriptor received some data using the ...@@ -1252,9 +1496,10 @@ Wait until a file descriptor received some data using the
# Simulate the reception of data from the network # Simulate the reception of data from the network
loop.call_soon(wsock.send, 'abc'.encode()) loop.call_soon(wsock.send, 'abc'.encode())
try:
# Run the event loop # Run the event loop
loop.run_forever() loop.run_forever()
finally:
# We are done, close sockets and the event loop # We are done, close sockets and the event loop
rsock.close() rsock.close()
wsock.close() wsock.close()
...@@ -1262,20 +1507,22 @@ Wait until a file descriptor received some data using the ...@@ -1262,20 +1507,22 @@ Wait until a file descriptor received some data using the
.. seealso:: .. seealso::
The :ref:`register an open socket to wait for data using a protocol * A similar :ref:`example <asyncio-register-socket>`
<asyncio-register-socket>` example uses a low-level protocol created by the using transports, protocols, and the
:meth:`AbstractEventLoop.create_connection` method. :meth:`loop.create_connection` method.
The :ref:`register an open socket to wait for data using streams * Another similar :ref:`example <asyncio-register-socket-streams>`
<asyncio-register-socket-streams>` example uses high-level streams using the high-level :func:`asyncio.open_connection` function
created by the :func:`open_connection` function in a coroutine. and streams.
Set signal handlers for SIGINT and SIGTERM Set signal handlers for SIGINT and SIGTERM
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Register handlers for signals :py:data:`SIGINT` and :py:data:`SIGTERM` using (This example only works on UNIX.)
the :meth:`AbstractEventLoop.add_signal_handler` method::
Register handlers for signals :py:data:`SIGINT` and :py:data:`SIGTERM`
using the :meth:`loop.add_signal_handler` method::
import asyncio import asyncio
import functools import functools
...@@ -1286,16 +1533,17 @@ the :meth:`AbstractEventLoop.add_signal_handler` method:: ...@@ -1286,16 +1533,17 @@ the :meth:`AbstractEventLoop.add_signal_handler` method::
print("got signal %s: exit" % signame) print("got signal %s: exit" % signame)
loop.stop() loop.stop()
loop = asyncio.get_event_loop() async def main():
for signame in ('SIGINT', 'SIGTERM'): loop = asyncio.get_running_loop()
loop.add_signal_handler(getattr(signal, signame),
for signame in {'SIGINT', 'SIGTERM'}:
loop.add_signal_handler(
getattr(signal, signame),
functools.partial(ask_exit, signame)) functools.partial(ask_exit, signame))
print("Event loop running forever, press Ctrl+C to interrupt.") await asyncio.sleep(3600)
print("pid %s: send SIGINT or SIGTERM to exit." % os.getpid())
try: print("Event loop running for 1 hour, press Ctrl+C to interrupt.")
loop.run_forever() print(f"pid {os.getpid()}: send SIGINT or SIGTERM to exit.")
finally:
loop.close()
This example only works on UNIX. asyncio.run(main())
Doc/library/asyncio-eventloops.rst deleted 100644 → 0
View file @ 735171e3
.. currentmodule:: asyncio
Event loops
===========
**Source code:** :source:`Lib/asyncio/events.py`
Event loop functions
--------------------
The following functions are convenient shortcuts to accessing the methods of the
global policy. Note that this provides access to the default policy, unless an
alternative policy was set by calling :func:`set_event_loop_policy` earlier in
the execution of the process.
.. function:: get_event_loop()
Equivalent to calling ``get_event_loop_policy().get_event_loop()``.
.. function:: set_event_loop(loop)
Equivalent to calling ``get_event_loop_policy().set_event_loop(loop)``.
.. function:: new_event_loop()
Equivalent to calling ``get_event_loop_policy().new_event_loop()``.
.. function:: get_running_loop()
Return the running event loop in the current OS thread. If there
is no running event loop a :exc:`RuntimeError` is raised.
.. versionadded:: 3.7
.. _asyncio-event-loops:
Available event loops
---------------------
asyncio currently provides two implementations of event loops:
:class:`SelectorEventLoop` and :class:`ProactorEventLoop`.
.. class:: SelectorEventLoop
Event loop based on the :mod:`selectors` module. Subclass of
:class:`AbstractEventLoop`.
Use the most efficient selector available on the platform.
On Windows, only sockets are supported (ex: pipes are not supported):
see the `MSDN documentation of select
<https://msdn.microsoft.com/en-us/library/windows/desktop/ms740141%28v=vs.85%29.aspx>`_.
.. class:: ProactorEventLoop
Proactor event loop for Windows using "I/O Completion Ports" aka IOCP.
Subclass of :class:`AbstractEventLoop`.
Availability: Windows.
.. seealso::
`MSDN documentation on I/O Completion Ports
<https://msdn.microsoft.com/en-us/library/windows/desktop/aa365198%28v=vs.85%29.aspx>`_.
Example to use a :class:`ProactorEventLoop` on Windows::
import asyncio, sys
if sys.platform == 'win32':
loop = asyncio.ProactorEventLoop()
asyncio.set_event_loop(loop)
.. _asyncio-platform-support:
Platform support
----------------
The :mod:`asyncio` module has been designed to be portable, but each platform
still has subtle differences and may not support all :mod:`asyncio` features.
Windows
^^^^^^^
Common limits of Windows event loops:
- :meth:`~AbstractEventLoop.create_unix_connection` and
:meth:`~AbstractEventLoop.create_unix_server` are not supported: the socket
family :data:`socket.AF_UNIX` is specific to UNIX
- :meth:`~AbstractEventLoop.add_signal_handler` and
:meth:`~AbstractEventLoop.remove_signal_handler` are not supported
- :meth:`EventLoopPolicy.set_child_watcher` is not supported.
:class:`ProactorEventLoop` supports subprocesses. It has only one
implementation to watch child processes, there is no need to configure it.
:class:`SelectorEventLoop` specific limits:
- :class:`~selectors.SelectSelector` is used which only supports sockets
and is limited to 512 sockets.
- :meth:`~AbstractEventLoop.add_reader` and :meth:`~AbstractEventLoop.add_writer` only
accept file descriptors of sockets
- Pipes are not supported
(ex: :meth:`~AbstractEventLoop.connect_read_pipe`,
:meth:`~AbstractEventLoop.connect_write_pipe`)
- :ref:`Subprocesses <asyncio-subprocess>` are not supported
(ex: :meth:`~AbstractEventLoop.subprocess_exec`,
:meth:`~AbstractEventLoop.subprocess_shell`)
:class:`ProactorEventLoop` specific limits:
- :meth:`~AbstractEventLoop.create_datagram_endpoint` (UDP) is not supported
- :meth:`~AbstractEventLoop.add_reader` and :meth:`~AbstractEventLoop.add_writer` are
not supported
The resolution of the monotonic clock on Windows is usually around 15.6 msec.
The best resolution is 0.5 msec. The resolution depends on the hardware
(availability of `HPET
<https://en.wikipedia.org/wiki/High_Precision_Event_Timer>`_) and on the Windows
configuration. See :ref:`asyncio delayed calls <asyncio-delayed-calls>`.
.. versionchanged:: 3.5
:class:`ProactorEventLoop` now supports SSL.
Mac OS X
^^^^^^^^
Character devices like PTY are only well supported since Mavericks (Mac OS
10.9). They are not supported at all on Mac OS 10.5 and older.
On Mac OS 10.6, 10.7 and 10.8, the default event loop is
:class:`SelectorEventLoop` which uses :class:`selectors.KqueueSelector`.
:class:`selectors.KqueueSelector` does not support character devices on these
versions. The :class:`SelectorEventLoop` can be used with
:class:`~selectors.SelectSelector` or :class:`~selectors.PollSelector` to
support character devices on these versions of Mac OS X. Example::
import asyncio
import selectors
selector = selectors.SelectSelector()
loop = asyncio.SelectorEventLoop(selector)
asyncio.set_event_loop(loop)
Event loop policies and the default policy
------------------------------------------
Event loop management is abstracted with a *policy* pattern, to provide maximal
flexibility for custom platforms and frameworks. Throughout the execution of a
process, a single global policy object manages the event loops available to the
process based on the calling context. A policy is an object implementing the
:class:`AbstractEventLoopPolicy` interface.
For most users of :mod:`asyncio`, policies never have to be dealt with
explicitly, since the default global policy is sufficient (see below).
The module-level functions
:func:`get_event_loop` and :func:`set_event_loop` provide convenient access to
event loops managed by the default policy.
Event loop policy interface
---------------------------
An event loop policy must implement the following interface:
.. class:: AbstractEventLoopPolicy
Event loop policy.
.. method:: get_event_loop()
Get the event loop for the current context.
Returns an event loop object implementing the :class:`AbstractEventLoop`
interface. In case called from coroutine, it returns the currently
running event loop.
Raises an exception in case no event loop has been set for the current
context and the current policy does not specify to create one. It must
never return ``None``.
.. versionchanged:: 3.6
.. method:: set_event_loop(loop)
Set the event loop for the current context to *loop*.
.. method:: new_event_loop()
Create and return a new event loop object according to this policy's
rules.
If there's need to set this loop as the event loop for the current
context, :meth:`set_event_loop` must be called explicitly.
The default policy defines context as the current thread, and manages an event
loop per thread that interacts with :mod:`asyncio`. An exception to this rule
happens when :meth:`~AbstractEventLoopPolicy.get_event_loop` is called from a
running future/coroutine, in which case it will return the current loop
running that future/coroutine.
If the current thread doesn't already have an event loop associated with it,
the default policy's :meth:`~AbstractEventLoopPolicy.get_event_loop` method
creates one when called from the main thread, but raises :exc:`RuntimeError`
otherwise.
Access to the global loop policy
--------------------------------
.. function:: get_event_loop_policy()
Get the current event loop policy.
.. function:: set_event_loop_policy(policy)
Set the current event loop policy. If *policy* is ``None``, the default
policy is restored.
Customizing the event loop policy
---------------------------------
To implement a new event loop policy, it is recommended you subclass the
concrete default event loop policy :class:`DefaultEventLoopPolicy`
and override the methods for which you want to change behavior, for example::
class MyEventLoopPolicy(asyncio.DefaultEventLoopPolicy):
def get_event_loop(self):
"""Get the event loop.
This may be None or an instance of EventLoop.
"""
loop = super().get_event_loop()
# Do something with loop ...
return loop
asyncio.set_event_loop_policy(MyEventLoopPolicy())
Doc/library/asyncio-exceptions.rst 0 → 100644
View file @ 7c7605ff
.. currentmodule:: asyncio
==========
Exceptions
==========
.. exception:: TimeoutError
The operation has exceeded the given deadline.
.. note::
This exception is different from the builtin :exc:`TimeoutError`
exception.
.. exception:: CancelledError
The operation has been cancelled.
This exception can be caught to perform custom operations on
when asyncio Tasks are cancelled. In almost all situations the
exception must always be re-raised.
.. note::
This exception is a subclass of :exc:`Exception`, so it can be
accidentally suppressed by ``try..except`` block::
try:
await operation
except Exception:
# The cancellation is broken because the *except* block
# suppresses the CancelledError exception.
log.log('an error has occurred')
Instead, the following pattern should be used::
try:
await operation
except asyncio.CancelledError:
raise
except Exception:
log.log('an error has occurred')
.. exception:: InvalidStateError
Invalid internal state of :class:`Task` or :class:`Future`.
Can be raised in situations like setting a result value for a
*Future* object that already has a result value set.
.. exception:: SendfileNotAvailableError
The "sendfile" syscall for is not available for the given
socket or file type.
A subclass of :exc:`RuntimeError`.
.. exception:: IncompleteReadError
Incomplete read error.
Raised by :ref:`asyncio streams <asyncio-streams>` APIs.
This exception is a subclass of :exc:`EOFError`.
.. attribute:: expected
Total number (:class:`int`) of expected bytes.
.. attribute:: partial
Read :class:`bytes` string before the end of stream was reached.
.. exception:: LimitOverrunError
Reached the buffer limit while looking for a separator.
Raised by :ref:`asyncio streams <asyncio-streams>` APIs.
.. attribute:: consumed
Total number of to be consumed bytes.
Doc/library/asyncio-platforms.rst 0 → 100644
View file @ 7c7605ff
.. currentmodule:: asyncio
.. _asyncio-platform-support:
=================
Platforms Support
=================
The :mod:`asyncio` module has been designed to be portable,
but some platforms have subtle differences and limitations.
All Platforms
=============
* :meth:`loop.add_reader` and :meth:`loop.add_writer`
cannot be used to monitor file IO.
Windows
=======
All event loops on Windows do not support the following methods:
* :meth:`loop.create_unix_connection` and
:meth:`loop.create_unix_server` are not supported.
The :data:`socket.AF_UNIX` socket family is specific to UNIX/
* :meth:`loop.add_signal_handler` and
:meth:`loop.remove_signal_handler` are not supported.
:class:`SelectorEventLoop` has the following limitations:
* :class:`~selectors.SelectSelector` is used to wait on socket events:
it supports sockets and is limited to 512 sockets.
* :meth:`loop.add_reader` and :meth:`loop.add_writer` only accept
socket handles (e.g. pipe file descriptors are not supported).
* Pipes are not supported, so the :meth:`loop.connect_read_pipe`
and :meth:`loop.connect_write_pipe` methods are not implemented.
* :ref:`Subprocesses <asyncio-subprocess>` are not supported, i.e.
:meth:`loop.subprocess_exec` and :meth:`loop.subprocess_shell`
methods are not implemented.
:class:`ProactorEventLoop` has the following limitations:
* The :meth:`loop.create_datagram_endpoint` method
is not supported.
* The :meth:`loop.add_reader` and :meth:`loop.add_writer`
methods are not supported.
The resolution of the monotonic clock on Windows is usually around 15.6
msec. The best resolution is 0.5 msec. The resolution depends on the
hardware (availability of `HPET
<https://en.wikipedia.org/wiki/High_Precision_Event_Timer>`_) and on the
Windows configuration.
.. _asyncio-windows-subprocess:
Subprocess Support on Windows
-----------------------------
:class:`SelectorEventLoop` on Windows does not support subproceses,
so :class:`ProactorEventLoop` should be used instead::
import asyncio
asyncio.set_event_loop_policy(
asyncio.WindowsProactorEventLoopPolicy())
asyncio.run(your_code())
The :meth:`policy.set_child_watcher()
<AbstractEventLoopPolicy.set_child_watcher>` function is also
not supported, as :class:`ProactorEventLoop` has a different mechanism
to watch child processes.
macOS
=====
Modern macOS versions are fully supported.
.. rubric:: macOS <= 10.8
On macOS 10.6, 10.7 and 10.8, the default event loop
uses :class:`selectors.KqueueSelector`, which does not support
character devices on these versions. The :class:`SelectorEventLoop`
can be manually configured to use :class:`~selectors.SelectSelector`
or :class:`~selectors.PollSelector` to support character devices on
these older versions of macOS. Example::
import asyncio
import selectors
selector = selectors.SelectSelector()
loop = asyncio.SelectorEventLoop(selector)
asyncio.set_event_loop(loop)
Doc/library/asyncio-policy.rst 0 → 100644
View file @ 7c7605ff
.. currentmodule:: asyncio
.. _asyncio-policies:
========
Policies
========
An event loop policy, a global per-process object, controls
management of the event loop. Each event loop has a default
policy, which can be changed and customized using the API.
A policy defines the notion of context and manages a
separate event loop per context. The default policy
defines context to be the current thread.
By using a custom event loop policy, the behavior of
:func:`get_event_loop`, :func:`set_event_loop`, and
:func:`new_event_loop` functions can be customized.
Policy objects should implement the APIs defined
in the abstract base class :class:`AbstractEventLoopPolicy`.
Access the Policy
=================
The following functions can be used to get and set the policy
for the current process:
.. function:: get_event_loop_policy()
Return the current process-wide policy.
.. function:: set_event_loop_policy(policy)
Set the current process-wide policy to *policy*.
If *policy* is set to ``None``, the default policy is restored.
Policy Objects
==============
The abstract event loop policy base class is defined as follows:
.. class:: AbstractEventLoopPolicy
An abstract base class for asyncio policies.
.. method:: get_event_loop()
Get the event loop for the current context.
Return an event loop object implementing the
:class:`AbstractEventLoop` interface.
This method should never return ``None``.
.. versionchanged:: 3.6
.. method:: set_event_loop(loop)
Set the event loop for the current context to *loop*.
.. method:: new_event_loop()
Create and return a new event loop object.
This method should never return ``None``.
.. method:: get_child_watcher()
Get a child process watcher object.
Return a watcher object implementing the
:class:`AbstractChildWatcher` interface.
This function is Unix specific.
.. method:: set_child_watcher(watcher)
Get the current child process watcher to *watcher*.
This function is Unix specific.
asyncio ships with the following built-in policies:
.. class:: DefaultEventLoopPolicy
The default asyncio policy. Uses :class:`SelectorEventLoop`
on both Unix and Windows platforms.
There is no need to install the default policy manually; asyncio
is configured to use it automatically.
.. class:: WindowsProactorEventLoopPolicy
An alternative event loop policy that uses the
:class:`ProactorEventLoop` event loop implementation.
Availability: Windows.
Process Watchers
================
A process watcher allows customization of how an event loop monitors
child processes on Unix. Specifically, the event loop needs to know
when a child process has finished its execution.
In asyncio, child processes are created with
:func:`create_subprocess_exec` and :meth:`loop.subprocess_exec`
functions.
asyncio defines an abstract base class :class:`AbstractChildWatcher`
that child watchers should implement, and has two different
implementations: :class:`SafeChildWatcher` (configured to be used
by default) and :class:`FastChildWatcher`.
See also the :ref:`Subprocess and Threads <asyncio-subprocess-threads>`
section.
The following two functions can be used to customize the watcher
implementation used by the asyncio event loop:
.. function:: get_child_watcher()
Return the current child watcher for the current policy.
.. function:: set_child_watcher(watcher)
Set the current child watcher to *watcher* for the current
policy. *watcher* must implement methods defined in the
:class:`AbstractChildWatcher` base class.
.. note::
Third-party event loops implementations might not support
custom child watchers. For such event loops, using
:func:`set_child_watcher` might have no effect or even can
be prohibited.
.. class:: AbstractChildWatcher
.. method:: add_child_handler(pid, callback, \*args)
Register a new child handler.
Arrange for ``callback(pid, returncode, *args)`` to be called
when a process with PID equal to *pid* terminates. Specifying
another callback for the same process replaces the previous
handler.
*callback* callable must be thread-safe.
.. method:: remove_child_handler(pid)
Removes the handler for process with PID equal to *pid*.
The function returns ``True`` if the handler was successfully
removed, ``False`` if there was nothing to remove.
.. method:: attach_loop(loop)
Attach the watcher to an event loop.
If the watcher was previously attached to an event loop, then
it is first detached before attaching to the new loop.
Note: loop may be ``None``.
.. method:: close()
Close the watcher.
This method has to be called to ensure that underlying
resources are cleaned-up.
.. class:: SafeChildWatcher
This implementation avoids disrupting other code spawning processes
by polling every process explicitly on a :py:data:`SIGCHLD` signal.
This is a safe solution but it has a significant overhead when
handling a big number of processes (*O(n)* each time a
:py:data:`SIGCHLD` is received).
asyncio uses this implementation by default.
.. class:: FastChildWatcher
This implementation reaps every terminated processes by calling
``os.waitpid(-1)`` directly, possibly breaking other code spawning
processes and waiting for their termination.
There is no noticeable overhead when handling a big number of
children (*O(1)* each time a child terminates).
Custom Policies
===============
To implement a new event loop policy, it is recommended to subclass
:class:`DefaultEventLoopPolicy` and override the methods for which
custom behavior is wanted, e.g.::
class MyEventLoopPolicy(asyncio.DefaultEventLoopPolicy):
def get_event_loop(self):
"""Get the event loop.
This may be None or an instance of EventLoop.
"""
loop = super().get_event_loop()
# Do something with loop ...
return loop
asyncio.set_event_loop_policy(MyEventLoopPolicy())
Doc/library/asyncio-protocol.rst
View file @ 7c7605ff
.. currentmodule:: asyncio .. currentmodule:: asyncio
+++++++++++++++++++++++++++++++++++++++++++++
Transports and protocols (callback based API)
+++++++++++++++++++++++++++++++++++++++++++++
**Source code:** :source:`Lib/asyncio/transports.py` ========================
Transports and Protocols
========================
.. rubric:: Preface
Transports and Protocols are used by **low-level** event loop
APIs such as :meth:`loop.create_connection`. They require using
callback-based programming style and enable high-performance
implementations of network or IPC protocols (e.g. HTTP).
Essentially, transports and protocols should only be used in
libraries and frameworks and never in high-level asyncio
applications.
This documentation page covers both `Transports`_ and `Protocols`_.
.. rubric:: Introduction
At the highest level, the transport is concerned with *how* bytes
are transmitted, while the protocol determines *which* bytes to
transmit (and to some extent when).
A different way of saying the same thing: a transport is an
abstraction for a socket (or similar I/O endpoint) while a protocol
is an abstraction for an application, from the transport's point
of view.
Yet another view is simply that the transport and protocol interfaces
together define an abstract interface for using network I/O and
interprocess I/O.
There is always a 1:1 relationship between transport and protocol
objects: the protocol calls transport methods to send data,
while the transport calls protocol methods to pass it data that
has been received.
Most of connection oriented event loop methods
(such as :meth:`loop.create_connection`) usually accept a
*protocol_factory* argument used to create a *Protocol* object
for an accepted connection, represented by a *Transport* object.
Such methods usually return a tuple of ``(transport, protocol)``.
.. rubric:: Contents
This documentation page contains the following sections:
* The `Transports`_ section documents asyncio :class:`BaseTransport`,
:class:`ReadTransport`, :class:`WriteTransport`, :class:`Transport`,
:class:`DatagramTransport`, and :class:`SubprocessTransport`
classes.
* The `Protocols`_ section documents asyncio :class:`BaseProtocol`,
:class:`Protocol`, :class:`BufferedProtocol`,
:class:`DatagramProtocol`, and :class:`SubprocessProtocol` classes.
* The `Examples`_ section showcases how to work with transports,
protocols, and low-level event loop APIs.
**Source code:** :source:`Lib/asyncio/protocols.py`
.. _asyncio-transport: .. _asyncio-transport:
...@@ -14,77 +67,145 @@ Transports ...@@ -14,77 +67,145 @@ Transports
========== ==========
Transports are classes provided by :mod:`asyncio` in order to abstract Transports are classes provided by :mod:`asyncio` in order to abstract
various kinds of communication channels. You generally won't instantiate various kinds of communication channels.
a transport yourself; instead, you will call an :class:`AbstractEventLoop` method
which will create the transport and try to initiate the underlying
communication channel, calling you back when it succeeds.
Once the communication channel is established, a transport is always Transport objects are always instantiated by an
paired with a :ref:`protocol <asyncio-protocol>` instance. The protocol can ref:`asyncio event loop <asyncio-event-loop>`.
then call the transport's methods for various purposes.
:mod:`asyncio` currently implements transports for TCP, UDP, SSL, and asyncio implements transports for TCP, UDP, SSL, and subprocess pipes.
subprocess pipes. The methods available on a transport depend on The methods available on a transport depend on the transport's kind.
the transport's kind.
The transport classes are :ref:`not thread safe <asyncio-multithreading>`. The transport classes are :ref:`not thread safe <asyncio-multithreading>`.
.. versionchanged:: 3.6
The socket option ``TCP_NODELAY`` is now set by default.
Transports Hierarchy
BaseTransport --------------------
-------------
.. class:: BaseTransport .. class:: BaseTransport
Base class for transports. Base class for all transports. Contains methods that all
asyncio transports share.
.. class:: WriteTransport(BaseTransport)
A base transport for write-only connections.
Instances of the *WriteTransport* class are returned from
the :meth:`loop.connect_write_pipe` event loop method and
are also used by subprocess-related methods like
:meth:`loop.subprocess_exec`.
.. class:: ReadTransport(BaseTransport)
A base transport for read-only connections.
Instances of the *ReadTransport* class are returned from
the :meth:`loop.connect_read_pipe` event loop method and
are also used by subprocess-related methods like
:meth:`loop.subprocess_exec`.
.. class:: Transport(WriteTransport, ReadTransport)
Interface representing a bidirectional transport, such as a
TCP connection.
The user never instantiates a transport directly; they call a
utility function, passing it a protocol factory and other
information necessary to create the transport and protocol.
Instances of the *Transport* class are returned from or used by
event loop methods like :meth:`loop.create_connection`,
:meth:`loop.create_unix_connection`,
:meth:`loop.create_server`, :meth:`loop.sendfile`, etc.
.. class:: DatagramTransport(BaseTransport)
.. method:: close() A transport for datagram (UDP) connections.
Close the transport. If the transport has a buffer for outgoing Instances of the *DatagramTransport* class are returned from
the :meth:`loop.create_datagram_endpoint` event loop method.
.. class:: SubprocessTransport(BaseTransport)
An abstraction to represent a connection between a parent and its
child OS process.
Instances of the *SubprocessTransport* class are returned from
event loop methods :meth:`loop.subprocess_shell` and
:meth:`loop.subprocess_exec`.
Base Transport
--------------
.. method:: BaseTransport.close()
Close the transport.
If the transport has a buffer for outgoing
data, buffered data will be flushed asynchronously. No more data data, buffered data will be flushed asynchronously. No more data
will be received. After all buffered data is flushed, the will be received. After all buffered data is flushed, the
protocol's :meth:`connection_lost` method will be called with protocol's :meth:`protocol.connection_lost()
<BaseProtocol.connection_lost>` method will be called with
:const:`None` as its argument. :const:`None` as its argument.
.. method:: is_closing() .. method:: BaseTransport.is_closing()
Return ``True`` if the transport is closing or is closed. Return ``True`` if the transport is closing or is closed.
.. versionadded:: 3.5.1 .. method:: BaseTransport.get_extra_info(name, default=None)
Return information about the transport or underlying resources
it uses.
*name* is a string representing the piece of transport-specific
information to get.
.. method:: get_extra_info(name, default=None) *default* is the value to return if the information is not
available, or if the transport does not support querying it
with the given third-party event loop implementation or on the
current platform.
Return optional transport information. *name* is a string representing For example, the following code attempts to get the underlying
the piece of transport-specific information to get, *default* is the socket object of the transport::
value to return if the information doesn't exist.
This method allows transport implementations to easily expose sock = transport.get_extra_info('socket')
channel-specific information. if sock is not None:
print(sock.getsockopt(...))
Categories of information that can be queried on some transports:
* socket: * socket:
- ``'peername'``: the remote address to which the socket is connected, - ``'peername'``: the remote address to which the socket is
result of :meth:`socket.socket.getpeername` (``None`` on error) connected, result of :meth:`socket.socket.getpeername`
(``None`` on error)
- ``'socket'``: :class:`socket.socket` instance - ``'socket'``: :class:`socket.socket` instance
- ``'sockname'``: the socket's own address, - ``'sockname'``: the socket's own address,
result of :meth:`socket.socket.getsockname` result of :meth:`socket.socket.getsockname`
* SSL socket: * SSL socket:
- ``'compression'``: the compression algorithm being used as a string, - ``'compression'``: the compression algorithm being used as a
or ``None`` if the connection isn't compressed; result of string, or ``None`` if the connection isn't compressed; result
:meth:`ssl.SSLSocket.compression` of :meth:`ssl.SSLSocket.compression`
- ``'cipher'``: a three-value tuple containing the name of the cipher
being used, the version of the SSL protocol that defines its use, and - ``'cipher'``: a three-value tuple containing the name of the
the number of secret bits being used; result of cipher being used, the version of the SSL protocol that defines
its use, and the number of secret bits being used; result of
:meth:`ssl.SSLSocket.cipher` :meth:`ssl.SSLSocket.cipher`
- ``'peercert'``: peer certificate; result of - ``'peercert'``: peer certificate; result of
:meth:`ssl.SSLSocket.getpeercert` :meth:`ssl.SSLSocket.getpeercert`
- ``'sslcontext'``: :class:`ssl.SSLContext` instance - ``'sslcontext'``: :class:`ssl.SSLContext` instance
- ``'ssl_object'``: :class:`ssl.SSLObject` or :class:`ssl.SSLSocket`
instance - ``'ssl_object'``: :class:`ssl.SSLObject` or
:class:`ssl.SSLSocket` instance
* pipe: * pipe:
...@@ -94,49 +215,41 @@ BaseTransport ...@@ -94,49 +215,41 @@ BaseTransport
- ``'subprocess'``: :class:`subprocess.Popen` instance - ``'subprocess'``: :class:`subprocess.Popen` instance
.. method:: set_protocol(protocol) .. method:: BaseTransport.set_protocol(protocol)
Set a new protocol. Switching protocol should only be done when both Set a new protocol.
protocols are documented to support the switch.
.. versionadded:: 3.5.3 Switching protocol should only be done when both
protocols are documented to support the switch.
.. method:: get_protocol .. method:: BaseTransport.get_protocol()
Return the current protocol. Return the current protocol.
.. versionadded:: 3.5.3
.. versionchanged:: 3.5.1
``'ssl_object'`` info was added to SSL sockets.
Read-only Transports
--------------------
ReadTransport .. method:: ReadTransport.is_reading()
-------------
.. class:: ReadTransport
Interface for read-only transports.
.. method:: is_reading()
Return ``True`` if the transport is receiving new data. Return ``True`` if the transport is receiving new data.
.. versionadded:: 3.7 .. versionadded:: 3.7
.. method:: pause_reading() .. method:: ReadTransport.pause_reading()
Pause the receiving end of the transport. No data will be passed to Pause the receiving end of the transport. No data will be passed to
the protocol's :meth:`data_received` method until :meth:`resume_reading` the protocol's :meth:`protocol.data_received() <Protocol.data_received>`
is called. method until :meth:`resume_reading` is called.
.. versionchanged:: 3.7 .. versionchanged:: 3.7
The method is idempotent, i.e. it can be called when the The method is idempotent, i.e. it can be called when the
transport is already paused or closed. transport is already paused or closed.
.. method:: resume_reading() .. method:: ReadTransport.resume_reading()
Resume the receiving end. The protocol's :meth:`data_received` method Resume the receiving end. The protocol's
:meth:`protocol.data_received() <Protocol.data_received>` method
will be called once again if some data is available for reading. will be called once again if some data is available for reading.
.. versionchanged:: 3.7 .. versionchanged:: 3.7
...@@ -144,30 +257,27 @@ ReadTransport ...@@ -144,30 +257,27 @@ ReadTransport
transport is already reading. transport is already reading.
WriteTransport Write-only Transports
-------------- ---------------------
.. class:: WriteTransport
Interface for write-only transports.
.. method:: abort() .. method:: WriteTransport.abort()
Close the transport immediately, without waiting for pending operations Close the transport immediately, without waiting for pending operations
to complete. Buffered data will be lost. No more data will be received. to complete. Buffered data will be lost. No more data will be received.
The protocol's :meth:`connection_lost` method will eventually be The protocol's :meth:`protocol.connection_lost()
<BaseProtocol.connection_lost>` method will eventually be
called with :const:`None` as its argument. called with :const:`None` as its argument.
.. method:: can_write_eof() .. method:: WriteTransport.can_write_eof()
Return :const:`True` if the transport supports :meth:`write_eof`, Return :const:`True` if the transport supports
:const:`False` if not. :meth:`~WriteTransport.write_eof`, :const:`False` if not.
.. method:: get_write_buffer_size() .. method:: WriteTransport.get_write_buffer_size()
Return the current size of the output buffer used by the transport. Return the current size of the output buffer used by the transport.
.. method:: get_write_buffer_limits() .. method:: WriteTransport.get_write_buffer_limits()
Get the *high*- and *low*-water limits for write flow control. Return a Get the *high*- and *low*-water limits for write flow control. Return a
tuple ``(low, high)`` where *low* and *high* are positive number of tuple ``(low, high)`` where *low* and *high* are positive number of
...@@ -177,48 +287,52 @@ WriteTransport ...@@ -177,48 +287,52 @@ WriteTransport
.. versionadded:: 3.4.2 .. versionadded:: 3.4.2
.. method:: set_write_buffer_limits(high=None, low=None) .. method:: WriteTransport.set_write_buffer_limits(high=None, low=None)
Set the *high*- and *low*-water limits for write flow control. Set the *high*- and *low*-water limits for write flow control.
These two values (measured in number of These two values (measured in number of
bytes) control when the protocol's bytes) control when the protocol's
:meth:`pause_writing` and :meth:`resume_writing` methods are called. :meth:`protocol.pause_writing() <BaseProtocol.pause_writing>`
If specified, the low-water limit must be less than or equal to the and :meth:`protocol.resume_writing() <BaseProtocol.resume_writing>`
high-water limit. Neither *high* nor *low* can be negative. methods are called. If specified, the low-water limit must be less
than or equal to the high-water limit. Neither *high* nor *low*
can be negative.
:meth:`pause_writing` is called when the buffer size becomes greater :meth:`~BaseProtocol.pause_writing` is called when the buffer size
than or equal to the *high* value. If writing has been paused, becomes greater than or equal to the *high* value. If writing has
:meth:`resume_writing` is called when the buffer size becomes less been paused, :meth:`~BaseProtocol.resume_writing` is called when
than or equal to the *low* value. the buffer size becomes less than or equal to the *low* value.
The defaults are implementation-specific. If only the The defaults are implementation-specific. If only the
high-water limit is given, the low-water limit defaults to an high-water limit is given, the low-water limit defaults to an
implementation-specific value less than or equal to the implementation-specific value less than or equal to the
high-water limit. Setting *high* to zero forces *low* to zero as high-water limit. Setting *high* to zero forces *low* to zero as
well, and causes :meth:`pause_writing` to be called whenever the well, and causes :meth:`~BaseProtocol.pause_writing` to be called
buffer becomes non-empty. Setting *low* to zero causes whenever the buffer becomes non-empty. Setting *low* to zero causes
:meth:`resume_writing` to be called only once the buffer is empty. :meth:`~BaseProtocol.resume_writing` to be called only once the
Use of zero for either limit is generally sub-optimal as it buffer is empty. Use of zero for either limit is generally
reduces opportunities for doing I/O and computation sub-optimal as it reduces opportunities for doing I/O and
concurrently. computation concurrently.
Use :meth:`get_write_buffer_limits` to get the limits. Use :meth:`~WriteTransport.get_write_buffer_limits`
to get the limits.
.. method:: write(data) .. method:: WriteTransport.write(data)
Write some *data* bytes to the transport. Write some *data* bytes to the transport.
This method does not block; it buffers the data and arranges for it This method does not block; it buffers the data and arranges for it
to be sent out asynchronously. to be sent out asynchronously.
.. method:: writelines(list_of_data) .. method:: WriteTransport.writelines(list_of_data)
Write a list (or any iterable) of data bytes to the transport. Write a list (or any iterable) of data bytes to the transport.
This is functionally equivalent to calling :meth:`write` on each This is functionally equivalent to calling :meth:`write` on each
element yielded by the iterable, but may be implemented more efficiently. element yielded by the iterable, but may be implemented more
efficiently.
.. method:: write_eof() .. method:: WriteTransport.write_eof()
Close the write end of the transport after flushing buffered data. Close the write end of the transport after flushing buffered data.
Data may still be received. Data may still be received.
...@@ -227,36 +341,38 @@ WriteTransport ...@@ -227,36 +341,38 @@ WriteTransport
(e.g. SSL) doesn't support half-closes. (e.g. SSL) doesn't support half-closes.
DatagramTransport Datagram Transports
----------------- -------------------
.. method:: DatagramTransport.sendto(data, addr=None) .. method:: DatagramTransport.sendto(data, addr=None)
Send the *data* bytes to the remote peer given by *addr* (a Send the *data* bytes to the remote peer given by *addr* (a
transport-dependent target address). If *addr* is :const:`None`, the transport-dependent target address). If *addr* is :const:`None`,
data is sent to the target address given on transport creation. the data is sent to the target address given on transport
creation.
This method does not block; it buffers the data and arranges for it This method does not block; it buffers the data and arranges
to be sent out asynchronously. for it to be sent out asynchronously.
.. method:: DatagramTransport.abort() .. method:: DatagramTransport.abort()
Close the transport immediately, without waiting for pending operations Close the transport immediately, without waiting for pending
to complete. Buffered data will be lost. No more data will be received. operations to complete. Buffered data will be lost.
The protocol's :meth:`connection_lost` method will eventually be No more data will be received. The protocol's
called with :const:`None` as its argument. :meth:`protocol.connection_lost() <BaseProtocol.connection_lost>`
method will eventually be called with :const:`None` as its argument.
BaseSubprocessTransport .. _asyncio-subprocess-transports:
-----------------------
.. class:: BaseSubprocessTransport Subprocess Transports
---------------------
.. method:: get_pid() .. method:: SubprocessTransport.get_pid()
Return the subprocess process id as an integer. Return the subprocess process id as an integer.
.. method:: get_pipe_transport(fd) .. method:: SubprocessTransport.get_pipe_transport(fd)
Return the transport for the communication pipe corresponding to the Return the transport for the communication pipe corresponding to the
integer file descriptor *fd*: integer file descriptor *fd*:
...@@ -269,25 +385,25 @@ BaseSubprocessTransport ...@@ -269,25 +385,25 @@ BaseSubprocessTransport
or :const:`None` if the subprocess was not created with ``stderr=PIPE`` or :const:`None` if the subprocess was not created with ``stderr=PIPE``
* other *fd*: :const:`None` * other *fd*: :const:`None`
.. method:: get_returncode() .. method:: SubprocessTransport.get_returncode()
Return the subprocess returncode as an integer or :const:`None` Return the subprocess return code as an integer or :const:`None`
if it hasn't returned, similarly to the if it hasn't returned, similarly to the
:attr:`subprocess.Popen.returncode` attribute. :attr:`subprocess.Popen.returncode` attribute.
.. method:: kill() .. method:: SubprocessTransport.kill()
Kill the subprocess, as in :meth:`subprocess.Popen.kill`. Kill the subprocess, as in :meth:`subprocess.Popen.kill`.
On POSIX systems, the function sends SIGKILL to the subprocess. On POSIX systems, the function sends SIGKILL to the subprocess.
On Windows, this method is an alias for :meth:`terminate`. On Windows, this method is an alias for :meth:`terminate`.
.. method:: send_signal(signal) .. method:: SubprocessTransport.send_signal(signal)
Send the *signal* number to the subprocess, as in Send the *signal* number to the subprocess, as in
:meth:`subprocess.Popen.send_signal`. :meth:`subprocess.Popen.send_signal`.
.. method:: terminate() .. method:: SubprocessTransport.terminate()
Ask the subprocess to stop, as in :meth:`subprocess.Popen.terminate`. Ask the subprocess to stop, as in :meth:`subprocess.Popen.terminate`.
This method is an alias for the :meth:`close` method. This method is an alias for the :meth:`close` method.
...@@ -296,11 +412,11 @@ BaseSubprocessTransport ...@@ -296,11 +412,11 @@ BaseSubprocessTransport
On Windows, the Windows API function TerminateProcess() is called to On Windows, the Windows API function TerminateProcess() is called to
stop the subprocess. stop the subprocess.
.. method:: close() .. method:: SubprocessTransport.close()
Ask the subprocess to stop by calling the :meth:`terminate` method if the Ask the subprocess to stop by calling the :meth:`terminate` method
subprocess hasn't returned yet, and close transports of all pipes if the subprocess hasn't returned yet, and close transports of all
(*stdin*, *stdout* and *stderr*). pipes (*stdin*, *stdout* and *stderr*).
.. _asyncio-protocol: .. _asyncio-protocol:
...@@ -308,65 +424,62 @@ BaseSubprocessTransport ...@@ -308,65 +424,62 @@ BaseSubprocessTransport
Protocols Protocols
========= =========
:mod:`asyncio` provides base classes that you can subclass to implement asyncio provides a set of abstract base classes that should be used
your network protocols. Those classes are used in conjunction with to implement network protocols. Those classes are meant to be used
:ref:`transports <asyncio-transport>` (see below): the protocol parses incoming together with :ref:`transports <asyncio-transport>`.
data and asks for the writing of outgoing data, while the transport is
responsible for the actual I/O and buffering.
When subclassing a protocol class, it is recommended you override certain Subclasses of abstract base protocol classes can implement some or
methods. Those methods are callbacks: they will be called by the transport all methods. All those methods are callbacks: they are called by
on certain events (for example when some data is received); you shouldn't transports on certain events, for example when some data is received.
call them yourself, unless you are implementing a transport. Base protocol methods are not supposed to be called by anything but
the corresponding transport.
.. note::
All callbacks have default implementations, which are empty. Therefore,
you only need to implement the callbacks for the events in which you
are interested.
Base Protocols
--------------
.. class:: BaseProtocol
Protocol classes Base protocol with methods that all protocols share.
----------------
.. class:: Protocol .. class:: Protocol(BaseProtocol)
The base class for implementing streaming protocols (for use with The base class for implementing streaming protocols
e.g. TCP and SSL transports). (TCP, Unix sockets, etc).
.. class:: BufferedProtocol .. class:: BufferedProtocol(BaseProtocol)
A base class for implementing streaming protocols with manual A base class for implementing streaming protocols with manual
control of the receive buffer. control of the receive buffer.
.. versionadded:: 3.7 .. class:: DatagramProtocol(BaseProtocol)
**Important:** this has been added to asyncio in Python 3.7
*on a provisional basis*! Treat it as an experimental API that
might be changed or removed in Python 3.8.
.. class:: DatagramProtocol The base class for implementing datagram (UDP) protocols.
The base class for implementing datagram protocols (for use with .. class:: SubprocessProtocol(BaseProtocol)
e.g. UDP transports).
.. class:: SubprocessProtocol
The base class for implementing protocols communicating with child The base class for implementing protocols communicating with child
processes (through a set of unidirectional pipes). processes (unidirectional pipes).
Connection callbacks Base Protocol
-------------------- -------------
These callbacks may be called on :class:`Protocol`, :class:`DatagramProtocol` All asyncio protocols can implement Base Protocol callbacks.
and :class:`SubprocessProtocol` instances:
.. rubric:: Connection Callbacks
Connection callbacks are called on all protocols, exactly once per
a successful connection. All other protocol callbacks can only be
called between those two methods.
.. method:: BaseProtocol.connection_made(transport) .. method:: BaseProtocol.connection_made(transport)
Called when a connection is made. Called when a connection is made.
The *transport* argument is the transport representing the The *transport* argument is the transport representing the
connection. You are responsible for storing it somewhere connection. The protocol is responsible for storing the reference
(e.g. as an attribute) if you need to. to its transport.
.. method:: BaseProtocol.connection_lost(exc) .. method:: BaseProtocol.connection_lost(exc)
...@@ -376,65 +489,77 @@ and :class:`SubprocessProtocol` instances: ...@@ -376,65 +489,77 @@ and :class:`SubprocessProtocol` instances:
The latter means a regular EOF is received, or the connection was The latter means a regular EOF is received, or the connection was
aborted or closed by this side of the connection. aborted or closed by this side of the connection.
:meth:`~BaseProtocol.connection_made` and :meth:`~BaseProtocol.connection_lost`
are called exactly once per successful connection. All other callbacks will be
called between those two methods, which allows for easier resource management
in your protocol implementation.
The following callbacks may be called only on :class:`SubprocessProtocol` .. rubric:: Flow Control Callbacks
instances:
.. method:: SubprocessProtocol.pipe_data_received(fd, data) Flow control callbacks can be called by transports to pause or
resume writing performed by the protocol.
Called when the child process writes data into its stdout or stderr pipe. See the documentation of the :meth:`~WriteTransport.set_write_buffer_limits`
*fd* is the integer file descriptor of the pipe. *data* is a non-empty method for more details.
bytes object containing the data.
.. method:: SubprocessProtocol.pipe_connection_lost(fd, exc) .. method:: BaseProtocol.pause_writing()
Called when one of the pipes communicating with the child process Called when the transport's buffer goes over the high-water mark.
is closed. *fd* is the integer file descriptor that was closed.
.. method:: SubprocessProtocol.process_exited() .. method:: BaseProtocol.resume_writing()
Called when the child process has exited. Called when the transport's buffer drains below the low-water mark.
If the buffer size equals the high-water mark,
:meth:`~BaseProtocol.pause_writing` is not called: the buffer size must
go strictly over.
Streaming protocols Conversely, :meth:`~BaseProtocol.resume_writing` is called when the
buffer size is equal or lower than the low-water mark. These end
conditions are important to ensure that things go as expected when
either mark is zero.
Streaming Protocols
------------------- -------------------
The following callbacks are called on :class:`Protocol` instances: Event methods, such as :meth:`loop.create_server`,
:meth:`loop.create_unix_server`, :meth:`loop.create_connection`,
:meth:`loop.create_unix_connection`, :meth:`loop.connect_accepted_socket`,
:meth:`loop.connect_read_pipe`, and :meth:`loop.connect_write_pipe`
accept factories that return streaming protocols.
.. method:: Protocol.data_received(data) .. method:: Protocol.data_received(data)
Called when some data is received. *data* is a non-empty bytes object Called when some data is received. *data* is a non-empty bytes
containing the incoming data. object containing the incoming data.
.. note::
Whether the data is buffered, chunked or reassembled depends on Whether the data is buffered, chunked or reassembled depends on
the transport. In general, you shouldn't rely on specific semantics the transport. In general, you shouldn't rely on specific semantics
and instead make your parsing generic and flexible enough. However, and instead make your parsing generic and flexible enough. However,
data is always received in the correct order. data is always received in the correct order.
The method can be called an arbitrary number of times during
a connection.
However, :meth:`protocol.eof_received() <Protocol.eof_received>`
is called at most once and, if called,
:meth:`protocol.data_received() <Protocol.data_received>`
won't be called after it.
.. method:: Protocol.eof_received() .. method:: Protocol.eof_received()
Called when the other end signals it won't send any more data Called when the other end signals it won't send any more data
(for example by calling :meth:`write_eof`, if the other end also uses (for example by calling :meth:`transport.write_eof()
<WriteTransport.write_eof>`, if the other end also uses
asyncio). asyncio).
This method may return a false value (including ``None``), in which case This method may return a false value (including ``None``), in which case
the transport will close itself. Conversely, if this method returns a the transport will close itself. Conversely, if this method returns a
true value, closing the transport is up to the protocol. Since the true value, closing the transport is up to the protocol. Since the
default implementation returns ``None``, it implicitly closes the connection. default implementation returns ``None``, it implicitly closes the
connection.
.. note::
Some transports such as SSL don't support half-closed connections, Some transports such as SSL don't support half-closed connections,
in which case returning true from this method will not prevent closing in which case returning true from this method will not prevent closing
the connection. the connection.
:meth:`data_received` can be called an arbitrary number of times during
a connection. However, :meth:`eof_received` is called at most once
and, if called, :meth:`data_received` won't be called after it.
State machine: State machine:
...@@ -446,20 +571,18 @@ State machine: ...@@ -446,20 +571,18 @@ State machine:
-> connection_lost -> end -> connection_lost -> end
Streaming protocols with manual receive buffer control Buffered Streaming Protocols
------------------------------------------------------ ----------------------------
.. versionadded:: 3.7 .. versionadded:: 3.7
**Important:** :class:`BufferedProtocol` has been added to **Important:** this has been added to asyncio in Python 3.7
asyncio in Python 3.7 *on a provisional basis*! Consider it as an *on a provisional basis*! This is as an experimental API that
experimental API that might be changed or removed in Python 3.8. might be changed or removed completely in Python 3.8.
Event methods, such as :meth:`AbstractEventLoop.create_server` and Buffered Protocols can be used with any event loop method
:meth:`AbstractEventLoop.create_connection`, accept factories that that supports `Streaming Protocols`_.
return protocols that implement this interface.
The idea of BufferedProtocol is that it allows to manually allocate The idea of ``BufferedProtocol`` is that it allows to manually allocate
and control the receive buffer. Event loops can then use the buffer and control the receive buffer. Event loops can then use the buffer
provided by the protocol to avoid unnecessary data copies. This provided by the protocol to avoid unnecessary data copies. This
can result in noticeable performance improvement for protocols that can result in noticeable performance improvement for protocols that
...@@ -489,13 +612,15 @@ instances: ...@@ -489,13 +612,15 @@ instances:
.. method:: BufferedProtocol.eof_received() .. method:: BufferedProtocol.eof_received()
See the documentation of the :meth:`Protocol.eof_received` method. See the documentation of the :meth:`protocol.eof_received()
<Protocol.eof_received>` method.
:meth:`get_buffer` can be called an arbitrary number of times during :meth:`~BufferedProtocol.get_buffer` can be called an arbitrary number
a connection. However, :meth:`eof_received` is called at most once of times during a connection. However, :meth:`protocol.eof_received()
and, if called, :meth:`get_buffer` and :meth:`buffer_updated` <Protocol.eof_received>` is called at most once
won't be called after it. and, if called, :meth:`~BufferedProtocol.get_buffer` and
:meth:`~BufferedProtocol.buffer_updated` won't be called after it.
State machine: State machine:
...@@ -509,10 +634,11 @@ State machine: ...@@ -509,10 +634,11 @@ State machine:
-> connection_lost -> end -> connection_lost -> end
Datagram protocols Datagram Protocols
------------------ ------------------
The following callbacks are called on :class:`DatagramProtocol` instances. Datagram Protocol instances should be constructed by protocol
factories passed to the :meth:`loop.create_datagram_endpoint` method.
.. method:: DatagramProtocol.datagram_received(data, addr) .. method:: DatagramProtocol.datagram_received(data, addr)
...@@ -530,76 +656,116 @@ The following callbacks are called on :class:`DatagramProtocol` instances. ...@@ -530,76 +656,116 @@ The following callbacks are called on :class:`DatagramProtocol` instances.
In many conditions though, undeliverable datagrams will be silently In many conditions though, undeliverable datagrams will be silently
dropped. dropped.
.. note::
Flow control callbacks On BSD systems (macOS, FreeBSD, etc.) flow control is not supported
---------------------- for datagram protocols, because send failures caused by
writing too many packets cannot be detected easily.
These callbacks may be called on :class:`Protocol`, The socket always appears 'ready' and excess packets are dropped; an
:class:`DatagramProtocol` and :class:`SubprocessProtocol` instances: :class:`OSError` with ``errno`` set to :const:`errno.ENOBUFS` may
or may not be raised; if it is raised, it will be reported to
:meth:`DatagramProtocol.error_received` but otherwise ignored.
.. method:: BaseProtocol.pause_writing()
Called when the transport's buffer goes over the high-water mark. .. _asyncio-subprocess-protocols:
.. method:: BaseProtocol.resume_writing() Subprocess Protocols
--------------------
Called when the transport's buffer drains below the low-water mark. Datagram Protocol instances should be constructed by protocol
factories passed to the :meth:`loop.subprocess_exec` and
:meth:`loop.subprocess_shell` methods.
.. method:: SubprocessProtocol.pipe_data_received(fd, data)
:meth:`pause_writing` and :meth:`resume_writing` calls are paired -- Called when the child process writes data into its stdout or stderr
:meth:`pause_writing` is called once when the buffer goes strictly over pipe.
the high-water mark (even if subsequent writes increases the buffer size
even more), and eventually :meth:`resume_writing` is called once when the
buffer size reaches the low-water mark.
.. note:: *fd* is the integer file descriptor of the pipe.
If the buffer size equals the high-water mark,
:meth:`pause_writing` is not called -- it must go strictly over. *data* is a non-empty bytes object containing the received data.
Conversely, :meth:`resume_writing` is called when the buffer size is
equal or lower than the low-water mark. These end conditions .. method:: SubprocessProtocol.pipe_connection_lost(fd, exc)
are important to ensure that things go as expected when either
mark is zero. Called when one of the pipes communicating with the child process
is closed.
*fd* is the integer file descriptor that was closed.
.. method:: SubprocessProtocol.process_exited()
Called when the child process has exited.
Examples
========
.. _asyncio-tcp-echo-server-protocol:
TCP Echo Server
---------------
TCP echo server using the :meth:`loop.create_server` method, send back
received data and close the connection::
import asyncio
class EchoServerClientProtocol(asyncio.Protocol):
def connection_made(self, transport):
peername = transport.get_extra_info('peername')
print('Connection from {}'.format(peername))
self.transport = transport
def data_received(self, data):
message = data.decode()
print('Data received: {!r}'.format(message))
print('Send: {!r}'.format(message))
self.transport.write(data)
print('Close the client socket')
self.transport.close()
.. note::
On BSD systems (OS X, FreeBSD, etc.) flow control is not supported
for :class:`DatagramProtocol`, because send failures caused by
writing too many packets cannot be detected easily. The socket
always appears 'ready' and excess packets are dropped; an
:class:`OSError` with errno set to :const:`errno.ENOBUFS` may or
may not be raised; if it is raised, it will be reported to
:meth:`DatagramProtocol.error_received` but otherwise ignored.
async def main():
# Get a reference to the event loop as we plan to use
# low-level APIs.
loop = asyncio.get_running_loop()
Coroutines and protocols server = await loop.create_server(
------------------------ lambda: EchoServerClientProtocol(),
'127.0.0.1', 8888)
Coroutines can be scheduled in a protocol method using :func:`ensure_future`, async with server:
but there is no guarantee made about the execution order. Protocols are not await server.serve_forever()
aware of coroutines created in protocol methods and so will not wait for them.
To have a reliable execution order,
use :ref:`stream objects <asyncio-streams>` in a
coroutine with ``await``. For example, the :meth:`StreamWriter.drain`
coroutine can be used to wait until the write buffer is flushed.
asyncio.run(main())
Protocol examples
================= .. seealso::
The :ref:`TCP echo server using streams <asyncio-tcp-echo-server-streams>`
example uses the high-level :func:`asyncio.start_server` function.
.. _asyncio-tcp-echo-client-protocol: .. _asyncio-tcp-echo-client-protocol:
TCP echo client protocol TCP Echo Client
------------------------ ---------------
TCP echo client using the :meth:`AbstractEventLoop.create_connection` method, send TCP echo client using the :meth:`loop.create_connection` method, send
data and wait until the connection is closed:: data and wait until the connection is closed::
import asyncio import asyncio
class EchoClientProtocol(asyncio.Protocol): class EchoClientProtocol(asyncio.Protocol):
def __init__(self, message, loop): def __init__(self, message, on_con_lost, loop):
self.message = message self.message = message
self.loop = loop self.loop = loop
self.on_con_lost = on_con_lost
def connection_made(self, transport): def connection_made(self, transport):
transport.write(self.message.encode()) transport.write(self.message.encode())
...@@ -610,99 +776,99 @@ data and wait until the connection is closed:: ...@@ -610,99 +776,99 @@ data and wait until the connection is closed::
def connection_lost(self, exc): def connection_lost(self, exc):
print('The server closed the connection') print('The server closed the connection')
print('Stop the event loop') self.on_con_lost.set_result(True)
self.loop.stop()
async def main():
# Get a reference to the event loop as we plan to use
# low-level APIs.
loop = asyncio.get_running_loop()
loop = asyncio.get_event_loop() on_con_lost = loop.create_future()
message = 'Hello World!' message = 'Hello World!'
coro = loop.create_connection(lambda: EchoClientProtocol(message, loop),
transport, protocol = await loop.create_connection(
lambda: EchoClientProtocol(message, on_con_lost, loop),
'127.0.0.1', 8888) '127.0.0.1', 8888)
loop.run_until_complete(coro)
loop.run_forever()
loop.close()
The event loop is running twice. The # Wait until the protocol signals that the connection
:meth:`~AbstractEventLoop.run_until_complete` method is preferred in this short # is lost and close the transport.
example to raise an exception if the server is not listening, instead of try:
having to write a short coroutine to handle the exception and stop the await on_con_lost
running loop. At :meth:`~AbstractEventLoop.run_until_complete` exit, the loop is finally:
no longer running, so there is no need to stop the loop in case of an error. transport.close()
asyncio.run(main())
.. seealso:: .. seealso::
The :ref:`TCP echo client using streams <asyncio-tcp-echo-client-streams>` The :ref:`TCP echo client using streams <asyncio-tcp-echo-client-streams>`
example uses the :func:`asyncio.open_connection` function. example uses the high-level :func:`asyncio.open_connection` function.
.. _asyncio-tcp-echo-server-protocol: .. _asyncio-udp-echo-server-protocol:
TCP echo server protocol UDP Echo Server
------------------------ ---------------
TCP echo server using the :meth:`AbstractEventLoop.create_server` method, send back UDP echo server using the :meth:`loop.create_datagram_endpoint`
received data and close the connection:: method, send back received data::
import asyncio import asyncio
class EchoServerClientProtocol(asyncio.Protocol):
class EchoServerProtocol:
def connection_made(self, transport): def connection_made(self, transport):
peername = transport.get_extra_info('peername')
print('Connection from {}'.format(peername))
self.transport = transport self.transport = transport
def data_received(self, data): def datagram_received(self, data, addr):
message = data.decode() message = data.decode()
print('Data received: {!r}'.format(message)) print('Received %r from %s' % (message, addr))
print('Send %r to %s' % (message, addr))
print('Send: {!r}'.format(message)) self.transport.sendto(data, addr)
self.transport.write(data)
print('Close the client socket')
self.transport.close()
loop = asyncio.get_event_loop() async def main():
# Each client connection will create a new protocol instance print("Starting UDP server")
coro = loop.create_server(EchoServerClientProtocol, '127.0.0.1', 8888)
server = loop.run_until_complete(coro)
# Serve requests until Ctrl+C is pressed # Get a reference to the event loop as we plan to use
print('Serving on {}'.format(server.sockets[0].getsockname())) # low-level APIs.
try: loop = asyncio.get_running_loop()
loop.run_forever()
except KeyboardInterrupt:
pass
# Close the server # One protocol instance will be created to serve all
server.close() # client requests.
loop.run_until_complete(server.wait_closed()) transport, protocol = await loop.create_datagram_endpoint(
loop.close() lambda: EchoServerProtocol(),
local_addr=('127.0.0.1', 9999))
:meth:`Transport.close` can be called immediately after try:
:meth:`WriteTransport.write` even if data are not sent yet on the socket: both await asyncio.sleep(3600) # Serve for 1 hour.
methods are asynchronous. ``await`` is not needed because these transport finally:
methods are not coroutines. transport.close()
.. seealso::
The :ref:`TCP echo server using streams <asyncio-tcp-echo-server-streams>` asyncio.run(main())
example uses the :func:`asyncio.start_server` function.
.. _asyncio-udp-echo-client-protocol: .. _asyncio-udp-echo-client-protocol:
UDP echo client protocol UDP Echo Client
------------------------ ---------------
UDP echo client using the :meth:`AbstractEventLoop.create_datagram_endpoint` UDP echo client using the :meth:`loop.create_datagram_endpoint`
method, send data and close the transport when we received the answer:: method, send data and close the transport when we received the answer::
import asyncio import asyncio
class EchoClientProtocol: class EchoClientProtocol:
def __init__(self, message, loop): def __init__(self, message, loop):
self.message = message self.message = message
self.loop = loop self.loop = loop
self.transport = None self.transport = None
self.on_con_lost = loop.create_future()
def connection_made(self, transport): def connection_made(self, transport):
self.transport = transport self.transport = transport
...@@ -719,75 +885,46 @@ method, send data and close the transport when we received the answer:: ...@@ -719,75 +885,46 @@ method, send data and close the transport when we received the answer::
print('Error received:', exc) print('Error received:', exc)
def connection_lost(self, exc): def connection_lost(self, exc):
print("Socket closed, stop the event loop") print("Connection closed")
loop = asyncio.get_event_loop() self.on_con_lost.set_result(True)
loop.stop()
async def main():
# Get a reference to the event loop as we plan to use
# low-level APIs.
loop = asyncio.get_running_loop()
loop = asyncio.get_event_loop()
message = "Hello World!" message = "Hello World!"
connect = loop.create_datagram_endpoint( transport, protocol = await loop.create_datagram_endpoint(
lambda: EchoClientProtocol(message, loop), lambda: EchoClientProtocol(message, loop),
remote_addr=('127.0.0.1', 9999)) remote_addr=('127.0.0.1', 9999))
transport, protocol = loop.run_until_complete(connect)
loop.run_forever()
transport.close()
loop.close()
.. _asyncio-udp-echo-server-protocol:
UDP echo server protocol
------------------------
UDP echo server using the :meth:`AbstractEventLoop.create_datagram_endpoint`
method, send back received data::
import asyncio
class EchoServerProtocol:
def connection_made(self, transport):
self.transport = transport
def datagram_received(self, data, addr):
message = data.decode()
print('Received %r from %s' % (message, addr))
print('Send %r to %s' % (message, addr))
self.transport.sendto(data, addr)
loop = asyncio.get_event_loop()
print("Starting UDP server")
# One protocol instance will be created to serve all client requests
listen = loop.create_datagram_endpoint(
EchoServerProtocol, local_addr=('127.0.0.1', 9999))
transport, protocol = loop.run_until_complete(listen)
try: try:
loop.run_forever() await protocol.on_con_lost
except KeyboardInterrupt: finally:
pass
transport.close() transport.close()
loop.close()
asyncio.run(main())
.. _asyncio-register-socket: .. _asyncio-register-socket:
Register an open socket to wait for data using a protocol Connecting Existing Sockets
--------------------------------------------------------- ---------------------------
Wait until a socket receives data using the Wait until a socket receives data using the
:meth:`AbstractEventLoop.create_connection` method with a protocol, and then close :meth:`loop.create_connection` method with a protocol::
the event loop ::
import asyncio import asyncio
from socket import socketpair import socket
# Create a pair of connected sockets
rsock, wsock = socketpair()
loop = asyncio.get_event_loop()
class MyProtocol(asyncio.Protocol): class MyProtocol(asyncio.Protocol):
transport = None
def __init__(self, loop):
self.transport = None
self.on_con_lost = loop.create_future()
def connection_made(self, transport): def connection_made(self, transport):
self.transport = transport self.transport = transport
...@@ -795,35 +932,102 @@ the event loop :: ...@@ -795,35 +932,102 @@ the event loop ::
def data_received(self, data): def data_received(self, data):
print("Received:", data.decode()) print("Received:", data.decode())
# We are done: close the transport (it will call connection_lost()) # We are done: close the transport;
# connection_lost() will be called automatically.
self.transport.close() self.transport.close()
def connection_lost(self, exc): def connection_lost(self, exc):
# The socket has been closed, stop the event loop # The socket has been closed
loop.stop() self.on_con_lost.set_result(True)
# Register the socket to wait for data
connect_coro = loop.create_connection(MyProtocol, sock=rsock)
transport, protocol = loop.run_until_complete(connect_coro)
# Simulate the reception of data from the network async def main():
loop.call_soon(wsock.send, 'abc'.encode()) # Get a reference to the event loop as we plan to use
# low-level APIs.
loop = asyncio.get_running_loop()
# Run the event loop # Create a pair of connected sockets
loop.run_forever() rsock, wsock = socket.socketpair()
# Register the socket to wait for data.
transport, protocol = await loop.create_connection(
lambda: MyProtocol(loop), sock=rsock)
# Simulate the reception of data from the network.
loop.call_soon(wsock.send, 'abc'.encode())
# We are done, close sockets and the event loop try:
rsock.close() await protocol.on_con_lost
finally:
transport.close()
wsock.close() wsock.close()
loop.close()
asyncio.run(main())
.. seealso:: .. seealso::
The :ref:`watch a file descriptor for read events The :ref:`watch a file descriptor for read events
<asyncio-watch-read-event>` example uses the low-level <asyncio-watch-read-event>` example uses the low-level
:meth:`AbstractEventLoop.add_reader` method to register the file descriptor of a :meth:`loop.add_reader` method to register an FD.
socket.
The :ref:`register an open socket to wait for data using streams The :ref:`register an open socket to wait for data using streams
<asyncio-register-socket-streams>` example uses high-level streams <asyncio-register-socket-streams>` example uses high-level streams
created by the :func:`open_connection` function in a coroutine. created by the :func:`open_connection` function in a coroutine.
.. _asyncio-subprocess-proto-example:
loop.subprocess_exec() and SubprocessProtocol
---------------------------------------------
An example of a subprocess protocol using to get the output of a
subprocess and to wait for the subprocess exit.
The subprocess is created by th :meth:`loop.subprocess_exec` method::
import asyncio
import sys
class DateProtocol(asyncio.SubprocessProtocol):
def __init__(self, exit_future):
self.exit_future = exit_future
self.output = bytearray()
def pipe_data_received(self, fd, data):
self.output.extend(data)
def process_exited(self):
self.exit_future.set_result(True)
async def get_date():
# Get a reference to the event loop as we plan to use
# low-level APIs.
loop = asyncio.get_running_loop()
code = 'import datetime; print(datetime.datetime.now())'
exit_future = asyncio.Future(loop=loop)
# Create the subprocess controlled by DateProtocol;
# redirect the standard output into a pipe.
transport, protocol = await loop.subprocess_exec(
lambda: DateProtocol(exit_future),
sys.executable, '-c', code,
stdin=None, stderr=None)
# Wait for the subprocess exit using the process_exited()
# method of the protocol.
await exit_future
# Close the stdout pipe.
transport.close()
# Read the output which was collected by the
# pipe_data_received() method of the protocol.
data = bytes(protocol.output)
return data.decode('ascii').rstrip()
if sys.platform == "win32":
asyncio.set_event_loop_policy(
asyncio.WindowsProactorEventLoopPolicy())
date = asyncio.run(get_date())
print(f"Current date: {date}")
Doc/library/asyncio-queue.rst
View file @ 7c7605ff
.. currentmodule:: asyncio .. currentmodule:: asyncio
======
Queues Queues
====== ======
**Source code:** :source:`Lib/asyncio/queues.py` asyncio queues are designed to be similar to classes of the
:mod:`queue` module. Although asyncio queues are not thread-safe,
Queues: they are designed to be used specifically in async/await code.
* :class:`Queue` Note that methods on asyncio queues don't have a *timeout* parameter;
* :class:`PriorityQueue` use :func:`asyncio.wait_for` function to do queue operations with a
* :class:`LifoQueue` timeout.
asyncio queue API was designed to be close to classes of the :mod:`queue` See also the `Examples`_ section below.
module (:class:`~queue.Queue`, :class:`~queue.PriorityQueue`,
:class:`~queue.LifoQueue`), but it has no *timeout* parameter. The
:func:`asyncio.wait_for` function can be used to cancel a task after a timeout.
Queue Queue
----- =====
.. class:: Queue(maxsize=0, \*, loop=None) .. class:: Queue(maxsize=0, \*, loop=None)
A queue, useful for coordinating producer and consumer coroutines. A first in, first out (FIFO) queue.
If *maxsize* is less than or equal to zero, the queue size is infinite. If If *maxsize* is less than or equal to zero, the queue size is
it is an integer greater than ``0``, then ``await put()`` will block infinite. If it is an integer greater than ``0``, then
when the queue reaches *maxsize*, until an item is removed by :meth:`get`. ``await put()`` blocks when the queue reaches *maxsize*
until an item is removed by :meth:`get`.
Unlike the standard library :mod:`queue`, you can reliably know this Queue's Unlike the standard library threading :mod:`queue`, the size of
size with :meth:`qsize`, since your single-threaded asyncio application won't the queue is always known and can be returned by calling the
be interrupted between calling :meth:`qsize` and doing an operation on the :meth:`qsize` method.
Queue.
This class is :ref:`not thread safe <asyncio-multithreading>`. This class is :ref:`not thread safe <asyncio-multithreading>`.
.. versionchanged:: 3.4.4 .. attribute:: maxsize
New :meth:`join` and :meth:`task_done` methods.
Number of items allowed in the queue.
.. method:: empty() .. method:: empty()
...@@ -45,26 +45,16 @@ Queue ...@@ -45,26 +45,16 @@ Queue
Return ``True`` if there are :attr:`maxsize` items in the queue. Return ``True`` if there are :attr:`maxsize` items in the queue.
.. note:: If the queue was initialized with ``maxsize=0`` (the default),
then :meth:`full()` never returns ``True``.
If the Queue was initialized with ``maxsize=0`` (the default), then
:meth:`full()` is never ``True``.
.. coroutinemethod:: get() .. coroutinemethod:: get()
Remove and return an item from the queue. If queue is empty, wait until Remove and return an item from the queue. If queue is empty,
an item is available. wait until an item is available.
This method is a :ref:`coroutine <coroutine>`.
.. seealso::
The :meth:`empty` method.
.. method:: get_nowait() .. method:: get_nowait()
Remove and return an item from the queue.
Return an item if one is immediately available, else raise Return an item if one is immediately available, else raise
:exc:`QueueEmpty`. :exc:`QueueEmpty`.
...@@ -72,26 +62,16 @@ Queue ...@@ -72,26 +62,16 @@ Queue
Block until all items in the queue have been gotten and processed. Block until all items in the queue have been gotten and processed.
The count of unfinished tasks goes up whenever an item is added to the The count of unfinished tasks goes up whenever an item is added
queue. The count goes down whenever a consumer thread calls to the queue. The count goes down whenever a consumer thread calls
:meth:`task_done` to indicate that the item was retrieved and all work on :meth:`task_done` to indicate that the item was retrieved and all
it is complete. When the count of unfinished tasks drops to zero, work on it is complete. When the count of unfinished tasks drops
:meth:`join` unblocks. to zero, :meth:`join` unblocks.
This method is a :ref:`coroutine <coroutine>`.
.. versionadded:: 3.4.4
.. coroutinemethod:: put(item) .. coroutinemethod:: put(item)
Put an item into the queue. If the queue is full, wait until a free slot Put an item into the queue. If the queue is full, wait until a
is available before adding item. free slot is available before adding item.
This method is a :ref:`coroutine <coroutine>`.
.. seealso::
The :meth:`full` method.
.. method:: put_nowait(item) .. method:: put_nowait(item)
...@@ -107,54 +87,111 @@ Queue ...@@ -107,54 +87,111 @@ Queue
Indicate that a formerly enqueued task is complete. Indicate that a formerly enqueued task is complete.
Used by queue consumers. For each :meth:`~Queue.get` used to fetch a task, a Used by queue consumers. For each :meth:`~Queue.get` used to
subsequent call to :meth:`task_done` tells the queue that the processing fetch a task, a subsequent call to :meth:`task_done` tells the
on the task is complete. queue that the processing on the task is complete.
If a :meth:`join` is currently blocking, it will resume when all items
have been processed (meaning that a :meth:`task_done` call was received
for every item that had been :meth:`~Queue.put` into the queue).
Raises :exc:`ValueError` if called more times than there were items If a :meth:`join` is currently blocking, it will resume when all
placed in the queue. items have been processed (meaning that a :meth:`task_done`
call was received for every item that had been :meth:`~Queue.put`
into the queue).
.. versionadded:: 3.4.4 Raises :exc:`ValueError` if called more times than there were
items placed in the queue.
.. attribute:: maxsize
Number of items allowed in the queue.
Priority Queue
PriorityQueue ==============
-------------
.. class:: PriorityQueue .. class:: PriorityQueue
A subclass of :class:`Queue`; retrieves entries in priority order (lowest A variant of :class:`Queue`; retrieves entries in priority order
first). (lowest first).
Entries are typically tuples of the form: (priority number, data). Entries are typically tuples of the form
``(priority_number, data)``.
LifoQueue LIFO Queue
--------- ==========
.. class:: LifoQueue .. class:: LifoQueue
A subclass of :class:`Queue` that retrieves most recently added entries A variant of :class:`Queue` that retrieves most recently added
first. entries first (last in, first out).
Exceptions Exceptions
^^^^^^^^^^ ==========
.. exception:: QueueEmpty .. exception:: QueueEmpty
Exception raised when the :meth:`~Queue.get_nowait` method is called on a This exception is raised when the :meth:`~Queue.get_nowait` method
:class:`Queue` object which is empty. is called on an empty queue.
.. exception:: QueueFull .. exception:: QueueFull
Exception raised when the :meth:`~Queue.put_nowait` method is called on a Exception raised when the :meth:`~Queue.put_nowait` method is called
:class:`Queue` object which is full. on a queue that has reached its *maxsize*.
Examples
========
Queues can be used to distribute workload between several
concurrent tasks::
import asyncio
import random
import time
async def worker(name, queue):
while True:
# Get a "work item" out of the queue.
sleep_for = await queue.get()
# Sleep for the "sleep_for" seconds.
await asyncio.sleep(sleep_for)
# Notify the queue that the "work item" has been processed.
queue.task_done()
print(f'{name} has slept for {sleep_for:.2f} seconds')
async def main():
# Create a queue that we will use to store our "workload".
queue = asyncio.Queue()
# Generate random timings and put them into the queue.
total_sleep_time = 0
for _ in range(20):
sleep_for = random.uniform(0.05, 1.0)
total_sleep_time += sleep_for
queue.put_nowait(sleep_for)
# Create three worker tasks to process the queue concurrently.
tasks = []
for i in range(3):
task = asyncio.create_task(worker(f'worker-{i}', queue))
tasks.append(task)
# Wait until the queue is fully processed.
started_at = time.monotonic()
await queue.join()
total_slept_for = time.monotonic() - started_at
# Cancel our worker tasks.
for task in tasks:
task.cancel()
# Wait until all worker tasks are cancelled.
await asyncio.gather(*tasks, return_exceptions=True)
print('====')
print(f'3 workers slept in parallel for {total_slept_for:.2f} seconds')
print(f'total expected sleep time: {total_sleep_time:.2f} seconds')
asyncio.run(main())
Doc/library/asyncio-stream.rst
View file @ 7c7605ff
...@@ -2,83 +2,107 @@ ...@@ -2,83 +2,107 @@
.. _asyncio-streams: .. _asyncio-streams:
+++++++++++++++++++++++++++++ =======
Streams (coroutine based API) Streams
+++++++++++++++++++++++++++++ =======
**Source code:** :source:`Lib/asyncio/streams.py` Streams are high-level async/await-ready primitives to work with
network connections. Streams allow send and receive data without
using callbacks or low-level protocols and transports.
Stream functions Here's an example of a TCP echo client written using asyncio
================ streams::
.. note:: import asyncio
async def tcp_echo_client(message):
reader, writer = await asyncio.open_connection(
'127.0.0.1', 8888)
print(f'Send: {message!r}')
writer.write(message.encode())
data = await reader.read(100)
print(f'Received: {data.decode()!r}')
print('Close the connection')
writer.close()
The top-level functions in this module are meant as convenience wrappers asyncio.run(tcp_echo_client('Hello World!'))
only; there's really nothing special there, and if they don't do
exactly what you want, feel free to copy their code.
.. coroutinefunction:: open_connection(host=None, port=None, \*, loop=None, limit=None, ssl=None, family=0, proto=0, flags=0, sock=None, local_addr=None, server_hostname=None, ssl_handshake_timeout=None) .. rubric:: Stream Functions
A wrapper for :meth:`~AbstractEventLoop.create_connection()` returning a (reader, The following top-level asyncio functions can be used to create
writer) pair. and work with streams:
The reader returned is a :class:`StreamReader` instance; the writer is
a :class:`StreamWriter` instance.
When specified, the *loop* argument determines which event loop to use, .. coroutinefunction:: open_connection(host=None, port=None, \*, \
and the *limit* argument determines the buffer size limit used by the loop=None, limit=None, ssl=None, family=0, \
proto=0, flags=0, sock=None, local_addr=None, \
server_hostname=None, ssl_handshake_timeout=None)
Establish a network connection and return a pair of
``(reader, writer)``.
The returned *reader* and *writer* objects are instances of
:class:`StreamReader` and :class:`StreamWriter` classes.
The *loop* argument is optional and can always be determined
automatically when this method is awaited from a coroutine.
*limit* determines the buffer size limit used by the
returned :class:`StreamReader` instance. returned :class:`StreamReader` instance.
The rest of the arguments are passed directly to The rest of the arguments are passed directly to
:meth:`AbstractEventLoop.create_connection`. :meth:`loop.create_connection`.
This function is a :ref:`coroutine <coroutine>`.
.. versionadded:: 3.7 .. versionadded:: 3.7
The *ssl_handshake_timeout* parameter. The *ssl_handshake_timeout* parameter.
.. coroutinefunction:: start_server(client_connected_cb, host=None, port=None, \*, loop=None, limit=None, family=socket.AF_UNSPEC, flags=socket.AI_PASSIVE, sock=None, backlog=100, ssl=None, reuse_address=None, reuse_port=None, ssl_handshake_timeout=None, start_serving=True) .. coroutinefunction:: start_server(client_connected_cb, host=None, \
port=None, \*, loop=None, limit=None, \
family=socket.AF_UNSPEC, \
flags=socket.AI_PASSIVE, sock=None, \
backlog=100, ssl=None, reuse_address=None, \
reuse_port=None, ssl_handshake_timeout=None, \
start_serving=True)
Start a socket server, with a callback for each client connected. The return Start a socket server.
value is the same as :meth:`~AbstractEventLoop.create_server()`.
The *client_connected_cb* callback is called whenever a new client The *client_connected_cb* callback is called whenever a new client
connection is established. It receives a reader/writer pair as two connection is established. It receives a ``(reader, writer)`` pair
arguments, the first is a :class:`StreamReader` instance, as two arguments, instances of the :class:`StreamReader` and
and the second is a :class:`StreamWriter` instance. :class:`StreamWriter` classes.
*client_connected_cb* accepts a plain callable or a *client_connected_cb* can be a plain callable or a
:ref:`coroutine function <coroutine>`; if it is a coroutine function, :ref:`coroutine function <coroutine>`; if it is a coroutine function,
it will be automatically converted into a :class:`Task`. it will be automatically wrapped into a :class:`Task`.
When specified, the *loop* argument determines which event loop to use, The *loop* argument is optional and can always be determined
and the *limit* argument determines the buffer size limit used by the automatically when this method is awaited from a coroutine.
:class:`StreamReader` instance passed to *client_connected_cb*.
The rest of the arguments are passed directly to *limit* determines the buffer size limit used by the
:meth:`~AbstractEventLoop.create_server()`. returned :class:`StreamReader` instance.
This function is a :ref:`coroutine <coroutine>`. The rest of the arguments are passed directly to
:meth:`loop.create_server`.
.. versionadded:: 3.7 .. versionadded:: 3.7
The *ssl_handshake_timeout* and *start_serving* parameters. The *ssl_handshake_timeout* and *start_serving* parameters.
.. coroutinefunction:: open_unix_connection(path=None, \*, loop=None, limit=None, ssl=None, sock=None, server_hostname=None, ssl_handshake_timeout=None) .. coroutinefunction:: open_unix_connection(path=None, \*, loop=None, \
limit=None, ssl=None, sock=None, \
A wrapper for :meth:`~AbstractEventLoop.create_unix_connection()` returning server_hostname=None, ssl_handshake_timeout=None)
a (reader, writer) pair.
When specified, the *loop* argument determines which event loop to use, Establish a UNIX socket connection and return a pair of
and the *limit* argument determines the buffer size limit used by the ``(reader, writer)``.
returned :class:`StreamReader` instance.
The rest of the arguments are passed directly to Similar to :func:`open_connection` but operates on UNIX sockets.
:meth:`~AbstractEventLoop.create_unix_connection()`.
This function is a :ref:`coroutine <coroutine>`. See also the documentation of :meth:`loop.create_unix_connection`.
Availability: UNIX. Availability: UNIX.
...@@ -90,27 +114,16 @@ Stream functions ...@@ -90,27 +114,16 @@ Stream functions
The *path* parameter can now be a :term:`path-like object` The *path* parameter can now be a :term:`path-like object`
.. coroutinefunction:: start_unix_server(client_connected_cb, path=None, \*, loop=None, limit=None, sock=None, backlog=100, ssl=None, ssl_handshake_timeout=None, start_serving=True) .. coroutinefunction:: start_unix_server(client_connected_cb, path=None, \
\*, loop=None, limit=None, sock=None, \
Start a UNIX Domain Socket server, with a callback for each client connected. backlog=100, ssl=None, ssl_handshake_timeout=None, \
start_serving=True)
The *client_connected_cb* callback is called whenever a new client
connection is established. It receives a reader/writer pair as two
arguments, the first is a :class:`StreamReader` instance,
and the second is a :class:`StreamWriter` instance.
*client_connected_cb* accepts a plain callable or a
:ref:`coroutine function <coroutine>`; if it is a coroutine function,
it will be automatically converted into a :class:`Task`.
When specified, the *loop* argument determines which event loop to use, Start a UNIX socket server.
and the *limit* argument determines the buffer size limit used by the
:class:`StreamReader` instance passed to *client_connected_cb*.
The rest of the arguments are passed directly to Similar to :func:`start_server` but operates on UNIX sockets.
:meth:`~AbstractEventLoop.create_unix_server()`.
This function is a :ref:`coroutine <coroutine>`. See also the documentation of :meth:`loop.create_unix_server`.
Availability: UNIX. Availability: UNIX.
...@@ -123,6 +136,13 @@ Stream functions ...@@ -123,6 +136,13 @@ Stream functions
The *path* parameter can now be a :term:`path-like object`. The *path* parameter can now be a :term:`path-like object`.
.. rubric:: Contents
* `StreamReader`_ and `StreamWriter`_
* `StreamReaderProtocol`_
* `Examples`_
StreamReader StreamReader
============ ============
...@@ -159,8 +179,6 @@ StreamReader ...@@ -159,8 +179,6 @@ StreamReader
If the EOF was received and the internal buffer is empty, If the EOF was received and the internal buffer is empty,
return an empty ``bytes`` object. return an empty ``bytes`` object.
This method is a :ref:`coroutine <coroutine>`.
.. coroutinemethod:: readline() .. coroutinemethod:: readline()
Read one line, where "line" is a sequence of bytes ending with ``\n``. Read one line, where "line" is a sequence of bytes ending with ``\n``.
...@@ -171,8 +189,6 @@ StreamReader ...@@ -171,8 +189,6 @@ StreamReader
If the EOF was received and the internal buffer is empty, If the EOF was received and the internal buffer is empty,
return an empty ``bytes`` object. return an empty ``bytes`` object.
This method is a :ref:`coroutine <coroutine>`.
.. coroutinemethod:: readexactly(n) .. coroutinemethod:: readexactly(n)
Read exactly *n* bytes. Raise an :exc:`IncompleteReadError` if the end of Read exactly *n* bytes. Raise an :exc:`IncompleteReadError` if the end of
...@@ -180,8 +196,6 @@ StreamReader ...@@ -180,8 +196,6 @@ StreamReader
:attr:`IncompleteReadError.partial` attribute of the exception contains :attr:`IncompleteReadError.partial` attribute of the exception contains
the partial read bytes. the partial read bytes.
This method is a :ref:`coroutine <coroutine>`.
.. coroutinemethod:: readuntil(separator=b'\\n') .. coroutinemethod:: readuntil(separator=b'\\n')
Read data from the stream until ``separator`` is found. Read data from the stream until ``separator`` is found.
...@@ -208,7 +222,8 @@ StreamReader ...@@ -208,7 +222,8 @@ StreamReader
.. method:: at_eof() .. method:: at_eof()
Return ``True`` if the buffer is empty and :meth:`feed_eof` was called. Return ``True`` if the buffer is empty and :meth:`feed_eof`
was called.
StreamWriter StreamWriter
...@@ -299,7 +314,8 @@ StreamWriter ...@@ -299,7 +314,8 @@ StreamWriter
StreamReaderProtocol StreamReaderProtocol
==================== ====================
.. class:: StreamReaderProtocol(stream_reader, client_connected_cb=None, loop=None) .. class:: StreamReaderProtocol(stream_reader, client_connected_cb=None, \
loop=None)
Trivial helper class to adapt between :class:`Protocol` and Trivial helper class to adapt between :class:`Protocol` and
:class:`StreamReader`. Subclass of :class:`Protocol`. :class:`StreamReader`. Subclass of :class:`Protocol`.
...@@ -314,36 +330,8 @@ StreamReaderProtocol ...@@ -314,36 +330,8 @@ StreamReaderProtocol
accidentally calling inappropriate methods of the protocol.) accidentally calling inappropriate methods of the protocol.)
IncompleteReadError Examples
=================== ========
.. exception:: IncompleteReadError
Incomplete read error, subclass of :exc:`EOFError`.
.. attribute:: expected
Total number of expected bytes (:class:`int`).
.. attribute:: partial
Read bytes string before the end of stream was reached (:class:`bytes`).
LimitOverrunError
=================
.. exception:: LimitOverrunError
Reached the buffer limit while looking for a separator.
.. attribute:: consumed
Total number of to be consumed bytes.
Stream examples
===============
.. _asyncio-tcp-echo-client-streams: .. _asyncio-tcp-echo-client-streams:
...@@ -354,28 +342,26 @@ TCP echo client using the :func:`asyncio.open_connection` function:: ...@@ -354,28 +342,26 @@ TCP echo client using the :func:`asyncio.open_connection` function::
import asyncio import asyncio
async def tcp_echo_client(message, loop): async def tcp_echo_client(message):
reader, writer = await asyncio.open_connection('127.0.0.1', 8888, reader, writer = await asyncio.open_connection(
loop=loop) '127.0.0.1', 8888)
print('Send: %r' % message) print(f'Send: {message!r}')
writer.write(message.encode()) writer.write(message.encode())
data = await reader.read(100) data = await reader.read(100)
print('Received: %r' % data.decode()) print(f'Received: {data.decode()!r}')
print('Close the socket') print('Close the connection')
writer.close() writer.close()
message = 'Hello World!' asyncio.run(tcp_echo_client('Hello World!'))
loop = asyncio.get_event_loop()
loop.run_until_complete(tcp_echo_client(message, loop))
loop.close()
.. seealso:: .. seealso::
The :ref:`TCP echo client protocol <asyncio-tcp-echo-client-protocol>` The :ref:`TCP echo client protocol <asyncio-tcp-echo-client-protocol>`
example uses the :meth:`AbstractEventLoop.create_connection` method. example uses the low-level :meth:`loop.create_connection` method.
.. _asyncio-tcp-echo-server-streams: .. _asyncio-tcp-echo-server-streams:
...@@ -391,35 +377,33 @@ TCP echo server using the :func:`asyncio.start_server` function:: ...@@ -391,35 +377,33 @@ TCP echo server using the :func:`asyncio.start_server` function::
data = await reader.read(100) data = await reader.read(100)
message = data.decode() message = data.decode()
addr = writer.get_extra_info('peername') addr = writer.get_extra_info('peername')
print("Received %r from %r" % (message, addr))
print("Send: %r" % message) print(f"Received {message!r} from {addr!r}")
print(f"Send: {message!r}")
writer.write(data) writer.write(data)
await writer.drain() await writer.drain()
print("Close the client socket") print("Close the connection")
writer.close() writer.close()
loop = asyncio.get_event_loop() async def main():
coro = asyncio.start_server(handle_echo, '127.0.0.1', 8888, loop=loop) server = await asyncio.start_server(
server = loop.run_until_complete(coro) handle_echo, '127.0.0.1', 8888)
addr = server.sockets[0].getsockname()
print(f'Serving on {addr}')
# Serve requests until Ctrl+C is pressed async with server:
print('Serving on {}'.format(server.sockets[0].getsockname())) await server.serve_forever()
try:
loop.run_forever() asyncio.run(main())
except KeyboardInterrupt:
pass
# Close the server
server.close()
loop.run_until_complete(server.wait_closed())
loop.close()
.. seealso:: .. seealso::
The :ref:`TCP echo server protocol <asyncio-tcp-echo-server-protocol>` The :ref:`TCP echo server protocol <asyncio-tcp-echo-server-protocol>`
example uses the :meth:`AbstractEventLoop.create_server` method. example uses the :meth:`loop.create_server` method.
Get HTTP headers Get HTTP headers
...@@ -434,30 +418,34 @@ Simple example querying HTTP headers of the URL passed on the command line:: ...@@ -434,30 +418,34 @@ Simple example querying HTTP headers of the URL passed on the command line::
async def print_http_headers(url): async def print_http_headers(url):
url = urllib.parse.urlsplit(url) url = urllib.parse.urlsplit(url)
if url.scheme == 'https': if url.scheme == 'https':
connect = asyncio.open_connection(url.hostname, 443, ssl=True) reader, writer = await asyncio.open_connection(
url.hostname, 443, ssl=True)
else: else:
connect = asyncio.open_connection(url.hostname, 80) reader, writer = await asyncio.open_connection(
reader, writer = await connect url.hostname, 80)
query = ('HEAD {path} HTTP/1.0\r\n'
'Host: {hostname}\r\n' query = (
'\r\n').format(path=url.path or '/', hostname=url.hostname) f"HEAD {url.path or '/'} HTTP/1.0\r\n"
f"Host: {url.hostname}\r\n"
f"\r\n"
)
writer.write(query.encode('latin-1')) writer.write(query.encode('latin-1'))
while True: while True:
line = await reader.readline() line = await reader.readline()
if not line: if not line:
break break
line = line.decode('latin1').rstrip() line = line.decode('latin1').rstrip()
if line: if line:
print('HTTP header> %s' % line) print(f'HTTP header> {line}')
# Ignore the body, close the socket # Ignore the body, close the socket
writer.close() writer.close()
url = sys.argv[1] url = sys.argv[1]
loop = asyncio.get_event_loop() asyncio.run(print_http_headers(url))
task = asyncio.ensure_future(print_http_headers(url))
loop.run_until_complete(task)
loop.close()
Usage:: Usage::
...@@ -467,6 +455,7 @@ or with HTTPS:: ...@@ -467,6 +455,7 @@ or with HTTPS::
python example.py https://example.com/path/page.html python example.py https://example.com/path/page.html
.. _asyncio-register-socket-streams: .. _asyncio-register-socket-streams:
Register an open socket to wait for data using streams Register an open socket to wait for data using streams
...@@ -476,14 +465,18 @@ Coroutine waiting until a socket receives data using the ...@@ -476,14 +465,18 @@ Coroutine waiting until a socket receives data using the
:func:`open_connection` function:: :func:`open_connection` function::
import asyncio import asyncio
from socket import socketpair import socket
async def wait_for_data():
# Get a reference to the current event loop because
# we want to access low-level APIs.
loop = asyncio.get_running_loop()
async def wait_for_data(loop): # Create a pair of connected sockets.
# Create a pair of connected sockets rsock, wsock = socket.socketpair()
rsock, wsock = socketpair()
# Register the open socket to wait for data # Register the open socket to wait for data.
reader, writer = await asyncio.open_connection(sock=rsock, loop=loop) reader, writer = await asyncio.open_connection(sock=rsock)
# Simulate the reception of data from the network # Simulate the reception of data from the network
loop.call_soon(wsock.send, 'abc'.encode()) loop.call_soon(wsock.send, 'abc'.encode())
...@@ -498,17 +491,14 @@ Coroutine waiting until a socket receives data using the ...@@ -498,17 +491,14 @@ Coroutine waiting until a socket receives data using the
# Close the second socket # Close the second socket
wsock.close() wsock.close()
loop = asyncio.get_event_loop() asyncio.run(wait_for_data())
loop.run_until_complete(wait_for_data(loop))
loop.close()
.. seealso:: .. seealso::
The :ref:`register an open socket to wait for data using a protocol The :ref:`register an open socket to wait for data using a protocol
<asyncio-register-socket>` example uses a low-level protocol created by the <asyncio-register-socket>` example uses a low-level protocol and
:meth:`AbstractEventLoop.create_connection` method. the :meth:`loop.create_connection` method.
The :ref:`watch a file descriptor for read events The :ref:`watch a file descriptor for read events
<asyncio-watch-read-event>` example uses the low-level <asyncio-watch-read-event>` example uses the low-level
:meth:`AbstractEventLoop.add_reader` method to register the file descriptor of a :meth:`loop.add_reader` method to watch a file descriptor.
socket.
Doc/library/asyncio-subprocess.rst
View file @ 7c7605ff
...@@ -2,137 +2,91 @@ ...@@ -2,137 +2,91 @@
.. _asyncio-subprocess: .. _asyncio-subprocess:
Subprocess ============
========== Subprocesses
============
**Source code:** :source:`Lib/asyncio/subprocess.py` This section describes high-level async/await asyncio APIs to
create and manage subprocesses.
Windows event loop Here's an example of how asyncio can run a shell command and
------------------ communicate its result back::
On Windows, the default event loop is :class:`SelectorEventLoop` which does not import asyncio
support subprocesses. :class:`ProactorEventLoop` should be used instead.
Example to use it on Windows::
import asyncio, sys
if sys.platform == 'win32':
loop = asyncio.ProactorEventLoop()
asyncio.set_event_loop(loop)
.. seealso::
:ref:`Available event loops <asyncio-event-loops>` and :ref:`Platform
support <asyncio-platform-support>`.
Create a subprocess: high-level API using Process
-------------------------------------------------
.. coroutinefunction:: create_subprocess_exec(\*args, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds)
Create a subprocess.
The *limit* parameter sets the buffer limit passed to the
:class:`StreamReader`. See :meth:`AbstractEventLoop.subprocess_exec` for other
parameters.
Return a :class:`~asyncio.subprocess.Process` instance.
This function is a :ref:`coroutine <coroutine>`.
.. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds)
Run the shell command *cmd*.
The *limit* parameter sets the buffer limit passed to the
:class:`StreamReader`. See :meth:`AbstractEventLoop.subprocess_shell` for other
parameters.
Return a :class:`~asyncio.subprocess.Process` instance. async def run(cmd):
proc = await asyncio.create_subprocess_shell(
cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE)
It is the application's responsibility to ensure that all whitespace and stdout, stderr = await proc.communicate()
metacharacters are quoted appropriately to avoid `shell injection
<https://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_
vulnerabilities. The :func:`shlex.quote` function can be used to properly
escape whitespace and shell metacharacters in strings that are going to be
used to construct shell commands.
This function is a :ref:`coroutine <coroutine>`. print(f'[{cmd!r} exited with {proc.returncode}]')
if stdout:
print(f'[stdout]\n{stdout.decode()}')
if stderr:
print(f'[stderr]\n{stderr.decode()}')
Use the :meth:`AbstractEventLoop.connect_read_pipe` and asyncio.run(run('ls /zzz'))
:meth:`AbstractEventLoop.connect_write_pipe` methods to connect pipes.
will print::
Create a subprocess: low-level API using subprocess.Popen ['ls /zzz' exited with 1]
--------------------------------------------------------- [stderr]
ls: /zzz: No such file or directory
Run subprocesses asynchronously using the :mod:`subprocess` module. Because all asyncio subprocess functions are asynchronous and asyncio
provides many tools to work with such functions, it is easy to execute
and monitor multiple subprocesses in parallel. It is indeed trivial
to modify the above example to run a few commands at once::
.. coroutinemethod:: AbstractEventLoop.subprocess_exec(protocol_factory, \*args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) async def main():
await asyncio.gather(
run('ls /zzz'),
run('sleep 1; echo "hello"'))
Create a subprocess from one or more string arguments (character strings or asyncio.run(main())
bytes strings encoded to the :ref:`filesystem encoding
<filesystem-encoding>`), where the first string
specifies the program to execute, and the remaining strings specify the
program's arguments. (Thus, together the string arguments form the
``sys.argv`` value of the program, assuming it is a Python script.) This is
similar to the standard library :class:`subprocess.Popen` class called with
shell=False and the list of strings passed as the first argument;
however, where :class:`~subprocess.Popen` takes a single argument which is
list of strings, :func:`subprocess_exec` takes multiple string arguments.
The *protocol_factory* must instantiate a subclass of the See also the `Examples`_ subsection.
:class:`asyncio.SubprocessProtocol` class.
Other parameters:
* *stdin*: Either a file-like object representing the pipe to be connected Creating Subprocesses
to the subprocess's standard input stream using =====================
:meth:`~AbstractEventLoop.connect_write_pipe`, or the constant
:const:`subprocess.PIPE` (the default). By default a new pipe will be
created and connected.
* *stdout*: Either a file-like object representing the pipe to be connected .. coroutinefunction:: create_subprocess_exec(\*args, stdin=None, \
to the subprocess's standard output stream using stdout=None, stderr=None, loop=None, \
:meth:`~AbstractEventLoop.connect_read_pipe`, or the constant limit=None, \*\*kwds)
:const:`subprocess.PIPE` (the default). By default a new pipe will be
created and connected.
* *stderr*: Either a file-like object representing the pipe to be connected Create a subprocess.
to the subprocess's standard error stream using
:meth:`~AbstractEventLoop.connect_read_pipe`, or one of the constants
:const:`subprocess.PIPE` (the default) or :const:`subprocess.STDOUT`.
By default a new pipe will be created and connected. When
:const:`subprocess.STDOUT` is specified, the subprocess's standard error
stream will be connected to the same pipe as the standard output stream.
* All other keyword arguments are passed to :class:`subprocess.Popen` The *limit* argument sets the buffer limit for :class:`StreamReader`
without interpretation, except for *bufsize*, *universal_newlines* and wrappers for :attr:`Process.stdout` and :attr:`Process.stderr`
*shell*, which should not be specified at all. (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr*
arguments).
Returns a pair of ``(transport, protocol)``, where *transport* is an Return a :class:`~asyncio.subprocess.Process` instance.
instance of :class:`BaseSubprocessTransport`.
This method is a :ref:`coroutine <coroutine>`. See the documentation of :meth:`loop.subprocess_exec` for other
parameters.
See the constructor of the :class:`subprocess.Popen` class for parameters. .. coroutinefunction:: create_subprocess_shell(cmd, stdin=None, \
stdout=None, stderr=None, loop=None, \
limit=None, \*\*kwds)
.. coroutinemethod:: AbstractEventLoop.subprocess_shell(protocol_factory, cmd, \*, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, \*\*kwargs) Run the shell command *cmd*.
Create a subprocess from *cmd*, which is a character string or a bytes The *limit* argument sets the buffer limit for :class:`StreamReader`
string encoded to the :ref:`filesystem encoding <filesystem-encoding>`, wrappers for :attr:`Process.stdout` and :attr:`Process.stderr`
using the platform's "shell" syntax. This is similar to the standard library (if :attr:`subprocess.PIPE` is passed to *stdout* and *stderr*
:class:`subprocess.Popen` class called with ``shell=True``. arguments).
The *protocol_factory* must instantiate a subclass of the Return a :class:`~asyncio.subprocess.Process` instance.
:class:`asyncio.SubprocessProtocol` class.
See :meth:`~AbstractEventLoop.subprocess_exec` for more details about See the documentation of :meth:`loop.subprocess_shell` for other
the remaining arguments. parameters.
Returns a pair of ``(transport, protocol)``, where *transport* is an .. note::
instance of :class:`BaseSubprocessTransport`.
It is the application's responsibility to ensure that all whitespace and It is the application's responsibility to ensure that all whitespace and
metacharacters are quoted appropriately to avoid `shell injection metacharacters are quoted appropriately to avoid `shell injection
...@@ -141,106 +95,125 @@ Run subprocesses asynchronously using the :mod:`subprocess` module. ...@@ -141,106 +95,125 @@ Run subprocesses asynchronously using the :mod:`subprocess` module.
escape whitespace and shell metacharacters in strings that are going to be escape whitespace and shell metacharacters in strings that are going to be
used to construct shell commands. used to construct shell commands.
This method is a :ref:`coroutine <coroutine>`. .. note::
The default event loop that asyncio is pre-configured
to use on **Windows** does not support subprocesses.
See :ref:`Subprocess Support on Windows <asyncio-windows-subprocess>`
for details.
.. seealso:: .. seealso::
The :meth:`AbstractEventLoop.connect_read_pipe` and asyncio has also *low-level* APIs to work with subprocesses:
:meth:`AbstractEventLoop.connect_write_pipe` methods. :meth:`loop.subprocess_exec`, :meth:`loop.subprocess_shell`,
:meth:`loop.connect_read_pipe`, :meth:`loop.connect_write_pipe`,
as well as the :ref:`Subprocess Transports <asyncio-subprocess-transports>`
and :ref:`Subprocess Protocols <asyncio-subprocess-protocols>`.
Constants Constants
--------- =========
.. data:: asyncio.subprocess.PIPE .. data:: asyncio.subprocess.PIPE
Special value that can be used as the *stdin*, *stdout* or *stderr* argument Can be passed to the *stdin*, *stdout* or *stderr* parameters.
to :func:`create_subprocess_shell` and :func:`create_subprocess_exec` and
indicates that a pipe to the standard stream should be opened. If *PIPE* is passed to *stdin* argument, the
:attr:`Process.stdin <asyncio.subprocess.Process.stdin>` attribute
will point to a :class:`StreamWriter` instance.
If *PIPE* is passed to *stdout* or *stderr* arguments, the
:attr:`Process.stdout <asyncio.subprocess.Process.stdout>` and
:attr:`Process.stderr <asyncio.subprocess.Process.stderr>`
attributes will point to :class:`StreamReader` instances.
.. data:: asyncio.subprocess.STDOUT .. data:: asyncio.subprocess.STDOUT
Special value that can be used as the *stderr* argument to Can be passed to the *stderr* parameter to redirect process'
:func:`create_subprocess_shell` and :func:`create_subprocess_exec` and *stderr* to *stdout*.
indicates that standard error should go into the same handle as standard
output.
.. data:: asyncio.subprocess.DEVNULL .. data:: asyncio.subprocess.DEVNULL
Special value that can be used as the *stdin*, *stdout* or *stderr* argument Can be passed as the *stdin*, *stdout* or *stderr* parameters
to :func:`create_subprocess_shell` and :func:`create_subprocess_exec` and to redirect the corresponding subprocess' IO to :data:`os.devnull`.
indicates that the special file :data:`os.devnull` will be used.
Process Interacting with Subprocesses
------- =============================
Both :func:`create_subprocess_exec` and :func:`create_subprocess_shell`
functions return instances of the *Process* class. It is a high-level
wrapper that allows to watch for subprocesses completion and
communicate with them.
.. class:: asyncio.subprocess.Process .. class:: asyncio.subprocess.Process
A subprocess created by the :func:`create_subprocess_exec` or the An object that wraps OS processes created by the
:func:`create_subprocess_shell` function. :func:`create_subprocess_exec` and :func:`create_subprocess_shell`
functions.
This class is designed to have a similar API to the
:class:`subprocess.Popen` class, but there are some
notable differences:
* unlike Popen, Process instances do not have an equivalent to
the :meth:`~subprocess.Popen.poll` method;
* the :meth:`~asyncio.subprocess.Process.communicate` and
:meth:`~asyncio.subprocess.Process.wait` methods don't take a
*timeout* parameter: use the :func:`wait_for` function;
* the :meth:`Process.wait() <asyncio.subprocess.Process.wait>` method
is asynchronous, whereas :meth:`subprocess.Popen.wait` method
is implemented as a blocking busy loop;
The API of the :class:`~asyncio.subprocess.Process` class was designed to be * the *universal_newlines* parameter is not supported.
close to the API of the :class:`subprocess.Popen` class, but there are some
differences:
* There is no explicit :meth:`~subprocess.Popen.poll` method This class is :ref:`not thread safe <asyncio-multithreading>`.
* The :meth:`~subprocess.Popen.communicate` and
:meth:`~subprocess.Popen.wait` methods don't take a *timeout* parameter:
use the :func:`wait_for` function
* The *universal_newlines* parameter is not supported (only bytes strings
are supported)
* The :meth:`~asyncio.subprocess.Process.wait` method of
the :class:`~asyncio.subprocess.Process` class is asynchronous whereas the
:meth:`~subprocess.Popen.wait` method of the :class:`~subprocess.Popen`
class is implemented as a busy loop.
This class is :ref:`not thread safe <asyncio-multithreading>`. See also the See also the :ref:`Subprocess and Threads <asyncio-subprocess-threads>`
:ref:`Subprocess and threads <asyncio-subprocess-threads>` section. section.
.. coroutinemethod:: wait() .. coroutinemethod:: wait()
Wait for child process to terminate. Set and return :attr:`returncode` Wait for child process to terminate.
attribute.
This method is a :ref:`coroutine <coroutine>`. Set and return the :attr:`returncode` attribute.
.. note:: .. note::
This will deadlock when using ``stdout=PIPE`` or ``stderr=PIPE`` and This method can deadlock when using ``stdout=PIPE`` or
the child process generates enough output to a pipe such that it ``stderr=PIPE`` and the child process generates so much output
blocks waiting for the OS pipe buffer to accept more data. Use the that it blocks waiting for the OS pipe buffer to accept
:meth:`communicate` method when using pipes to avoid that. more data. Use the :meth:`communicate` method when using pipes
to avoid this condition.
.. coroutinemethod:: communicate(input=None) .. coroutinemethod:: communicate(input=None)
Interact with process: Send data to stdin. Read data from stdout and Interact with process:
stderr, until end-of-file is reached. Wait for process to terminate.
The optional *input* argument should be data to be sent to the child
process, or ``None``, if no data should be sent to the child. The type
of *input* must be bytes.
:meth:`communicate` returns a tuple ``(stdout_data, stderr_data)``. 1. send data to *stdin* (if *input* is not ``None``);
2. read data from *stdout* and *stderr*, until EOF is reached;
3. wait for process to terminate.
If a :exc:`BrokenPipeError` or :exc:`ConnectionResetError` exception is The optional *input* argument is the data (:class:`bytes` object)
raised when writing *input* into stdin, the exception is ignored. It that will be sent to the child process.
occurs when the process exits before all data are written into stdin.
Note that if you want to send data to the process's stdin, you need to Return a tuple ``(stdout_data, stderr_data)``.
create the Process object with ``stdin=PIPE``. Similarly, to get anything
other than ``None`` in the result tuple, you need to give ``stdout=PIPE``
and/or ``stderr=PIPE`` too.
This method is a :ref:`coroutine <coroutine>`. If either :exc:`BrokenPipeError` or :exc:`ConnectionResetError`
exception is raised when writing *input* into *stdin*, the
exception is ignored. This condition occurs when the process
exits before all data are written into *stdin*.
.. note:: If its desired to send data to the process' *stdin*,
the process needs to be created with ``stdin=PIPE``. Similarly,
The data read is buffered in memory, so do not use this method if the to get anything other than ``None`` in the result tuple, the
data size is large or unlimited. process has to be created with ``stdout=PIPE`` and/or
``stderr=PIPE`` arguments.
.. versionchanged:: 3.4.2 Note, that the data read is buffered in memory, so do not use
The method now ignores :exc:`BrokenPipeError` and this method if the data size is large or unlimited.
:exc:`ConnectionResetError`.
.. method:: send_signal(signal) .. method:: send_signal(signal)
...@@ -255,67 +228,81 @@ Process ...@@ -255,67 +228,81 @@ Process
.. method:: terminate() .. method:: terminate()
Stop the child. On Posix OSs the method sends :py:data:`signal.SIGTERM` Stop the child.
to the child. On Windows the Win32 API function
:c:func:`TerminateProcess` is called to stop the child. On Posix OSs the method sends :py:data:`signal.SIGTERM` to the
child process.
On Windows the Win32 API function :c:func:`TerminateProcess` is
called to stop the child process.
.. method:: kill() .. method:: kill()
Kills the child. On Posix OSs the function sends :py:data:`SIGKILL` to Kill the child.
the child. On Windows :meth:`kill` is an alias for :meth:`terminate`.
On Posix OSs the function sends :py:data:`SIGKILL` to the child
process.
On Windows this method is an alias for :meth:`terminate`.
.. attribute:: stdin .. attribute:: stdin
Standard input stream (:class:`StreamWriter`), ``None`` if the process Standard input stream (:class:`StreamWriter`) or ``None``
was created with ``stdin=None``. if the process was created with ``stdin=None``.
.. attribute:: stdout .. attribute:: stdout
Standard output stream (:class:`StreamReader`), ``None`` if the process Standard output stream (:class:`StreamReader`) or ``None``
was created with ``stdout=None``. if the process was created with ``stdout=None``.
.. attribute:: stderr .. attribute:: stderr
Standard error stream (:class:`StreamReader`), ``None`` if the process Standard error stream (:class:`StreamReader`) or ``None``
was created with ``stderr=None``. if the process was created with ``stderr=None``.
.. warning:: .. warning::
Use the :meth:`communicate` method rather than :attr:`.stdin.write Use the :meth:`communicate` method rather than
<stdin>`, :attr:`.stdout.read <stdout>` or :attr:`.stderr.read <stderr>` :attr:`process.stdin.write() <stdin>`,
to avoid deadlocks due to streams pausing reading or writing and blocking :attr:`await process.stdout.read() <stdout>` or
the child process. :attr:`await process.stderr.read <stderr>`
to avoid deadlocks due to streams pausing reading or writing
and blocking the child process.
.. attribute:: pid .. attribute:: pid
The identifier of the process. Process identification number (PID).
Note that for processes created by the :func:`create_subprocess_shell` Note that for processes created by the :func:`create_subprocess_shell`
function, this attribute is the process identifier of the spawned shell. function, this attribute is the PID of the spawned shell.
.. attribute:: returncode .. attribute:: returncode
Return code of the process when it exited. A ``None`` value indicates Return code of the process when it exits.
that the process has not terminated yet.
A ``None`` value indicates that the process has not terminated yet.
A negative value ``-N`` indicates that the child was terminated by signal A negative value ``-N`` indicates that the child was terminated
``N`` (Unix only). by signal ``N`` (Unix only).
.. _asyncio-subprocess-threads: .. _asyncio-subprocess-threads:
Subprocess and threads Subprocess and Threads
---------------------- ----------------------
asyncio supports running subprocesses from different threads, but there asyncio built-in event loops support running subprocesses from
are limits: different threads, but there are the following limitations:
* An event loop must run in the main thread * An event loop must run in the main thread.
* The child watcher must be instantiated in the main thread, before executing
subprocesses from other threads. Call the :func:`get_child_watcher`
function in the main thread to instantiate the child watcher.
The :class:`asyncio.subprocess.Process` class is not thread safe. * The child watcher must be instantiated in the main thread,
before executing subprocesses from other threads. Call the
:func:`get_child_watcher` function in the main thread to instantiate
the child watcher.
Note, that alternative event loop implementations might not share
the above limitations; please refer to their documentation.
.. seealso:: .. seealso::
...@@ -323,97 +310,43 @@ The :class:`asyncio.subprocess.Process` class is not thread safe. ...@@ -323,97 +310,43 @@ The :class:`asyncio.subprocess.Process` class is not thread safe.
<asyncio-multithreading>` section. <asyncio-multithreading>` section.
Subprocess examples Examples
------------------- --------
Subprocess using transport and protocol
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Example of a subprocess protocol using to get the output of a subprocess and to
wait for the subprocess exit. The subprocess is created by the
:meth:`AbstractEventLoop.subprocess_exec` method::
import asyncio
import sys
class DateProtocol(asyncio.SubprocessProtocol):
def __init__(self, exit_future):
self.exit_future = exit_future
self.output = bytearray()
def pipe_data_received(self, fd, data):
self.output.extend(data)
def process_exited(self):
self.exit_future.set_result(True)
async def get_date(loop):
code = 'import datetime; print(datetime.datetime.now())'
exit_future = asyncio.Future(loop=loop)
# Create the subprocess controlled by the protocol DateProtocol,
# redirect the standard output into a pipe
transport, protocol = await loop.subprocess_exec(
lambda: DateProtocol(exit_future),
sys.executable, '-c', code,
stdin=None, stderr=None)
# Wait for the subprocess exit using the process_exited() method
# of the protocol
await exit_future
# Close the stdout pipe
transport.close()
# Read the output which was collected by the pipe_data_received() An example using the :class:`~asyncio.subprocess.Process` class to
# method of the protocol control a subprocess and the :class:`StreamReader` class to read from
data = bytes(protocol.output) the *stdout*.
return data.decode('ascii').rstrip()
if sys.platform == "win32": The subprocess is created by the :func:`create_subprocess_exec`
loop = asyncio.ProactorEventLoop()
asyncio.set_event_loop(loop)
else:
loop = asyncio.get_event_loop()
date = loop.run_until_complete(get_date(loop))
print("Current date: %s" % date)
loop.close()
Subprocess using streams
^^^^^^^^^^^^^^^^^^^^^^^^
Example using the :class:`~asyncio.subprocess.Process` class to control the
subprocess and the :class:`StreamReader` class to read from the standard
output. The subprocess is created by the :func:`create_subprocess_exec`
function:: function::
import asyncio.subprocess import asyncio
import sys import sys
async def get_date(): async def get_date():
code = 'import datetime; print(datetime.datetime.now())' code = 'import datetime; print(datetime.datetime.now())'
# Create the subprocess, redirect the standard output into a pipe # Create the subprocess; redirect the standard output
# into a pipe.
proc = await asyncio.create_subprocess_exec( proc = await asyncio.create_subprocess_exec(
sys.executable, '-c', code, sys.executable, '-c', code,
stdout=asyncio.subprocess.PIPE) stdout=asyncio.subprocess.PIPE)
# Read one line of output # Read one line of output.
data = await proc.stdout.readline() data = await proc.stdout.readline()
line = data.decode('ascii').rstrip() line = data.decode('ascii').rstrip()
# Wait for the subprocess exit # Wait for the subprocess exit.
await proc.wait() await proc.wait()
return line return line
if sys.platform == "win32": if sys.platform == "win32":
loop = asyncio.ProactorEventLoop() asyncio.set_event_loop_policy(
asyncio.set_event_loop(loop) asyncio.WindowsProactorEventLoopPolicy())
else:
loop = asyncio.get_event_loop() date = asyncio.run(get_date())
print(f"Current date: {date}")
date = loop.run_until_complete(get_date())
print("Current date: %s" % date)
loop.close() See also the :ref:`same example <asyncio-subprocess-proto-example>`
written using low-level APIs.
Doc/library/asyncio-task.rst
View file @ 7c7605ff
...@@ -63,7 +63,7 @@ the coroutine object returned by the call doesn't do anything until you ...@@ -63,7 +63,7 @@ the coroutine object returned by the call doesn't do anything until you
schedule its execution. There are two basic ways to start it running: schedule its execution. There are two basic ways to start it running:
call ``await coroutine`` or ``yield from coroutine`` from another coroutine call ``await coroutine`` or ``yield from coroutine`` from another coroutine
(assuming the other coroutine is already running!), or schedule its execution (assuming the other coroutine is already running!), or schedule its execution
using the :func:`ensure_future` function or the :meth:`AbstractEventLoop.create_task` using the :func:`ensure_future` function or the :meth:`loop.create_task`
method. method.
...@@ -129,7 +129,7 @@ Example of coroutine displaying ``"Hello World"``:: ...@@ -129,7 +129,7 @@ Example of coroutine displaying ``"Hello World"``::
.. seealso:: .. seealso::
The :ref:`Hello World with call_soon() <asyncio-hello-world-callback>` The :ref:`Hello World with call_soon() <asyncio-hello-world-callback>`
example uses the :meth:`AbstractEventLoop.call_soon` method to schedule a example uses the :meth:`loop.call_soon` method to schedule a
callback. callback.
...@@ -159,7 +159,7 @@ using the :meth:`sleep` function:: ...@@ -159,7 +159,7 @@ using the :meth:`sleep` function::
The :ref:`display the current date with call_later() The :ref:`display the current date with call_later()
<asyncio-date-callback>` example uses a callback with the <asyncio-date-callback>` example uses a callback with the
:meth:`AbstractEventLoop.call_later` method. :meth:`loop.call_later` method.
Example: Chain coroutines Example: Chain coroutines
...@@ -190,32 +190,12 @@ Sequence diagram of the example: ...@@ -190,32 +190,12 @@ Sequence diagram of the example:
.. image:: tulip_coro.png .. image:: tulip_coro.png
:align: center :align: center
The "Task" is created by the :meth:`AbstractEventLoop.run_until_complete` method The "Task" is created by the :meth:`loop.run_until_complete` method
when it gets a coroutine object instead of a task. when it gets a coroutine object instead of a task.
The diagram shows the control flow, it does not describe exactly how things The diagram shows the control flow, it does not describe exactly how things
work internally. For example, the sleep coroutine creates an internal future work internally. For example, the sleep coroutine creates an internal future
which uses :meth:`AbstractEventLoop.call_later` to wake up the task in 1 second. which uses :meth:`loop.call_later` to wake up the task in 1 second.
InvalidStateError
-----------------
.. exception:: InvalidStateError
The operation is not allowed in this state.
TimeoutError
------------
.. exception:: TimeoutError
The operation exceeded the given deadline.
.. note::
This exception is different from the builtin :exc:`TimeoutError` exception!
Future Future
...@@ -231,7 +211,7 @@ Future ...@@ -231,7 +211,7 @@ Future
raise an exception when the future isn't done yet. raise an exception when the future isn't done yet.
- Callbacks registered with :meth:`add_done_callback` are always called - Callbacks registered with :meth:`add_done_callback` are always called
via the event loop's :meth:`~AbstractEventLoop.call_soon`. via the event loop's :meth:`loop.call_soon`.
- This class is not compatible with the :func:`~concurrent.futures.wait` and - This class is not compatible with the :func:`~concurrent.futures.wait` and
:func:`~concurrent.futures.as_completed` functions in the :func:`~concurrent.futures.as_completed` functions in the
...@@ -281,7 +261,7 @@ Future ...@@ -281,7 +261,7 @@ Future
The *callback* is called with a single argument - the future object. If the The *callback* is called with a single argument - the future object. If the
future is already done when this is called, the callback is scheduled future is already done when this is called, the callback is scheduled
with :meth:`~AbstractEventLoop.call_soon`. with :meth:`loop.call_soon`.
An optional keyword-only *context* argument allows specifying a custom An optional keyword-only *context* argument allows specifying a custom
:class:`contextvars.Context` for the *callback* to run in. The current :class:`contextvars.Context` for the *callback* to run in. The current
...@@ -344,11 +324,11 @@ Example combining a :class:`Future` and a :ref:`coroutine function ...@@ -344,11 +324,11 @@ Example combining a :class:`Future` and a :ref:`coroutine function
The coroutine function is responsible for the computation (which takes 1 second) The coroutine function is responsible for the computation (which takes 1 second)
and it stores the result into the future. The and it stores the result into the future. The
:meth:`~AbstractEventLoop.run_until_complete` method waits for the completion of :meth:`loop.run_until_complete` method waits for the completion of
the future. the future.
.. note:: .. note::
The :meth:`~AbstractEventLoop.run_until_complete` method uses internally the The :meth:`loop.run_until_complete` method uses internally the
:meth:`~Future.add_done_callback` method to be notified when the future is :meth:`~Future.add_done_callback` method to be notified when the future is
done. done.
...@@ -433,7 +413,7 @@ Task ...@@ -433,7 +413,7 @@ Task
logged: see :ref:`Pending task destroyed <asyncio-pending-task-destroyed>`. logged: see :ref:`Pending task destroyed <asyncio-pending-task-destroyed>`.
Don't directly create :class:`Task` instances: use the :func:`create_task` Don't directly create :class:`Task` instances: use the :func:`create_task`
function or the :meth:`AbstractEventLoop.create_task` method. function or the :meth:`loop.create_task` method.
Tasks support the :mod:`contextvars` module. When a Task Tasks support the :mod:`contextvars` module. When a Task
is created it copies the current context and later runs its coroutine is created it copies the current context and later runs its coroutine
...@@ -644,7 +624,7 @@ Task functions ...@@ -644,7 +624,7 @@ Task functions
.. seealso:: .. seealso::
The :func:`create_task` function and The :func:`create_task` function and
:meth:`AbstractEventLoop.create_task` method. :meth:`loop.create_task` method.
.. function:: wrap_future(future, \*, loop=None) .. function:: wrap_future(future, \*, loop=None)
......
Doc/library/asyncio.rst
View file @ 7c7605ff
...@@ -50,7 +50,8 @@ Table of contents: ...@@ -50,7 +50,8 @@ Table of contents:
:maxdepth: 3 :maxdepth: 3
asyncio-eventloop.rst asyncio-eventloop.rst
asyncio-eventloops.rst asyncio-policy.rst
asyncio-platforms.rst
asyncio-task.rst asyncio-task.rst
asyncio-protocol.rst asyncio-protocol.rst
asyncio-stream.rst asyncio-stream.rst
...@@ -58,6 +59,7 @@ Table of contents: ...@@ -58,6 +59,7 @@ Table of contents:
asyncio-sync.rst asyncio-sync.rst
asyncio-queue.rst asyncio-queue.rst
asyncio-dev.rst asyncio-dev.rst
asyncio-exceptions.rst
.. seealso:: .. seealso::
......
Doc/library/ipc.rst
View file @ 7c7605ff
...@@ -16,11 +16,11 @@ The list of modules described in this chapter is: ...@@ -16,11 +16,11 @@ The list of modules described in this chapter is:
.. toctree:: .. toctree::
asyncio.rst
socket.rst socket.rst
ssl.rst ssl.rst
select.rst select.rst
selectors.rst selectors.rst
asyncio.rst
asyncore.rst asyncore.rst
asynchat.rst asynchat.rst
signal.rst signal.rst
......
Doc/whatsnew/3.5.rst
View file @ 7c7605ff
...@@ -777,18 +777,18 @@ all changes introduced in Python 3.5 have also been backported to Python 3.4.x. ...@@ -777,18 +777,18 @@ all changes introduced in Python 3.5 have also been backported to Python 3.4.x.
Notable changes in the :mod:`asyncio` module since Python 3.4.0: Notable changes in the :mod:`asyncio` module since Python 3.4.0:
* New debugging APIs: :meth:`loop.set_debug() <asyncio.BaseEventLoop.set_debug>` * New debugging APIs: :meth:`loop.set_debug() <asyncio.loop.set_debug>`
and :meth:`loop.get_debug() <asyncio.BaseEventLoop.get_debug>` methods. and :meth:`loop.get_debug() <asyncio.loop.get_debug>` methods.
(Contributed by Victor Stinner.) (Contributed by Victor Stinner.)
* The proactor event loop now supports SSL. * The proactor event loop now supports SSL.
(Contributed by Antoine Pitrou and Victor Stinner in :issue:`22560`.) (Contributed by Antoine Pitrou and Victor Stinner in :issue:`22560`.)
* A new :meth:`loop.is_closed() <asyncio.BaseEventLoop.is_closed>` method to * A new :meth:`loop.is_closed() <asyncio.loop.is_closed>` method to
check if the event loop is closed. check if the event loop is closed.
(Contributed by Victor Stinner in :issue:`21326`.) (Contributed by Victor Stinner in :issue:`21326`.)
* A new :meth:`loop.create_task() <asyncio.BaseEventLoop.create_task>` * A new :meth:`loop.create_task() <asyncio.loop.create_task>`
to conveniently create and schedule a new :class:`~asyncio.Task` to conveniently create and schedule a new :class:`~asyncio.Task`
for a coroutine. The ``create_task`` method is also used by all for a coroutine. The ``create_task`` method is also used by all
asyncio functions that wrap coroutines into tasks, such as asyncio functions that wrap coroutines into tasks, such as
...@@ -805,10 +805,10 @@ Notable changes in the :mod:`asyncio` module since Python 3.4.0: ...@@ -805,10 +805,10 @@ Notable changes in the :mod:`asyncio` module since Python 3.4.0:
(Contributed by Yury Selivanov.) (Contributed by Yury Selivanov.)
* New :meth:`loop.set_task_factory() * New :meth:`loop.set_task_factory()
<asyncio.AbstractEventLoop.set_task_factory>` and <asyncio.loop.set_task_factory>` and
:meth:`loop.get_task_factory() <asyncio.AbstractEventLoop.get_task_factory>` :meth:`loop.get_task_factory() <asyncio.loop.get_task_factory>`
methods to customize the task factory that :meth:`loop.create_task() methods to customize the task factory that :meth:`loop.create_task()
<asyncio.BaseEventLoop.create_task>` method uses. (Contributed by Yury <asyncio.loop.create_task>` method uses. (Contributed by Yury
Selivanov.) Selivanov.)
* New :meth:`Queue.join() <asyncio.Queue.join>` and * New :meth:`Queue.join() <asyncio.Queue.join>` and
...@@ -822,7 +822,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.4.0: ...@@ -822,7 +822,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.4.0:
Updates in 3.5.1: Updates in 3.5.1:
* The :func:`~asyncio.ensure_future` function and all functions that * The :func:`~asyncio.ensure_future` function and all functions that
use it, such as :meth:`loop.run_until_complete() <asyncio.BaseEventLoop.run_until_complete>`, use it, such as :meth:`loop.run_until_complete() <asyncio.loop.run_until_complete>`,
now accept all kinds of :term:`awaitable objects <awaitable>`. now accept all kinds of :term:`awaitable objects <awaitable>`.
(Contributed by Yury Selivanov.) (Contributed by Yury Selivanov.)
...@@ -834,20 +834,20 @@ Updates in 3.5.1: ...@@ -834,20 +834,20 @@ Updates in 3.5.1:
method to check if the transport is closing or closed. method to check if the transport is closing or closed.
(Contributed by Yury Selivanov.) (Contributed by Yury Selivanov.)
* The :meth:`loop.create_server() <asyncio.BaseEventLoop.create_server>` * The :meth:`loop.create_server() <asyncio.loop.create_server>`
method can now accept a list of hosts. method can now accept a list of hosts.
(Contributed by Yann Sionneau.) (Contributed by Yann Sionneau.)
Updates in 3.5.2: Updates in 3.5.2:
* New :meth:`loop.create_future() <asyncio.BaseEventLoop.create_future>` * New :meth:`loop.create_future() <asyncio.loop.create_future>`
method to create Future objects. This allows alternative event method to create Future objects. This allows alternative event
loop implementations, such as loop implementations, such as
`uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster `uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster
:class:`asyncio.Future` implementation. :class:`asyncio.Future` implementation.
(Contributed by Yury Selivanov.) (Contributed by Yury Selivanov.)
* New :meth:`loop.get_exception_handler() <asyncio.BaseEventLoop.get_exception_handler>` * New :meth:`loop.get_exception_handler() <asyncio.loop.get_exception_handler>`
method to get the current exception handler. method to get the current exception handler.
(Contributed by Yury Selivanov.) (Contributed by Yury Selivanov.)
...@@ -856,13 +856,13 @@ Updates in 3.5.2: ...@@ -856,13 +856,13 @@ Updates in 3.5.2:
sequence appears. sequence appears.
(Contributed by Mark Korenberg.) (Contributed by Mark Korenberg.)
* The :meth:`loop.create_connection() <asyncio.BaseEventLoop.create_connection>` * The :meth:`loop.create_connection() <asyncio.loop.create_connection>`
and :meth:`loop.create_server() <asyncio.BaseEventLoop.create_server>` and :meth:`loop.create_server() <asyncio.loop.create_server>`
methods are optimized to avoid calling the system ``getaddrinfo`` methods are optimized to avoid calling the system ``getaddrinfo``
function if the address is already resolved. function if the address is already resolved.
(Contributed by A. Jesse Jiryu Davis.) (Contributed by A. Jesse Jiryu Davis.)
* The :meth:`loop.sock_connect(sock, address) <asyncio.BaseEventLoop.sock_connect>` * The :meth:`loop.sock_connect(sock, address) <asyncio.loop.sock_connect>`
no longer requires the *address* to be resolved prior to the call. no longer requires the *address* to be resolved prior to the call.
(Contributed by A. Jesse Jiryu Davis.) (Contributed by A. Jesse Jiryu Davis.)
......
Doc/whatsnew/3.6.rst
View file @ 7c7605ff
...@@ -824,7 +824,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0 ...@@ -824,7 +824,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0
(Contributed by Yury Selivanov in :issue:`28613`.) (Contributed by Yury Selivanov in :issue:`28613`.)
* The :func:`~asyncio.ensure_future` function and all functions that * The :func:`~asyncio.ensure_future` function and all functions that
use it, such as :meth:`loop.run_until_complete() <asyncio.BaseEventLoop.run_until_complete>`, use it, such as :meth:`loop.run_until_complete() <asyncio.loop.run_until_complete>`,
now accept all kinds of :term:`awaitable objects <awaitable>`. now accept all kinds of :term:`awaitable objects <awaitable>`.
(Contributed by Yury Selivanov.) (Contributed by Yury Selivanov.)
...@@ -836,18 +836,18 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0 ...@@ -836,18 +836,18 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0
method to check if the transport is closing or closed. method to check if the transport is closing or closed.
(Contributed by Yury Selivanov.) (Contributed by Yury Selivanov.)
* The :meth:`loop.create_server() <asyncio.BaseEventLoop.create_server>` * The :meth:`loop.create_server() <asyncio.loop.create_server>`
method can now accept a list of hosts. method can now accept a list of hosts.
(Contributed by Yann Sionneau.) (Contributed by Yann Sionneau.)
* New :meth:`loop.create_future() <asyncio.BaseEventLoop.create_future>` * New :meth:`loop.create_future() <asyncio.loop.create_future>`
method to create Future objects. This allows alternative event method to create Future objects. This allows alternative event
loop implementations, such as loop implementations, such as
`uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster `uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster
:class:`asyncio.Future` implementation. :class:`asyncio.Future` implementation.
(Contributed by Yury Selivanov in :issue:`27041`.) (Contributed by Yury Selivanov in :issue:`27041`.)
* New :meth:`loop.get_exception_handler() <asyncio.BaseEventLoop.get_exception_handler>` * New :meth:`loop.get_exception_handler() <asyncio.loop.get_exception_handler>`
method to get the current exception handler. method to get the current exception handler.
(Contributed by Yury Selivanov in :issue:`27040`.) (Contributed by Yury Selivanov in :issue:`27040`.)
...@@ -860,12 +860,12 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0 ...@@ -860,12 +860,12 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0
has been improved. has been improved.
(Contributed by Mark Korenberg in :issue:`28370`.) (Contributed by Mark Korenberg in :issue:`28370`.)
* The :meth:`loop.getaddrinfo() <asyncio.BaseEventLoop.getaddrinfo>` * The :meth:`loop.getaddrinfo() <asyncio.loop.getaddrinfo>`
method is optimized to avoid calling the system ``getaddrinfo`` method is optimized to avoid calling the system ``getaddrinfo``
function if the address is already resolved. function if the address is already resolved.
(Contributed by A. Jesse Jiryu Davis.) (Contributed by A. Jesse Jiryu Davis.)
* The :meth:`loop.stop() <asyncio.BaseEventLoop.stop>` * The :meth:`loop.stop() <asyncio.loop.stop>`
method has been changed to stop the loop immediately after method has been changed to stop the loop immediately after
the current iteration. Any new callbacks scheduled as a result the current iteration. Any new callbacks scheduled as a result
of the last iteration will be discarded. of the last iteration will be discarded.
...@@ -876,7 +876,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0 ...@@ -876,7 +876,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0
the :exc:`StopIteration` exception. the :exc:`StopIteration` exception.
(Contributed by Chris Angelico in :issue:`26221`.) (Contributed by Chris Angelico in :issue:`26221`.)
* New :meth:`loop.connect_accepted_socket() <asyncio.BaseEventLoop.connect_accepted_socket>` * New :meth:`loop.connect_accepted_socket() <asyncio.loop.connect_accepted_socket>`
method to be used by servers that accept connections outside of asyncio, method to be used by servers that accept connections outside of asyncio,
but that use asyncio to handle them. but that use asyncio to handle them.
(Contributed by Jim Fulton in :issue:`27392`.) (Contributed by Jim Fulton in :issue:`27392`.)
...@@ -884,7 +884,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0 ...@@ -884,7 +884,7 @@ Notable changes in the :mod:`asyncio` module since Python 3.5.0
* ``TCP_NODELAY`` flag is now set for all TCP transports by default. * ``TCP_NODELAY`` flag is now set for all TCP transports by default.
(Contributed by Yury Selivanov in :issue:`27456`.) (Contributed by Yury Selivanov in :issue:`27456`.)
* New :meth:`loop.shutdown_asyncgens() <asyncio.AbstractEventLoop.shutdown_asyncgens>` * New :meth:`loop.shutdown_asyncgens() <asyncio.loop.shutdown_asyncgens>`
to properly close pending asynchronous generators before closing the to properly close pending asynchronous generators before closing the
loop. loop.
(Contributed by Yury Selivanov in :issue:`28003`.) (Contributed by Yury Selivanov in :issue:`28003`.)
......
Doc/whatsnew/3.7.rst
View file @ 7c7605ff
...@@ -637,10 +637,10 @@ include: ...@@ -637,10 +637,10 @@ include:
(Contributed by Yury Selivanov in :issue:`32314`.) (Contributed by Yury Selivanov in :issue:`32314`.)
* asyncio gained support for :mod:`contextvars`. * asyncio gained support for :mod:`contextvars`.
:meth:`loop.call_soon() <asyncio.AbstractEventLoop.call_soon>`, :meth:`loop.call_soon() <asyncio.loop.call_soon>`,
:meth:`loop.call_soon_threadsafe() <asyncio.AbstractEventLoop.call_soon_threadsafe>`, :meth:`loop.call_soon_threadsafe() <asyncio.loop.call_soon_threadsafe>`,
:meth:`loop.call_later() <asyncio.AbstractEventLoop.call_later>`, :meth:`loop.call_later() <asyncio.loop.call_later>`,
:meth:`loop.call_at() <asyncio.AbstractEventLoop.call_at>`, and :meth:`loop.call_at() <asyncio.loop.call_at>`, and
:meth:`Future.add_done_callback() <asyncio.Future.add_done_callback>` :meth:`Future.add_done_callback() <asyncio.Future.add_done_callback>`
have a new optional keyword-only *context* parameter. have a new optional keyword-only *context* parameter.
:class:`Tasks <asyncio.Task>` now track their context automatically. :class:`Tasks <asyncio.Task>` now track their context automatically.
...@@ -651,11 +651,11 @@ include: ...@@ -651,11 +651,11 @@ include:
to ``asyncio.get_event_loop().create_task()``. to ``asyncio.get_event_loop().create_task()``.
(Contributed by Andrew Svetlov in :issue:`32311`.) (Contributed by Andrew Svetlov in :issue:`32311`.)
* The new :meth:`loop.start_tls() <asyncio.AbstractEventLoop.start_tls>` * The new :meth:`loop.start_tls() <asyncio.loop.start_tls>`
method can be used to upgrade an existing connection to TLS. method can be used to upgrade an existing connection to TLS.
(Contributed by Yury Selivanov in :issue:`23749`.) (Contributed by Yury Selivanov in :issue:`23749`.)
* The new :meth:`loop.sock_recv_into() <asyncio.AbstractEventLoop.sock_recv_into>` * The new :meth:`loop.sock_recv_into() <asyncio.loop.sock_recv_into>`
method allows reading data from a socket directly into a provided buffer making method allows reading data from a socket directly into a provided buffer making
it possible to reduce data copies. it possible to reduce data copies.
(Contributed by Antoine Pitrou in :issue:`31819`.) (Contributed by Antoine Pitrou in :issue:`31819`.)
...@@ -683,13 +683,13 @@ include: ...@@ -683,13 +683,13 @@ include:
can be used to determine if the writer is closing. can be used to determine if the writer is closing.
(Contributed by Andrew Svetlov in :issue:`32391`.) (Contributed by Andrew Svetlov in :issue:`32391`.)
* The new :meth:`loop.sock_sendfile() <asyncio.AbstractEventLoop.sock_sendfile>` * The new :meth:`loop.sock_sendfile() <asyncio.loop.sock_sendfile>`
coroutine method allows sending files using :mod:`os.sendfile` when possible. coroutine method allows sending files using :mod:`os.sendfile` when possible.
(Contributed by Andrew Svetlov in :issue:`32410`.) (Contributed by Andrew Svetlov in :issue:`32410`.)
* The new :meth:`Task.get_loop() <asyncio.Task.get_loop>` and * The new :meth:`Future.get_loop() <asyncio.Future.get_loop>` and
:meth:`Future.get_loop() <asyncio.Future.get_loop>` methods ``Task.get_loop()`` methods return the instance of the loop on which a task or
return the instance of the loop on which a task or a future were created. a future were created.
:meth:`Server.get_loop() <asyncio.Server.get_loop>` allows doing the same for :meth:`Server.get_loop() <asyncio.Server.get_loop>` allows doing the same for
:class:`asyncio.Server` objects. :class:`asyncio.Server` objects.
(Contributed by Yury Selivanov in :issue:`32415` and (Contributed by Yury Selivanov in :issue:`32415` and
...@@ -698,8 +698,8 @@ include: ...@@ -698,8 +698,8 @@ include:
* It is now possible to control how instances of :class:`asyncio.Server` begin * It is now possible to control how instances of :class:`asyncio.Server` begin
serving. Previously, the server would start serving immediately when created. serving. Previously, the server would start serving immediately when created.
The new *start_serving* keyword argument to The new *start_serving* keyword argument to
:meth:`loop.create_server() <asyncio.AbstractEventLoop.create_server>` and :meth:`loop.create_server() <asyncio.loop.create_server>` and
:meth:`loop.create_unix_server() <asyncio.AbstractEventLoop.create_unix_server>`, :meth:`loop.create_unix_server() <asyncio.loop.create_unix_server>`,
as well as :meth:`Server.start_serving() <asyncio.Server.start_serving>`, and as well as :meth:`Server.start_serving() <asyncio.Server.start_serving>`, and
:meth:`Server.serve_forever() <asyncio.Server.serve_forever>` :meth:`Server.serve_forever() <asyncio.Server.serve_forever>`
can be used to decouple server instantiation and serving. The new can be used to decouple server instantiation and serving. The new
...@@ -717,20 +717,20 @@ include: ...@@ -717,20 +717,20 @@ include:
(Contributed by Yury Selivanov in :issue:`32662`.) (Contributed by Yury Selivanov in :issue:`32662`.)
* Callback objects returned by * Callback objects returned by
:func:`loop.call_later() <asyncio.AbstractEventLoop.call_later>` :func:`loop.call_later() <asyncio.loop.call_later>`
gained the new :meth:`when() <asyncio.TimerHandle.when>` method which gained the new :meth:`when() <asyncio.TimerHandle.when>` method which
returns an absolute scheduled callback timestamp. returns an absolute scheduled callback timestamp.
(Contributed by Andrew Svetlov in :issue:`32741`.) (Contributed by Andrew Svetlov in :issue:`32741`.)
* The :meth:`loop.create_datagram_endpoint() \ * The :meth:`loop.create_datagram_endpoint() \
<asyncio.AbstractEventLoop.create_datagram_endpoint>` method <asyncio.loop.create_datagram_endpoint>` method
gained support for Unix sockets. gained support for Unix sockets.
(Contributed by Quentin Dawans in :issue:`31245`.) (Contributed by Quentin Dawans in :issue:`31245`.)
* The :func:`asyncio.open_connection`, :func:`asyncio.start_server` functions, * The :func:`asyncio.open_connection`, :func:`asyncio.start_server` functions,
:meth:`loop.create_connection() <asyncio.AbstractEventLoop.create_connection>`, :meth:`loop.create_connection() <asyncio.loop.create_connection>`,
:meth:`loop.create_server() <asyncio.AbstractEventLoop.create_server>`, :meth:`loop.create_server() <asyncio.loop.create_server>`,
:meth:`loop.create_accepted_socket() <asyncio.BaseEventLoop.connect_accepted_socket>` :meth:`loop.create_accepted_socket() <asyncio.loop.connect_accepted_socket>`
methods and their corresponding UNIX socket variants now accept the methods and their corresponding UNIX socket variants now accept the
*ssl_handshake_timeout* keyword argument. *ssl_handshake_timeout* keyword argument.
(Contributed by Neil Aspinall in :issue:`29970`.) (Contributed by Neil Aspinall in :issue:`29970`.)
...@@ -2360,11 +2360,11 @@ Changes in the Python API ...@@ -2360,11 +2360,11 @@ Changes in the Python API
(Contributed by Brett Cannon in :issue:`33169`.) (Contributed by Brett Cannon in :issue:`33169`.)
* In :mod:`asyncio`, * In :mod:`asyncio`,
:meth:`loop.sock_recv() <asyncio.AbstractEventLoop.sock_recv>`, :meth:`loop.sock_recv() <asyncio.loop.sock_recv>`,
:meth:`loop.sock_sendall() <asyncio.AbstractEventLoop.sock_sendall>`, :meth:`loop.sock_sendall() <asyncio.loop.sock_sendall>`,
:meth:`loop.sock_accept() <asyncio.AbstractEventLoop.sock_accept>`, :meth:`loop.sock_accept() <asyncio.loop.sock_accept>`,
:meth:`loop.getaddrinfo() <asyncio.AbstractEventLoop.getaddrinfo>`, :meth:`loop.getaddrinfo() <asyncio.loop.getaddrinfo>`,
:meth:`loop.getnameinfo() <asyncio.AbstractEventLoop.getnameinfo>` :meth:`loop.getnameinfo() <asyncio.loop.getnameinfo>`
have been changed to be proper coroutine methods to match their have been changed to be proper coroutine methods to match their
documentation. Previously, these methods returned :class:`asyncio.Future` documentation. Previously, these methods returned :class:`asyncio.Future`
instances. instances.
......
Doc/whatsnew/3.8.rst
View file @ 7c7605ff
...@@ -170,7 +170,7 @@ Deprecated ...@@ -170,7 +170,7 @@ Deprecated
* Passing an object that is not an instance of * Passing an object that is not an instance of
:class:`concurrent.futures.ThreadPoolExecutor` to :class:`concurrent.futures.ThreadPoolExecutor` to
:meth:`asyncio.AbstractEventLoop.set_default_executor()` is :meth:`asyncio.loop.set_default_executor()` is
deprecated and will be prohibited in Python 3.9. deprecated and will be prohibited in Python 3.9.
(Contributed by Elvis Pranskevichus in :issue:`34075`.) (Contributed by Elvis Pranskevichus in :issue:`34075`.)
...@@ -264,7 +264,7 @@ Changes in the Python API ...@@ -264,7 +264,7 @@ Changes in the Python API
* Asyncio tasks can now be named, either by passing the ``name`` keyword * Asyncio tasks can now be named, either by passing the ``name`` keyword
argument to :func:`asyncio.create_task` or argument to :func:`asyncio.create_task` or
the :meth:`~asyncio.AbstractEventLoop.create_task` event loop method, or by the :meth:`~asyncio.loop.create_task` event loop method, or by
calling the :meth:`~asyncio.Task.set_name` method on the task object. The calling the :meth:`~asyncio.Task.set_name` method on the task object. The
task name is visible in the ``repr()`` output of :class:`asyncio.Task` and task name is visible in the ``repr()`` output of :class:`asyncio.Task` and
can also be retrieved using the :meth:`~asyncio.Task.get_name` method. can also be retrieved using the :meth:`~asyncio.Task.get_name` method.
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7