Skip to content

C
cpython
Commit 06aa0c62 authored by Fred Drake's avatar Fred Drake
Browse files
Options

wrap lines to <80 characters before fixing errors

parent 074d045a
Hide whitespace changes
Inline Side-by-side
Showing with 190 additions and 178 deletions
Doc/library/asynchat.rst
View file @ 06aa0c62
......@@ -9,72 +9,77 @@
This module builds on the :mod:`asyncore` infrastructure, simplifying
asynchronous clients and servers and making it easier to handle protocols whose
elements are terminated by arbitrary strings, or are of variable length.
asynchronous clients and servers and making it easier to handle protocols
whose elements are terminated by arbitrary strings, or are of variable length.
:mod:`asynchat` defines the abstract class :class:`async_chat` that you
subclass, providing implementations of the :meth:`collect_incoming_data` and
:meth:`found_terminator` methods. It uses the same asynchronous loop as
:mod:`asyncore`, and the two types of channel, :class:`asyncore.dispatcher` and
:class:`asynchat.async_chat`, can freely be mixed in the channel map. Typically
an :class:`asyncore.dispatcher` server channel generates new
:class:`asynchat.async_chat` channel objects as it receives incoming connection
requests.
:mod:`asyncore`, and the two types of channel, :class:`asyncore.dispatcher`
and :class:`asynchat.async_chat`, can freely be mixed in the channel map.
Typically an :class:`asyncore.dispatcher` server channel generates new
:class:`asynchat.async_chat` channel objects as it receives incoming
connection requests.
.. class:: async_chat()
This class is an abstract subclass of :class:`asyncore.dispatcher`. To make
practical use of the code you must subclass :class:`async_chat`, providing
meaningful :meth:`collect_incoming_data` and :meth:`found_terminator` methods.
meaningful :meth:`collect_incoming_data` and :meth:`found_terminator`
methods.
The :class:`asyncore.dispatcher` methods can be used, although not all make
sense in a message/response context.
Like :class:`asyncore.dispatcher`, :class:`async_chat` defines a set of events
that are generated by an analysis of socket conditions after a :cfunc:`select`
call. Once the polling loop has been started the :class:`async_chat` object's
methods are called by the event-processing framework with no action on the part
of the programmer.
Like :class:`asyncore.dispatcher`, :class:`async_chat` defines a set of
events that are generated by an analysis of socket conditions after a
:cfunc:`select` call. Once the polling loop has been started the
:class:`async_chat` object's methods are called by the event-processing
framework with no action on the part of the programmer.
Unlike :class:`asyncore.dispatcher`, :class:`async_chat` allows you to define a
first-in-first-out queue (fifo) of *producers*. A producer need have only one
method, :meth:`more`, which should return data to be transmitted on the channel.
Unlike :class:`asyncore.dispatcher`, :class:`async_chat` allows you to
define a first-in-first-out queue (fifo) of *producers*. A producer need
have only one method, :meth:`more`, which should return data to be
transmitted on the channel.
The producer indicates exhaustion (*i.e.* that it contains no more data) by
having its :meth:`more` method return the empty string. At this point the
:class:`async_chat` object removes the producer from the fifo and starts using
the next producer, if any. When the producer fifo is empty the
:class:`async_chat` object removes the producer from the fifo and starts
using the next producer, if any. When the producer fifo is empty the
:meth:`handle_write` method does nothing. You use the channel object's
:meth:`set_terminator` method to describe how to recognize the end of, or an
important breakpoint in, an incoming transmission from the remote endpoint.
:meth:`set_terminator` method to describe how to recognize the end of, or
an important breakpoint in, an incoming transmission from the remote
endpoint.
To build a functioning :class:`async_chat` subclass your input methods
:meth:`collect_incoming_data` and :meth:`found_terminator` must handle the data
that the channel receives asynchronously. The methods are described below.
:meth:`collect_incoming_data` and :meth:`found_terminator` must handle the
data that the channel receives asynchronously. The methods are described
below.
.. method:: async_chat.close_when_done()
Pushes a ``None`` on to the producer fifo. When this producer is popped off the
fifo it causes the channel to be closed.
Pushes a ``None`` on to the producer fifo. When this producer is popped off
the fifo it causes the channel to be closed.
.. method:: async_chat.collect_incoming_data(data)
Called with *data* holding an arbitrary amount of received data. The default
method, which must be overridden, raises a :exc:`NotImplementedError` exception.
Called with *data* holding an arbitrary amount of received data. The
default method, which must be overridden, raises a
:exc:`NotImplementedError` exception.
.. method:: async_chat.discard_buffers()
In emergencies this method will discard any data held in the input and/or output
buffers and the producer fifo.
In emergencies this method will discard any data held in the input and/or
output buffers and the producer fifo.
.. method:: async_chat.found_terminator()
Called when the incoming data stream matches the termination condition set by
:meth:`set_terminator`. The default method, which must be overridden, raises a
:exc:`NotImplementedError` exception. The buffered input data should be
available via an instance attribute.
Called when the incoming data stream matches the termination condition set
by :meth:`set_terminator`. The default method, which must be overridden,
raises a :exc:`NotImplementedError` exception. The buffered input data
should be available via an instance attribute.
.. method:: async_chat.get_terminator()
......@@ -90,59 +95,59 @@ requests.
.. method:: async_chat.handle_read()
Called when a read event fires on the channel's socket in the asynchronous loop.
The default method checks for the termination condition established by
:meth:`set_terminator`, which can be either the appearance of a particular
string in the input stream or the receipt of a particular number of characters.
When the terminator is found, :meth:`handle_read` calls the
:meth:`found_terminator` method after calling :meth:`collect_incoming_data` with
any data preceding the terminating condition.
Called when a read event fires on the channel's socket in the asynchronous
loop. The default method checks for the termination condition established
by :meth:`set_terminator`, which can be either the appearance of a
particular string in the input stream or the receipt of a particular number
of characters. When the terminator is found, :meth:`handle_read` calls the
:meth:`found_terminator` method after calling :meth:`collect_incoming_data`
with any data preceding the terminating condition.
.. method:: async_chat.handle_write()
Called when the application may write data to the channel. The default method
calls the :meth:`initiate_send` method, which in turn will call
:meth:`refill_buffer` to collect data from the producer fifo associated with the
channel.
Called when the application may write data to the channel. The default
method calls the :meth:`initiate_send` method, which in turn will call
:meth:`refill_buffer` to collect data from the producer fifo associated
with the channel.
.. method:: async_chat.push(data)
Creates a :class:`simple_producer` object (*see below*) containing the data and
pushes it on to the channel's ``producer_fifo`` to ensure its transmission. This
is all you need to do to have the channel write the data out to the network,
although it is possible to use your own producers in more complex schemes to
implement encryption and chunking, for example.
Creates a :class:`simple_producer` object (*see below*) containing the data
and pushes it on to the channel's ``producer_fifo`` to ensure its
transmission. This is all you need to do to have the channel write the
data out to the network, although it is possible to use your own producers
in more complex schemes to implement encryption and chunking, for example.
.. method:: async_chat.push_with_producer(producer)
Takes a producer object and adds it to the producer fifo associated with the
channel. When all currently-pushed producers have been exhausted the channel
will consume this producer's data by calling its :meth:`more` method and send
the data to the remote endpoint.
Takes a producer object and adds it to the producer fifo associated with
the channel. When all currently-pushed producers have been exhausted the
channel will consume this producer's data by calling its :meth:`more`
method and send the data to the remote endpoint.
.. method:: async_chat.readable()
Should return ``True`` for the channel to be included in the set of channels
tested by the :cfunc:`select` loop for readability.
Should return ``True`` for the channel to be included in the set of
channels tested by the :cfunc:`select` loop for readability.
.. method:: async_chat.refill_buffer()
Refills the output buffer by calling the :meth:`more` method of the producer at
the head of the fifo. If it is exhausted then the producer is popped off the
fifo and the next producer is activated. If the current producer is, or becomes,
``None`` then the channel is closed.
Refills the output buffer by calling the :meth:`more` method of the
producer at the head of the fifo. If it is exhausted then the producer is
popped off the fifo and the next producer is activated. If the current
producer is, or becomes, ``None`` then the channel is closed.
.. method:: async_chat.set_terminator(term)
Sets the terminating condition to be recognised on the channel. ``term`` may be
any of three types of value, corresponding to three different ways to handle
incoming protocol data.
Sets the terminating condition to be recognized on the channel. ``term``
may be any of three types of value, corresponding to three different ways
to handle incoming protocol data.
+-----------+---------------------------------------------+
| term | Description |
......@@ -158,8 +163,8 @@ requests.
| | forever |
+-----------+---------------------------------------------+
Note that any data following the terminator will be available for reading by the
channel after :meth:`found_terminator` is called.
Note that any data following the terminator will be available for reading
by the channel after :meth:`found_terminator` is called.
.. method:: async_chat.writable()
......@@ -174,29 +179,29 @@ asynchat - Auxiliary Classes and Functions
.. class:: simple_producer(data[, buffer_size=512])
A :class:`simple_producer` takes a chunk of data and an optional buffer size.
Repeated calls to its :meth:`more` method yield successive chunks of the data no
larger than *buffer_size*.
A :class:`simple_producer` takes a chunk of data and an optional buffer
size. Repeated calls to its :meth:`more` method yield successive chunks of
the data no larger than *buffer_size*.
.. method:: simple_producer.more()
Produces the next chunk of information from the producer, or returns the empty
string.
Produces the next chunk of information from the producer, or returns the
empty string.
.. class:: fifo([list=None])
Each channel maintains a :class:`fifo` holding data which has been pushed by the
application but not yet popped for writing to the channel. A :class:`fifo` is a
list used to hold data and/or producers until they are required. If the *list*
argument is provided then it should contain producers or data items to be
written to the channel.
Each channel maintains a :class:`fifo` holding data which has been pushed
by the application but not yet popped for writing to the channel. A
:class:`fifo` is a list used to hold data and/or producers until they are
required. If the *list* argument is provided then it should contain
producers or data items to be written to the channel.
.. method:: fifo.is_empty()
Returns ``True`` iff the fifo is empty.
Returns ``True`` if and only if the fifo is empty.
.. method:: fifo.first()
......@@ -206,14 +211,14 @@ asynchat - Auxiliary Classes and Functions
.. method:: fifo.push(data)
Adds the given data (which may be a string or a producer object) to the producer
fifo.
Adds the given data (which may be a string or a producer object) to the
producer fifo.
.. method:: fifo.pop()
If the fifo is not empty, returns ``True, first()``, deleting the popped item.
Returns ``False, None`` for an empty fifo.
If the fifo is not empty, returns ``True, first()``, deleting the popped
item. Returns ``False, None`` for an empty fifo.
The :mod:`asynchat` module also defines one utility function, which may be of
use in network and textual analysis operations.
......@@ -221,8 +226,8 @@ use in network and textual analysis operations.
.. function:: find_prefix_at_end(haystack, needle)
Returns ``True`` if string *haystack* ends with any non-empty prefix of string
*needle*.
Returns ``True`` if string *haystack* ends with any non-empty prefix of
string *needle*.
.. _asynchat-example:
......@@ -231,19 +236,20 @@ asynchat Example
----------------
The following partial example shows how HTTP requests can be read with
:class:`async_chat`. A web server might create an :class:`http_request_handler`
object for each incoming client connection. Notice that initially the channel
terminator is set to match the blank line at the end of the HTTP headers, and a
flag indicates that the headers are being read.
:class:`async_chat`. A web server might create an
:class:`http_request_handler` object for each incoming client connection.
Notice that initially the channel terminator is set to match the blank line at
the end of the HTTP headers, and a flag indicates that the headers are being
read.
Once the headers have been read, if the request is of type POST (indicating that
further data are present in the input stream) then the ``Content-Length:``
header is used to set a numeric terminator to read the right amount of data from
the channel.
Once the headers have been read, if the request is of type POST (indicating
that further data are present in the input stream) then the
``Content-Length:`` header is used to set a numeric terminator to read the
right amount of data from the channel.
The :meth:`handle_request` method is called once all relevant input has been
marshalled, after setting the channel terminator to ``None`` to ensure that any
extraneous data sent by the web client are ignored. ::
marshalled, after setting the channel terminator to ``None`` to ensure that
any extraneous data sent by the web client are ignored. ::
class http_request_handler(asynchat.async_chat):
......@@ -281,4 +287,3 @@ extraneous data sent by the web client are ignored. ::
self.handling = True
self.ibuffer = []
self.handle_request()
Doc/library/asyncore.rst
View file @ 06aa0c62
......@@ -3,7 +3,8 @@
===============================================
.. module:: asyncore
:synopsis: A base class for developing asynchronous socket handling services.
:synopsis: A base class for developing asynchronous socket handling
services.
.. moduleauthor:: Sam Rushing <rushing@nightmare.com>
.. sectionauthor:: Christopher Petrilli <petrilli@amber.org>
.. sectionauthor:: Steve Holden <sholden@holdenweb.com>
......@@ -16,59 +17,62 @@ service clients and servers.
There are only two ways to have a program on a single processor do "more than
one thing at a time." Multi-threaded programming is the simplest and most
popular way to do it, but there is another very different technique, that lets
popular way to do it, but there is another very different technique, that lets
you have nearly all the advantages of multi-threading, without actually using
multiple threads. It's really only practical if your program is largely I/O
bound. If your program is processor bound, then pre-emptive scheduled threads
are probably what you really need. Network servers are rarely processor bound,
however.
bound. If your program is processor bound, then pre-emptive scheduled threads
are probably what you really need. Network servers are rarely processor
bound, however.
If your operating system supports the :cfunc:`select` system call in its I/O
library (and nearly all do), then you can use it to juggle multiple
communication channels at once; doing other work while your I/O is taking place
in the "background." Although this strategy can seem strange and complex,
especially at first, it is in many ways easier to understand and control than
multi-threaded programming. The :mod:`asyncore` module solves many of the
difficult problems for you, making the task of building sophisticated
high-performance network servers and clients a snap. For "conversational"
applications and protocols the companion :mod:`asynchat` module is invaluable.
The basic idea behind both modules is to create one or more network *channels*,
instances of class :class:`asyncore.dispatcher` and
:class:`asynchat.async_chat`. Creating the channels adds them to a global map,
used by the :func:`loop` function if you do not provide it with your own *map*.
communication channels at once; doing other work while your I/O is taking
place in the "background." Although this strategy can seem strange and
complex, especially at first, it is in many ways easier to understand and
control than multi-threaded programming. The :mod:`asyncore` module solves
many of the difficult problems for you, making the task of building
sophisticated high-performance network servers and clients a snap. For
"conversational" applications and protocols the companion :mod:`asynchat`
module is invaluable.
The basic idea behind both modules is to create one or more network
*channels*, instances of class :class:`asyncore.dispatcher` and
:class:`asynchat.async_chat`. Creating the channels adds them to a global
map, used by the :func:`loop` function if you do not provide it with your own
*map*.
Once the initial channel(s) is(are) created, calling the :func:`loop` function
activates channel service, which continues until the last channel (including any
that have been added to the map during asynchronous service) is closed.
activates channel service, which continues until the last channel (including
any that have been added to the map during asynchronous service) is closed.
.. function:: loop([timeout[, use_poll[, map[,count]]]])
Enter a polling loop that terminates after count passes or all open channels
have been closed. All arguments are optional. The *count* parameter defaults
to None, resulting in the loop terminating only when all channels have been
closed. The *timeout* argument sets the timeout parameter for the appropriate
:func:`select` or :func:`poll` call, measured in seconds; the default is 30
seconds. The *use_poll* parameter, if true, indicates that :func:`poll` should
be used in preference to :func:`select` (the default is ``False``).
Enter a polling loop that terminates after count passes or all open
channels have been closed. All arguments are optional. The *count*
parameter defaults to None, resulting in the loop terminating only when all
channels have been closed. The *timeout* argument sets the timeout
parameter for the appropriate :func:`select` or :func:`poll` call, measured
in seconds; the default is 30 seconds. The *use_poll* parameter, if true,
indicates that :func:`poll` should be used in preference to :func:`select`
(the default is ``False``).
The *map* parameter is a dictionary whose items are the channels to watch. As
channels are closed they are deleted from their map. If *map* is omitted, a
global map is used. Channels (instances of :class:`asyncore.dispatcher`,
:class:`asynchat.async_chat` and subclasses thereof) can freely be mixed in the
map.
The *map* parameter is a dictionary whose items are the channels to watch.
As channels are closed they are deleted from their map. If *map* is
omitted, a global map is used. Channels (instances of
:class:`asyncore.dispatcher`, :class:`asynchat.async_chat` and subclasses
thereof) can freely be mixed in the map.
.. class:: dispatcher()
The :class:`dispatcher` class is a thin wrapper around a low-level socket
object. To make it more useful, it has a few methods for event-handling which
are called from the asynchronous loop. Otherwise, it can be treated as a
normal non-blocking socket object.
object. To make it more useful, it has a few methods for event-handling
which are called from the asynchronous loop. Otherwise, it can be treated
as a normal non-blocking socket object.
Two class attributes can be modified, to improve performance, or possibly even
to conserve memory.
Two class attributes can be modified, to improve performance, or possibly
even to conserve memory.
.. data:: ac_in_buffer_size
......@@ -80,12 +84,13 @@ that have been added to the map during asynchronous service) is closed.
The asynchronous output buffer size (default ``4096``).
The firing of low-level events at certain times or in certain connection states
tells the asynchronous loop that certain higher-level events have taken place.
For example, if we have asked for a socket to connect to another host, we know
that the connection has been made when the socket becomes writable for the first
time (at this point you know that you may write to it with the expectation of
success). The implied higher-level events are:
The firing of low-level events at certain times or in certain connection
states tells the asynchronous loop that certain higher-level events have
taken place. For example, if we have asked for a socket to connect to
another host, we know that the connection has been made when the socket
becomes writable for the first time (at this point you know that you may
write to it with the expectation of success). The implied higher-level
events are:
+----------------------+----------------------------------------+
| Event | Description |
......@@ -101,11 +106,11 @@ that have been added to the map during asynchronous service) is closed.
During asynchronous processing, each mapped channel's :meth:`readable` and
:meth:`writable` methods are used to determine whether the channel's socket
should be added to the list of channels :cfunc:`select`\ ed or :cfunc:`poll`\ ed
for read and write events.
should be added to the list of channels :cfunc:`select`\ ed or
:cfunc:`poll`\ ed for read and write events.
Thus, the set of channel events is larger than the basic socket events. The full
set of methods that can be overridden in your subclass follows:
Thus, the set of channel events is larger than the basic socket events. The
full set of methods that can be overridden in your subclass follows:
.. method:: dispatcher.handle_read()
......@@ -116,9 +121,9 @@ set of methods that can be overridden in your subclass follows:
.. method:: dispatcher.handle_write()
Called when the asynchronous loop detects that a writable socket can be written.
Often this method will implement the necessary buffering for performance. For
example::
Called when the asynchronous loop detects that a writable socket can be
written. Often this method will implement the necessary buffering for
performance. For example::
def handle_write(self):
sent = self.send(self.buffer)
......@@ -127,15 +132,15 @@ set of methods that can be overridden in your subclass follows:
.. method:: dispatcher.handle_expt()
Called when there is out of band (OOB) data for a socket connection. This will
almost never happen, as OOB is tenuously supported and rarely used.
Called when there is out of band (OOB) data for a socket connection. This
will almost never happen, as OOB is tenuously supported and rarely used.
.. method:: dispatcher.handle_connect()
Called when the active opener's socket actually makes a connection. Might send a
"welcome" banner, or initiate a protocol negotiation with the remote endpoint,
for example.
Called when the active opener's socket actually makes a connection. Might
send a "welcome" banner, or initiate a protocol negotiation with the remote
endpoint, for example.
.. method:: dispatcher.handle_close()
......@@ -152,40 +157,40 @@ set of methods that can be overridden in your subclass follows:
.. method:: dispatcher.handle_accept()
Called on listening channels (passive openers) when a connection can be
established with a new remote endpoint that has issued a :meth:`connect` call
for the local endpoint.
established with a new remote endpoint that has issued a :meth:`connect`
call for the local endpoint.
.. method:: dispatcher.readable()
Called each time around the asynchronous loop to determine whether a channel's
socket should be added to the list on which read events can occur. The default
method simply returns ``True``, indicating that by default, all channels will
be interested in read events.
Called each time around the asynchronous loop to determine whether a
channel's socket should be added to the list on which read events can
occur. The default method simply returns ``True``, indicating that by
default, all channels will be interested in read events.
.. method:: dispatcher.writable()
Called each time around the asynchronous loop to determine whether a channel's
socket should be added to the list on which write events can occur. The default
method simply returns ``True``, indicating that by default, all channels will
be interested in write events.
Called each time around the asynchronous loop to determine whether a
channel's socket should be added to the list on which write events can
occur. The default method simply returns ``True``, indicating that by
default, all channels will be interested in write events.
In addition, each channel delegates or extends many of the socket methods. Most
of these are nearly identical to their socket partners.
In addition, each channel delegates or extends many of the socket methods.
Most of these are nearly identical to their socket partners.
.. method:: dispatcher.create_socket(family, type)
This is identical to the creation of a normal socket, and will use the same
options for creation. Refer to the :mod:`socket` documentation for information
on creating sockets.
This is identical to the creation of a normal socket, and will use the same
options for creation. Refer to the :mod:`socket` documentation for
information on creating sockets.
.. method:: dispatcher.connect(address)
As with the normal socket object, *address* is a tuple with the first element
the host to connect to, and the second the port number.
As with the normal socket object, *address* is a tuple with the first
element the host to connect to, and the second the port number.
.. method:: dispatcher.send(data)
......@@ -195,38 +200,41 @@ of these are nearly identical to their socket partners.
.. method:: dispatcher.recv(buffer_size)
Read at most *buffer_size* bytes from the socket's remote end-point. An empty
string implies that the channel has been closed from the other end.
Read at most *buffer_size* bytes from the socket's remote end-point.
An empty string implies that the channel has been closed from the other
end.
.. method:: dispatcher.listen(backlog)
Listen for connections made to the socket. The *backlog* argument specifies the
maximum number of queued connections and should be at least 1; the maximum value
is system-dependent (usually 5).
Listen for connections made to the socket. The *backlog* argument
specifies the maximum number of queued connections and should be at least
1; the maximum value is system-dependent (usually 5).
.. method:: dispatcher.bind(address)
Bind the socket to *address*. The socket must not already be bound. (The
format of *address* depends on the address family --- see above.) To mark the
socket as re-usable (setting the :const:`SO_REUSEADDR` option), call the
:class:`dispatcher` object's :meth:`set_reuse_addr` method.
format of *address* depends on the address family --- see above.) To mark
the socket as re-usable (setting the :const:`SO_REUSEADDR` option), call
the :class:`dispatcher` object's :meth:`set_reuse_addr` method.
.. method:: dispatcher.accept()
Accept a connection. The socket must be bound to an address and listening for
connections. The return value is a pair ``(conn, address)`` where *conn* is a
*new* socket object usable to send and receive data on the connection, and
*address* is the address bound to the socket on the other end of the connection.
Accept a connection. The socket must be bound to an address and listening
for connections. The return value is a pair ``(conn, address)`` where
*conn* is a *new* socket object usable to send and receive data on the
connection, and *address* is the address bound to the socket on the other
end of the connection.
.. method:: dispatcher.close()
Close the socket. All future operations on the socket object will fail. The
remote end-point will receive no more data (after queued data is flushed).
Sockets are automatically closed when they are garbage-collected.
Close the socket. All future operations on the socket object will fail.
The remote end-point will receive no more data (after queued data is
flushed). Sockets are automatically closed when they are
garbage-collected.
.. _asyncore-example:
......@@ -266,4 +274,3 @@ implement its socket handling::
c = http_client('www.python.org', '/')
asyncore.loop()
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment
GitLab Nexedi Edition | About GitLab | About Nexedi | 沪ICP备2021021310号-2 | 沪ICP备2021021310号-7