Merge heads

Doc/library/argparse.rst

@@ -1499,12 +1499,15 @@ Sub-commands
    * parser_class - class which will be used to create sub-parser instances, by
      default the class of the current parser (e.g. ArgumentParser)
 
-   * dest - name of the attribute under which sub-command name will be
+   * action_ - the basic type of action to be taken when this argument is
+     encountered at the command line
+
+   * dest_ - name of the attribute under which sub-command name will be
      stored; by default None and no value is stored
 
-   * help - help for sub-parser group in help output, by default None
+   * help_ - help for sub-parser group in help output, by default None
 
-   * metavar - string presenting available sub-commands in help; by default it
+   * metavar_ - string presenting available sub-commands in help; by default it
      is None and presents sub-commands in form {cmd1, cmd2, ..}
 
    Some example usage::

Doc/library/logging.handlers.rst

@@ -185,15 +185,16 @@ module, supports rotation of disk log files.
    You can use the *maxBytes* and *backupCount* values to allow the file to
    :dfn:`rollover` at a predetermined size. When the size is about to be exceeded,
    the file is closed and a new file is silently opened for output. Rollover occurs
-   whenever the current log file is nearly *maxBytes* in length; if *maxBytes* is
-   zero, rollover never occurs.  If *backupCount* is non-zero, the system will save
-   old log files by appending the extensions '.1', '.2' etc., to the filename. For
-   example, with a *backupCount* of 5 and a base file name of :file:`app.log`, you
-   would get :file:`app.log`, :file:`app.log.1`, :file:`app.log.2`, up to
-   :file:`app.log.5`. The file being written to is always :file:`app.log`.  When
-   this file is filled, it is closed and renamed to :file:`app.log.1`, and if files
-   :file:`app.log.1`, :file:`app.log.2`, etc.  exist, then they are renamed to
-   :file:`app.log.2`, :file:`app.log.3` etc.  respectively.
+   whenever the current log file is nearly *maxBytes* in length; if either of
+   *maxBytes* or *backupCount* is zero, rollover never occurs.  If *backupCount*
+   is non-zero, the system will save old log files by appending the extensions
+   '.1', '.2' etc., to the filename. For example, with a *backupCount* of 5 and
+   a base file name of :file:`app.log`, you would get :file:`app.log`,
+   :file:`app.log.1`, :file:`app.log.2`, up to :file:`app.log.5`. The file being
+   written to is always :file:`app.log`.  When this file is filled, it is closed
+   and renamed to :file:`app.log.1`, and if files :file:`app.log.1`,
+   :file:`app.log.2`, etc.  exist, then they are renamed to :file:`app.log.2`,
+   :file:`app.log.3` etc.  respectively.
 
    .. versionchanged:: 2.6
       *delay* was added.

Doc/library/ssl.rst

@@ -638,6 +638,13 @@ Constants
 
    .. versionadded:: 2.7.9
 
+.. data:: HAS_ALPN
+
+   Whether the OpenSSL library has built-in support for the *Application-Layer
+   Protocol Negotiation* TLS extension as described in :rfc:`7301`.
+
+   .. versionadded:: 2.7.10
+
 .. data:: HAS_ECDH
 
    Whether the OpenSSL library has built-in support for Elliptic Curve-based
@@ -864,9 +871,19 @@ SSL sockets also have the following additional methods and attributes:
 
    .. versionadded:: 2.7.9
 
+.. method:: SSLSocket.selected_alpn_protocol()
+
+   Return the protocol that was selected during the TLS handshake.  If
+   :meth:`SSLContext.set_alpn_protocols` was not called, if the other party does
+   not support ALPN, if this socket does not support any of the client's
+   proposed protocols, or if the handshake has not happened yet, ``None`` is
+   returned.
+
+   .. versionadded:: 2.7.10
+
 .. method:: SSLSocket.selected_npn_protocol()
 
-   Returns the higher-level protocol that was selected during the TLS/SSL
+   Return the higher-level protocol that was selected during the TLS/SSL
    handshake. If :meth:`SSLContext.set_npn_protocols` was not called, or
    if the other party does not support NPN, or if the handshake has not yet
    happened, this will return ``None``.
@@ -1034,6 +1051,20 @@ to speed up repeated connections from the same clients.
       when connected, the :meth:`SSLSocket.cipher` method of SSL sockets will
       give the currently selected cipher.
 
+.. method:: SSLContext.set_alpn_protocols(protocols)
+
+   Specify which protocols the socket should advertise during the SSL/TLS
+   handshake. It should be a list of ASCII strings, like ``['http/1.1',
+   'spdy/2']``, ordered by preference. The selection of a protocol will happen
+   during the handshake, and will play out according to :rfc:`7301`. After a
+   successful handshake, the :meth:`SSLSocket.selected_alpn_protocol` method will
+   return the agreed-upon protocol.
+
+   This method will raise :exc:`NotImplementedError` if :data:`HAS_ALPN` is
+   False.
+
+   .. versionadded:: 2.7.10
+
 .. method:: SSLContext.set_npn_protocols(protocols)
 
    Specify which protocols the socket should advertise during the SSL/TLS
@@ -1072,7 +1103,7 @@ to speed up repeated connections from the same clients.
 
    Due to the early negotiation phase of the TLS connection, only limited
    methods and attributes are usable like
-   :meth:`SSLSocket.selected_npn_protocol` and :attr:`SSLSocket.context`.
+   :meth:`SSLSocket.selected_alpn_protocol` and :attr:`SSLSocket.context`.
    :meth:`SSLSocket.getpeercert`, :meth:`SSLSocket.getpeercert`,
    :meth:`SSLSocket.cipher` and :meth:`SSLSocket.compress` methods require that
    the TLS connection has progressed beyond the TLS Client Hello and therefore

Doc/library/time.rst

@@ -222,12 +222,13 @@ The module defines the following functions and data items:
 
 .. function:: sleep(secs)
 
-   Suspend execution for the given number of seconds.  The argument may be a
-   floating point number to indicate a more precise sleep time. The actual
-   suspension time may be less than that requested because any caught signal will
-   terminate the :func:`sleep` following execution of that signal's catching
-   routine.  Also, the suspension time may be longer than requested by an arbitrary
-   amount because of the scheduling of other activity in the system.
+   Suspend execution of the current thread for the given number of seconds.
+   The argument may be a floating point number to indicate a more precise sleep
+   time. The actual suspension time may be less than that requested because any
+   caught signal will terminate the :func:`sleep` following execution of that
+   signal's catching routine.  Also, the suspension time may be longer than
+   requested by an arbitrary amount because of the scheduling of other activity
+   in the system.
 
 
 .. function:: strftime(format[, t])

Lib/httplib.py

@@ -313,6 +313,11 @@ class HTTPMessage(mimetools.Message):
                 hlist.append(line)
                 self.addheader(headerseen, line[len(headerseen)+1:].strip())
                 continue
+            elif headerseen is not None:
+                # An empty header name. These aren't allowed in HTTP, but it's
+                # probably a benign mistake. Don't add the header, just keep
+                # going.
+                continue
             else:
                 # It's not a header line; throw it back and stop here.
                 if not self.dict:
@@ -723,7 +728,7 @@ class HTTPConnection:
         endpoint passed to set_tunnel. This is done by sending a HTTP CONNECT
         request to the proxy server when the connection is established.
 
-        This method must be called before the HTML connection has been
+        This method must be called before the HTTP connection has been
         established.
 
         The headers argument should be a mapping of extra HTTP headers
@@ -1129,7 +1134,7 @@ class HTTP:
         "Accept arguments to set the host/port, since the superclass doesn't."
 
         if host is not None:
-            self._conn._set_hostport(host, port)
+            (self._conn.host, self._conn.port) = self._conn._get_hostport(host, port)
         self._conn.connect()
 
     def getfile(self):

Lib/rfc822.py

@@ -179,6 +179,11 @@ class Message:
                 lst.append(line)
                 self.dict[headerseen] = line[len(headerseen)+1:].strip()
                 continue
+            elif headerseen is not None:
+                # An empty header name. These aren't allowed in HTTP, but it's
+                # probably a benign mistake. Don't add the header, just keep
+                # going.
+                continue
             else:
                 # It's not a header line; throw it back and stop here.
                 if not self.dict:
@@ -202,7 +207,7 @@ class Message:
         data in RFC 2822-like formats with special header formats.
         """
         i = line.find(':')
-        if i > 0:
+        if i > -1:
             return line[:i].lower()
         return None
 

Lib/ssl.py

@@ -123,7 +123,7 @@ _import_symbols('ALERT_DESCRIPTION_')
 _import_symbols('SSL_ERROR_')
 _import_symbols('PROTOCOL_')
 
-from _ssl import HAS_SNI, HAS_ECDH, HAS_NPN
+from _ssl import HAS_SNI, HAS_ECDH, HAS_NPN, HAS_ALPN
 
 from _ssl import _OPENSSL_API_VERSION
 
@@ -365,6 +365,17 @@ class SSLContext(_SSLContext):
 
         self._set_npn_protocols(protos)
 
+    def set_alpn_protocols(self, alpn_protocols):
+        protos = bytearray()
+        for protocol in alpn_protocols:
+            b = protocol.encode('ascii')
+            if len(b) == 0 or len(b) > 255:
+                raise SSLError('ALPN protocols must be 1 to 255 in length')
+            protos.append(len(b))
+            protos.extend(b)
+
+        self._set_alpn_protocols(protos)
+
     def _load_windows_store_certs(self, storename, purpose):
         certs = bytearray()
         for cert, encoding, trust in enum_certificates(storename):
@@ -647,6 +658,13 @@ class SSLSocket(socket):
         else:
             return self._sslobj.selected_npn_protocol()
 
+    def selected_alpn_protocol(self):
+        self._checkClosed()
+        if not self._sslobj or not _ssl.HAS_ALPN:
+            return None
+        else:
+            return self._sslobj.selected_alpn_protocol()
+
     def cipher(self):
         self._checkClosed()
         if not self._sslobj:

Lib/test/test_httplib.py

@@ -164,6 +164,16 @@ class HeaderTests(TestCase):
         conn.request('GET', '/foo')
         self.assertTrue(sock.data.startswith(expected))
 
+    def test_malformed_headers_coped_with(self):
+        # Issue 19996
+        body = "HTTP/1.1 200 OK\r\nFirst: val\r\n: nval\r\nSecond: val\r\n\r\n"
+        sock = FakeSocket(body)
+        resp = httplib.HTTPResponse(sock)
+        resp.begin()
+
+        self.assertEqual(resp.getheader('First'), 'val')
+        self.assertEqual(resp.getheader('Second'), 'val')
+
 
 class BasicTest(TestCase):
     def test_status_lines(self):
@@ -461,7 +471,11 @@ class OfflineTest(TestCase):
         self.assertEqual(httplib.responses[httplib.NOT_FOUND], "Not Found")
 
 
-class SourceAddressTest(TestCase):
+class TestServerMixin:
+    """A limited socket server mixin.
+
+    This is used by test cases for testing http connection end points.
+    """
     def setUp(self):
         self.serv = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
         self.port = test_support.bind_port(self.serv)
@@ -476,6 +490,7 @@ class SourceAddressTest(TestCase):
         self.serv.close()
         self.serv = None
 
+class SourceAddressTest(TestServerMixin, TestCase):
     def testHTTPConnectionSourceAddress(self):
         self.conn = httplib.HTTPConnection(HOST, self.port,
                 source_address=('', self.source_port))
@@ -492,6 +507,24 @@ class SourceAddressTest(TestCase):
         # for an ssl_wrapped connect() to actually return from.
 
 
+class HTTPTest(TestServerMixin, TestCase):
+    def testHTTPConnection(self):
+        self.conn = httplib.HTTP(host=HOST, port=self.port, strict=None)
+        self.conn.connect()
+        self.assertEqual(self.conn._conn.host, HOST)
+        self.assertEqual(self.conn._conn.port, self.port)
+
+    def testHTTPWithConnectHostPort(self):
+        testhost = 'unreachable.test.domain'
+        testport = '80'
+        self.conn = httplib.HTTP(host=testhost, port=testport)
+        self.conn.connect(host=HOST, port=self.port)
+        self.assertNotEqual(self.conn._conn.host, testhost)
+        self.assertNotEqual(self.conn._conn.port, testport)
+        self.assertEqual(self.conn._conn.host, HOST)
+        self.assertEqual(self.conn._conn.port, self.port)
+
+
 class TimeoutTest(TestCase):
     PORT = None
 
@@ -537,6 +570,7 @@ class TimeoutTest(TestCase):
         self.assertEqual(httpConn.sock.gettimeout(), 30)
         httpConn.close()
 
+
 class HTTPSTest(TestCase):
 
     def setUp(self):
@@ -713,7 +747,8 @@ class TunnelTests(TestCase):
 @test_support.reap_threads
 def test_main(verbose=None):
     test_support.run_unittest(HeaderTests, OfflineTest, BasicTest, TimeoutTest,
-                              HTTPSTest, SourceAddressTest, TunnelTests)
+                              HTTPTest, HTTPSTest, SourceAddressTest,
+                              TunnelTests)
 
 if __name__ == '__main__':
     test_main()

Lib/test/test_rfc822.py

@@ -248,6 +248,12 @@ A test message.
         eq(rfc822.quote('foo\\wacky"name'), 'foo\\\\wacky\\"name')
         eq(rfc822.unquote('"foo\\\\wacky\\"name"'), 'foo\\wacky"name')
 
+    def test_invalid_headers(self):
+        eq = self.assertEqual
+        msg = self.create_message("First: val\n: otherval\nSecond: val2\n")
+        eq(msg.getheader('First'), 'val')
+        eq(msg.getheader('Second'), 'val2')
+
 
 def test_main():
     test_support.run_unittest(MessageTestCase)

Lib/test/test_ssl.py

@@ -1569,7 +1569,8 @@ else:
                 try:
                     self.sslconn = self.server.context.wrap_socket(
                         self.sock, server_side=True)
-                    self.server.selected_protocols.append(self.sslconn.selected_npn_protocol())
+                    self.server.selected_npn_protocols.append(self.sslconn.selected_npn_protocol())
+                    self.server.selected_alpn_protocols.append(self.sslconn.selected_alpn_protocol())
                 except socket.error as e:
                     # We treat ConnectionResetError as though it were an
                     # SSLError - OpenSSL on Ubuntu abruptly closes the
@@ -1678,7 +1679,8 @@ else:
         def __init__(self, certificate=None, ssl_version=None,
                      certreqs=None, cacerts=None,
                      chatty=True, connectionchatty=False, starttls_server=False,
-                     npn_protocols=None, ciphers=None, context=None):
+                     npn_protocols=None, alpn_protocols=None,
+                     ciphers=None, context=None):
             if context:
                 self.context = context
             else:
@@ -1693,6 +1695,8 @@ else:
                     self.context.load_cert_chain(certificate)
                 if npn_protocols:
                     self.context.set_npn_protocols(npn_protocols)
+                if alpn_protocols:
+                    self.context.set_alpn_protocols(alpn_protocols)
                 if ciphers:
                     self.context.set_ciphers(ciphers)
             self.chatty = chatty
@@ -1702,7 +1706,8 @@ else:
             self.port = support.bind_port(self.sock)
             self.flag = None
             self.active = False
-            self.selected_protocols = []
+            self.selected_npn_protocols = []
+            self.selected_alpn_protocols = []
             self.conn_errors = []
             threading.Thread.__init__(self)
             self.daemon = True
@@ -1927,11 +1932,13 @@ else:
                     'compression': s.compression(),
                     'cipher': s.cipher(),
                     'peercert': s.getpeercert(),
+                    'client_alpn_protocol': s.selected_alpn_protocol(),
                     'client_npn_protocol': s.selected_npn_protocol(),
                     'version': s.version(),
                 })
                 s.close()
-            stats['server_npn_protocols'] = server.selected_protocols
+            stats['server_alpn_protocols'] = server.selected_alpn_protocols
+            stats['server_npn_protocols'] = server.selected_npn_protocols
         return stats
 
     def try_protocol_combo(server_protocol, client_protocol, expect_success,
@@ -2787,6 +2794,55 @@ else:
             if "ADH" not in parts and "EDH" not in parts and "DHE" not in parts:
                 self.fail("Non-DH cipher: " + cipher[0])
 
+        def test_selected_alpn_protocol(self):
+            # selected_alpn_protocol() is None unless ALPN is used.
+            context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+            context.load_cert_chain(CERTFILE)
+            stats = server_params_test(context, context,
+                                       chatty=True, connectionchatty=True)
+            self.assertIs(stats['client_alpn_protocol'], None)
+
+        @unittest.skipUnless(ssl.HAS_ALPN, "ALPN support required")
+        def test_selected_alpn_protocol_if_server_uses_alpn(self):
+            # selected_alpn_protocol() is None unless ALPN is used by the client.
+            client_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+            client_context.load_verify_locations(CERTFILE)
+            server_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+            server_context.load_cert_chain(CERTFILE)
+            server_context.set_alpn_protocols(['foo', 'bar'])
+            stats = server_params_test(client_context, server_context,
+                                       chatty=True, connectionchatty=True)
+            self.assertIs(stats['client_alpn_protocol'], None)
+
+        @unittest.skipUnless(ssl.HAS_ALPN, "ALPN support needed for this test")
+        def test_alpn_protocols(self):
+            server_protocols = ['foo', 'bar', 'milkshake']
+            protocol_tests = [
+                (['foo', 'bar'], 'foo'),
+                (['bar', 'foo'], 'foo'),
+                (['milkshake'], 'milkshake'),
+                (['http/3.0', 'http/4.0'], None)
+            ]
+            for client_protocols, expected in protocol_tests:
+                server_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+                server_context.load_cert_chain(CERTFILE)
+                server_context.set_alpn_protocols(server_protocols)
+                client_context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
+                client_context.load_cert_chain(CERTFILE)
+                client_context.set_alpn_protocols(client_protocols)
+                stats = server_params_test(client_context, server_context,
+                                           chatty=True, connectionchatty=True)
+
+                msg = "failed trying %s (s) and %s (c).\n" \
+                      "was expecting %s, but got %%s from the %%s" \
+                          % (str(server_protocols), str(client_protocols),
+                             str(expected))
+                client_result = stats['client_alpn_protocol']
+                self.assertEqual(client_result, expected, msg % (client_result, "client"))
+                server_result = stats['server_alpn_protocols'][-1] \
+                    if len(stats['server_alpn_protocols']) else 'nothing'
+                self.assertEqual(server_result, expected, msg % (server_result, "server"))
+
         def test_selected_npn_protocol(self):
             # selected_npn_protocol() is None unless NPN is used
             context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)

Misc/NEWS

@@ -15,6 +15,14 @@ Core and Builtins
 Library
 -------
 
+- Issue #19996: Make :mod:`httplib` ignore headers with no name rather than
+  assuming the body has started.
+
+- Issue #20188: Support Application-Layer Protocol Negotiation (ALPN) in the ssl
+  module.
+
+- Issue #23248: Update ssl error codes from latest OpenSSL git master.
+
 - Issue #23098: 64-bit dev_t is now supported in the os module.
 
 - Issue #23063: In the disutils' check command, fix parsing of reST with code or

Modules/_ssl.c

@@ -105,6 +105,11 @@ struct py_ssl_library_code {
 # define HAVE_SNI 0
 #endif
 
+/* ALPN added in OpenSSL 1.0.2 */
+#if OPENSSL_VERSION_NUMBER >= 0x1000200fL && !defined(OPENSSL_NO_TLSEXT)
+# define HAVE_ALPN
+#endif
+
 enum py_ssl_error {
     /* these mirror ssl.h */
     PY_SSL_ERROR_NONE,
@@ -205,9 +210,13 @@ typedef struct {
     PyObject_HEAD
     SSL_CTX *ctx;
 #ifdef OPENSSL_NPN_NEGOTIATED
-    char *npn_protocols;
+    unsigned char *npn_protocols;
     int npn_protocols_len;
 #endif
+#ifdef HAVE_ALPN
+    unsigned char *alpn_protocols;
+    int alpn_protocols_len;
+#endif
 #ifndef OPENSSL_NO_TLSEXT
     PyObject *set_hostname;
 #endif
@@ -1408,7 +1417,20 @@ static PyObject *PySSL_selected_npn_protocol(PySSLSocket *self) {
 
     if (out == NULL)
         Py_RETURN_NONE;
-    return PyUnicode_FromStringAndSize((char *) out, outlen);
+    return PyString_FromStringAndSize((char *)out, outlen);
+}
+#endif
+
+#ifdef HAVE_ALPN
+static PyObject *PySSL_selected_alpn_protocol(PySSLSocket *self) {
+    const unsigned char *out;
+    unsigned int outlen;
+
+    SSL_get0_alpn_selected(self->ssl, &out, &outlen);
+
+    if (out == NULL)
+        Py_RETURN_NONE;
+    return PyString_FromStringAndSize((char *)out, outlen);
 }
 #endif
 
@@ -1924,6 +1946,9 @@ static PyMethodDef PySSLMethods[] = {
     {"version", (PyCFunction)PySSL_version, METH_NOARGS},
 #ifdef OPENSSL_NPN_NEGOTIATED
     {"selected_npn_protocol", (PyCFunction)PySSL_selected_npn_protocol, METH_NOARGS},
+#endif
+#ifdef HAVE_ALPN
+    {"selected_alpn_protocol", (PyCFunction)PySSL_selected_alpn_protocol, METH_NOARGS},
 #endif
     {"compression", (PyCFunction)PySSL_compression, METH_NOARGS},
     {"shutdown", (PyCFunction)PySSL_SSLshutdown, METH_NOARGS,
@@ -2032,6 +2057,9 @@ context_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
 #ifdef OPENSSL_NPN_NEGOTIATED
     self->npn_protocols = NULL;
 #endif
+#ifdef HAVE_ALPN
+    self->alpn_protocols = NULL;
+#endif
 #ifndef OPENSSL_NO_TLSEXT
     self->set_hostname = NULL;
 #endif
@@ -2091,7 +2119,10 @@ context_dealloc(PySSLContext *self)
     context_clear(self);
     SSL_CTX_free(self->ctx);
 #ifdef OPENSSL_NPN_NEGOTIATED
-    PyMem_Free(self->npn_protocols);
+    PyMem_FREE(self->npn_protocols);
+#endif
+#ifdef HAVE_ALPN
+    PyMem_FREE(self->alpn_protocols);
 #endif
     Py_TYPE(self)->tp_free(self);
 }
@@ -2117,6 +2148,30 @@ set_ciphers(PySSLContext *self, PyObject *args)
     Py_RETURN_NONE;
 }
 
+static int
+do_protocol_selection(int alpn, unsigned char **out, unsigned char *outlen,
+                      const unsigned char *server_protocols, unsigned int server_protocols_len,
+                      const unsigned char *client_protocols, unsigned int client_protocols_len)
+{
+    int ret;
+    if (client_protocols == NULL) {
+        client_protocols = (unsigned char *)"";
+        client_protocols_len = 0;
+    }
+    if (server_protocols == NULL) {
+        server_protocols = (unsigned char *)"";
+        server_protocols_len = 0;
+    }
+
+    ret = SSL_select_next_proto(out, outlen,
+                                server_protocols, server_protocols_len,
+                                client_protocols, client_protocols_len);
+    if (alpn && ret != OPENSSL_NPN_NEGOTIATED)
+        return SSL_TLSEXT_ERR_NOACK;
+
+    return SSL_TLSEXT_ERR_OK;
+}
+
 #ifdef OPENSSL_NPN_NEGOTIATED
 /* this callback gets passed to SSL_CTX_set_next_protos_advertise_cb */
 static int
@@ -2127,10 +2182,10 @@ _advertiseNPN_cb(SSL *s,
     PySSLContext *ssl_ctx = (PySSLContext *) args;
 
     if (ssl_ctx->npn_protocols == NULL) {
-        *data = (unsigned char *) "";
+        *data = (unsigned char *)"";
         *len = 0;
     } else {
-        *data = (unsigned char *) ssl_ctx->npn_protocols;
+        *data = ssl_ctx->npn_protocols;
         *len = ssl_ctx->npn_protocols_len;
     }
 
@@ -2143,23 +2198,9 @@ _selectNPN_cb(SSL *s,
               const unsigned char *server, unsigned int server_len,
               void *args)
 {
-    PySSLContext *ssl_ctx = (PySSLContext *) args;
-
-    unsigned char *client = (unsigned char *) ssl_ctx->npn_protocols;
-    int client_len;
-
-    if (client == NULL) {
-        client = (unsigned char *) "";
-        client_len = 0;
-    } else {
-        client_len = ssl_ctx->npn_protocols_len;
-    }
-
-    SSL_select_next_proto(out, outlen,
-                          server, server_len,
-                          client, client_len);
-
-    return SSL_TLSEXT_ERR_OK;
+    PySSLContext *ctx = (PySSLContext *)args;
+    return do_protocol_selection(0, out, outlen, server, server_len,
+                                 ctx->npn_protocols, ctx->npn_protocols_len);
 }
 #endif
 
@@ -2202,6 +2243,50 @@ _set_npn_protocols(PySSLContext *self, PyObject *args)
 #endif
 }
 
+#ifdef HAVE_ALPN
+static int
+_selectALPN_cb(SSL *s,
+              const unsigned char **out, unsigned char *outlen,
+              const unsigned char *client_protocols, unsigned int client_protocols_len,
+              void *args)
+{
+    PySSLContext *ctx = (PySSLContext *)args;
+    return do_protocol_selection(1, (unsigned char **)out, outlen,
+                                 ctx->alpn_protocols, ctx->alpn_protocols_len,
+                                 client_protocols, client_protocols_len);
+}
+#endif
+
+static PyObject *
+_set_alpn_protocols(PySSLContext *self, PyObject *args)
+{
+#ifdef HAVE_ALPN
+    Py_buffer protos;
+
+    if (!PyArg_ParseTuple(args, "s*:set_npn_protocols", &protos))
+        return NULL;
+
+    PyMem_FREE(self->alpn_protocols);
+    self->alpn_protocols = PyMem_Malloc(protos.len);
+    if (!self->alpn_protocols)
+        return PyErr_NoMemory();
+    memcpy(self->alpn_protocols, protos.buf, protos.len);
+    self->alpn_protocols_len = protos.len;
+    PyBuffer_Release(&protos);
+
+    if (SSL_CTX_set_alpn_protos(self->ctx, self->alpn_protocols, self->alpn_protocols_len))
+        return PyErr_NoMemory();
+    SSL_CTX_set_alpn_select_cb(self->ctx, _selectALPN_cb, self);
+
+    PyBuffer_Release(&protos);
+    Py_RETURN_NONE;
+#else
+    PyErr_SetString(PyExc_NotImplementedError,
+                    "The ALPN extension requires OpenSSL 1.0.2 or later.");
+    return NULL;
+#endif
+}
+
 static PyObject *
 get_verify_mode(PySSLContext *self, void *c)
 {
@@ -3188,6 +3273,8 @@ static struct PyMethodDef context_methods[] = {
                        METH_VARARGS | METH_KEYWORDS, NULL},
     {"set_ciphers", (PyCFunction) set_ciphers,
                     METH_VARARGS, NULL},
+    {"_set_alpn_protocols", (PyCFunction) _set_alpn_protocols,
+                           METH_VARARGS, NULL},
     {"_set_npn_protocols", (PyCFunction) _set_npn_protocols,
                            METH_VARARGS, NULL},
     {"load_cert_chain", (PyCFunction) load_cert_chain,
@@ -4100,6 +4187,14 @@ init_ssl(void)
     Py_INCREF(r);
     PyModule_AddObject(m, "HAS_NPN", r);
 
+#ifdef HAVE_ALPN
+    r = Py_True;
+#else
+    r = Py_False;
+#endif
+    Py_INCREF(r);
+    PyModule_AddObject(m, "HAS_ALPN", r);
+
     /* Mappings for error codes */
     err_codes_to_names = PyDict_New();
     err_names_to_codes = PyDict_New();

Tools/ssl/make_ssl_data.py

@@ -5,8 +5,7 @@ This script should be called *manually* when we want to upgrade SSLError
 `library` and `reason` mnemnonics to a more recent OpenSSL version.
 
 It takes two arguments:
-- the path to the OpenSSL include files' directory
-  (e.g. openssl-1.0.1-beta3/include/openssl/)
+- the path to the OpenSSL source tree (e.g. git checkout)
 - the path to the C file to be generated
   (probably Modules/_ssl_data.h)
 """
@@ -15,9 +14,10 @@ import datetime
 import os
 import re
 import sys
+import _ssl
 
 
-def parse_error_codes(h_file, prefix):
+def parse_error_codes(h_file, prefix, libcode):
     pat = re.compile(r"#define\W+(%s([\w]+))\W+(\d+)\b" % re.escape(prefix))
     codes = []
     with open(h_file, "r", encoding="latin1") as f:
@@ -26,7 +26,8 @@ def parse_error_codes(h_file, prefix):
             if match:
                 code, name, num = match.groups()
                 num = int(num)
-                codes.append((code, name, num))
+                # e.g. ("SSL_R_BAD_DATA", ("ERR_LIB_SSL", "BAD_DATA", 390))
+                codes.append((code, (libcode, name, num)))
     return codes
 
 if __name__ == "__main__":
@@ -34,12 +35,32 @@ if __name__ == "__main__":
     outfile = sys.argv[2]
     use_stdout = outfile == '-'
     f = sys.stdout if use_stdout else open(outfile, "w")
-    error_libraries = (
-        # (library code, mnemonic, error prefix, header file)
-        ('ERR_LIB_PEM', 'PEM', 'PEM_R_', 'pem.h'),
-        ('ERR_LIB_SSL', 'SSL', 'SSL_R_', 'ssl.h'),
-        ('ERR_LIB_X509', 'X509', 'X509_R_', 'x509.h'),
-        )
+    error_libraries = {
+        # mnemonic -> (library code, error prefix, header file)
+        'PEM': ('ERR_LIB_PEM', 'PEM_R_', 'crypto/pem/pem.h'),
+        'SSL': ('ERR_LIB_SSL', 'SSL_R_', 'ssl/ssl.h'),
+        'X509': ('ERR_LIB_X509', 'X509_R_', 'crypto/x509/x509.h'),
+        }
+
+    # Read codes from libraries
+    new_codes = []
+    for libcode, prefix, h_file in sorted(error_libraries.values()):
+        new_codes += parse_error_codes(os.path.join(openssl_inc, h_file),
+                                       prefix, libcode)
+    new_code_nums = set((libcode, num)
+                        for (code, (libcode, name, num)) in new_codes)
+
+    # Merge with existing codes (in case some old codes disappeared).
+    codes = {}
+    for errname, (libnum, errnum) in _ssl.err_names_to_codes.items():
+        lib = error_libraries[_ssl.lib_codes_to_names[libnum]]
+        libcode = lib[0]              # e.g. ERR_LIB_PEM
+        errcode = lib[1] + errname    # e.g. SSL_R_BAD_SSL_SESSION_ID_LENGTH
+        # Only keep it if the numeric codes weren't reused
+        if (libcode, errnum) not in new_code_nums:
+            codes[errcode] = libcode, errname, errnum
+    codes.update(dict(new_codes))
+
     def w(l):
         f.write(l + "\n")
     w("/* File generated by Tools/ssl/make_ssl_data.py */")
@@ -47,21 +68,19 @@ if __name__ == "__main__":
     w("")
 
     w("static struct py_ssl_library_code library_codes[] = {")
-    for libcode, mnemo, _, _ in error_libraries:
+    for mnemo, (libcode, _, _) in sorted(error_libraries.items()):
         w('    {"%s", %s},' % (mnemo, libcode))
     w('    { NULL }')
     w('};')
     w("")
 
     w("static struct py_ssl_error_code error_codes[] = {")
-    for libcode, _, prefix, h_file in error_libraries:
-        codes = parse_error_codes(os.path.join(openssl_inc, h_file), prefix)
-        for code, name, num in sorted(codes):
-            w('  #ifdef %s' % (code))
-            w('    {"%s", %s, %s},' % (name, libcode, code))
-            w('  #else')
-            w('    {"%s", %s, %d},' % (name, libcode, num))
-            w('  #endif')
+    for errcode, (libcode, name, num) in sorted(codes.items()):
+        w('  #ifdef %s' % (errcode))
+        w('    {"%s", %s, %s},' % (name, libcode, errcode))
+        w('  #else')
+        w('    {"%s", %s, %d},' % (name, libcode, num))
+        w('  #endif')
     w('    { NULL }')
     w('};')
     if not use_stdout:

setup.py

@@ -696,7 +696,7 @@ class PyBuildExt(build_ext):
         exts.append( Extension('audioop', ['audioop.c']) )
 
         # Disabled on 64-bit platforms
-        if sys.maxint != 9223372036854775807L:
+        if sys.maxsize != 9223372036854775807L:
             # Operations on images
             exts.append( Extension('imageop', ['imageop.c']) )
         else: