Make headerregistry fully part of the provisional api.

When I made the checkin of the provisional email policy, I knew that
Address and Group needed to be made accessible from somewhere.  The more
I looked at it, though, the more it became clear that since this is a
provisional API anyway, there's no good reason to hide headerregistry as
a private API.  It was designed to ultimately be part of the public API,
and so it should be part of the provisional API.

This patch fully documents the headerregistry API, and deletes the
abbreviated version of those docs I had added to the provisional policy
docs.

Doc/library/email.policy.rst

@@ -310,10 +310,10 @@ added matters.  To illustrate::
 
 .. note::
 
-   The remainder of the classes documented below are included in the standard
-   library on a :term:`provisional basis <provisional package>`.  Backwards
-   incompatible changes (up to and including removal of the feature) may occur
-   if deemed necessary by the core developers.
+   The documentation below describes new policies that are included in the
+   standard library on a :term:`provisional basis <provisional package>`.
+   Backwards incompatible changes (up to and including removal of the feature)
+   may occur if deemed necessary by the core developers.
 
 
 .. class:: EmailPolicy(**kw)
@@ -353,12 +353,12 @@ added matters.  To illustrate::
 
       A callable that takes two arguments, ``name`` and ``value``, where
       ``name`` is a header field name and ``value`` is an unfolded header field
-      value, and returns a string-like object that represents that header.  A
-      default ``header_factory`` is provided that understands some of the
-      :RFC:`5322` header field types.  (Currently address fields and date
-      fields have special treatment, while all other fields are treated as
-      unstructured.  This list will be completed before the extension is marked
-      stable.)
+      value, and returns a string subclass that represents that header.  A
+      default ``header_factory`` (see :mod:`~email.headerregistry`) is provided
+      that understands some of the :RFC:`5322` header field types.  (Currently
+      address fields and date fields have special treatment, while all other
+      fields are treated as unstructured.  This list will be completed before
+      the extension is marked stable.)
 
    The class provides the following concrete implementations of the abstract
    methods of :class:`Policy`:
@@ -465,167 +465,5 @@ header.  Likewise, a header may be assigned a new value, or a new header
 created, using a unicode string, and the policy will take care of converting
 the unicode string into the correct RFC encoded form.
 
-The custom header objects and their attributes are described below.  All custom
-header objects are string subclasses, and their string value is the fully
-decoded value of the header field (the part of the field after the ``:``)
-
-
-.. class:: BaseHeader
-
-   This is the base class for all custom header objects.  It provides the
-   following attributes:
-
-   .. attribute:: name
-
-      The header field name (the portion of the field before the ':').
-
-   .. attribute:: defects
-
-      A possibly empty list of :class:`~email.errors.MessageDefect` objects
-      that record any RFC violations found while parsing the header field.
-
-   .. method:: fold(*, policy)
-
-      Return a string containing :attr:`~email.policy.Policy.linesep`
-      characters as required to correctly fold the header according
-      to *policy*.  A :attr:`~email.policy.Policy.cte_type` of
-      ``8bit`` will be treated as if it were ``7bit``, since strings
-      may not contain binary data.
-
-
-.. class:: UnstructuredHeader
-
-   The class used for any header that does not have a more specific
-   type.  (The :mailheader:`Subject` header is an example of an
-   unstructured header.)  It does not have any additional attributes.
-
-
-.. class:: DateHeader
-
-   The value of this type of header is a single date and time value.  The
-   primary example of this type of header is the :mailheader:`Date` header.
-
-   .. attribute:: datetime
-
-      A :class:`~datetime.datetime` encoding the date and time from the
-      header value.
-
-      The ``datetime`` will be a naive ``datetime`` if the value either does
-      not have a specified timezone (which would be a violation of the RFC) or
-      if the timezone is specified as ``-0000``.  This timezone value indicates
-      that the date and time is to be considered to be in UTC, but with no
-      indication of the local timezone in which it was generated.  (This
-      contrasts to ``+0000``, which indicates a date and time that really is in
-      the UTC ``0000`` timezone.)
-
-      If the header value contains a valid timezone that is not ``-0000``, the
-      ``datetime`` will be an aware ``datetime`` having a
-      :class:`~datetime.tzinfo` set to the :class:`~datetime.timezone`
-      indicated by the header value.
-
-   A ``datetime`` may also be assigned to a :mailheader:`Date` type header.
-   The resulting string value will use a timezone of ``-0000`` if the
-   ``datetime`` is naive, and the appropriate UTC offset if the ``datetime`` is
-   aware.
-
-
-.. class:: AddressHeader
-
-   This class is used for all headers that can contain addresses, whether they
-   are supposed to be singleton addresses or a list.
-
-   .. attribute:: addresses
-
-      A list of :class:`.Address` objects listing all of the addresses that
-      could be parsed out of the field value.
-
-   .. attribute:: groups
-
-      A list of :class:`.Group` objects.  Every address in :attr:`.addresses`
-      appears in one of the group objects in the tuple.  Addresses that are not
-      syntactically part of a group are represented by ``Group`` objects whose
-      ``name`` is ``None``.
-
-   In addition to addresses in string form, any combination of
-   :class:`.Address` and :class:`.Group` objects, singly or in a list, may be
-   assigned to an address header.
-
-
-.. class:: Address(display_name='', username='', domain='', addr_spec=None):
-
-   The class used to represent an email address.  The general form of an
-   address is::
-
-      [display_name] <username@domain>
-
-   or::
-
-      username@domain
-
-   where each part must conform to specific syntax rules spelled out in
-   :rfc:`5322`.
-
-   As a convenience *addr_spec* can be specified instead of *username* and
-   *domain*, in which case *username* and *domain* will be parsed from the
-   *addr_spec*.  An *addr_spec* must be a properly RFC quoted string; if it is
-   not ``Address`` will raise an error.  Unicode characters are allowed and
-   will be property encoded when serialized.  However, per the RFCs, unicode is
-   *not* allowed in the username portion of the address.
-
-   .. attribute:: display_name
-
-      The display name portion of the address, if any, with all quoting
-      removed.  If the address does not have a display name, this attribute
-      will be an empty string.
-
-   .. attribute:: username
-
-      The ``username`` portion of the address, with all quoting removed.
-
-   .. attribute:: domain
-
-      The ``domain`` portion of the address.
-
-   .. attribute:: addr_spec
-
-      The ``username@domain`` portion of the address, correctly quoted
-      for use as a bare address (the second form shown above).  This
-      attribute is not mutable.
-
-   .. method:: __str__()
-
-      The ``str`` value of the object is the address quoted according to
-      :rfc:`5322` rules, but with no Content Transfer Encoding of any non-ASCII
-      characters.
-
-
-.. class:: Group(display_name=None, addresses=None)
-
-   The class used to represent an address group.  The general form of an
-   address group is::
-
-     display_name: [address-list];
-
-   As a convenience for processing lists of addresses that consist of a mixture
-   of groups and single addresses, a ``Group`` may also be used to represent
-   single addresses that are not part of a group by setting *display_name* to
-   ``None`` and providing a list of the single address as *addresses*.
-
-   .. attribute:: display_name
-
-      The ``display_name`` of the group.  If it is ``None`` and there is
-      exactly one ``Address`` in ``addresses``, then the ``Group`` represents a
-      single address that is not in a group.
-
-   .. attribute:: addresses
-
-      A possibly empty tuple of :class:`.Address` objects representing the
-      addresses in the group.
-
-   .. method:: __str__()
-
-      The ``str`` value of a ``Group`` is formatted according to :rfc:`5322`,
-      but with no Content Transfer Encoding of any non-ASCII characters.  If
-      ``display_name`` is none and there is a single ``Address`` in the
-      ``addresses` list, the ``str`` value will be the same as the ``str`` of
-      that single ``Address``.
+The custom header objects and their attributes are described in
+:mod:`~email.headerregistry`.

Lib/email/policy.py

@@ -4,7 +4,7 @@ code that adds all the email6 features.
 
 from email._policybase import Policy, Compat32, compat32
 from email.utils import _has_surrogates
-from email._headerregistry import HeaderRegistry as _HeaderRegistry
+from email.headerregistry import HeaderRegistry as HeaderRegistry
 
 __all__ = [
     'Compat32',
@@ -60,13 +60,13 @@ class EmailPolicy(Policy):
     """
 
     refold_source = 'long'
-    header_factory = _HeaderRegistry()
+    header_factory = HeaderRegistry()
 
     def __init__(self, **kw):
         # Ensure that each new instance gets a unique header factory
         # (as opposed to clones, which share the factory).
         if 'header_factory' not in kw:
-            object.__setattr__(self, 'header_factory', _HeaderRegistry())
+            object.__setattr__(self, 'header_factory', HeaderRegistry())
         super().__init__(**kw)
 
     # The logic of the next three methods is chosen such that it is possible to

Lib/test/test_email/test__headerregistry.py → Lib/test/test_email/test_headerregistry.py

@@ -5,72 +5,71 @@ from email import errors
 from email import policy
 from email.message import Message
 from test.test_email import TestEmailBase
-from email import _headerregistry
-# Address and Group are public but I'm not sure where to put them yet.
-from email._headerregistry import Address, Group
+from email import headerregistry
+from email.headerregistry import Address, Group
 
 
 class TestHeaderRegistry(TestEmailBase):
 
     def test_arbitrary_name_unstructured(self):
-        factory = _headerregistry.HeaderRegistry()
+        factory = headerregistry.HeaderRegistry()
         h = factory('foobar', 'test')
-        self.assertIsInstance(h, _headerregistry.BaseHeader)
-        self.assertIsInstance(h, _headerregistry.UnstructuredHeader)
+        self.assertIsInstance(h, headerregistry.BaseHeader)
+        self.assertIsInstance(h, headerregistry.UnstructuredHeader)
 
     def test_name_case_ignored(self):
-        factory = _headerregistry.HeaderRegistry()
+        factory = headerregistry.HeaderRegistry()
         # Whitebox check that test is valid
         self.assertNotIn('Subject', factory.registry)
         h = factory('Subject', 'test')
-        self.assertIsInstance(h, _headerregistry.BaseHeader)
-        self.assertIsInstance(h, _headerregistry.UniqueUnstructuredHeader)
+        self.assertIsInstance(h, headerregistry.BaseHeader)
+        self.assertIsInstance(h, headerregistry.UniqueUnstructuredHeader)
 
     class FooBase:
         def __init__(self, *args, **kw):
             pass
 
     def test_override_default_base_class(self):
-        factory = _headerregistry.HeaderRegistry(base_class=self.FooBase)
+        factory = headerregistry.HeaderRegistry(base_class=self.FooBase)
         h = factory('foobar', 'test')
         self.assertIsInstance(h, self.FooBase)
-        self.assertIsInstance(h, _headerregistry.UnstructuredHeader)
+        self.assertIsInstance(h, headerregistry.UnstructuredHeader)
 
     class FooDefault:
-        parse = _headerregistry.UnstructuredHeader.parse
+        parse = headerregistry.UnstructuredHeader.parse
 
     def test_override_default_class(self):
-        factory = _headerregistry.HeaderRegistry(default_class=self.FooDefault)
+        factory = headerregistry.HeaderRegistry(default_class=self.FooDefault)
         h = factory('foobar', 'test')
-        self.assertIsInstance(h, _headerregistry.BaseHeader)
+        self.assertIsInstance(h, headerregistry.BaseHeader)
         self.assertIsInstance(h, self.FooDefault)
 
     def test_override_default_class_only_overrides_default(self):
-        factory = _headerregistry.HeaderRegistry(default_class=self.FooDefault)
+        factory = headerregistry.HeaderRegistry(default_class=self.FooDefault)
         h = factory('subject', 'test')
-        self.assertIsInstance(h, _headerregistry.BaseHeader)
-        self.assertIsInstance(h, _headerregistry.UniqueUnstructuredHeader)
+        self.assertIsInstance(h, headerregistry.BaseHeader)
+        self.assertIsInstance(h, headerregistry.UniqueUnstructuredHeader)
 
     def test_dont_use_default_map(self):
-        factory = _headerregistry.HeaderRegistry(use_default_map=False)
+        factory = headerregistry.HeaderRegistry(use_default_map=False)
         h = factory('subject', 'test')
-        self.assertIsInstance(h, _headerregistry.BaseHeader)
-        self.assertIsInstance(h, _headerregistry.UnstructuredHeader)
+        self.assertIsInstance(h, headerregistry.BaseHeader)
+        self.assertIsInstance(h, headerregistry.UnstructuredHeader)
 
     def test_map_to_type(self):
-        factory = _headerregistry.HeaderRegistry()
+        factory = headerregistry.HeaderRegistry()
         h1 = factory('foobar', 'test')
-        factory.map_to_type('foobar', _headerregistry.UniqueUnstructuredHeader)
+        factory.map_to_type('foobar', headerregistry.UniqueUnstructuredHeader)
         h2 = factory('foobar', 'test')
-        self.assertIsInstance(h1, _headerregistry.BaseHeader)
-        self.assertIsInstance(h1, _headerregistry.UnstructuredHeader)
-        self.assertIsInstance(h2, _headerregistry.BaseHeader)
-        self.assertIsInstance(h2, _headerregistry.UniqueUnstructuredHeader)
+        self.assertIsInstance(h1, headerregistry.BaseHeader)
+        self.assertIsInstance(h1, headerregistry.UnstructuredHeader)
+        self.assertIsInstance(h2, headerregistry.BaseHeader)
+        self.assertIsInstance(h2, headerregistry.UniqueUnstructuredHeader)
 
 
 class TestHeaderBase(TestEmailBase):
 
-    factory = _headerregistry.HeaderRegistry()
+    factory = headerregistry.HeaderRegistry()
 
     def make_header(self, name, value):
         return self.factory(name, value)
@@ -149,13 +148,13 @@ class TestDateHeader(TestHeaderBase):
 
     def test_date_header_properties(self):
         h = self.make_header('date', self.datestring)
-        self.assertIsInstance(h, _headerregistry.UniqueDateHeader)
+        self.assertIsInstance(h, headerregistry.UniqueDateHeader)
         self.assertEqual(h.max_count, 1)
         self.assertEqual(h.defects, ())
 
     def test_resent_date_header_properties(self):
         h = self.make_header('resent-date', self.datestring)
-        self.assertIsInstance(h, _headerregistry.DateHeader)
+        self.assertIsInstance(h, headerregistry.DateHeader)
         self.assertEqual(h.max_count, None)
         self.assertEqual(h.defects, ())
 

Lib/test/test_email/test_pickleable.py

@@ -4,7 +4,7 @@ import copy
 import pickle
 from email import policy
 from email import message_from_string
-from email._headerregistry import HeaderRegistry
+from email.headerregistry import HeaderRegistry
 from test.test_email import TestEmailBase
 
 class TestPickleCopyHeader(TestEmailBase):

Lib/test/test_email/test_policy.py

@@ -5,7 +5,7 @@ import unittest
 import email.policy
 import email.parser
 import email.generator
-from email import _headerregistry
+from email import headerregistry
 
 def make_defaults(base_defaults, differences):
     defaults = base_defaults.copy()
@@ -185,11 +185,11 @@ class PolicyAPITests(unittest.TestCase):
     def test_default_header_factory(self):
         h = email.policy.default.header_factory('Test', 'test')
         self.assertEqual(h.name, 'Test')
-        self.assertIsInstance(h, _headerregistry.UnstructuredHeader)
-        self.assertIsInstance(h, _headerregistry.BaseHeader)
+        self.assertIsInstance(h, headerregistry.UnstructuredHeader)
+        self.assertIsInstance(h, headerregistry.BaseHeader)
 
     class Foo:
-        parse = _headerregistry.UnstructuredHeader.parse
+        parse = headerregistry.UnstructuredHeader.parse
 
     def test_each_Policy_gets_unique_factory(self):
         policy1 = email.policy.EmailPolicy()
@@ -197,10 +197,10 @@ class PolicyAPITests(unittest.TestCase):
         policy1.header_factory.map_to_type('foo', self.Foo)
         h = policy1.header_factory('foo', 'test')
         self.assertIsInstance(h, self.Foo)
-        self.assertNotIsInstance(h, _headerregistry.UnstructuredHeader)
+        self.assertNotIsInstance(h, headerregistry.UnstructuredHeader)
         h = policy2.header_factory('foo', 'test')
         self.assertNotIsInstance(h, self.Foo)
-        self.assertIsInstance(h, _headerregistry.UnstructuredHeader)
+        self.assertIsInstance(h, headerregistry.UnstructuredHeader)
 
     def test_clone_copies_factory(self):
         policy1 = email.policy.EmailPolicy()