improve abstract property support (closes #11610)

Thanks to Darren Dale for patch.

Doc/library/abc.rst

@@ -127,19 +127,18 @@ This module provides the following class:
    available as a method of ``Foo``, so it is provided separately.
 
 
-It also provides the following decorators:
+The :mod:`abc` module also provides the following decorators:
 
 .. decorator:: abstractmethod(function)
 
    A decorator indicating abstract methods.
 
-   Using this decorator requires that the class's metaclass is :class:`ABCMeta` or
-   is derived from it.
-   A class that has a metaclass derived from :class:`ABCMeta`
-   cannot be instantiated unless all of its abstract methods and
-   properties are overridden.
-   The abstract methods can be called using any of the normal 'super' call
-   mechanisms.
+   Using this decorator requires that the class's metaclass is :class:`ABCMeta`
+   or is derived from it.  A class that has a metaclass derived from
+   :class:`ABCMeta` cannot be instantiated unless all of its abstract methods
+   and properties are overridden.  The abstract methods can be called using any
+   of the normal 'super' call mechanisms.  :func:`abstractmethod` may be used
+   to declare abstract methods for properties and descriptors.
 
    Dynamically adding abstract methods to a class, or attempting to modify the
    abstraction status of a method or class once it is created, are not
@@ -147,12 +146,52 @@ It also provides the following decorators:
    regular inheritance; "virtual subclasses" registered with the ABC's
    :meth:`register` method are not affected.
 
-   Usage::
+   When :func:`abstractmethod` is applied in combination with other method
+   descriptors, it should be applied as the innermost decorator, as shown in
+   the following usage examples::
 
       class C(metaclass=ABCMeta):
           @abstractmethod
           def my_abstract_method(self, ...):
               ...
+          @classmethod
+          @abstractmethod
+          def my_abstract_classmethod(cls, ...):
+              ...
+          @staticmethod
+          @abstractmethod
+          def my_abstract_staticmethod(...):
+              ...
+
+          @property
+          @abstractmethod
+          def my_abstract_property(self):
+              ...
+          @my_abstract_property.setter
+          @abstractmethod
+          def my_abstract_property(self, val):
+              ...
+
+          @abstractmethod
+          def _get_x(self):
+              ...
+          @abstractmethod
+          def _set_x(self, val):
+              ...
+          x = property(_get_x, _set_x)
+
+   In order to correctly interoperate with the abstract base class machinery,
+   the descriptor must identify itself as abstract using
+   :attr:`__isabstractmethod__`. In general, this attribute should be ``True``
+   if any of the methods used to compose the descriptor are abstract. For
+   example, Python's built-in property does the equivalent of::
+
+      class Descriptor:
+          ...
+          @property
+          def __isabstractmethod__(self):
+              return any(getattr(f, '__isabstractmethod__', False) for
+                         f in (self._fget, self._fset, self._fdel))
 
    .. note::
 
@@ -177,6 +216,8 @@ It also provides the following decorators:
               ...
 
    .. versionadded:: 3.2
+   .. deprecated:: 3.3
+       Use :class:`classmethod` with :func:`abstractmethod` instead
 
 
 .. decorator:: abstractstaticmethod(function)
@@ -192,18 +233,19 @@ It also provides the following decorators:
               ...
 
    .. versionadded:: 3.2
+   .. deprecated:: 3.3
+       Use :class:`staticmethod` with :func:`abstractmethod` instead
 
 
 .. decorator:: abstractproperty(fget=None, fset=None, fdel=None, doc=None)
 
    A subclass of the built-in :func:`property`, indicating an abstract property.
 
-   Using this function requires that the class's metaclass is :class:`ABCMeta` or
-   is derived from it.
-   A class that has a metaclass derived from :class:`ABCMeta` cannot be
-   instantiated unless all of its abstract methods and properties are overridden.
-   The abstract properties can be called using any of the normal
-   'super' call mechanisms.
+   Using this function requires that the class's metaclass is :class:`ABCMeta`
+   or is derived from it. A class that has a metaclass derived from
+   :class:`ABCMeta` cannot be instantiated unless all of its abstract methods
+   and properties are overridden. The abstract properties can be called using
+   any of the normal 'super' call mechanisms.
 
    Usage::
 
@@ -220,6 +262,9 @@ It also provides the following decorators:
           def setx(self, value): ...
           x = abstractproperty(getx, setx)
 
+   .. deprecated:: 3.3
+       Use :class:`property` with :func:`abstractmethod` instead
+
 
 .. rubric:: Footnotes
 

Doc/whatsnew/3.3.rst

@@ -352,6 +352,23 @@ curses
 
 (Contributed by Iñigo Serna in :issue:`6755`)
 
+abc
+---
+
+Improved support for abstract base classes containing descriptors composed with
+abstract methods. The recommended approach to declaring abstract descriptors is
+now to provide :attr:`__isabstractmethod__` as a dynamically updated
+property. The built-in descriptors have been updated accordingly.
+
+  * :class:`abc.abstractproperty` has been deprecated, use :class:`property`
+    with :func:`abc.abstractmethod` instead.
+  * :class:`abc.abstractclassmethod` has been deprecated, use
+    :class:`classmethod` with :func:`abc.abstractmethod` instead.
+  * :class:`abc.abstractstaticmethod` has been deprecated, use
+    :class:`property` with :func:`abc.abstractmethod` instead.
+
+(Contributed by Darren Dale in :issue:`11610`)
+
 faulthandler
 ------------
 

Include/object.h

@@ -473,6 +473,7 @@ PyAPI_FUNC(int) PyObject_HasAttrString(PyObject *, const char *);
 PyAPI_FUNC(PyObject *) PyObject_GetAttr(PyObject *, PyObject *);
 PyAPI_FUNC(int) PyObject_SetAttr(PyObject *, PyObject *, PyObject *);
 PyAPI_FUNC(int) PyObject_HasAttr(PyObject *, PyObject *);
+PyAPI_FUNC(int) _PyObject_IsAbstract(PyObject *);
 PyAPI_FUNC(PyObject *) _PyObject_GetAttrId(PyObject *, struct _Py_Identifier *);
 PyAPI_FUNC(int) _PyObject_SetAttrId(PyObject *, struct _Py_Identifier *, PyObject *);
 PyAPI_FUNC(int) _PyObject_HasAttrId(PyObject *, struct _Py_Identifier *);

Lib/abc.py

@@ -26,7 +26,8 @@ def abstractmethod(funcobj):
 
 
 class abstractclassmethod(classmethod):
-    """A decorator indicating abstract classmethods.
+    """
+    A decorator indicating abstract classmethods.
 
     Similar to abstractmethod.
 
@@ -36,6 +37,9 @@ class abstractclassmethod(classmethod):
             @abstractclassmethod
             def my_abstract_classmethod(cls, ...):
                 ...
+
+    'abstractclassmethod' is deprecated. Use 'classmethod' with
+    'abstractmethod' instead.
     """
 
     __isabstractmethod__ = True
@@ -46,7 +50,8 @@ class abstractclassmethod(classmethod):
 
 
 class abstractstaticmethod(staticmethod):
-    """A decorator indicating abstract staticmethods.
+    """
+    A decorator indicating abstract staticmethods.
 
     Similar to abstractmethod.
 
@@ -56,6 +61,9 @@ class abstractstaticmethod(staticmethod):
             @abstractstaticmethod
             def my_abstract_staticmethod(...):
                 ...
+
+    'abstractstaticmethod' is deprecated. Use 'staticmethod' with
+    'abstractmethod' instead.
     """
 
     __isabstractmethod__ = True
@@ -66,7 +74,8 @@ class abstractstaticmethod(staticmethod):
 
 
 class abstractproperty(property):
-    """A decorator indicating abstract properties.
+    """
+    A decorator indicating abstract properties.
 
     Requires that the metaclass is ABCMeta or derived from it.  A
     class that has a metaclass derived from ABCMeta cannot be
@@ -88,7 +97,11 @@ class abstractproperty(property):
             def getx(self): ...
             def setx(self, value): ...
             x = abstractproperty(getx, setx)
+
+    'abstractproperty' is deprecated. Use 'property' with 'abstractmethod'
+    instead.
     """
+
     __isabstractmethod__ = True
 
 

Lib/numbers.py

@@ -5,7 +5,7 @@
 
 TODO: Fill out more detailed documentation on the operators."""
 
-from abc import ABCMeta, abstractmethod, abstractproperty
+from abc import ABCMeta, abstractmethod
 
 __all__ = ["Number", "Complex", "Real", "Rational", "Integral"]
 
@@ -50,7 +50,8 @@ class Complex(Number):
         """True if self != 0. Called for bool(self)."""
         return self != 0
 
-    @abstractproperty
+    @property
+    @abstractmethod
     def real(self):
         """Retrieve the real component of this number.
 
@@ -58,7 +59,8 @@ class Complex(Number):
         """
         raise NotImplementedError
 
-    @abstractproperty
+    @property
+    @abstractmethod
     def imag(self):
         """Retrieve the imaginary component of this number.
 
@@ -272,11 +274,13 @@ class Rational(Real):
 
     __slots__ = ()
 
-    @abstractproperty
+    @property
+    @abstractmethod
     def numerator(self):
         raise NotImplementedError
 
-    @abstractproperty
+    @property
+    @abstractmethod
     def denominator(self):
         raise NotImplementedError
 

Lib/test/test_abc.py

@@ -10,14 +10,7 @@ import abc
 from inspect import isabstract
 
 
-class TestABC(unittest.TestCase):
-
-    def test_abstractmethod_basics(self):
-        @abc.abstractmethod
-        def foo(self): pass
-        self.assertTrue(foo.__isabstractmethod__)
-        def bar(self): pass
-        self.assertFalse(hasattr(bar, "__isabstractmethod__"))
+class TestLegacyAPI(unittest.TestCase):
 
     def test_abstractproperty_basics(self):
         @abc.abstractproperty
@@ -29,10 +22,12 @@ class TestABC(unittest.TestCase):
         class C(metaclass=abc.ABCMeta):
             @abc.abstractproperty
             def foo(self): return 3
+        self.assertRaises(TypeError, C)
         class D(C):
             @property
             def foo(self): return super().foo
         self.assertEqual(D().foo, 3)
+        self.assertFalse(getattr(D.foo, "__isabstractmethod__", False))
 
     def test_abstractclassmethod_basics(self):
         @abc.abstractclassmethod
@@ -40,7 +35,7 @@ class TestABC(unittest.TestCase):
         self.assertTrue(foo.__isabstractmethod__)
         @classmethod
         def bar(cls): pass
-        self.assertFalse(hasattr(bar, "__isabstractmethod__"))
+        self.assertFalse(getattr(bar, "__isabstractmethod__", False))
 
         class C(metaclass=abc.ABCMeta):
             @abc.abstractclassmethod
@@ -58,7 +53,7 @@ class TestABC(unittest.TestCase):
         self.assertTrue(foo.__isabstractmethod__)
         @staticmethod
         def bar(): pass
-        self.assertFalse(hasattr(bar, "__isabstractmethod__"))
+        self.assertFalse(getattr(bar, "__isabstractmethod__", False))
 
         class C(metaclass=abc.ABCMeta):
             @abc.abstractstaticmethod
@@ -98,6 +93,163 @@ class TestABC(unittest.TestCase):
             self.assertRaises(TypeError, F)  # because bar is abstract now
             self.assertTrue(isabstract(F))
 
+
+class TestABC(unittest.TestCase):
+
+    def test_abstractmethod_basics(self):
+        @abc.abstractmethod
+        def foo(self): pass
+        self.assertTrue(foo.__isabstractmethod__)
+        def bar(self): pass
+        self.assertFalse(hasattr(bar, "__isabstractmethod__"))
+
+    def test_abstractproperty_basics(self):
+        @property
+        @abc.abstractmethod
+        def foo(self): pass
+        self.assertTrue(foo.__isabstractmethod__)
+        def bar(self): pass
+        self.assertFalse(getattr(bar, "__isabstractmethod__", False))
+
+        class C(metaclass=abc.ABCMeta):
+            @property
+            @abc.abstractmethod
+            def foo(self): return 3
+        self.assertRaises(TypeError, C)
+        class D(C):
+            @C.foo.getter
+            def foo(self): return super().foo
+        self.assertEqual(D().foo, 3)
+
+    def test_abstractclassmethod_basics(self):
+        @classmethod
+        @abc.abstractmethod
+        def foo(cls): pass
+        self.assertTrue(foo.__isabstractmethod__)
+        @classmethod
+        def bar(cls): pass
+        self.assertFalse(getattr(bar, "__isabstractmethod__", False))
+
+        class C(metaclass=abc.ABCMeta):
+            @classmethod
+            @abc.abstractmethod
+            def foo(cls): return cls.__name__
+        self.assertRaises(TypeError, C)
+        class D(C):
+            @classmethod
+            def foo(cls): return super().foo()
+        self.assertEqual(D.foo(), 'D')
+        self.assertEqual(D().foo(), 'D')
+
+    def test_abstractstaticmethod_basics(self):
+        @staticmethod
+        @abc.abstractmethod
+        def foo(): pass
+        self.assertTrue(foo.__isabstractmethod__)
+        @staticmethod
+        def bar(): pass
+        self.assertFalse(getattr(bar, "__isabstractmethod__", False))
+
+        class C(metaclass=abc.ABCMeta):
+            @staticmethod
+            @abc.abstractmethod
+            def foo(): return 3
+        self.assertRaises(TypeError, C)
+        class D(C):
+            @staticmethod
+            def foo(): return 4
+        self.assertEqual(D.foo(), 4)
+        self.assertEqual(D().foo(), 4)
+
+    def test_abstractmethod_integration(self):
+        for abstractthing in [abc.abstractmethod, abc.abstractproperty,
+                              abc.abstractclassmethod,
+                              abc.abstractstaticmethod]:
+            class C(metaclass=abc.ABCMeta):
+                @abstractthing
+                def foo(self): pass  # abstract
+                def bar(self): pass  # concrete
+            self.assertEqual(C.__abstractmethods__, {"foo"})
+            self.assertRaises(TypeError, C)  # because foo is abstract
+            self.assertTrue(isabstract(C))
+            class D(C):
+                def bar(self): pass  # concrete override of concrete
+            self.assertEqual(D.__abstractmethods__, {"foo"})
+            self.assertRaises(TypeError, D)  # because foo is still abstract
+            self.assertTrue(isabstract(D))
+            class E(D):
+                def foo(self): pass
+            self.assertEqual(E.__abstractmethods__, set())
+            E()  # now foo is concrete, too
+            self.assertFalse(isabstract(E))
+            class F(E):
+                @abstractthing
+                def bar(self): pass  # abstract override of concrete
+            self.assertEqual(F.__abstractmethods__, {"bar"})
+            self.assertRaises(TypeError, F)  # because bar is abstract now
+            self.assertTrue(isabstract(F))
+
+    def test_descriptors_with_abstractmethod(self):
+        class C(metaclass=abc.ABCMeta):
+            @property
+            @abc.abstractmethod
+            def foo(self): return 3
+            @foo.setter
+            @abc.abstractmethod
+            def foo(self, val): pass
+        self.assertRaises(TypeError, C)
+        class D(C):
+            @C.foo.getter
+            def foo(self): return super().foo
+        self.assertRaises(TypeError, D)
+        class E(D):
+            @D.foo.setter
+            def foo(self, val): pass
+        self.assertEqual(E().foo, 3)
+        # check that the property's __isabstractmethod__ descriptor does the
+        # right thing when presented with a value that fails truth testing:
+        class NotBool(object):
+            def __nonzero__(self):
+                raise ValueError()
+            __len__ = __nonzero__
+        with self.assertRaises(ValueError):
+            class F(C):
+                def bar(self):
+                    pass
+                bar.__isabstractmethod__ = NotBool()
+                foo = property(bar)
+
+
+    def test_customdescriptors_with_abstractmethod(self):
+        class Descriptor:
+            def __init__(self, fget, fset=None):
+                self._fget = fget
+                self._fset = fset
+            def getter(self, callable):
+                return Descriptor(callable, self._fget)
+            def setter(self, callable):
+                return Descriptor(self._fget, callable)
+            @property
+            def __isabstractmethod__(self):
+                return (getattr(self._fget, '__isabstractmethod__', False)
+                        or getattr(self._fset, '__isabstractmethod__', False))
+        class C(metaclass=abc.ABCMeta):
+            @Descriptor
+            @abc.abstractmethod
+            def foo(self): return 3
+            @foo.setter
+            @abc.abstractmethod
+            def foo(self, val): pass
+        self.assertRaises(TypeError, C)
+        class D(C):
+            @C.foo.getter
+            def foo(self): return super().foo
+        self.assertRaises(TypeError, D)
+        class E(D):
+            @D.foo.setter
+            def foo(self, val): pass
+        self.assertFalse(E.foo.__isabstractmethod__)
+
     def test_metaclass_abc(self):
         # Metaclasses can be ABCs, too.
         class A(metaclass=abc.ABCMeta):

Lib/test/test_property.py

@@ -128,6 +128,29 @@ class PropertyTests(unittest.TestCase):
         self.assertEqual(newgetter.spam, 8)
         self.assertEqual(newgetter.__class__.spam.__doc__, "new docstring")
 
+    def test_property___isabstractmethod__descriptor(self):
+        for val in (True, False, [], [1], '', '1'):
+            class C(object):
+                def foo(self):
+                    pass
+                foo.__isabstractmethod__ = val
+                foo = property(foo)
+            self.assertIs(C.foo.__isabstractmethod__, bool(val))
+
+        # check that the property's __isabstractmethod__ descriptor does the
+        # right thing when presented with a value that fails truth testing:
+        class NotBool(object):
+            def __nonzero__(self):
+                raise ValueError()
+            __len__ = __nonzero__
+        with self.assertRaises(ValueError):
+            class C(object):
+                def foo(self):
+                    pass
+                foo.__isabstractmethod__ = NotBool()
+                foo = property(foo)
+            C.foo.__isabstractmethod__
+
 
 # Issue 5890: subclasses of property do not preserve method __doc__ strings
 class PropertySub(property):

Misc/ACKS

@@ -220,6 +220,7 @@ Tom Culliton
 Antonio Cuni
 Brian Curtin
 Lisandro Dalcin
+Darren Dale
 Andrew Dalke
 Lars Damerow
 Evan Dandrea

Misc/NEWS

@@ -416,6 +416,8 @@ Core and Builtins
 Library
 -------
 
+- Issue #11610: Introduce a more general way to declare abstract properties.
+
 - Issue #13591: A bug in importlib has been fixed that caused import_module
   to load a module twice.
 

Objects/descrobject.c

@@ -1380,6 +1380,43 @@ property_init(PyObject *self, PyObject *args, PyObject *kwds)
     return 0;
 }
 
+static PyObject *
+property_get___isabstractmethod__(propertyobject *prop, void *closure)
+{
+    int res = _PyObject_IsAbstract(prop->prop_get);
+    if (res == -1) {
+        return NULL;
+    }
+    else if (res) {
+        Py_RETURN_TRUE;
+    }
+
+    res = _PyObject_IsAbstract(prop->prop_set);
+    if (res == -1) {
+        return NULL;
+    }
+    else if (res) {
+        Py_RETURN_TRUE;
+    }
+
+    res = _PyObject_IsAbstract(prop->prop_del);
+    if (res == -1) {
+        return NULL;
+    }
+    else if (res) {
+        Py_RETURN_TRUE;
+    }
+    Py_RETURN_FALSE;
+}
+
+static PyGetSetDef property_getsetlist[] = {
+    {"__isabstractmethod__",
+     (getter)property_get___isabstractmethod__, NULL,
+     NULL,
+     NULL},
+    {NULL} /* Sentinel */
+};
+
 PyDoc_STRVAR(property_doc,
 "property(fget=None, fset=None, fdel=None, doc=None) -> property attribute\n"
 "\n"
@@ -1445,7 +1482,7 @@ PyTypeObject PyProperty_Type = {
     0,                                          /* tp_iternext */
     property_methods,                           /* tp_methods */
     property_members,                           /* tp_members */
-    0,                                          /* tp_getset */
+    property_getsetlist,                        /* tp_getset */
     0,                                          /* tp_base */
     0,                                          /* tp_dict */
     property_descr_get,                         /* tp_descr_get */

Objects/funcobject.c

@@ -814,6 +814,27 @@ static PyMemberDef cm_memberlist[] = {
     {NULL}  /* Sentinel */
 };
 
+static PyObject *
+cm_get___isabstractmethod__(classmethod *cm, void *closure)
+{
+    int res = _PyObject_IsAbstract(cm->cm_callable);
+    if (res == -1) {
+        return NULL;
+    }
+    else if (res) {
+        Py_RETURN_TRUE;
+    }
+    Py_RETURN_FALSE;
+}
+
+static PyGetSetDef cm_getsetlist[] = {
+    {"__isabstractmethod__",
+     (getter)cm_get___isabstractmethod__, NULL,
+     NULL,
+     NULL},
+    {NULL} /* Sentinel */
+};
+
 PyDoc_STRVAR(classmethod_doc,
 "classmethod(function) -> method\n\
 \n\
@@ -865,7 +886,7 @@ PyTypeObject PyClassMethod_Type = {
     0,                                          /* tp_iternext */
     0,                                          /* tp_methods */
     cm_memberlist,              /* tp_members */
-    0,                                          /* tp_getset */
+    cm_getsetlist,                              /* tp_getset */
     0,                                          /* tp_base */
     0,                                          /* tp_dict */
     cm_descr_get,                               /* tp_descr_get */
@@ -969,6 +990,27 @@ static PyMemberDef sm_memberlist[] = {
     {NULL}  /* Sentinel */
 };
 
+static PyObject *
+sm_get___isabstractmethod__(staticmethod *sm, void *closure)
+{
+    int res = _PyObject_IsAbstract(sm->sm_callable);
+    if (res == -1) {
+        return NULL;
+    }
+    else if (res) {
+        Py_RETURN_TRUE;
+    }
+    Py_RETURN_FALSE;
+}
+
+static PyGetSetDef sm_getsetlist[] = {
+    {"__isabstractmethod__",
+     (getter)sm_get___isabstractmethod__, NULL,
+     NULL,
+     NULL},
+    {NULL} /* Sentinel */
+};
+
 PyDoc_STRVAR(staticmethod_doc,
 "staticmethod(function) -> method\n\
 \n\
@@ -1017,7 +1059,7 @@ PyTypeObject PyStaticMethod_Type = {
     0,                                          /* tp_iternext */
     0,                                          /* tp_methods */
     sm_memberlist,              /* tp_members */
-    0,                                          /* tp_getset */
+    sm_getsetlist,                              /* tp_getset */
     0,                                          /* tp_base */
     0,                                          /* tp_dict */
     sm_descr_get,                               /* tp_descr_get */

Objects/object.c

@@ -840,6 +840,29 @@ PyObject_SetAttrString(PyObject *v, const char *name, PyObject *w)
     return res;
 }
 
+int
+_PyObject_IsAbstract(PyObject *obj)
+{
+    int res;
+    PyObject* isabstract;
+    _Py_IDENTIFIER(__isabstractmethod__);
+
+    if (obj == NULL)
+        return 0;
+
+    isabstract = _PyObject_GetAttrId(obj, &PyId___isabstractmethod__);
+    if (isabstract == NULL) {
+        if (PyErr_ExceptionMatches(PyExc_AttributeError)) {
+            PyErr_Clear();
+            return 0;
+        }
+        return -1;
+    }
+    res = PyObject_IsTrue(isabstract);
+    Py_DECREF(isabstract);
+    return res;
+}
+
 PyObject *
 _PyObject_GetAttrId(PyObject *v, _Py_Identifier *name)
 {