Python ABC's Implementation

In Python we normally define abstract classes by inheriting from abc.ABC, but the abstract classes logic really lives in the abc.ABCMeta metaclass. Given that the abc.ABC class has abc.ABCMeta as its metaclass, classes that inherit from abc.ABC also have abc.ABCMeta as its metaclass. This means that we can declare an abstract class by explicitly setting its metaclass MyAbstractClass(metaclass=abc.ABCMeta) (and indeed I think this was the initial way to declare abstract classes) or just by inheriting from the abc.ABC class (I guess this was introduced so that people would not even need to think about metaclasses).

PEP-3119 gives this interesting explanation of how abstract classes are implemented.

Implementation: The @abstractmethod decorator sets the function attribute __isabstractmethod__ to the value True. The ABCMeta.__new__ method computes the type attribute __abstractmethods__ as the set of all method names that have an __isabstractmethod__ attribute whose value is true. It does this by combining the __abstractmethods__ attributes of the base classes, adding the names of all methods in the new class dict that have a true __isabstractmethod__ attribute, and removing the names of all methods in the new class dict that don’t have a true __isabstractmethod__ attribute. If the resulting __abstractmethods__ set is non-empty, the class is considered abstract, and attempts to instantiate it will raise TypeError.

So this is a very interesting use case of metaclasses. As explained in the above paragraph, when we define an abstract class we are creating an instance of ABCMeta, so the __new__ method of ABCMeta is invoked. I was thinking that further leveraging metaclasses, ABCMeta would also have a __call__ method so that when you try to create an instance of an abstract class that __call__ method would get invoked, and it would perform that check of the __abstractmethods__ attribute, but I see in the source code of ABCMeta that such __call__ method does not exist. Quite intrigued by this, I downloaded cPython source code and searching for the "Can't instantiate abstract class" error message I found that this check is directly done in typeobject.c, in the function that I think creates an object.

So the __abstractmethods__ attribute is calculated when the class is defined. This is important because it does not play so good with the dynamic features of Python. Let's say we create a class that inherits from and abstract class an in which we do not implement one of the inherited abstract methods, so the class is "marked" as abstract. If then we dynamically add that method to the class, the __abstractmethods__ are not recalculated, and the class continues to be considered as abstract and not instantiable (we get the "Exception: Can't instantiate abstract class" error if we try to instantiate it). Let's see an example:

from abc import ABC, abstractmethod, update_abstractmethods

class Widget(ABC):
    @abstractmethod
    def draw(self):
        pass

    @abstractmethod
    def move(self):
        pass

class MovableItem:
    def move(self):
        print(f"I'm moving")

class Button(Widget):
    def draw(self):
        print(f"I'm drawing myself")

Button.move = MovableItem.move

try:
    # this instantiation fails
    b1 = Button()
except Exception as ex:
    print(f"Exception: {ex}")
#Exception: Can't instantiate abstract class Button with abstract method move

Hopefully we can ask the "abstract status" to be recalculated, by using update_abstractmethods

update_abstractmethods(Button)

try:
    # works fine
    b1 = Button()
    b1.move()
except Exception as ex:
    print(f"Exception: {ex}")

There's an additional issue that can not be fixed with update_abstractmethods. If I define a class that inherits from an abstract class and gets the implementation of one of the abstract methods from another parent class, such method is not considered as implemented, and it shows up in the __abstractmethods__ of the class, making it uninstantiable. This behaviour seems a bit odd to me, and I've verified that for example in Kotlin if one class implements an interface and extends a class, methods in the parent class can provide the implementation of the interface without a problem. If you read carefully the explanation in that paragraph above about how abstract classes are implemented, this behaviour makes sense, but as I've said it's not what I would expect.

abstract/inheritance1.py

from abc import ABC, abstractmethod, update_abstractmethods

class Widget(ABC):
    @abstractmethod
    def draw(self):
        pass

    @abstractmethod
    def move(self):
        pass


class MovableItem:
    def move(self):
        print(f"I'm moving")


class Button(Widget, MovableItem):
    def draw(self):
        print(f"I'm drawing myself")

try:
    b1 = Button()
    # though I'm inheriting the move method from MovableItem, the abstract one in Widget remains and I get an exception creating the object
except Exception as ex:
    print(f"Exception: {ex}")
# Exception: Can't instantiate abstract class Button with abstract method move

You can fix it by defining the problematic method in the derived class and invoking from it the implementation in the parent class.

class Button2(Widget, MovableItem):
    def draw(self):
        print(f"I'm drawing myself")
    
    def move(self):
        # if we use super the resolution ends up in the move of the abstract, Widget class, that does nothing
        #super().move()
        MovableItem.move(self)


b2 = Button2()
b2.draw()
b2.move()
# I'm drawing myself
# I'm moving

By the way, there's an interesting discussion here about the best way to check if a class is abstract.