Python Dynamism Oddities

I've always been fascinated by features provided in a greater or lesser level by dynamic languages. Adding new attributes/properties to an object (either instance or class), changing the class (prototype in JavaScript) of an existing object, modifying the inheritance chain, hooking into the getting/setting of attributes... There's no doubt that Python is not short of these beautiful features (adding also the concept of metaclasses). I'll share here some quirks, rules, limitations... of some of these features that I've learnt about in the last weeks.

Freely adding new attributes to an object is what I consider one of the main features of a dynamic language. In Python we can do that with our custom classes and instances of those classes, but not with an instance of the object class itself, which throws an "'object' object has no attribute" Exception. This happens because an instance of object does not have a __dict__ attribute, that is where instance attributes (unless we are using slots) are kept. If we create an empty class inheriting from object, its instances have __dict__ and there's no problem in adding attibutes to it. We can read more about this here.

o1 = object()
print(dir(o1))
# o1 does not have __dict__
# ['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']

try:
    o1.city = "Paris"
except Exception as ex:
    print(f"Exception: {ex}")
#Exception: 'object' object has no attribute 'city'

class Ob(object):
    pass
    
o2 = Ob()
print(dir(o2))
# o2 has __dict__
#['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__']

o2.city = "Paris"
print(o2.city)
#Paris

A Python object has a __class__ attribute pointing to its class (let's say Class1), and that attribute can be set to another class (let's say Class2), changing dynamically the behaviour of that object (now it has Class2 methods rather than Class1 ones). A class has a __bases__ attribute that points to a tuple containing its inheritance chain. We can set that attribute to a different tuple, hence changing its inheritance chain, but there's a gotcha with that. If you class just derives from object, trying to reassign its __bases__ will throw an exception:
__bases__ assignment: 'XX' deallocator differs from 'object'
However, if your class derives from another class (for example just an empty class deriving from object) the reassignment to __bases__ works without a problem. The exception seems to be a bug that is known and remains open since 2003!!! (which is pretty odd to say the least). When the __bases__ reassignment works fine it also causes the MRO (method resolution order, __mro__ attribute) of the class to be recalculated.

class A1:
    pass

class A2:
    pass

class B:
    pass

# for classes that directly inherit from object __bases__ reassignment fails
try:
    B.__bases__ = (A1, A2)
except Exception as ex:
    print(f"exception! {ex}")
# exception! __bases__ assignment: 'A1' deallocator differs from 'object'

# but if we inherit from any other class (for example this "empty" one) it works
class Empty:
    pass

class C (Empty):
    pass

print(f"C.__mro__ before __bases__ reassignment: {C.__mro__}")
# C.__mro__ before __bases__ reassignment: (, , )
C.__bases__ = (A1, A2)
print(f"C.__mro__ before __bases__ reassignment: {C.__mro__}")
# C.__mro__ after __bases__ reassignment: (, , , )

Classes do not have a __metaclass__ attribute, which makes sense. As the metaclass is the class used to create a class, it's the __class__ attribute of the class object which points to the metaclass (same as the __class__ object in an instance of a class points to its class). This said, we could think that we can change the metaclass of a class by reassigning its __class__ attribute, but this reassignment works for instances, not for classes, for which we get an exception: __class__ assignment only supported for mutable types or ModuleType subclasses

class Meta1(type):
    pass

class A2(metaclass=Meta1):
    def __init__(self,  name):
        self.name = name

print(f"A2 metaclass: {A2.__class__}")
# A2 metaclass: 
print(f"A2 metaclass: {type(A2)}")
# A2 metaclass: 

try:
    A2.__class__ = type
except BaseException as ex:
    print(f"Exception! trying to change the metaclass: {ex}")
# Exception! trying to change the metaclass: __class__ assignment only supported for mutable types or ModuleType subclasses

Metaclasses are mainly used to manage how classes (instances of that metaclass) are created, so changing the metaclass of an existing class is useless for that. However, metaclasses can also have an effect on how instances of a class are created, by providing a custom __call__ method in the metaclass. (a = A() will invoke __call__ in A's metaclass). So we can change the __call__ method of a metaclass to modify how instances of the class are created

def _custom_call(cls, *args, **kwargs):
    print(f"invoking {cls.__name__} construction")
    instance = type.__call__(cls, *args, **kwargs)
    # Additional customization if needed
    return instance

Meta1.__call__ = _custom_call
a2 = A2("Iyan")
# invoking A2 construction
print(f"a2 type: {type(a2).__name__}")
# a2 type: A2
print(f"a2.name: {a2.name}")
# a2.name: Iyan

There's an additional use of metaclasses that was unknown to me, defining a different MRO (method resolution order, stored in the __mro__ attribute) for classes of that metaclass. I discovered this feature in this post. Defining a custom mro() method in the metaclass changes the MRO of a class at class creation type. You can not change the MRO of an existing class by reassigning its __mro__ attribute, as it's readonly. There's a crazy dicussion about a hack for that here.

By overriding mro() on the metaclass we can define a custom __mro__ for our class. Python will then traverse it instead of the default implementation, which is provided by type.mro().