Python Metaclasses

Metaclasses are a very interesting topic. The basic idea is In object-oriented programming, a metaclass is a class whose instances are classes.. That means that basically all Object Oriented languages have metaclasses. In C# all classes are instances of Type, in Java all classes are instances of Class... So the interesting thing comes when we can define custom metaclasses, as we can do for example in Groovy and Python.

I talked about Groovy metaclasses so many years ago. Custom metaclasses in Grooy are frequently used, while that's not the case in Python. The reason for this is that most of the features that require metaclasses in Groovy (expanding and object or class, interception...) are present in python through different means (__getattribute__, __getattr__...)

This means that custom metaclasses are rarelly necessary in Python, but anyway, I find it useful to have an (at least partial) understanding of how they work and their relation with callables and the attribute lookup mechanism.

This is probably the most in depth article I've found about metaclasses. I can not say I've fully understood it... but with it and other readings I think I have the basic idea of how things work. Let's go

type
Classes and functions are objects. object is an instance of type. type is an instance of type and inherits from object. A class is an instance of type. A metaclass is an instance of type and inherits from type. To find the metaclass of a class A, we just do type(A).

Callables
When python comes across this: x() it checks what class x is an instance of, and will call the __call__ method in that class with x as first parameter.

Creation of an Instance of a class
When creating an instance of a class A (a = A()) it invokes its metaclass .__call__ (so for "normal" classes it's type.__call__ and for classes with a custom metaclass it's CustomMetaclass.__call__). Then type.__call__(cls, *args, *kwargs) basically does this: invokes cls.__new__(cls) (that unless overriden is object.__new__) and with the returned instance object it invokes cls.__init__(instance)

Creation of a class
When creating a class A (class A:) it's like doing A = type("A", bases, attributes). So it invokes __call__ in type's metaclass, that happens to be type itself, so type.__call__(type). Then this invokes type.__new__ and type.__init__ (by the way, in case you are wondering whether type.__new__ is just the same as object.__new__, no, they are not)

Custom metaclass
- When creating a class with a custom metaclass (class A(metaclass=MetaB):), we are creating an instance of MetaB, so we end up calling MetaB.__new__ and MetaB.__init____.
- When creating an instance of a class A that has a metaclass MetaB (a = A()), this calls MetaB.__call__ (that could have been overriden in MetaB, or otherwise will be type.__call__)

From all the above, we can get to the 2 main uses of metaclasses:
- Defining __new__ and/or __init__ in the metaclass we can manage how instances of that metaclass (classes) are created. - Defining a custom __call__ method in the metaclass we can manage how instances of classes of that metaclass are created (singleton?).

This article makes a good read about the above. Just to make the signatures clear:

class MyMeta(type):
    def __new__(self,name,base,ns):
        print("meta new")
        return type.__new__(self,name,base,ns)
    def __init__(self,name,base,ns):
        print("meta init") self.cloned = False
        type.__init__(self,name,base,ns) class 

metaclasse and method lookup

Metaclasses do not add extra layers to the attribute lookup mechanism. Let's say we have "class A" and "a = A()". When looking up an attribute at1 in object a (a.at1), python searches first in a's __dict__ (or slots) and if not there, it'll check what class a is an instance of (that is, A), and will search in A's __dict__ and in that of its base classes. It will not use A's metaclass in this lookup mechanims at all.

If we search an attribute in a class (A.att1), python searches first in A.__dict__, then as it sees that is an instance of type (or a derived metaclass) or maybe just because it sees that it has a __bases__ attribute, it checks in the __dict__ of its parents. If not there, as A is an instance of type, it will search in type.__dict__. So yes, in this case it's using the metaclass, but just because it's the first "instance of", but it does not go deeper with the "instance of" of "instance of" of...

metaclasses and inheritance Custom metaclasses are inherited from your base classes. With multiple inheritance it becomes a bit complex, it's perfectly explained in the superb article that I mentioned above.

Immutable
You can not switch the metaclass of a class for a different metaclass once it has been created (and contrary to what some posts say, it makes sense to want to do that, as changing the metaclass of a class can change how instances of the class are created). However, there's an easy workaround, similar to what I do in my previous post. Create a new class that inherits from the one we want to modify and assign it the new metaclass. I mean:

class Meta1(type):
	pass
	
class A(metaclass=Meta1):
	pass

class Meta2(type):
	pass	

# now, let's "change" the metaclass of A from Meta1 to Meta2

class A2(A, metaclass=Meta2):
	pass
	

And then use A2 rather than A for creating new instances. We've previously seen that for already existing instances the metaclass of it's class does not play any role, so not being able to change it does not pose any problem.