Python "Power Instances"

We saw in my recent post about making an instance callable (__call__()) that the implicit search for __dunder__ methods will start searching for them in the class of the instance, not in the instance itself. So this happens also with __getattribute__, __setattribute__, we can change the look-up mechanism for an object by defining a __getattribute__, __setattribute__ in its class, but defining it directly in the instance will have no effect. Something similar happens also for descriptors, they are executed (__get__(), __set__()...) when found in a class, if found in an instance they are returned as such. I showed in that post a technique for overcoming this "limitation". We create a new class, add the dunder method(s) to it, and change the class of our instance to that new class (by reassigning the __class__ attribute). We could generalize this technique, having classes that create "power-instances", instances that we can easily not only do callable, but also define a __getattribute__, add descriptors, and why not, change its inheritance mechanism. All this for an specific instance, not for the whole class (which would affect all its instances).

So I've come up with the idea of having a decorator that decorates classes, turning them into classes which instances are "power-instances" on which we can alter the __getattribute__ behaviour, do __callable__ etc. The decorator returns a new class (PowerInstance class) inheriting from the original one and with additional methods (to do it callable, for adding instance methods, for adding properties, for interception, for adding a base class...). When this class is used for creating an instance it will return an instance not of this class, but of a new class (yes, we create a new class on each instantiation), that inherits from this PowerInstance class. The different additional methods that we defined in PowerInstance will perform the actions (add the specific __call__, __getattribute__(), modify inheritance...) on this new class. This is possible because we can return an instance of a new class rather than of the expected one by defining a new __new__() method in the class. OK, let's see the implementation:

from types import MethodType

# decorator that enhances a class so that we can add functionality just to the instance, not to the whole class
# it creates a new class PowerInstance that inherits from the original class
def power_instance(cls):
    class PowerInstance(cls):
        # each time we create an instance of this class, we create an instance of new Child class 
        # (in this Child class is where we add the extra things that we can't directly add to the instance)
        def __new__(_cls, *args, **kwargs):
            # it's interesting to note what someone says in OS that returning instances of different classes 
            # in a sense breaks object orientation, cause you would expect getting instances of a same class
            class PerInstanceClass(_cls):
                pass
            return super().__new__(PerInstanceClass)
            # remember this is how a normal __new__() looks like:
                # def __new__(cls, *args, **kwargs):
                #     return super().__new__(cls)
                  
        def _add_to_instance(self, item, item_name):
            # type(self) returns me the specific class (PerInstanceClass) created for this particular instance           
            setattr(type(self), item_name, item)

        def add_instance_method(self, fn, fn_name: str):
            self._add_to_instance(fn, fn_name)

        def add_instance_property(self, prop, prop_name: str):
            self._add_to_instance(prop, prop_name)

        def do_callable(self, call_fn):
            type(self).__call__ = call_fn

        def intercept_getattribute(self, call_fn):
            type(self).__getattribute__ = call_fn

        def do_instance_inherit_from(self, cls):
            # create a new class that inherits from my current class and from the provided one
            class PerInstanceNewChild(type(self), cls):
                pass
            #NewChild.__name__ = type(self).__name__
            self.__class__ = PerInstanceNewChild
    
    # there's no functools.wraps for a class, but we can do this, so the new class has more meaningful attributes
    for attr in '__doc__', '__name__', '__qualname__', '__module__':
        setattr(PowerInstance, attr, getattr(cls, attr))
    return PowerInstance
    

And now one usage example. Notice how we enrich the p1 instance and this affects only that specific instance, if I create a new instance of that class, it's "clean" of those features.

@power_instance
class Person:
    def __init__(self, name: str):
        self.name = name

    def say_hi(self, who: str) -> str:
        return f"{self.name} says Bonjour to {who}"

print(f"Person.__name__: {Person.__name__}")
p1 = Person("Iyan")
print(p1.say_hi("Antoine"))

def say_bye(self, who: str):
    return f"{self.name} says Bye to {who}"
p1.add_instance_method(say_bye, "say_bye")
print(p1.say_bye("Marc"))

p1.add_instance_property(property(lambda self: self.name.upper()), "upper_name")
print(p1.upper_name)

p1.do_callable(lambda self: f"[[{self.name.upper()}]]")
print(p1())

# Person.__name__: Person
# Iyan says Bonjour to Antoine
# Iyan says Bye to Marc
# IYAN
# [[IYAN]]

print("------------------")

# verify that say_bye is only accessible by p1, not by p2
p2 = Person("Francois")
print(p2.say_hi("Antoine"))

try:
    print(p2.say_bye("Marc"))
except Exception as ex:
    print(f"Exception! {ex}")

try:
    print(p2())
except Exception as ex:
    print(f"Exception! {ex}")

# Francois says Bonjour to Antoine
# Exception! 'PerInstanceClass' object has no attribute 'say_bye'
# Exception! 'PerInstanceClass' object is not callable

Let's do our instance extend another class:

class Animal:
    def growl(self, who: str) -> str:
        return f"{self.name} is growling to {who}"
        
p1.do_instance_inherit_from(Animal)

print(p1.growl("aa"))
print(p1.say_hi("bb"))
print(p1.say_bye("cc"))
print(p1())
print(f"mro: {type(p1).__mro__}")
print(f"bases: {type(p1).__bases__}")

# Iyan is growling to aa
# Iyan says Bonjour to bb
# Iyan says Bye to cc
# [[IYAN]]
# mro: (.PowerInstance.do_instance_inherit_from..PerInstanceNewChild'>, .PowerInstance.__new__..PerInstanceClass'>, , , , )
# bases: (.PowerInstance.__new__..PerInstanceClass'>, )

And now some interception via __getattribute__:

# intercept attribute access in the instance
print("- Interception:")

# interceptor that does NOT use "self" in the interception code
def interceptor(instance, attr_name):
    attr = object.__getattribute__(instance, attr_name)
    if not callable(attr):
        return attr
    
    def wrapper(*args, **kwargs):
        print(f"before invoking {attr_name}")
        # attr is already a bound method (if that's the case)
        res = attr(*args, **kwargs)
        print(f"after invoking {attr_name}")
        return res
    return wrapper

p1.intercept_getattribute(interceptor)
print(p1.say_hi("Antoine"))

p3 = Person("Francois")
# interceptor that does use "self" in the interception code
def interceptor2(instance, attr_name):
    attr = object.__getattribute__(instance, attr_name)
    if not callable(attr):
        return attr
    
    def wrapper(self, *args, **kwargs):
        print(f"before invoking {attr_name} in instance: {type(self)}")
        # attr is already a bound method (if that's the case)
        res = attr(*args, **kwargs)
        print(f"after invoking {attr_name} in instance: {type(self)}")
        return res
    
    return MethodType(wrapper, instance)        

p3.intercept_getattribute(interceptor2)
print(p3.say_hi("Antoine"))

# before invoking say_hi
# after invoking say_hi
# Iyan says Bonjour to Antoine

# before invoking say_hi in instance: .PowerInstance.__new__..PerInstanceClass'>
# after invoking say_hi in instance: .PowerInstance.__new__..PerInstanceClass'>
# Francois says Bonjour to Antoine

This thing of being able to have "constructors" that return something different from the expected "new instance of the class" is not something unique to Python. In JavaScript a function used as constructor (invoked via new) can return a new object, different from the one that new has created and passed over to it as "this". In Python construction-initialization is done in 2 steps, __new__ and __init__ methods, that are invoked by the __call__ method of the metaclass, so we normally end up in: type.__call__. In this discussion you can see a rough implementation of such type.__call__

# A regular instance method of type; we use cls instead of self to
# emphasize that an instance of a metaclass is a class
def __call__(cls, *args, **kwargs):
    rv = cls.__new__(cls, *args, **kwargs)  # Because __new__ is static!
    if isinstance(rv, cls):
        rv.__init__(*args, **kwargs)
    return rv

# Note that __init__ is only invoked if __new__ actually returns an instance of the class calling __new__ in the first place.

It's interesting to note that someone says in stackoverflow that returning from a constructor an instance of class different to that of the constructor, particularly a different class each time, in a sense breaks object orientation, cause normally you expect all instances returned by a constructor to be of the same class (this expectation does not exist if what you use is a factory function).