Python Decorators Implementation

There's something in the inner workings of the amazing multimethod library that we saw in my previous post that has had me pretty confused. In that post I outline how the multidispatch decorator works internally, but for the multimethod decorator, things are more complex. From my previous post we know we use it like this:

class Formatter:
    def __init__(self, wrapper: str):
        self.wrapper = wrapper

    @multimethod
    def format(self, item: str, starter: str):
        return f"{starter}{self.wrapper}{item}{self.wrapper}"
    
    @multimethod
    def format(self, item: int, starter: str):
        return f"{starter}{self.wrapper * 2}{item}{self.wrapper * 2}"   

multimethod is a class based decorator. In the first invokation it returns an instance of the multimethod class, that will store the format function and its signature. For each new invokation of the decorator it has to store each of those additional overload functions and signatures in that already existing instance, rather than creating a new instance each time. For doing that, the decorator checks if in the current scope (the class declaration scope) already exists a variable with the name of the function being decorated that already points to an instance of multimethod. For that it inspects the previous frame in the stack, like this (taken from its source code):

    def __new__(cls, func):
        homonym = inspect.currentframe().f_back.f_locals.get(func.__name__)
        if isinstance(homonym, multimethod):
            return homonym
        ...

That's really, really nice code, but it's another thing what I could not grasp. We know that Python decorators are just callables (functions or classes) that are invoked receiving the function (or class) being decorated as parameter. So they are normally explained like this:

@my_deco
def fn(): 
   # whatever

Is (in principle) just equivalent (syntactic sugar) to this:

def fn():
   # whatever
fn = my_deco(fn)

The problem is that it would mean that our example above is indeed translated by the Python compiler into something like this:

    def format(self, item: str, starter: str):
        return f"{starter}{self.wrapper}{item}{self.wrapper}"
    format = multimethod(format)
    # here format is pointing to a multimethod instance, good
    
   
    def format(self, item: int, starter: str):
        return f"{starter}{self.wrapper * 2}{item}{self.wrapper * 2}" 
    # but here format is pointing the the function that we've just defined, so when we apply the decorator again and it does the isinstance(homonym, multimethod) check, it will be False! so this can not work
    format = multimethod(format)

I've explained the problem in the comments. When defining the second format function, the format variable in the current scope is set to that second function, so it's no longer pointing the the multimethod instance previously created, so the isinstance(homonym, multimethod) check will be False! and a new multimethod instance will be created, so the whole thing can not work!

Well, after discussing this with Claude AI we've found an explanation. When applying a decorator to a function, the function definition doesn't immediately overwrite the namespace (setting a variable with the name of the function to point to the function), what is really happening is this:

1. def format(...) creates a temporary function object
2. @multimethod is applied to that temporary function object
3. The decorator returns an object (which could be the existing multimethod)
4. Only then does format = assignment happen

So indeed a decorator works more like this:

format = my_deco(
	def format():
	    # whatever
)

But as Python lacks statement lambdas, the above code is not valid, and hence the different articles explaining decorators can not use that as the pseudo-code for what the runtime really does, and use an inaccurate approximation. Usually that's not a problem, but for this very specific case that approximation prevented me from envisioning how this particular decorator works.