Python Class Body and Lexical Scope

In my previous post about class creation in Python I mentioned how a code object is created for the code in the class body, and then that code object is executed as a function receiving a namespace object (created by __prepare__) as its locals. The class body will add attributes to that namespace, and can use whatever is already present in that namespace (if __prepare__ has put something there). This has made me wonder if apart from that, the class body can have access to its enclosing scope. Just remember that in this post we saw that methods in a class have access to its enclosing scope (they close over variables defined outside the class).

So the answer is YES, and it's just the closures mechanism in action. Let's see an example:

def create_class(id: str):
    class MyClass:
        # the class initialiation (the class body) has access to "external" variables in the enclosing scope, such as "id"
        # the code in the class body is placed in a codeobject that will run in a function having trapped (closed over) the "id" free variable
        class_id = id
        
        print(f"Free variables: {inspect.currentframe().f_code.co_freevars}")
        # Free variables: ('id',)

        def __init__(self, value):
            self.value = value

        def display(self):
            print(f"MyClass value: {self.value}, Class ID: {self.class_id}")

    return MyClass

cl = create_class("123")
instance = cl("aa")
instance.display()

# Free variables: ('id',)
# locals: {'__module__': '__main__', '__qualname__': 'create_class..MyClass', '__firstlineno__': 7, 'class_id': '123'}
# MyClass value: aa, Class ID: 123

As you can see, that class body (my understanding is that Python will execute the code object corresponding to that class body by putting it in a "synthetic function") has access to the id variable in the outer scope and can assign it to one of its attributes. We can see that 'id' in the list of freevars for the code object of the class body (that we get accessing the current frame from the class body itself). However, I came across something that confused me. If I print locals() from the class body, I can't see 'id' there. That's very strange, if I do the same from a normal function, locals shows both "normal" variables and those that the function has trapped in its closure, but, as I've said, for the class body, 'id' is missing in locals:

def create_class(id: str):
    class MyClass:
        # the class initialiation (the class body) has access to "external" variables in the enclosing scope, such as "id"
        # the code in the class body is placed in a codeobject that will run in a function having trapped (closed over) the "id" free variable
        class_id = id
        
        print(f"Free variables: {inspect.currentframe().f_code.co_freevars}")
        # Free variables: ('id',)

        # notice that locals() does not show the id free var
        # that's because we are running in "class scope" and locals() just shows its namespace, not the closure cells. The closure is a separate object that holds references to the free variables, and it is not part of the local namespace of the class body. However, the class body can still access the free variable "id" through the closure.
        print(f"locals: {locals()}")
        # {'__module__': '__main__', '__qualname__': 'create_class..MyClass', '__firstlineno__': 4, 'class_id': '123'}

While for a normal function, it's well there:

def outer(id):
    def inner(value):
        # locals() shows "id" free variable because we are in a function scope (optimized scope)
        print(f"locals: {locals()}")
        # {'value': 'bb', 'id': '456'}
        print(f"Inner function value: {value}, Outer ID: {id}")
    return inner

inner_func = outer("456")
inner_func("bb")

So, why is that? At the time of the Python3.13 release I wrote a post about locals(), f_locals and the "local namespace" (this is related to PEP-667). Well, indeed what I mention on that post (based on other articles) about the "local namespace" as a sort of dictionary is not correct for normal functions in recent Python versions. In "normal" functions we are working in an Optimized Scope. In this optimized scope local variables are not placed in a dictionary and accessed by key, but in a fastarray, and accessed by index (you can see this using dis to check the bytecodes of a function). This locals fastarray, part of the _PyInterpreterFrame for each running function, contains both local variables (including function arguments and those local variables that are cells, cellvars, because they are trapped by inner functions in its closure) and variables trapped by the function itself in its closure:

In CPython's frame object, the `fastlocals` array is laid out as:

[regular locals] [cellvars] [freevars]

At the beginning of a function that has variables in its __closure__ the `COPY_FREE_VARS` bytecode instruction copies cell references from `__closure__` into the frame's fastlocals array for quick access!*
After `COPY_FREE_VARS` executes, all variables (normal locals, cellvars and freevars) are accessed from the fastlocals array during function execution.

By the way, regarding the aforementioned _PyInterpreterFrame, I'll leverage to copy here some GPT wisdom about frames in recent (Python 3.11 and above) Python versions

The _PyInterpreterFrame is an internal C struct introduced in Python 3.11 that represents a stack frame for execution, aiming to improve performance by reducing the overhead of allocating full Python PyFrameObject objects.

Purpose: It holds the execution state for code objects, including local variables, globals, builtins, and the instruction pointer (f_lasti).
Performance: Unlike older Python versions where every frame was a full heap-allocated PyFrameObject, _PyInterpreterFrame is designed to be lightweight and often lives on the C stack, reducing garbage collection pressure.

The traditional PyFrameObject still exists, but it has been relegated to a "shadow" role. It is now treated purely as a compatibility API wrapper.

Python only creates a PyFrameObject on demand when a tool or a piece of code explicitly asks to inspect the call stack. This process is often referred to as materializing a frame.