Saturday, 29 August 2026

Python Object Oriented Enums

Years ago, when I wrote this post about Python enums I mentioned this:

if you want something more advanced like the Java/Kotlin enums, where we can have multiple attributes as values and have instance methods (the archetypical Java Planets example), you can use the powerful aenum module (advanced enums), that just implements the Planets example.

Well, I've just found out that the standard Python enums support "enum members with associated data". You can add methods and properties to your enum classes, so you have a functionality pretty similar to the powerful Java enums. I've felt a bit like an idiot, cause after learning this from a code discussion with a GPT I realised that it's not any "python hidden feature", the standard enums documentation comes with a Planet example.


class Planet(Enum):
    MERCURY = (3.303e+23, 2.4397e6)
    VENUS   = (4.869e+24, 6.0518e6)
    EARTH   = (5.976e+24, 6.37814e6)
    MARS    = (6.421e+23, 3.3972e6)
    JUPITER = (1.9e+27,   7.1492e7)
    SATURN  = (5.688e+26, 6.0268e7)
    URANUS  = (8.686e+25, 2.5559e7)
    NEPTUNE = (1.024e+26, 2.4746e7)
    
    def __init__(self, mass, radius):
        self.mass = mass       # in kilograms
        self.radius = radius   # in meters
        
    @property
    def surface_gravity(self):
        # universal gravitational constant  (m3 kg-1 s-2)
        G = 6.67300E-11
        return G * self.mass / (self.radius * self.radius)

Planet.EARTH.value

Planet.EARTH.surface_gravity

That's the equivalent to the Java Planet one that years ago showed the world that enums could be much more than what we were used to.


public enum Planet {
    MERCURY (3.303e+23, 2.4397e6),
    VENUS   (4.869e+24, 6.0518e6),
    EARTH   (5.976e+24, 6.37814e6),
    MARS    (6.421e+23, 3.3972e6),
    JUPITER (1.9e+27,   7.1492e7),
    SATURN  (5.688e+26, 6.0268e7),
    URANUS  (8.686e+25, 2.5559e7),
    NEPTUNE (1.024e+26, 2.4746e7);

    private final double mass;   // in kilograms
    private final double radius; // in meters
    Planet(double mass, double radius) {
        this.mass = mass;
        this.radius = radius;
    }
    private double mass() { return mass; }
    private double radius() { return radius; }

    // universal gravitational constant  (m3 kg-1 s-2)
    public static final double G = 6.67300E-11;

    double surfaceGravity() {
        return G * mass / (radius * radius);
    }
    double surfaceWeight(double otherMass) {
        return otherMass * surfaceGravity();
    }

"Traditional" enums, like those in C or C# are just "a group of named constant values with type safety", they answered this question "How do I give names to a set of integer constants?". Java enums are "classes with a fixed set of instances defined at compile time", and they answer this question "How do I represent a fixed set of domain objects?"

Before Java added enums to the language, people were using the "Typesafe Enum Pattern" pattern to simulate "object-oriented enums". By the way, years ago I wrote about emulating Java enums in C#.

Java enums have an extra feature that is not directly implemented by Python enums, each member of the enum can have its own implementation of some methods (it's a per-instance override). Let's see an example:


enum Operation {
	PLUS { public int apply(int a, int b) { return a + b; } },
	
	TIMES { public int apply(int a, int b) { return a * b; } };
	
	public abstract int apply(int a, int b);
}

Python does not have a direct equivalent, a member can’t override a method. But there are idiomatic workarounds: carry the behaviour as data (a callable in the tuple) and dispatch on self inside one method:


class Operation(enum.Enum):
	PLUS = (operator.add,)
	TIMES = (operator.mul,)

	def __init__(self, fn: Callable[[int, int], int]) -> None:
		self.fn = fn

	def apply(self, a: int, b: int) -> int:
		return self.fn(a, b)

In Python we can dynamically look up an enum member both by name Color["RED"] and by value (Color("red")). The latter is not supported in Java. Related to this, we can customize behaviour when trying to access an element using a non existing value, by means of defining a _missing_ method:


from enum import Enum
class Color(Enum):
    RED = "red"
    BLUE = "blue"
    @classmethod
    def _missing_(cls, value):
        value = value.lower()
        for member in cls:
            if member.value == value:
                return member
        return None

Color.RED is Color("rEd")
# True

Saturday, 15 August 2026

Linux Dynamic Linking

A few months ago I wrote a post about glibc, it's a good time now to follow up with a related element of linux systems, ld-linux.so, the dynamic linker or dynamic loader.

If you need a refresh on what linking is, this wikipedia article should be enough. In this post I'll be talking about dynamic linking on Linux (performed by ld-linux.so), do not confuse it with the static linking performed by the ld tool (part of the binutils package, in turn part of the build-essential metapackage) to create an executable copying inside it the library code it references.

So dynamic linking means loading into a process memory at runtime the libraries (shared objects, .so's) that it needs. This loading is mainly done when the process starts (based on the imports defined in the ELF binary file), but can also be performed dynamically at any point during the process execution, with the dlopen function. In both cases, ld-linux.so (at least in Ubuntu its exact name is: /lib64/ld-linux-x86-64.so.2) is used to perform this loading and linking.

If ld-linux.so is used to load so's, and ld-linux is a so itself, this looks like a chicken-egg problem. Well, it's the kernel itself who takes care of loading it when a process is launched:

A binary names its loader in the ELF .interp section (readelf -p .interp /bin/ls → /lib64/ld-linux-x86-64.so.2). On execve the kernel reads .interp, hands control to that interpreter, which maps the libraries and jumps into the program.

Notice that the dynamic linker/loader is also known as the ELF interpreter.

When you run a dynamically linked program, the Linux kernel reads the executable's Executable and Linkable Format (ELF) header. It looks for a specific section called .interp (or the PT_INTERP program header), which contains the hardcoded string path to this exact interpreter. How the ELF Interpreter Works: Rather than running your program directly, the operating system kernel actually loads and hands control over to ld-linux first. The interpreter then performs several crucial tasks:
- Finds Dependencies: It scans your binary to see which shared libraries (such as libc.so) it needs.
- Loads Libraries: It locates those .so files on the disk and maps them into the program's memory space.
- Resolves Symbols: It performs "relocations," fixing memory references so your program knows exactly where library functions exist in memory.
- Launches Program: Once the environment is ready, it hands control back to your program's main entry point.

There's a strong relation between glibc and ld-linux. They are built together carrying the same version:

The loader (ld.so) and libc.so.6 are a version-locked matched pair. They are built from the same source tree in the same build and talk to each other over a private, unstable ABI: the GLIBC_PRIVATE symbols.
nm -D /lib64/ld-linux-x86-64.so.2 | grep GLIBC_PRIVATE # loader DEFINES them (T/D)nm -D /lib/x86_64-linux-gnu/libc.so.6 | grep GLIBC_PRIVATE # libc has them UNDEFINED (U)

libc.so.6 lists symbols like _dl_allocate_tls@GLIBC_PRIVATE, __tunable_get_val@GLIBC_PRIVATE, _dl_find_dso_for_object@GLIBC_PRIVATE as undefined — “the loader must hand these to me at startup.” The names, semantics, and the layouts behind them (TCB/TLS, _rtld_global) change freely between glibc versions.

ld-linux.so can be executed on its own, as a normal executable. If you run it with the --help version (/lib64/ld-linux-x86-64.so.2 --help) you get this interesting information.

You have invoked 'ld.so', the program interpreter for dynamically-linked ELF programs. Usually, the program interpreter is invoked automatically when a dynamically-linked executable is started.

You may invoke the program interpreter program directly from the command line to load and run an ELF executable file; this is like executing that file itself, but always uses the program interpreter you invoked, instead of the program interpreter specified in the executable file you run. Invoking the program interpreter directly provides access to additional diagnostics, and changing the dynamic linker behavior without setting environment variables (which would be inherited by subprocesses).

libc.so also happens to have an entry point, but it seems to have no other function that showing some information about itself: (/lib/x86_64-linux-gnu/libc.so.6 --version)

GNU C Library (Ubuntu GLIBC 2.39-0ubuntu8.8) stable release version 2.39.
Copyright (C) 2024 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
Compiled by GNU CC version 13.3.0.
libc ABIs: UNIQUE IFUNC ABSOLUTE
Minimum supported kernel: 3.2.0

Invoking ld-linux.so with --version in the same system you'll see that it's the same version as glibc, /lib64/ld-linux-x86-64.so.2 --version

ld.so (Ubuntu GLIBC 2.39-0ubuntu8.8) stable release version 2.39.

Notice that in the glibc information you can see "Minimum supported kernel: 3.2.0". Bearing in mind that my current kernel is 6.8.0, so we can say that glibc is quite little demanding with regards to kernel evolution.

Probably the ldd tool resonates with you. It's just a bash script wrapper (/usr/bin/ldd) around ld-linux.so

Saturday, 8 August 2026

Python Function Call

Over the years I've come to appreciate a real lot how coherent the Python object model is. For example, the callable concept applies to every object that can be invoked, starting by functions. When you define a function, you have an object that is an instance of the types.FunctionType class, a class that has a __call__ method. In contrast, Kotlin has the invokable concept, but this is an extra (the invoke operator) that you can add to your classes, but the basic invokable element, a method/function, is not an instance of a class with an invoke operator. Functions/methods in Kotlin (at the Kotlin language level, then at a JVM or JavaScript runtime things differ) are not objects, we have to get a function reference or a method reference (with the :: syntax) to treat them as objects.

So having in mind that the callable concept applies to normal functions and methods has made me to dive deeper into what I explained in a recent post:

At the C level, all Python objects are represented by the PyObject structure, which points to a PyTypeObject (its type). The type object defines how instances of that type behave.

To make calls fast and avoid creating temporary tuple/dict arguments, CPython uses the vectorcall protocol (Py_TPFLAGS_HAVE_VECTORCALL).
When the interpreter encounters a CALL opcode, it ultimately looks at the target object's type to find its vectorcall entry point:

- Pure Python Functions (PyFunction_Type): The vectorcall pointer points to _PyFunction_Vectorcall. This function extracts the function's PyCodeObject (func->func_code), allocates a new _PyInterpreterFrame on the evaluation stack, and pushes it to _PyEval_EvalFrameDefault.

- Built-in/Native Functions (PyCFunction_Type): Native functions (like math.sqrt or print) are wrapped in a PyCFunctionObject. Their vectorcall pointer points to _PyCFunction_Vectorcall. This function extracts the underlying C function pointer (func->m_ml->ml_meth) and invokes the compiled C code directly.

But, based on the "callable protocol", when the interpreter encounters a CALL opcode, it should check if the type of the object has a __call__ method and invoke it, right? How does this fit with the above? Well, we have to separate language semantics from implementation:

It's a classic example of language semantics vs. implementation details:

At the language level (Python semantics): Everything is an object, and every object that can be invoked implements the callable protocol via __call__. Defining __call__ on function and builtin_function_or_method keeps the object model clean, predictable, and fully inspectable.

At the runtime level (C implementation): The interpreter is allowed to use any shortcut it wants as long as it doesn't break the language semantics. Since invoking fn() is the most critical operation in the language, CPython uses type-checking fast paths (ob_type) to skip the overhead of attribute lookups.

Going deeper:

Looking up __call__ via the attribute resolution mechanism on every function call would carry massive performance overhead. Here is how CPython handles this under the hood and how it fits with the previous explanation. 1. Fast Paths Bypass __call__ Entirely

For pure Python functions (types.FunctionType) and standard C builtins (types.BuiltinFunctionType), CPython completely bypasses the __call__ method lookup.

Even though type(func).__call__ exists at the Python language level (defined on function / PyFunction_Type), the bytecode interpreter does not execute a dictionary lookup for __call__ when you write func().

Instead:

The interpreter checks Py_TYPE(callable).

If it matches &PyFunction_Type or &PyCFunction_Type, it executes the direct fast paths explained previously (allocating a _PyInterpreterFrame or invoking the vectorcall C function pointer directly).

As for custom callables (instances of classes with a __call__ method), the mechanism is also optimized for performance (it's not doing an expensive MRO __call__ search for any object). The structure representing and object's type, PyTypeObject, has 2 fields, tp_call and tp_vectorcall_offset serving different generations of CPython's calling API. tp_vectorcall_offset is used among others for bound methods and functools.partial. tp_call is used for classes that define a __call__ method (and their derived classes). In that case tp_call points to the generic C wrapper that performs the lookup for "__call__" in the object's class dictionary/MRO and invokes it (and so, if the object does not have a tp_call no search is performed at all).

If we have a function: def fn(), we can invoke it normally, via fn(), but also like this: fn.__call__(). The latter invokation form ends up creating the corresponding _PyInterpreterFrame and running fn's code_object in _PyEval_EvalFrameDefault(), but does it through quite a few extra steps. First we have the look up of the __call__ attribute in the fn object, which will find it in the Function class (types.FunctionType). Retrieving it will give us a bound method (types.MethodType), where __self__ is the fn function, and __func__ is the __call__ function. Then, we have the normal invokation of that bound method, that will end up executing the code object for the fn function.