Friday, 27 February 2026

Python and Prototype-based languages

In my previous post I explained how class level attributes and the lookup mechanism in Python allow us sharing an attribute between instances through the class while it's only being read, and then getting it added to the instance when it's written through the instance rather than the class.

So we share the continent attribute between instances in Python like this:



>>> class Person:
...     continent: str = "Europe"
...     def __init__(self, name):
...         self.name = name
...         
>>> p1 = Person("Francois")
>>> p2 = Person("Iyan")
>>> print(p1.continent)
Europe
>>> print(p2.continent)
Europe
>>> p2.continent = "Asia"
>>> print(p1.continent)
Europe
>>> print(p2.continent)
Asia


This this not the first time I see this behaviour, I've known since many years ago that it exists in JavaScript, via the prototype chain, like this:


let person_proto = { continent: "Europe" }
let p1 = { name: "Francois" }
Object.setPrototypeOf(p1, person_proto);

let p2 = { name: "Iyan" }
Object.setPrototypeOf(p2, person_proto);

console.log(p1.continent); // Europe
console.log(p2.continent); // Europe

p2.continent = "Asia"

console.log(p1.continent); // Europe
console.log(p2.continent); // Asia

Notice that we can do it either directly setting the [[Prototype]] via Object.setPrototypeOf, or indirectly by setting the value in the prototype of a function and using that function as constructor:


console.log("-------------------")
function Person(name) {
    this.name = name;
}
Person.prototype.continent = "Europe";
p1 = new Person("Francois")
p2 = new Person("Iyan")

console.log(p1.continent); // Europe
console.log(p2.continent); // Europe

p2.continent = "Asia"

console.log(p1.continent); // Europe
console.log(p2.continent); // Asia

This feels like a feature of Prototype-based languages (JavaScript is by far the most widely used Prototype-based language), so one could wrongly wonder if somehow Python is also a Prototype-based language. Maybe it's like modern JavaScript, where classes are just syntactic sugar on top of prototypes. No, it's not. Python is a class based language, only that it behaves like a prototype‑based system in how it resolves attributes, but the defining point is that Python is not a prototype‑based language in how it constructs inheritance chains .

From the wikipedia article:

In prototype-based languages that use delegation, the language runtime is capable of dispatching the correct method or finding the right piece of data simply by following a series of delegation pointers (from object to its prototype) until a match is found. All that is required to establish this behavior-sharing between objects is the delegation pointer.

In JavaScript these "delegation pointers" make up the prototype chain. Each object is linked to it's prototype (via the [[Prototype]] internal property), and the attribute look up mechanism uses that chain to search for attributes. In Python that "delegation chain" is the MRO (Method Resolution Order) that depends on the __class__ and __bases__ attributes. Python dynamism allows us setting dynamically the class of an object (same as in JavaScript we can alter the prototype chain of an object) by setting its __class__ attribute to a different class. I made use of this technique in this previous post. So, having said this, why is it that Python is not considered as a Prototype based language?

Because there is a limitation. The __class__ attribute can only point to classes (objects that are instances of type), not to simple objects (because in Python there's a clear difference between objects that are classes (or metaclasses), that is, types, and objects that are not. If we try to set __class__ to "non type" object, we get an exception: TypeError: __class__ must be set to a class, not 'xxxx' object


class Animal:
	pass
	
class Person:
	pass
	
a1 = Animal()
p1 = Person()

a1.__class__ = Person

a1.__class__ = p1
Traceback (most recent call last):
  File "", line 1, in 
    a1.__class__ = p1
    ^^^^^^^^^^^^
TypeError: __class__ must be set to a class, not 'Person' object


That's the key, only classes can be used as "prototypes", so we're in a class based language.

Sunday, 22 February 2026

Python Class-Level Type Hints

Notice that in this post I'm talking about "standard" Python classes, not about dataclasses. I recently became aware of the possibility of using class-level type hints in your classes. The thing is that when reading the documentation I found it rather confusing. To make sense of it we have to be pretty aware of the difference between the intent that we express with those class hints and its runtime effects. So we have this example in the documentation:


class BasicStarship:
    captain: str = 'Picard'               # instance variable with default
    damage: int                           # instance variable without default
    stats: ClassVar[dict[str, int]] = {}  # class variable

The 'damage: int' part is the one that I knew about "class-level typehints" and was clear to me. We declare an attribute and its type, but we don't initialize it. Python takes this just as typing information, it has no runtime impact (other than being added to that class __annotations__), we are not creating an attribute in the class object.

The 'captain: str = 'Picard'' is what I could not understand. For me it's like the normal way of adding a class attribute, only that additionally you indicate the type, so how can it be that the doc says that it's an "instance variable with default". Well, it's the type-checking meaning vs the runtime effect. I am right that we get an attribute created at the class level (in the class __dict__), just see:


>>> class User:
...     continent: str = "Europe"
...     active = True
...

>>> User.__dict__
mappingproxy({'__module__': '__main__', '__firstlineno__': 1, '__annotations__': {'continent': }, 'continent': 'Europe', 'active': True, '__static_attributes__': (), '__dict__': , '__weakref__': , '__doc__': None})

>>> User.continent
'Europe'

>>> User.active
True

But for the type checker what that typed declaration means is that instances of that class will have a captain (or continent in my example) attribute. This could feel contradictory, but given how attribute look up works it's perfectly fine. Initially the 'captain' attribute is created at the class level. If we read it through an instance (my_ship.captain) the look up mechanism won't find it in the instance, but in the class, and return it. Then, when we write to it through an instance (not through the class) the writing will be done in the instance, so a 'captain' attribute will be added to the instance. That's fine, indeed, it's very nice, while the attribute is not being written to, just read, it's being shared between instances, kept in the class (and saving memory), then, as soon as you write to it, it's shadowed by the instance.


s = BasicStarship()
print(s.captain)       # "Picard" via class lookup
s.captain = "Xuan"     # creates an instance attribute
print(s.__dict__)      # {'captain': 'Xuan'}
print(BasicStarship.__dict__['captain'])  # 'Picard'

We can sumarize it like this:

Type hints alone do not create attributes; they only declare intent.
If you want the attribute to exist on the class (and thus be visible via Foo.x), you must assign a default value.

By the way, this is not the first time I see this behaviour of reading values from a "parent object" until we write the value to the object itself, shadowing it. This is just how things work in JavaScript with the [[Prototype]] chain.

I'm not much of a fan of defining instance attributes at the class level. It's true that it makes very explicit that an attribute is part of the public contract of the class, but I think most of the time it's a bit boilerplate. Type-checkers and autocomplete work perfectly fine with the classical style of initializing in the __init__ method, and if an attribute is internal/private and should not be considered part of the public API we should just follow the convention of starting it with '_'. So normally I would write the above code like this:



class AdvancedStarship:
    # stats = {} mypy will complain about this, because it is not a ClassVar
    stats: ClassVar[dict[str, int]] = {}  # class variable
    
    def __init__(self, damage: int, captain: str = 'Picard') -> None:
        self.captain = captain
        self.damage = damage



The case where these class-level type hints feel very useful to me is for Protocols, making unnecessary to declare the "data part" of the protocol with properties (get/set descriptors), that is the approach I used to follow so far.



from typing import Protocol

class Foo(Protocol):
    x: int  # part of the interface

class Bar:
    def __init__(self):
        self.x = 42  # matches Foo


It's also useful if we have attributes that won't be set in __init__, but in some later method call. This way we make them part of the class contract and initialize them to a default value (probably None), shared by all instances via the class attribute (as we saw with BasicStarship.captain), and then get it added to each instance when it gets set to a specific value.

Sunday, 15 February 2026

Logical Assignment Operator and More

I've recently come across the Logical OR Assignment (||=), and the Nullish Coalescing Assignment (??=) operators in JavaScript. They are not a revolution, just a shortcut for the usage of the OR (||) operator and the nullish coalescing operator in assignment situations. We use "||=" for falsy values and "??=" for nullish (null, undefined) values. Let's see:


// for "falsy" values
> let name = "";
> name ||= "default";
'default'
> name ||= "default2";
'default'

// is equivalent to:
> name = ""
> name = name || "default";
'default'
> name = name || "default2";
'default'

// for strict null or undefined values:
> let name = null; // or name = undefined
> name ??= "default";
'default'
> name ??= "default2";
'default'

// is equivalent to:
> name = null;
> name = name ?? "default";
'default'
> name = name ?? "default2";
'default'


Python does not have a 'None coalescing' operator (so obviously it does not have a 'None coalescing assignment' operator) so as equivalent we have to use an if-else expression. We have the 'or' operator (that we can use with falsy values), but not an "or assignment" operator. So the equivalent code to the above JavaScript is quite more verbose:


# for "falsy" values
> name = ""
> name = name or "default"
'default'
> name = name || "default2"
'default'

# for strict None values:
> name = null
> name = name if name is not None else "default"
'default'
> name = name if name is not None else "default2"
'default'

As the if-else pattern is quite verbose, we can write a simple coalesce function (I've just remembered that such function is almost standard SQL) to make code more straightforward.


def coalesce(value, default_value):
    return value if value is not None else default_value

a = coalesce(a, "default value")

As for other languages, Kotlin has the || operator and the :? null coalescing operator, but not a shortcut form to use during assignment. Ruby has a logical or assignment operator that we can use with nil and false (the only falsy values in Ruby). It feels strange that Ruby does not have a null coalescing operator, so if we want to be strict and deal only with null (nil), we have to use the so rich Ruby syntax differently:


# for null coalescing assignment
# like JavaScript: a = a ?? "default" 
# or Kotlin: a = a ?: "default"

a = "default" if a.nil?
# or
a = a.nil? ? "default" : a


Reached this point I think it'll be good to remember what are considered falsy values (those that, when evaluated in a boolean context, are considered as false) in different languages:

  • JavaScript: false, null, undefined, 0, ""
  • Python: False, None, 0, "", [], {}, set()
  • Rubynil, false
  • Kotlinfalse. Kotlin does NOT perform truthy/falsy coercion, it's fully, strictly typed:trying to use a non boolean value in a condition causes a compilation error.

As you can see the main (and very important) difference between JavaScript and Python is that in Python empty containers are falsy.

Saturday, 7 February 2026

Python Attribute Lookup and Dunders

I already talked in the past about Python descriptors [1] and [2] (referencing also the complex attribute lookup process). Somehow I've recently realised of how some commonly used attributes are managed with descriptors present in classes or metaclasses. First, I'll paste here the conclusions after an interesting chat with a GPT regarding the attibute lookup process:

1) Instance attribute lookup (obj.attr)

This is (conceptually) what object.__getattribute__(obj, name) does:

a) Check for a data descriptor on the class or its MRO
Search type(obj).__mro__ for name in each class’s __dict__.
If found and it’s a data descriptor (has __set__ or __delete__), return descriptor.__get__(obj, type(obj)).

b) Check the instance’s own dictionary
If obj.__dict__ exists and contains name, return obj.__dict__[name].
Note: If the class defines __slots__ without __dict__, this step may not exist.

c) Check for a non-data descriptor or other attribute on the class/MRO
Search type(obj).__mro__ for name.
If found and it’s a non-data descriptor (has __get__ only), return descriptor.__get__(obj, type(obj)).
Otherwise, return the found value as-is.

d) Fallback: __getattr__
If nothing above produced a value, and type(obj) defines __getattr__(self, name), call it and return its result.

e) Otherwise
Raise AttributeError.

2) class attribute lookup (C.attr)

Conceptually, type.__getattribute__(C, name) does this:

a) Metaclass MRO — data descriptors first
Search type(C).__mro__. If name is found and it’s a data descriptor (__set__ or __delete__ present), return descriptor.__get__(None, C).

b) Class MRO (C and its bases) — regular attributes & descriptors
Search C.__mro__ (starting with C, then bases):
If found and it’s a descriptor (__get__), return descriptor.__get__(None, C) (note obj=None).
Otherwise, return the raw value.

c) Metaclass MRO — non-data descriptors and other attributes
If found on the metaclass MRO and it’s a descriptor, return descriptor.__get__(C, type(C)) (here, the “instance” is the class C) Otherwise return the value.

e) Fallback
If not found and the metaclass defines __getattr__(cls, name), call it.
Else raise AttributeError.

Let's see now some examples of attributes that are indeed descriptors:

__name__ of a class (Person.__name__). One could think that it's just an attribute directly in the class object, but if it were that way, I could acces it via an instance of the class (person1.__name__) that is not the case. So indeed __name__ is a descriptor in the metaclass (and exactly the same for __bases__ or __doc__):


>>> class Person:
...     pass
...     
>>> Person().__name__
Traceback (most recent call last):
    Person().__name__
AttributeError: 'Person' object has no attribute '__name__'

>>> Person.__name__
'Person'

>>> Person.__dict__["__name__"]
Traceback (most recent call last):
    Person.__dict__["__name__"]
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^
KeyError: '__name__'

>>> type(Person).__dict__["__name__"]
attribute '__name__' of 'type' objects
>>> type(type(Person).__dict__["__name__"])
class 'getset_descriptor'

>>> type(Person).__dict__["__bases__"]
attribute '__bases__' of 'type' objects
>>> type(type(Person).__dict__["__bases__"])
class 'getset_descriptor'>

>>> type(type(Person).__dict__["__doc__"])
class 'getset_descriptor'


__class__ of an instance or __class__ of a class. This one does not seem be based on descriptors, but (my discussion with a GPT is a bit confusing) it seem like it's managed specially by the look up algorithm.


>>> p1 = Person()
>>> p1.__class__
class '__main__.Person'

>>> type.__class__
class 'type'

>>> type(p1.__dict__["__class__"])
Traceback (most recent call last):
    type(p1.__dict__["__class__"])
         ~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: '__class__'

>>> type(Person.__dict__["__class__"])
Traceback (most recent call last):
    type(Person.__dict__["__class__"])
         ~~~~~~~~~~~~~~~^^^^^^^^^^^^^
KeyError: '__class__'


Dunder attributes. It's interesting to note that there are 2 categories of __dunder__ attributes (those that start and end by "__").
- On one hand we have those like the ones we've just seen, these are Special Attributes (Metadata), that are used to store metadata: __name__, __class__, __bases__, __mro__, __dict__, __module__, __doce__, __annotations__.
- And on the other hand we have Special Methods (Behavioral Hooks), that are used to implement Python's syntactic sugar:

__call__: ob(), Invokation
__getitem__: ob[key]
__setitem__: ob[key] = value 
__getattr__: Fallback for missing attributes
__getattribute__: Intercepts all attribute access
__iter__, __next__: Iteration
__str__, __repr__: String representation
__eq__, __lt__, etc: Comparisons
__enter__, __exit__: Context managers
__add__, __mul__, etc: Arithmetic operations

Notice that if you access a Behavioral Hook "on your own" (I mean, you explicitly do: obj.__call__() or obj.__iter__()) the normal look up mechanism applies (using the object and its class). However, when used in the intended way (when you do obj(), or iter(obj)) the look up is done only in the class of the object (and if an object is a class it's done in its metaclass) not in the object itself.