In this previous post I explained how Python allows the usage of any object, not just type objects, for its annotations "Annotations can be any valid Python expression". Annotations are used to provide metadata, and while normally that metadata is just typing information, we can provide any sort of metadata for custom use at runtime (and as we saw we have the Annotated mechanism to combine both typing and custom metadata. While doing some tests at that time I noticed that VS Code (the Pylance extension) would warn (with: "Call expression not allowed in type expression" or "Variable not allowed in type expression") against using annotations like this (that as I've said is perfectly valid):
# metadata for parameters
@dataclass
class ValueRange:
lo: int
hi: int
# pylance warning: Call expression not allowed in type expression
def create_post_1(
title: ValueRange(5, 20),
content: ValueRange(5, 100),
) -> dict:
return {"title": title, "content": content}
val_range = ValueRange(1, 10)
# pylance warning: Variable not allowed in type expression
def fn2(a: val_range) -> None:
pass
# but it's stored OK in the function annotations
print(annotationlib.get_annotations(fn2))
# {'a': ValueRange(lo=1, hi=10), 'return': None}
So if this works fine at runtime (you can see that the annotation is stored along with the function) why pylance warns against it? Because we have to differentiate what is valid for runtime use vs what is valid at type checking time. From a GPT:
The runtime accepts arbitrary expressions. Static type checkers do not. This is so because static typing tools parse Python, but they don’t execute it, and they don’t compile it to bytecode either. Type checkers rely on syntactic patterns, not runtime behavior
This is something I had never thought about before. Python type checkers analyze your code without executing anything (they are static). So the types in the type annotations that they are going to analyze have to be expressed in a direct, static form, not as the result of executing an expression (as they are not going to execute that expression). The different Python type checkers (mypy, Pylance, pyre...) parse python source code into an AST (normally different from CPython AST) and analyze it, they do not run code, indeed they do not even compile the code to bytecodes to create code objects. That's why they can only work with type expressions (annotation expressions), that follow a specific syntax, not with any expression. From here: Note that while annotation expressions are the only expressions valid as type annotations in the type system, the Python language itself makes no such restriction: any expression is allowed.
Type checkers operate on expressions that syntactically denote types, which basically is a type name or a generic type. And this has sparked my curiosity about how generic types (MyClass[T]) work. For the type checker it's simple, as it does not execute anything, it just has to parse that particular syntax. But what's the runtime meaning of such generic expression?
Well, it's subscripted access to an object (to a class). When we do my_instance["x"] this searches for a __getitem__ method in my_instance's type. So MyClass["x"] should just search __getitem__ in MyClass's type (that is, its metaclass). That's correct, but given that Python designers have always considered metaclasses like particularly complex and/or exotic, they decided to introduce (via PEP560) a hook to make easier to implement subscripted access to classes. Rather than having to define a metaclass for MyClass, we can directly define a __class_getitem__ method in MyClass.