Sunday, 30 November 2025

exec, eval, return and ruby

In this post about expressions I mentioned that Kotlin features return expressions, which is a rather surprising feature. Let's see it in action:


// Kotlin code:
fun getCapital(country: Country) {
     val city = country?.capital ?: return "Paris"
     // this won't run if we could not find a capital
     logger.log("We've found the capital")
     return city
}

Contrary to try or throw expressions, that can be simulated (in JavaScript, Python...) with a function [1], [2], there's no way to use a "return() function" to mimic them (it would exit from that function itself, not from the calling one). Well, it came to my mind that maybe we could use a trick in JavaScript with eval() (I already knew that it would not work in Python with exec()), but no, it does not work in JavaScript either.


// JavaScript code:
function demo() {
    eval("return 42;");
    console.log("This will never run");
}

console.log(demo());
// Output: SyntaxError: Illegal return statement


JavaScript gives us a SyntaxError when we try that because that return can not work in the way we intend (returning from the enclosing function) so it prevents us from trying it. The code that eval compiles and runs is running inside the eval function, it's not as if it were magically placed inline in the enclosing function, so return (or break, or continue) would just return from eval itself, not from the enclosing function, and to prevent confusion, JavaScript forbids it.

The reason why I thought that maybe this would be possible is because as I had already explained in this previous post JavaScript eval() is more powerful than Python exec(), as it allows us modifying and even adding variables to the enclosing function. As a reminder:


// JavaScript code:
function declareNewVariable() {
    // has to be declared as "var" rather than let to make it accessible outside the block
    let block = "var a = 'Bonjour';";
    eval(block);
    console.log(`a: ${a}`)
}

declareNewVariable();
// a: Bonjour


This works because when JavaScript compiles and executes a "block" of code with eval() it gives it access to the scope chain of the enclosing function.

Python could have also implemented this feature, but it would be very problematic in performance terms. Each Python function stores its variables in an array (I think it's the f_localsplus attribute of the internal frame/interpreter object, not to be confused with the higher level PyFrameObject wrapper), and the bytecode access to variables by index in that array (using LOAD_FAST, STORE_FAST instructions), not by name . exec() accepts an arbitrary dictionary to be used as locals, meaning that it will access to that custom locals or to the one created from the real locals, as a dictionary lookup (with LOAD_NAME, STORE_NAME). Basically there's not an easy way to reconcile both approaches. Well, indeed exec() could have been designed as receiving by default a write-through proxy like the one created by frame.f_locals. That would allow modifying variables from the enclosing function, but would not work for adding variables to it (see this post). So I guess Python designers have seen it as more coherent to prevent both cases rather than having one case work (modification of variable) and another case not (addition of a new variable). As for the PyFrameObject stuff that I mention, some GPT information:

In Python 3.11+, the local variables and execution state are stored in interpreter frames (also called "internal frames"), which are lower-level C structures that are much more lightweight than the old PyFrameObject.
When you call sys._getframe() or use debugging tools, CPython creates a PyFrameObject on-demand that acts as a Python-accessible wrapper around the internal frame data. This wrapper is what you can inspect from Python code, but it's only created when needed.

So all in all we can say (well, a GPT says...)

Bottom line: Neither Python’s exec() nor JavaScript’s eval() can magically splice control-flow into the caller’s code. They both create separate compilation units. JavaScript feels “closer” because eval() shares lexical scope, but the AST boundaries still apply.

After all this, one interesting question comes up, is there any language where the equivalent to eval/exec allows us returning from the enclosing function? The answer is Yes, Ruby (and obviously it also allows modifying and adding new variables to the enclosing function). Additionally notice that ruby also supports return expressions (well, everything in ruby is an expression).


# ruby code:
def example
  result = eval("return 5")
  puts "This won't execute"
end

example  # returns 5

Ruby's eval is much more powerful than JavaScript's or Python's - it truly executes code as if it were written inline in the enclosing context.

The "as if" is important. It's not that Ruby compiles the code passed to eval and somehow embeds it in the middle of the currently running function. That could be possible I guess in a Tree parsing interpreter, modifying the AST of the current function, but Ruby has long ago moved to bytecode and JIT. What really happens is this

Ruby's eval compiles the string to bytecode and then executes it in the context of the provided binding, which includes:

- Local variables
- self (the current object)
- The control flow context (the call stack frame)

That last part is key. When you pass a Binding object, you're not just passing variables - you're passing a reference to the actual execution frame. So when the evaled code does return, break, or next (Ruby's continue), it operates on that captured frame. Here's where it gets wild.

The Binding object idea (an object that represents the execution context of a function) is amazing. By default (when you don't explicitly provide a binding object) the binding object represents the current execution frame, but you can even pass as binding the execution frame of another function!!! You can get access to variables from another function, and if that function is still active (it's up in the call stack) you can even return from that function, meaning you can make control flow jump from one function to another one up in the stack chain!

eval operates on a Binding object (which you can pass explicitly), and that binding captures the complete execution context - local variables, self, the surrounding scope, everything. You can even capture and pass bindings around

Just notice that Python allows a small subset of the binding object functionality by allowing us to explicitly provide custom dictionaries as locals and globals to exec().

Sunday, 23 November 2025

How Exceptions Work

It's been quite a while since I first complained about the lack of safe-navigation and coalesce operators in Python, and provided a basic alternative. I've also complained about the lack of try-expressions in Python, and also provided a basic alternative. Indeed, there's not a strong reason for having 2 separate functions, I think we can just use do_try for the safe-get and coalesce option.



def do_try(action: Callable, exceptions: BaseException | list[BaseException] | None = Exception, on_except: Any | None = None) -> Any:
    """
    simulate 'try expressions'
    on_except can be a value or a Callable (that receives the Exception)
    """
    try:
        return action()
    except exceptions as ex:
        return on_except(ex) if (on_except and callable(on_except)) else on_except

person = Person()
embassy = do_try(lambda: person.country.main_cities[0])


I also complained of how absurd it feels having a get method for ditionaries but not for collections. That means that we end up writing code like this:


x = items[i] if len(items) > i else "default"

Of course, that would not be necessary if we had safe-navigation, but as we don't have it, we can just use the do_try function:


x = do_try(lambda: items[i], "default")

And here comes the interesting part, obviously using do_try means using try-except under the covers, which when compared to using an if conditional seems something to avoid in performance terms, right? Well, I've been revisiting a bit the internals and cost of exceptions. Since version 3.11 Python has zero-cost exceptions. This means that (as in Java) having try-except blocks in your code does not have any performance effect if no exception is thrown/raise, the only costs occur if an exception is actually raised: The "zero cost" refers to the cost when no exception is raised. There is still a cost when exceptions are thrown.

Modern Python uses exception tables. For each function containing try-except blocks an exception table is created linking the try part to the handling code in the except part. Exception tables are created at compile time and stored in the code object. Then at runtime if an exception occurs the interpreter will consult the exception table to find the handler for the given exception and jump to it. Obviously creating an exception object, searching the exception table and jumping to the handler has a cost. Given that in Python compilation occurs when we launch the script, just before we can run the code, we can say that this exception table creation also has a runtime cost, but it's mininum as it happens only once per function (when the function is created), not every time the function is executed. That's where the cost happens if an exception is raised: creating the Exception object, unwinding and jumping to the handler.

Throwing/raising an exception felt like a low level mechanism to me, but it's not at all.

Language-level exceptions are software constructs managed by the runtime (JVM for Java, CPython for Python). They do not involve the OS unless the program crashes. So when you use a throw/raise statement in your code there's not any sort if software interrupt, it's just one more (or several) instruction. The python interpreter will come across a RAISE_VARARGS bytecode instruction, and it will search in the exception table for the current function and/or the functions in the call stack, trying to find an exception handler.

Notice that the same happens in Java-JVM. The Java Compiler creates an exception table for each method and stores it in the .class file. This table maps bytecode ranges to handlers (catch blocks) and the type of exception they handle. When the class loader loads the class the JVM stores this table in the method’s metadata.. Given that the JVM comes with JIT compilation, there's an additional level. When the JIT compiles the method:

The JIT generates native machine code for the method.
It also creates a new exception table for the compiled code, because:
The original bytecode offsets are no longer relevant.
The JIT needs to map native instruction addresses to handler entry points.

This table is stored alongside the compiled code in the JVM’s internal structures.

So once a method has been compiled by the JIT at runtime we'll have two exception tables, the initial one for the bytecode form of the method (that is kept around in case we have to deoptimize from native back to bytecodes), and the table for native code. Notice that when the JIT compiles the bytecodes to native code we'll incur a very small extra cost for the creation of this additional table.

With all the above, using do_try() for safe indexed access seems a bit overkill (unless we're sure that access is very rarely going to fail and throw), and having a specific commodity function for it makes sense:


def get_by_index(sequence: Sequence[T], index: int, default: Any = None) -> Any:
    """
    Safely access an element by index in a sequence, where sequence is any class supporting __getitem__ and __len__.
    like: list, str, tuple, and bytes
    Usage: get_index(my_list, 2, default='Not Found')
    """
    return sequence[index] if 0 <= index < len(sequence) else default
    

We could generalize the function for nested access, but once we start to loop with if conditions at some number of iterations the try-except will probably end up being better for performance.

Friday, 14 November 2025

FICXixon 2024

I guess as you get older (OK, let's say mature, to sound nicer) traditions become more and more important. One more year, and one more edition of FICXixón is almost here, and as usual, I realise I have not published yet my post about the previous edition, so here it goes:

FICXixon 62 edition took place from November 15th to 23rd. This was a dry November month here, which makes me rather angry, I love rainy weather, I've always loved it, but now even more probably due my youth memories (and I guess my Galician heritage also plays its part). All in all I attended 6 films, which is my record since I returned to Xixón in 2018. This year I was not so busy at work, and had more time to check the programme and attend screenings. I watched 2 excellent films, a good film, an interesting documentary, and 2 films that were OK, but I would not watch again. From the micro-reviews below you can guess who is who.

  • The Antique, Friday 15, 19:00, OCine. The best film I've watched in this edition. I was quite in doubt between this one and Bird, but the trailer of "The Antique" looked so good (additionally, a story set in Russia is more appealing to me now that one set in UK), and in the end I settled on this gorgeous Georgian film. An old flat in a historical building in central Saint Petersbourg, the snow covered streets, an old man approaching the end, a gorgeous Georgian woman, antiquities, social unrest. I think I've said enough
  • Una Ballena, Saturday 16, 22:00, Teatro Jovellanos. Basque film mixing neo-noir and horror-fantasy. It had excellent reviews, but I'm not sure why it did not work for me. There was an "Encuentro con el público" after the screening, where the director and the main actress, the beautiful Ingrid, would discuss the film with the audience, some people expressed how (positively) shocked they were with the film, and honestly I felt a bit out of place.
  • La prisionnere de Bordeaux, Tuesday 19, 21:30, Yelmo Ocimax. When FICXixón 2022 dedicated a retrospective to Patricia Mazuay I watched three of her films, and I loved 2 of them. And I loved even more the "Encuentros con el público" with her. She was so funny and expressive. So when I found out that they were programming her last film I could not miss it, furthermore when it stars the charming Isabelle Huppert, one of my favorite actresses. So I even took the bus to go to the Yelmo cinema located far away in the outskirts, something I'd never done before for a FIC film! I did not regret it, the film is the kind of bizarre, funny, melancholic product that you could expect from these 2 crazy women.
  • When the Light Breaks, Wednesday 20, 22:00, Teatro Jovellanos. Excellent Icelandic drama. Grieving for a loved one is even worse when you are (and he was) young, and even worse when you have to hide how broken you are cause you had a secret relation. The light sequences at the start and the end of the film are mesmerizing.
  • Que se sepa (Indarkeriaren pi(h)artzunak), Thursday 21, 22:00, Escuela de Comercio. Interesting and necessary Basque documentary about one more of the many episodes of sorrow and pain brought up by the long and bloody conflict in the Basque Country. This time ETA and the Spanish Government join forces to kill an innocent man and destroy his family.
  • Fréwaka, Saturday 22, 19:15, OCine. It was preceded by a Colombian short, "La noche del Minotauro". Irish horror film, it was entertaining I think, but had little effect on me, so little that 1 year after I hardly remember anything about it.

Sunday, 9 November 2025

CGNAT

I've got a rather basic network knowledge and I've lately come across a problem/limitation I was not aware of and that I think is increasingly common, CGNAT. With my Internet Provider (Telecable) in Asturies, my FTTH router (a nice ZTE F6640) has a stable IP. I mean, it's not static, but it rarely changes (even after rebooting it). So when I recently felt that it could be convenient for me to occasionally connect to one of the computers in my LAN from outside, I thought it would be feasible.

So let's say I want to be able to ssh into my RasPI 5 from downtown while I discuss with my friends about how woke ideology is destroying humanity. The DHCP server in my router is configured to provide a static IP to all significant devices in my LAN, let's say 192.168.1.5 for my rasPi5. To make the port 22 in my rasPi accessible from outside I have to configure port forwarding in my router. It's just a matter of telling the router "forward incoming connections to one of your ports (let's say 10022) to port 22 in 192.168.1.5". I'd never done it before, but seems like something that has existed for decades and should work. So I connected my laptop to my mobile phone hotspot, to simulate the "I'm on the outside world thing", and tried. And tried, and tried... to not avail.

Checking some forums with similar questions involving other Internet providers in Spain I came across this fucking technology: CGNAT

Carrier-grade NAT (CGN or CGNAT), also known as large-scale NAT (LSN), is a type of network address translation (NAT) used by ISPs in IPv4 network design. With CGNAT, end sites, in particular residential networks, are configured with private network addresses that are translated to public IPv4 addresses by middlebox network address translator devices embedded in the network operator's network, permitting the sharing of small pools of public addresses among many end users. This essentially repeats the traditional customer-premises NAT function at the ISP level.

My internet provider in Asturies continues to use IPv4 (that's not the case in France, where to my surprise I found recently that it's using IPv6), and given that it has not enough public IP addresses for all its customers, it's adding an extra NAT (Network Address Translation) Layer.

I had got my router public address using curl ident.me, that gave me a nice and public 85.152.xxx.yyy address, but if I connect to my fiber router and check in it, I see a different one: 100.102.x.y. Well, that's not a public IP, and an indicator that my ISP is using CGNAT, as explained here.

If it's any of the following, then your router doesn't have a public IP address:

  • 192.168.x.x
  • 10.x.x.x
  • 172.16.x.x through 172.31.x.x
  • 100.64.x.x through 100.127.x.x

The last one is usually indicative of your ISP using CGNAT.

Summing up, my laptop has a 192.168. private IP address. My fiber Router faces the outside world with another private IP address (100.102.). Me and other customers in my area are connected to another upstream router in my ISP network, and this one faces the outside world with the 85.152.xxx.yyy public IP that I can see with ident.me. So in order for the connection from the outside to my RasPi to work I would also have to set up port-forwarding in that upstream ISP router shared with my "neighbours". So, no way...

Well, there's another way (that I have not tried) to set up this, a sort of reverse approach. In the last year I've been using SSH tunnels to connect to some non public servers at work through a "Bastion" work server with a public IP. With a standard SSH tunnel I basically create a SSH connection to that Bastion server telling it (to the Bastion server) that any connection that goes through it (through that "tunnel") has to be forwarded to another server. There are also reverse SSH Tunnels, where I create a SSH connection to a server (a tunnel) telling that server that any connections it receives to a certain port have to be forwarded to "me" through that tunnel, to a certain port on my machine. So if you have a server on the internet (Azure, AWS...) you could use it to create a reverse SSH tunnel to your PC located behind CGNAT. All this is explained for example here.