Sunday, 26 January 2025

JavaScript vs Python: arguments vs locals()

It's not uncommon to have a function that acts as an adapter, wrapping another existing function, having its same parameters and performing some action before and/or after invoking the original function with those parameters. JavaScript provides a very convenient way to invoke the original function without having to write again that full list of arguments in the call, just using the arguments array-like object and the spread operator. I mean:


function doAction(a, b, c) {

}
function checkParams(a, b, c) {
	if (a === "undefined") {
		return null;
	}
	return doAction(...arguments);
}

In many cases the arguments object is not necessary, we can just use the rest operator in the adapter function rather than the list of arguments. That's great when the adapter function is not doing anything with the arguments, for example when creating generic wrappers like this:


function doAction(a, b, c) {

}

function createLoggingWrapper(fn) {
	return (...params) => {
		console.log("started");
		res = fn(...params);
		console.log("finished");
		return res;
	};
}

In Python the equivalent to the rest-spread operaters is the packing-unpacking operator (that given that Python features named parameters has extra support for that) . So we can write the second example just like this:


from functools import wraps

def doAction(a, b, c):
    print(f"{a}{b}{c}")


def createLoggingWrapper(fn):
    @wraps(fn)
    def logged(*args, **kwargs):
        print("started")
        res = fn(*args, **kwargs)
        print("finished")
        return res
    return logged

doAction = createLoggingWrapper(doAction)
doAction("AA", c="CC", b="BB")

But what about the first example? We don't have a direct equivalent to the arguments object, but we have a workaround that is almost equivalent, using locals(). I already talked about the locals() function in a previous post. It returns the list of variables in the "local namespace" at that moment (that is function arguments and local variables), so if we run it before any local variable is declared we can use it as the JavaScript arguments object. Let's see:


def format(name, age, city, country):
    return f"formatting: {name} - {age} [{city}, {country}]"

def format_with_validation(name, age, city, country):
    if city:
        print("validation OK")
        return format(**locals()) # equivalent to: format(name=name, age=age, city=city, country=country)
    else:
        print("validation KO")
        return None

print(format_with_validation("Francois", 40, "Paris", "France"))
print(format_with_validation("Francois", 40, "Lyon", "France"))

# validation OK
# formatting: Francois - 40 [Paris, France]
# validation OK
# formatting: Francois - 40 [Lyon, France]

There's one gotcha. locals() returns also those variables (free-vars) trapped by closures. So if our wrapping function happened to be working as a closure, this technique will not work, as we'll be passing extra parameters to the wrapped function (the "counter" variable in the example below).


def format(name, age, city, country):
    return f"formatting: {name} - {age} [{city}, {country}]"

def create_format_with_validation():
    counter = 0
    def format_with_validation(name, age, city, country):
        nonlocal counter       
        counter +=1
        print(f"invokation number: {counter}")
        if city:
            print("validation OK")
            return format(**locals()) # equivalent to: format(name=name, age=age, city=city, country=country)
        else:
            print("validation KO")
            return None
    return format_with_validation

format_with_validation = create_format_with_validation()
try:
    print(format_with_validation("Francois", 40, "Paris", "France"))
except Exception as ex:
    print(f"Error: {ex}")
    # TypeError: format() got an unexpected keyword argument 'counter'. Did you mean 'country'?


Well, we can easily fix that by removing from the object returned by locals the entries corresponding to trapped variables. Indeed, Python is so amazingly instrospective that we can automate that. The code object of one function has a co_freevars attribute that gives us the names of the freevars used by that function. So we can write a "remove_free_vars" function and do this:


def remove_free_vars(loc_ns: dict, fn) -> dict:
    return {
        key: value for key, value in loc_ns.items()
        if key not in fn.__code__.co_freevars
    }

def create_format_with_validation_closure_aware():
    counter = 0
    def format_with_validation(name, age, city, country):
        nonlocal counter       
        counter +=1
        print(f"invokation number: {counter}")
        if city:
            print("validation OK")
            return format(**(remove_free_vars(locals(), format_with_validation))) 
        else:
            print("validation KO")
            return None
    return format_with_validation

format_with_validation = create_format_with_validation_closure_aware()
print(format_with_validation("Francois", 40, "Paris", "France"))

There's one additional gotcha. As locals() returns a dictionary-like object, if the function to be invoked has been defined restricting that some of its arguments can not be provided as named arguments (that odd feature that I described here).

Monday, 20 January 2025

Python Generator with Calculated Values

I frequently take a look into the Python discussion forum, mainly to the "ideas" section, where people propose things they would like to see in the language or in libraries. Though sadly most of the time most requests for additions to the language are promptly rejected (sometimes with absurd reaons, as if given by people that do not use other modern languages...) the proposed alternatives and the ideas themselves can be pretty interesting. Sometime ago I came across an idea that I was not immediatelly grasping, Literal syntax for generatos.

We know generators generate values lazily, as requested. But if we had a literal syntax we would be already providing all values, so why not just use a list? Well, because the idea would be supporting that some values are generated by functions, that get invoked when the iteration requests it. Well, the guy showed the workaround that he's using, and it's pretty, pretty, interesting:


queries: Iterator[str] = (lambda: (
    (yield "abc"),
    (yield t if (t := expensive_func()) else None),
    (yield "something else"),
))()

Honestly it took me quite a while to understand what's going on there. So he's using an immediatelly invokable lambda with an expression containing multiple expressions (this is the trick that I explained some time ago), where each subexpression is yielding a value. In Python yield can be used both as a statement and as an expression. The clear case for the latter is when we write a = yield "whatever", for using with the send(value) method). In this example, for the yield to be considered as an expression it has to be wrapped with parentheses.

It looks pretty nice, and it would have never occurred to me. The alternative that I could have envisioned would be a generator factory like this:


def lazy_gen(values: Iterable) -> Generator:
    """
    returns a Generator object
    """
    def gen_fn_wrapper():
        for it in values:
            yield it() if callable(it) else it
    return gen_fn_wrapper()

Indeed, maybe this approach is easier to understand. Let's see an example:


def expensive_format(msg: str):
    sleep(2)
    return msg.upper()

print("started")
messages: Generator[str, None, None] = (lambda: (
    (yield "abc"),
    (yield expensive_format("my message")),
    (yield expensive_format("my message 2")),
    (yield "something else"),
))()

for message in messages:
    print(message)

print("- second approach")

messages = lazy_gen([
    "abc", 
    lambda: expensive_format("my message"),
    partial(expensive_format, "my message 2"),
    "something else",
])
for message in messages:
    print(message)


There's another approach that they mention in the discussion thread and that uses a trick that I find rather confusing. We can use a decorator to immediatelly invoke a function that we've just defined. This means that we can define a normal generator function that contains statements and invoke it to obtain a generator object that will get assigned to a variable with the name of the function that we've just decorated. As I've said, it feels terribly confusing, but I'll include it here for the sake of completion. I have to say that defining the decorator inline with a lambda feels pretty cool :-) Let's see:



@lambda x: x() # Or @operator.call
def messages():
    yield "abc"
    yield expensive_format("my message")
    yield expensive_format("my message 2")
    yield "something else"

for message in messages:
    print(message)          


yield expressions are, well, expressions, which means that in principle we can use them anywhere an expression can be used, for example as argument to a function. I mean


def f1():
    yield "a"
    print(str((yield "b")))
    yield "d"
  

gen = f1()
print(next(gen))
#'a'
print(next(gen))
#'b'
print(gen.send(11))
#11
#'d'


Wednesday, 15 January 2025

As Bestas

I had pending to watch As Bestas, a French-Spanish (Galician) film since quite a while ago. I wanted to watch it with my parents, particularly with my Galician father, but as it runs for more than 2 hours it was not easy to find the appropriate moment. This Christmas Eve has been the time. I didn't know that it was based on a true story that happened also in rural Galicia, to a Dutch couple that settled there, between 1996 and 2014. The film is particularly appealing to me cause though I am and feel so deeply Asturian, France is my second country, my second identity, and Galicia is the land of all my paternal ancestors, so I consider it my third country and identity. Dialogs in the film are in French, Galician and some Spanish I think also. It felt odd to me that I could understand better the parts in French than those in Galician, you can never disappoint your ancestors enough...

So in the film we have a French couple of "neo-rurals" that have settled somewhere in the rural interior of Galicia. These are not "digital nomads" teleworking from a small paradise, but 2 hard worker idealists that want to make a living working the land (in a traditional way it seems). From the beginning we see that they do not get on well with some of their neighbours, 2 of them in particular, 2 brothers. At first we can think that this hostility, this distrust, is because of some sort of xenophobia, which would be paradoxal in a land, Galicia, that in spite of its wild natural wealth has never managed to feed all its children, making Galicians (even more than Asturians) one of the most migrant people in the world (you can find Galicians anywhere, indeed the joke says that North Americans came across one Galician in the Moon when they landed there).

The permanent tension between the French couple and these 2 brothers, that bully them and keep a continuous threatening attitude to them creates a thick and oppressive atmosphere, accentuated by the Galician landscape and the backward feeling of the village. The "bar" where good part of the interaction between the French man and the 2 brothers takes place feels like taken out from another century. This reminds me of one discussion with one friend of mine (also an Asturian with Galician ancestry) when he told me about how poor and less developed some parts of rural Galicia feel when compared to their Asturian counterparts. It's strange, cause on the other side, Galician cities (particularly Vigo), that is the part of Galicia that I know, feel way more cosmopolitan and developed than Asturian cities.

As the film evolves, the reason for the resentment that the 2 brothers express for the French couple unveils. The couple were some of the main opponents that refused selling their lands to an energy company that wanted to set up an Eolian farm in the village, and needed for it the lands of all the villagers, so this refusal prevented other villagers from selling. We arrive then to the most intense moment of the film, the discussion where the main brother explains full of bitterness and hatred how his miserable life would have changed if he had sold his lands and left the village for the city. It's a point where you end up empathizing with a character that up to that point had been profoundly revolting.

Well, I think that's all, I can not tell more without fully spoiling the film. Just reserve 2 hours of your life (OK, a bit more if you have to search the torrent and download it) and watch it.

Sunday, 5 January 2025

Closing Javascript Generators

In this previous post about closing Python generators I mentioned that JavaScript generators had a similar feature that would deserve a post on its own, so here it is.

JavaScript generators have a return() method. We can think of it as partially equivalent to Python's close() method. This is so for the simple (and main I guess) use cases that I explained in my previous post (use it as a replacement for break, and when you are passing the generator around to other methods and one of them can decide to close it). For example:


function* citiesGen() {
    yield "Paris";
    yield "Porto";
    return "Europe";
}

// using .return() rather than break
let cities = citiesGen();
for (let city of cities) {
    if (city == "Porto") {
        cities.return();
        console.log("closing generator");
    }
    console.log(city);
}
/*
Paris
closing generator
Porto
*/


Then we have the more advanced cases, for which indeed finding a use case seems not so apparent to me. Here it's where the differences with Python's close() are important. JavaScript return() accepts a value, that will be returned as part of the value-done pair returned when the generator is finished. This "when it's finished" is key, as a try-finally in the generator code can prevent the return() call from finishing the generator in that call. It will continue to produce values as instructed from the finally part, and once completed will return the value that we had passed in the return() call. The theory:

The return() method, when called, can be seen as if a return value; statement is inserted in the generator's body at the current suspended position, where value is the value passed to the return() method. Therefore, in a typical flow, calling return(value) will return { done: true, value: value }. However, if the yield expression is wrapped in a try...finally block, the control flow doesn't exit the function body, but proceeds to the finally block instead. In this case, the value returned may be different, and done may even be false, if there are more yield expressions within the finally block.

And the practice:




function* citiesGen2() {
    yield "Paris";
    try {
        yield "Lyon";
        yield "Porto";
        return "Stockholm";
    }
    finally {
        yield "Lisbon";
        yield "Berlin";
    }
}

cities = citiesGen2();
console.log(cities.next());
console.log(cities.next());
console.log(cities.return("Over"));
console.log(cities.next());
console.log(cities.next());
console.log(cities.next());


// { value: 'Paris', done: false }
// { value: 'Lyon', done: false }
// { value: 'Lisbon', done: false }
// { value: 'Berlin', done: false }
// { value: 'Over', done: true }
// { value: undefined, done: true }


If this feels odd to you, you're not alone :-D. This is quite different from Python, where a call to close() always finishes the generator, even if we are catching the Exception and returning something from it.

generator.close()

Raises a GeneratorExit at the point where the generator function was paused. If the generator function catches the exception and returns a value, this value is returned from close(). If the generator function is already closed, or raises GeneratorExit (by not catching the exception), close() returns None. If the generator yields a value, a RuntimeError is raised. If the generator raises any other exception, it is propagated to the caller. If the generator has already exited due to an exception or normal exit, close() returns None and has no other effect.

Same as Python, JavaScript generators also have a throw() method, and again, I see no much use for it.