Access to Non Existing Index or Key

JavaScript, Kotlin and Python have some important differences when accessing indexes or keys that do not exist in a List/Array or Dictionary/Map. Notice that in Python we use the term attribute for fields in an object, and property for the classic getter-setter thing (that is implemented via descriptors). In JavaScript we use the term property for fields, and those properties can be a getter-setter descriptor or just a data descriptor. In Kotlin we use the term property, that always involves a getter and maybe a setter (that can be just the default ones that have no extra logic but accessing the backing field).

JavaScript is very permissive (I'm not surprised...). Accessing a non existing index of an array, a non existing key of a dictionary (a Map) or a non existing property of an object will not throw an exception, but return undefined.

 
> let ar = [];
> ar[2];
undefined
> 
> let dic = new Map();
> dic.get("name");
undefined
> 
> let ob = {};
> ob.name;
undefined

Python is not that permissive and any of the 3 accesses (missing index, missing key or missing attribute) will cause a different exception

ar = []
ar[2]
IndexError: list index out of range 

d = {}
d["name"]
KeyError: 'name'

ob = object()
ob.name
AttributeError: 'object' object has no attribute 'name'

For safe access to a dictionary key we can use the .get(key, default) method, and for safe access to an object attribute we can use getattr(ob, attr_name, default). Oddly enough, lists do not have a .get(index, default) method. Checking this question the reasonings for not having such method seem totally absurd to me. It's one of those occasions where the "being pythonic" and "zen of python" crap really infurates me. Python is an amazing language, but it could be even better if those that make the excellent work of developing and maintaining it were a bit more open to change.

In Kotlin, as it's a static language, accessing a non existent property of an object makes no sense, as the compiler prevents that. As for accessing a not existing index or key, the behaviour is a bit odd. A non existing index in an Array or List will throw an exception, but a non existing key in a map will just return null.

    val ls = listOf("a", "b")
    try {
        println(ls[4])
    }
    catch (e: Throwable) {
        println("${e::class.simpleName} - ${e.message}")
        // ArrayIndexOutOfBoundsException
    }

    val dic = mapOf(
        "France" to "Paris",
        "Austria" to "Vienna",
    )
    try {
        println(dic["Italy"]) // No exception, returns Null
    }
    catch (e: Throwable) {
        println("${e::class.simpleName} - ${e.message}")
    }

We can use getOrNull with an Array or List to get the same behaviour that we have with Maps. We also have getOrElse to return a default value rather than null. With Maps we can use both getOrElse or getOrDefault. Notice that the list and map retrieval behaviours are explained here and here

List Comprehensions vs Chaining Style

For sure list-comprehensions are a beautiful Python feature, but I've realised that I've been overusing them lately, applying a list comprehension to another list-comprehension, which makes the code not particularly clear. This has led me to compare it with the common approach followed in other languages: chaining methods on collections (JavaScript array extras or lodash, linq, pydash in python itself, Kotlin collections methods...). Let's compare some examples:

Using nested list comprehensions like this in Python seems pretty elegant to me:

cities = [ city.upper()
	for country in countries if country.name.startswith("c")
	for city in country.cities if city.name.startswith("c")
]

We can write an equivalent in "chaining style" in JavaScript like this:

let cities = countries.filter(country => country.startsWith("c"))
	.flatmap(country => country.cities)
	.filter(city => city.startsWith("c")
	.map(city => city.toUpper());

In this case I think I prefer the list-comprehensions option.

Nested list-comprehensions are also very nice if I want to combine values from the main and the nested collection. For example here, where I want a listing of country-city:

country_and_cities = [ [country.name, city.name]
	for country in countries 
	for city in country.cities 
]

That I would write in "chaining style" in JavaScript like this

let countryAndCities = countries.flatMap(country => country.cities.map(city => [country.name, city.name]));

Let's go now with those cases where I think list-comprehensions make code not so pleasant to read.

For example for parsing some lines read from a CSV into a list of objects we would write:

@dataclass
class Person:
	name: str
	age: int
	
countries = [Person(items[0], int(items[2])
	for items in [
		line.split(",") 
		for line in lines if not line.startswith("--")
	]
]

We would write it in "chaining style" in javascript like this:

lines.filter(line => !line.startsWith("--")
.map(line => {
	let [name, age] = line.split(",");
	return new Person(name, int(population));
});

And for dumping a list of objects to a CSV string we have this Python code with list-comprehensions:

users = [Person("Francois", 55), Person("Iyan", 45)]

"\n".join([user_line for user_line in 
    [";".join(user.name, str(user.age))
    for user in users]
])

That I think looks better in "chaining style" with javascript like this (or using pydash in python for example)

users.map(user => [user.name, user.age].join(";"))
	.join("\n");

Python Walrus Operator

The walrus operator aka Assignment Expressions was added to Python a few years ago, and it seems pretty interesting to me as I had never seen it before in other languages. As the name says, it allows us to have expressions that assign to a value. The main sample in the PEP-572 documentation gives us some useful use case:

# Handle a matched regex
if (match := pattern.search(data)) is not None:
    # Do something with match

# A loop that can't be trivially rewritten using 2-arg iter()
while chunk := file.read(8192):
   process(chunk)

# Reuse a value that's expensive to compute
[y := f(x), y**2, y**3]

# Share a subexpression between a comprehension filter clause and its output
filtered_data = [y for x in data if (y := f(x)) is not None]

I've started to use it in conditionals like this:

if (us := get_user()) and us.startswith("a"):
    print(f"yes, {us} starts with a")
else:
    print("no")

Given that in Kotlin almost everything is an expression I was expecting it to treat assignments as expressions also, but that's not the case. When searching about that I found out to my surprise that Java provides this feature!.

Though I'm still going through the process of interiorizing when/how to use Kotlin's scope functions and similar ones like takeIf, I've managed to come up to rewrite the conditional above like this:

    getUser()?.takeIf {
	    it.startsWith("a")
    }?.let { println("yes, ${it} starts with a") }

We can use walrus in Python when passing parameters to a function (even with named parameters):

def say(txt):
    print(txt)   

say(msg := "hi")
# hi
print(msg)
# 'hi'

say(txt=(msg := "hey"))
# hey
print(msg)
# hey

We can use it also as an indexer

items = [1,2,3,4,5,6]
pos = 0
items[pos:(pos:=pos+2)]
Out[46]: [1, 2]
pos
Out[47]: 2

Which seems pretty useful to write code like this:

def chunks_generator_fn(chunk_size, items):
    cur_pos = 0
    while cur_pos < len(items):
        next_chunk = items[cur_pos:(cur_pos := cour_pos + chunk_size]
        yield next_chunk

And one more use case:

if (index := index + 1) > end:
        index = 0

# looks better to me than:
index += 1
if index > end:
        index = 0        

forEach Method vs for of loop

While I'm using JavaScript here, this post applies to any language providing a forEach() iteration method as an alternative to traditional loops. I think it was in 2005 with that beautiful library called prototype.js that I first came across the possibility of iterating an array using a method (in modern JavaScript it would be the Array.prototype.forEach() method) rather than a "normal" loop construct. At that time it seemed so cool and using it was like a way of bragging. Over the years I've found myself using it more and more rarely. Of course I love all the iterate-transform methods: map, filter, etc in Array extras or lodash, but when all I need is iterating an array to apply some action I tend to use the "for of" loop.

I've been thinking lately if there's any advantage in using forEach() rather than for of. Well, indeed forEach() has some limitations. First, if your action is an async function forEach() won't work (though we saw that it's pretty simple to implement an async counterpart). Then you have to simulate "continue" using "return" (but in those cases I think it's more natural to use a normal loop), and we can't simulate "break". Of course we could write a new forEach() that treats an specific exception (for example BreakError) as a break.

When those limitations do not apply (we don't need await, break or continue), one could wonder if forEach() makes our code more declarative, more functional. I general terms, I don't think so, and I'm not alone [1] and [2]

However, there are specific cases when forEach() can make our code look better. Let's see:

- Combined with the safe navigation operator:

users?.forEach(user -> /* doWhatever */);

//probably looks better than:

if (users){
	for (const user of users) {
	 /* do whatever */
	}
}

- When you have already chained calls to other Array Extras methods

getUsers().filter(user -> user.country == "France")
	.forEach(user -> /* doWhatever */);

// maybe looks better than:

for (const user of getUsers.filter(user -> user.country == "France")){
	/* do whatever */
}

Infiesto

Infiesto is a dark Spanish thriller film that has become rather popular in Netflix. I've recently watched it and I've pretty enjoyed it. The plot is good, not particularly memorable, but entertaining, with good ideas (nothing new, but a good mix) that should have been further developed with an extended duration, or better, in a series.

The film is named after an Asturian village, L'Infiestu/Infiesto and is filmed in Asturies (mainly) and Galiza (the church in ruins with its Cruceiro, and the Hospital San Rafael in A Coruña). So yes, it's obvious that I was pretty interested on this film since I knew about it. The opening scene is an aerial view of Mieres, followed by a girl crossing the bridge that joins the railway station to the town.

We see some magnificent industrial ruins, massive, alive and well, industrial facilities (Arcelor Tabaza), run down farms, and of course, an abandoned coal mine, a testament to a nearby time of wealth and progress (yes, many Asturians still see coal as the source of life that it represented for our land, and not as the evil global warming monster...). There's a gloomy, oppressive, decadent atmosphere, rain, mud and humidity everywhere (that reminds me again that this Autumn and Winter are not behaving as they should). Asturies is a perfect setting to create this dark aesthetic, it's not chauvinism, it's reallity.

To my surprise (before watching the film I thought it had been entirely filmed in the "Cuenca Minera", the former coal mining areas), my hometown, Xixón, is well present in the film. The Police headquarters are set 10 minutes walking from my flat! There are road views of Avenida Constitución and the film ends with an aerial view of the southern neighbourhoods (Pumarín, Nuevu Xixón).

Unfortunately the film is too short, it ends too fast, and as I aforementioned I think the idea and scenario would have been great for a much longer development as a series. We could hope that this film could inspire other directors to leverage the Asturian landscapes and ruins for future works. The internationally recognized Asturian director Sergio G Sánchez made an excellent use of the Asturian landscape in his last work The Girl in the Mirror. It's like a teenage dark fantasy thriller, that is not exactly my favorite kind of story (I think it's mainly the teenage thing what does not please me), but the presence of the Asturian landscape is sublime (the opening scene with the bus going down that winding mountains road, the cave, the coast, the coastal village (L.luarca)... Something that I have to admit that I found a bit bothering in "The girl in the mirror" (and that has become so common) is the overrepresentation of homosexuality. We've moved from a time when films kept the LG community invisible, trying to sweep under the carpet something that felt uncomfortable (or plainly wrong) to the society of that time, to the current time where every film or series has to have at least one gay and one lesbian couple (or at least bisexual). I think this is way more than the proportion of homosexuals in the real world, so it's a way of deforming reality. I'm really fine with faking reality to promote positive, healthy, superior values (so for example I would pretty agree with showing a bigger proportion of vegans/vegetarians in films than the real one), but as I don't think homosexuality is better (or worse) than heterosexuality, I don't see any reason to promote it (or to discourage it).

JavaScript vs Python Closures

I rencently published a post comparing the dynamic creation of a function in JavaScript and Python, through the use of the magic functions eval and execute, that dynamically compile a string of code into bytecodes to be run by the VM.

For most cases javascript eval and Python execute are equally powerful, but eval has some freak and rarely used superpowers that Python lacks. Let's see:

Difference 1. In JavaScript ff the block passed to eval defines a new variable, this variable is accessible from the code in the function that invoked eval.

function declareNewVariable() {
    // has to be declared as "var" rather than let to make it accessible outside the block
    let block = "var a = 'Bonjour';";
    eval(block);
    console.log(`a: ${a}`)
}

declareNewVariable();
// a: Bonjour

As I've said, trying to do something similar in Python won't work:

def declare_new_variable():
    block = "a = 'Bonjour'";
    exec(block);
    print(f"a: {a}")

declare_new_variable()

# Exception:
# NameError: name 'a' is not defined

I've found a discussion about this limitation where they mention this:

You can't use exec() to set locals in a function, unless the name was already assigned to in a given function. This is a hard limitation due optimisations to how a local namespace in a function is accessed

Difference 2. In JavaScript ff the block passed to eval defines a new function, this function can become a closure, as it traps other variables defined both in the block itself and in the function invoking eval. This makes sense with how the scope chain works in javascript

function closureInEval() {
    let counter = 0;
    // both options work fine:
    // option 1, using a function declaration
    // let fnSt = "function printAndCount(){ console.log(counter); counter++;}";
    // eval(fnSt);
    //option 2, using an expression
    let printAndCount = eval("() => { console.log(counter); counter++;}");
    printAndCount();
    printAndCount();
    printAndCount();
}

closureInEval();

// 0
// 1
// 2

As I've said, trying to do something similar in Python won't work:

def test_closure_with_modify_access_1():
    fn_st = "\n".join([
    "def print_and_count(msg):", 
    "    nonlocal counter",
    "    counter += 1",
    "    print(str(counter) + ' - ' + msg)"
    ])

    # this does not work. The function that I define inside exec can not bind to the counter external variable as a nonlocal for the closure
    def create_eval_closure():
        d = {}
        counter = 0
        exec(fn_st + "\n" + "d['fn'] = " + "print_and_count")
        return d["fn"]

    print_and_count = create_eval_closure()
    print_and_count("a")
    print_and_count("b")
    print_and_count("c")

test_closure_with_modify_access_1()
# no binding for nonlocal 'counter' found (, line 2)

We get a no binding for nonlocal 'counter' found error cause the counter variable has not been trapped by the new function that we have just compiled. I've found some discussions about this issue, like this one, where they mention this:

When you pass a string to exec or eval, it compiles that string to a code object before considering globals or locals... There's no way for compile to know that a is a freevar

The above example will also fail if we declare the counter variable as part of the string passed to execute(), I mean:

    fn_st = "\n".join([
    "counter = 0",        
    "def print_and_count(msg):", 
    "    nonlocal counter",
    "    counter += 1",
    "    print(str(counter) + ' - ' + msg)"
    ])

What is perfectly fine in Python is passing to execute() a string with a wrapper function that defines a new variable and creates another function that is accessing to that outer variable. In this case this inner function becomes a closure trapping that variable defined in the same execute block.

def create_function(fn_st, fn_name):
    d = {}
    exec(fn_st + "\n" + "d['fn'] = " + fn_name)
    return d["fn"]

def test_closure_with_modify_access_3():
    # I can create a closure, but wrapping the closure creation in another function
    fn_st = "\n".join([
    "def create_closure():",
    "       counter = 0",
    "       def print_and_count(msg):", 
    "           nonlocal counter",
    "           counter += 1",
    "           print(str(counter) + ' - ' + msg)",
    "       return print_and_count"
    ])

    closure_creator = create_function(fn_st, "create_closure")
    print_and_count = closure_creator()
    print_and_count("a")
    print_and_count("b")
    print_and_count("c")

# 1 - a
# 2 - b
# 3 - c

Run Remote Application

Years ago I wrote about running applications remotely. In that case I was running the remote application without interacting with it, just obtaining its output. These days I've needed something a bit differente. We want some users (with 0 linux knowledge) to run a Linux command line interactive application (select option, press Y/N, etc), from windows. The ideal solution is that they just click a shortcut and a terminal (PuTTY) opens, connects to the server and starts the application. This way the user can work on the application without knowing anything about ssh connections, navigating the filesystem, etc...

Putty can be started from the command line with the server to connect to, user and password. Furthermore, we can provide it with a file with a list of commands (-m option) to run one the connection is established. If we want the remote session and the terminal to remain open once the commands are finished, we have a flag "-t" for that. All in all we have this:

putty.exe -ssh user@server -pw myPassword -t -m commands.txt

Having the commands in a separate file can be a bit problematic for some use cases, so searching how to overcome this limitation I came across KiTTY, a PuTTY fork. It supports all the PuTTY arguments and some additional ones, like "-cmd" that allows us to pass a command to run (so obviously we can run several commands joining them with "&" or ";"). So with KiTTY we have:

kitty.exe -ssh user@server -pw myPassword -t -cmd "python3.10 /apps/my_app/main.py"

Notice that while for this specific use case of opening a terminal and connection from the command line... I'm using PuTTY and KiTTY, for my normal work with Linux Servers I use since a long, long while MobaXterm. One gorgeous piece of software, that incorporates a sftp browser, splitting your screen for multiple connecions and typing the same commands in all of them, and many, many more features. Furthermore, it's developed in Toulouse!