Friday, 15 November 2024

Python Functions Disallowing Named Arguments

There are some builtin Python functions (written in C) that do not accept named (keyword) arguments (so they force the use of positional ones), for example (range, map, filter...) If you try to pass them a keyword argument you'll get an error: [function] takes no keyword arguments. This feels like an odd limitation to me, and furthermore, very confusing, as when looking into the documentation (for example for functions filter and map) I see no indication of this restriction, so you "learn by trying".

I've been trying to find information about the reason for this odd behaviour, and have not found much. There is this discussion that says that some builtin functions are defined with a "fast header" that does not accept keyword arguments.

This is because they use only a fast simple header ...(... PyObject *args) and therefore all named parameters are rejected automatically. (Python can never introspect into names of arguments in C source. :-)

Many other C functions have a header ...(... PyObject *args, PyObject *kwds) and they support exact list of names explicit by implementing much more complicated validation PyArg_ParseTupleAndKeywords. These names must be written to docs strings manually.

This is a bit confusing to me, but here they go some of my ideas on this (comparing it to other platforms). From what I've read the JVM has no notion of named arguments, meaning that the Kotlin compiler takes care of them when producing the bytecodes for the function call, and has a minimum performance implications at runtime:

Using named arguments produces the same bytecode as using positional arguments. However, changing the order of the arguments leads to the creation of temporary variables. This is because the arguments need to be evaluated in the order they appear, but passed to the method in the original order.

On the other hand, Python bytecode has the notion of Keyword arguments at the bytecode level, so it's the interpreter at runtime who takes care of them, which I guess has some minor performance implication. I would say that when writing native functions that you want to be as fast as possible, preventing a function from being invoked with keyword arguments provides some "nano-improvement".

Thanks to this searching I've found a surprising feature. I already knew that you can use the * symbol in your function signature to force all parameters after it to be passed as named (keyword) arguments, but I did not know that python 3.8 (PEP-570) introduced Positional Only Parameters. Using the / symbol in your function signature, parameters defined before it can only be provided as positional arguments, not as keyword ones.

At first sight it seems like an odd addition. That some builtin functins behave like that is a disturbance for me, so why would I want to disturb others by using that in some of my own functions? Well, there's a pretty good explanation here. Of the 4 reasons given there, the one that really makes this feature useful and valuable to me is the last one:

Since the parameters to the left of / are not exposed as possible keywords, the parameters names remain available for use in **kwargs:

Honestly I'd never thought about that potential problem when using variable number of keyword arguments. When you receive in your packed **kwargs an argument with a name that you already use for your other parameters you get an exception (Exception: [function] got multiple values for argument '[xxx]'), as the interpreter does not know to what parameter that named argument refers to. Let's see an example with the problem and the solution using /



def format_msg(msg: str):
    return f"[[{msg}]]"

def log_call(msg, fn, *args, **kwargs):
    print(f"{fn.__name__} {msg}")
    return fn(*args, **kwargs)
    

# no keywork argument, so it works fine
print(log_call("invoked", format_msg, "hi"))
#format invoked
#Out[11]: 'hi'

# but here we have a problem
try:
    print(log_call("invoked", format, msg="hi"))
except Exception as ex:
    print(f"Exception: {ex}")

# TypeError                                 Traceback (most recent call last)
# Input In [12], in ()
# ----> 1 log_call("invoked", format, msg="hi")
# TypeError: log_call() got multiple values for argument 'msg'


# that we can prevent by redefining the function like this:
def log_call(msg, fn, /, *args, **kwargs):
    print(f"{fn.__name__} {msg}")
    return fn(*args, **kwargs)
    

print(log_call("invoked", format_msg, msg="hi"))
# format_msg invoked
# [[hi]]'


Now that I'm aware of this potential problem in Python a further question arises, how do they manage this in Kotlin? Well, the thing is that they don't have this problem, cause kotlin supports variable number of unnamed arguments, but not of named arguments. The vararg modifier used in a function signature denotes an array of arguments, but there's not an additional modifier for denoting a dictionary of parameters (like python's **). Related to this, the spread operator * only applies to arrays, there's not an equivalent to Python's ** (packing/unpacking) for dictionaries.

Friday, 8 November 2024

Javascript Function.bind

In my previous post I talked about how in Python we have 2 different functions (functools.partial and functools.partialmethod) for using partial function application to functions or methods, due to how callables and bound methods work. A natural question is, what about JavaScript?

JavaScript comes with a Function.prototype.bind function. It allows us to bind a "this" value (thisArg) and other arguments (arg1, arg2...) to a function. When binding values to a function that does not use the "this" value (or just uses it for the 'global this') we can pass null or undefind as thisArg to bind. So Function.prototype.bind will serve us well for binding to a plain function, and for "extracting" a method from a class to use it as a function. I mean:



function format(it1, it2, it3, it4) {
    return `${it1} - ${it2} _ ${it3} . ${it4}`;
}

// function.bind works good for functions 
let fn1 = format.bind(null, "aa", "bb");
console.log(fn1("cc", "dd"));



class Formatter {
    constructor(name) {
        this.name = name;
    }

    format(it1, it2, it3, it4) {
        return `${this.name} + ${it1} - ${it2} _ ${it3} . ${it4}`;
    }
}

//and for a method that we want to "extract" from the class to use as a function 
let format2 = Formatter.prototype.format.bind({name: "Xuan"}, "aa");
console.log(format2("bb", "cc", "dd"));
 

Notice that arrow functions do not have a dynamic 'this', but what used to be called a lexical 'this' (the term seems to have fallen into disuse). They'll search for "this" in their scope chain [1]. This means that trying to bind a "this" value to an arrow function is useless, it will always search "this" in its scope chain. On the other side, binding other parameters to an arrow function is perfectly fine.



// function.bind works fine with arrow functions when binding normal parameters

let greet = (msg, fromX, toY) => `${msg} from ${fromX} to ${toY}`;
console.log(greet.name); //greet
let sayHi = greet.bind(null, "Hi");
console.log(sayHi.name); //bound greet
console.log(sayHi("Xose", "Francois"));
//Hi from Xose to Francois

// but binding "this" will not work. We get a bound function, but the "this" value will not be used by the arrow, it will continue to use the lexical this
let greet2 = (fromX, toY) => `${this.msg} from ${fromX} to ${toY}`;
let info = {msg: "Hi"};
let sayHi2 = greet2.bind(info); 
console.log(sayHi2("Xose", "Francois"));
//undefined from Xose to Francois


Using Function.prototype.bind for binding values to a method (that we'll continue to use as method, not as a function) rather than a normal function is a bit particular. So we have a class with a method that receives several parameters (apart from "this") and we want to bind some of them. Because of Function.prototype.bind expecting us to bind something as "this" we have to invoke bind passing that instance as thisArg and then attach the bound function to the instance itself. That's a bit odd, but OK for binding for a particular instance:



let formatter = new Formatter("Francois");
formatter.format = formatter.format.bind(formatter, "aa", "bb")
console.log(formatter.format("cc", "dd"));


But what if we want the parameter binding for all instances of the class, so that it works for any new instance that we create. For that our best bet is implementing our own partial function.



function partial(fn, ...trappedArgs) {
    return function(...args) {
        return fn.call(this, ...trappedArgs, ...args);
    }
}

// it works fine used with methods
Formatter.prototype.format = partial(Formatter.prototype.format, "aa", "bb")
let formatter2 = new Formatter("Francois");
console.log(formatter2.format("cc", "dd"));

//and when used with plain functions
let fn2 = partial(format, "aa", "bb");
console.log(fn2("cc", "dd"));


Thursday, 31 October 2024

Python Decorator Classes applied to Methods

When writing my post about functools.partial I noticed the presence of a functools.partialmethod function to apply partial to functions that will be used as methods. I had not realised of this before, but knowing how methods are implemented (functions implement the descriptor protocol, so when a function attached to a class is accessed through an object (receiver) the __get__ of the function gets invoked and returns a bound method) the issue becomes clear. From the documentation:

To support automatic creation of methods, functions include the __get__() method for binding methods during attribute access. This means that functions are non-data descriptors that return bound methods during dotted lookup from an instance.

functools.partial does not return another function (a Function is both a callable and descriptor), but a callable object that is an instance of a class other than Function (that has a __call__ method but is not a descriptor). Because of that, you can not use it as a method cause not having a __get__ you are losing the bound-method part. That's why a separate partialmethod function is needed.

We have this same problem with decorators that have been implemented as classes rather than as functions. When you apply one of these decorators, defined lets say as class MyDecorator, you create an instance of that MyDecorator class, and that instance is a callable object. If you apply MyDecorator to a method, obtaining a callable is not enough, we also need it also to have a __get__ that returns a bound method. Taking inspiration from here and here I've set up an example:


import types

class LoggedCall:
    def __init__(self, f):
        self.func = f
        # for the __name__, __doc__, etc attributes to be set to those of the wrapped function
        functools.update_wrapper(self, f)
        

    def __call__(self, *args, **kwargs):
        print(f"{self.func.__name__} invoked")
        return self.func(*args, **kwargs)

    def __get__(self, instance, cls):
        # the reason for adding the "if instance is" is not in case we apply it to an independent function (that would work fine)
        # is in case we decide to call it through the class and not an instance (see example below)           
        return types.MethodType(self, instance) if instance is not None else self
    

class Person:
    def __init__(self, name: str):
        self.name = name

    @LoggedCall
    def say_hi(self, someone: str):
        print(f"{self.name} says hi to {someone}")


p1 = Person("Iyan")
p1.say_hi("Francois")

# it for this odd way to invoke it for which we need to add the "if instance" in the __get__
Person.say_hi(p1, "Antoine")


Notice that this example is quite a bit contrived. Rather than using a decorator class we could have used a decorator function, that feels way more natural, and needs no extra bound-method creation from our part:


def log_call(func):
    functools.wraps(func)
    def wrapper(*args, **kwargs):
        print(f"{func.__name__} invoked")
        return func(*args, **kwargs)
    return wrapper


class Person2:
    def __init__(self, name: str):
        self.name = name

    @log_call
    def say_hi(self, someone: str):
        print(f"{self.name} says hi to {someone}")


p1 = Person("Iyan")
p1.say_hi("Francois")


Notice how in both cases I'm using functools.wraps of functools.update_wrapper for the wrapper function to look like the wrapped function (attributes: __module__, __name__, __qualname__, __annotations__, __type_params__, and __doc__)

So all in all, when adding a callable object that is not a function to a class as a method we need that callable to behave like a function, returning a bound-method when being accessed. For that we have to make the class of our callable object a descriptor also, with a __get__ method that returns a bound-method (just as functions do).

While writing this post I've gone throug a couple discussions [1] and [2] about using decorator functions vs decorator classes. Even if we want to keep state, I'm much more inclined to use a decorator function returning a closure than to use a decorator class.

Monday, 21 October 2024

Zarréu, Rain and Colomines

I love rainy weather and I love rainy places, it's been like that for most of my life. For European standards Asturies is a rainy region, 1000 mms per year in the main cities, above 1500 mms in mountain areas, allegedly 2000 mms in Picos d'Europa (but there are no meteo stations in the right places to measure that). One of the rainiest places in Asturies is on its southwest corner, L.leitariegos and Degaña allegedly should be well around 1700 mms, but in years (like the last one) when rains enter more from the SW than from the NW they can be well above that. From early September 2024 to October 10th there's been a combination of NW, W and SW situations that has left 450 mms in 40 days in Degaña (Zarréu)! Wow, that's massive and the best is that those have been strong but well sustained rains allowing the soil to absorb the water and not causing any sort of destructive floodings. All this being said, in the last year I've got a fascination with that hidden corner of Asturies called Degaña.

I've never set foot there as reaching the place in public transport is very, very complex, so I've done some virtual travel via google. Degaña has a coal mining past that brought it so many jobs and prosperity that are now long gone (as in neighbouring L.laciana (Vil.lablinu) in Llión, and in the other Asturian coal mining basins "cuenques mineres"). Because of that when you take a virtual-look into Zarréu you see one part of the village made up of traditional housing and another parts consisting of housing blocks built in the 60's (I guess) to house all the population growth gifted by the coal mining activities. It's the same kind of housing blocks that you'll find all over Asturies (frequently known as "colomines"), that in the outside look pretty similar to Soviet Krushevkas.

Like those, they were built with a max height of 4 floors, to avoid the need for elevators and save money. The ones that you see in Zarréu range from 2 to 4 floors, and have not seen any sort of restoration, so they look pretty old (but with the charm of a glorious past). To my astonishment, on one side of those blocks a very nice piece of modern architecture pops out. In the past I wrote this post expressing my delighment with some modern public housing buildings that I had just found in Mieres. As I say in that post, for Asturies that kind of construction is absolutely amazing, but it's something that you can find in so many places in France (except Toulouse, of course, the "Architectural redneck capital of Europe"). But this building in Zarréu is amazing in itself, not just in comparison to what is common in Asturies. The combination of modernity (shapes and black facades) and tradition (those slate tiles and those small balconies that are like a reinterpretation of the magnificent Asturian "corredor") is just unparallel here.



After the initial shock I investigated a bit to see how such a beautifully surprise came into existence (read: [1] and [2]). They were completed in 2009, as modern public housing intended for young people and coal miners. At that time there were still coal mines going on, and though the activity was clearly in decline, I think nobody could think at that time that in 10 years the Central Government would have closed them all (in the whole country) and that 2 years later it would close all the coal-fired power stations in the country!!!??? in their stupid plan of "achieving" the fastest energetic transition in the world, with a total despite for all those that will be left behind by this pseudo-ecologic delirium. We are talking of definitive closing, with the coal mines in Degaña and Ibias closing from one day to another, and with coal-fired power stations being dismantled almost immediatelly, so that following the German approach of putting some of them back to work to try to compensate the energy crisis due to the NATO war against Russia were totally impossible... So in retrospect that building was quite probably totally unnecessary (indeed it seems a twin building was also planned, but of course the idea was abandoned), as all that area (Ibias, Degaña, L.laciana...) has suffered a massive population decline, with the few young locals having mainly moved away in their search for survival (finding a job), and I guess that probably quite a few retired coal miners have also moved (like many in Mieres and Llangréu previously did, moving to Uviéu and Xixón)

The masterminds behind this gorgeous construction are the Asturian architect Nacho Ruiz Allén and the Navarrese architect Jose Antonio Ruiz Esquiroz. The've worked together in several other projects, among them this mesmerizing viewpoint also in Asturies: Walkway-Lookout over Rioseco Mines.

Do a google search for images of the place, it's outstanding:

Thursday, 17 October 2024

Create Class Instance given a String with its Name

Reflection has always fascinated me, from basic introspection to creating code at runtime, to more crazy ideas like modifying at runtime how the language behaves (how loops work, how inheritance works, how attribute lookup works). The other day, working with Python, I needed a not particularly unusual reflective feature, create instances of a class which name we have in a string. Think of a plugin system, you don't know at development time which plugins you'll have available, and at runtime you are provided with the name of a particular plugin class (a class that someone has made available in the modules search path) that has to be instantiated.

This is a pretty easy task. Classes live inside modules, so the first step is loading (if it's not already loaded) the corresponding module for that class. Once we have a reference to the module we can access to the class object using getattr() and invoke it (classes are callable) to create an instance.



# we have  dblib.oradb.py module, with an OracleConnector class in it
full_class_name = "dblib.oradb.OracleConnector"
parts = param["param_type"].split(".")
mod_name_parts, class_name = (parts[:-1], parts[-1])

# load module
mod = importlib.import_module(".".join(mod_name_parts))

# retrieve a class object
connector_cls = getattr(mod, class_name)

# invoke the "constructor" with the corresponding parameters
connector = connector_cls("geodata", "user1")

# obviously the module is now loaded in sys.modules
mod == getattr(sys.modules, "mod_name")
# True

If we know that the module for the class is already loaded and with the class added to the global namespace, we could obtain a reference to the class using eval(), but that's not a particularly good approach. I mean:


from dblib.oradb.OracleConnector import *

class_name = "OracleConnector"
connector = eval(class_name)


What about Kotlin (for the JVM) / Java? There's not an extra syntax or library in Kotlin for this, so it's just the same as in Java. While in Python modules are the "loading unit", in Java that "loading unit" is the class (classes are loaded from the file system by the Class Loader as needed). The equivalent to the 2 steps process in Python: load module and retrieve class object, is a single step in Java, get a Class object by calling Class.forName(classNameString). The next part is more complicated in Java than in Python. Python classes are callables, so we just have to invoke the class object to obtain the class instance. In Java we have to obtain the constructor function from the class, and the thing that I had almost forgotten is that due to overloading we can have multiple constructors (or multiple methods with the same name), so to retrieve the correct constructor we have to provide the types of the different parameters that we are going to use in the ensuing invokation. The main idea is this (simplest possible case, as we are using a parameterless constructor)



val classToLoad: Class = Class.forName(classNameString)
//simplest possible case, just get a parameterless constructor
val ob = classToLoad.getDeclaredConstructor().newInstance();


And now a more complex/verbose case using a constructor that expects parameters (taken from here)



class MyClass {
    public MyClass(Long l, String s, int i) {

    }
}

Class classToLoad = Class.forName(classNameString);

Class[] cArg = new Class[3]; //Our constructor has 3 arguments
cArg[0] = Long.class; //First argument is of *object* type Long
cArg[1] = String.class; //Second argument is of *object* type String
cArg[2] = int.class; //Third argument is of *primitive* type int

Long l = new Long(88);
String s = "text";
int i = 5;

classToLoad.getDeclaredConstructor(cArg).newInstance(l, s, i);


I think I had never thought until today about the fact that though Kotlin supports method overloading we use it less frequently than in Java, and that's just because optional parameters eliminates some of the cases where we would be using method overloading.

Thursday, 10 October 2024

Partial Function Application in Python

I won't explain here the difference between partial function application and currying, there are millions of articles, posts and discussions about it everywhere. As well explained by wikipedia, partial application refers to the process of fixing a number of arguments of a function, producing another function of smaller arity. JavaScript comes with function.bind, and Python comes with functools.partial. I rarely use it though. Most times I just need to bind a parameter to a function expecting few parameters, and for that I tend to just use a lambda, I mean:


from functools import partial

def wrap_msg(wrapper: str, msg: str) -> str:
    return f"{wrapper}{msg}{wrapper}"

wrap1 = partial(wrap_msg, "||")
print(wrap1("aaa"))

wrap2 = lambda msg: wrap_msg("||", msg)
print(wrap2("aaa"))


In the above case it's a matter of taste, but for other use cases there are clear differences. First, let's see what functools.partial exactly does:

Return a new partial object which when called will behave like func called with the positional arguments args and keyword arguments keywords. If more arguments are supplied to the call, they are appended to args. If additional keyword arguments are supplied, they extend and override keywords. Roughly equivalent to:

So if we want to bind a parameter that is not the first one we'll have to bind it as a named parameter, and then we will have to invoke the resulting function passing all the parameters located after it as named ones. That's a constraint that probably makes more convenient using a lambda. On the other side, given the they extend and override keywords feature, binding a parameter as a named one gives as a very nice feature, we can use it to turn parameters into default parameters. We use partial to bind those parameters for which we've realised we want a default value, and then we still can invoke the resulting function providing those values if we want. I mean:


cities = ["Xixon", "Paris", "Lisbon"]
def join_and_wrap(items: Iterable[Any], jn: str, wr: str) -> str:
    return f"{wr}{jn.join(items)}{wr}"

join_and_wrap_with_defaults = partial(join_and_wrap, jn=".", wr="||")

# make use of both defaults
print(join_and_wrap_with_defaults(cities))

# "overwrite" one of the defaults
print(join_and_wrap_with_defaults(cities, jn=";"))
print(join_and_wrap_with_defaults(cities, wr="--"))

I was checking what others had to say about all this, and this post is an interesting one. They mention how functools.partial is pretty convenient to prevent the Closure in a loop issue. As you can see in the code below it's less verbose than the strategy of creating a wrapper function that works as scope. Notice that indeed with partial we are not creating a closure (it return a callable object that is different from a function, and traps the bound arguments as attributes of the object, not in the cells of a __closure__).


greetings = ["Bonjour", "Bon día", "Hola", "Hi"]

# remember the "closure in a loop issue" as the the variable has not a "iteration scope"

print("'- Closure in a loop' issue")
greet_fns = []
for greeting in greetings:
    def say_hi(name):
        return f"{greeting} {name}"
    greet_fns.append(say_hi)

for fn in greet_fns:
    print(fn("Francois"))

#Hi Francois
#Hi Francois
#Hi Francois
#Hi Francois

print("------------------------------")

print("- Fixed 1")
greet_fns = []
def create_scope(greeting):
    def say_hi(name):
        return f"{greeting} {name}"
    return say_hi
for greeting in greetings:
    greet_fns.append(create_scope(greeting))

for fn in greet_fns:
    print(fn("Francois"))

#Bonjour Francois
#Bon día Francois
#Hola Francois
#Hi Francois

print("------------------------------")

print("- Fixed 2")
greet_fns = []
def say_hi(greeting, name):
    return f"{greeting} {name}"
for greeting in greetings:
    greet_fns.append(partial(say_hi, greeting))

for fn in greet_fns:
    print(fn("Francois"))

#Bonjour Francois
#Bon día Francois
#Hola Francois
#Hi Francois

Reading about this stuff I've come across the better partial module, that implements an interesting approach to partial application, that feels like something in between the "classic" partial and currying. It creates partial functions that create other partial functions. You invoke them with real values and place-holders, until you have provided all the parameters. Sure this explanation is rather unclear, so I better copy-paste here an example from the module github.



from better_partial import partial, _
# Note that "_" can be imported under a different name if it clashes with your conventions

@partial
def f(a, b, c, d, e):
  return a, b, c, d, e

f(1, 2, 3, 4, 5)
f(_, 2, _, 4, _)(1, 3, 5)
f(1, _, 3, ...)(2, 4, 5)
f(..., a=1, e=5)(_, 3, _)(2, 4)
f(..., e=5)(..., d=4)(1, 2, 3)
f(1, ..., e=5)(2, ..., d=4)(3)
f(..., e=5)(..., d=4)(..., c=3)(..., b=2)(..., a=1)
f(_, _, _, _, _)(1, 2, 3, 4, 5)
f(...)(1, 2, 3, 4, 5)


Wednesday, 2 October 2024

Using Suspend Functions to Prevent Stack Overflows in Kotlin

This post is the Kotlin counterpart to these 2 previous JavaScript and Python posts. Avoiding stack overflows by means of using suspendable functions (or async-await in JavaScript, Python). This technique does not have much of a real practical purpose, the standard mechanism to prevent stack overflows is using trampolines, so consider this post as a brain exercise that has helped me to better understand how coroutines and suspendable functions work (same as the aformentioned posts helped me better understand async/await).

When a kotlin suspend function gets suspended at a suspension point it returns to the calling function, so the corresponding stack frame is released (the function will be later resumed, in the sentence following that suspension point, by "something" calling the resume method of the continuation object corresponding to that function. If we are not stacking things in the stack, there's no risk of a stack overflow. So if we add in our recursive function a call to a trivial suspend function before the next call to the recursive function, we should avoid the stack overflow. The main thing here is clarifying what "trivial" suspend function we can use for this.

Let's start by showing a recursive function that causes a stack overflow:


fun wrapText1(txt: String, times: Int): String = 
    if (times <= 0) txt
    else wrapText1("[$txt]", times - 1)
	
try {
	println(wrapText1("b", 10000))
}
catch (ex: StackOverflowError) {
	println("StackOverflow!!!")
}

// StackOverflow!!!


First of all, that case could be fixed by leveraging Kotlin's compiler support for Tail Call Optimizations. That function is Tail Recursive, so we can just mark it with the tailrec modifier and the compiler will transform that recursion into a loop, so no more overflows.


tailrec fun wrapText1(txt: String, times: Int): String = 
    if (times <= 0) txt
    else wrapText1("[$txt]", times - 1)

That said, let's forget about the tailrec power. That works for tail recursive functions, while the idea of leveraging suspendable functions to prevent stack overflows will work the same for tail and non-tail recursion. Notice that I´ll run my recursive function in a coroutine created with the runBlocking builder using an event loop dispatcher, so everything runs in a single thread, as in JavaScript and Python asyncio.

Let's first try using a suspend function that indeed does nothing. It does not invoke another "real" suspendable function. As you can see in my post from 2021, in JavaScript awaiting on a resolved Promise (or just on a normal object) is enough to avoid the stack overflow, as it causes the function to be suspended, returning and releasing its stack frame. The "then" part to run after the await is not executed inmmediatelly, but put on the microtask queue, meaning that the function returns to the event loop (and this will execute the "then handler" taken from the microtasks queue). Asynchrony in Kotlin is implemented in a quite different way (continuations...) and I was quite convinced that this "doNothing" function would not cause a suspension, but I had to confirm it.


suspend fun doNothingSuspend(): String {
    return "a"
}

// as expected this one DOES NOT prevent Stack Overflows. Calling a function marked as suspend but that is not really suspending is useless for this
// there won't be suspension. So it's like Python asyncio
suspend fun wrapTextAsync1(txt: String, times: Int): String {
    return if (times <= 0) txt
    else {
        doNothingSuspend()
        wrapTextAsync1("[$txt]", times - 1)
    }
}

try {
	println(wrapText("b", 10000))
}
catch (ex: StackOverflowError) {
	println("StackOverflow!!!")
}

//StackOverflow!!!


Yes, we get an ugly stack overflow. Calling that suspend function does not cause a suspension cause the complex code in which a suspend function is transformed by the compiler will not return an COROUTINE_SUSPENDED value, that is what the calling function checks in turn for returning to its calling function (and so on down to the event loop) or continuing with the next sentence (that is what happens in this case). This is a bit related to what happens in Python, where marking a function that "does not really do something asynchronous" as async (a sort of "fake async function") will not really suspend the function. Because of how coroutines are implemented in Python (similar to generators), the chain of async calls will get down to the Task that controls the chain of coroutine calls, and as what has been returned is not an awaitable, the Task will just invoke send() in the first coroutine in the chain, which will go through the different coroutines in the chain recreating their stacks, up to the coroutine from which we had invoked the "fake async function", that will continue, hence running sequentially, not interrupted by a suspension, without having returned the control to the event loop.

So if we want to have a real suspension in our recursive function, returning from the function and avoiding a stack overflow we'll have to invoke a function that really suspends. The immediate option that comes to mind is using a sort of sleep (delay, in the Kotlin coroutines universe) with a minimum value. I say "minimum value" cause passing just 0 to kotlinx.coroutines.delay will return immediatelly without causing any suspension (it won't return a COROUTINE_SUSPENDED value).


suspend fun wrapTextAsync2(txt: String, times: Int): String {
    return if (times <= 0) txt
    else {
        // using a delay(0) is NOT an option, as it returns immediatelly and no suspension happens
        delay(1)
        wrapTextAsync2("[$txt]", times - 1)
    }
}


That works fine. delay(1) returns to the event loop and after that delay the corresponding continuation will resume the recursive function in the next sentence in the function (the next call to that wrapTextAsync2) with a clean stack. But obviously there's an important performance penalty due to the delay.

If you've read this previous post you'll be also familiar with kotlinx.coroutines.yield. Reading the implementation details I was a bit dubious as to whether running my code in an environment consisting of an event loop created by runBlocking with no multiple suspend functions running concurrently (so no items waiting in the event loop queue) would be enough for yield() to cause a suspension, but if you invoke the function below, with whatever huge times value, you'll see that no stack overflow happens. So yes, yield() prevents a stack overflow.


suspend fun wrapTextAsync3(txt: String, times: Int): String {
    return if (times <= 0) txt
    else {
        yield()
        wrapTextAsync3("[$txt]", times - 1)
    }
}