Kotlin Asynchronous Programming

One of the first things I looked into when I started to fiddle with Kotlin was if it had the async-await "magic keywords" and it took me a while to understand that the approach to asynchrous programming in Kotlin is rather different from the strategies followed in JavaScript, Python or C#.

Asynchronous programming in JavaScript revolves around Promises, both if you use the async-await magic or if you go more old-school and explicitely chain Promises. A function marked as async in JavaScript implicitly returns a Promise, and when you await on that Promise the compiler "breaks" the function transforming it into a state-machine so that when that promise is resolved the function is invoked again in the point after the await (the corresponding state in the state-machine). Asynchrony in C# revolves around Tasks, and methods marked as async return a Task. In Python asynchronous programming revolves around awaitables (Tasks, Futures and Coroutine objects) and when a function marked as async (a coroutine function returning a coroutine object) performs a real asynchrouns operation (it gets "suspended"), it will yield a Future to the event loop (the python approach is more confusing to me, but I managed to more or less explain it last year)

In Kotlin the key element in asynchronous programming are suspendable functions (marked with the suspend keyword) and the Continuation Passing Style (CSP) that comes with them. Indeed suspendable functions are a broader concept, as they are also used for the equivalent to generators in JavaScript and Python. As you can guess from its name a suspendable function can be suspended at different points, suspension points, points from where the function is calling other suspendable functions. Once the function invoked from the suspension point completes the caller/calling function will be resumed from that suspension point. Each suspend function is indeed a state machine. Well, this feels like async functions in other languages, but the internals are a bit different. Suspendable functions do not return Promises/Futures... but receive something that at first (wrong) sight seemed like callbacks to me, Continuations. A Continuation contains the function to be invoked, its state (the values of its variables) and the point where the function has to be resumed (state machine), so it's more complex than a simple callback. From this excellent article:

Each suspendable lambda is compiled to a continuation class, with fields representing its local variables, and an integer field for current state in the state machine. Suspension point is where such lambda can suspend: either a suspending function call or suspendCoroutineUninterceptedOrReturn intrinsic call.

So for each suspendable function we have a Continuation class that contains the code of the function, its variables and the point where it has been suspended (the function is made up of a switch-case state machine, so that point is a label for that switch). When a suspendable function f1 invokes another suspendable function f2 it passes over to it its own continuation object. f2 will set in its own continuation a reference to the f1's continuation. This way, once the invoked suspend function f2 completes the continuation for the caller function f1 will be invoked. If in turn f2 invokes another suspendable function f3, this will be added to the chain of continuations: f3 -> f2 -> f1. You can read about this for example here or here

Notice that the way this chain of Continuations is created is like the opposite of what we do in JavaScript (or C#, or Python) when creating a chain of Promises, where when an async function returns a Promise the caller function attaches to the Promise the next function to execute (via .then()

Apart from suspend functions and continuations the other key aspect of asynchrony in Kotlin are coroutines. A chain of suspend function calls runs in a coroutine. A suspend function can only be invoked from another suspend function or from a Coroutine. From here.

A coroutine is an instance of a suspendable computation. It is conceptually similar to a thread, in the sense that it takes a block of code to run that works concurrently with the rest of the code. However, a coroutine is not bound to any particular thread. It may suspend its execution in one thread and resume in another one.

So a coroutine is a "suspendable execution flow", similar to threads or processes, but we don't create coroutines (instances of classes inheriting from the AbstractCoroutine class) directly, we create them through a CoroutineBuilder function. CoroutineBuilders like launch or runBlocking are extension methods of the CoroutineScope interface. The source code for coroutine builders is for sure pretty interesting. We can see there that a coroutine builder creates a CoroutineContext that passes over to the Coroutine constructor and then the builder invokes Coroutine.start().

CoroutineScopes and CoroutineContexts are the other essential pieces of Kotlin asynchrony. They are well explained here, where we can learn that a CoroutineScope holds a CorotuineContext, and that the CoroutineContext is a set of various elements. The main elements are the Job of the coroutine and its Dispatcher. The Dispatcher determines what thread/s will be used for executing the different suspend functions of the coroutine. A Continuation has a reference to its Context. From another article I noted down this: "When resuming, continuations don't invoke their functions directly, but they ask the dispatcher to do it. The Dispatcher is referenced from a CoroutineContext, that is referenced from the Continuation."

Another good article about coroutines is this one

There's an important usage difference that comes from the differences between Promises/Tasks/Futures... and Continuations. In JavaScript when you invoke an async function you have to explicitly state (by means of the await keyword) that you are "suspending" at that point until the Promise is resolved. In Kotlin there's no option to indicate if you want to suspend or not, any invocation to a suspend function means that the caller function will get suspended (if the called one really does something asynchronous). I guess the explanation for this is because you can only pass your continuation to the suspend function at the point of invokation, while with promises you can decide to await for the result just as you obtain the Promise, or await for it later (doing something else in between). The problem with this is that when your suspend function invokes several functions (some "normal" funtions, some suspend functions), you don't know at first sight if it's suspending at those invokations points or not, you have to check if the invoked function is a "normal" or suspend one. In one hand forcing you to use a sort of "await" keyword when invoking a suspend function would be redundant, but on the other hand would make code more clear to me.

What can be a bit confusing is that while Kotlin does not need async-await keywords for writing asynchronous code, it does indeed provide an async() function and an await() function, that you can need in certain cases. async() is a coroutine builder, that contrary to other builders like launch(), returns a Deferred (a sort of Promise/Future) that completes when the corresponding coroutine finishes. In turn, Deferred has an await suspendable method. We can see an example combining both functions here. Related to the previous paragraph, if you want to invoke a suspend function, but not get your current function suspended (as when in JavaScript we invoke an async function but do not await for it at that point) you can wrap the call to the suspend function in a block and pass it to async().

Reading a bit about asynchrony in Raku (that pretty interesting descendent of the honorable Perl language) I've seen that they have Promises also, but they keep() or break() rather than resolve() or reject(). It seemed odd at first sight, but indeed that's what we use in natural language, keeping or breaking promises, not resolving or rejecting.