Friday 27 January 2023

Kotlin "::" Syntax

The :: symbol is used in Kotlin to address some syntactic particularities. We saw last week that functions are first-class citizens that can be passed around and returned, but it happens that the way to get a reference to a function or method is a bit odd. In Python and JavaScript we can assign a function (independent function, method...) to a variable just doing: let fn = functionName; let fn = obj.methodName;. In Kotlin we have to use ::: Maybe this has to do with the fact that in Kotlin there are at least 2 cases in which you can invoke a function without parentheses (trailing lambda expressions and infix notation) and because of that it could be difficoult for the compiler to determine if we are getting a function reference or invoking it.

Function References. So as I've just said, to assing, pass over, return... an existing function you have to obtain a function reference, using ::


fun sayHi(from: String, to: String) {
    println("Hi from $from to $to")
}

    // for this the compiler creates a Singleton class
    val greet = ::sayHi
    greet("Xose", "Francois")


The Kotlin compiler creates a singleton class implementing one of the "Function Types interfaces", kotlin.jvm.functions.Function2, that from its invoke method calls into the sayHi function.

Kotlin also allows you to immediatelly invoke the function reference. In that case, as you don't need a "Function object" to pass around, no class is created, it just generates exactly the same bytecodes than for a direct call to the function. I mean, these 2 lines are exactly the same:


   (::sayHi)("Xose", "Francois")
    sayHi("Xose", "Francois")  

Method References. We can get a reference to a method. If we get the method from the class it will be un Unbound Method Reference (when invoking it we have to pass as first parameter the "receiver"). If we get the method from an instance we get a Bound Method Reference.
Let's see an example for Unbound Method References


class Formatter constructor(val wrap: String) {
    fun format(msg: String): String {
        return "${wrap}-${msg}-${wrap}"
    }
}

fun testUnboundMethodReference2(){
    // for this one the compiler creates a new Singleton class
    val format: (Formatter, String) -> String = Formatter::format
    println(format(Formatter("|"), "AA"))
}   

For that case the compiler creates a Singleton class also implementing the Function2 "Function Type Interface", with an invoke method that receives 2 parameters and uses the first parameter as receiver for calling into format.

We can also invoke a method reference immediatelly, in that case no new class is created, it's just the same as doing a normal call through the receiver


fun testUnboundMethodReference1(){
    // no new class gets created. Indeed the bytecodes are basically the same as in testNormalMethodCall
    println((Formatter::format)(Formatter("|"), "AA"))
}

And now an example forBound Method References


fun testBoundMethodReference(){
    // obviously for this the compiler also creates a new class (not a singleton, it has the receiver (formatter object) as a property)   
    val formatter = Formatter("|")
    val format: (String) -> String = formatter::format
    println(format("AA"))
}   

The compiler creates a new class with a property pointing to the receiver and an invoke method that receives one string as parameter and invokes "format" through the receiver.

We can also get a references to a constructor by using: myFn = ::ClassName. Again a new singleton class is created by the compiler. As Kotlin does not use "new" to create instances, we can pass around that constructor reference as a factory to other methods, same as we can do in Python.


fun testConstructorReference(){
    val factory: (String) -> Formatter = ::Formatter
    val formatter = factory("|")
    println(formatter.format("AA"))
}

Finally we can also get bound and unbound refereces to properties.
I've just realized that what I've posted so far here is almost like a rewrite, with different samples and some information about what the Kotlin compiler generates, of what comes in the Callable References" documentation.

The additional use of the :: syntax is to obtain an object (an instance of KClass) that contains information about one class. We can get it directly from the class Person::class or from an instance myPerson::class. We should not confuse a Kotlin KClass with a Python metaclass. A KClass is just information about a class, while a Python metaclass is a class used to create other classes (this idea does not exist in Kotlin).

Sunday 22 January 2023

Throw as en Expression

Thanks to learning Kotlin I've got introduced to some features and techniques that I had never thought about. I guess it's because Kotlin is the most funtional programming friendly language that I've used. I've read somewhere that in Kotlin every statement is either a declaration or an expression. Thanks to this we have "if-else expressions", "switch expressions", "try expressions", "throw-retur-break-continue expressions"... The idea of a throw expression seemed odd to me, until that I saw an example:


fun fooOffensive(value: String) {
     val number = value.toIntOrNull() ?: throw IllegalArgumentException("The given argument is not a valid number string")
    // do something with number
}


It's nice, really nice. Neither JavaScript nor Python (this is obvious, as while Python has many, many strengths, it's, among modern languages, one of the less functional programming friendly ones) has this feature, but I've realised that we can easily get the same effect of throw expressions. Just by defining a throwException function we can write code using the ternary operator in JavaScript like this:


function throwException(ex){
    throw ex;
}

function processMessage(msg){
    msg = msg.startsWith("Msg-") ? msg.toUpperCase() : throwException(new Error("Wrong format"));
    console.log(`Processing message: ${msg}`);
}

let msgs = ["Msg-Connect", "aaa"]
for (let msg of msgs){
    try{
        processMessage(msg);
    }
    catch (ex){
        console.log(`Error!!! - ${ex}`);
    }
}

// Processing message: MSG-CONNECT
// Error!!! - Error: Wrong format


And code using optional chaining like this:


let europe = {
    countries: {
        France: {
            population: 65000000,
            cities: {
                Paris: {
                    population: 2000000,
                    code: 75
                }
            }
        }
    }
}

function getCityInfo(city){
    let code = europe?.countries?.France?.cities?.[city]?.code ?? throwException(new Error("Missing City"));
    console.log(`getting city info for ${code}`);
}

let cities = ["Paris", "Lyon"]
for (let city of cities){
    try{
        getCityInfo(city);
    }
    catch (ex){
        console.log(`Error!!! - ${ex}`);
    }
}


// getting city info for 75
// Error!!! - Error: Missing City


In Python we can leverage an equivalent raise_exception function when using the (sort of) "ternary operator":


def raise_exception(ex: BaseException):
    raise ex

def process_message(msg):
    msg = msg.upper() if msg.startswith("Msg-") else raise_exception(Exception("Wrong format"))
    print(f"Processing message: {msg}")


msgs = ["Msg-Connect", "aaa"]
for msg in msgs:
    try:
        process_message(msg);
    except Exception as ex:
        print(f"Error!!! - {ex}")
        
# Processing message: MSG-CONNECT
# Error!!! - Wrong format


As we know Python lacks so far a "safe navigation - optional chaining" operator (there are renewed discussions, and the usual absurd oppositions based on "it's unpythonic, it makes code hard to read (for non real programmers)..." and so... As we saw in this post, we can get similar safe navigation effect with a rather clear syntax, just by using a function. I've adapted such function to throw exceptions if requested. It receives a lambda (already trapping in its closure the variable to "navigate") and an optional second parameter. If the safe navigation fails, if the second parameter has not been provided the function returns None, if provided, if it's an instance of Exception it throws it, otherwise returns the value.



def raise_exception(ex: BaseException):
    raise ex
    
def safe_access(fn, default = None):
    """ returns None by default if something in the chain fails. 
        If the default value is provided and it's an Exception, it throws it, otherwise it returns it
    """
    try:
        return fn()
    except (TypeError, KeyError, IndexError, AttributeError):
        return raise_exception(default) if isinstance(default, BaseException) else default


class Person:
    def __init__(self, name, age, profession):
        self.name = name
        self.age = age
        self.profession = profession

class Profession:
    def __init__(self, name, salary):
        self.name = name
        self.salary = salary
    
    def format_profession(self, st1, st2):
        return f"{st1}-{self.name}-{st2}"

p1 = Person("Francois", 4, Profession("programmer", 50000))

print(safe_access(lambda: p1.name))
print(safe_access(lambda: p1.city))
print(safe_access(lambda: p1.city.name, "Paris"))
try:
    print(safe_access(lambda: p1.city.name, Exception("Missing city")))
except Exception as ex:
    print(f"Error: {ex}")
print(safe_access(lambda: p1.profession.salary))
print(safe_access(lambda: p1.profession.format_profession("¡¡", "!!").upper()[:6]))
print(safe_access(lambda: p1.profession.format_profession("¡¡", "!!").upper()[:6]))

# invoking a non callable
print(safe_access(lambda: p1.profession.salary()))


Thursday 19 January 2023

Kotlin Function Types

I've recently started to fiddle with Kotlin and it's being a really nice experience. It reminds me a bit of my excitement in 2011 when I discovered Groovy. Well, as I've always been a fan of dynamic languages I would say that Groovy still seems more advanced to me, but given how much attention Kotlin has gained, Kotlin seems to me the best option to leverage the power of a JVM. There are tons of concepts that suddenly come up when you start to learn Kotlin, most of them sound more or less familiar from other languages, but they've helping me to rethink how I do things in other languages, why other languages do things differently, etc.

I'll talk today about Function Types, and anything that I'll say in this post is related to Kotlin code compiled for the JVM. So in Kotlin, as in any other modern language, functions are first-class citizens. They are like objects that can be assigned, passed around and returned. Kotlin is statically typed, and for the JVM functions are not objects, they are always methods of a class. Function Types are the mechanism that allows Kotlin to manage function as objects with well defined types (the type of their parameters and of its return). Hum, I've just realized that I have just rewritten with other words what I learnt 1 month ago in this excellent article.

From a Type Checking point of view Function Types are more or less the same that when in Python we use the Callable type with Type Hints. A Kotlin Function Type like this: (String, Int) -> String that represents a function that receives a string and an int and returns a string is equivalent to this Type Hint in Python: Callable[[str, int], str]. However, Function Types are more than a Kotlin facility to help with Type Checking. In the JVM everything is typed, so Function Types have to be expressed somehow at the JVM level, so in the end Function Types are syntactic sugar for an Interface containing an invoke method. As I aforementioned, when Kotlin code is compiled to Java bytecodes, Kotlin functions have to end up being methods in a class. So for each "function object" (lambda expression, anonymous function, function reference) that we define the Kotlin compiler will create a class that implements an interface that corresponds to the Function Type for that function. Decompiling with javap -c the .class files generated by kotlinc you'll see that these are the kotlin.jvm.functions.FunctionX interfaces, for the Function Type that I'm using as example we have: kotlin.jvm.functions.Function2<java.lang.String, java.lang.Integer, java.lang.String>. This is explained here.

Kotlin has also Function Types with Receiver, something like: String.(Int) -> String. This is used to represent functions that can be invoked as if they were methods of the first parameter (the receiver), I mean, "hi".say(2). We could say that they are an additional layer of Syntax Sugar, cause the interface for String.(Int) -> String is the same interface that we saw before for (String, Int) -> String, that is: kotlin.jvm.functions.Function2<java.lang.String, java.lang.Integer, java.lang.String>. Given that in the end both Function Types represent the same interface a Function Type with Receiver can be invoked through the receiver or with the receiver as its first explicit parameter. Also, I can do assignmet between compatible types of Funtion Types with and without receiver. What is not allowed is invoking a normal Function Type as a compatible Function Type with Receiver. I hope the code below makes a bit more clear what I mean.


class City constructor(var name: String, var population: Int) {
    fun multiplyPopulation(factor: Int){
        population *= factor
    }   
}

val paris = City("Paris", 2000000)

// declare a Function Type with Receiver
var multiplier2: City.(Int) -> Unit
// assign a function reference to it
multiplier2 = City::multiplyPopulation 

// can invoke it both through the receiver
paris.multiplier2(2)

// or passing the receiver as parameter
multiplier2(paris, 2)

// -------------------------------

// declare a Function Type
var multiplier3: (City, Int) -> Unit = City::multiplyPopulation 

multiplier3(paris, 2)

// I CAN NOT invoke a "normal function type" as a "function type with receiver"
//paris.multiplier3(2) // Compilation Error: unresolved reference: multiplier3


// I can assign a "normal" Function Type to a compatible Function Type with Receiver
multiplier2 = multiplier3
// and invoke it one way or another
multiplier2(paris, 2)
paris.multiplier2(2)

// I can also assign a Function Type with Receiver to a "normal" Function Type
multiplier3 = multiplier2
multiplier3(paris, 2)

// but as expected this does not compile
// paris.multiplier3(2)

With the examples that I've used so far one could think that this idea of Function Types with Receiver is not particularly interesting. Being able to write "a".say(2) rather than say("a", 2) is not something that makes the code particularly more expressive or beautiful. Right, but I think the main idea of Functions Types with receiver is using them with Lambda Expressions with Receiver when passing them as parameters to another function. The apply scope function is the perfect example. We can write code like this:


city.apply {
	name = "a"
	age = 47
}

because the apply function is defined as receiving a Function Type with Receiver, like this:


public inline fun  T.apply(block: T.() -> Unit): T {
    block()
    return this
}

I guess Kotlin guys (some of them coming from Java) will love the "implicit this" feature (the block() line is the same as this.block(), the name = "a" is the same as this.name = "a"), but for me sometimes it makes code less straightforward. I guess it's because I've got so used to JavaScript, where "this" can be dynamic or "static" ("bound" through function.bind or trapped as "lexical this" in arrow functions), but you have to explicitly type it, and to Python where the first parameter of a method is just named "self" by convention and you have to type it both in the method signature and when using it.

Tuesday 10 January 2023

Creating function at runtime

It's a common mantra that creating code at runtime from a string (with eval or some similar function for those languages providing this feature) is dangerous and should be avoided. Yes, that's right, but there are still cases where it's very useful and not dangerous at all. Lately I've been using it in Python in a small, "private" app used only by a few people that are also developers, and has been really helpful. I remember having used "eval" in Perl quite a few years ago, but I'll just talk about JavaScript and Python.

Let's say we want to dynamically "compile" some code into a function (not just eval it and run it once) as we'll be using that dynamic code multiple times.

JavaScript provides the eval function. It accepts a string representing a expression or 1 to n statements. When we provide an expression it returns the result of such expression. In JavaScript we can declare functions as expressions, either with an arrow function expression or a function expression. So to create that dynamic function we would write its code in a string for any of those 2 types of expressions, eval it, and assign the eval result to a variable.


// option 1: 
// arrow function in a string
let sfn1 = "st => console.log('st: ' + st)"
let fn1 = eval(sfn1)
// fn1 points now to an anonymous function
fn1("bb");
// st: bb

// fn1.name === "" 


// option 2: 
// function expression in a string
let sfn2 = "(function dynFn(st){ console.log('st: ' + st); })";
let fn2 = eval(sfn2);

fn2("bb");
// st: b

// fn2.name === dynFn 


Notice 2 interesting things with option 2. First, in order for it to be considered a function expression I have to wrap int in "()", otherwise it would be considered a function statement. Second, I can give the function a name, which is always useful. For arrow functions JavaScript compiler is smart enough to assign a name to the function when declared and immediatelly assigned, I mean: let fn = st => console.log('st: ' + st);, but that does not work when the declaration is in an eval.

Python provides 3 functions to deal with code in strings: eval, compile and exec. This question explains the differences pretty well. Notice what I mentioned in my previous post:
The only Function Expressions in Python are lambdas (that in Python can only contain an expression). We still lack (multi)statement lambdas.
his makes the python code a bit more verbose again. I'm going to use the exec function. I'll put in the string a function declaration, and in order to have access that function that we're going to compile dynamically, I'll have to assign it to a variable present in the scope where exec is run. There's a trick with this, as explained here. The code that we compile and run inside exec has read access to the variables in the scope where exec is run, but not write access. This is a bit like the thing with closures and having to use the nonlocal declaration, but that will not work here. The thing is that being "a" a variable declared outside the "exec", doing "a = v" inside the exec will not make the variable 'a' point to v, but if 'a' points to an object we can modify the contents of that object referenced by a, I mean doing "a.b = v" or "d['b'] = v" will perfectly work. So in the end we have something like this:


fn_st = (
"def multiply(num1, num2):\n"
"    print('multiplying numbers')\n"
"    return num1 * num2\n"
)

def create_function(fn_st, fn_name):
    d = {}
    exec(fn_st + "\n" + "d['fn'] = " + fn_name)
    return d["fn"]


new_fn = create_function(fn_st, "multiply")
print("created")
print(new_fn(2, 3))

# multiplying numbers
# 6