Sunday 26 December 2021

Promises and catch

I've recently been working on JavaScript code written by other people where they were using .then()/.catch() chains rathern than async-await magic, and I should not change that, so I've had to review a bit how then()/.catch() works (thanks to async-await I've rarely used them in the last years).

The first point is a reminder about promises rejection in general. When inside .then() we want to reject the promise, should we use Promise.reject(new Error()) or throw new Error()?
Well, it's basically the same. If you do a throw inside a .then or a .catch the .then or .catch will trap the exception and reject the current promise. You can read more about this here.

Both methods produce the exact same response. The .then() handler catches the thrown exception and turns it into a rejected promise automatically.

The second point is that I had almost forgottent that the .then() method accepts a second parameter, a handler function to be called when the promise has been rejected. So, what's the difference between that and chaining a .catch() call? I mean:



getData()
.then(
	data => {/* do whatever */},
	error => {/* treat error */}
);

vs

getData()
.then(data => {/* do whatever */})
.catch(error => {/* treat error */});



Well, the behaviour is almost the same, but there's a subtle difference. If the success handler throws an error, in the first case, that error is not being caught, but in the second case, the .catch, will catch that error. It's well explained here

The next thing is with how many .catch() calls you need. If you have a chain of .then() calls and want to catch any exception in any of them, and apply the same treatment, you need just one .catch() at the end of the call. This means that this code:


async function getProcessedMessage(){
	try{
		let msg = await getMessage();
		msg = await correctOrtography(msg);
		msg = await replaceOddWords(msg);
		msg = await addExtraFormatting(msg);
		return msg;
	}
	catch(error){
		console.log("unable to perform the whole processing");
		return "";
	}
}

is equivalent to this:


function getProcessedMessage(){
	return getMessage()
	.then(correctOrtography)
	.then(replaceOddWords)
	.then(addExtraFormatting)
	.catch(error => {
		console.log("unable to perform the whole processing");
		return "";
	})
}


If what we want is managing exceptions call by call (I mean, differently for different .then() calls), to do some treatment (logging the exception and throwing a "higher level one") or recovering (we replace by a default value and continue with the ensuing calls) then we'll have a .catch() for each .then(). This code:


async function getProcessedMessage(){
	let msg;
	try{
		msg = await getMessage();
	}
	catch(error){
		console.log("failed to retrieve message, we'll use the default");
		msg = "default";
	}
	try {
		msg = await translate(msg);
	}
	catch(error){
		console.log("low level error: " + error.message);
		throw new Error("failed to translate);		
	}
	return msg;
}

is equivalent to this:


function getProcessedMessage(){
	getMessage().
	.catch(error){
		console.log("failed to retrieve message, we'll use the default");
		return "default";
	}
	.then(translate)
	.catch(error){
		console.log("low level error: " + error.message);
		throw new Error("failed to translate);			
	}
}

Thursday 23 December 2021

FIC Xixón 2019

Since I started this blog, for every edition of the Xixon Internationl Film Festival (FICXixon) during which I was in town, I've done a review of the screenings that I attended to. When starting to write the one for 2021 I've realised that I had missed to complete and publish the one from 2019 (in 2020 due to the pandemic the festival was only online, and I did not watch any film). So well, 2 years late, here it is!

Though in 2019 I was in Xixón during the 9 days of the festival, I only attended 3 films of this 57th edition. After the years that I missed the festival cause I was living abroad I no longer have that urge to attend 1 (or more) films per day, and mainly, I only look into films being screened in Teatro Jovellanos, the main site for the festival. Also, I missed a couple of films cause I did not hurry enough to get the tickets... This said, these are the films I watched

  • Thursday, 2019/11/21, La hija de un Ladrón, Belén Funes, Spain, 2019 A good film. It goes through the life of a young Spanish girl striving to get a life for for her and her little kid. She has not had an easy life, with a missing mother; a father that has spent time in prison, an odd relation with the father of her child... and lives for the moment in a shared flat managed by the Catalan social aid system (she's not part of any 'minority' and does not seem to be into drugs or have committed any crime, so don't expect she'll be considered of particular interest to receive much help from the social system...) The girl is a fighter, doing her best to get and keep a job, and to protect her kid and her little brother. Recorded in a rather realistic style, probably this is not a film that you will particularly remember, but you won't quit it either, and you'll have the feeling of having spent your time correctly.

  • Friday, 2019/11/22, Rounds, by Stephan Komandarev, Boulgarie, Serbie, France, 2019. I think this is the second Bulgarian film that I've watched in my life (at least that I'm aware of) and as my previous Bulgarian experience, this is an excellent work, really excellent. It follows the lives of 3 pairs of Sofia policemen working the night shift, and combines masterfully black humor and dark, brutal events to portrait the harsh reallity that those living in that souther slav country face day after day. You can read more here.

  • Saturday, 2019/11/23, Synonyms, Nadav Lapid, Israel-Germany-France, 2018. An absolute piece of crap. There's nothing in this film that is worth wasting your time in front of it.

Sunday 28 November 2021

Yield based Trampolines

Some weeks ago I came across this article about using trampoline functions with generators to avoid stack overflows in recursive functions. I've recently written about "classic" trampolines for non tail recursions and about an alternative, using await, so I decided to transport the technique explained in that article to a javascript example.

Let's see first an example of adapting a Tail recursive function to be used with a trampoline function to prevent stack overflows in a runtime lacking of TCO (like node.js). We have a recursive factorial like this:


function recursiveFactorial(n, accumulator) {
    if (n <= 0) return accumulator;

    return recursiveFactorial(n-1, n*accumulator);
}

And we transform it into this:


function trampolineReadyFactorial(n, accumulator){
    if (n <= 0) return accumulator;

    return () => recursiveFactorial(n-1, n*accumulator);
}

To use it with a trampoline that we create with a factory:


function createTrampoline(fn){
    return (...args) => {
        let result = fn(...args);

        while (typeof result === "function")
            result = result();
        return result;
    };
}

let factorialTrampoline = createTrampoline(trampolineReadyFactorial);
console.log("factorialTrampoline: " + factorialTrampoline(50, 1));

Now let's do it differently, let's rewrite the "trampoline enabled" recursive function, using generators as explained in that python article.


//there's no recursion cause invoking the generator function returns a generator object, and the code
//inside the generator function is not executed until we call next() on the returned generator object (iterable/iterator)
function* yieldBasedFactorial(n, accumulator){
    if (n <= 0) 
        yield accumulator;
    else
        yield yieldBasedFactorial(n-1, n*accumulator);
}

So now the "trampoline enabled" function no longer returns a function (containing the "recursive" call), but a generator object (iterable-iterator). Calling next() on that generator object will perform the next "recursive" call). We can create a trampoline for this kind of "trampoline enabled" functions, like this:


function createTrampolineFromYieldBasedFn(yieldBasedFn){
    return (...args) => {
        let genObj = yieldBasedFn(...args);
        let item = genObj.next();
        while (item.value.next !== undefined )
            item = item.value.next();
        return item.value;
    }
}

let yieldBasedFactorialTrampoline = createTrampolineFromYieldBasedFn(yieldBasedFactorial);
let result = yieldBasedFactorialTrampoline(50, 1);
console.log("yield based Trampoline 2: " + result);

The trampoline function looks more complex to me in this case, but one could say that the adapted, recursive-like function, seems more close to the original with that yield. As usual, I've uploaded the code to a gist

Sunday 21 November 2021

Infinite Sequence and More

It's good that I still find interest in bizarre programming topics :-) I recently came across an article about recursive iterators in javascript. What mainly called my attention was the recursive way to create and endless iterator. The "classical", iterative way is something along these lines:


function* infiniteIterable(iter){
    while (true){
        yield* iter;
    }
} 

The recursive way (I managed to write it myself without looking it on the article, to make sure I understood it) is this:


function* infiniteIterableViaRecursion(iter){
    yield* iter;
    yield* infiniteIterableViaRecursion(iter);
} 

Somehow I also came across this other post with an odd title Eliminate try..catch when using async/await by chaining .catch() clauses. What!? Avoiding the "Promise Hell" of a mass of chained then-catch calls and writing it as sequential code is why we all love async-await, so this seemed senseless. Well, looking into the article he refers to a case for which it makes good sense, when the async call returns a value and even in case of an exception we still want to return a (sort of default) value. So rather than this:


let value;
try {
  value = await db.get('nonexistent'); // throws NotFoundError
} catch {
  value = null;
}

We write this, that is quite more concise:


const value = await db.get('nonexistent').catch(() => null);

Sunday 14 November 2021

LongChamp Marseille

Recently I've been lucky to spend (again) a few days in Marseille. I find this city mesmerizing, in spite of its massive problems: Le Grand Remplacement (it's not a theory, it's a painful reality in at least 1/2 of the city, where non-muslims have become a clear minority), the huge presence of far-left scum and the islamization and criminality that comes along with both previous issues... Marseille is placed in a privileged natural setting, encircled by mountains and the sea, and with a 150 meters high hill right at its center, from where the magical Notre-Dame de la Garde dominates the whole city. Additionally, I absolutely love its urban landscape, a dense city where the old and the new embrace each other and where high-rise constructions spring up unorderly here and there.

I thought I had already written about Marseille after my initial discovery of this impressive city, but I've just realised that unfortunately I never finished a rather long article that I was writing in August 2014!. All I've published about Marseille is this short post. It's strange that I did not mention in it the incredible place that I'm going to describe now, LongChamps.

LongChamp is a park, a garden and gorgeous palace Palais Longchamp, with a beautiful, imposing and delicate fountain (Chateau d'eau). It's the kind of classic French refinement that maybe you would not expect to find in Marseille, though the city has some other excellent examples of magnificent "so French" architecture, e.g. the Prefecture, the Banque de France and the Caisse d'Epargne buildings (the 3 of them really close). Indeed, my first time there, in 2014, I was almost shocked. After that, it was under renovation for like almost 2 years, and now it looks even better. This renovation included improvements to the lightning of the monument, which reminds me that I have pending to go there one day at night. The main sculpture and the cascade are one of those exquisite pieces of art that make you think about what their creators could feel the day it was complete, to what extend they were aware of the beauty they were gifting us with.

From the wikipedia article I've learnt that the achievement that it celebrates, the construction of the Canal de Marseille is still relevant these days, as it still provides 2/3's of its drinking water to Marseille. This is in contrast with another of the great French construction works of all times, the Canal du Midi, that remains an essential source of visual joy for most Toulousains, but no longer has much practical use.

I've come to profoundly hate French rap, save for some Patriotic acts like Tepa (Repose en Paix, combatant) or Millesime K. but well, there is a clip by Soprano with some nice views of LongChamp (you can just mute the sound).

Monday 1 November 2021

DbContext Creation

There are several ways to create a DBContext in (modern) .Net and they are explained here. There 2 of them that I normally use. This one, when I'm not using any IoC container:


var connectionstring = "Data Source=blogging.db";

var optionsBuilder = new DbContextOptionsBuilder<BloggingContext>();
optionsBuilder.UseSqlite(connectionstring);
optionsBuilder.UseLazyLoadingProxies();

BloggingContext dbContext = new BloggingContext(optionsBuilder.Options);

The above is pretty clear. We set several options through a builder object, build the options object and pass it over to the DBContext constructor, so that we get a DBContext fully set with all its options. This is much better than if we wereconstructing the object and then setting options on it, that would allow having a partially initialized (and useless) object.

This other mechanism is used when using the built-in IoC container provided by the .Net Base Class Library:


var connectionstring = "Data Source=blogging.db";
ServiceCollection services = new ServiceCollection();
services.AddDbContextFactory<BloggingContext>( optionsBuilder => {
	optionsBuilder.UseSqlite(connectionstring);
	optionsBuilder.UseLazyLoadingProxies();
});

var provider = services.BuildServiceProvider();

BloggingContext dbContext = provider.GetService<BloggingContext>();


Usually I just copy/paste it from some article or from a previous project and don't pay it much more attention, but it has always seemed a bit odd to me, so I've been reflecting a bit on it. We register in the IoC container our DbContext, providing a callback function that configures the DbContextOptionsBuilder and will be invoked when the Container instanciates the DbContext. This callback is not a factory, as it does not create the DbContext, it helps to configure it. This technique is possible becaue the built-in container has "extra knowledge" about DbContexts. I could think of 2 different ways to create the DbContext through the container that look a bit more natural to me.

1. Almost any IoC container allows registering parameters to pass to a constructor. For example in Autofac we can register constant values or register a factory function that is invoked by the container to create the non constant parameter each time it needs to provide the service. We would need that for our case with the DbContextOptions. I think the built-in .Net container does not support this.

2. Registering a Factory that creates a new DbContext each time the IoC container needs one. Indeed we can do that with the built-in container:


            var connectionstring = "Data Source=blogging.db";
            ServiceCollection services = new ServiceCollection();
            
            services.AddTransient<BloggingContext>(s => {
                var optionsBuilder = new DbContextOptionsBuilder<BloggingContext>();
                optionsBuilder.UseSqlite(connectionstring);
                optionsBuilder.UseLazyLoadingProxies();
                return new BloggingContext(optionsBuilder.Options);
            });
            var provider = services.BuildServiceProvider();
            return provider.GetService<BloggingContext>();

I think this seems more natural to me as it's a generic technique that I'm more accustomed to use with other containers and for different object types, but it's slightly more verbose than the specific technique featured by the built-in container.

Saturday 16 October 2021

Method References

I've been recently reading about method references in Java, particularly interested in whether invokedynamic is also used with them (as it happens with lambdas). The answer is yes and as you can read in that document the main difference is that for lambdas the Java compiler needs to "desugar" the lambda into a private method, which is obviously unnecessary for method references.

When the compiler encounters a lambda expression, it first lowers (desugars) the lambda body into a method whose argument list and return type match that of the lambda expression, possibly with some additional arguments (for values captured from the lexical scope, if any.) At the point at which the lambda expression would be captured, it generates an invokedynamic call site, which, when invoked, returns an instance of the functional interface to which the lambda is being converted. This call site is called the lambda factory for a given lambda. The dynamic arguments to the lambda factory are the values captured from the lexical scope. The bootstrap method of the lambda factory is a standardized method in the Java language runtime library, called the lambda metafactory. The static bootstrap arguments capture information known about the lambda at compile time (the functional interface to which it will be converted, a method handle for the desugared lambda body, information about whether the SAM type is serializable, etc.)

Method references are treated the same way as lambda expressions, except that most method references do not need to be desugared into a new method; we can simply load a constant method handle for the referenced method and pass that to the metafactory.

This other article gives a deeply interesting explanation of how lambdas and invokedynamic works.

Regarding method references (do not confuse them with the related, low level Method Handles), this article explains them pretty well. As I expected you can get a method reference to an instance method that gets its "this" value ("receiver object" in the article) bound to that instance, and you can get method references to static methods. The not so obvious thing is that you can get references to instance methods that are not bound to a "this" value ("References to unbound non-static methods" in the article). They'll use as "this/receiver object" the first parameter received when being invoked. That's pretty nice, as you can reuse the same method reference object with different receivers.

This has made me think about how delegates work in .Net, I could remember that we have something very similar, about which I wrote in one of my first posts. We call that Open Instance Delegates, but their creation is rather less friendly than in Java, so they remain as a not particularly well known feature. As a reminder:


MethodInfo mInf = typeof(Person).GetMethod("DoSomethingInstance");
  Func openDelegate = (Func)(Delegate.CreateDelegate(typeof(Func), mInf));
  //now let's call our Open Delegate with different instances of Person
  Console.WriteLine(openDelegate(p1,"formatting: ", 3));
  Console.WriteLine(openDelegate(new Person(){Name = "Xurde"},"formatting: ", 4));
  Console.WriteLine(openDelegate(new Person(){Name = "Nel"},"formatting: ", 2));


Another nice feature of Java Method references is that we can have references to constructors, using this syntax: ClassName::new. This is not possible in C#, I think because the CLR does not allow binding delegates to ConstructorInfo objects, only to MethodInfo objects. This is not a big deal, because we can just create a lambda that invokes the constructor: () => MyConstructor(), but anyway, it's nice that Java allows to directly reference the constructor.

In my old post I've seen that C# delegates provide also Close Static Delegates, that is a delegate pointing to a static method and where the delegate Target ("this") is bound to a value that will be passed as the first parameter to the static method. I still can not see much use for this, but well, just another feature.

Sunday 10 October 2021

Coroutine Sample

From time to time some Computer Science concept comes to my mind and keeps me busy reflecting about it. Recently I disserted about Promises vs Futures, now I've been thinking about what a coroutine really is. In principle for me a coroutine was a function that could return multiple times, getting suspended on each return and continuing from that point in the next invocation. You get this construct in languges like C#, JavaScript and Python when using await or yield.

Well, we can say that I was on the right track, but that the thing is a bit more complex. Reading here and there, with A Curious Course on Coroutines and Concurrency being a real eye opener, I find that there's a difference between generators and coroutines. Generators are functions that contain yield statements and use them to produce (generate) a sequence of results rather than a single result, to create iterables/iterators. A coroutine also contains yield statements, but uses them not (or not only) for producing values, but to consume values. C# lacks this feature, but in Python and JavaScript we can send values to a generator function (that becomes a coroutine then) by means of .send(value) or .next(value).

Long time ago I posted about how this could be used to simulate the async/await magic, but other than that (or the complex collaborative multitasking cases, using trampolines and so on), it's not easy for me to think of particularly interesting use cases for "simple" coroutines. The grep example in the above mentioned pdf could be rewritten with just a closure for example. Well, in the end I have come up with what I think is a good use case.

I'm going to invoke from "A" a method in ObjectB multiple times, passing different values, but these values that we have in "A" should be "enriched-updated" before sending, and this update follows a sequencial logic. I can put that logic in a coroutine, that I place between A and ObjectB. In my Example ObjectB is a User with a method haveMeal. "A" is the "feeding" logic that prepares the meals, but rather than sending them directly to ObjectB.haveMeal, it will send them to a coroutine that adds medication (morning, lunch or dinner ones) to that meal, and will then send that "enriched meal" to ObjectB.

The coroutine looks like this


import { User } from "./User.js";
import { Meal } from "./Meal.js";

let user = new User("Francoise");

//coroutine that consumes meals, adds medicines to them and sends them to a user
function* medicinesAdderFn(user){
    while (true){
        let medsGroups = [["morning pill"], ["afternoon pill"], ["dinner pill"]];
        for (let meds of medsGroups){
            let meal = yield null;
            meal.addMedicines(meds);
            user.haveMeal(meal);         
        }

        // let meal = yield null;
        // meal.addMedicines(["morning pill"])
        // user.haveMeal(meal);

        // meal = yield null;
        // meal.addMedicines(["afternoon pill"]);
        // user.haveMeal(meal);

        // meal = yield null;
        // meal.addMedicines(["dinner pill"])
        // user.haveMeal(meal);

    }
}

let meals = [
    new Meal("breakfast", ["croissant", "compota"], null, ["coffee", "water"]),
    new Meal("lunch", ["soup", "beans"], ["flan"], ["water"]),
    new Meal("dinner", ["pure", "cookies"], null, ["ekko", "water"]),
    new Meal("breakfast", ["pain au chocolat", "compota"], null, ["coffee", "water"]),
    new Meal("lunch", ["soup", "pasta"], ["riz au lait"], ["water"]),
    new Meal("dinner", ["omelet", "magdalenes"], null, ["ekko", "water"])
];

let medicinesAdder = medicinesAdderFn(user);


//initial call to "prime" the generator (runs the generator until executing the first yield)
medicinesAdder.next();
//normal calls to the generator providing values
for (let meal of meals){
    medicinesAdder.next(meal);    
}

And you can see the full sample here

Sunday 3 October 2021

Use await to avoid Stack Overflows

In my last post I mentioned that I should revisit my posts [1] and [2] about avoiding stack overflows. Well' I've done so and I've realised that while the technique that I describe there (using a timeout to exit the current call and then enter the next "recursion") makes sense in a pre-async/await world, but we can use a much more simple approach based on async-await. This technique that I'm going to show now is valid both for tail and not tail recursions

First let's see the code. I can transform these recursive functions (that in node, that lacks of Tail Call Optimizations, would crash when used with a large number


function factorial(n) {
	//return (n !== 0) ? n * factorial(n - 1) : 1;
	if(n === 0 || n === 1){
		return 1;
	}
	else{
		return n * factorial(n-1);
	}
}

function tailFactorial(n, total = 1) {
	if (n === 0) {
		return total;
	}    
	else{
		// proper tail call
		return tailFactorial(n - 1, n * total);
	}
}



Into these, that will not overflow the stack, cause basically we are no longer recursivelly stacking new frames in the stack, the await keyword is sort of avoiding the recursion



async function factorialWithAwait(n) {
	if(n === 0 || n === 1){
		return 1;
	}
	else{
		await "nothing";
		return (n * await factorialWithAwait(n-1));
	}
}


async function tailFactorialWithAwait(n, total = 1, resFn){
	if (n === 0){
		return total;
	}
	else{
		//this await prevents the stack overflow as we are exiting the function before the recursive call!
		await "nothing";
		//if we comment the await above, we'll get the stack overflows, cause this await here is AFTER the recursive call, so we continue to stack them up!
		return await tailFactorialWithAwait(n-1, n * total);
	}
}


This works thanks to how the eventloop and the microtask queue work in JavaScript.

To put it more simply, when a promise is ready, its .then/catch/finally handlers are put into the queue; they are not executed yet. When the JavaScript engine becomes free from the current code, it takes a task from the queue and executes it.

That means that adding an "await 'nothing'" in our code will cause that when JavaScript comes across this await for an already resolved value, it will add the ".then part" (that is what comes after the await) to the microtask queue, and will exit the current function (so releasing the current stack frame). Then it will take the task from the microtask queue and execute it.

As usual I've uploaded the code to a gist

Saturday 2 October 2021

From Recursion to Trampolines

I've recently come across one more piece of Groovy beauty. Closures (the objects used to represent a block of code), have multiple methods, the "usual suspects" (like memoize, curry...) and also so cool stuff like trampoline.

Trampolines are a technique used to adapt a tail recursive function into an iterative one, just by turning the recursive call into a function (usually called thunk) that is returned and will be used in a loop. Well, this article in JavaScript explains it pretty good.

All the articles I've seen use trampolines for Tail recursion (this is necessary as most languages do not implement Tail Call Optimizations), but what if we have a recursive function that peforms additional actions (post actions) after the recursive call? Last year I wrote 2 posts ([1] and [2]) about preventing stack overflows in recursive functions, both for tail and non tail recursion. The technique that I use there (and that probably I'll revisit in short) had nothing to do with trampolines, but the way I saved "post actions" for executing them after the recursive calls, helped me find a technique for using trampolines with not Tail recursion

In each call to the "recursive function adapted for trampoline iteration", apart from returning a function with the next "pseudo recursive call" I return a function with the post action, that I stack up and will execute after the recursion. Well, I better copy here the code:



let st = "helloworld";

//normal recursive (non tail recursion) function
function recursiveTransformString (source){
    if (source.length === 0)
        return "";
    if (source.length === 1)
        return source.toUpperCase() + source;

    
    let curItem = source[0];
    let transformed = recursiveTransformString(source.substring(1));
    return curItem.toUpperCase() + transformed + curItem;
}

console.log(recursiveTransformString(st));
//HELLOWORLDdlrowolleh

//the above function adapted to be used in a trampoline
function transformStringAdaptedForTrampoline (source){
    if (source.length === 0)
        return [""];
    if (source.length === 1)
        return [source.toUpperCase() + source];
    
    let curItem = source[0];
    let nextRecursion = () => transformStringAdaptedForTrampoline(source.substring(1));
    let postAction = (result) => curItem.toUpperCase() + result + curItem;
    return [undefined, nextRecursion, postAction];
}

//trampoline factory
function createTrampolineWithPostAction(fn){
    return (...args) => {
        let postActions = [];
        let [result, nextRecursion, postAction] = fn(...args);
        while (result === undefined){
            postActions.push(postAction);
            [result, nextRecursion, postAction] = nextRecursion();
        }

        while(postActions.length){
            result = postActions.pop()(result);
        }
        return result;
    }
}

let trampolined = createTrampolineWithPostAction(transformStringAdaptedForTrampoline);

console.log(trampolined(st));
//HELLOWORLDdlrowolleh



As usual I've uploaded the code to a gist

For further reading, this article is another good read about trampolines.

Sunday 26 September 2021

Linux Process Memory Consumption

Properly understanding the memory consumption of a process is quite an endeavor. For Windows I've given up. Over the years I've gone through different articles and discussions about the WorkingSet, Private Bytes and so on... and I've not managed to get a full understanding. In Linux things seem to make quite more sense.

I have several recent posts that could join this current one in a sort of "Understanding Memory series". This one about the whole Memory consumption in your system, this about the available physical memory, and this one where among other things I talk about the Virtual Address Space.

I'm going to talk now about the physical memory consumed by a specific process and about "shared things". With the top command we see 3 memory values: VIRT, RES and SHR. VIRT is the virtual address space, so as I explained in previous articles it has nothing to do with physical memory and normally it's not something we should care about. RES is the important one, it's the physical memory used by the process, and it's the same value that we see with the ps v pid command under the RSS column.

Well, things are not so simple. Processes share resources among them. I can think of 2 main ones: SO's (shared objects/shared libraries, that is, Linux equivalent to dll's) and the code segment between different instances of the same process. From what I can read here and here, the memory taken by a SO will be included in the RSS value of each process using that library (though it's taking, at least for most part of the SO, physical memory only once). The same goes for the code segment space, for each instance of the same process the RSS value will include the code segment space, though it's loaded only once in physical memory. This means that if you add up the RSS value of all the processes in your system it will give you a far bigger value than the real physical memory consumption that is provided by free -m.

RSS is the Resident Set Size and is used to show how much memory is allocated to that process and is in RAM. It does not include memory that is swapped out. It does include memory from shared libraries as long as the pages from those libraries are actually in memory. It does include all stack and heap memory.

Remember that code segment pages are shared among all of the currently running instances of the program. If 26 ksh processes are running, only one copy of any given page of the ksh executable program would be in memory, but the ps command would report that code segment size as part of the RSS of each instance of the ksh program.

When a process forks, both the parent and the child will show with the same RSS. However linux employs copy-on-write so that both processes are really using the same memory. Only when one of the processes modifies the memory will it actually be duplicated. This will cause the free number to be smaller than the top RSS sum.

There's an additional point to take into account, Shared Memory. Contrary to what you could think, when talking about "Shared Memory" in linux systems we are not talking about shared libraries, but about a different concept, memory reserved by shmget or mmap. Honestly I know almost nothing about it, I'm just telling what I've read here and here

"shared libraries" != "shared memory". shared memory is stuff like shmget or mmap. The wording around memory stuff is very tricky. Using the wrong word in the wrong place can totally screw up the meaning of a sentence

This Shared Memory is the SHR value on top's output. All processes sharing a block of shared memory will have it counted under their SHR value (but not under RSS).

The RSS value doesn't include shared memory. Because shared memory isn't owned by any one process, top doesn't include it in RSS.

Tuesday 14 September 2021

Truffle Interpreters

In my previos post about Graal VM I mentioned that maybe I would write an additional post about Truffle. This is how I summarized in that post what Truffle is supposed to be:

The Truffle Language Implementation Framework. It allows "easily" writing programming languages implementations as interpreters (written in Java) for self-modifying Abstract Syntax Trees (what?!). It has allowed the creation of very efficient Ruby, Python and JavaScript implementations to run in the GraalVM. As these implementations are written in Java you can run them on a standard JVM, but they'll be slow, as it's the GraalVM JIT compiler who allows these implementations to run efficiently, so that's why Truffle is so linked to GraalVM. This is a really good (and thick) read.

This idea of writing high performance interpreters seems odd to me. I know very little about compilers, interpreters and VM's, but it seems to go against the last decades of common Computer Science knowledge. Indeed, the Java guys spent much time and efforts to adapt the JVM, by means of the Da Vinci project and the new invokedynamic bytecode instruction, to strongly increase its performance with dynamic languages. After doing that, all of a sudden comes Truffle and you can write interpreters for languages that you compile to "self-modifying AST's" (not to Java bytecodes), and those implementations of JavaScript, Ruby... seem to be faster that the implementations that compiled those languages to Java bytecodes and that leveraged that recent and magical invokedynamic instruction! Indeed, Oracle has discontinued Nashorn (that was developed along with the invokedynamic-Da Vinci thing and that compiled JavaScript to Java bytecodes containing invokedynamic), and replaced it with Graal.js, that is based on Truffle.

Truffle is intended to run with the GraalVM Compiler, so OK, as this is an optimized JIT, your interpreter, written in Java and compiled to bytecodes, will end up compiled into a very optimized machine code thanks to all the optimizations that this cool JIT will do overtime, but anyway, we are optimizing an interpreter... and there can not be such a difference between the GraalVM Compiler and the Hotspot JIT to allow for this big change that Truffle seems to mean.

Well, I think I've finally understood what's the magic with Truffle. This article has been quite esential (and this other one has also helped). First we have that the AST that gets constructed for the program that you are interpreting will evolve (based on runtime profiling) from dynamically typed nodes into statically typed nodes. And then comes the real magic, the Partial Evaluation technique. Your interpreter code (the java bytecodes, not the code being interpreted) that was written to work with any types, will be especialized to work with the types being used in these executions. This is the Futamura projections, your interpreter gets specialized to run a specific source code!

So in the end your interpreter will ask Graal to compile to native code especialized versions of the different classes that make up the interpreter. From one of the articles:

Partial evaluation is a very interesting topic in theoretical computer science – see the Wikipedia article on Futamura projections for some pretty mind-bending ideas. But for our case, it means compiling specific instances of a class, as opposed to the entire class, with aggressive optimizations, mainly inlining, constant folding and dead code elimination.

All the above seems terribly complex (and magic) to me, and at some point I still wonder if I'm really understanding it correctly. The Truffle-Ruby documentation here has a couple of paragraphs that seem to confirm that my understanding is correct.

The Truffle framework gets the bytecode representation of all of the AST interpreter methods involved in running your Ruby method, combines them into something like a single Java method, optimizes them together, and emits a single machine code function.

TruffleRuby doesn’t use invokedynamic, as it doesn’t emit bytecode. However it does have an optimizing method dispatch mechanism that achieves a similar result.

Same as Ruby has now the "classic" JRuby (that continues to use as far as I know a strange mix of a Ruby interpreter written in Java and a runtime compiler to java bytecodes, that then will be either interpreted or JIT compiled by the JVM) and the new Truffle interpreter, it would be interesting to see if other languages plan a similar move. For the moment Groovy has no plans for that. After the performance improvements they reached when moving from callsite caching to indy (invokedynamic), they don't seem to see any reason to create a Groovy Truffle interpreter. Bear in mind also that as with all Java code, they are getting additional performance improvements "for free", just by running with the GraalVM JIT rather than the C2 Hotspot JIT.

Monday 6 September 2021

Groovy Types, AsType

It was such a long time ago that I expressed my profound admiration for Groovy. At that time I was impressed by its metaprogramming capabilites, recently I've been delighted by what a beautiful platform for DSL's it provides (Jenkins pipelines...), and lately it's its approach to Types that has caught my attention.

The dynamic vs static typing, and strong vs weak vs duck typing debate can be a bit confusing. I talked about it long in the past. Groovy's approach to type checking is pretty interesting, and this article is an excellent reference. In the last years it has gained support for static type checking (checks done at compile time) by means of the @TypeChecked and @CompileStatic annotation, but I'm not much interested on it. What's really appealing to me is that when working in the "dynamic mindset", its Optional typing approach allows us to move between dynamic typing and duck typing. When you declare types for variables or method/function parameters, Groovy will do that type check a runtime, but if you don't declare the type, it will be declared as Object, but no type checks will be applied, and the Groovy invokation magic (callsite caching in the past, invokedynamic in modern versions) will give you a duck typing behaviour. It will try to find a method with the given name, will try to invoke it, and if it works, it works :-)

All this thinking about type systems reminded me of my discovery of structural typing via TypeScrit time ago, and made me wonder if when using type declarations Groovy would support structural typing at runtime (I mean, rather than checking if an object has an specific type, checking if the "shapes" match, that is, the expected type and the real type has the same methods). The answer is No, (but a bit).

In other words, Groovy does not define structural typing. It is however possible to make an instance of an object implement an interface at runtime, using the as coercion operator.

You can see that there are two distinct objects: one is the source object, a DefaultGreeter instance, which does not implement the interface. The other is an instance of Greeter that delegates to the coerced object.

This thing of the "as operator" (AsType method) and the creation of a new object that delegates calls to the original object... sounds like a Proxy, n'est ce pas? So I've been investigating a bit more, and yes, Groovy dynamically creates a proxy class for you (and an instance of that proxy). Thanks to the proxy you'll go through the runtime type checks, and then it will try to invoke the requested method in the original object, and if it can not find it you'll get a runtime error.


import java.lang.reflect.*;

interface Formatter{
    String applySimpleFormat(String txt);
    String applyDoubleFormat(String txt);
}

class SimpleFormatter implements Formatter {
    private String wrap1;
    private String wrap2;
    
    SimpleFormatter(String w1, String w2){
        this.wrap1 = w1;
        this.wrap2 = w2;
    }

    String applySimpleFormat(String txt){
        return wrap1 + txt + wrap1;
    }

    String applyDoubleFormat(String txt){
        return wrap2 + wrap1 + txt + wrap1 + wrap2;
    }
}

class TextModifier{
    String applySimpleFormat(String txt){
        return "---" + txt + "---";
    }
}

def void formatAndPrint(String txt, Formatter formatter){
    System.out.println(formatter.applySimpleFormat(txt));
}

def formatter1 = new SimpleFormatter("+", "*");
formatAndPrint("bonjour", formatter1);

def modifier = new TextModifier();
try{
    formatAndPrint("bonjour", modifier); //exception
}
catch (ex){
    //runtime exception, when invoking the function it checks if modifier is an instance of Formatter
     println("- Exception 1: " + ex.getMessage());
}    

Formatter formatter2;
try{
    formatter2 = modifier;
}
catch (ex){
    //runtime exception, it tries to do a cast
// Cannot cast object 'TextModifier@1fdf1c5' with class 'TextModifier' to class 'Formatter'
    println("- Exception 2: " + ex.getMessage());
}   


formatter2 = modifier.asType(Formatter); //same as: modifier as Formatter;

//this works fine thanks to the Proxy created by asType
formatAndPrint("bonjour", formatter2); 

println "formatter2.class: " + formatter2.getClass().name;
//TextModifier1_groovyProxy

//but it is not a java.lang.reflect.Proxy
println ("isProxyClass: " + Proxy.isProxyClass(formatter2.class));


As you can see in the code above, Groovy creates a "TextModifier1_groovyProxy" class, but it does so through a mechanism other than java.lang.reflect.Proxy, as the Proxy.isProxyClass check returns false.

I'll leverage this post to mention another interesting, recent feature regarding types in a dynamic language. Python type hints and mypy. You can add type declarations to your python code (type hints), but they have no effect at runtime, nothing is checked by the interpreter, your code remains as dynamic and duck typed as always, the type hints work just as a sort of documentation. However, you can run a static type checker like mypy on your source code to simulate compile time static typing. This is in a sense similar to what you do with TypeScript, only that you are not transpiling from one language (TypeScript) to another (javascript), you are only doing the type checking part.

Sunday 29 August 2021

Proxying a Function, Callable Attempt and More

As of late I've been again delving into some unfrequent features found in some languages like Groovy and Phyton. One of those features is the "callable" concept in python. Any object is callable (that is, invokable with "()") if it has a __call__ method in its class (or any class up in its inheritance chain). It wrongly came to my mind that we could simulate this feature in javascript by means of creating a Proxy for a function with the apply trap.

I have to admit that I have never used proxies for functions. It's so simple to wrap a function with another function that I've never seen the reason to use the Proxy machinery. So it seemed cool to have found a use case (my main idea behind doing an object callable would be to express with a "myInstance();" call that I want to invoke the main, most important method in that object). Well, if I had fully read the documentation, I would have seen this: The target must be a callable itself. That is, it must be a function object.. So my approach was useless. The curious thing is that you won't get an error when creating the Proxy with the apply trap for a "non function", but when you try to invoke it.


class Formatter{
    constructor(){
        this.defaultWrapText = "||";
    }

    format(str){
        return str.toLowerCase();
    }

    wrapAndFormat(txt, wrapText){
        wrapText = this.defaultWrapText || wrapText;
        return wrapText + this.format(txt) + wrapText;
    }
}

let formatter1 = new Formatter();
console.log(formatter1.wrapAndFormat("HI"));

//let's try to do the formatter1 object a "callable" that would invoke "format". It's like a way to express a "main-default" method in the object 
let callableF1 = new Proxy(formatter1, {
    apply: function(target, thisArg, argumentsList) {
        console.log("invoking as callable");
        return target.format(...argumentsList);
      }
});

//does not work: "caught TypeError: callableF1 is not a function"
// from MDN: The target must be a callable itself. That is, it must be a function object. 
try{
    console.log(callableF1("HeY"));
}
catch(ex){
    console.log(`${ex.name}:\n${ex.message}`);
}

After this I found this article where a very smart guy explains how to do callable objects in JavaScript by inheriting from Function.

The use of a Proxy around the original object/function should be completely transparent to the final user, I already talked about it. For example if I get the name of a proxied function, the name of the target function will be returned. However, there is one case where this "Proxy transparency" seems to fail. If we do a "toString" on a proxied function, we don't get the source of the target function, but a "function () { [native code] }" string, quite odd. Let's see both cases:


function sayHi(){
	console.log("hi");
}

console.log(sayHi.toString());

//output:
/*
function sayHi(){
	console.log("hi");
}
*/

let f3 = new Proxy(sayHi, {
	apply(target, thisArg, argumentsList) {
        console.log("invoking sayHi");
        return target();
	}
});

f3();

//the Proxy is transparent, it returns the name of the target
console.log(f3.name);
//sayHi

//however, toString does not return the source of the target function!
console.log(f3.toString());
//output:
//function () { [native code] }


The behaviour of Function.prototype.bind when applied to a Proxy has quite surprised me. As it returns a new "bound" function, I was expecting that this new function would not be proxied, but it is. So it returns a new Proxy object around the new bound function!


function formatText(txt, str1, str2){
    return str1 + str2 + txt + str2 + str1;
}

let formatProxied = new Proxy(formatText, {
    apply(target, thisArg, argumentsList) {
        console.log("invoking formatText");
        return target(...argumentsList);
	}
});

console.log("- using proxy:");
console.log(formatProxied("Bonjour", "|", "+"));
console.log(formatProxied.name); //formatText

console.log("---------------");
console.log("- after binding");

let boundFormatProxied = formatProxied.bind(null, "Bonjour");
console.log(boundFormatProxied("*", "+"));
//it returns a new Proxy object that proxies the new bound function!!!

// I can see how the name now is "bound formatText"
console.log(boundFormatProxied.name);

console.log("---------------");

// output:
- using proxy:
invoking formatText
|+Bonjour+|
formatText
---------------
- after binding
invoking formatText
*+Bonjour+*
bound formatText
---------------


Notice in the example above that when we bind a function, the name of the returned function is: "bound 'original name'"

Sunday 22 August 2021

Torture-free Town

In the last years my perception of Spain has changed a lot. I still has a very strong Asturian and European identity (and that plays a role in my strong attachment to France, that at one point was the maximum expression of European civilization), but I'm not "in conflict" with Spain anymore. I no longer think Asturies should break away from Spain. After many centuries of living together with the other different cultures of the Peninsula I think there are enough common points to stay together (though for sure it's really important that the image that Spain projects outside stops being focused on Southern and Mediterranean Spain and finally starts to show the sheer diversity and plurality of the country). This said, there are things that the world associates with Spain that I profoundly distaste and continue to ashame me (me and a huge part of the population). Obviously bullfighting is one of them.

Modern "Spanish bullfighting" is rooted in southern Spain, but the disease was spread all over the country, as a form of trying to create a common tradition working as one of the basis of a fictitiously homogeneous country (as I've just said, Spain is a very heterogenous country due to its geography, climate, history...). This form of public torture made its way into northern Spain long ago. In Asturies, Uvieu used to have bullfights (corridas), but they declined in popularity and took place for the last time in 2007. After that the bullring (plaza de toros) went into a dilapidated state and though there are plans to restore it, it will be used only for cultural purposes (concerts...), and there's no intention to bring back torture to it. Unfortunately, for whatever the reason, bullfighting remained popular enough (I read years ago that it was maintained thanks to public fonds) in my town, Xixón, to continue perpetrating these acts of sadism for several days each year in August, during the "Semana Grande" (festivities week). In the last years the demonstrations and gatherings in front of the "Plaza de Toros" to protest against this heinous spectacle and demand its discontinuation have grown in strenght, but no signs from the city council made us think that they would come to an end. The previous, center-"conservative", major was indeed fond of this brutal "tradition", and the current "progressist" one had not given any sign of intending to change this (which indeed was rather disappointing).

So the announcement this week from the major that bullfighting was over has come as a beautiful surprise. While other Spanish minicipalities or autonomous communities officially banned bullfighting, the procedure in Xixón has been fair more simple. The license for organizing bullfights in town expires this year, so a new bid should be organized for the next years, and the city council has decided not to organize it. Simple and straightforward. Obviously a change of major could bring them back, but well, let's enjoy the present!

Obviosly local rednecks have condemned this and are trying to stop it, but I hope it will be to not avail. The local branch of the conservative party PP is collecting signatures against this measure, and the bastards of Vox (there are some stances of this party regarding immigration and criminality that I fully support, but their positions regarding Asturian language, Euthanasia, Animal Rights... make me feel a profund disdain for these idiots) promise to fight back...

It seems like what finally prompted the local council to take this decision was the fact that in this year's bullfights one of the bulls being tortured and murdered was named "feminista" and another one "nigeriano". Obviously this is disgusting, but animal rights and public honor should have been enough to make them take this step.

For all those saying that stopping bullfighting in our town will have negative economical consequences (in commerces, cafes...), well, for sure the abolishment of enslavement meant that some people stopped earning some money... It's funny cause one day after I said this in a family discussion, the local major made some similar remarks.

Same as in Uviéu, the intention is to reuse the bullring for cultural acts (concerts). Indeed I think it has already been used like that in the past. Honestly, I find the architecture of bullrings really ugly (it's not just that they smell to blood and cruelty, it's that the architecture has this sort of Arabic (neo-mudejar) trails that really disturb me), and also in order to prevent future majors to bring bullfighting back, I'd really love to see the building demolished.

You can find some information about this in English here

Sunday 15 August 2021

Kalifat

I've recently watched Kalifat/Caliphate, an extraordinary Swedish series. I can not express how good this series is, it kept me glued to the screen for its 8 episodes, and with the last ones I almost had anxiety to watch what happened next.

If you know nothing about radical Islam and how it brainwashes people and seeds an infinite hatred in our Western world, this series will wake you up to the enormous threat that we're facing. If you have some knowledge about this topic, this series will move you to the next level. If you know quite a lot about this topic (like me) you'll be amazed by how real this series is, how perfectly it shows the horrendous reality to which our societies are confronted, how carefully it shows our enemies, the ones that have travelled to Syria to rape women and kill infidels, the ones that here at home take advantage of our freedoms and welfare state to wage war against us, the ones that seemed to be normal people but indeed had inside that tribal hatred against us that is easily waken up by some smart liar, and finally, the idiotic "pure European" loser that in order to give sense to his pathetic life goes through an "express conversion" and turns into the most fanatical and hate fueled Islamist (as a decade ago he would have joined a neonazi gang...)

The story is captivating and the characters are excellent. The immigrant family of middle-eastern origin gives us such a good portrait of how people coming from a same culture can move forward or backwards. The father is a perfect example of integration/assimilation. He does not like joining another part of the family that have not integrated ("they continue to live as if they where there..." he says) and feels horrified when during a family gathering that part of the family expresses their joy for the Charlie Hebdo massacre... His eldest daughter is the perfect example of a piece of shit. Alredy before undergoing her Islamist tranformation she expresses her disdain for "the whites" (yes, those horrible Swedish "vikings" that have given her everything she has). His wife is also integrated, but she is more "Muslim friendly". When the daughter starts to show her first signs of radicalization (she decides to wear a Hijab and starts to talk about how oppressed Muslims are...) she's tolerant for her new dress-code, and avoids confronting her saying tha she knows nothing about politics (while the father fairly reacts very strongly against the hijab and her comments). Then, when the mother finds that her daughter has been watching jihadist videos she finally understand what is happening and is in shock. As for the younger sister, well, I'll just say that at the end of the film she becomes the icing of the cake. If we were lucky enough to have a second season of this masterpiece I think she should play a central role.

The "Swedish" couple that joined ISIS and live in Raqqa are also pretty interesting. They man is a weak, confused idiot with a low IQ, so the perfect prey to be drawn into hatred and jihad. His wife, Petri, is strong and smart and at the present time she does not look like a fanatical, so maybe (this is my guessing here) she was drawn into the ISIS madness slightly deluded, thinking about joining a perfect and pure Muslim society, but not thinking about behading people and supporting the rape and enslavement of infidel women.

As for the Swedish police woman, Fatima, the character is perfect. A Swedish woman of Bosniak descent (we learn that she's Bosniak because she wears a Sarajevo t-shirt and eats Balkan food when she is hidden in one family's friend flat), bearing a Muslim name but completely unrelated to that religion (she drinks beer, smokes some pot, enjoys sex not being married... ). She is stubborn, obsessive, does not necessary follow orders if they make no sense, and is willing to risk everything to fight the Islamist monster. Choosing these origins for the main character has been a beautiful choice from the director, that is a Swedish guy born in Sarajevo himself. Probably he is quite aware of how the Salafists are trying to take control of the once very moderate Bosnian Islam. I love how she does not seem to empathize at all with Petri. She's going to help Petri just in exchange for information, but she sees her as the only one to blame for her situation (as she says to her colleague "she's there because she decided to go there"), and during most of the story she's using and putting pressure on her without any sort of kindness. It's only at the end of the story when she seems to get some feelings for her, but this is indeed intermixed with the fact that after Petri has fullfilled her part of the agreement, now she has to do her own, whatever it costs.

I could spend hours talking about this gorgeous piece of cinematography, but you better enjoy it yourself.

Saturday 7 August 2021

GraalVM

I've been reading about GraalVM recently, and it's an interesting piece of technology. The thing is that I had to read several articles to get a clear understanding of what GraalVM is. At first sight you could think that it's a completely new Java VM, that's wrong. It's some specific pieces of the Java VM that are new, not the whole one (the Garbage Collectors and Heap Memory structure, Class Loader, ByteCode format... remain the same). GraalVM is a high-performance JDK distribution designed to accelerate the execution... When you download the Community GraalVM what you are getting basically is the HotSpot/OpenJDK with some extra (very important) pieces. Mainly

  • The GraalVM Compiler. This is the essential part, and the one that is important for 95% of people. The essential idea is that this ins not a new Java source code)compiler (like javac) but a new JIT compiler, that compiles Java Bytecodes to Native code.
  • The GraalVM Native Image. This technology compiles java applications ahead of time to native code. The resulting native applications do not run on a JVM (there's no need for an interpreter, JIT or I think even class loaders), but it needs some runtime components (GC, Thread scheduling...) that are called "Substrate VM" (though I don't think this can be considered as a VM).
  • The Truffle Language Implementation Framework. This is a beast on its own and maybe I'll write a separate post about it. It allows "easily" writing programming languages implementations as interpreters (written in Java) for self-modifying Abstract Syntax Trees (what?!). It has allowed the creation of very efficient Ruby, Python and JavaScript implementations to run in the GraalVM. As these implementations are written in Java you can run them on a standard JVM, but they'll be slow, as it's the GraalVM JIT compiler who allows these implementations to run efficiently, so that's why Truffle is so linked to GraalVM. This is a really good (and thick) read.

GraalVM Compiler
The Java HotSpot VM uses an hybrid model to run the bytecode that javac generates and puts in your .class files. It first runs your code with an interpreter, then uses a fast but not optimized JIT compiler, C1, to transform to native code those methods that are being used frequently (hot methods). Then those methods that are called very frequently (very hot methods) are JIT compiled by C2, an optimized but slower JIT compiler. You can read a good explanation here. This is awesome, but is not specific to the Hotspot JVM, as I explaine here and here. Even though the C2 JIT compiler seems particularly advanced with its use of Profile Guided Optimization, some smart folks thought they could replace it with an even (much) better beast, and that's what the GraalVM Compiler is, an speculative, profile-guided JIT Compiler to act as a replacement for C2.

The Hot Spot JVM features the JVMCI (Java Virtual Machine Compiler interface). Your code can use this interface to pass bytecodes to a JIT compiler, get them compiled and then install that native code into the VM. Through JVMCI you can plug a JIT compiler implementing that interface into the VM, that's how the GraalVM (JIT) Compiler is added to the VM replacing the C2 JIT.

The GraalVM Compiler is included in your normal OpenJDK (and Oracle JDK) distribution. To use it rather than the standard C2 JIT compiler, you have to start the VM with these options: -XX:+UnlockExperimentalVMOptions -XX:+EnableJVMCI -XX:+UseJVMCICompiler
The other, more common option, is installing the GraalVM JDK distribution, that comes with the GraalVM Compiler already switched on, along with the other related technologies that I've previously mentioned: Native Image, Truffle, Substrate...

The GraalVM Compiler is written in Java, so its bytecodes have to be converted to native code at some point, this is what is called bootstrapping. From what I've understood here and what I can envision on my own, I think there are 2 options, either compile it ahead of time with Native Image, or use C1 to initially compile it to not optimized native code, and then allow it to optimize itself as its methods become hot.

It's mentioned here that the GraalVM compiler uses an Intermediate Representation (IR). I was a bit confused about what this means with regards to standard Java bytecodes, and this article came to the rescue. Basically: Java ByteCode -> IR -> Machine Code

In short, the just-in-time compiler will convert Java bytecode into SSA IR. To be more precise, it is an IR graph containing control flow and data flow. Each bytecode corresponds to several nodes (note that some bytecodes do not have corresponding IR nodes). Then, the just-in-time compiler optimizes on the IR graph.

In all this post I've been saying that GraalVM Compiler is a replacement for the C2 JIT compiler, but I've found this article saying that it replaces both C1 and C2, I think that's wrong. In most places I've read, like here, it clearly says that it only replaces the C2.