Wednesday 25 December 2019

Tour Triangle

Warning: Please, notice that I consider France as my second country and I love it, so don't take the beginning of this post as an attack against France, I just criticise a common attitude in one part of the French population that, as someone that loves architecture, rather bothers me, and that for someone that has lived most of his life in a city (Xixón) where 15 stories buildings are just everywhere and people do not particularly complain about them, is pretty difficult to understand (notice that it goes far beyond the grand-ensemble trauma, and anyway, it's not HLM buildings that cause troubles, it's some of the people living there who do it.

The irrational hatred for High Rise Buildings could almost be considered as an essential part of the modern French identity. It's not that everyone hates them, indeed I guess part of the population does not care too much about them, but the percentage of people that vehemently hates any building higher than 50 meters and that is willing to create collectives, flood the media with lies about the "evils" of such buildings, go to court or whatever... in order to stop its construction is absolutely perplexing. Probably Marseille and Lyon are the 2 cities less affected by this redneck attitude, but unfortunately, Toulouse is right on the other side of the spectrum. This is the city where these collectives of "rednecks against the modern urban landscape" has an uncomprehensible power, furthermore, they even redefine the definition of high-rise, for these idiots which main interest in life is to hinder the development of the city, any building beyond 30 meters is a skyscrapper....

Things in Paris have been looking better in these last years. In 2018 Paris intra-muros saw the completion of the first sky-scrapper built in this century, the new TGI (notice of course that La Defense, that continues to thrive with interesting developments, is outside the limits of the "ville de Paris"), and the "Tours Duo" by Jean Nouvelle are well under construction (and there are plans for another skyscrapper in this same area). Furthermore, and this is what motivates this post, I've read recently that finally, the Tour Triangle has managed to overcome the last 2 (of many) appeals presented by some redneck association, and is now confirmed to become a new addition to the Parisian sky line. The images in in this article are just amazing, it looks massively beautiful!

Sunday 15 December 2019

IQueryable and Default Interface Methods

The discussion in the .Net world as to whether it's OK to have repositories returning IQueryable rather than IEnumerable has been rather present all these years. The common consensus is that it's not, and your repositories should not return an IQueryable. There are multiple reasons for this, but from an architecture point, probably this is one of the best I've read.

The point is that using IQueryable, you're asking for a query builder and not for a model. A query builder specifies how the query should be, how to get the data. Then why are we using a repository? Isn't the repo's job to know how to get the thing we want? We're using the repository because we want to specify what to get, not how to get it.

This goes along the idea of repositories having methods for "fat queries", so that the repository itself does the filtering, rather than returning a "collection" on which you apply filters. A similar approach is combining Repository and Specification pattern, your repository no longer have specific methods for each fat query, but you pass it over a specification with the query itself and it's the repository who performs the query.

While I agree with all the above, the idea of returning an IEnumerable and filtering it later, does not always seem like such a sin to me, after all we use Linq to Objects all the time. The big problem with returning an IEnumerable is that we would lose all the magic of IQueryable transforming our filters into a query and running it for us. The Linq methods for IEnumerable and IQueryable have been implemented as extension methods, and as we know this is a compiler trick, and method override and runtime resolution do not apply. If my IPersonRepository.GetItems returns and IEnumerable, my PersonSqlServerRepository.GetItems has to return an IEnumerable also in its signature (as C# lacks of Method return type covariance). We can respect that signature but return an IQueryable anyway, I mean:

IEnumerable GetItems()
{
 ...
 return dbContext.Persons; //returns and IQueryable
}

But the problem is that then, when doing myPersonsRepository.GetItems().Where(.....); the C# compiler would set a call to System.Linq.Enumerable.Where (rather than System.Linq.Queryable.Where), so we would be retrieving all the items from the Database, and then performing the filtering.

Now that C# 8 has added Default Interface Methods, if all the linq methods where added as default methods to IEnumerable and IQueryable the whole thing would change. As the method resolution would be done at runtime, calling Where in the above case would end up going to the IQueryable.Where, and generating the consequent sql query. This would be really sweet, but I've not heard so far about any plans of "copy-pasting" (you can not just move them due to backwards compatibility) the methods in System.Linq.Queryable into IQueryable as default interface methods.

Wednesday 4 December 2019

Promise.all with a Limit

A few months ago I wrote a post about how to limit the number of async operations that we run in parallel in JavaScript. Last week I came up with something related, but a bit different. What if we want to run n async operations and wait for all of them to complete, but we want to limit how many of them we run in parallel. A good example of this is if we want to do several http requests and combine their results, but we don't want to flood the server with all the requests at the same time (indeed browsers manage this themselves limiting how many requests to the same server they open in parallel, so we won't be overloading the server, but at JavaScript level the requests are already launched, and the timeout starts to count...). Waiting for all the async operations to complete is what we get with Promise.all, but it does not provide a limit option.

Well, the idea seems simple to implement, here it goes my take on it:

class PromiseAllWithLimit{
    //tasks: Array of functions that return a Promise
    constructor(tasks, maxItems){
        this.allTasks = tasks;
        this.maxItems = maxItems;
        this.taskResults = [];
        this.nextTaskId = 0;
        this.resolveAll = null; //points to the resolve function to invoke when all tasks are complete
        this.completedCounter = 0;
    }

    //tasks: Array of functions that return a Promise
    //returns a Promise that gets resolved when all tasks are done
    run(){
         while(this.nextTaskId < this.maxItems){
            this._runNextTask();
        }
        return new Promise((res) => this.resolveAll = res);
    }

    _runNextTask(){
        //we need this var to get it trapped in the closure to run in the task.then()
        let curTaskId = this.nextTaskId
        let nextTask = this.allTasks[this.nextTaskId];
        this.nextTaskId++;
        nextTask().then((result) => {
            this.completedCounter++;
            this.taskResults[curTaskId] = result;
            if (this.nextTaskId < this.allTasks.length){
                this._runNextTask();
            }
            else{
                //no more tasks to launch
                //if all tasks are complete, complete the All Promise
                if(this.completedCounter === this.allTasks.length){
                    this.resolveAll(this.taskResults);
                }
            }
        });
    }
}

That we can use like this:

function getRandomInt(max) {
    return Math.floor(Math.random() * Math.floor(max));
  }

  //returns Promise
function getPost(postId){
    console.log("getPost " + postId + " started");
    return new Promise(res => {
        setTimeout(() => {
            console.log("---> getPost " + postId + " finishing");
            res(`${postId} - content`)
        }, 2000 * (1 + getRandomInt(2)));
    });
}

(async () => {
    let tasks = [];
    for (let i=0; i<10; i++){
        tasks.push(() => getPost("postId_" + i));
    }
    let results = await new PromiseAllWithLimit(tasks, 5).run();
    console.log("all tasks are complete");
    console.log(results.join("\n"));
})();

There's an important point to notice. While Promise.all receives an array of promises, our PromiseAllWithLimit receives an array of functions that return a Promise. I'm calling these functions "tasks" in my code (that have little to do with .Net Tasks), let's think of them as "lazy-promises". I can not directly pass Promises, cause that would mean that we are already running all the functions, that is just what we want to limit. We return a Promise that will get resolved once all the "tasks" are complete (so we store its resolve "callback" for later invokation).

I've uploaded the code to this gist.

Sunday 24 November 2019

Tiered Compilation

.Net core 3.0 has come up with a huge surprise for me, Tiered Compilation (it exists since 2.1, but now it's enabled by default). Years ago I had written about how odd it seemed to me that contrary to Java HotSpot and to all modern JavaScript engines (that initially either interpret or JIT compile a method with a fast, lower-quality Jitter and then, if considered appropriate, will replace it, hot-swap, with a optimised Jitted version) .Net continued to use a single pass JIT compiler. Now things have changed.

Now .Net core comes with a fast, lower quality JIT compiler (tier-0) and with a slower, higher quality JIT compiler (tier-1). At runtime, methods are initially compiled to Native code with tier-0, and if they are executed more than 30 times, they'll get compiled again (by tier-1) and hot-swapped. This article provides some useful diagrams. Long in short, calls to compiled methods have now an additional level of indirection, where a count and jump to "initial, non-optimised native code" is done until the counter threshold is reached, then the "count and jump" is replace by a "jump to the new optimised native code". Several versions of the compiled native code get stored and over time could get hot-swapped again, but I have not found information about when this could be done.

Reading some other articles [1], [2] you'll find that .Net already had since the beginning 2 different quality JIT compilers. A faster one (equivalent to current tier-0) was used for debug builds, and an optimised one (equivalent to tier-1) for release builds. The thing is that you were stuck to one version or another, and no hot-swapping existed. There's also the question of why not using an interpreter rather than a not optimised JIT (and then the optimised JIT for frequently invoked methods), as done by Java Hot-Spot or some JavaScript engines. The main answer is that right now .Net does not have a good-enough interpreter (but it seems Mono has introduced a high-quality interpreter... so who knows if this could change in the future).

Finally, we have to take into account that .Net Core provides also Ready To Run images (R2R). This means that the compiler will compile your C# code (I guess this also exists for VB.Net and F#) to native code rather than CIL Bytecodes. This way when the application is run, the tier-0 compiler will not be used against that code, so start-up times are even faster, cause we already have the not-optimised native code, but the counting and recompiling if appropriate with the tier-1 JIT compiler still applies, so we can end up anyway with highly optimised code.

Sunday 17 November 2019

10 Years Post

It's now 10 years since I started this blog. If you read my first post, you'll see that I was not so sure I would manage to persist on it and make it one part of my life. I can say that I've succeeded. Posting here has become one part of my life, I've turned it into an obligation, sometimes I don't feel much like writing, but I force myself to do it, I force myself to continue, to focus on something. I've written some posts in some really difficult moments of my life, and when looking back it amazes me that certain month of certain year I were able to put certain things aside and focus on writing some paragraphs unrelated to that situation and that maybe no more than 10 people would ever read. It's nice when someone still manages to surprise himself in a positive way.

10 years have passed, and obviously many things have occurred. For someone moving from thirty-something to forty-something I guess I have not gone through some of the expected processes: finally finding a "life-partner", having children... Well, for an Asturian these processes are not so evident, this is probably the place on Earth with the lowest fertility rate, and I think the percentage of people my age that have not found (and many of us have never cared at all) a person to share life with is probably really, really high. It's not a surprise at all, 10 years ago I had no interest on finding "that person", and hopefully that interest has not emerged over these years (I say hopefully cause I'm quite sure this egocentric bastard would not manage to keep a steady relationship). As I said, I've gone through some bad moments, some of them pretty bad, but it has helped me to better appreciate the good moments, and better understand what makes up those moments: family, friends, places...

Not having taken those expected steps does not mean that things have been static, on the contrary, I've changed so much in these years. My political positions have turned like 160 degrees, my main interests, the things that I appreciate the most... all this has changed a real lot. Something that in 2009 was hardly an idea became a necessity, almost an obsession (in 2012-13), to live abroad for a long time. I finally made it and ended up in one country that had never appealed to me that much, France, and it has become my second identity, my second motherland... Several years there quenched my thirst for change, and for several reasons I realised that it was time to return to my primary home, that now I appreciate much more than before. I keep my strong links with France, and one could say that in a sense I've changed from being a "citizen of the world" to being a "citizen of two countries and one continent".
My musical interests have not remained static either, but after many evolutions and having broadened my taste pretty much, now I mainly favour the same styles I loved 20 years ago.
As for work, from an external perspective little has changed, but my internal attitude is now so different. I no longer dream of "doing something significant", I still enjoy learning and coding (well, sometimes at least), but regarding work all I aspire to is to get easy assignments and reduce stress.

Many people feel sad (even terrified) about getting old. I think it was the same for me in the past, but not now. It's not because I don't appreciate this life, just the contrary. As time goes by we are all aware that the end is closer, but not everyone is aware of how having reached this point in the path is an achievement. People die at any age, under any situation, so being able to look back and see how the traversed path has got longer should relieve us and make us feel fortunate, make us give thanks to whoever-whatever we think has helped us reach this point. Already quite a long time ago I said this sentence "Cada día vivido es una victoria frente a la muerte / Each day we live is a victory against death". Some friends thought of it as a very depressive conclusion, about an obsession with the idea of death stalking us I guess... but it's not like that really, or at least only partially. For me it's light coming out of darkness... We are our memories, and the more memories the more we are. The last pages of Serotonin by Houllebecq (a crazy, amazing book) quite shocked me (I read it in French, which is one of my most unexpected achievements of this decade). The main character, sank in a depression that is literally killing him, prints pictures of his life and wallpaper his room, what is intended to be his last room, with them. They say that when you're going to die you see all your life pass at high speed, this was like a low speed version that provided pleasures that the present no longer hosted. The more things you have to fill that sequence, I think the more sense you can say your life has had.

This has been a very personal post, and if you have read it to this point, most likely you are someone that is a part of my life. If that's the case, I want to say Thanks, for having been there all this time and for remaining there in the future. I plan to continue to write boring paragraphs here, and I hope we'll be lucky to share a "20 years post" in 10 years time. I also hope we'll be sharing passages of our lives face to face, in front of a coffee (or a beer for many of you) more frequently than we do. You know that I'm a socially lazy person, but I appreciate more and more those shared moments. On se voit!

Saturday 16 November 2019

Iterating multiple times

There's something in the behaviour of IEnumerables-IEnumerators created by an Iterator method in C# that is quite confusing. Let's start by the behaviour of JavaScript generators, that is the expected one for me, and let's see then the oddities of the C# version.

In JavaScript, a generator function returns a generator object, an object that is both an iterable and an iterator. So when asking it for an iterator (via [Symbol.iterator]) it returns itself. Because of this, if we try to iterate multiple times our iterable, the iteration will only happen the first time. During this first iteration the iterable-iterator object reaches its end, so next iterations will do nothing.

function* getCountries(){
 yield "France";
 yield "Belgium";
 yield "Portugal";
}

let countriesGenOb = getcountries();

let countriesIterator1 = countriesGenOb[Symbol.iterator]();
console.log(countriesGenOb === countriesIterator1 ? "same object" : "different object"); //same object


console.log("- first iteration:");
for (let country of countriesGenOb){
 console.log(city);
}

//France
//Belgium
//Portugal

console.log("-------------");

console.log("- second iteration:");
//no iteration is done, countriesGenOb[Symbol.iterator] is returning the same object
//that was already iterated to the end in the previous loop
//very interesting, this behaviour is different from C#, here the generator object (this is both iterable and iterator) is returning itself, rather than a copy
for (let country of countriesGenOb){
 console.log(city);
}

//Nothing gets printed

let countriesIterator2 = countriesGenOb[Symbol.iterator]();
let countriesIterator3 = countriesGenOb[Symbol.iterator]();
console.log(countriesGenOb === countriesIterator2 ? "Same reference" : "Different reference"); 
//same reference
console.log(countriesGenOb === countriesIterator3 ? "Same reference" : "Different reference"); 
//same reference

The behaviour in C# is different and surprising. An Iterator method returns an object that implements both IEnumerable and IEnumerator, but we can iterate it multiple times. This is like that because it seems that when we call GetEnumerator on and IEnumerable-IEnumerator that has already been enumerated, it does not return a reference to the same object, but to a new object implementing also IEnumerable and IEnumerator.

  var countries = GetCountries();
  Console.WriteLine(countries is IEnumerable); 
  //True
  Console.WriteLine(countries is IEnumerator); 
  //True

  Console.WriteLine(countries == countries.GetEnumerator() ? "Same reference" : "Different reference");
  //Same reference

  //the Iterator method is returning an IEnumerable/IEnumerator object, but the thing is that calling to GetEnumerator returns a new instance, rather than the object itself
  //Because of that the 2 loops do a whole iteration, and enumerator1 and 2 are different objects.
  Console.WriteLine("- first iteration:");
  foreach(string country in countries)
   Console.WriteLine(country);
  
  //France
  //Belgium
  //Portugal
  
  Console.WriteLine("-----------------");
  Console.WriteLine(countries == countries.GetEnumerator() ? "Same reference" : "Different reference");
  //Different reference
  
  Console.WriteLine("- second iteration:");
  foreach(string country in countries)
   Console.WriteLine(country);
  
  //France
  //Belgium
  //Portugal

  Console.WriteLine("-----------------");

  var enumerator1 = countries.GetEnumerator();

  Console.WriteLine((enumerator1 == countries) ? "Same reference" : "Different reference"); 
  //Different reference
  

  var enumerator2 = countries.GetEnumerator();
  Console.WriteLine((enumerator1 == enumerator2) ? "Same reference" : "Different reference"); 
  //Different reference


<-- CSharp and JavaScript current sources F:\Main\MyWebCreatures\deploytonenyures\SourceCode\GeneratorsIterators_IterateMultipleTimes -->

Sunday 10 November 2019

Deferred-Lazy Iterable

When in my last post I talked about how to provide lodash-like functionality to Async Iterables, I did not realise that indeed lodash (or underscore for the matter) do not work with normal Iterables. The Array methods work with arrays and the Collection methods work with Objects. Well, Iterables are objects, but if you check the source code for example for map you'll see that it checks if the object is an array (in which case it traverses it with a while-length loop) or just an Object, in which case it traverses it using Object.keys... so it won't work for non-Array iterables. The solution is converting your iterable to an Array, which in modern JavaScript is so clean as doing [...myIterable],

var _ = require('lodash');

function* citiesGeneratorFn(){
 let cities = ["Xixon", "Toulouse", "Lisbon", "Tours"];
 for (city of cities){
  yield city;
 }
}

let cities = citiesGeneratorFn();

//lodash methods do not work with iterables
console.log("with iterable");
console.log(_.map(cities, city => city.toUpperCase()).join(";"));
//nothing gets printed

//we have to convert the iterable to an array
console.log("with array");
console.log(_.map([...cities], city => city.toUpperCase()).join(";"));
//XIXON;TOULOUSE;LISBON;TOURS

Problem with this is that you are losing one of the main features of iterables, that they are deferred. First, if my generator function*: citiesGeneratorFn above were doing something like retrieving the city data via a server call, I would not need to do that retrieval until the moment when I were doing a next call to get that city, but when converting to array, we are doing all those calls in advance, even if then I'm only going to use the first city. Second, as lodash does not support iterables, all the map, filter... functions are applied to all the elements of the array, even if again we are only going to use the first one. Because of this is why a library like lazy.js (of which I talked recently) exists.

So, it came to my mind how would I go in order to try to implement something like that, a map, filter... functions that worked nicely with iterables and hence deferred. Thanks to generators it's pretty straightforward. I create an ExtendedIterable class that will wrap the original iterable. This class features map and filter methods. The important thing is that each of these methods returns a new ExtendedIterable (so I can chain the calls) that applies the function logic to the value to return, one by one, taking advantage again of generators. All in all, we have this:

class ExtendedIterable{
    constructor(iterable){
        this.iterable = iterable;
    }

    *[Symbol.iterator](){
        for (let it of this.iterable){
            yield it;
        }
    }

    map(fn){
        //I need to use the old "self" trick, cause we can not declare a generator with an arrow function
        //so as we are using a normal function, "this" is dynamic, so we have to trap it via "self"
        let self = this;
        function* _map(fn){
            for (let it of self.iterable){
                yield fn(it);
            }
        };
        let newIterable = _map(fn);
        return new ExtendedIterable(newIterable);
    }

    filter(fn){
        let self = this;
        function* _filter(fn){
            for (let it of self.iterable){
                if (fn(it))
                    yield it;
            }
        };
        let newIterable = _filter(fn);
        return new ExtendedIterable(newIterable);
    }
}

function* citiesGeneratorFn(){
 let cities = ["Xixon", "Toulouse", "Lisbon", "Tours"];
 for (city of cities){
  yield city;
 }
}
let cities = citiesGeneratorFn();

//I could just use the array, and array is iterable and the citiesGenerator is not doing any processing, just retrieving the value directly
//let cities = ["Xixon", "Toulouse", "Lisbon", "Tours"];

let extendedIterable = new ExtendedIterable(cities);

let result = extendedIterable.map(it => it.toUpperCase())
    .filter(it => it.startsWith("T"));

for (let it of result){
    console.log(it);
} 

//or just convert to array and join it
//console.log([...result].join(";"));

I've uploaded the code to a gist. This is just a POC, if you want something real, go for lazy.js.

Thursday 7 November 2019

Async Iterables

I've been playing around lately with Async Iterables (the JavaScript equivalent to C# Async Streams), and I've thought I should share it here for future reference. Same as an object is Iterable if it has a [Symbol.Iterator] method, it is asynchronously iterable if it features a [Symbol.asyncIterator] method. Both method return an object (the iterator) with a next method, that in the first case returns an object {done: bool, value: any} and in the second case a Promise of {done: bool, value: any}. Notice that in both cases the method is called next while in C# we have IEnumerator.MoveNext but IAsyncEnumerator.MoveNextAsync.

I've written some basic code where I define an Iterable class implementing the [Symbol.asyncIterator] method in 3 ways. The first and the second create the iterator "manually" (the only difference is that in one case I use Promise.then and in the other the magic of async-await. In the third case I leverage generators.

function getPost(postId){
    return new Promise((res) => {
        setTimeout(() => {
            if (postId != 4)
                res("post: " + postId + " content");
            else
                res(null);
        }, 500);
    });
}

//in all cases the iterator returned by asyncIterator returns a Promise<{done, value}>
class InfoBox1{
    [Symbol.asyncIterator]() {
        return {
            postId: 0,
            next(){ 
                return getPost(this.postId++)
     .then(post => post === null ? {done: true}: {done: false, value: post});
            }
        };
    }
}

class InfoBox2{
    [Symbol.asyncIterator]() {
        return {
            postId: 0,
            async next(){ 
                let post = await getPost(this.postId++);
                return post === null ? {done: true}
                    : {done: false, value: post};
            }
        };
    }
}


class InfoBox4{
    //"*" cause it contains "yield" statements
    //"async" cause it contains "await" statements
    async *[Symbol.asyncIterator]() {
       let postId = 0;
        while (true){
            let post = await getPost(postId++);
            if (post !== null)
                yield post;
            else 
                return;
        }
    }
}


async function asyncMain(){
    let infoBox = new InfoBox1();
    for await (let post of infoBox){
        console.log(post);
    }
    
    console.log("--------------");
    
    infoBox = new InfoBox2();
    for await (let post of infoBox){
        console.log(post);
    }

    console.log("--------------");

    infoBox = new InfoBox4();
    for await (let post of infoBox){
        console.log(post);
    }
}

asyncMain();

It's important to notice how we can declare async next(), and async *[Symbol.asyncIterator]() but bear in mind that while we can declare an arrow function async, we can not use arrow functions as generators.

It would be pretty convenient to be able to use map, filter... and similar functions with Async Iterables. There are some attempts on this, like wu.js. Well, I've come up with a different approach. We could transform the Async Iterable into an Observable, and then leverage the many rxjs operators. Transforming the Async Iterable into an Observable is as simple as this:

function asyncIterableToObservable(asyncIterable){
    return new Observable(async (observer) => {
        for await (let item of asyncIterable){
            observer.next(item);
        }
        observer.complete();
    })
}

And now you can leverage rxjs operators:

async function* citiesGeneratorFunc() {
    let cities = ["Xixon", "Toulouse", "Lisbon", "Tours"];
    for(let city of cities){
        yield await new Promise(res => {
            setTimeout(()=>res(city.toUpperCase()), 600);
        });
    }
}

let citiesGeneratorObj = citiesGeneratorFunc();
let citiesObservable = asyncIterableToObservable(citiesGeneratorObj);
citiesObservable.pipe(
 filter(it => it.startsWith("T")),
 map(it => "[" + it + "]"))
).subscribe(it => console.log(it));

//[Toulouse]
//[Tours]

Friday 1 November 2019

Lazy Split

Those working with C# should be rather familiar with the idea of performing lazy operations on collections. With linq to collections, when you chain operations (provided as extension methods by System.Linq.Enumerable) on an IEnumerable, these operations are not executed until the first time you iterate the resulting Enumerable (this is called deferred execution). Furthermore, whenever possible, the operations are performed one by one, as we retrieve each element during the iteration. For example, for the Select (equivalent to Map in other languages) or Where (equivalent to filter in other languages) methods, when we retrieve the first element, we will execute the delegate provided to Select for only one element, and the delegate provided to Where only until we obtain the first element fullfilling the condition. This is called lazy evaluation. However, a method like Sort needs to execute its delegate for all elements before retrieving the first item, this is called eager evaluation. All this is much better explained in this post.

The other day I came across lazy.js that brings this concept of deferred execution of operations on collections to the JavaScript world. Basically they have implemented the methods provided by underscore-lodash in a "lazy manner". They are all deferred, and I guess whenever possible the evaluation will be lazy rather than eager. Lazy.js is not only about collections, it also includes lazy funcionality for strings, like for example a lazy split. Honestly I'd never thought about something like this, but it makes much sense for huge strings.

Out of curiosity, I decided to implement my own lazy split, which is pretty straightforward thanks to generators:

function* lazySplit(source, separator){
 let curPos = 0;
 curPart = "";
 while (curPos < source.length){
  curPart += source[curPos];
  if ((curPart).endsWith(separator)){
   yield curPart.substring(0, curPart.length - separator.length);
   curPart = "";
  }
  curPos++;
 }
 yield curPart;
}

function doTest(str){
 console.log("- splitting: " + str);
 let generatorObj = lazySplit(str, ";");
 for (let it of generatorObj){
  console.log(it);
 }
 console.log("--------------\n");
}

doTest("a;bc;de");
doTest("a;bc;");
doTest("a;;de");
doTest(";;");

.Net lacks a lazy split in the standard library, so other people have implemented it.

Saturday 19 October 2019

Hot vs Cold Observables

Having read some of the multiple articles on the Hot vs Cold observables thing like this, I used to think that I have a good understanding of it, but the other day I came across a question in StackOverflow that quite confused me. I did my own test to verify that it really worked like that.

import { Observable, Subject, ReplaySubject, from, of, range } from 'rxjs';
import { map, filter, switchMap, flatMap, tap, mapTo} from 'rxjs/operators';

const subject1 = new Subject();

let ob1 = subject1.pipe(
    tap(it => console.log("tap: " + it)),
    map (it => {
  console.log("map: " + it);
  return it * 2;
 })
);

ob1.subscribe(it => console.log(it + " received in subscription 1"));
ob1.subscribe(it => console.log(it + " received in subscription 2"));

subject1.next(20);

We get this output:

tap: 20
map: 20
40 received in subscription 1
tap: 20
map: 20
40 received in subscription 2

I would have expected the ouput to be like this:

tap: 20
map: 20
40 received in subscription 1
40 received in subscription 2

I'd never checked how pipe and operators are implemented, so I was thinking that when we pipe several operators a sort of chain is created, so that when a subscribe is run, the different observables are created and linked, but this chain of observables would be created only once, regardless of how many subscription we were doing, so we would have:

subject -> tap Observable -> map Observable -> subscriber 1
                                           |
                                           |-> subscriber 2

Reading this article about creating your custom operators has helped me to understand that the above chain is created each time we call subscribe, so what we really get is this:

subject -> tap Observable -> map Observable -> subscriber 1
       |
       |-> tap Observable2 -> map Observable2 -> subscriber 2

Sunday 13 October 2019

IEqualityComparer and GetHashCode

I've always found this a bit confusing, but it has not been until this week that I decided to try to understand the reasons behind it. Some methods in Linq to Objects (System.Linq.Enumerable) take an IEqualityComparer as parameter, for example Distinct and Contains. Both methods have an overload that requires no EqualityComparer. In that case the default EqualityComparer for the class is used, that means that if the class does not implement System.IEquatable and it's a reference type, we'll end up using reference equality. So using Distinct or Contains when we are concerned about references pointing to the same object is pretty straight forward.

When we want to compare based on some property in the object, (the Id, the Name...) it's obvious that we need to provide the comparison "mechanism". In javascript for functions like find, findIndex, includes... we pass a function expecting two values and returning true or false. That's what in principle I would be expecting that should be needed in .Net, a delegate, but no, we are forced to provide an IEqualityComparer, that has an Equals method and also a GetHashCode method. The reason for this is that the comparison logic in Distinct and Contains not only uses the Equals method (in that case it could have been designed to just receive a delegate), but also the HashCode. It's well explained here. Hash functions are not perfect, they have collisions (the hash of 2 different values can be the same), but we need to make sure that for 2 objects, if IEqualityComparer.Equals is true, the GetHashCode for both objects is also the same (but the reverse does not need to be true). With this, and based on some explanations I've read (I've taken a fast look into the source code, but got a bit lost) we can think of the Distinct method being implemented sort of like this:

Each unique element found so far is stored in a HashTable, where the key is the EqualityComparer.GetHashCode. Now, when checking if another element is unique, rather than comparing it with all items from the start of the collection, for the part of the collection going from the start to this current element we can just check against the hash table (much faster than checking element by element), if we don't find it there, we'll have to continue comparing item by item against the remaining part of the collection.

So this GetHashCode is clearly a way to speed up those methods where the collection is going to be iterated multiple times (like Distinct). However, for methods like Contains, not sure how it comes into play (probably it compares hashCodes first before resorting to Equals?). The odd thing is that methods similar to Contains, like First or Any, do not use an IEqualityComparer, but just a delegate...

Saturday 5 October 2019

C# Asynchronous (pull) Streams

C# 8 is finally out, with two main, long awaited features: Default Interface Methods and Asynchronous (Pull) Streams (Asynchronous Enumeration). Regarding the latter, already 3 years ago I had posted about how useful it would be to have something like that, and I think it will be good to do a recap (I also posted about types of streams early this year).

At that time I had said that in some cases we could deal with Asynchronous Enumeration by returning an IEnumerable<Task<T>>. This technique is only valid when knowing if we can get a new element or not (so MoveNext returns true or false) is synchronous, and the asynchronous part is obtaining the value Task<T> that will be set in Current. An example:

 private static Task<string> GetBlogAsync(int id)
 {
  return Task.Delay(500).ContinueWith(task => "Blog " + id.ToString());
 }


 private static IEnumerable<Task<string>> GetBlogs()
 {
  for (var i=0; i<5; i++)
  {
   yield return GetBlogAsync(i);
  }
 }
 

 static async Task Main(string[] args)
 {
  foreach (var blogTask in GetBlogs())
  {
   Console.WriteLine(blogTask.Result);
  }
 }

The above code can be written in C# 8 (thanks to the introduction of IAsyncEnumerable and IAsyncEnumerator) like this:

        private static async IAsyncEnumerable<string> GetBlogs2()
        {
            for (var i=0; i<5; i++)
            {
                yield return await GetBlogAsync(i);
            }
        }
  
  static async Task Main(string[] args)
        {
            await foreach (var blog in GetBlogs2())
            {
                Console.WriteLine(blog);
            }
        }
    }

For this case, where we can know if we have reached the end of the iteration (i==5) without trying to get the element, both options are equally valid, but we have to understand that there's a difference in what we are doing. The difference between IEnumerable<Task<T>> and IAsyncEnumerable<T> is that for the former, MoveNext() returns a bool, and Current returns a Task<T>. For the latter, MoveNextAsync() returns a Task<bool> and Current returns T. Thanks to this, IAsyncEnumerable works fine both when the iteration end condition is known synchronously or asynchronously.

Modern javascript also includes this feature, known as Asynchronous Iterators, and this article makes a really good read about it. By the way, I've always found it rather confusing that the Iterator protocol uses Symbol.iterator rather than Symbol.getIterator for the name of the function that defines an object as iterable and returns the iterator. It is a function, so it should have a verb name, not a noun...

Sunday 29 September 2019

Functional Programming

I have to admit that I've never been into Functional Programming. I mean, I've used and loved for quite long some techniques related to Functional Programming, like: map, filter, forEach, closures, partial functions... but I've always used this as part of my OOP programming, I've never got into some other functional techniques and all the functional hype.

Last year I wrote this crappy post where as you can see I still could hardly see any particular use to function currying. Well, that has changed in the last days thanks to this article about currying and this article about lodash-fp.

The main element about currying that I had been missing is that a curried function can be invoked not just passing one argument at each call, but several arguments in one call (this is called Variadic Currying). I mean:

function myFunc(a, b, c){....}

//given a curried version of myFunc we can do:
myCurriedFunc(a)(b)(c);

//but also:
myCurriedFunc(a, b)(c);
myCurriedFunc(a)(b, c);
myCurriedFunc(a, b, c);

Not taking into account Variadic Currying, one could see no particular difference between Partial Function Application and Currying.

var _ = require('lodash');

function wrapTwice(wrapper, txt){
    return `${wrapper.repeat(2)} ${txt} ${wrapper.repeat(2)}`;
}

const wrapWithXPartial = _.partial(wrapTwice, "X");
const wrapWithXCurry = _.curry(wrapTwice)("X");

console.log(wrapWithXPartial("hi"));
//XX hi XX

console.log(wrapWithXCurry("hi"));
//XX hi XX

But Variadic Currying allows us to write this:

const curriedWrapTwice = _.curry(wrapTwice);
console.log(curriedWrapTwice("X", "hi"));
console.log(curriedWrapTwice("X")("hi"));

In the first invocation we are invoking the curried function just in the same was as we would invoke the original function. So we could just curry all our functions beforehand and then invoke them normally (with all the parameters) or in the curried style (one or x parameters at each time). Now you can wonder, "OK, and what for?". To answer this, you just have to read the second aforementioned article, and play around with lodash/fp. You'll also learn a piece of jargon "point-free programming" to throw around from time to time during an office coffee break.

By the way, the introductory paragraph:
The lodash/fp module promotes a more functional programming (FP) friendly style by exporting an instance of lodash with its methods wrapped to produce immutable auto-curried iteratee-first data-last methods.
could let you speechless (I've sent it to one friend to have fun with his reaction :-)... so a basic explanation:

  • auto-curried: What I mentioned above, the lodash/fp functions (map, filter...) have already been curried.
  • iteratee-first data-last: The parameters order with respect to their lodash counterparts has been inverted. We have lodash.map(collection, callback), and lodash/fp.map(callback, collection)

Friday 27 September 2019

Composing Functions

The other day somehow I ended reading this post and found an amazing piece of code:

const pipe = (...fns) => a => fns.reduce((b, f) => f(b), a);

Composing functions, that is, creating a function that chains several function calls passing the result of each function to the next... is nothing new, but the way this guy implements it using reduce quite caught my eye. Also, those 2 consecutive arrows puzzled me a bit. We have a function returning another function, but as both are written as arrows, the code seemed a bit strange to me at first... .

After this I was thinking about ways to compose functions provided by some commont javascript libraries. In lodash we have 2 options, _.chain and _.flow.

_.chain is not really a way to compose, but an alternative. When we want to compose functions provided by lodash, we can wrap our array/collection in a chainable object, and then chain method calls to that object. It's explained here.

let cities = [
    "Paris",
    null,
    "Toulouse",
    "Xixon",
    "",
    "Berlin"
];

result = _.chain(cities)
    .compact()
    .take(3)
    .reverse()
    .value();
//result = [ 'Xixon', 'Toulouse', 'Paris' ]

Rather than _.chain we can use _.flow, like this:

result = _.flow([
    _.compact,
    (ar) => _.take(ar, 3),
    _.reverse
])(cities);

Notice that with _.flow we are creating the composed function, that here we are invoking immediatelly, but that we could reuse for multiple calls. _.chain is just a one go approach. The other big difference is that _.chain is only valid when using lodash functions, while with _.flow we could be composing any sort of function.

The documentation of _.flow says something that at first read confused me a bit: "Creates a function that returns the result of invoking the given functions with the this binding of the created function". Well, when composing non-method functions most times they are not using "this" (as in the example above), so we don't care about it. But this is important when composing methods of a same object. Let's say I have this class:

class Formatter{
    constructor(repetitions){
        this.repetitions = repetitions;
    }

    format1(txt){
        return `${"[" .repeat(this.repetitions)} ${txt} ${"]" .repeat(this.repetitions)}`;
    }

    format2(txt){
        return `${"(" .repeat(this.repetitions)} ${txt} ${")" .repeat(this.repetitions)}`;
    }
}

If we compose the calls to format1 and format2 like this we will obviously be missing "this" in the method, and the result will be wrong:

let formatter = new Formatter(2);

let formatFn = _.flow([
    formatter.format1,
    formatter.format2
]);

console.log(formatFn("hi"));
// hi

With what we've read about "the binding of the created function", all we need to fix it is calling function.prototype.bind:

let boundFormatFn = formatFn.bind(formatter);
console.log(boundFormatFn("hi"));
// (( [[ hi ]] ))

If lodash were not providing this feature, we could just create additional functions to wrap each method call, or directly invoke bind on each method-function

let formatFn2 = _.flow([
    (st) => formatter.format1(st),
    (st) => formatter.format2(st)
]);
console.log(formatFn2("hi"));
//(( [[ hi ]] ))
console.log("------------------");

let formatFn3 = _.flow([
    formatter.format1.bind(formatter),
    formatter.format2.bind(formatter)
]);
console.log(formatFn3("hi"));
//(( [[ hi ]] ))

I've uploaded all the code to this gist

By the way, if you want to read some javascript programming way more interesting than this post, you can check this article about currying that I found recently.

Sunday 22 September 2019

Getting Used To

Humans have this amazing ability to adapt to our changing world. This is an essential evolutionary trait, and mainly, those who fail to adapt, fail to survive. But sometimes this capacity to adapt is not that good, sometimes adapting to changes is not the right option, the right option is fighting back those changes and reverting them.

In the last year France has shown me several examples of this kind of negative adaptation:

During the Gillet Jaunes crisis (that 'movement' that at the start was dubious, eclectic, incoherent and had none of my sympathy, and that soon became one more delirium of French far-left fascism, ultra-violence and reject for anyone that does not belong to their fucking sect) many shops (not just banks, also insurances, shops that would be worth to loot...) had to cover their windows with wood plaques to try to avoid the attacks. Initially this was done just for the day when the violent gatherings were due, Saturdays, but over time, as this yellow sickness was becoming chronic, businesses had no other choice that keeping these protections permanently to reduce costs. Overtime some of these wood plaques would get painted to try to make them attractive. There was a sense of normalization, as if having to cover windows shops from the attacks of some far-left parasites (and the hordes of your average muslim-banlieu-antiFrance-criminal (racaille) that joined them to enjoy the violence, insult, harass and attack normal citizens, burn stuff, and loot shops) was just something unavoidable, as if the state should not use as much force as necessary to immediately stop this madness.

Months ago I read a report (not sure if in Le Monde or Liberation, for the most part I no longer read those leftist pamphlets other than if redirected there from some other site making fun of their islamo-collaborationist articles), that quite shocked me. It explained the 'astuces' (tricks) that a group of young girls living in 'banlieu parisien' (the outskirts, somewhere outside the peripherique - 20 arrondissements) had to use when they wanted to go to Paris by public transport to enjoy a night out. As it was not a good idea to wear a nice, a bit revealing maybe, dress in a wagon that could be infected by undesirables (do I need to explain to what "culture" do 90% of them belong to...?), the girls would go dressed normally with their night clothes in a bag to get changed later on in the flat of one friend living in Paris proper. The crazy thing is that this article was not denouncing this situation, this lack of freedom, as something unacceptable, but just talking about it as a normal, almost cool, situation. This thing of the girls getting changed in one friend flat is not new, but I associate it with the dark times of Franco's dictatorship in Spain, a mechanism to avoid the anger of conservative society, but now, in XXI century France (well, I can easily imagine the same situation in Brussels...) this has become normal.

If you watch this documentary about Saint Denis, that city in the Paris Metropolitan area that hosts the basilica where many French kings are buried and that has now become a Muslim enclave (well, good part of the Seine Saint Denis department, le 93, is a Muslim enclave...), you'll find many shockingly disturbing things, but there's something that particularly caught my attention. Obviously this city is way much cheaper than Paris, and there are some rather beautiful hausmannian buildings (I can say the same for Montreuil, one city in the same department (province) where I've spent some time), so some Parisians are willing to move there to enjoy a much better property. We follow the real state agent walking a potential client along the city's main street, and the conversation is interesting. The client notes that them 'French people of European descent' are a minority in the place, but seems no particularly concerned. Then the agent gives her the 3 main advices for living in the city: never pull out your smartphone on the street, always hold your bag tightly and on the front side, and when driving always make sure your doors are well closed and never put your bag on a seat. Again, the client does not seem particularly surprised, and we learn she will end up buying a property and moving to St Denis. From this to having to live in closed, protected communities like in South Africa, Mexico... there are just a few steps... If you think I'm going too far... there's another fact revealed by the documentary that absolutely goes in this line. In the last decade many companies have set offices in Saint Denis, leveraging the low prices in an extensive area of industrial ruins that has been turned into offices. Apart from the huge security measures adopted to control the access to the premises, many companies have had to put in place a system of mini-buses (navettes) so that those employees going to work by public transport can be driven safely from the Metro station to the Offices and back.

Saturday 21 September 2019

Amerika Square

I think Amerika Square is the second Greek film that I've watched in my life (I won't consider Costas Gravas films as Greek, he's French-Greek and all his studies and career has gone by in France). Amerika Square tells a tragic, very present story, with a dark sense of humor. Two friends in their 40's deal in a totally opposite way with the tragic consequences of the refugee crisis in their neighborhood in Athenas.

While the film takes a leftist stance, the way it portraits the nationalistic-xenophobic character is not totally absurd or sectarian, on the contrary, it makes you understand (understanding is not the same as agreeing) why he has ended up thinking like that. It shows the drama of those that have lost their land and references by being forced to flee their home, but also the drama of those that because of the economical crisis and the changes to their neighborhood brought about by the massive arrival of "the others", no longer feel at home. The element in the psyche of this character that is senseless to me is that he seems to profess the same level of hate for all immigrants, regardless of whether they are Polish Christians, Peaceful Asian guys, or Muslims. I think it's quite unlikely to find an attitude like that save in the most extreme far-right bastards.

A very human film that makes you understand the desperation on both sides. This understanding stems from the fact that on both sides the film shows us the good ones. The main immigrants, save for the Russian maphia, are nice people that just want a chance for a better life (they are not Islamist scum or social aid parasites). The same goes for the 'xenophobe', he's not an educated motherfucker with some crazy theory of race and ancestry, but a poor guy that can not find a job, suffers seeing the degradation of his neighborhood and longs for the peaceful, cohesive society of his childhood.

The other Greek film that I'm aware of having watched, Plato's Academy also had immigration, economical crisis and xenophobia as its main topic. I watched that film in FIC Xixon 2010, so we were in the middle of the huge crisis caused by the economy crash of 2008, but quite before the migrant crisis, and a bit before the rise of Golden Dawn, the nazi scum party. Already at that time ordinary Greeks were fucked by their poor economy and some of them were angry to 'the others', the Albanians mainly.

Saturday 31 August 2019

Promise Reject vs Throw

Promises are designed to be resolved or rejected (by the code ran by the promise), but occasionally I've seen code that throws an exception rather than rejecting, so is there any difference? Well, sometimes yes and sometimes no. Let's say I have an "old school" callback based asynchronous function

function getPost(url, successFn){
 let posts = {
  "blog.fr/aaa": "AAA post"
 };
 //simulate async operation
 setTimeout(()=> successFn(posts[url]), 1000);
}

Now I have a function that makes use of it, adding some validation logic and "modernizing" it by returning a Promise.

function validateAndGetPostAsync(url){
 return new Promise((res, rej) => {
  if (!url)
   return rej("empty url");
  
  getPost(url, (post) => {
   if (!post)
    return rej("post not found");
   res(post);
  });
 });
}

It's rejecting the promise if the initial validation fails or if the returned post is null. Using it with await and a try-catch block works nicely:

let urls = ["blog.fr/aaa", null, "blog.fr/aaa"];
 for (let url of urls){
  try{
   console.log(`url: ${url} ${await validateAndGetPostAsync(url)}`);
  }
  catch (ex){
   console.log(`exception: ${ex}`);
  }
 }
 console.log("after retrieving blogs");
 
 //url: blog.fr/aaa - AAA post
 //exception: empty url
 //exception: post not found

Now, if rather than rejecting we use a throw:

function validateAndGetPostAsync2(url){
 return new Promise((res, rej) => {
  if (!url)
   throw "empty url";
  
  getPost(url, (post) => {
   if (!post)
    throw "post not found";
   res(post);
  });
 });
}

and run the same example as before:

 for (let url of urls){
  try{
   console.log(`url: ${url} - ${await validateAndGetPostAsync2(url)}`);
  }
  catch (ex){
   console.log(`exception: ${ex}`);
  }
 }
 console.log("after retrieving blogs");
  
 //Output:
 // url: blog.fr/aaa - AAA post
 // exception: empty url

 // D:\Main\MyPrograms\programs_JavaScript\promises\throwVsReject.js:31
         // throw "post not found";
         // ^
 //post not found

We can see that while the throwing in the initial validation is being caught, the throw in the validation after obtaining the post is not caught (node prints the exception and crashes, the "after retrieving blogs" line is never printed). So what's happening? Well, it's easy, validateAndGetPostAsync is calling getPost, that is an "old school" asynchronous function, it invokes a callback once it gets the post, rather than returning a Promise. This asynchronous callback runs in a different stack from the one that launched the code, so a try-catch block surrounding the callback won't catch the Exception. When invoking reject rather than throwing, this is not a problem. Why? Well, the await-try-catch is just magic that the compiler translates to a .then().catch() chain (more or less). A call to reject() obviously rejects the promise and is captured by the Promise.prototype.catch(). However, when doing a throw, the Promise constructor wraps in a try-catch block the call to the provided function, and that catch block will reject the promise. If the code runs outside that try (because it's running asynchronously in another stack) it won't be captured and no call to the reject callback will happen.

Sunday 25 August 2019

13 Minutes

13 Minutes is a really important film, cause it tells a really important story, that of Georg Elser the man that was about to kill Hitler in November 1939, little after the start of WWII. It's hard to believe that this event is so little known. I'm not by far as educated as I'd like to be, but you know that I'm pretty much into history and social/political stuff, I've been tons of times in Germany... and I'd never heard about this. On the other hand, I've watched documentaries and one film about the assassination attempt of July 1944. This attempt would have moved forward several months the end of the war, but it pales down in importance when compared to the one of 1939 that could have just stopped it from the beginning.

The story of Georg Elser is that of a real hero, an incredible example of humanism. A humble, incredibly smart (the nazi officials could not believe that he had managed to build that bomb) left wing christian (though close to the German Communist Party he was not affiliated) that fed up with what was happening before his eyes decided to take matters into his own hands and work alone for one year in a thorough plan to bomb out the Adolf motherfucker. Hitler left the venue 13 minutes before the bomb exploded, so those 13 minutes would cause suffering and destruction as humanity had never experienced before. I'm not going to tell you more, you just have to watch the film and search some information about this hero.

The German people is deeply aware of its recent history, of its failure as a society that allowed the Nazi monstrosity to deceive it, poison it and almost destroy it (well, with the rise of AFD this awareness seems to be starting to fade, and I don't say this for whatever the opinions of AFD regarding immigration, but because of their minimization or even approval of multiple aspects of the nazi regime). German society has lived with this feeling of guilt and responsibility for what happened, and you can still feel it today in facts like that the German left (contrary to the pro-Arab obsession of most of the European left) has never been particularly supportive of the Palestinians (unfortunately, this has started to change in the last years, caused by the massive increase of Muslims in the country, that as in the rest of Europe infiltrate any association, political group, whatever... with the only objective to push their own communitarian agenda). I'm waiting for the day when Muslims will infiltrate LGTB organizations to ask for homosexuality to be considered again a sickness and demand funds to get healed... OK, back to the point, so why has this hero been kept relatively hidden? Is it maybe to avoid any lessening in this feeling of guilt? Maybe knowing that there were individuals inside that society that fought against its destruction would reduce the idea of a whole society gone crazy and would reduce the sense of guilt? I don't think so, in that case Germans that saved Jews would also be kept under the carpet...

Maybe the problem lies in the fact that the bombing killed 8 people. One of them was a waitress, I don't know if she was a nazi sympathizer or a beautiful human being, the other 7 were part of the audience, so they were fucking nazis. The dead of an allegedly innocent person as collateral damage of an action that would have saved millions of lives should not undermine the value of such action, so I don't see this as an explanation... So honestly, I have no idea... Reading the Wikipedia entry I can see that he has received some recognition in the last decades, but anyway it seems too poor to me...

Watching this film reminded me of this really interesting German film about Sophie Scholl and the White Rose resistence movement. Another must see that shows us how one (small) part of the German people fought to their last breath against the Nazi madness.

Wednesday 14 August 2019

Privatize Method

Somehow some days ago I ended up performing one more of those useless programming experiments that still manage to keep me entertained. It's important to understand that the private accessors provided by languages like C# and Java are a compile time mechanism to enforce restrictions on the code that we write, but they should not be confused with a real security mechanism. They can be circumvented at runtime with Reflection, and that's not a flaw in the system, it's that they are not intended to be really "secret". Years ago I wrote about this and presented a trick where Reflection itself was used (by examining the call stack) to prevent this the use of Reflection for invoking private methods. Similar techniques can be used in Java.

As I explain in that post, you can get privacy in JavaScript by means of defining your public methods as closures that trap the private members. This indeed gives you a level of privacy way superior to the one provided by the private accessor in other languages. Anyway, few people found it necessary to mess their code with that technique. The last versions of JavaScript have remained reluctant to providing us with private accessors, but as the TypeScript compiler comes with them and almost everyone is using TypeScript... in a sense it's as if they were already a part of JavaScript...

Notice that getting by TypeScript privacy is even easier than using reflection, all you need is casting to any:

(myInstance as any).privateMethod();

For whatever the reason it came to my mind the idea of making a JavaScript method private at runtime. You have defined the method public and while the application is running you decide that it should go private. The closures trick has to be used when defining the methods in the class, so that's not an option. What we can do is replacing the existing method by a new one that checks the call stack to see if it's being called from another method in the same class and depending on it invokes the original method or throws an exception. Getting the caller of one function in JavaScript used to be quite straightforward, by means of function.caller, but it's not standard and is not available in strict mode. The other option, arguments.caller, is plain obsolete.

There's a pretty inventive solution (of course I did not come up with it, I just googled a bit). When you create an Error object, its stack property gets filled with a string with the call stack information, something like this:

 Error
    at calledFromInsideClass (D:\Main\MyPrograms\programs_JavaScript\PrivatizeMethod\PrivatizeMethod.js:51:23)
    at City.privatizedMethod (D:\Main\MyPrograms\programs_JavaScript\PrivatizeMethod\PrivatizeMethod.js:60:13)
    at City.growCity (D:\Main\MyPrograms\programs_JavaScript\PrivatizeMethod\PrivatizeMethod.js:30:14)
    at Object. (D:\Main\MyPrograms\programs_JavaScript\PrivatizeMethod\PrivatizeMethod.js:71:6)
    at Module._compile (internal/modules/cjs/loader.js:689:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
    at Module.load (internal/modules/cjs/loader.js:599:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
    at Function.Module._load (internal/modules/cjs/loader.js:530:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)

This has quite surprised me. In C#, the stack trace of an Exception only gets populated if you throw the exception, not just by creating the Exception object. The thing is that you can examine that string to see if you're being called from inside or outside the class. There are some articles about parsing this string [1] and [2]. For my proof of concept I've just used some basic, error-prone code based on some of the code I've seen there.

So I have created a privatizeMethod function that receives a "class" and the name of a method, and replaces that method (in the class.prototype) by a new method that when invoked checks (through the calledFromInsideClass function) if it's being invoked by a method in the class. The stack at that point is: calledFromInsideClass <- myPrivatizedMethod <- caller method, so we are checking the third entry in the stack trace.

function calledFromInsideClass(className:string):boolean{
    let stackTrace = (new Error()).stack; // Only tested in latest FF and Chrome
    console.log("stackTrace: " + stackTrace);
    stackTrace = (stackTrace as string).replace(/^Error\s+/, ''); // Sanitize Chrome
    let callerLine = stackTrace.split("\n")[2]; // 1st item is this function, 2nd item is the privatizedMethod, 3rd item is the caller
    return callerLine.includes(className + ".");
}


function privatizeMethod(classFn: any, methodName: string){
    let originalMethod = classFn.prototype[methodName];
    let privatizedMethod = function(){
        if (calledFromInsideClass(classFn.name)){
            console.log("privatized method called from inside");
            return originalMethod.call(this, ...arguments);
        }
        else{
            throw new Error(`privatized method ${originalMethod.name} called`);
        }
    };
    classFn.prototype[methodName] = privatizedMethod;
}

//privatize the the City.growPopulation method
privatizeMethod(City, City.prototype.growPopulation.name);

I've uploaded a full example (in typescript) here.

Wednesday 7 August 2019

Throttling Async Calls

The TPL and Parallel.ForEach make it easy to run multiple instances of a same operation in parallel, taking advantage of the number of processing units (processor cores) in your machine. This is great for CPU bound operations, but what about async I/O bound operations (async http calls for example)?. If the operation returns a Task, it makes no sense to start several threads and keep them idle waiting for the asynchronous result. We should use a single thread to launch each operation one after another (without awaiting for the Task to be completed) and let a continuation to be invoked in Thread Pool thread once the result is available (or maybe push the Tasks in a collection and do a WaitAll). Something like this (consider it pseudocode)


var postTasks = List<Task<string>>();
foreach (var url in urls)
{
 contents.Add(this.getPost(url));
}
var posts = postTasks.WaitAll();

That looks good, but in many occasions launching a huge number of async operations in parallel is not a good idea. Let's say we are sending those http requests to the same server, we don't want to overload it (or make it think that we're doing a DOS attack!). Thinking about this, I found this discussion in SO explaining the options to throttle these async operations. The usage of SemaphoreSlim is pretty interesting.

A questions comes to mind, how to deal with this same problem in that different universe called JavaScript, where awaitables are called Promises rather than Tasks and where the runtime is single-threaded? I've been playing a bit with this, and here it goes what I've came up with.

I've created a simple TicketManager class that is like a sort of SemaphoreSlim. It has a number of available tickets (as many as parallel async calls we want to do). Before invoking the async function we'll request a ticket, receiving a Promise that will be resolved as soon as an execution slot is available for us, and after completing the async function we'll release it. If there are no tickets available we'll await for one (getTicket returns a Promise).

class TicketProvider{
    constructor(maxActiveTickets){
        this.activeTickets = 0;
        this.maxActiveTickets = maxActiveTickets;
        this.waitingTickets = []; //array of resolve function, to get invoked once we have a free slot, so the code waiting for the ticket can move on
    }

    //returns Promise
    getTicket(){
        if (this.activeTickets {
                //save the resolve function so that we can invoke it when another ticket gets released
    this.waitingTickets.push(res);
            });
        }
    }

    releaseTicket(){
        this.activeTickets--;
        let nextTicketToResolve = this.waitingTickets.shift();
        if (nextTicketToResolve) {
            nextTicketToResolve();
        }
    }
}

So when a ticket is requested, if we still have some slot available we return a resolved promise, so the client will immediatelly continue. If not, we return a Promise. This Promise will get resolved when a slot gets realeased. Promises can not be resolved from outside (contrary to .Net, where a Task can be resolved by an associated TaskCompletionSource), it's from "inside" the Promise that we can resolve it, by means of the function received as parameter in the Promise constructor. So we have to create a Promise that checks at intervals if a slot has been released, and in that case resolves itself store that resolve function in the TicketManager.

Given an async function like this:

function getPost(postId:number):Promise{
    console.log("getPost " + postId + " started");
    return new Promise(res => {
        setTimeout(() => {
            console.log("---> getPost " + postId + " finishing");
            res(`${postId} - content`)
        }, 2000 * (1 + getRandomInt(2)));
    });
}

We can leverage this TicketManager like this (notice that I've wrapped in an additional function the call to the async call, in order to wait for it and then call to releaseTicket) in order to run no more than 3 post requests at the same time.

async function getPostAndReleaseTicket(postId:number, ticketProvider:TicketProvider):Promise{
    let post = await getPost(postId);
    ticketProvider.releaseTicket();
    return post;
}

async function runWithThrottling1(){
    let ticketProvider = new TicketProvider(3);
    let curPos = 0;
    let requests = [];
    while (curPos < 10){
        let ticket = await ticketProvider.getTicket();
        requests.push(getPostAndReleaseTicket(curPos, ticketProvider));
        curPos++;
    }
    await Promise.all(requests);
    console.log("all posts retrieved");
}

Or like this (now the wrapper funtion takes care of both getting the ticket and releasing it)

async function getTicketAndGetPostAndReleaseTicket(postId:number, ticketProvider:TicketProvider):Promise{
    await ticketProvider.getTicket();
    let post = await getPost(postId);
    ticketProvider.releaseTicket();
    return post;
}

async function runWithThrottling2(){
    let ticketProvider = new TicketProvider(3);
    let curPos = 0;
    let requests = [];
    while (curPos < 10){
        requests.push(getTicketAndGetPostAndReleaseTicket(curPos, ticketProvider));
        curPos++;
    }
    await Promise.all(requests);
    console.log("all posts retrieved");
}

I've uploaded the code to a gist

Sunday 4 August 2019

The Factory

I've recently quite enjoyed with a certain surprise an excellent Russian film, The Factory. Well, it's done with the support of Canal+ and somehow Armenia has also something to do with it, so in the end it's a Russian-French-Armenian production.

If you follow this blog I guess you'll know that I feel a profound distaste for Russia. An ultraconservative society, where ultra-nationalism, racism and xenophobia are the guiding rules for a good part of the population. A population that continues to rise to power bastards like Poutin, such an enemy of European values... I consider that an appropriate environment is essential to spur creativity, so with this in mind I'm not much inclined to expect anything interesting coming out of Russian soil (other than beautiful women...). But this film is an interesting exception.

The film depicts the kind of Russia that we expect. A desolate country, where most of the population live in dark and ugly soviet buildings that look as obsolete as the factories where they strive to get an income to manage to pay for their basic needs and keep the routine going. Where oligarchs rule supreme managing the lives of others as disposable pieces in their game to become richer and richer. One of these factories is one the few remaining economic activities of an anonymous, neglected city. One day, the oligarch owning the factory announces that he's closing it as it's no longer profitable. The instinctive reaction of the workers, threatening with a strike, makes the millionaire laugh. One of the workers, that wears external and internal scars from the war (Chechnya I guess) comes up with a different solution, kidnapping the oligarch, asking for a huge ransom and fleeing away. He convinces 5 other fellow workers to join him.

The kidnapping starts OK, but things will soon get twisted, and this ex-soldier will be back in a war commanding his unexperienced comrades. I won't tell you more, just try to find this film and watch it. There is action for sure, but this is more about personal drama, betrayal, redemption, hidden intentions, revenge... and all set in a dark, tragic atmosphere.

As an Asturian I know quite well what the closing of a factory (or a shipyard, or a coal mine...) means for a territory (when the government of that territory has no capacity to find a viable replacement economical activity).When the Spanish Government decided to close (or to do nothing to prevent it) a huge part of our productive fabric (starting in the late 70's and still ongoing now in the shape of a stupid "ecological transition" law) Asturian workers started their responses with the classic approach, strikes. Then measures scaled up into hunger strikes, locking themselves in the coal mines, walking hundreds of kilometers to Madrid to protest before the government... Things would get tougher, first blocking highways with burning tyres and then spiraling up into some levels of violence rarely seen in the western world (and supported or approved by a large percentage of the population, indeed, many of us have interiorised this kind of violent resistence as an essential part of the Asturian identity, perceiving it as a continuity in our history), using authentic urban war tactics against the armed forces (Naval Xixón, "valles mineros"...). This Russian approach of kidnapping was never tried, quite a shame!

Saturday 13 July 2019

Threads Mashup

After writing this post I was thinking a bit about how we could do to cancel a chain of async-await calls in JavaScript. Standard promises are not cancellable (unless that you implement your own mechanism, as they do in Bluebird), but I'm thinking more about at least skipping the ensuing await calls. This post gives some good hints on how to proceed. This usage of a Cancellation Token sent me back to the C#-Tasks world, where Cancellation Tokens are the mechanism used for cancelling tasks. Tasks in .Net are used for 2 different things, IO async operations (where the Task does not involve a new Thread, same as Promises in JavaScript), and CPU intensive operations where you do a Task.Run to run the code in a separate thread and leverage your CPU cores. The rest of this article deals with the Threading and Tasks involving threads" world, not with the IO async operations represented with Promises/Tasks universe.

Cancellation Tokens are a collaborative mechanism. Someone tells the code running in a Thread created via Task.Run or Task.Factory.Start "please stop" and that code is occasionally checking for that kind of requests (through the Cancellation Token) and stops itself. You could use something similar for Threads started via the old Thread.Start mechanism. Use some property, closure variable, whatever shared between the caller and the callee, and have the callee checking for it and quitting if requested.

Threads started with Thread.Start provide you with a more radical cancellation mechanism, Thread.Abort. There's no collaboration here, someone tells the Runtime to stop the thread, and the runtime will do so if certain circumstances are met. Canceling a Thread without its permission can be risky, it can let the system in an unstable state, so some restrictions apply:

  • Thread.Abort will not try to cancel a thread that is running native code. It makes sense, but causes some absurd situation. A thread stopped in a Console.ReadLine() is deep into native code, so it can not be aborted, but this is a pretty harmless operation.
  • The thread is given the chance to manage these abortions. A ThreadAbortException is thrown inside the thread, that can catch it and do some clean up. ThreadAbortExcpetion is a very special exception, even if you catch it, it gets re-thrown, so it will ultimately kill the thread. In order to avoid that, the thread has to call Thread.ResetAbort() inside the catch block.

Related to this, I was revisiting how unhandled exceptions in one thread affect the whole process. An unhandled exception will not just kill the involved thread, but the whole process. This is the same regardless of whether the Thread is foreground or background (background threads are those that will not keep a process running if all the foreground ones are finished). This is not affected either by whether the Thread is a normal thread or a .Net ThreadPool one.

ThreadAbortExceptions are also particular in this sense. As I've said, if we don't call ResetAbort the exception will get to the top of the thread and finish it, but it won't crash the process.

Finally, I've learnt that since .Net 4.5 unhandled exceptions in a Thread started with Task.Run (or Task.Factory.Start) will not tear down the process. I guess Task.Run wraps the call to the provided delegate in a try-catch block.

An additional note, Thread.Abort is not available in .Net Core

Basic REST stuff

I have a very limited knowledge (being optimistic) about REST API's (I mean, using the right url's, the HATEOAS thing...) and the other day I came across a couple of points that I'll share.

I was doing a GET request and the last parameter was a decimal value, something like this:

http://myserver.org/documents/docId/version

http://myserver.org/documents/a123bb/1.1

The above would give a server error (thrown by the http server itself, not by the ASP.NET Web API controller). The problem is because finishing the url with a ".1" the http server was interpreting that I was requesting a file. You can easily fix this by adding a "/" at the end, I mean:

http://myserver.org/documents/a123bb/1.1/

The next issue I came across was with an optional parameter not placed at the end of the url. I want the children of a document for a given version, but the version parameter is optional, as if missing we'll use the last available document. I wanted to use something like this:

http://myserver.org/documents/a123bb/1.1/children

http://myserver.org/documents/a123bb/children

[HttpGet]
[Route("{docId}/{version}/children")]
public HttpResponseMessage GetDocumentChildren(string docId, string version = null){}

The above would not work for the second url. The controller would not understand that version was missing there. In the end I had to add a second route to the controller, so I ended up with this:

[HttpGet]
[Route("{docId}/{version}/children")]		
[Route("{docId}/children")]
public HttpResponseMessage GetDocumentChildren(string docId, string version = null){}	

Arrow Functions and binding

When doing some tests with Jasmine I wanted to spy on one method invoked from the constructor (I mean method and constructor of the same class). As you need to set the spy before having the instance created, you'll have to spy on the class method (so you'll end up spying on each instance of the class, not just a particular one), not on the instance method. As you know, JavaScript classes are syntax sugar on top of the prototypes mechanism. Apart from "recording" the call, I did not want to invoke the original method, but a function assigning some mock data to one property of the object. So I was setting my spy like this

spyOn(MyClass.prototype, "myMethod").and.callFake(() => this.items = mockData.items);

let myInstance = new MyClass();
console.log(myInstance.items); //undefined!

I had read that the function provided to callFake would receive as this the object on which we are spying, but the above code was not setting the property. Can you spot the problem? Well, it's simple, we should use a function expression rather than an arrow function:

spyOn(MyClass.prototype, "myMethod").and.fake(function(){
 this.items = mockData.items;
});

Other than its cool syntax Arrow functions trap the lexical this rather than using a dynamic this. This is very useful for event handlers, callbacks... and has made the typical "self-closure" pattern disappear, I mean:

self = this;
myButton.addEventListener("click", function(){
 self.myMethod();
});

//has given place to:
myButton.addEventListener("click", () => this.myMethod());

An arrow function binds to the function the this that exists at the moment of creating the function. The arrow function gets bound to that this similar to when you call function.prototype.bind. Once you have a bound function, calling bind on it again will not have a real effect. You get a new function wrapping the bound one, but the invokation will call the internal function with the first bound this. I guess function.prototype.bind creates a wrapper that uses function.prototype.call, and function.prototype.call is useless with bound functions:


class Person{
 constructor(name){
  this.name = name;
 }
 
 sayHi(){
  return "Hi, I'm " + this.name;
 }
}

let f1 = Person.prototype.sayHi.bind({name: "Francois"});
console.log(f1());

console.log(f1.call({name: "Chloe"}));

let f2 = f1.bind({name: "Chloe"});
console.log(f2());

//Hi, I'm Francois
//Hi, I'm Francois
//Hi, I'm Francois

I assume Jasmine is using using Function.prototype.bind to provide the this to the function invoked from callFake, so an arrow won't make it.

Notice that both functions created with Function.prototype.bind and arrow functions do not have a .prototype property (obviously they still have a [[Prototype]] property as every JavaScript object) and can not be used as a constructor with new (you'll get an "is not a constructor" error)

Sunday 30 June 2019

CAP des Banlieus

J'en ai marré/I'm fed up:

  • I'm fed up of Racialists that consider Racialism the new antiracism.
  • I'm fed up of "Identitarians" that defend the identity of the country from which their ancestors fled escaping from poverty rather than defending the country where they've been born and grown, the country that has given them all the chances that they have decided to drop so they can live happy in their victimism.
  • I'm fed up of communitarists that try to close themselves in a community defined by religion, skin color or external countries, rather than the national community.
  • I'm fed up of so called "antifascists" that work hand in hand with IslamoFascists and xenophobic indigenists.
  • I'm fed up of religious foundamentalists trying to impose on us the shit laws of their medieval society.
  • I'm fed up of "Anticolonial activists" (Indigenists), that fight against the European colononialism of centuries ago (when a more advanced culture extended its control over less advanced societies) and defend the current colonialization of European soil by backward cultures...
  • I'm fed up of Houria Bouteldja and Medine (le "rappeur" islamiste), and demand them to be immediatelly remigrated into their beloved Algerian shitholes.

  • I'm fed up of the Rap anti-French, anti-white and islamo-supremacist.

  • I'm fed up of people that denounce the European slave trade but have nothing to say about the Arab slave trade (that was much worse in numbers and probably way more cruel).
  • I'm fed up of those who know nothing about Barbary corsairs seizing European women and children and turning them into sex slaves (hum, sounds like ISIS, maybe there's a certain religion as common basis in both cases...)
  • I'm fed up of Maghrebian and Turkish ultra-nationalists that with all their xenophobia and histerical nationalism make any European "facha" look like a love and flowers hippy.

  • I'm fed up of seeing people born and raised in France to leverage any opportunity to wave the flag of their parents country rather than the French flag.
  • I'm fed up of seeing those same people spit of the French flag and make monkey sounds when the Marseillaise resounds.
  • I'm fed up of seeing those same people using their "piece d'identité" (French ID card) to receive the social aids that allow them to spend their days complaining and conspiring against France, rather than working to keep it alive.
  • I'm fed up of retarded people that think that all cultures are equal.
  • I'm fed up of all those that think that asking immigrants to integrate and assimilate is fascism...
  • I'm fed up of the leftist scum that defends and promotes all the things I'm fed up with. Melenchon, Benoit Hamon... History will remind you as what you are: traitors, cowards, collabos and fascists (you try to impose on us a failed model of multicultural/multiconflictual society without asking for our approval).

And today, I'm particularly fed up with La Coupe d'Afrique des banlieus/African Cup of the "French" suburban neighbourhoods. So this month the African Footbal Cup is being played, and people of African descent born and raised in France decided to play an African Cup in their neighbourhoods. So they made teams based on the country of their parents/grandparents, put on the t-shirts of those countries, waved flags of those countries, sweared loyalty to those countries hitting their chests, sang the anthems of those countries, felt an irrational pride for those countries, shitholes (with all my respect) like Algeria, Mali, Senegal, Turkey (yeah, the Turks decided to join the party, even if they hate Africans with all their soul)... where most of them have rarely set foot... and so on and so on. All of this on French soil, perpetrated by people born and raised in France and taking advantage of the French System... The black guy that came up with the idea explains that he decided to make teams based on nationalities to avoid the conflicts that would have erupted if they had made the teams based on neighbourhoods... OK, it would make sense if all those people had a feeling of being French and their "parents nation" were just a third level identity, but that's not the case at all. The guy goes on to tell us that he does not need to justify how French he is, he is obviously French because he has a French ID (Francais de papiers), pays with euros and drives a French car... wow, I really can't refute that...

You don't need to understand French to understand this video. All you need to understand is that all those people wearing a foreign "uniform", waving a foreign flag and singing a foreign anthem are doing so in France, and were born and raised in France. You also need to understand the the leftist journalists in the video (save for 1 of them, obviously not a leftist) can not believe how absolutely cool the whole thing is. How a great example of the "Vivre ensemble"/Living together, they are making... Yes, living together among Africans, "a cote des Francais" (apart from the French).

Cerise sur le gateu (icing on the cake), there's something that I guess is not in accordance with the French Laicité principles...There's been a match between a Muslim team and Christian team! Wow, c'est vraiment trop cool, n'est-ce pas? et apres!? (it's too cool, right? what's next!?) Maybe a match between gays and heteros, or between gays and bisexuals or... wow, there are so many ideas flowing...

Friday 28 June 2019

Promises vs Observables

Reactive Extensions (I've only used them in JavaScript/TypeScript with rxjs) is a really powerful tool and the more one plays with them more aware one becomes. Their aim is making easy to work with push streams (sequences) of data (I had given a classification of streams types here). It gives you so much power when compared to basic/classic event listeners.

On the other side, when working with an asynchronous operation that returns just once (it can return a single value or a collection, but it returns it just once) returning a Promise and using async/await moves you into a world miles ahead of the classic callback paradigm used for continuing asynchronous operations.

From the above, it would seem clear that we'll use Observables or Promises + async + await depending on whether we have multiple data or single data. But then, you find that the Angular HttpClient returns an Observable rather than a Promise and it seems odd. Even worse, if understanding how the magic of async/await and Promises (or Tasks in .Net) was not an easy endeavor for you, it hurts hard to read people saying that Promises are no longer cool and only Observables rule... what the fuck!?

There are many articles comparing Promises and Observables, this one is particularly good. You'll find also many discussions about why even when dealing only with a single value Observables are supposed to be always superior to Promises, and you'll find many disidents, like me :-). Observables can be cancelled and they are lazy, that's why even for a single data operation like an http request, they are supposed to be superior. That's true and that's false. Let me explain.

For those situations where you could need to cancel the http request, or you want to set it but run it lazily, yes, returning and Observable is a better option, but how often do you need any of those 2 features for a http request? Honestly, I hardly can think of a single case where I've needed that... On the other side, even after becoming familiar with pipe, flatmap, of... I think the cleanliness of the code that you can write with async-await lives in a different galaxy than the one you write with Observables. Let's see an example.

Let's say I have 2 async operations that I want to run sequentially, passing the result of the first one to the second one. Each operation would be like a http request, but in order to be able to run this code without depending on a server I'm just doing a sort of simulation.


function getUserId(userName:string, callback: (id:number) => void){
 let users: { [key: string]: number } = {
  anonymous: -1,
  francois: 1,
  chloe: 2
 };
 setTimeout(() => callback(users[userName] as number), 2000);
}

function getPermissions(id:number, callback: (permissions:string[]) => void){
 let permissions: { [key: number]: string[] } = {
  1: ["r", "w"],
  2: ["r"]
 };
 setTimeout(() => callback(permissions[id]), 2000);
}

Promises + async + await

I wrap each operation in a Promise returning function:

function promisifiedGetUserId(userName:string): Promise{
 return new Promise(res => getUserId(userName, res));
}

function promisifiedGetPermissions(id:number): Promise{
 return new Promise(res => getPermissions(id, res));
}

And now I can use them like this:

async function promisesTest(userName:string){
 let id = await promisifiedGetUserId(userName);
 console.log("id: " + id);

 let permissions: string[];
 if (id === -1){
  permissions = [];
 }
 else{
  permissions =  await promisifiedGetPermissions(id);
 }
 
 console.log("permissions: " + permissions.join(","));
} 

//sort of C#'s "async main"
(async () => {
 await promisesTest("francois");
 await promisesTest("anonymous");
})();

Observables

I wrap each operation in an Observable returning function:

function observableGetUserId(userName:string): Observable{
 return new Observable(observer => getUserId(userName, id => observer.next(id)));
}

function observableGetPermissions(id:number): Observable{
 return new Observable(observer => getPermissions(id, permissions => observer.next(permissions)));
}

And now I can use them like this:

function observablesTest(userName:string){
 observableGetUserId(userName).pipe(
  flatMap(id => {
   console.log("id: " + id);
   if (id === -1){
    return of([]);
   }
   else{
    return observableGetPermissions(id);
   }
  })
 ).subscribe(permissions => console.log("permissions: " + permissions.join(",")));
}

observablesTest("francois");
//observablesTest("anonymous");
 

I think the first code is way more clear and natural, don't you think? So Promises, await and async are alive and well! I've uploaded the code into a gist.

This said, I think the decision done by Angular of returning an Observable with their http client is OK. Converting that Observable to a Promise is as simple as invoking myObservable.toPromise(), so you can still do good use of async/await. At the same time, if you are in one of those cases where canceability and laziness are useful to use, that Observable is what you need, so in the end Angular is giving you the most versatile solution.