Saturday 31 August 2019

Promise Reject vs Throw

Promises are designed to be resolved or rejected (by the code ran by the promise), but occasionally I've seen code that throws an exception rather than rejecting, so is there any difference? Well, sometimes yes and sometimes no. Let's say I have an "old school" callback based asynchronous function

function getPost(url, successFn){
 let posts = {
  "blog.fr/aaa": "AAA post"
 };
 //simulate async operation
 setTimeout(()=> successFn(posts[url]), 1000);
}

Now I have a function that makes use of it, adding some validation logic and "modernizing" it by returning a Promise.

function validateAndGetPostAsync(url){
 return new Promise((res, rej) => {
  if (!url)
   return rej("empty url");
  
  getPost(url, (post) => {
   if (!post)
    return rej("post not found");
   res(post);
  });
 });
}

It's rejecting the promise if the initial validation fails or if the returned post is null. Using it with await and a try-catch block works nicely:

let urls = ["blog.fr/aaa", null, "blog.fr/aaa"];
 for (let url of urls){
  try{
   console.log(`url: ${url} ${await validateAndGetPostAsync(url)}`);
  }
  catch (ex){
   console.log(`exception: ${ex}`);
  }
 }
 console.log("after retrieving blogs");
 
 //url: blog.fr/aaa - AAA post
 //exception: empty url
 //exception: post not found

Now, if rather than rejecting we use a throw:

function validateAndGetPostAsync2(url){
 return new Promise((res, rej) => {
  if (!url)
   throw "empty url";
  
  getPost(url, (post) => {
   if (!post)
    throw "post not found";
   res(post);
  });
 });
}

and run the same example as before:

 for (let url of urls){
  try{
   console.log(`url: ${url} - ${await validateAndGetPostAsync2(url)}`);
  }
  catch (ex){
   console.log(`exception: ${ex}`);
  }
 }
 console.log("after retrieving blogs");
  
 //Output:
 // url: blog.fr/aaa - AAA post
 // exception: empty url

 // D:\Main\MyPrograms\programs_JavaScript\promises\throwVsReject.js:31
         // throw "post not found";
         // ^
 //post not found

We can see that while the throwing in the initial validation is being caught, the throw in the validation after obtaining the post is not caught (node prints the exception and crashes, the "after retrieving blogs" line is never printed). So what's happening? Well, it's easy, validateAndGetPostAsync is calling getPost, that is an "old school" asynchronous function, it invokes a callback once it gets the post, rather than returning a Promise. This asynchronous callback runs in a different stack from the one that launched the code, so a try-catch block surrounding the callback won't catch the Exception. When invoking reject rather than throwing, this is not a problem. Why? Well, the await-try-catch is just magic that the compiler translates to a .then().catch() chain (more or less). A call to reject() obviously rejects the promise and is captured by the Promise.prototype.catch(). However, when doing a throw, the Promise constructor wraps in a try-catch block the call to the provided function, and that catch block will reject the promise. If the code runs outside that try (because it's running asynchronously in another stack) it won't be captured and no call to the reject callback will happen.

Sunday 25 August 2019

13 Minutes

13 Minutes is a really important film, cause it tells a really important story, that of Georg Elser the man that was about to kill Hitler in November 1939, little after the start of WWII. It's hard to believe that this event is so little known. I'm not by far as educated as I'd like to be, but you know that I'm pretty much into history and social/political stuff, I've been tons of times in Germany... and I'd never heard about this. On the other hand, I've watched documentaries and one film about the assassination attempt of July 1944. This attempt would have moved forward several months the end of the war, but it pales down in importance when compared to the one of 1939 that could have just stopped it from the beginning.

The story of Georg Elser is that of a real hero, an incredible example of humanism. A humble, incredibly smart (the nazi officials could not believe that he had managed to build that bomb) left wing christian (though close to the German Communist Party he was not affiliated) that fed up with what was happening before his eyes decided to take matters into his own hands and work alone for one year in a thorough plan to bomb out the Adolf motherfucker. Hitler left the venue 13 minutes before the bomb exploded, so those 13 minutes would cause suffering and destruction as humanity had never experienced before. I'm not going to tell you more, you just have to watch the film and search some information about this hero.

The German people is deeply aware of its recent history, of its failure as a society that allowed the Nazi monstrosity to deceive it, poison it and almost destroy it (well, with the rise of AFD this awareness seems to be starting to fade, and I don't say this for whatever the opinions of AFD regarding immigration, but because of their minimization or even approval of multiple aspects of the nazi regime). German society has lived with this feeling of guilt and responsibility for what happened, and you can still feel it today in facts like that the German left (contrary to the pro-Arab obsession of most of the European left) has never been particularly supportive of the Palestinians (unfortunately, this has started to change in the last years, caused by the massive increase of Muslims in the country, that as in the rest of Europe infiltrate any association, political group, whatever... with the only objective to push their own communitarian agenda). I'm waiting for the day when Muslims will infiltrate LGTB organizations to ask for homosexuality to be considered again a sickness and demand funds to get healed... OK, back to the point, so why has this hero been kept relatively hidden? Is it maybe to avoid any lessening in this feeling of guilt? Maybe knowing that there were individuals inside that society that fought against its destruction would reduce the idea of a whole society gone crazy and would reduce the sense of guilt? I don't think so, in that case Germans that saved Jews would also be kept under the carpet...

Maybe the problem lies in the fact that the bombing killed 8 people. One of them was a waitress, I don't know if she was a nazi sympathizer or a beautiful human being, the other 7 were part of the audience, so they were fucking nazis. The dead of an allegedly innocent person as collateral damage of an action that would have saved millions of lives should not undermine the value of such action, so I don't see this as an explanation... So honestly, I have no idea... Reading the Wikipedia entry I can see that he has received some recognition in the last decades, but anyway it seems too poor to me...

Watching this film reminded me of this really interesting German film about Sophie Scholl and the White Rose resistence movement. Another must see that shows us how one (small) part of the German people fought to their last breath against the Nazi madness.

Wednesday 14 August 2019

Privatize Method

Somehow some days ago I ended up performing one more of those useless programming experiments that still manage to keep me entertained. It's important to understand that the private accessors provided by languages like C# and Java are a compile time mechanism to enforce restrictions on the code that we write, but they should not be confused with a real security mechanism. They can be circumvented at runtime with Reflection, and that's not a flaw in the system, it's that they are not intended to be really "secret". Years ago I wrote about this and presented a trick where Reflection itself was used (by examining the call stack) to prevent this the use of Reflection for invoking private methods. Similar techniques can be used in Java.

As I explain in that post, you can get privacy in JavaScript by means of defining your public methods as closures that trap the private members. This indeed gives you a level of privacy way superior to the one provided by the private accessor in other languages. Anyway, few people found it necessary to mess their code with that technique. The last versions of JavaScript have remained reluctant to providing us with private accessors, but as the TypeScript compiler comes with them and almost everyone is using TypeScript... in a sense it's as if they were already a part of JavaScript...

Notice that getting by TypeScript privacy is even easier than using reflection, all you need is casting to any:

(myInstance as any).privateMethod();

For whatever the reason it came to my mind the idea of making a JavaScript method private at runtime. You have defined the method public and while the application is running you decide that it should go private. The closures trick has to be used when defining the methods in the class, so that's not an option. What we can do is replacing the existing method by a new one that checks the call stack to see if it's being called from another method in the same class and depending on it invokes the original method or throws an exception. Getting the caller of one function in JavaScript used to be quite straightforward, by means of function.caller, but it's not standard and is not available in strict mode. The other option, arguments.caller, is plain obsolete.

There's a pretty inventive solution (of course I did not come up with it, I just googled a bit). When you create an Error object, its stack property gets filled with a string with the call stack information, something like this:

 Error
    at calledFromInsideClass (D:\Main\MyPrograms\programs_JavaScript\PrivatizeMethod\PrivatizeMethod.js:51:23)
    at City.privatizedMethod (D:\Main\MyPrograms\programs_JavaScript\PrivatizeMethod\PrivatizeMethod.js:60:13)
    at City.growCity (D:\Main\MyPrograms\programs_JavaScript\PrivatizeMethod\PrivatizeMethod.js:30:14)
    at Object. (D:\Main\MyPrograms\programs_JavaScript\PrivatizeMethod\PrivatizeMethod.js:71:6)
    at Module._compile (internal/modules/cjs/loader.js:689:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:700:10)
    at Module.load (internal/modules/cjs/loader.js:599:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:538:12)
    at Function.Module._load (internal/modules/cjs/loader.js:530:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:742:12)

This has quite surprised me. In C#, the stack trace of an Exception only gets populated if you throw the exception, not just by creating the Exception object. The thing is that you can examine that string to see if you're being called from inside or outside the class. There are some articles about parsing this string [1] and [2]. For my proof of concept I've just used some basic, error-prone code based on some of the code I've seen there.

So I have created a privatizeMethod function that receives a "class" and the name of a method, and replaces that method (in the class.prototype) by a new method that when invoked checks (through the calledFromInsideClass function) if it's being invoked by a method in the class. The stack at that point is: calledFromInsideClass <- myPrivatizedMethod <- caller method, so we are checking the third entry in the stack trace.

function calledFromInsideClass(className:string):boolean{
    let stackTrace = (new Error()).stack; // Only tested in latest FF and Chrome
    console.log("stackTrace: " + stackTrace);
    stackTrace = (stackTrace as string).replace(/^Error\s+/, ''); // Sanitize Chrome
    let callerLine = stackTrace.split("\n")[2]; // 1st item is this function, 2nd item is the privatizedMethod, 3rd item is the caller
    return callerLine.includes(className + ".");
}


function privatizeMethod(classFn: any, methodName: string){
    let originalMethod = classFn.prototype[methodName];
    let privatizedMethod = function(){
        if (calledFromInsideClass(classFn.name)){
            console.log("privatized method called from inside");
            return originalMethod.call(this, ...arguments);
        }
        else{
            throw new Error(`privatized method ${originalMethod.name} called`);
        }
    };
    classFn.prototype[methodName] = privatizedMethod;
}

//privatize the the City.growPopulation method
privatizeMethod(City, City.prototype.growPopulation.name);

I've uploaded a full example (in typescript) here.

Wednesday 7 August 2019

Throttling Async Calls

The TPL and Parallel.ForEach make it easy to run multiple instances of a same operation in parallel, taking advantage of the number of processing units (processor cores) in your machine. This is great for CPU bound operations, but what about async I/O bound operations (async http calls for example)?. If the operation returns a Task, it makes no sense to start several threads and keep them idle waiting for the asynchronous result. We should use a single thread to launch each operation one after another (without awaiting for the Task to be completed) and let a continuation to be invoked in Thread Pool thread once the result is available (or maybe push the Tasks in a collection and do a WaitAll). Something like this (consider it pseudocode)


var postTasks = List<Task<string>>();
foreach (var url in urls)
{
 contents.Add(this.getPost(url));
}
var posts = postTasks.WaitAll();

That looks good, but in many occasions launching a huge number of async operations in parallel is not a good idea. Let's say we are sending those http requests to the same server, we don't want to overload it (or make it think that we're doing a DOS attack!). Thinking about this, I found this discussion in SO explaining the options to throttle these async operations. The usage of SemaphoreSlim is pretty interesting.

A questions comes to mind, how to deal with this same problem in that different universe called JavaScript, where awaitables are called Promises rather than Tasks and where the runtime is single-threaded? I've been playing a bit with this, and here it goes what I've came up with.

I've created a simple TicketManager class that is like a sort of SemaphoreSlim. It has a number of available tickets (as many as parallel async calls we want to do). Before invoking the async function we'll request a ticket, receiving a Promise that will be resolved as soon as an execution slot is available for us, and after completing the async function we'll release it. If there are no tickets available we'll await for one (getTicket returns a Promise).

class TicketProvider{
    constructor(maxActiveTickets){
        this.activeTickets = 0;
        this.maxActiveTickets = maxActiveTickets;
        this.waitingTickets = []; //array of resolve function, to get invoked once we have a free slot, so the code waiting for the ticket can move on
    }

    //returns Promise
    getTicket(){
        if (this.activeTickets {
                //save the resolve function so that we can invoke it when another ticket gets released
    this.waitingTickets.push(res);
            });
        }
    }

    releaseTicket(){
        this.activeTickets--;
        let nextTicketToResolve = this.waitingTickets.shift();
        if (nextTicketToResolve) {
            nextTicketToResolve();
        }
    }
}

So when a ticket is requested, if we still have some slot available we return a resolved promise, so the client will immediatelly continue. If not, we return a Promise. This Promise will get resolved when a slot gets realeased. Promises can not be resolved from outside (contrary to .Net, where a Task can be resolved by an associated TaskCompletionSource), it's from "inside" the Promise that we can resolve it, by means of the function received as parameter in the Promise constructor. So we have to create a Promise that checks at intervals if a slot has been released, and in that case resolves itself store that resolve function in the TicketManager.

Given an async function like this:

function getPost(postId:number):Promise{
    console.log("getPost " + postId + " started");
    return new Promise(res => {
        setTimeout(() => {
            console.log("---> getPost " + postId + " finishing");
            res(`${postId} - content`)
        }, 2000 * (1 + getRandomInt(2)));
    });
}

We can leverage this TicketManager like this (notice that I've wrapped in an additional function the call to the async call, in order to wait for it and then call to releaseTicket) in order to run no more than 3 post requests at the same time.

async function getPostAndReleaseTicket(postId:number, ticketProvider:TicketProvider):Promise{
    let post = await getPost(postId);
    ticketProvider.releaseTicket();
    return post;
}

async function runWithThrottling1(){
    let ticketProvider = new TicketProvider(3);
    let curPos = 0;
    let requests = [];
    while (curPos < 10){
        let ticket = await ticketProvider.getTicket();
        requests.push(getPostAndReleaseTicket(curPos, ticketProvider));
        curPos++;
    }
    await Promise.all(requests);
    console.log("all posts retrieved");
}

Or like this (now the wrapper funtion takes care of both getting the ticket and releasing it)

async function getTicketAndGetPostAndReleaseTicket(postId:number, ticketProvider:TicketProvider):Promise{
    await ticketProvider.getTicket();
    let post = await getPost(postId);
    ticketProvider.releaseTicket();
    return post;
}

async function runWithThrottling2(){
    let ticketProvider = new TicketProvider(3);
    let curPos = 0;
    let requests = [];
    while (curPos < 10){
        requests.push(getTicketAndGetPostAndReleaseTicket(curPos, ticketProvider));
        curPos++;
    }
    await Promise.all(requests);
    console.log("all posts retrieved");
}

I've uploaded the code to a gist

Sunday 4 August 2019

The Factory

I've recently quite enjoyed with a certain surprise an excellent Russian film, The Factory. Well, it's done with the support of Canal+ and somehow Armenia has also something to do with it, so in the end it's a Russian-French-Armenian production.

If you follow this blog I guess you'll know that I feel a profound distaste for Russia. An ultraconservative society, where ultra-nationalism, racism and xenophobia are the guiding rules for a good part of the population. A population that continues to rise to power bastards like Poutin, such an enemy of European values... I consider that an appropriate environment is essential to spur creativity, so with this in mind I'm not much inclined to expect anything interesting coming out of Russian soil (other than beautiful women...). But this film is an interesting exception.

The film depicts the kind of Russia that we expect. A desolate country, where most of the population live in dark and ugly soviet buildings that look as obsolete as the factories where they strive to get an income to manage to pay for their basic needs and keep the routine going. Where oligarchs rule supreme managing the lives of others as disposable pieces in their game to become richer and richer. One of these factories is one the few remaining economic activities of an anonymous, neglected city. One day, the oligarch owning the factory announces that he's closing it as it's no longer profitable. The instinctive reaction of the workers, threatening with a strike, makes the millionaire laugh. One of the workers, that wears external and internal scars from the war (Chechnya I guess) comes up with a different solution, kidnapping the oligarch, asking for a huge ransom and fleeing away. He convinces 5 other fellow workers to join him.

The kidnapping starts OK, but things will soon get twisted, and this ex-soldier will be back in a war commanding his unexperienced comrades. I won't tell you more, just try to find this film and watch it. There is action for sure, but this is more about personal drama, betrayal, redemption, hidden intentions, revenge... and all set in a dark, tragic atmosphere.

As an Asturian I know quite well what the closing of a factory (or a shipyard, or a coal mine...) means for a territory (when the government of that territory has no capacity to find a viable replacement economical activity).When the Spanish Government decided to close (or to do nothing to prevent it) a huge part of our productive fabric (starting in the late 70's and still ongoing now in the shape of a stupid "ecological transition" law) Asturian workers started their responses with the classic approach, strikes. Then measures scaled up into hunger strikes, locking themselves in the coal mines, walking hundreds of kilometers to Madrid to protest before the government... Things would get tougher, first blocking highways with burning tyres and then spiraling up into some levels of violence rarely seen in the western world (and supported or approved by a large percentage of the population, indeed, many of us have interiorised this kind of violent resistence as an essential part of the Asturian identity, perceiving it as a continuity in our history), using authentic urban war tactics against the armed forces (Naval Xixón, "valles mineros"...). This Russian approach of kidnapping was never tried, quite a shame!