Bytecodes, ASTs and so on

These pieces of news: [1], [2] and [3] about WebAssembly have made a pretty intereting read. It's not that I am interested in using it, I think 99% of web development projects will continue to be done in an every day more powerful and beautiful JavaScript, but the concept is pretty interesting since a technical stand point.

Being able to write high performance code for the web so that game engines can run smoothly on the browser is cool, but as I've said, I guess few people will ever need it. Having your Browser Virtual Machine running for a same "application" two parsers (one for JavaScript and one for WebAssembly) compiling to a common bytecode language is cool, but nothing new (you've been able since a long while to mix Python and Ruby into a C# application, and the same for Java). What is funny is that in the CLR and the JVM the first citizens were the static, "low level" languages (C#, Java...), and then the dynamic and more feature filled languages (Python, JavaScript, Groovy) were added as second citizens, for the web it has been right the contrary path.

What I find really interesting is that the code written in this "binary format for the web" is not bytecodes, in the sense that is not a set of instructions for a stack based (CLR, standard JVM) or registers based (Dalvik) Virtual Machine, but a bit upper level, a binary representation of an Abstract Syntax Tree (AST). Does this ring a bell? Well, it's just what is done with the DLR languages, that are compiled at runtime into a DLR AST that will be then transformed into .Net bytecodes.

The main performance advantage offered by WebAsm over JavaScript is that parsing this binary AST into bytecodes is way faster than parsing JavaScript. Though quite unrelated, my mind established a link between this and JerryScript, a JavaScript engine for small embedded devices. In the article they explain how they managed to make this engine fit into just 64KBs of memory:

From the perspective of IoT, we only focused on memory footprint.

JerryScript is a pure interpreter in contrast with multi-stage adaptive JIT engines today. So it has no overhead for storing compiled code. Even the parser doesn’t store AST. It produces bytecode line-by-line directly from the source code. For data representation, objects in JerryScript are optimized for size. JerryScript is using compressed pointers, bytecode blocks with fixed size, a pre-allocated object pool and multiple representation of Number objects to achieve both standard compliance and memory optimization. We will keep continue to reduce the memory footprint in various ways.

WPF vs WinForms

I don't like WPF. I've ever found it complex and overarchitectured, with a rather steep learning curve (and most times the VS designer does not seem able to draw effectively a XAML form) . For thin client applications a HTML5 SPA using your favorite JavaScript Framework/toolkit is usually more than enough. For thick clients sometimes is also enough. For those that it's not and for purely client applications you can try some of the HTML5 and node combinations discussed here. Anyway, for business line applications, Windows client applications are still needed and demanded. For those cases, I still would prefer to avoid WPF and stick to Windows Forms, and seems like I'm not the only one.

Sure there are many horribly architectured Windows Forms applications out there, just a continuation of the classical vb5 and vb6 nightmares. Obviously that's not Windows Forms fault, and many people have been developing pretty well desinged WinForms applications. The use of MVC (well, indeed I think it is MVP) demonstrated here is quite similar to what I've used in the few WinForms projects I've worked on.

With this in mind, the other day I was wondering if there is really any advantage in using WPF over WinForms. The resulting "research" did not give me conclusive answers, but made me refresh my poor knowledge of how Windows GUI applications work.

Apart from XAML, the recommended patterns and the new controls, under the covers the main evolution from WinForms to WPF is that WPF uses Direct X for rendering while WinForms uses GDI. DirectX can discharge this drawing to the Graphic Card, while GDI will do it in your CPU. This means that WPF drawing should be much faster that WinForms drawing. However, in principle WPF objects are more heavy than WinForms objects (in .Net terms). You can read more about it in these interesting StackOverflow posts [1] and [2]

One interesting point raised there is:

That being said, if you have a LOT of controls, WPF can actually outperform Windows Forms by a fair amount. In Windows Forms, each control requires a separate window handle, and receives its own message sets. With a lot of controls, this can actually slow things down pretty dramatically.

So in "Windows terms" WPF controls are lighter. But, what does this window handle and messages thing mean?

The most essential feature of any Windows GUI application (either .Net or native) is the existence of a Message Loop. The main thread in your windows application runs in a loop waiting for messages (a click, a timer, a refresh...) and when a message is received it will pass it to the code (Windows Procedure) associated to the affected Window. The notion of what a Window is at that level is quite important. At this level a Window (a Win32 Window) is not just your "normal Window" (with is blue title bar, close button...) but also controls like buttons, combo boxes, edits... you can see it here. They are created with the CreateWindow Win32 function, that is how under the covers WinForms, classic vb, MFC,... create their controls.

Both WinForms and WPF are based on the Windows message loop (it's created and kept hidden from you by Application.Run), but WinForms works much closer to the Win32 reality. Underneath it uses the CreateWindow function in user32.dll for creating most of its controls. In turn, user32 uses GDI for drawing these controls. From here:

User32.dll is a core windows dll used for windowing and other interactive user program tasks. WPF does not use it much.
Windows.Forms is mostly a .net wrapper around User32, and so is still based entirely on GDI,GDI+, and window handles.

WPF uses CreateWindow just for creating "real windows" and then it will create its controls by drawing on these Windows via DirectX (as if it were drawing shapes). As an analogy, think of how html controls work. An html button or textbox looks like its WinForms counterpart, but it's not a Windows Control, but a shape that the browser draws (for example via Cairo in Firefox)

You can verify this by running Winspector and trying to select a control in a WinForms application and a WPF application.

In the WinForms application you can select controls, for example a button:

While in the WPF application the only thing you can select is the "normal Window":

Set [[Prototype]]

When years ago I first heard about the plans to add classes to ES6 I was terrified they could intend to mutate the language from its Prototype based nature to a standard Class based paradigm. Almost immediately I found comments explaining that the addition of classes would be just syntactic sugar, and these classes would be translated by the compiler into the prototype chains we are used to.

Now that ES6 as a standard is a reality, and hopefully most ES engines will be implementing it pretty soon, it feels like it's time to get ready for it. On one side you can read here how indeed classes are just syntactic sugar.

ES6 classes are a simple sugar over the prototype-based OO pattern.

But furthermore you can read here that a standard way has been added to set the [[Prototype]] (aka internal Prototype) of an Object, by means of the Object.setPrototypeOf(obj, proto) method. We will no longer need to use the non standard __proto__ property, neither for setting nor for getting (as since ES5 we've had Object.getPrototypeOf(obj)).

It's noteworthy though the warning in the MDN about how slow setPrototypeOf can be.

DLR vs InvokeDynamic

I found out the other day that PowerShell 3.0 makes use of the DLR. It came as a surprise, as after the abandonment by Microsoft of the Iron languages, I thought they were setting it aside, just keeping it in the framework but with no plans of using it for anything. Hopefully it's not like that, and I guess the reason for not hearing anything about changes or evolutions in the DLR in the last versions of the CLR is that it's complete enough and there's no pressing need for improvements.

This finding, along with this excellent article about invokedynamic in the Java Platform, made me think again of the differences between both approaches for speeding up dynamic languages in the JVM and the CLR. Last year I had already discussed in this previous article how Java 8 implements Lambdas by means of invokedynamic and generating at run time the classes needed to contain the state of the lambdas, on the other side either C#, Scala and Groovy (at least in the pre-Java 8 times) create these classes fully at compile time.

Let's forget about Java Lambdas and let's focus on the "dynamic performance boosting techniques". Both invokedynamic and the DLR make massive use of runtime code generation, but in rather different ways.

invokedynamic When the JVM interpreter first finds an invokedynamic bytecode instruction, it will call into a bootstrap method and will obtain a CallSite object wrapping a MethodHandle. So the invokedynamic instruction will be replaced by a call to that MethodHandle hosted in the CallSite. This call to the bootstrap happens only once for each invokedynamic instruction found. As the code evolves, the MethodHandle in the CallSite can be replaced by another MethodHandle. Regarding the bootstrapping process, I think this statement I found somewhere is pretty important for its understanding:

"the key thing about invokedynamic is that it's essentially a JVM-level macro that defers lambda translation strategy to LambdaMetaFactory which is a library class."

and from this other article:

Whenever a invokedynamic call site is created in bytecode, it references a so-called bootstrap method for deciding what method should be invoked. A bootstrap method is implemented in plain Java and serves as a lookup routine for locating the method that should be called for the invokedynamic instruction. This lookup is only performed a single time. It is however possible to manually rebind a dynamic call site at a later time.

So the key part now is understanding what a MethodHandle is. Well, it's basically an object containing a "method pointer". So, it's basically the same as a .net delegate? yes and no. In .net we have multiple delegate types, classes that inherit from System.MulticastDelegate. We create (well, since .net 2.0 we normally use the Action and Func ones provided by the Framework) delegate types for the different method signatures that we need. This way, the invokation of a delegate is type safe and fast, as the the type correctness is guaranteed at compile time. In Java we have a unique MethodHandle class, so this type safety is guaranteed differently. The JVM itself knows about MethodHandles, and they are invoked through what is known as a "polymorphic signature", so it allows to call MethodHandle.invokeExact with different parameters in the different call sites.

DLR. The procedure to improve .net as a host for dynamic languages did not entail the creation of any new bytecode instruction or changes to the Virtual Machine, but a rather smart use of existing features like Expression Trees, Delegates and LightWeight Code Generation. This series of posts about the DLR internals gives and excellent explanation. Summing it up, in those locations where a dynamic call is performed (for example through the use of a variable declared as dynamic in C#) the corresponding compiler will insert the creation of a CallSite object and will invoke the delegate referenced by its Target property. So, basically this sounds just very similar to the CallSite-MethodHandle combo in the Java Platform. The CallSite creation in .net is done at compile time and in Java at runtime, so looks like the Java approach has to be more dynamic. Well, not really, the CallSite creation in .net is the less important part, the juicy thing happens at runtime, when depending on what object we really have at that point each time, dynamic code generation is used to create a new delegate (doing heavy use of a cache system) to bind to the CallSite.Target. This is pretty sophisticated and is explained here. I assume the JVM uses a similar Cache and decision system for rebinding the MethodHandle in a CallSite, but I have not investigated about it.

I've read in some places that the JVM approach is better and hence it's better suited for dynamic languages. This is one of those sources. It basically says (I think) that in the JVM when finally native code is generated this can be inlined, but it's not like that for the CLR. Honestly, this is a too advanced topic for me, so I can not say how true and how an impact it really makes. Anyway, if not the best, I think the CLR + DLR have proved to be a rather good host for dynamic languages, and with this in mind (and with all their investment in TypeScript) I'd love to see Microsoft coming up with an implementation of ES5-ES6 for the CLR (like Oracle has done with Nashorn)

Solidarity with Heroes

When a state welcomes back heroes with charges of terrorism one can not feel more than contempt and despite for that state. The 2 Spanish Freedom Fighters that have been risking their lives for 6 months in Rojava confronting that gang of rapists, islamo-fascists and perverts called ISIS have been detained by the Spanish repressive apparatus. You can read about it here and already posted about how the Spanish government exercises its repressive role against far-left militants that had sided with the pro-russian side in Novorossiya. It seems they intend to use the same charges against these 2 Humanitarian Fighters, "joining a terrorist organization", and "putting at risk the interests of the 'Spanish nation'"...

Both accusations are demented and delusional and prove how rotten the Spanish government is. Considering the YPG/YPJ as a terrorist group is like saying that being vegetarian is a crime against humanity... The second accusation "risks for the interests of Spain" is particularly stupid, as it indicates that the Spanish government acknowledges the links between the Turkish state and ISIS. I assume that the Spanish government does not recognize ISIS as a state. So, which other states we have there?. Irak and Syria? Both of them are fighting against ISIS (OK, yes, Assad is against the Kurdish liberation process, but I guess that's not a concern for the Spanish government). We have one country left, Turkey. Oh, yes, we all know that Turkey and its IslamoNationalist government are big allies of the West... (sick world). For sure the Spanish government does not want to irritate Erdogan, that friendly murderer... So, what we can deduce from this is that they admit that fighting against ISIS can irritate Turkey, and hence there's a link between ISIS and the Erdogan beast... Thanks Rajoy, you are admitting what we already knew...

There are millions of proofs of how the Turkish government collaborates with ISIS, but this article explaining how the terrorists entered Kobane some weeks ago through the Turkish border is revolting. Maybe this was the best way Erdogan could find to take revenge against the ascent of the HDP, the left-wing pro-Kurdish party.

I really think Kurdish people in Europe should conduct actions against the interests of the Turkish government, something similar to what an Armenian organization did in the 70's and 80's.

AD Fast Querying

The other day at work I went through again a issue I had already experienced in previous occasions. Querying the Windows AD for some basic need can be terribly slow. I have no idea about why, but I've suffered this while querying different AD's (from different companies). So this time I've gone through an alternative route.

As far as I know the recommended way to query the AD from modern versios of .Net is using the classes in the System.DirectoryServices.AccountManagement namespace, like PrincipalContext and UserPrincipal, so if you want to get the AD groups a user is member of you would do something like this:

private static List<string> GetGroupNamesBasedOnPC(string userName, string domainName)
 {
  List<string> result;

  using (PrincipalContext pc = new PrincipalContext(ContextType.Domain, domainName))
  {
   using (PrincipalSearchResult<.Principal> src = UserPrincipal.FindByIdentity(pc, userName).GetGroups(pc))
   {
    result = src.Select(sr => sr.SamAccountName).ToList();
   }
  }
  return result;
 }

The above code is terribly sloooooowwwwww.

Hopefully, there are alternatives (don't ask me why these alternatives are way faster, I have no clue).

• You can use the very fast net user command like this:
  net user userName /domain
  notice that it will search the current domain, so the "/domain" string is right that, you don't have to replace it by the domain name, I mean:
  net user francoise12 /domain
  The problem with that approach is that the returned string comes formatted for the Windows Console and some characters can be trimmed. So large group names will be trimmed and this command is not valid for general cases.
  
  
• Another option is using the gpresult command, as along with information about the installed gpo's it will also return the list a groups for the current user, and this time with no odd formatting causing trimming.
  gpresult /user ng3419a /r
  The problem with this is that it only works for the current user, you can't pass the user name as parameter
  
  
• And finally the good one:
  net group groupName /domain
  Here you'll get a listing with just the group names, one per row, so no trimming problems. The windows commandline is not that primitive as many people use to think (and yes, of course with PowerShell is really powerful), so if you just want to verify if a user belongs to a certain group you can pipe the output to the find command like this:
  net group groupName /domain | find /I /C userName
  If you want to run it from a .Net application you just do this (notice that net group is a command, not a program, so the process that you have to start is cmd.exe)

  string command = "/c net group /domain " + group + " | find /I /C \"" + user + "\"";
  Process process = new Process();
  process.StartInfo.FileName = "cmd.exe";
  process.StartInfo.Arguments = command;
  process.StartInfo.WindowStyle = ProcessWindowStyle.Hidden;
  //need these 2 lines in order to read from the stdout
  process.StartInfo.UseShellExecute = false;
  process.StartInfo.RedirectStandardOutput = true;
  process.Start();
  process.WaitForExit();// Waits here for the process to exit.
  string output = process.StandardOutput.ReadToEnd(); 
  bool belongsToGroup = output[0] == '1';
  

Simulating await with ES6 Generators

When I first read about ES6 generators I wrote a post about its differences with C# "iterators" (I'll say it again, horrible naming choice by Microsoft). At that time I read how people were taking advantage of them to write code resembling C#s async/await magic. It seemed hard and interesting to me to understand at first sight how both things were related, but I had no time to look into it at that time. These days, after glancing through this thorough and brilliant article I decided to try to understand it.

The first point to consider is that as mentioned in the article, generator objects created by a generator function implement 2 different interfaces: data producers (iterators, the Symbol.iterator thing), that's what you are used to in C#, and data consumers. The latter is pretty interesting as it's the feature that makes the await emulation possible.

The fundamental idea is that generator objects combine the producer and consumer role in a "next" method that receives a value and returns a value. You represent it like this in your generator function:

var b = yield a;

The essential part in that line is that it really means this:

return a to the current "next" call;
read b from the next "next" call;
and execute the following lines until the next "yield" is found and you return a new value from there);

between both actions, the generator function gets "suspended".

This idea of having your code inside a sort of function (in the end it's a kind of illusion, that function is bended by the compiler into a generator object where state is maintained so that this suspension/reentrancy can really work) is the basis of how async/await works in C#. So, if you return from your yield a sort of Task object that when completed would call the reentrant function again passing it its return value, you basically get the same. Calling the reentrant funtion means calling the generator.next from an async loop, so your Task objects will need a reference to the async loop function (that you will call from the continuation (nextStep property in my example) that you have added to the Task), so that they can continue the cycle.

The JavaScript kind of equivalent to C# Tasks are Promises, and it's what is used by the different libraries (like co) enabling this sort of async/await programming. For exploratory reasons, in order to fully understand this whole thing, when writing my "async/await enabling" function, I've decided to use a very basic TaskInfo object rather than Promises, I think it makes easier to understand how the whole think works with no need to know anything about promises. Indeed, seems like co also works with a sort of lightweight alternative to promises called thunks

I've put up the code in this gist.