Thursday, 29 March 2012

When a Process dies

In the end, software is not unlike biological entities, sooner or later all gets to an end. I've been pondering a bit about how a process can be finished off, what happens at that point, what code we can assume is run, how resources are cleaned up... and so on... and I think it's a good idea to put together my findings and conclusions. This will be focused on the Windows OS family and the .Net platform.

First of all, you should read this excellent article, almost all you need to know is there. In turn, part of the information there is taken from here.

The distinction between cooperative and abnormal process shutdown seems really important, but in the end probably is not that much. One of the main concerns would be whether the resources used by the application will be released or not. Well, given that we place our clean up code in finally blocks and in finalizers, the fact that even in a cooperative shutdown the finally blocks are not run, and the finalizers are tried, but they won't run if they take too long, could seem worrying. Actually, it's not that worrying, because when a process finishes (either gracefully or abruptly) the OS will take care of cleaning up its resources (decrease the reference count in the corresponding Kernel objects... and so on) as you can read here. Most times, the only code that is inside our finalizers is code to release native resources (CloseHandle...), so it seems like that task will be carried out by the OS anyway. Likewise, the situation is similar for finally blocks, usually we put resources release code there (think of a using construct, that in the end comes down to a try-finally), as those resources will end up being released by the OS, so that does not seem like a problem.

With this in mind, one could think, why does the runtime run the Finalizers in a cooperative shutdown if anyway the OS is going to release resources? Well, in some cases I think your finalizer could do something more than just releasing the native resources (write a log, send a last message over the network, try to flush a buffer). I'm saying "I think" cause almost in every article that you can come across, when they talk about finalization and dispose they only talk about releasing resources.
Contrary to what I initially thought (this article has been really helpful), it should be safe to use a Managed, non finalizable object from your finalization code, what you have to refrain from is using Finalizable objects, cause we don't know in what order objects are finalized, so that object could already have been finalized and therefore would be unable to do its work properly. Anyway, also in this article they stick to the "release resources" mantra:

Generalizing this principle, in a Dispose method it’s safe to clean up all resources that an object is holding onto, whether they are managed objects or native resources. However, in a finalizer it is only safe to clean up objects that are not finalizable, and generally the finalizer should only be releasing native resources.

There are a few more things I've learned that I'd like to add here

  • How does Task Manager kill processes? First of all, you'll have noticed that sometimes killing a process from the Applications tab fails, but killing it from the Processes tab works OK. As it's well explained here, different mechanisms are used in both cases. Killing it from the Process tab does a call to TerminateProcess, that will finish off the process without giving it any chance to run any extra code. Well, it's interesting to see that as explained here, the OS will try to complete all pending IO operations before the process exits.

    The TerminateProcess function is used to unconditionally cause a process to exit. The state of global data maintained by dynamic-link libraries (DLLs) may be compromised if TerminateProcess is used rather than ExitProcess. TerminateProcess initiates termination and returns immediately. This stops execution of all threads within the process and requests cancellation of all pending I/O. The terminated process cannot exit until all pending I/O has been completed or canceled. A process cannot prevent itself from being terminated.

  • How does sending Ctrl-C to a console application differ from the above?. The main difference is that your console application can have registered a handler routine via SetConsoleCtrlHandler to do some extra processing. The default handler for the Ctrl-C signal just calls to ExitProcess. This question is also worth a read.

Saturday, 24 March 2012

C# Object Initializers in JavaScript

A very short entry for a trick I came up with today

I'm rather keen on C#'s Object Initializers, and it's something that to some degree I've been missing in JavaScript. I'm saying just "to some degree" because on many occasions in JavaScript we just have enough with Object Literals and don't go down the way of declaring a "constructor" function and adding functions to the prototype (maybe we don't intend to have more than one instance of that object)...

Anyway, what could be the equivalent to this C#'s code:

Person p = new Person{
Name = "xose",
Age = 36

Well, jQuery's extend function comes more than handy here, so we can write this:

//constructor function
function person(){

var p = $.extend(new person(),{
name: "xose",
age : 36
Very simple, but so helpful...

Sunday, 18 March 2012

We Are The Night / Wir Sind Die Nacht

This film seemed to have all the ingredients to delight me, and indeed it did. Long in short, it's a film about a group of hyper attractive wealthy female vampires (with a nice touch of femme fatales) living/partying/killing/feeding in modern day Berlin.

Actually, there's much more to this film that just that. The plot turns around a troublesome young woman that gets turned into a vampire and how she copes with her new condition. The story confronts us with some "phylosophical" issues recurrent to most decent vampire films: immortality, loneliness, doubt, despair and emptiness after seeing your loved ones die (this one is brilliantly depicted here when one 20 something vampire sees die an 80 something woman that happpens to be the child she had had before been turned into a vampire), fear to destroy those you love... It's quite interesting to see how differently these vampires live their current condition, the "master", the one that turned the others, seems happy, but obsessed seeking a missing love, another one, turned a few years ago, is still really excited about the marvels of immortality and is living her new life at its most, and finally, we have the depressed type, one that can't seem to cope with the death of all those she loved and the impossibility to establish links with anyone outside of their circle.

The film is set on modern day Berlin, probably my favorite place in the world, and there are tons of those views I so much love of this city (the omnipresent TV tower, the elevated Bahn stations, the street art everywhere, the plattenbau). Part of the action takes place in one location also used in the recent film Hanna, the abandoned Spreepark, and the final scenes are set on Teufelsberg one place to which I have a visit pending for my next escape to Berlin.

Saturday, 17 March 2012

The Secret in Their Eyes

I'm rather sleepy so don't feel much like posting anything now, but I've just finished watching this outstanding film and I feel obliged to pay it tribute with a recommendation here

This Oscar winning Argentinian film has really caught me by surprise. I had read some very good reviews, but frankly, I didn't expect so much. I've watched some pretty good Spanish thrillers and dramas, but honestly, I hardly remember having watched any Argentinian film. Though classified as a Crime-Thriller, there's much more to it than that. Yes, the plot revolves around a terrible crime and how over the years the characters involved strive to solve it, but the really interesting thing is how this crime shaped their lives, either ruining them or pouring some more desolation on them or just preventing them from taking the right turn leading them to happiness. In the end, getting to grips with this mystery will exorcise their fears and bring them the strength to start a new path, far away from their empty lives.

To sum up, a slow paced film with a high emotional dose (or maybe it's just that with thirty-something one already has a good bunch of marks in his past that could have used as turning points but for some reason didn't... that all these stories about letting opportunities go by exert a higher effect in me) that I absolutely recommend

Thursday, 15 March 2012

.Net GC update

Time ago I wrote about the .Net GC. When I stumpled upon this news piece in InfoQ talking about GC's latency mode, it sounded like new to me, so I had to read through that old post again to refresh my mind and try to complement it.

So, the thing is that in .Net we have 2 GC modes:

  • Server GC. Under Server GC there's not Background (previously Concurrent) GC. When a GC runs we have a stop the world approach, this should be OK for some server apps where a pause in the application could be attributed to Network latency itself. Under Server GC the latency mode is always GCLatencyMode.Batch
  • Workstation GC. Under Workstation GC we can use the Background (Concurrent) GC or not. By default, the value for GCSettings.LatencyMode should be Interactive, which means that the Concurrent GC is enabled.

    Interactive Enables garbage collection concurrency and reclaims objects while the application is running. This is the default mode for garbage collection on a workstation and is less intrusive. It balances responsiveness with throughput. This mode is equivalent to garbage collection on a workstation that is concurrent. It is not available on the server garbage collector.

    We could dynamically set it to GCLatencyMode.Batch, which would prevent the Concurrent GC. Finally, we have the LowLatency (and now the SustainedLowLatency) mode, that should not be used throughout the whole lifetime of the application, but only at some intervals for which we want to avoid GC pauses (as they exemplify somewhere, we are in the middle of an animation and don't want it to get paused disturbing the user). So, this mode prevents the GC from running, but we should understand that this is risky, cause if we prevent it from running for too long, we could end up with an OutOfMemoryException.

You can read more about this here and here

Wednesday, 7 March 2012

C# vs Java: Exceptions

The Web is full of C# vs Java comparisons, (heated) debates, swears, lies, love, hate... so why shouldn't I add my 2 cents!?

  • I think it's obvious that one of the first differences (annoyance I would call it) that one finds between Java and C# are Checked Exceptions. It's rather odd that such a conservative language like Java decided to include a so experimental feature (suffice to say that no other popular language had or has included it, much less after the disconform and frustration that they seem to cause in many programmers).

    The many people that dislike checked exceptions (I'm not going to discuss if they have some benefits, the net is full of heated discussions about it, but I'm one more of those many people that coming from any of those (many) languages that lack of them finds them disturbing and can't get accustomed to them) try to avoid them to a certain extent by deriving all their custom exceptions from RuntimeException instead of Exception. Anyway, all the Java base library is full of them, so for most common operations (IO, Threading) you'll end up having to deal with them (out of tiredness in many cases it just means swallowing them).

    Due to this general frustration, the designers of more friendly and beautiful languages like Groovy or Scala decided not to care about Checked exceptions. You can invoke a method that throw checked exceptions and the compiler won't force you to wrap it into a try-catch or add a throws declaration to the method signature. Good, but, how do they achieve this?

    Well, thought the throws clausule exists at the bytecode level, the runtime does not give a damn about it when interpreting or JIT compiling those bytecodes (and neither does the bytecode verifier). This means that it's the Java compiler, not the runtime, who's imposing you the checked exceptions restrictions. The Groovy and Scala compilers just don't do that... This nice explanation:

    While the JVM bytecode includes checked exception specifications on methods, the bytecode verifier that runs before any bytecode is executed specifically doesn't check that the methods actually conform to exception specifications. You could write a program that took existing JVM bytecode and removed all exception specifications, and the resulting bytecode would be perfectly valid and run identically to the original (barring reflection).

    can be found here
  • Another interesting difference between Java and C# with respect to exceptions has to do with Exception rethrowing (and this time I would say it's C# who has a "bad" (let's say prone to confusion) behaviour.

    In C#, when we are inside a catch block and we want to rethrow the exception, we have two options: throw ex; and throw;. The former rethrows the exception, but previously removes the stack trace, the latter just rethrows the exception with its whole stack trace.
    In Java we only have throw ex;, that just rethrows the exception with its whole stack trace (same as C#'s throw;). If we wanted to clean the Stack trace we would have to create a new exception and throw it. I think I've never needed to clean the Stack trace before throwing a exception (in those cases where I wanted to hide internals I would be throwing a new custom exception, not a cleaned up version of the original exception, so for me that option given by C# only adds confusion. You can read more here and here.

  • Catching multiple exceptions. Java 7 added Multicatch. You can catch different types of exceptions in the same catch block:

    catch (FirstException | SecondException ex) {
        throw ex;
    So far, in C# you can only catch one type of exception in each catch block, so you would have to use something a bit more verbose like the technique described here
    catch (Exception ex)            
        if (ex is FormatException ||
            ex is OverflowException)
            WebId = Guid.Empty;

  • Automatic Resource Management Java 7 has finally added ARM, that works basically the same as C#'s using blocks (available in C# since the initial version). In Java the objects eligible for ARM have to implement the new Autoclosable interface, in C# they have to implement IDisposable. In both cases what the compiler does under the covers is wrap the code in a try, and add a finally where the calls to close or Dispose are done. A good way to avoid boilerplate code. You can read more here

Sunday, 4 March 2012

32 vs 64 bits again

This post is a bit of complementary of this one I wrote some months ago. This week, someone at work said it was possible to install Windows 7 64 bits on a 32 bits processor. Well, I was pretty certain that was absolutely impossible (same as you can't run a 64 bits application on a 32 bits OS). Obviously the 64 bits OS will make use of the 64 bits registers (otherwise it would not be a 64 bits OS), and those registers are not available in your 32 bits processor (what I've heard is that it's possible to install it on a VM, but I think the performace is terrible, imagine what the VM will have to do to emulate the 64 bits registers...). Anyway, I don't like entering into a discussion before doing some previous extra research.

After checking a few articles I confirmed that I was correct and my colleage confused. A 64 bits OS can't run on a 32 bits processor, same as a 64 bits application can't run on a 32 bits OS. This is one of the best articles that I found, with some rather interesting tables:

Another interesting point they mention there and that should also seem obvious, but I'd never thought of it before, that you can't run 32 bit drivers on a 64 bits OS. The driver's kernel mode code can not run inside the WOW64 emulation system

One more thing that came to my mind is, given than as I explained in my post from October, 64 bits windows applications can't load 32 bits components (dll's, COM components...) what happens for example with Internet Explorer 64 bits and all those intranet applications that use some client side COM component? (we have several of them at work, some of them close to critical)
The answer is here. Your 64 bits Windows system includes 2 Internet Explorer versions, one 64 bits (the default one), and one 32 bits. So for those cases all you need is just launching the 32 bits version.

I think another interesting topic is how much memory is now available to the OS and to an application.

  • In 32 bits systems (processor and OS), we had a maximum of 4 Physical GBs of memory accessible to the OS. That's the maximum address you can reference to from those 32 bits registers. However, you end up having quite less than 4 GBs of physical space to you, normally around 3 GBs. Why? because of Memory Mapped IO. The memory and registers of the I/O devices are mapped to (associated with) the upper physical addresses in the main memory. Bear in mind that then the (let's say) 512 MBs of your Graphic Card are reducing the amount of Physical Memory available to the OS, not because the Card is storing data there, but because those addresses are reserved to communicate with the GC physical memory... That's why usually a Windows 32 bits OS can't see more than 3 GBs of physical RAM...

    With a 64 bits system, as much more memory is addressable the memory addresses reserved for IO mapping will be further up from the top of the Physical memory available in your system.

    In the Virtual Memory arena, in a 32 bits system a 32 bits process has 4 GBs of Virtual Memory available to him, from which the 2 first GBs are reserved for the kernel, so indeed it's only 2 GBs of Virtual Memory what is available to the processor. Well, I can think of many applications for which only 2 GBs of memory seems pretty scarce...

  • In 64 bits systems, the theoretical limit of Physical Memory that can be addressed is 2 to the power of 64 bytes (that's 16 TBs I think), but in practice most OSs (well, at least Windows) impose restrictions (in Windows this goes from 8 GBs to 2 TBs, you can check the specific figures here)

    For the Virtual Memory, the limitations imposed to the Physical Memory are not applied, and then we have that for each 64 bits process there are 8 TBs of User Virtual Memory available to it (the other 8 TBs are for the Kernel).

This is a good read regarding all this.

I guess this write up has helped me understand how important 64 bits Systems are now not just for big servers, but for a normal user (I'm limited to 3GBs of Physical memory in both my laptops, and well, with Visual Studio, Eclipse, FireFox and Lotus Notes running, that´s far from optimus.