Putting Science back into Fiction

Years ago I used to be pretty much into both General Science and Science Fiction, but over the years I've been going away from it, moving towards history/geography/social issues for the former, and towards drama kind of films for the latter. For the first switch I think it's mainly due to having had the chance to do some travelling, which has awaken my early interests (when I was a kid I was crazy about geography), for the second, I just would say that in the last years I haven't come across too many good SciFic films.

With such set up, Europa Report has come as a particularly beautiful surprise. No doubt that the backbone of the story is pretty exciting, the first manned trip to Europa (that Jupiter's moon that many scientists believe could be home to other life forms), but over similarly promising base ideas the ensuing development fails miserably in too many occasions. In this case a powerful and excellent story is built upon it. A bit slow paced, a bit claustrophobic, a bit emotional, with an interesting narrative formula (parts of the story are told by external sources that seems unclear how have gained that information). No unnecessary "easy horrors" or flamboyant FX, but a beautiful portrait of the human thirst for knowledge, for understanding... and the noble willingness to sacrifice it all in that search for answers.

To me, great part of the success of this film lies on how they've been able to weave a passionate work of fiction around an appealing scientific fact, that chances are that Europa hosts life. Furthermore, it seems like they've pretty much stick to current scientific knowledge when depicting the Moon.

The aspect of the moon Europa was based for accuracy on data from NASA and JPL's maps of the moon's surface

Well, I don't think there's much more that I can say other than recommending you to watch it.

Java Closures Limitations

I had read some time ago something about the limitations of Java 8 lambdas with regards to modifying state, but I hadn't had time to test it myself. Finally I've been able to give it a go and here are my findings.

This is one of the most typical examples of closures that I can think of, a function that keeps a counter of how many times it's been invoked:

	public static Supplier<String> getPrinterCounterFails(){
		int counter = 0;
		Supplier<String> f1 = () -> {
			System.out.println("function invokation: " + Integer.toString(counter));
			counter++;
			return "a";
		};
		return f1;
	}

The compiler will reject it with this message: local variables referenced from a lambda expression must be final or effectively final

So well, I felt quite puzzled by this limitation. Closures are functions with state, Java 8 does not seem to provide that, it's more like it provides functions with immutable state.

That said, it's indeed quite easy to work around this limitation. If you're trapping a primitive value, as Java 8 forces you to declare it final you can't modify it's state, but if you Wrap that value into another object, then the reference to that object will be final, but you can change the contents of that object, hence the primitive value that you've wrapped there.

	public static Supplier<String> getPrinterCounter(){
		//don't need to explicitely set it as final, as it's not being reassigned, it's effectively final
		/*final*/ int[] counterWrapper = new int[]{0};
		Supplier<String> f1 = () -> {
			System.out.println("function invokation: " + Integer.toString(counterWrapper[0]));
			counterWrapper[0]++;
			return "a";
		};
		return f1;
	}

It's been many years since the last time I wrote any Python code (I quite liked the language at first, but over time I ended up moving away from it because of the syntax, seriously, I've turned really intolerant to "non-C syntax"), but I think to remember that closures there had this same kind of limitation.

C# closures don't have this limitation, so this code will work nicely.

	public static Action GetPrinterCounter(){
		int counter = 0;
		return () => {
			Console.WriteLine("function invokation: " + counter.ToString());
			counter++;
		};
	}

It's needless to say that the almighty JavaScript also lacks this limitation.

Well, if we think in terms of how closures are implemented, and not in terms of what a closure should be, things look more clear. In JavaScript, variable resolution is based on [[scope]], ExecutionContext Objects and so on. Basically, a function points to an object where all the arguments and local variables are stored, and that in turn points to the same kind of object in the "parent function". These objects form a chain (similar to the prototype chain) and variables are looked up in this chain of objects. With such implementation, it's clear that this limitation can not exist in JavaScript.

C# and Java have nothing to do with the above. While .Net languages and now Java support functions as first class objects, their underlying platforms, the CLR and the JVM do not. I mean, neither of them has the notion of an object being a function. In both cases lambdas (and anonymous methods) will be desugared into normal (oddly named) methods. In C#, if the function needs state (i.e. it's a closure) this method will be created in a separate class containing fields for the state trapped by the closure. I think it's the same that Scala does. Java follows a more complex approach, the method is created inside the current class, and it's not until runtime (through invokeDynamic and Lambda MetaFactory magic) that a new class (implementing the required functional interface) is created. Then, in .Net a delegate object will be used to invoke this method, in Java we'll use directly the functional interface. Please, note that the above explained for java is a rough approximation based on my incomplete understanding of how lambda translation, invokedynamic and MethodHandles work in Java, I plan to write a long post about it once I've had more time to dive into it.

The problem comes for cases where the variable trapped by the closure could be trapped by another closure, or be modified by the outer function where the closure is created. In that case the compiler would need to do some black magic so that those different variables (a local variable in the outer function, a field contained in the generated class for the Closure) are kept in sync, meaning that if one is changed to point to another memory location or contain a different value (reference vs value type/primitive type), the other does so. Java compiler designers decided not to go through that trouble and just force you to declare these variables final (or be effectively final), while C# compiler designers considered it worth the effort. This means that even code like this works nicely in C#

	public static void ComplexTest(){
		int counter = 0;
		Action a1 = () => {
			Console.WriteLine("inside a1, counter: " + counter.ToString());
			counter++;
		};
		Action a2 = () => {
			Console.WriteLine("inside a2, counter: " + counter.ToString());
			counter++;
		};
		Console.WriteLine("ComplexTest, counter: " + counter.ToString());
		a1();
		a1();
		Console.WriteLine("ComplexTest, counter: " + counter.ToString());
		a2();
                a1();
		Console.WriteLine("ComplexTest, counter: " + counter.ToString());
	}

//then invoke ComplexTest
ComplexTest();
//and this is the output
ComplexTest, counter: 0
inside a1, counter: 0
inside a1, counter: 1
ComplexTest, counter: 2
inside a2, counter: 2
inside a1, counter: 3
ComplexTest, counter: 4
output:

Indeed, I had already talked about this more than 2 years ago in this post. You'll find there more information about what the C# compiler is doing and the generated bytecodes.

Albi

Before coming to Toulouse several people told me that for sure I had to pay a visit to Carcasonne, that it was such a beautiful and impressive place. Of course once I'd had a first taste of Toulouse my next steps were visiting Bordeaux (likewise some people had insisted on how nice it was, and yes, it's an incredible place) and then visiting Carcasonne. Obviously I liked the place, "la cité" (the medieval citadel) is impressive, but the rest of the city has no much to see and as my expectations were so high I didn't feel so amazed.

On the other side, when I paid my first visit to Albi, back in February, my expectations were not that high. I mainly expected a smaller version of downtown Toulouse, which of course would be nice, but nothing new to me. Wow, once there I felt fascinated by this "ville", it's much more than a "small Toulouse". Of course, you'll draw much similarities between both old towns (and with Montauban's one), as the 3 of them make up the trinity of Languedoc-style red brick architecture. I was back in Albi a few weeks ago, and the feeling of delight was the same.

My first impressions when walking from the train station to the "Centre de Ville" reinforced that perspective of being in a small Toulouse: the tree lined old boulevards, the Jeanne d'Arc statue (you'll find them in all French cities, but I think this is the only one I remember were she's standing rather than riding a horse), a nice park with astonishing gardens (since I'm here I've gone crazy about gardens, French people have such an incredible taste for gardens, and for architecture and cakes :-) a "Monument aux Morts" that is like a brick brother of the one in Toulouse...

Once in the city center, my first view was that of the Cathedral. Yeah, it's as impressive as all guide books say. The harsh and imposing walls remind me of Los Jacobinos in Toulouse, but taken to another dimension. I remember how much it captivated me many years ago when I read that the main idea of Gothic cathedrals with their large windows and towers and those high ceilings that seem to get lost in heaven... was to make men feel so small before the power of God. That vision is taken one step further here, those walls made me feel before an unconquerable military fortress, and probably that's what their constructors aimed, as it was built in the middle of a land of "heretics" after the bloody Albingensian Crusade. Once inside, things could not get any better, it's astonishing. Those painted ceilings are like a Gothic Sistine Chapel, and then the superb Rood Screen, sporting some of the most elaborate sets of gothic stone filigrees that I can think of.

The old town is much more than the cathedral: the beautiful Saint Salvi church, the red brick medieval buildings everywhere (I've ever felt a fascination for timber-frame houses, but the brick timber-frames that you find in Languedoc are a next level), the so charming Castelviel district... but the real ice of the cake will come when once in the Berbie palace you first get the astonishing views of the garden, the Tarn river with its soft waterfall, the houses leaning over the river and the imposing bridges, it's one of those images that will stay forever in your memory. In a way, this mesmerizing view reminded me of the one that you get from the castle in Cesky Krumlov, though the houses and the river are so different, something established the connection in my mind.

After enjoying the views and the garden for a long while, you should stroll to the river walking through some of the most ancient streets in the city, cross the Pont-View to the right bank, and then cross back to the main bank via the "Pont du 22 Aout 1944" (marking the liberation of Albi from the Nazi scum, just 3 days after Toulouse) that rises proudly over the Tarn for quite some meters (this elevated position could remind you of the bridges in Porto). If you continue along the Boulevard Pompidou you'll end up in another beautiful French garden and enjoy some "modern" (after so much medieval buildings) classic French architecture. Stroll back to the station and carry these memories forever with you. Then, you can even write a crappy post that in no way can express how enjoyable this little city is.

Seul Contre Tous

Seul Contre Tous (I Stand Alone) is a quite shocking film. I was not aware of it until I got it recommended by one friend. Bearing in mind that it's the first work by Gaspar Noé, the guy behind the devastating Irreversible, I should have already laid my eyes on it long ago. Indeed, I find it odd that this film is not framed in the French Extremity genre.

We're before a rather unconventional film: the story, the narrative, the aesthetic... We could consider the leading character as a sort of Antihero, but in my concept of Antihero it's necessary to be able to gain some sort of identification with him, and in this case I can't. Having been a Vegetarian on moral basis for almost 20 years, it's clear that it would be difficult that I could feel any affection for a butcher (that's our man's "profession") from the start, but as the film goes on things don't get any better, he's a piece a shit (no matter how deeply he's been a victim of society), a rather stinky one. At several points of the story it seems like there's going to be a twist and he's going to transform into a hero, but it never happens. First, when he expounds his philosophical views on loneliness, I pretty agree on start up, but then he takes it way too far. Then, when he's in a bar and the bartender rejects to serve a coffee to a well-behaved customer just cause he's an Arab (with a "you can go for a mint-tea somewhere else" and crap like that) you think our man is going to explode and blow the head of that racist scum, but nothing happens, and furthermore, later on he'll end up showing his homophobic and racist side. About his sickening approach to sexuality, well, I think that's all I can say, sickening.

The film is set in the allegedly crisis-riddend late 80's France. I say allegedly cause also at present newspapers here (I'm still living in France, though sadly I think that only for just a few more weeks) are always talking about how stagnant the economy is and how terrifying the unemployment figures are, but being an Asturian, my notion of crisis is somehow different (25% unemployment, rows of closed shops, rows of "Gold buy and sell" shops, very aged population...). There are several shots in the film that I think try to transmit a sense of political unrest: some disgusting far-right graffiti ("our land, our blood"), walls with huge slogans asking for the vote for the PCF (French Communist Party) and CGT (one trade union).

When he was he was strolling along Lille beside that nurse I had the feeling that the architecture looked quite more British than French, well, seems it's not my mind playing tricks as you can read here

.

New IT Jargon

IT is a complex, constantly evolving world where it's rather difficoult for someone with an average brain to keep up with just a small part of the many things one can find interesting. What is quite simple (and sometimes can be pretty helpful) is keeping up with the new vocabulary. When I say helpful, I mean that knowing to name a few of the last cool IT things can make you look smarter than you are (but notice that some managers can consider that knowing what an Acronym stands for is right the same as being an expert on it, so it can end up causing you some trouble).

In the last weeks I've come across with some names/acronyms that can help you look cool :-D

• NoSQL is one of the most misleading terms of the last years. OK, NoSQL stores do not use SQL, but that's not the real point, the real point is that those Data Stores are not based on the Relational Model we find in RDBMS's. So when I read somewhere that some people had started to use the term NoRel I immediately loved it.
• Polyglot Persistence. When I first knew about the term polyglot programming I really liked it, it was a fast way to define what the programming world had evolved into, it's no longer enough to be proficient in one language/platform and blindly choose it for all your projects. I guess with Polyglot Persistence it's going to be right the same.
• BASE. The counterpart to ACID in a NoRel world.
• MEAN. I think it's been some years since the last time I heard about LAMP, and suddenly a modern, geek incarnation of that web stack is here: MongoDB, Express, AngularJS and Node. I'm almost certain I'll never use this combination. I love Node (but mainly as a scripting platform or just as a way to test JavaScript "proofs of concept") and I'm looking forward to have some time to play with Mongo and AngularJS, but I don't see myself doing any personal Web Development project, and in most companies the LOB (Line of Business) web applications are and will continue to be Java or .Net.
• MVW. Probably you're a bit fed up of having to stop to think about the distinction among MVC, MVP, MVVM... to explain someone which one (or variation of which one) you're using. Saying "I'm using Model View Whatever" can come really handy to spare you time in fruitless debates.
• Hoisting. In JavaScript variables declarations (but not initializations) are hoisted (that is, moved to the top). Function declaration (not function expressions) are also hoisted.
• IIEF. If you've been into JavaScript development in the last years is pretty likely that you've done some use of them, but chances are that you don't know their name: IIFE (Immediately Invoked Function Expression), I mean:

  (function () {  // open IIFE
                  var tmp = 100 - x;
                  ...
              }()); 
  

File Locking

I've gone through some problems with File Locking at work lately, so I think I'll write some notes about it here. We had an application appending lines (results) to a csv file in a shared folder. If that file is opened with Notepad++, results will continue to be appended successfully (and notepad++ will ask you to reload the file as it's been modified), however, if it's opened with Excel by someone with write access permissions on that file, new lines will fail to be appended to the file (we won't lose them as in that case we write to a secondary file, with means that in the end we'll have to consolidate both files). So Excel is locking the file while notepad++ isn't.

We requested Write access to be removed for our users, and just in case we also asked them to always copy the file to a local folder and open their local copy. Anyway, we've found lately that errors continue to happen (as we can see that new lines get added to the secondary file). After some bewilderment we finally found the reason, the errors were happening when a user was copying the file to their local folder (the file has grown enough to take some seconds to get copied, time enough for experiencing collisions with the main process trying to append new results to the file).

I have to admit that I had not much clear how "file locking" works, and I was quite confused on some points. The only kind of locking that I had in mind was that 2 processes (threads indeed) can not write to the same file at the same time, but from the above it seems like there are more cases where locking happens (I guess copying a file would mean just opening it in Read mode), so first I've done some basic tests with a .Net program opening a file with a StreamReader, StreamWriter or FileStrem with differen parameters, and trying to read/write from/to that file with some basic commands: echo hi ≶≶ File.txt / type File.txt.
Notice that I've performed all these tests on Windows, I'll test this on Linux when I have a chance.

1)One process opens a file for writing to it (either with a StreamWriter or a FileStream):

using (StreamWriter sw = new StreamWriter(filePath)) //careful, the StreamWriter empties the file!

//or:

using (FileStream fs = new FileStream(filePath, FileMode.Append, FileAccess.Write))

As expected, if another process tries to write to the same file (just a simple echo time >> file.txt), it'll fail with a The process cannot access the file because it is being used by another process error.

Notice however, that there will be no problem to read from this file from another process (a simple type file.txt)

2)Now, let's go for the interesting case, let's open a file for reading(with a StreamReader or a FileStream):

using (StreamReader sr = new StreamReader(filePath))

//or

using (FileStream fs = new FileStream(filePath, FileMode.Open, FileAccess.Read))

and let's try to write to it (again with just a simple echo time >> file.txt), It'll fail So it seems like by default opening a file for reading prevents us from writing to it. This quite fits with the unexpected behaviour that we'd come across when trying to write while copying.
Again, notice that there will be no problem to read from the file from another process.

These results disconcerted me a bit, I guess there has to be a way to open a file for reading while allowing other processes to write to it. Trying to find some more information about File Locking in Windows it happened again that the Wikipedia seems to have just the perfect amount of information.

Microsoft Windows uses three distinct mechanisms to manage access to shared files:

• using share-access controls that allow applications to specify whole-file access-sharing for read, write, or delete
• using byte-range locks to arbitrate read and write access to regions within a single file
• by Windows file systems disallowing executing files from being opened for write or delete access

...

The sharing mode parameter in the CreateFile function used to open files determines file-sharing. Files can be opened to allow sharing the file for read, write, or delete access. Subsequent attempts to open the file must be compatible with all previously granted sharing-access to the file. When the file is closed, sharing-access restrictions are adjusted to remove the restrictions imposed by that specific file open.
Byte-range locking type is determined by the dwFlags parameter in the LockFileEx function used to lock a region of a file. The Windows API function LockFile can also be used and acquires an exclusive lock on the region of the file.

If you look at the different overloads for the FileStream constructor you'll find a few ones with a FileShare parameter. This is what the article refers to as "file access sharing". In my examples, I've not been using any of those overloads, so I guess I'm in this case:

Windows inherits the semantics of share-access controls from the MS-DOS system, where sharing was introduced in MS–DOS 3.3. Thus, an application must explicitly allow sharing; otherwise an application has exclusive read, write, and delete access to the file (other types of access, such as those to retrieve the attributes of a file are allowed.)

I found then this excellent answer, where a guy opens a file with FileShare.ReadWrite and then copies it by blocks. I've put it into my own FileCopyTaker class and done some tests, and it works like a charm. I can copy a large file and at the same time append additional information to it from another process.

I've uploaded it here.

It's clear from this that doing a file copy from Windows Explorer (or via the copy command) is not using this file-sharing feature at all, and one could wonder why. Well, I think that's the correct default behaviour. In most cases, modifying a file while it's being copied would leave the file in an unusable state (when the copy finishes the other process could still be writing information to the file, so it would be incomplete, and in most cases (save for my particular case of appending a line to a text file) it would mean broken.

You'll find people wondering why there's not a Windows function to check whether a file is Locked. The answer is clear, it would not be useful, as you could run into synchronization issues, I mean, given this code

if (!isFileLocked(filePath)){
	//Do stuff with the file
}

between the moment when isFileLocked returns false, and the moment you start to do things with it, another process could be locking the file and your code would crash... so in anyway, you should always put your file access code into a try-catch to deal with the possibility of the file being locked

Thread Suspend, Resume and Abort

At first sight aborting, suspending or resuming threads (in the .Net arena) seems pretty simple, the Thread class sports methods for each of these 3 tasks. However, if you check the documentation, you'll see that both Suspend and Resume are marked as obsolete and highly discouraged:

Do not use the Suspend and Resume methods to synchronize the activities of threads. You have no way of knowing what code a thread is executing when you suspend it. If you suspend a thread while it holds locks during a security permission evaluation, other threads in the AppDomain might be blocked. If you suspend a thread while it is executing a class constructor, other threads in the AppDomain that attempt to use that class are blocked. Deadlocks can occur very easily.

As for the native Win32 funcion SuspendThread you'll find equally discouraging statements:

This function is primarily designed for use by debuggers. It is not intended to be used for thread synchronization. Calling SuspendThread on a thread that owns a synchronization object, such as a mutex or critical section, can lead to a deadlock if the calling thread tries to obtain a synchronization object owned by a suspended thread. To avoid this situation, a thread within an application that is not a debugger should signal the other thread to suspend itself. The target thread must be designed to watch for this signal and respond appropriately.

Thread.Abort is not marked as obsolete and no advice against its use is explicitly given in MSDN, but you'll find multiple articles and discussions (like this pointing against its use, just for the same reasons as Suspend.

The basic idea is that these methods are too invasive, they'll just Suspend or Abort a thread irrespective of what it's doing at that moment, which can be pretty risky. We need then a more collaborative way to do this, basically a Thread should be periodically stopping to do its main task and checking for Suspend or Abort requests at moments when it can really proceed with such order, aborting/suspending itself accordingly. Obviously the "periodically stop to do its main task" thing can be complicated to accomplish depending on what that main task is... but that's apart from the suspend/abort logic.

For Abort it seems pretty simple. A given thread would just check for an Abort flag, and when true, it would just end.

while(!this.abort){ //check for Abort requests
     DoWork();
}

For Suspend/Resume it's a bit more complicated, we can check for a Suspend flag, but once suspended, how do we check for the Resume order? Well, signals are the main communication mechanism among threads, so we should just use that, a ManuaResetEvent initially set as signaled, we would indicate a Suspend request by resetting it, and then send a Resume command by setting it again.

while(!this.abort){ //check for Abort requests
     DoWork();
     this.resumeSuspend.WaitOne(); //check for Resume/Suspend requests
    }

To really understand it you'll have to see this full sample