Friday, 29 January 2010

Some Linq definitions

Sure Linq is nothing new at this point in time, but still today I think there are some misconceptions, terms wrongly used... and I think a short recap can be useful (at least as an easily accessible note for myself)

not exclusive to Linq, but highly related.
A Lambda is just an abbreviated way to create an anonymous delegate or an Expression<TDelegate>. It makes heavy use of the type inference feature added in C# 3.0.
Lambdas have three parts

  • the input parameters on the left

  • the => operator (that reads like "goes to")

  • the expression or statement on the right

Based on this third part we have 2 types of Lambdas:
Expression Lambdas and Statement Lambdas.

  • Statement Lambdas have brackets that delimit a "statement block". This block can contain any C# code, so this type of lambdas can be used instead of any delegate. It's just syntax sugar that the compiler turns into a delegate.

    sample - you can check this with our good friend Reflector :-)

    c =>{ Console.WriteLine("called");
    return c.StartsWith("L");

    is equivalent to this:

    delegate (string c) {
    return c.StartsWith("L");

  • Expression Lambdas contain just one expression. This is the kind of lambdas more commonly used. Again this is just syntax sugar, and depending on where they're being used, compiler can turn them into a delegate or an Expression<TDelegate> (an expression tree).

    When we have this code (the Where here is the System.Linq.Queryable.Where extension method, that expects an Expression<TDelegate>):

    MyDataContext dc = new MyDataContext();
    dc.Contacts.Where(contact => contact.FirstName.StartsWith("Greg"));

    this is translated into this:

    contacts = dc.Contacts.Where(Expression.Lambda<Func<Contact, bool>>(Expression.Call(Expression.Property(CS$0$0000 = Expression.Parameter(typeof(Contact), "c"), (MethodInfo) methodof(Contact.get_FirstName)), (MethodInfo) methodof(string.StartsWith), new Expression[] { Expression.Constant("Greg", typeof(string)) }), new ParameterExpression[] { CS$0$0000 }));

    however, if we have this code (the Where here is the System.Linq.Enumerable.Where extension method, that expects a Func<T, bool> delegate):

    List cities = new List(){"Berlin", "London", "Xixon", "Llangreu"};
    results = cities.Where(c => c.StartsWith("L"));

    it gets translated into something like this:

    results = cities.Where(delegate(string c){
    return c.StartsWith("L");

    Of course Expression<TDelegate> (Expression Tree) are terribly important for IQueryable Linq Providers (like Linq to Sql). An Expression Tree is just an AST that gets turned into code (for example TSql when that IQueryable Provider is Linq To Sql) at runtime. It's easy to see that this is a rather complex thing and that's why only expression lambdas can be turned into Expression<TDelegate> (I'm talking about .Net 3.5, fortunately that will be different with the advent of .Net 4.0 and DLR)

Given an Expression<TDelegate> we can turn it into a TDelegate just by calling its Compile methods (this is really beautiful, at compile time we just create an AST and it's at runtime when it gets turned into CIL). Nevertheless we can't convert a "good candidate" delegate (I mean, one which body is just one expression) into an Expression<TDelegate>. There's not a "magic" method to do that at runtime, it's just "compiler magic" what turns a lambda expression into an Expression<TDelegate>. To do this at runtime we would need to get the CIL for that Delegate (that's easy: myDelegate.Method.GetMethodBody()) and then translate it into an Expression Tree (that's not easy...) I've done some googling and haven't found any implementation of this.

When writing "Linq code" we have two ways (two syntaxes) to write our code:

  • Query Syntax (Query Expressions)

  • Method Syntax (Standard Query Operators Method Calls)

The first one is just syntax sugar that gets translated by the compiler into method calls. That means that this:

//Query Syntax
IEnumerable<string> results = from c in cities
where c.StartsWith("L")
select c;

and this:

//Method Syntax
IEnumerable<string> result = cities.Where(c => c.StartsWith("L"));

are just the same.
At a compiler level I'm not sure if we have a two pass thing or not. I mean, maybe the compiler does some preprocessing turns the first code block into the second one, and then the CS to CIL compilation is done, or maybe the first code block is directly compiled to CIL.

Monday, 25 January 2010

Carriers (Infectados)

I watched this film last weekend. It was a rather pleasant surprise to me. I had not read any critics about it, I just recalled having seen some trailer somewhere when it was released that made me add it to my list of medium priority films to watch some day.

I was expecting the more or less typical apocalyptic film where some sort of infection is wiping humanity from the face of the Earth, turning humans into horrific zombies and giving us a good dose of action and violence... So yes, I was thinking more in something like "28 (days-weeks) later", "Resident Evil" or even "Hills have eyes", films, all of them, that I rather like.

The thing is that I was rather mistaken about the plot. Humanity is on the verge of extinction for some unknown infectious disease, but there are no zombies anywhere, people just get infected, deteriorate and die, without entering in any blood thirsty, violent zombie state. So, we have a pandemic situation that is the basis for a drama in which the 4 main characters will confront us with the pain of losing your loved ones, taking very harsh decisions, trust and mistrust, pursuing a very distant hope, and, in the end, loneliness.
I don't like giving a numeric rate to things cause I think my ratings would fluctuate too much from one moment to another, so I just will say that this is a must-see film.

Thursday, 21 January 2010

Func vs Predicate

These last days I've been giving a short course on .Net development at work to some colleages with a Java background (poor guys, having had to listen to my ramblings on programming sure has not been an easy task :-)

Thing is that having talked first about the generic delegate types (Predicate<T>, Action<...> and Func<...>), when then I moved to Linq to Objects and showing a sample with the Where method, I was assuming this method would expect a Predicate<T>. So, when I opened the SDK documentation it came as a surprise to me that in fact it expects a Func<T, bool> (normally you just use a lambda and let the compiler the task of replacing it with the appropiate delegate, so you don't need to pay much attention to the specific delegate type used).
If we already have a Predicate<T> type, that returns a bool, and which name follows semantics commonly used in other environments for similar purposes, a function returning true or false, what's the sense of using Func<T, bool>?
It seems less meaningful to me (OK, you inmediately read the bool thing, but anyway Predicate seems more clear to me).

Of course, this is a rather unoriginal question that many other have done before, so checking this thread in stackoverflow, I found an explanation by Jon Skeet that I rather like:

I've wondered this before. I like the Predicate delegate - it's nice and descriptive. However, you need to consider the overloads of Where:

Where<T>(IEnumerable<T>, Func<T, bool>)
Where<T>(IEnumerable<T>, Func<T, int, bool>)

That allows you to filter based on the index of the entry as well. That's nice and consistent, whereas:

Where<T>(IEnumerable<T>, Predicate<T>)
Where<T>(IEnumerable<T>, Func<T, int, bool>)

wouldn't be.

Based on other answers, we could think that if Predicate<T> and Func<...> had been introduced in the same version of the Framework, maybe Predicate<T> would have not made it in favor of Func<T, bool>, but well, there are other cases where having a Delegate Type with a very semantic name has seemed useful to the Framework Architects, that's the case of Converter<TInput, TOutput> that could have been omitted in favor of Func<TInput, TOutput>.

Friday, 15 January 2010

Assumptions revisited

"Life is suffering" is a common simplistic approach to Buddhist teachings, "Life is learning" is one of my prefered approachs to existence. What I mean with this is that learning new stuff is one of the things that better manages to make me feel very alive.
Finding out that some of my previously acquired pieces of knowledge has been reshaped by science, experience or whatever, is something that I find terribly attractive.

In the last weeks I've came across a couple of these things:

  • I had been taught that over the centuries that evolution moved the human larynx to a lower position to allow us produce a higher variety of sounds. In principle this feature was unique to humans, being one of the anatomical basis for our sophisticated language (and with it, one of the basis for our complexity as species). This descended larynx means that we can choke easier when eating, but this downside seems more than worth bearing in mind all the advantages that language gives us.
    Well, it seems larynx location is not that unique as we thought... In the interesting BBC Horizon documentary "Why do we talk?" they show that recent investigation in the University of Vienna seems to have proven that other animals are able to lower their larynx when barking, growling or whatever, but anyway they have not developed a basic talked language. They can dynamically lower down their larynx and reconfigure their vocal tract into a human like configuration, so the constraint that is keeping chimps or dogs from developing a basic vocal language is not an anatomical one, but purely a matter of brain configuration.

    This makes me thing that the evolutionary decision of lowering down our larynx was not a so clever one. I guess it would be better having it a bit upper, so we could not choke accidentally while eating, and then dynamically lower it down when we want to talk (well mannered people don't eat and talk at the same time :-D
    This should be no wonder, unfortunately evolution does not make big plans looking further into the future, it just picks the best solution that at a given moment random mutations provide us. As species and environment evolve, previous decisions could be not so fitting for the current scenario. I guess an "Intelligent designer" would be intelligent and generous enough to "design for the future" and give us solutions that would not look inappropiate after just a few milleniums... Would like to know what excuse creationists can make...

  • Another common myth is the one that says that with respect to taste, our tongue is divided in several highly specialized areas, each of them responsible for sensign one specific taste. Well, this theory, commonly known as Tongue map seems to be absolutely false. I had already read about it sometime ago, and after watching the documentary about Taste in this interesting series of documentaries about The Human Senses (in Spanish language here) I felt compelled to check what wikipedia had to say about it.
    To sum up, this "Tongue map" theory has been disproven, and every part of the tongue includes receptors for every basic taste.

Thursday, 7 January 2010

Open Delegates, Closed Delegates...

Now that we're so close to welcome the new, feature packed, so longed C# 4.0, it seems like a good idea to me to review some of the features that were added to the language back in 2005 (in its 2.0 incarnation) and that not having been so publicized as Anonymous Delegates (and the consequent lovely Closures), Iterator Blocks... we could have missed (in fact, I was not aware of these 2 improvements I'm going to talk about until 1 year ago).
So, these new (in 2005...) characteristics are Open Instance Delegates and Closed Static Delegates.
Maybe both characteristics could be considered part of the Delegate Relaxation concept that sometimes we can read in some pages, though in principle this Relaxation thing is used as a synonymous to the Delegate Variance characteristics also added in C# 2.0 (I plan to write something about variance in the near future).

As we all know, delegates are not much more than a reference to a method (well, a MethodInfo object pointed by the Method property), and a reference to the "this" object for that method (the Target property). If the delegate points to an Instance Method, the Target property points to the instance object that we used when creating the delegate. If the delegate points to a static method this Target property is null.
When we go to Native Code, the "this" object is passed to the function (method) as just one more parameter. So, when in runtime the JIT translates the CIL code to Native Code, each call to a delegate expecting n parameters involves calling to the native code representation of the MethodInfo, pushing into the stack those n parameters plus (in first position) the object pointed by the Target property (if it's not null). So it seems it should be feasible, fun and useful to play with combinations of Delegate Type and Target values to in the end invoke a method (instance or static) with all the needed parameters expected at the low level.
For example, it could seem useful to be able to alter an existing delegate object to change the "this" (Target), without having to resort to creating a new delegate object.
It can be done with CIL or it can be done with Reflection (and I should paste some example here...)

Or, instead of using a Closed Delegate (one with the Target Property pointing to somewhere), we can use an Open Instance Delegate (usually shortened as Open Delegate) which Target is null. So, in this case, how do we pass the instance parameter?
easy, we pass it as the first parameter in the Delegate invocation. What this means is that if we have an instance method of the Person class, that expects one string and one int as parameters, we'll have to use a delegate type that expects a Person, a string and an int as parameters. OK, and how do we create this Open Delegate?

Suppose we have a Person class with a method like this:
public string DoSomethingInstance(string st, int num) {return "";}

Given that the normal, closed delegate thing works like this:

 Person p1 = new Person(){Name = "Iyan"};
//the normal way, create a closed delegate
Func<string, int, string> closedDelegate = new Func<string, int, string>(p1.DoSomethingInstance);

we would like to be able to do this:
 Func<Person, string, int, string> openDelegate = new Func<Person,string, int, string>(p1.DoSomethingInstance);

but this is not possible (compile time error), it seems as though the compiler can not distinguish if we're trying to create an Open Delegate there or we're just making a mistake.

so, what other way we have to create delegates? well, we have the Delegate.CreateDelegate method, that we normally use when we depend on runtime information, but that this time could also be the way to go. There's a whole bunch of overloads for this method, but if we check just the first one Delegate.CreateDelegate Method (Type, MethodInfo), after a disappointing "Creates a delegate of the specified type to represent the specified static method" if we read on we find a promising:
"The MethodInfo describing the static or instance method the delegate is to represent. Only static methods are supported in the .NET Framework version 1.0 and 1.1."

so, if we write:
 MethodInfo mInf = typeof(Person).GetMethod("DoSomethingInstance");
Func<Person, string, int, string> openDelegate = (Func<Person, string, int, string>)(Delegate.CreateDelegate(typeof(Func<Person, string, int, string>), mInf));
//now let's call our Open Delegate with different instances of Person
Console.WriteLine(openDelegate(p1,"formatting: ", 3));
Console.WriteLine(openDelegate(new Person(){Name = "Xurde"},"formatting: ", 4));
Console.WriteLine(openDelegate(new Person(){Name = "Nel"},"formatting: ", 2));

and test it, it works fine!

And now, let's think the other way. Could it be possible to point to a static method with a delegate which has bound as Target the first parameter expected by the static method? (when we create a delegate to a static method the normal way Target is null).
Yes, it's possible, and is called Closed Static Delegate. It does not seem so useful to me as the Open ones, but anyway if we're going to call that method with the same first parameter but varying other parameters, it can be helpful. After all, it's not much different from the partial function application that we have in Python or in the all-powerful JavaScript.

so, if we have a:
public static string DoSomethingStatic(string st, int num){return "";}

we can do (again we have to resort to one of the Delegate.CreateDelegate overloads):
string st = "Nenyures";
MethodInfo mInf2 = typeof(Person).GetMethod("DoSomethingStatic");
Func<int, string> closedStaticDeleg = (Func<int, string>)(Delegate.CreateDelegate(typeof(Func<int, string>), st, mInf2));

and now that we've bound the string "Nenyures" to the static method wrapped in the delegate (i.e. assigned it to the Target property), we can do a call like this:


To sum up:
By default when we create a delegate pointing to an instance method, we're creating a closed delegate (Target property points to the instance object used when instantiating the delegate) and when we create a delegate poiting to a static method, we're creatina an open delegate (Target property is null), but all this can be altered for fun and profit...

you can check the source here.

Saturday, 2 January 2010

Allergy Planet

I'm really much into watching documentaries (general science, history, social issues, trips..) and BBC Horizon is always a reference place to search for new ones (mainly in the "soft science" arena). Sure they have some dull, boring ones, but they have some absolutely fantastic ones that give me that portion of "food for brain" that I'm always eager for.

Last week I watched one of their best episodes in a while, Allergy Planet.
Yes, I assume you've guessed it's about allergies. In the last 50 years the number of people affected by allergies (and the types of allergies) has risen sharply, and it's getting even worse, being a really serious health problem.
Allergies are basically a failure of our immune system that starts to respond to harmless substances (allergens).

I had already learnt some years ago that one of the explanations for the increase in allergies is that in our "hyper clean" (wash your hands again and again, scrub the floor again and again...) developed societies, we get exposed to a very limited number of pathogens, so our immune system does not get enough training, and being unable to distinguish the "bad guys"(pathogens) from some of the harmless ones (most of the allergens), it starts unnecessary (and harmful) immune responses. Related to this, the documentary explains how the absence of gutworms seems to increase the likelihood of developing allergies. IgE, the antibody involved in most allergies, is also used to fight gutworms. Evolution provided humans (for which gutworms used to be a serious issue) with the ability to produce good amounts of IgE. Problem is that now, good part of the population is no longer affected by gutworms and this IgE, having lost his job (binding to these pathogens), decides to reemploy himself in binding to innocuous allergens (giving rise to allergic reactions). This is one more of the problems (the other one I can think of is Stress) caused by the fact that in the last centuries Natural Evolution has lagged behind the evolution of our societies.

All this reminds me of something I read somewhere some weeks ago, that guys born through cesarean could be missing contact with some bacteria that would play an important role in the development of their immune system (babies born in the "traditional way" get exposed to bacteria in mum's vagina that will help them to develop their immune system, well, there are some nuances here, in some cases he can get exposed to life threatening bacteria or viruses... biology is too complex...)
All this does not mean that "all dirt is good", as air pollution (that exposes us to many allergens) seems to be an important cause for allergies.
There's another case in which too much hygiene increases the likelihood for developing allergies, this time not for underexposure to pathogens, but for overexposure to allergens. Washing up our hands too often may lead to our skin barrier (an outer layer of our skin that protects us from external elements) getting damaged, allowing too many allergens to enter our body and making our Immune System go nuts.

So, to sum up, some level of exposure to pathogens is good for our Immune System to get trained, and low level of exposure to allergens helps us avoid allergies.

Other great BBC Horizon documentaries that I've watched in the last months:

  • The Secret life of your body clock

  • God on the Brain. Abut brain conditions that influence religious experiences. Here I learnt a new cool word, neurotheology.

  • How to survive a disaster One of the most interesting things here is how our fear to ridiculous (for example, being the first one in raising the alarm when some smoke starts to get into the room) can lead us to total disaster.

  • Not a BBC Horizon one, but given that I mentioned stress in one of the paragraphs above, this is a must addition to this list. Stress, portrait of a killer with the great Robert Sapolsky. This is the guy that time ago said in an interview in Redes this great sentence (I'm translating it from Spanish from my distant memories, so it's anything by literal): You can't be a biologist if you haven't repeated this sentence at least 1000 times: genes do not determine anything, they determine something under a specific environment