Local Functions and Iterators

In the end I find the introduction of Local Functions in C# 6 really useful, as I have learnt a couple of things reading about their uses. Thanks to them I've got a refresh and improvement of my understanding of how the compiler manages Iterators and Async methods. As explained here, in both cases local functions provide a useful pattern to manage validation in Iterators and Async Methods. This post will focus on Iterators.

I've had since its inception a more or less clear a view of the magic used by the compiler when it comes across an Iterator method. It creates a class implementing both IEnumerable and IEnumerator and moves the code of the iterator method into the MoveNext method in that class (that acts as a sort of State Machine), replacing the "yield" keyword (that does not exist at MSIL level) with assignments to the "Current" support field. No need to enter into more details when you have a perfect explanation here.

The thing is that maybe one could think that when we have code like this:

public IEnumerablet<string> GetMainDocuments(string url)
{
 if (url == null){
  throw new Exception("Empty!");
 }
 for (var i=0; i<5; i++){
  yield return downloader.download(url + "/item/i");
 }
}

The compiler could put the code that goes before the loop in a separate method, not in the MoveNext. A method that would be executed before we started the iteration with MoveNext, just invoked from the constructor of the state machine for example. This would make sense for certain cases of validation code, where we could want an immediate crash rather than a delayed crash when we decided to iterate. But we have to be aware that one of the basis of iterators is "lazy evaluation/deferred execution". In a way they are a bit like "promises", we have a "promise of a sequence" but each of its items does not materialize until the moment when we ask for it in the next iteration step. Have this in mind, and think that there are many cases where the "pre-iteration" code should be done just at the moment of starting to iterate: maybe it returns values used during the iteration, maybe it's still a validation, but one that depends on dynamic factors and has to be done just at the moment of starting to iterate, not before. So as the compiler can not differenciate those cases, absolutely all the code in your iterator method is put inside the MoveNext of the state machine.

For example think about this case:

public IEnumerable<string> GetMainDocuments(string pattern, DBHelper dbHelper)
{
 if (pattern == null){
  throw new Exception("Empty!");
 }
 
 if (!dbHelper.IsDbUp()){
  throw new Exception("DB down!!!");
 }
 
 IEnumerable<string> documents = dbHelper.GetDocuments("whatever query " + pattern + " whatever");
 
 for (var i=0; i<5; i++){
  yield return downloader.download(url + "/item/i");
 }
}

For the first contitional it would be useful to do it in a no deferred way, but for the second conditional, it makes more sense to do it just at the moment when data start to be needed. And for the third, again it makes more sense to do it not deferred (or maybe you wanted a snapshot at the moment when GetMainDocuments was called?) Yes, there are many possibilities, and the compiler can not just guess. So just have to adapt to his rules. Anything, inside the iterator will be deferred, so if there is code that you don't want lazy you'll have to separate it yourself

public IEnumerable<string> GetMainDocuments(string url)
{
 IEnumerable<string> getDocuments(string url){
  for (var i=0; i<5; i++){
   yield return downloader.download(url + "/item/i");
  }
 }
 
 if (url == null){
  throw new Exception("Empty!");
 }
 return getDocuments(url);
}
 

I have no idea of how the different JavaScript engines manage generators, but the issue with deferred execution is the same. Nothing of the code in your generator function will be executed until you start to iterate the created generator object.

function* nameGenerator(){
 //this console.log won't be executed until the first call to "next"
 console.log("inside iteration");
 yield "xuan";
 yield "xana";
 yield "iyan";
};

let gen = nameGenerator();

console.log("generator created\n");

console.log("starting loop");
for(let name of gen){
 console.log(name);
} 

//output:

// generator created

// starting loop
// inside iteration
// xuan
// xana
// iyan