AsyncEnumerable

This is a topic that has got me confused a few times, and now that I have seen a reference to something similar for a future ES version, I thought of writing a short reference here.

Let's start by a necessary clarification. When we talk about enumerating/iterating/looping in an asynchronous way, there are 2 quite different things:

• Obtaining the item is not time costly and asynchronous, what is asynchronous is treating the item. This used to be the most common case for me [1] 2. It's what I've usually called async-loops. You would have a list of items and an asynchronous function to run on each of them, and you will pass as callback a function to continue with the iteration. I've written about it a few times. With the advent of async the code is now pretty simple, as the compiler takes care of all the heavy lifting. We now can write code like this (in C#):

  foreach (var item in myEnumerable)
  {
  await treatItemAsynchronously(item);
  }
  

• The other case is when obtaining the iteration item is time costly and hence implemented asynchronously. Thinking in terms of C# and IEnumerable/IEnumerator, that would mean having a sort of async MoveNext. Then, the treatment of the item could also be asynchronous, so we would have case 1 and case 2 together.

This post is focusing on the second case. It would be nice to be able to write something like this (which is just "fiction syntax":

foreach await (var item in myAsyncEnumerable){}

I've read somewhere of awaiting for a method returning a Task<IEnumerable<T>>. That makes no sense. What we could do is to return an IEnumerable<Task<T>> and use it this way:

class FilesRetriever
{
public IEnumerable<Task<String>>> GetFiles(){...}
...
}

var filesRetriever = new FilesRetriever(new List<string>(){"file1", "file2"});
			foreach (Task<string> fileTask in filesRetriever.GetFiles())
			{
				var fil = await fileTask;
				Console.WriteLine(fil);
			}

That is not so bad, but there is a problem. It works fine for cases where the the stop condition of the Enumerator (MoveNext returning false) is known beforehand, for example:

		private string GetFile(string path)
		{
			Thread.Sleep(1000);
			return "[[[" + path.ToUpper() + "]]]";
		}
		
		
		public IEnumerable<Task<String>> GetFiles()
		{
			foreach (var path in this.paths)
			{
				yield return Task.Run(() => {
				                      	return this.GetFile(path);
				                      });
			}
		}

But if that stop condition is only known depending on the iteration item (stop when the last retrieved file is empty for example), this approach would not be valid.

We could think then of some interface like this:

	public interface IAsyncEnumerable
	{
		async Task<bool> MoveNext();
		
		async Task<T> GetCurrent();
	}

that could be combined with a new foreach async loop construct. The loop would get suspended and the execution flow of the current thread would continue outside this function. Then once the Task.Result of that MoveNext is available another thread would continue (the sort of ContinueWith continuation) with GetCurrent, its treatment and the next iteration of the loop. This feature should come with the possibility of doing yield return await.... I assume combining the compiler magic used for yield and await will not be easy.

I've read that there are some requests for something similar

, and one guy implemented a pretty smart alternative back 5 years in time

In ES land, they got the async/await a bit later, but they are striding to get this async iterators thing in the short term. You can read about the proposal here