Peter Ritchie's MVP Blog : Async Functions, .NET Development

C#, Async, Limits, oh my!

PeterRitchie — Thu, 19 Jan 2012 18:54:35 GMT

One of the great sessions at Codemash was a dual-speaker session with Bill Wagner and Jon Skeet—Async from the Inside.

In that session Bill and Jon describe (in great detail) the state machine that the C# compiler generates when it compiles async code involving the await keyword. When the Async CTP was released this state machine was one of the first things I noticed when I was reflecting through some generated code. I too noticed the type of the state variable (int) and wondered, at the time, if that would be an issue. All the information Bill, Jon and I portray is what we’ve learned from reflection, none of this is “insider information”.

Some background: a state machine is generated for each method that uses the await keyword. It manages weaving back-together the asynchronous calls that you so lovingly linearly describe in an async method. The state machine not only weaves together the various “callbacks” that it generates, but it also (now) manages the state changes of the data being modified between and by asynchronous calls. Wonderfully complex stuff.

The state variable is basically an enum of the various states between async calls that the generated code can be in. I won’t get into too much detail about that (you should have gone Codemash like all the other cool people :) but needless to say having a “limit” on the number of states a method can be in sounds a bit scary.

Of course Jon, at one point, brought the same thing up in the session about the “int” state variable. This reminded me that I wanted to look into it further—not because I wanted to break the compiler (directly) but to know what the limits are, if any.

A couple of days later while I had some time waiting around at airports on my way home. I thought I’d test my theory. If you follow me on Twitter you probably saw some of my discoveries in real time. For those of you who didn’t, here’s an aggregation of what I found.

First of all, the state machine represents the async states a single method can be in. This is represented by a signed int. the only negative value that seems [1] to mean anything is –1, leaving 2,147,483,647 states (or, roughly, await invocations) that can occur in an async method.

First glance, this seems disconcerting. I quickly wrote some code to generate a method with 2,147,483,648 await invocations in it (that’s Int32.MaxValue + 1 for those without OCD). Needless to say, that took a few minutes to generate on my laptop, and I have an SSD (which I’ve clocked at increasing my IO by about 7 times on average). That generated a 45 gigabyte file (one class, one method).

Problems with the type int for the state variable started to seem ridiculous. But, I didn’t stop there.

Now I’m doing everything outside of Visual Studio (VS) from this point on. I’m running the app to generate the code from within VS; but everything else is from the command-line. So, I run CSC on the 45 gig file honestly expecting and internal compiler error. But, what happens is that I get a CS1504 error ‘file.cs’ could not be opened (‘Arithmetic result exceeded 32-bits. ‘). Now, the state variable is 32-bits so, it sounds like that could be the culprit. But, if you look at the error message, it can’t even open the file. I tried opening the file in VS and it told me the file couldn’t be found…

Okay, at this point it’s seeming even more unlikely that a type of int for the state variable is even remotely going to be an issue. But, now I’m curious about what 32-bit value has been exceeded. My theory is now the number of lines in the file… The compiler has to keep track of each line in the file in case it has to dump an error about it, maybe that’s the 32-bit value? I modify my code generation to limit the the number of await invocations so the number of lines in the file is 2,147,483,647 (kind of a binary search, if this works then I know it still could be the number of lines in the file). Same error.

The error isn’t from the # of lines. Now my theory is that the overflow is from trying to allocate enough memory load the file (keep in mind, I’ve got 8 GB of RAM and I’m trying to load a 45GB file; but, I have yet to get an out of memory error). So, I modify my code to generate a file that is approaching 2,147,483,647 bytes in size. Things are much faster now… I try again. Now I get the same CS1504 error but the message is ‘file.cs’ could not be opened (‘Not enough storage available to complete this operation’) (I’ve got 100 GB free space on the drive…). Interesting. I’ve lessened the data requirement and only now effectively getting “out of memory” errors.

Now I’m just looking for a file that the compiler will load—I’ve given up on some super-large number of await statements… Longer story, short, I kept halving the size of the file until I reached about 270 megabytes in size then the compiler finally succeeded (meaning ~540 Megabytes failed).

At this point, I’ve successfully proven that a type of int for the state variable is not an issue. If the compiler could load the 540 megabyte file and I somehow could use 8 bytes per await invocation (“await x;” for example) then I could never reach more than about 70,778,880 await calls in a single method. Of course, I’m way off here; that number is even much lower; but 70,778,880 is about 3% of 2,147,483,647. Clearly int is the smallest type that could store anything close to 70,778,880 or less…

Of course, I’m completely ignoring the fact that a 540 MB cs file is completely unmanageable in VS or a project in general; but, why get caught up in silly realities like that.

This state machine is almost identical to those generated by enumerator methods (yield return). If we assumed that the async state machine generation code is “inherited” (by pragmatic reuse) from the enumerator method state machine generator, we can assume it has very similar limits (but even smaller)—meaning you’d never get close to overflowing its int state variable.

HTMYL.

[1] Again, this all observed; I’m surmising the compiler never uses any other negative value.

Testing Strategies Involving Async Functions

PeterRitchie — Thu, 04 Nov 2010 16:26:00 GMT

Some things to keep in mind when writing units tests for code that use async methods: You're not trying to test the framework's "awaitability" and you're not trying to test framework methods that are "awaitable". You want to test your code in certain isolation contexts. One context, of course, is independent of asynchronicity--do individual units of code that don't depend on asynchronous invocation "work"... e.g. "Task<string> MyMethodAsync()", you want to have a unit test that make sure this method does what it's supposed to do (one being it returns a "valid" Task<string> object, the other being that individual side-effects occur, if any). Another context is dependent on asynchronicity--do individual units of code that do depend on asynchronous invocation (or depend on something being invoked asynchronously) "work".

It's the second context that seems to be the hardest to grasp for most people to grasp and action. Let's take this example code:

	private async void startButton_Click(object sender, EventArgs e)
	{
		try
		{
			startButton.Enabled = false;
			var webRequest = WebRequest.Create("http://google.ca");
			using (var response = await webRequest.GetResponseAsync())
			using (var stream = response.GetResponseStream())
			{
				if (stream == null) return;
				using (var reader = new StreamReader(stream))
				{
					textBox.Text = await reader.ReadToEndAsync();
				}
			}
		}
		finally
		{
			startButton.Enabled = true;
		}
	}

A couple invariants that we want to test might be that the button is disabled during the asynchronous operations and enabled after the asynchronous operations. It's not immediately obvious what to in order to validate these invariants.

When the compiler encounters the await operator, it actually goes searching for method that can act upon the type of the object being returned by the method used with the await operator. In our first use of await (on GetResponseAsync()) the return type is Task<T> (Task<WebResponse> specifically, but for our example I'll use Task<T>). There's various criteria the computer uses to search for this method, one method is to search for extension methods that match the name and return an "awaiter" type that has the following methods: BeginAwait and EndAwait (more details can be found in the Aync CTP documentation). The compiler doesn't allow us to inject an awaiter type in the normal run-time dependency injection semantics; but it does allow us to implement an awaiter that does support dependency injection at run-time. To do this I would create an ITaskAwaiter interface like this:

	public interface ITaskAwaiter<out T>
	{
		bool BeginAwait(Action continuation);
		T EndAwait();
	}

I then need to create a GetAwaiter(this Task<T>) extension method to return an implementation of this type. The built-in System.Runtime.CompilerServices.TaskAwaiter<T> is public; but, unfortunately it's constructor (and the ability to pass in a Task<T>) object is internal. So, we can't simply wrap the built-in awaiter and delegate to it as a default. We actually have to write an awaiter that mimics what the built-in one does. For example:

	public class TaskAwaiter<T> : ITaskAwaiter<T>
	{
		private readonly Task<T> task;
 
		public TaskAwaiter(Task<T> task)
		{
			this.task = task;
		}
 
		public bool BeginAwait(Action continuation)
		{
			if (task.IsCompleted) return false;
			var synchronizationContext = SynchronizationContext.Current;
			Action<Task> action = theTask => 
			                      	{
			                      		if (synchronizationContext != null)
			                      		{
			                      			synchronizationContext.Post(state => continuation(), null);
			                      		}
			                      		else
			                      		{
			                      			continuation();
			                      		}
			                      	};
 
			task.ContinueWith(action, CancellationToken.None, TaskContinuationOptions.ExecuteSynchronously, TaskScheduler.Current);
			return true;
		}
 
		public T EndAwait()
		{
			return task.Result;
		}
	}

This class is given the Task<T> in question, it implements a BeginWait method that sets up an Action delegate to invoke the continuation given to it in a specific synchronization context, then tells the task to use that new action as it's continuation via a call to ContinueWith. When the task is completed EndAwait will be called and we simply return the result of the task.

Now, in order to add the ability to inject a customer awaiter, we need to provide the GetAwaiter extension method. For example:

	namespace PRI.Extensions
	{
		static class TaskExtensions
		{
			static public ITaskAwaiter<TResult> GetAwaiter<TResult>(this Task<TResult> task)
			{
				return TaskAwaiterFactory<TResult>.CreateTaskAwaiter != null
					? TaskAwaiterFactory<TResult>.CreateTaskAwaiter(task) : new TaskAwaiter<TResult>(task);
			}
		}
	}

This extension method invokes a factory method delegate to create the ITaskAwaiter<T> object (or simply creates our new default awaiter). This factory method looks like this:

	internal static class TaskAwaiterFactory<TResult>
	{
		private static Func<Task<TResult>, ITaskAwaiter<TResult>> createTaskAwaiter = task =>
		                                                                              	{
		                                                                              		Current = new TaskAwaiter<TResult>(task);
		                                                                              		return Current;
		                                                                              	};
		public static Func<Task<TResult>, ITaskAwaiter<TResult>> CreateTaskAwaiter
		{
			get { return createTaskAwaiter; }
			set
			{
				createTaskAwaiter = task =>
				                    	{
				                    		Current = value(task);
				                    		return Current;
				                    	};
			}
		}
		public static ITaskAwaiter<TResult> Current { get; private set; }
	}

We can override this factory method and give a delegate to code that creates another type of ITaskAwaiter<T> implementation. We also decorate the call to that provided delegate with something that keeps track of the Current awaiter (more on that later). Now, we simply need to add a using statement within our namespace for the compiler to user our GetAwaiter when it compiles an await operator and use our factory. For example:

	namespace PRI.MainApplication
	{
		using PRI.Extensions;
 
		//...

We're now all set to inject a new awaiter and be able to isolate our code from the framework and validate some invariants. In our case, we'd like to check to make sure that the button is disabled during invocation of asynchronous operations and enabled when done. So, we can create a spy awaiter that basically does nothing and sends back a fake or dummy object. In our example we actually have two different types of Task<T> objects being awaited; so, we need to create a couple of awaiter spys. One is generic and one is specific. For example:

	public class WebResponseTaskAwaiterSpy : ITaskAwaiter<WebResponse>
	{
		private class FakeWebResponse : WebResponse
		{
			public override Stream GetResponseStream()
			{
				return new MemoryStream(System.Text.Encoding.ASCII.GetBytes("got some text"));
			}
		}
		private readonly MainForm form;
		public bool ButtonEnabledState { get; private set; }
		public WebResponseTaskAwaiterSpy(MainForm form)
		{
			this.form = form;
		}
 
		public bool BeginAwait(Action continuation)
		{
			return false;
		}
 
		public WebResponse EndAwait()
		{
			ButtonEnabledState = form.startButton.Enabled;
			return new FakeWebResponse();
		}
	}
 
	public class TaskAwaiterSpy<T> : ITaskAwaiter<T>
	{
		private readonly MainForm form;
		public bool ButtonEnabledState { get; private set; }
		public TaskAwaiterSpy(MainForm form)
		{
			this.form = form;
		}
 
		public bool BeginAwait(Action continuation)
		{
			return false;
		}
 
		public T EndAwait()
		{
			ButtonEnabledState = form.startButton.Enabled;
			return default(T);
		}
	}

Both classes don't actually execute the task because we know task execution works and we know it works asynchronously--we're not trying to test that. The clases return false from BeginAwait to tell the generated code that the asynchronous code "completed" (we lie). The generated code the calls the EndAwait method to get the result of the so-called asynchronous operation. In both cases, we get the state of the button in EndAwait. In the case of the generic spy, we simply return a dummy--the default value for that type parameter. In the case of the WebResponse spy, we can't send back a dummy because that would be a null value and we'd get NullReferenceEexceptions that would abort our test. So, we create a FakeWebResponse object and send that back that basically creates a canned response stream in memory and sends it back and that will be used in our second await.

Now, we can write a unit test for startButton_Click method. For example:

	MainForm form = CreateForm();
	TaskAwaiterFactory<WebResponse>.CreateTaskAwaiter = task => new WebResponseTaskAwaiterSpy(form);
	TaskAwaiterFactory<String>.CreateTaskAwaiter = task => new TaskAwaiterSpy<String>(form);
 
	form.startButton_Click(null, EventArgs.Empty);
 
	var spy1 = TaskAwaiterFactory<WebResponse>.Current as WebResponseTaskAwaiterSpy;
	var spy2 = TaskAwaiterFactory<String>.Current as TaskAwaiterSpy<String>;
 
	Assert.IsTrue((spy2 != null && !spy2.ButtonEnabledState) && (spy1 != null && !spy1.ButtonEnabledState));
	Assert.IsTrue(form.startButton.Enabled);

This code injects our two types of awaiters (our spies), invokes the method, then checks the spies for the values we're looking for as well as the current state of our button (we assume the current state at this point is the same state immediately after the asynchronous operation completed) to validate our invariants.