Jon Skeet: Coding Blog : Eduasync, async

Eduasync 20: Changes between the VS11 Preview and the Visual Studio 11 Beta

skeet — Tue, 06 Mar 2012 19:54:54 GMT

A while I ago I blogged about what had changed under the hood of async between the CTP and the VS11 Preview. Well, now that the VS11 Beta is out, it's time to do it all again...

Note that the code in this post is in the Eduasync codebase, under a different solution (Eduasync VS11.sln). Many of the old existing projects won't compile with VS11 beta, but I'd rather leave them as they are for posterity, showing the evolution of the feature.

Stephen Toub has an excellent blog post covering some of this, so while I'll mention things he's covered, I won't go into much detail about them. Let's start off there though...

(EDIT: Stephen has also mailed me with some corrections, which I've edited in - mostly without indication, as the post has been up for less than seven hours, and it'll make for a better reading experience.)

Awaiter pattern changes

The awaiter pattern is now not just a pattern. The IsCompleted property and GetResult method are still "loose" but OnCompleted is now part of an interface: INotifyCompletion. Awaiters have to implement INotifyCompleted, but may also implement ICriticalNotifyCompletion and its UnsafeOnCompleted method.

The OnCompleted method is just as it was before, and needs to flow the execuction context; the UnsafeOnCompleted method is simpler, as it doesn't need to flow the execution context. All of this only matters if you're implementing your own awaiters, of course. (More details in Stephen's blog post. I've found this area somewhat confusing, so please do read his post carefully!)

Skeleton method changes

Just as I have previously, I'm using the (entirely unofficial) term "skeleton method" to mean the very short method created by the compiler with the same signature as an async method: this is the entry point to the async method, effectively, which creates and starts the state machine containing all the real logic.

There are two changes I've noticed in the skeleton method. Firstly, for some reason the state machine state numbers have changed. Whereas previously a state number of 0 meant "initial or running", positive values meant "between calls, or navigating back to the await expression" and -1 meant "finished", now -1 means "initial or running", non-negative means "between calls, or navigating back to await expression" and -2 means "finished". It's not clear why this change has been made, given that it requires an extra assignment at the start of every skeleton method (to set the state to -1).

More importantly, the skeleton method no longer calls MoveNext directly on the state machine that it's built. Instead, it calls Start<TStateMachine> on the AsyncTaskMethodBuilder<T> (or whichever method builder it's using). It passes the state machine by reference (presumably for efficiency), and TStateMachine is constrained to implement the now-public-and-in-mscorlib IAsyncStateMachine interface. I'll come back to the relationship between the state machine and the builder later on.

Task caching

(Code is in project 30: TaskCaching)

It's possible for an async method to complete entirely synchronously. In this situation, the result is known before the method returns, and the task returned by the method is already in the RanToCompletionState. If two tasks have already run to completion with the same value, they can be (apparently) regarded as equivalent... so the beta now caches a task in this situation, for some types and values. (Apparently the preview cached too, but I hadn't noticed, and the beta caches more.) According to my experiments and some comments:

For int, tasks with values -1 to 8 inclusive are cached
For bool, both values (true and false)
For char, byte, sbyte, short, ushort, uint, long, ulong, IntPtr and UIntPtr tasks with value 0 (or '\0') are cached
For reference types, null is cached
For other types, no tasks are cached

EDIT: After they've completed, tasks are normally immutable except for disposal - the cached tasks are tweaked slightly to make disposal a no-op.

State machine interface changes

In the VS11 preview release, each state machine implemented an interface, but that interface was internal to the generated assembly, and contained a single method (SetMoveNextDelegate). It's now a public interface with two methods:

Personally I'm not keen on the naming of "MoveNext" - I can't help but feel that if we didn't have the "naming baggage" of IEnumerator and the fact that at least early on, the code generator was very similar to that used for iterator blocks, we'd have something different. (It is moving to the next state of the state machine, but it still doesn't quite feel right.) I'd favour something like "ContinueExecution". However, it doesn't matter - it obviously does what you'd expect, and you're not going to be calling this yourself.

SetStateMachine is a stranger beast. The documentation states:

Configures the state machine with a heap-allocated replica.

... which says almost nothing, really. The implementation is always simple, just like SetMoveNextDelegate was, although this time it delegates to the builder for the real work (a common theme, as we'll see):

void IAsyncStateMachine.SetStateMachine(IAsyncStateMachine param0)
{
this.<>t__builder.SetStateMachine(param0);
}

Now AsyncTaskMethodBuilder.SetStateMachine is also documented pretty sparsely:

Associates the builder with the specified state machine.

Again, no real help. However, we'll see that it's the builder which is responsible for calling OnContinue now, and as it can call MoveNext on an IStateMachine, it makes sense to tell it which state machine it's associated with... but can't it do that directly?

Well, not quite. The problem (as I understand it) is around when boxing occurs. We initially create the state machine on the stack, and it contains the builder. (Both are structs.) That's fine until we need a continuation, but then we've got to be able to get back to the current state later, after the current stack frame has been blown away. So we need to box the state machine. That will create a copy of the current builder (within the box). We need the builder within the boxed state machine to contain a reference to the same box. So the order has to be:

Box the state machine
Tell the state machine about the boxed reference
The state machine tells its nested builder about the boxed reference

Back when the state machine was in charge of the boxing, this went via the delegate: the act of creating the box was implicit when creating the delegate, and then casting the delegate target to the interface type allowed a reference to the newly-created delegate to be set within the copy. This is similar, but using the builder instead. It's hard to follow, but of course it's not going to matter.

State machine field changes

There are various kinds of fields in the state machine:

Those corresponding with local variables and parameters in the async method
The state
The field(s) associated with awaiters
(In the preview/beta) The field associated with the "current execution stack" at the point of an await expression
(In the CTP) An unused "disposing" field

Of these, I believe only the awaiters have actually changed, but before we talk about that, let's revisit local variables.

Local variable hoisting

I've just noticed that the local variables are only hoisted to fields when its scope contains an await expressions, but in that case all local variables of that scope are hoisted, whether or not they're used "across" awaits. It would be possible to hoist only those which need to be maintained between executions, but then you wouldn't be able to see the others when debugging, which would be somewhat confusing. Likewise local variables of the same type which are never propagated across the same await could be aliased. For example, consider this async method:

static async Task<int> M(Random rng)
{
    int x = rng.Next(1000);
    int y = x + rng.Next(1000);

    await Task.Yield();

    int z = y + rng.Next(1000);

    await Task.Yield();
    return z;
}

If the compiler could be confident you didn't need to debug through this code, it could make do with one field of type "Random" and one field of type "int" - x can be a completely local variable in MoveNext (it's not used between two awaits) and y and z can be aliased (we never need the value of y after we've first written to z).

Local variable aliasing probably isn't particularly useful for "normal" methods as the JIT may be able to do it (so long as you don't have a debugger attached, potentially) but in this case we expect the state machine to be boxed at some point, so potentially does make a difference (while the stack is typically reasonably small, you could have a lot of outstanding async methods in some scenarios). Maybe in a future release, the C# compiler could have an aggressive optimization mode around this, to be turned on explicitly. (I don't think it should be a particularly high priority, mind you.)

Awaiter fields

(Code is in project 31, AwaiterFields.)

Awaiter fields have changed a bit over the course of async's history.

In the CTPs (all of them, I believe) each await expression had its own awaiter field in the state machine, with a type corresponding to the declared awaiter type from the awaitable. (Remember that the awaitable is the thing you await, such as a task, and the awaiter is what you get back from calling GetAwaiter on the awaitable).

In the VS11 Preview, there was always a single awaiter field of type object. From what I saw, it was usually populated with a single-element array containing an awaiter. For value type awaiters (i.e. where the awaiter is a struct) this is somewhat similar to boxing, but maintaining strong typing, so calls to IsCompleted etc can still be made. It's possible that reference type awaiters were stored without the array creation, as it would serve no purpose. (I don't have any machines with just the preview installed to verify this.)

In the Beta, we have a mixture. If there are any reference type awaiters, they all end up being stored in a single field of type object, which is then cast back to the actual type when required. (Don't forget that only one awaiter can be "active" at a time, which makes this possible.) This includes awaiters of an interface type - it's only the compile-time type declared as the return type of the GetAwaiter method of the awaitable which is important.

If any of the awaiter types are value types, each of these types gets its own field. So there might be a TaskAwaiter<int> field and a TaskAwaiter<string> field, for example. However, there can still be "sharing" going on: if there are multiple await expressions all of the same value type awaiter, they will all share a single field. (This all feels a little like the JITting of generics, but it's somewhat coincidental.)

MoveNext method changes

(Code is in project 32, BetaStateMachine)

As I've mentioned earlier, the builder is now responsible for a lot more of the work than it was in earlier versions. The majority of the code remains the same as far as I can tell, in terms of handling branching, evaluating expressions with multiple await expressions and so on. The code in the source repository shows what the complete state machine looks like, but for the sake of clarity, I'll just focus on a single snippet. If we have an await expression like this:

await x;

then the state machine code in the VS11 Preview would look something like this:

localTaskAwaiter = x.GetAwaiter();
if (localTaskAwaiter.IsCompleted)
{
    goto AwaitCompleted;
}
this.state = 1;
TaskAwaiter[] awaiterArray = { localTaskAwaiter };
this.awaiter = awaiterArray;
Action continuation = this.MoveNextDelegate;
if (continuation == null)
{
    Task<int> task = this.builder.Task;
    continuation = MoveNext;
    ((IStateMachine) continuation.Target).SetMoveNextDelegate(continuation);
}
awaiterArray[0].OnCompleted(continuation);
...
return;

(That's just setting up the await, of course - there's then the bit where the result is fetched, but that's less interesting. There's also the matter of doFinallyBodies.)

In the VS11 Beta, it's something this instead (for an awaiter type "AwaiterType" which implements ICriticalNotifyCompletion, in a state machine of type ThisStateMachine).

localAwaiter = x.GetAwaiter();
if (!localAwaiter.IsCompleted)
{
    state = 0;
    awaiterField = localAwaiter;
    builder.AwaitUnsafeOnCompleted<AwaiterType, ThisStateMachine>(ref localAwaiter, ref this);
    doFinallyBodies = false;
    return;
}

If the awaiter type only implements INotifyCompletion, it calls AwaitOnCompleted instead. Note how the calls are generic (but both type variables are constrained to implement appropriate interfaces) which avoids boxing.

The call to the builder will call back to the state machine's SetStateMachine method if this is the first awaiter that hasn't already completed within this execution of the async method. So that handles the section which checked for the continuation being null in the first block of code. Most of the rest of the change is explained by the difference in awaiter types, and obviously AwaitOnCompleted/AwaitUnsafeOnCompleted also ends up calling into OnCompleted on the awaiter itself.

Mutable value type awaiters

(Code is in project 32, MutableAwaiters)

One subtle difference which really shouldn't hurt people but is fun to explore is what happens if you have an awaiter which is a mutable value type. Due to the way awaiters were carefully handled pre-beta, mutations which were conducted as part of OnCompleted would still be visible in GetResult. That's not the case in the beta (as Stephen mentions in his blog post). Mind you, it doesn't mean that all mutations will be ignored... just ones in OnCompleted. A mutation from IsCompleted is still visible, as shown here:

public struct MutableAwaiter : INotifyCompletion
{
     private string message;

     public MutableAwaiter(string message)
     {
         this.message = message;
     }

     public bool IsCompleted
     {
         get
         {
             message = "Set in IsCompleted";
             return false;
         }
     }

     public void OnCompleted(Action action)
     {
         message = "Set in OnCompleted";
         // Ick! Completes inline. Never mind, it's only a demo...
         action();
     }

     public string GetResult()
     {
         return message;
     }
}

What would you expect to be returned from this awaiter? You can verify that all three members are called... but "Set in IsCompleted" is returned. That's because IsCompleted is called before the awaiter value is copied into a field within the state machine. Even though the state machine passes the awaiter by reference, it's passing the local variable, which is of course a separate variable from the field.

I'm absolutely not suggesting that you should rely on any of this behaviour. If you really need to be able to mutate your awaiter, make it a reference type.

Conclusion

The main changes in the Beta are around the interactions between the AsyncTaskMethodBuilder (et al) and the state machine, including the new interfaces for awaiters. There's been quite a bit of optimization, although I still see room for a bit more:

When there's only a single kind of reference type awaiter, the field for storing it could be of that type rather than of type object, removing the need for an execution-time cast
The "stack" variable could be removed in some cases, and made into a specific type in many others
With appropriate optimization flags, local variables which aren't used await expressions could stay local to the state machine instead of being hoisted, and hoisted variables could be aliased in some cases.

One thing which concerns me slightly is how the C# language specification is going to change - the addition of the new interfaces is definitely going to mean more complexity from this previously "tidy" feature. I'm sure it's worth it for the sake of efficiency and the like, but part of me sighs at every added tweak.

So, is this now close to the finished version of async? Only time will tell. I haven't checked whether dynamic awaitables have finally been introduced... if they have, I'll put that in the next post.

Eduasync part 19: ordering by completion, ahead of time...

skeet — Mon, 16 Jan 2012 22:22:43 GMT

Today's post involves the MagicOrdering project in source control (project 28).

When I wrote part 16 of Eduasync, showing composition in the form of majority voting, one reader mailed me a really interesting suggestion. We don't really need to wait for any of the tasks to complete on each iteration of the loop - we only need to wait for the next task to complete. Now that sounds impossible - sure, it's great if we know the completion order of the tasks, but half the point of asynchrony is that many things can be happening at once, and we don't know when they'll complete. However, it's not as silly as it sounds.

If you give me a collection of tasks, I'll give you back another collection of tasks which will return the same results - but I'll order them so that the first returned task will have the same result as whichever of your original tasks completes first, and the second returned task will have the same result as whichever of your original tasks completes second, and so on. They won't be the same tasks as you gave me, reordered - but they'll be tasks with the same results. I'll propagate cancellation, exceptions and so on.

It still sounds impossible... until you realize that I don't have to associate one of my returned tasks with one of your original tasks until it has completed. Before anything has completed, all the tasks look the same. The trick is that as soon as I see one of your tasks complete, I can fetch the result and propagate it to the first of the tasks I've returned to you, using TaskCompletionSource<T>. When the second of your tasks completes, I propagate the result to the second of the returned tasks, etc. This is all quite easy using Task<T>.ContinueWith - barring a few caveats I'll mention later on.

Once we've built a method to do this, we can then really easily build a method which is the async equivalent of Parallel.ForEach (and indeed you could write multiple methods for the various overloads). This will execute a specific action on each task in turn, as it completes... it's like repeatedly calling Task.WhenAny, but we only actually need to wait for one task at a time, because we know that the first task in our "completion ordered" collection will be the first one to complete (duh).

Show me the code!

Enough description - let's look at how we'll demonstrate both methods, and then how we implement them.

private static async Task PrintDelayedRandomTasksAsync()
{
    Random rng = new Random();
    var values = Enumerable.Range(0, 10).Select(_ => rng.Next(3000)).ToList();
    Console.WriteLine("Initial order: {0}", string.Join(" ", values));

    var tasks = values.Select(DelayAsync);

    var ordered = OrderByCompletion(tasks);

    Console.WriteLine("In order of completion:");
    await ForEach(ordered, Console.WriteLine);
}

/// <summary>
/// Returns a task which delays (asynchronously) by the given number of milliseconds,
/// then return that same number back.
/// </summary>
private static async Task<int> DelayAsync(int delayMillis)
{
    await TaskEx.Delay(delayMillis);
    return delayMillis;
}

The idea is that we're going to create 10 tasks which each just wait for some random period of time, and return the same time period back. We'll create them in any old order - but obviously they should complete in (at least roughly) the same order as the returned numbers.

Once we've created the collection of tasks, we'll call OrderByCompletion to create a second collection of tasks, returning the same results but this time in completion order - so ordered.ElementAt(0) will be the first task to complete, for example.

Finally, we call ForEach and pass in the ordered task collection, along with Console.WriteLine as the action to take with each value. We await the resulting Task to mimic blocking until the foreach loop has finished. Note that we could make this a non-async method and just return the task returned by ForEach, given that that's our only await expression and it's right at the end of the method. This would be marginally faster, too - there's no need to build an extra state machine. See Stephen Toub's article about async performance for more information.

ForEach

I'd like to get ForEach out of the way first, as it's so simple: it's literally just iterating over the tasks, awaiting them and propagating the result to the action. We get the "return a task which will wait until we've finished" for free by virtue of making it an async method.

/// <summary>
/// Executes the given action on each of the tasks in turn, in the order of
/// the sequence. The action is passed the result of each task.
/// </summary>
private static async Task ForEach<T>(IEnumerable<Task<T>> tasks, Action<T> action)
{
    foreach (var task in tasks)
    {
        T value = await task;
        action(value);
    }
}

Simple, right? Let's get onto the meat...

OrderByCompletion

This is the tricky bit, and I've actually split it into two methods to make it slightly easier to comprehend. The PropagateResult method feels like it could be useful in other composition methods, too.

The basic plan is:

Copy the input tasks to a list: we need to work out how many there are and iterate over them, so let's make sure we only iterate once
Create a collection of TaskCompletionSource<T> references, one for each input task. Note that we're not associating any particular input task with any particular completion source - we just need the same number of them
Declare an integer to keep track of "the next available completion source"
Attach a continuation to each input task which will be increment the counter we've just declared, and propagate the just-completed task's status
Return a view onto the collection of TaskCompletionSource<T> values, projecting each one to its Task property

Once you're happy with the idea, the implementation isn't too surprising (although it is quite long):

/// <summary>
/// Returns a sequence of tasks which will be observed to complete with the same set
/// of results as the given input tasks, but in the order in which the original tasks complete.
/// </summary>
private static IEnumerable<Task<T>> OrderByCompletion<T>(IEnumerable<Task<T>> inputTasks)
{
    // Copy the input so we know it'll be stable, and we don't evaluate it twice
    var inputTaskList = inputTasks.ToList();

    // Could use Enumerable.Range here, if we wanted...
    var completionSourceList = new List<TaskCompletionSource<T>>(inputTaskList.Count);
    for (int i = 0; i < inputTaskList.Count; i++)
    {
        completionSourceList.Add(new TaskCompletionSource<T>());
    }

    // At any one time, this is "the index of the box we've just filled".
    // It would be nice to make it nextIndex and start with 0, but Interlocked.Increment
    // returns the incremented value...
    int prevIndex = -1;

    // We don't have to create this outside the loop, but it makes it clearer
    // that the continuation is the same for all tasks.
    Action<Task<T>> continuation = completedTask =>
    {
        int index = Interlocked.Increment(ref prevIndex);
        var source = completionSourceList[index];
        PropagateResult(completedTask, source);
    };

    foreach (var inputTask in inputTaskList)
    {
        // TODO: Work out whether TaskScheduler.Default is really the right one to use.
        inputTask.ContinueWith(continuation,
                               CancellationToken.None,
                               TaskContinuationOptions.ExecuteSynchronously,
                               TaskScheduler.Default);
    }

    return completionSourceList.Select(source => source.Task);
}

/// <summary>
/// Propagates the status of the given task (which must be completed) to a task completion source
/// (which should not be).
/// </summary>
private static void PropagateResult<T>(Task<T> completedTask,
    TaskCompletionSource<T> completionSource)
{
    switch (completedTask.Status)
    {
        case TaskStatus.Canceled:
            completionSource.TrySetCanceled();
            break;
        case TaskStatus.Faulted:
            completionSource.TrySetException(completedTask.Exception.InnerExceptions);
            break;
        case TaskStatus.RanToCompletion:
            completionSource.TrySetResult(completedTask.Result);
            break;
        default:
            // TODO: Work out whether this is really appropriate. Could set
            // an exception in the completion source, of course...
            throw new ArgumentException("Task was not completed");
    }
}

You'll notice there are a couple of TODO comments there. The exception in PropagateResult really shouldn't happen - the continuation shouldn't be called when the task hasn't completed. I still need to think carefully about how tasks should propagate exceptions though.

The arguments to ContinueWith are more tricky: working through my TimeMachine class and some unit tests with Bill Wagner last week showed just how little I know about how SynchronizationContext, the task awaiters, task schedulers, and TaskContinuationOptions.ExecuteSynchronously all interact. I would definitely need to look into that more deeply before TimeMachine was really ready for heavy use... which means you should probably be looking at the TPL in more depth too.

Conclusion

Sure enough, when you run the code, the results appear in order, as the tasks complete. Here's one sample of the output:

Initial order: 335 468 1842 1991 2512 2603 270 2854 1972 1327
In order of completion:
270
335
468
1327
1842
1972
1991
2512
2603
2854

TODOs aside, the code in this post is remarkable (which I can say with modesty, as I've only refactored it from the code sent to me by another reader and Stephen Toub). It makes me smile every time I think about the seemingly-impossible job it accomplishes. I suspect this approach could be useful in any number of composition blocks - it's definitely one to remember.

Eduasync part 18: Changes between the Async CTP and the Visual Studio 11 Preview

skeet — Thu, 12 Jan 2012 01:34:31 GMT

In preparation for CodeMash, I've been writing some more async code and decompiling it with Reflector. This time I'm using the Visual Studio 11 Developer Preview - the version which installs alongside Visual Studio 2010 under Windows 7. (Don't ask me about any other features of Visual Studio 11 - I haven't explored it thoroughly; I've really only used it for the C# 5 bits.)

There have been quite a few changes since the CTP - they're not visible changes in terms of code that you'd normally write, but the state machine generated by the C# compiler is reasonably different. In this post I'll describe the differences, as best I understand them. There are still a couple of things I don't understand (which I'll highlight within the post) but overall, I think I've got a pretty good handle on why the changes have been made.

I'm going to assume you already have a reasonable grasp of the basic idea of async and how it works - the way that the compiler generates a state machine to represent an async method or anonymous function, with originally-local variables being promoted to instance variables within the state machine, etc. If the last sentence was a complete mystery to you, see Eduasync part 7 for more information. I don't expect you to remember the exact details of what was in the previous CTP though :)

Removal of iterator block leftovers

In the CTP, the code for async methods was based on the iterator block implementation. I suspect that's still the case, but possibly sharing just a little less code. There used to be a few methods and fields which weren't used in async methods, but now they're gone:

There's no now constructor, so no need for the "skeleton" method which replaces the real async method to pass in 0 as the initial state.
There's no Dispose method.
There's no disposing field.

It's nice to see these gone, but it's not terribly interesting. Now on to the bigger changes...

Large structural changes

There's a set of related structural changes which don't make sense individually. I'll describe them first, then look at how it all hangs together, and my guess as to the reasoning behind.

The state machine is now a struct

The declaration of the nested type for the state machine is now something like this:

[CompilerGenerated]
private struct StateMachine : IStateMachine
{
    // Fields common to all async state machines
    // (with caveats)
    private int state;
    private object awaiter;
    public AsyncTaskMethodBuilder<int> builder;
    public Action moveNextDelegate;
    private object stack;

    // Hoisted local variables

    // Methods
    [DebuggerHidden]
    public void SetMoveNextDelegate(Action action) { ... }
    public void MoveNext() { ... }
}

The caveats around the common field are in terms of the return type of the async method (which determines the type of builder used) and whether or not there are any awaits (if there are no awaits, the stack and awaiter fields aren't generated).

Note that throughout this blog post I've changed the names of fields and types - in reality they're all "unspeakable" names including angle-brackets, just like all compiler-generated names.

There's a new assembly-wide interface

As you can see from the code above, the state machine implements an interface (actually called <>t__IStateMachine). One of these is created in the global namespace in each assembly that contains at least one async method or anonymous function, and it looks like this:

[CompilerGenerated]
internal interface IStateMachine
{
void SetMoveNextDelegate(Action action);
}

The implementation for this method is always the same, and it's trivial:

[DebuggerHidden]
public void SetMoveNextDelegate(Action action)
{
this.moveNextDelegate = action;
}

Simplified skeleton method

The method which starts the state machine, which I've been calling the "skeleton" method everywhere, is now a bit simpler than it was. Something like this:

public static Task<int> FooAsync()
{
    StateMachine machine = new StateMachine();
    machine.builder = AsyncVoidMethodBuilder.Create();
    machine.MoveNext();
    return machine.builder.Task;
}

In fact if you decompile the IL, you'll see that it doesn't explicitly initialize the variable to start with - it just declares it, sets the builder field and then calls MoveNext(). That's not valid C# (as all the struct's fields aren't initialized), but it is valid IL. It's equivalent to the code above though. Note how there's nothing to set the continuation - where previously the moveNextDelegate field would be populated within the skeleton method.

Just-in-time delegate creation

Now that the skeleton method doesn't create the delegate representing the continuation, it can be done when it's first required - which is when we first encounter an await expression for an awaitable which hasn't already completed. (If the awaitable has completed before we await it, the generated code skips the continuation and just uses the results immediately and synchronously).

The code for that delegate creation is slightly trickier than you might expect, however. It looks something like this:

Action action = this.moveNextDelegate;
if (action == null)
{
    Task<int> task = this.builder.Task;
    action = new Action(this.MoveNext);
    ((IStateMachine) action.Target).SetMoveNextDelegate(action);
}

There are two oddities here, one of which I mostly understand and one of which I don't understand at all.

I really don't understand the "task" variable here. Why do we need to exercise the AsyncTaskMethodBuilder.Task property? We don't use the result anywhere... does forcing this flush some memory buffer? I have no clue on this one. (See the update at the bottom of the post...)

The part about setting the delegate via the interface makes more sense, but it's subtle. You might expect code like this:

// Looks sensible, but is actually slightly broken
Action action = this.moveNextDelegate;
if (action == null)
{
action = new Action(this.MoveNext);
this.moveNextDelegate = action;
}

That would sort of work - but we'd end up needing to recreate the delegate each time we encountered an appropriate await expression. Although the above code saves the value to the field, it saves it within the current value of the state machine... after we've boxed that value as the target of the delegate. The value we want to mutate is the one within the box - which is precisely why there's an interface, and why the code casts to it.

We can't even just unbox and then set the field afterwards - at least in C# - because the unbox operation is always followed by a copy operation in normal C#. I believe it would be possible for the C# compiler to generate IL which unboxed action.Target without the copy, and then set the field in that. It's not clear to me why the team went with the interface approach instead... I would expect that to be slower (as it requires dynamic dispatch) but I could easily be wrong. Of course, it would also make it impossible to decompile the IL to C#, which would make my talks harder, but don't expect the C# team to bend the compiler implementation for my benefit ;)

(As an aside to all of this, I've gone back and forth on whether the "slightly broken" implementation would recreate the delegate on every appropriate await, or only two. I think it would end up being on every occurrence, as even though on the second occurrence we'd be operating within the context of the first boxed instance, the new delegate would have a reference to a new boxed copy each time. It does my head in a little bit, trying to think about this... more evidence that mutable structs are evil and hard to reason about. It's not the wrong decision in this case, hidden far from the gaze of normal developers, but it's a pain to reason about.)

Single awaiter variable

In the CTP, each await expression generated a separate field within the state machine, and that field was always of the exact awaiter type. In the VS11 Developer Preview, there's always exactly one awaiter field (assuming there's at least one await expression) and it's always of type object. It's used like this:

// Single local variable used by both continuation and first-time paths
TaskAwaiter<int> localAwaiter;

...

if (conditions-for-first-time-execution)
{
      // Code before await

      localAwaiter = task.GetAwaiter();
      if (localAwaiter.IsCompleted)
      {
          goto Await1Completed;
      }
      this.state = 1;
      TaskAwaiter<int>[] awaiterArray = { localAwaiter };
      this.awaiter = awaiterArray;
      // Lazy delegate creation goes here
      awaiterArray[0].OnCompleted(action);
      return;
}
// Continuation would get into here
localAwaiter = ((TaskAwaiter<int>[]) this.awaiter)[0];
this.awaiter = null;
this.state = 0;
Await1Completed:
int result = localAwaiter.GetResult();
localAwaiter = default(TaskAwaiter<int>);

I realize there's a lot of code here, but it does make some sense:

The value of the awaiter field is always either null, or a reference to a single-element array of the awaiter type for one of the await expressions.
A single localAwaiter variable is shared between the two code paths, populated either from the awaitable (on the initial code path) or by copying the value from the array (in the second code path).
The field is always set to null and the local variable is set to its default value after use, presumably for the sake of garbage collection

It's basically a nice way of using the fact that we'll only ever need one awaiter at a time. It's not clear to me why an array is used instead of either using a reference to the awaiter for class-based awaiters, or simply by boxing for struct-based awaiters. The latter would need the same "unbox without copy" approach discussed in the previous section - so if there's some reason why that's actually infeasible, it would explain the use of an array here. We can't use the interface trick in this case, as the compiler isn't in control of the awaiter type (so can't make it implement an interface).

Expression stack preservation

This one is actually a fix to a bug in the async CTP, which I've written about before. We're used to the stack containing our local variables (in the absence of iterator blocks, captured variables etc, and modulo the stack being an implementation detail) but it's also used for intermediate results within a single statement. For example, consider this block of code:

int x = 10;
int y = 5;
int z = x + 50 * y;

That last line is effectively:

Load the value of x onto the stack
Load the value 50 onto the stack
Load the value of y onto the stack
Multiply the top two stack values (50 and y) leaving the result on the stack
Add the top two stack values (x and the previously-computed result) leaving the result on the stack
Store the top stack value into z

Now suppose we want to turn y into a Task<int>:

int x = 10;
Task<int> y = Task.FromResult(5);
int z = x + 50 * await y;

Our state machine needs to make sure that it will preserve the same behaviour as the synchronous version, so it needs the same sort of stack. In the new-style state machine, all of that stack is saved in the "stack" field. It's only one field, but may need to represent multiple different types within the code at various different await expressions - in the code above, for example, it represents two int values. As far as I can discover, the C# compiler generates code which uses the actual type of the value it needs, if it only requires a single value. If it needs multiple values, it uses an appropriate Tuple type, nesting tuples if it goes beyond the number of type parameters supported by the Tuple<...> family of types. So in our case above, we end up with code a bit like this:

// Local variable used in both paths
Tuple<int, int> tuple;

...

// Code before the await
if (conditions-for-first-time-execution)
{
    ...
    tuple = new Tuple<int, int>(this.x, 50);
    this.stack = tuple;
    ...
}

// Continuation would get into here
tuple = (Tuple<int, int>) this.stack;
// IL copies the values from the tuple onto the stack at this point
this.stack = null;

...

// Both the fast and slow code paths get here eventually
this.z = stack0 + stack1 * awaiter.GetResult()

I say it's a bit like this, because it's hard to represent the IL exactly in C# in this case. The tuple is only created if it's needed, i.e. not in the already-completed fast path. In that case, the values are loaded onto the stack but not then put into the tuple - execution skips straight to the code which uses the values already on the stack.

When the awaitable isn't complete immediately, then a Tuple<int, int> is created, stored in the "stack" field, and the continuation is handed to the awaiter. On continuation, the tuple is loaded back from the "stack" field (and cast accordingly), the values are loaded onto the stack - and then we're back into the common code path of fetching the value and performing the add and multiply operations.

Conclusion

As far as I'm aware, those are the most noticeable changes in the generated code. There may well still be a load more changes in Task<T> and the TPL in general - I wouldn't be at all surprised - but that's harder to investigate.

I'm sure all of this has been done in the name of performance (and correctness, in the case of stack preservation). The state machine is now much smaller in terms of the number of fields it requires, and objects are created locally as far as possible (including the state machine itself only requiring heap allocation if there's ever a "slow" awaitable). I suspect there's still some room for optimization, however:

Both the awaiter and the delegate use careful boxing and either arrays or a mutating interface to allow the boxed value to be changed. I suspect that using unbox with the concrete type, but without copying the value, would be more efficient. I may attempt to work this theory up into a test at some point.
If there's only one awaiter type (usually TaskAwaiter<T> for some T), that type could be used instead of object, potentially reducing heap optimization
I've no idea why the builder.Task property is explicitly fetched and then the results discarded
If there's only one await expression, the "stack" field can be strongly typed, which would also avoid boxing if only a single value needs to be within that stack
The stack field could be removed entirely when it's not needed for intermediate stack value preservation. (I believe that would be the case reasonably often.)

The use of mutable value types is really fascinating (for me, at least) - I'm sure most people on the C# team would still say they're evil, but when they're used in a carefully controlled environment where real developers don't have to reason about their behaviour, they can be useful.

Next time, I'll hopefully get back to the idea I promised to write up before, about ordering a collection of tasks in completion order... before they've completed. (Well, sort of.) Should be fun...

Update (January 16th 2012)

Stephen Toub got in touch with me after I posted the original version of this blog entry, to explain the use of the Task property. Apparently the idea is that at this point, we know we're going to return out of the state machine, so the skeleton method is going to access the Task property anyway. However, as we haven't scheduled the continuation yet we also know that nothing will be accessing the Task property on a different thread. If we access it now for the first time, we can lazily allocate the task in the same thread that created the AsyncMethodBuilder, with no risk of contention. If we can force there to be a task ready and waiting for whatever accesses it later, we don't need any synchronization in that area.

So why might we want to allocate the task lazily in the first place? Well, don't forget that we might never have to wait for an await (as it were). We might just have an async method which takes the fast path everywhere. If that's the case, then for certain cases (e.g. a non-generic, successfully completed task, or a Task<bool> which again has completed successfully) we can reuse the same instance repeatedly. Apparently this laziness isn't yet part of the VS11 Developer Preview, but the reason for the property access is in preparation for this.

Another case of micro-optimization - which is fair enough when it's at a system level :)

Eduasync part 17: unit testing

skeet — Fri, 25 Nov 2011 21:46:45 GMT

In the last post I showed a method to implement "majority voting" for tasks, allowing a result to become available as soon as possible. At the end, I mentioned that I was reasonably confident that it worked because of the unit tests... but I didn't show the tests themselves. I felt they deserved their own post, as there's a bigger point here: it's possible to unit test async code. At least sometimes.

Testing code involving asynchrony is generally a pain. Introducing the exact order of events that you want is awkward, as is managing the threading within tests. With a few benefits with async methods:

We know that the async method itself will only execute in a single thread at a time
We can control the thread in which the async method will execute, if it doesn't configure its awaits explicitly
Assuming the async method returns Task or Task<T>, we can check whether or not it's finished
Between Task<T> and TaskCompletionSource<T>, we have a way of injecting tasks that we understand

Now in our sample method we have the benefit of passing in the tasks that will be awaited - but assuming you're using some reasonably testable API to fetch any awaitables within your async method, you should be okay. (Admittedly in the current .NET framework that excludes rather a lot of classes... but the synchronous versions of those calls are also generally hard to test too.)

The plan

For our majority tests, we want to be able to see what happens in various scenarios, with tasks completing at different times and in different ways. Looking at the test cases I've implemented I have the following tests:

NullSequenceOfTasks
EmptySequenceOfTasks
NullReferencesWithinSequence
SimpleSuccess
InputOrderIsIrrelevant
MajorityWithSomeDisagreement
MajorityWithFailureTask
EarlyFailure
NoMajority

I'm not going to claim this is a comprehensive set of possible tests - it's a proof of concept more than anything else. Let's take one test as an example: MajorityWithFailureTask. The aim of this is to pass three tasks (of type Task<string>) into the method. One will give a result of "x", the second will fail with an exception, and the third will also give a result of "x". The events will occur in that order, and only when all three results are in should the returned task complete, at which point it will also have a success result of "x".

So, the tricky bit (compared with normal testing) is introducing the timing. We want to make it appear as if tasks are completing in a particular order, at predetermined times, so we can check the state of the result between events.

Introducing the TimeMachine class

Okay, so it's a silly name. But the basic idea is to have something to control the logical flow of time through our test. We're going to ask the TimeMachine to provide us with tasks which will act in a particular way at a given time, and then when we've started our async method we can then ask it to move time forward, letting the tasks complete as they go. It's probably best to look at the code for MajorityWithFailureTask first, and then see what the implementation of TimeMachine looks like. Here's the test:

[Test]
public void MajorityWithFailureTask()
{
    var timeMachine = new TimeMachine();
    // Second task gives a different result
    var task1 = timeMachine.AddSuccessTask(1, "x");
    var task2 = timeMachine.AddFaultingTask<string>(2, new Exception("Bang!"));
    var task3 = timeMachine.AddSuccessTask(3, "x");

    var resultTask = MoreTaskEx.WhenMajority(task1, task2, task3);
    Assert.IsFalse(resultTask.IsCompleted);

    // Only one result so far - no consensus
    timeMachine.AdvanceTo(1);
    Assert.IsFalse(resultTask.IsCompleted);

    // Second result is a failure
    timeMachine.AdvanceTo(2);
    Assert.IsFalse(resultTask.IsCompleted);

    // Third result gives majority verdict
    timeMachine.AdvanceTo(3);
    Assert.AreEqual(TaskStatus.RanToCompletion, resultTask.Status);
    Assert.AreEqual("x", resultTask.Result);
}

As you can see, there are two types of method:

AddSuccessTask / AddFaultingTask / AddCancelTask (not used here) - these all take the time at which they're going to complete as their first parameter, and the method name describes the state they'll reach on completion. The methods return the task created by the time machine, ready to pass into the production code we're testing.
AdvanceTo / AdvanceBy (not used here) - make the time machine "advance time", completing pre-programmed tasks as it goes. When those tasks complete, any continuations attached to them also execute, which is how the whole thing hangs together.

Now forcing tasks to complete is actually pretty simple, if you build them out of TaskCompletionSource<T> to start with. So all we need to do is keep our tasks in "time" order (which I achieve with SortedList), and then when we're asked to advance time we move through the list and take the appropriate action for all the tasks which weren't completed before, but are now. I represent the "appropriate action" as a simple Action, which is built with a lambda expression from each of the Add methods. It's really simple:

public class TimeMachine
{
    private int currentTime = 0;
    private readonly SortedList<int, Action> actions = new SortedList<int, Action>();

    public int CurrentTime { get { return currentTime; } }

    public void AdvanceBy(int time)
    {
        AdvanceTo(currentTime + time);
    }

    public void AdvanceTo(int time)
    {
        // Okay, not terribly efficient, but it's simple.
        foreach (var entry in actions)
        {
            if (entry.Key > currentTime && entry.Key <= time)
            {
                entry.Value();
            }
        }
        currentTime = time;
    }

    public Task<T> AddSuccessTask<T>(int time, T result)
    {
        TaskCompletionSource<T> tcs = new TaskCompletionSource<T>();
        actions[time] = () => tcs.SetResult(result);
        return tcs.Task;
    }

    public Task<T> AddCancelTask<T>(int time)
    {
        TaskCompletionSource<T> tcs = new TaskCompletionSource<T>();
        actions[time] = () => tcs.SetCanceled();
        return tcs.Task;
    }

    public Task<T> AddFaultingTask<T>(int time, Exception e)
    {
        TaskCompletionSource<T> tcs = new TaskCompletionSource<T>();
        actions[time] = () => tcs.SetException(e);
        return tcs.Task;
    }
}

Okay, that's a fair amount of code for a blog posts (and yes, it could do with some doc comments etc!) but considering that it makes life testable, it's pretty simple.

So, is that it?

It works on my machine... with my test runner... in simple cases...

When I first ran the tests using TimeMachine, they worked almost immediately. This didn't surprise me nearly as much as it should have done. You see, when the tests execute, they use async/await in the normal way - which means the continuations are scheduled on "the current task scheduler". I have no idea what the current task scheduler is in unit tests. Or rather, it feels like something which is implementation specific. It could easily have worked when running the tests from ReSharper, but not from NCrunch, or not from the command line NUnit test runner.

As it happens, I believe all of these run tests on thread pool threads with no task scheduler allocated, which means that the continuation is attached to the task to complete "in-line" - so when the TimeMachine sets the result on a TaskCompletionSource, the continuations execute before that call returns. That means everything happens on one thread, with no ambiguity or flakiness - yay!

However, there are two problems:

The words "I believe" aren't exactly confidence-inspiring when it comes to testing that your software works correctly.
Our majority voting code only ever sees one completed task at a time - we're not testing the situation where several tasks complete so quickly together that the continuation doesn't get chance to run before they've all finished.

Both of these are solvable with a custom TaskScheduler or SynchronizationContext. Without diving into the docs, I'm not sure yet which I'll need, but the aim will be:

Make TimeMachine implement IDisposable
In the constructor, set the current SynchronizationContext (or TaskScheduler) to a custom one having remembered what the previous one was
On disposal, reset the context
Make the custom scheduler keep a queue of jobs, such that when we're asked to advance to time T, we complete all the appropriate tasks but don't execute any continuations, then we execute all the pending continuations.

I don't yet know how hard it will be, but hopefully the Parallel Extensions Samples will help me.

Conclusion

I'm not going to claim this is "the" way of unit testing asynchronous methods. It's clearly a proof-of-concept implementation of what can only be called a "test framework" in the loosest possible sense. However, I hope it gives an example of a path we might take. I'm looking forward to seeing what others come up with, along with rather more polished implementations.

Next time, I'm going to shamelessly steal an idea that a reader mailed me (with permission, of course). It's insanely cool, simple and yet slightly brain-bending, and I suspect will handy in many situations. Love it.

Eduasync part 16: Example of composition: majority voting

skeet — Tue, 22 Nov 2011 23:18:41 GMT

Note: For the rest of this series, I'll be veering away from the original purpose of the project (investigating what the compiler is up to) in favour of discussing the feature itself. As such, I've added a requirement for AsyncCtpLib.dll - but due to potential distribution restrictions, I've felt it safest not to include that in the source repository. If you're running this code yourself, you'll need to copy the DLL from your installation location into the Eduasync\lib directory before it will build - or change each reference to it.

One of the things I love about async is the compositional aspect. This is partly due to the way that the Task Parallel Library encourages composition to start with, but async/await makes it even easier by building the tasks for you. In the next few posts I'll talk about a few examples of interesting building blocks. I wouldn't be surprised to see an open source library with a proper implementation of some of these ideas (Eduasync is not designed for production usage) whether from Microsoft or a third party.

In project 26 of Eduasync, I've implemented "majority voting" via composition. The basic idea is simple, and the motivation should be reasonably obvious in this day and age of redundant services. You have (say) five different tasks which are meant to be computing the same thing. As soon as you have a single answer which the majority of the tasks agree on, the code which needs the result can continue. If the tasks disagree, or fail (or a combination leading to no single successful majority result), the overall result is failure too.

My personal experience with services requiring a majority of operations to return is with Megastore, a storage system we use at Google. I'm not going to pretend to understand half of the details of how Megastore works, and I'm certainly not about to reveal any confidential information about its internals or indeed how we use it, but basically when discussing it with colleagues at around the time that async was announced, I contemplated what a handy feature async would be when implementing a Megastore client. It could also be used in systems where each calculation is performed in triplicate to guard against rogue errors - although I suspect the chances of those systems being implemented in C# are pretty small.

It's worth mentioning that the implementation here wouldn't be appropriate for something like a stock price service, where the result can change rapidly and you may be happy to tolerate a small discrepancy, within some bounds.

The API

Here's the signatures of the methods we'll implement:

public static Task<T> WhenMajority<T>(params Task<T>[] tasks)

public static Task<T> WhenMajority<T>(IEnumerable<Task<T>> tasks)

Obviously the first just delegates to the second, but it's helpful to have both forms, so that we can pass in a few tasks in an ad hoc manner with the first overload, or a LINQ-generated sequence of tasks with the second.

The name is a little odd - it's meant to match WhenAll and WhenAny, but I'm sure there are better options. I'm not terribly interested in that at the moment.

It's easy to use within an async method:

Task<int> firstTask = firstServer.ComputeSomethingAsync(input);
Task<int> secondTask = selectServer.ComputeSomethingAsync(input);
Task<int> thirdTask = thirdServer.ComputeSomethingAsync(input);

int result = await MoreTaskEx.WhenMajority(firstTask, secondTask, thirdTask);

Or using the LINQ-oriented overload:

var tasks = servers.Select(server => server.ComputeSomethingAsync(input));
int result = await MoreTaskEx.WhenMajority(tasks);

Of course we could add an extension method (dropping the When prefix as it doesn't make as much sense there, IMO):

int result = await servers.Select(server => server.ComputeSomethingAsync(input))
.MajorityAsync();

The fact that we've stayed within the Task<T> model is what makes it all work so smoothly. We couldn't easily express the same API for other awaitable types in general although we could do it for any other specific awaitable type of course. It's possible that it would work using dynamic, but I'd rather avoid that :) Let's implement it now.

Implementation

There are two parts to the implementation, in the same way that we implemented LINQ operators in Edulinq - and for the same reason. We want to go bang immediately if there are any clear input violations - such as the sequence of tasks being null or empty. This is in line with the Task-based Asynchronous Pattern white paper:

An asynchronous method should only directly raise an exception to be thrown out of the MethodNameAsync call in response to a usage error*. For all other errors, exceptions occurring during the execution of an asynchronous method should be assigned to the returned Task.

Now it occurs to me that we don't really need to do this in two separate methods (one for precondition checking, one for real work). We could create an async lambda expression of type Func<Task<T>>, and make the method just return the result of invoking it - but I don't think that would be great in terms of readability.

So, the first part of the implementation performing validation is really simple:

public static Task<T> WhenMajority<T>(params Task<T>[] tasks)
{
    return WhenMajority((IEnumerable<Task<T>>) tasks);
}

public static Task<T> WhenMajority<T>(IEnumerable<Task<T>> tasks)
{
    if (tasks == null)
    {
        throw new ArgumentNullException("tasks");
    }
    List<Task<T>> taskList = new List<Task<T>>(tasks);
    if (taskList.Count == 0)
    {
        throw new ArgumentException("Empty sequence of tasks");
    }
    foreach (var task in taskList)
    {
        if (task == null)
        {
            throw new ArgumentException("Null task in sequence");
        }
    }
    return WhenMajorityImpl(taskList);
}

The interesting part is obviously in WhenMajorityImpl. It's mildly interesting to note that I create a copy of the sequence passed in to start with - I know I'll need it in a fairly concrete form, so it's appropriate to remove any laziness at this point.

So, here's WhenMajorityImpl, which I'll then explain:

private static async Task<T> WhenMajorityImpl<T>(List<Task<T>> tasks)
{
    // Need a real majority - so for 4 or 5 tasks, must have 3 equal results.
    int majority = (tasks.Count / 2) + 1;
    int failures = 0;
    int bestCount = 0;

    Dictionary<T, int> results = new Dictionary<T, int>();
    List<Exception> exceptions = new List<Exception>();
    while (true)
    {
        await TaskEx.WhenAny(tasks);
        var newTasks = new List<Task<T>>();
        foreach (var task in tasks)
        {
            switch (task.Status)
            {
                case TaskStatus.Canceled:
                    failures++;
                    break;
                case TaskStatus.Faulted:
                    failures++;
                    exceptions.Add(task.Exception.Flatten());
                    break;
                case TaskStatus.RanToCompletion:
                    int count;
                    // Doesn't matter whether it was there before or not - we want 0 if not anyway
                    results.TryGetValue(task.Result, out count);
                    count++;
                    if (count > bestCount)
                    {
                        bestCount = count;
                        if (count >= majority)
                        {
                            return task.Result;
                        }
                    }
                    results[task.Result] = count;
                    break;
                default:
                    // Keep going next time. may not be appropriate for Created
                    newTasks.Add(task);
                    break;
            }
        }
        // The new list of tasks to wait for
        tasks = newTasks;

        // If we can't possibly work, bail out.
        if (tasks.Count + bestCount < majority)
        {
            throw new AggregateException("No majority result possible", exceptions);
        }
    }
}

I should warn you that this isn't a particularly efficient implementation - it was just one I wrote until it worked. The basic steps are:

Work out how many results make a majority, so we know when to stop
Keep track of how many "votes" our most commonly-returned result has, along with the counts of all the votes
Repeatedly:

Wait (asynchronously) for at least of the remaining tasks to finish (many may finish "at the same time")
Start a new list of "tasks we're going to wait for next time"
Process each task in the current list, taking an action on each state:

If it's been cancelled, we'll treat that as a failure (we could potentially treat "the majority have been cancelled" as a cancellation, but for the moment a failure is good enough)
If it's faulted, we'll add the exception to the list of exceptions, so that if the overall result ends up as failure, we can throw an AggregateException with all of the individual exceptions
If it's finished successfully, we'll check the result:

Add 1 to the count for that result (the dictionary will use the default comparer for the result type, which we assume is good enough)
If this is greater than the previous "winner" (which could be for the same result), check for it being actually an overall majority, and return if so.

If it's still running (or starting), add it to the new task list

Check whether enough tasks have failed - or given different results - so ensure that a majority is now impossible. If so, throw an AggregateException to say so. This may have some exceptions, but it may not (if there are three tasks which gave different results, none of them actually failed)

Each iteration of the "repeatedly" will have a smaller list to check than before, so we'll definitely terminate at some point.

I mentioned that it's inefficient. In particular, we're ignoring the fact that WhenAny returns a Task<Task<T>>, so awaiting that will actually tell us a task which has finished. We don't need to loop over the whole collection at that point - we could just remove that single task from the collection. We could do that efficiently if we kept a Dictionary<Task<T>, LinkedListNode<Task<T>> and a LinkedList<Task<T>> - we'd just look up the task which had completed in the dictionary, remove its node from the list, and remove the entry from the dictionary. We wouldn't need to create a new collection each time, or iterate through all of the old one. However, that's a job for another day... as is allowing a cancellation token to be passed in, and a custom equality comparer.

Conclusion

So we can make this implementation smarter and more flexible, certainly - but it's not insanely tricky to write. I'm reasonably confident that it works, too - as I have unit tests for it. They'll come in the next part. The important point from this post is that by sticking within the Task<T> world, we can reasonably easily create building blocks to allow for composition of asynchronous operations. While it would be nice to have someone more competent than myself write a bullet-proof, efficient implementation of this operation, I wouldn't feel too unhappy using a homegrown one in production. The same could not have been said pre-async/await. I just wouldn't have had a chance of getting it right.

Next up - the unit tests for this code, in which I introduce the TimeMachine class.

Eduasync part 14: Data passing in coroutines

skeet — Wed, 29 Jun 2011 17:18:20 GMT

(This post covers project 19 in the source code.)

Last time we looked at independent coroutines running in a round-robin fashion. This time we'll keep the round-robin scheduling, but add in the idea of passing data from one coroutine to another. Each coroutine will act on data of the same type, which is necessary for the scheme to work when one coroutine could "drop out" of the chain by returning.

Designing the data flow

It took me a while to get to the stage where I was happy with the design of how data flowed around these coroutines. I knew I wanted a coordinator as before, and that it should have a Yield method taking the value to pass to the next coroutine and returning an awaitable which would provide the next value when it completed. The tricky part was working out what to do at the start of each method and the end. If the method just took a Coordinator parameter, we wouldn't have anything to do with the value yielded by the first coroutine, because the second coroutine wouldn't be ready to accept it yet. Likewise when a coroutine completed, we wouldn't have another value to pass to the next coroutine.

Writing these dilemmas out in this post, the solution seems blindingly obvious of course: each coroutine should accept a data value on entry, and return one at the end. At any point where we transfer control, we provide a value and have a value which is required by something. The final twist is to make the coordinator's Start method take an initial value and return the value returned by the last coroutine to complete.

So, that's the theory... let's look at the implementation.

Initialization

I've changed the coordinator to take all the coroutines as a constructor parameter (of the somewhat fearsome declaration "params Func<Coordinator<T>, T, Task<T>>[] coroutines") which means we don't need to implement IEnumerable pointlessly any more.

This leads to a code skeleton of this form:

private static void Main(string[] args)
{
    var coordinator = new Coordinator<string>(FirstCoroutine,
                                              SecondCoroutine,
                                              ThirdCoroutine);
    string finalResult = coordinator.Start("m1");
    Console.WriteLine("Final result: {0}", finalResult);
}

private static async Task<string> FirstCoroutine(
    Coordinator<string> coordinator,
    string initialValue)
{
    ...
}

// Same signature for SecondCoroutine and ThirdCoroutine

Last time we simply had a Queue<Action> internally in the coordinator as the actions to invoke. You might be expecting a Queue<Func<T, T>> this time - after all, we're passing in data and returning data at each point. However, the mechanism for that data transfer is "out of band" so to speak. The only time we really "return" an item is when we reach the end of a coroutine. Usually we'll be providing data to the next step using a method. Likewise the only time a coroutine is given data directly is in the first call - after that, it will have to fetch the value by calling GetResult() on the awaiter which it uses to yield control.

All of this is leading to a requirement for our constructor to convert each coroutine delegate into a simple Action. The trick is working out how to deal with the data flow. I'm going to include SupplyValue() and ConsumeValue() methods within the coordinator for the awaiter to use, so it's just a case of calling those appropriately from our action. In particular:

When the action is called, it should consume the current value.
It should then call the coroutine passing in the coordinator ("this") and the initial value.
When the task returned by the coroutine has completed, the result of that task should be used to supply a new value.

The only tricky part here is the last bullet - and it's not that hard really, so long as we remember that we're absolutely not trying to start any new threads. We just want to hook onto the end of the task, getting a chance to supply the value before the next coroutine tries to pick it up. We can do that using Task.ContinueWith, but passing in TaskContinuationOptions.ExecuteSynchronously so that we use the same thread that the task completes on to execute the continuation.

At this point we can implement the initialization part of the coordinator, assuming the presence of SupplyValue() and ConsumeValue():

public sealed class Coordinator<T>
{
    private readonly Queue<Action> actions;
    private readonly Awaitable awaitable;

    public Coordinator(params Func<Coordinator<T>, T, Task<T>>[] coroutines)
    {
        // We can't refer to "this" in the variable initializer. We can use
        // the same awaitable for all yield calls.
        this.awaitable = new Awaitable(this);
        actions = new Queue<Action>(coroutines.Select(ConvertCoroutine));
    }

    // Converts a coroutine into an action which consumes the current value,
    // calls the coroutine, and attaches a continuation to it so that the return
    // value is used as the new value.
    private Action ConvertCoroutine(Func<Coordinator<T>, T, Task<T>> coroutine)
    {
        return () =>
        {
            Task<T> task = coroutine(this, ConsumeValue());
            task.ContinueWith(ignored => SupplyValue(task.Result),
                TaskContinuationOptions.ExecuteSynchronously);
        };
    }
}

I've broken ConvertCoroutine into a separate method so that we can use it as the projection for the Select call within the constructor. I did initially have it within a lambda expression within the constructor, but it was utterly hideous in terms of readabililty.

One suggestion I've received is that I could declare a new delegate type instead of using Func<Coordinator<T>, T, Task<T>> to represent a coroutine. This could either be a non-generic delegate nested in the generic coordinator class, or a generic stand-alone delegate:

public delegate T Coroutine<T>(Coordinator<T> coordinator, T initialValue);

// Or nested...
public sealed class Coordinator<T>
{
public delegate T Coroutine(Coordinator<T> coordinator, T initialValue);
}

Both of these would work perfectly well. I haven't made the change at the moment, but it's certainly worth considering. The debate about whether to use custom delegate types or Func/Action is one for another blog post, I think :)

The one bit of the initialization I haven't explained yet is the "awaitable" field and the Awaitable type. They're to do with yielding - so let's look at them now.

Yielding and transferring data

Next we need to work out how we're going to transfer data and control between the coroutines. As I've mentioned, we're going to use a method within the coordinator, called from the coroutines, to accomplish this. The coroutines have this sort of code:

private static async Task<string> FirstCoroutine(
    Coordinator<string> coordinator,
    string initialValue)
{
    Console.WriteLine("Starting FirstCoroutine with initial value {0}",
                      initialValue);
    ...
    string received = await coordinator.Yield("x1");
    Console.WriteLine("Returned to FirstCoroutine with value {0}", received);
    ...
    return "x3";
}

The method name "Yield" here is a double-edged sword. The word has two meanings - yielding a value to be used elsewhere, and yielding control until we're called back. Normally it's not ideal to use a name that can mean subtly different things - but in this case we actually want both of these meanings.

So, what does Yield need to do? Well, the flow control should look something like this:

Coroutine calls Yield()
Yield() calls SupplyValue() internally to remember the new value to be consumed by the next coroutine
Yield() returns an awaitable to the coroutine
Due to the await expression, the coroutine calls GetAwaiter() on the awaitable to get an awaiter
The coroutine checks IsCompleted on the awaiter, which must return false (to prompt the remaining behaviour)
The coroutine calls OnCompleted() passing in the continuation for the rest of the method
The coroutine returns to its caller
The coordinator proceeds with the next coroutine
When we eventually get back to this coroutine, it will call GetResult() to get the "current value" to assign to the "received" variable.

Now you'll see that Yield() needs to return some kind of awaitable type - in other words, one with a GetAwaiter() method. Previously we put this directly on the Coordinator type, and we could have done that here - but I don't really want anyone to just "await coordinator" accidentally. You should really need to call Yield() in order to get an awaitable. So we have an Awaitable type, nested in Coordinator.

We then need to decide what the awaiter type is - the result of calling GetAwaiter() on the awaitable. This time I decided to use the Coordinator itself. That means people could accidentally call IsCompleted, OnCompleted() or GetResult(), but I figured that wasn't too bad. If we were to go to the extreme, we'd create another type just for the Awaiter as well. It would need to have a reference to the coordinator of course, in order to actually do its job. As it is, we can make the Awaitable just return the Coordinator that created it. (Awaitable is nested within Coordinator<T>, which is how it can refer to T without being generic itself.)

public sealed class Awaitable
{
    private readonly Coordinator<T> coordinator;

    internal Awaitable(Coordinator<T> coordinator)
    {
        this.coordinator = coordinator;
    }

    public Coordinator<T> GetAwaiter()
    {
        return coordinator;
    }
}

The only state here is the coordinator, which is why we create an instance of Awaitable on the construction of the Coordinator, and keep it around.

Now Yield() is really simple:

public Awaitable Yield(T value)
{
SupplyValue(value);
return awaitable;
}

So to recap, we now just need the awaiter members, SupplyValue() and ConsumeValue(). Let's look at the awaiter members (in Coordinator) to start with. We already know that IsCompleted will just return false. OnCompleted() just needs to stash the continuation in the queue, and GetResult() just needs to consume the "current" value and return it:

public bool IsCompleted { get { return false; } }

public void OnCompleted(Action continuation)
{
actions.Enqueue(continuation);
}

public T GetResult()
{
return ConsumeValue();
}

Simple, huh? Finally, consuming and supplying values:

private T currentValue;
private bool valuePresent;

private void SupplyValue(T value)
{
    if (valuePresent)
    {
        throw new InvalidOperationException
            ("Attempt to supply value when one is already present");
    }
    currentValue = value;
    valuePresent = true;
}

private T ConsumeValue()
{
    if (!valuePresent)
    {
        throw new InvalidOperationException
            ("Attempt to consume value when it isn't present");
    }
    T oldValue = currentValue;
    valuePresent = false;
    currentValue = default(T);
    return oldValue;
}

These are relatively long methods (compared with the other ones I've shown) but pretty simple. Hopefully they don't need explanation :)

The results

Now that everything's in place, we can run it. I haven't posted the full code of the coroutines, but you can see it on Google Code. Hopefully the results speak for themselves though - you can see the relevant values passing from one coroutine to another (and in and out of the Start method).

Starting FirstCoroutine with initial value m1
Yielding 'x1' from FirstCoroutine...
    Starting SecondCoroutine with initial value x1
    Starting SecondCoroutine
    Yielding 'y1' from SecondCoroutine...
        Starting ThirdCoroutine with initial value y1
        Yielding 'z1' from ThirdCoroutine...
Returned to FirstCoroutine with value z1
Yielding 'x2' from FirstCoroutine...
    Returned to SecondCoroutine with value x2
    Yielding 'y2' from SecondCoroutine...
        Returned to ThirdCoroutine with value y2
        Finished ThirdCoroutine...
Returned to FirstCoroutine with value z2
Finished FirstCoroutine
    Returned to SecondCoroutine with value x3
    Yielding 'y3' from SecondCoroutine...
    Returned to SecondCoroutine with value y3
    Finished SecondCoroutine
Final result: y4

Conclusion

I'm not going to claim this is the world's most useful coroutine model - or indeed useful at all. As ever, I'm more interested in thinking about how data and control flow can be modelled than actual usefulness.

In this case, it was the realization that everything should accept and return a value of the same type which really made it all work. After that, the actual code is pretty straightforward. (At least, I think it is - please let me know if any bits are confusing, and I'll try to elaborate on them.)

Next time we'll look at something more like a pipeline model - something remarkably reminiscent of LINQ, but without taking up as much stack space (and with vastly worse readability, of course). Unfortunately the current code reaches the limits of my ability to understand why it works, which means it far exceeds my ability to explain why it works. Hopefully I can simplify it a bit over the next few days.

Eduasync part 13: first look at coroutines with async

skeet — Wed, 22 Jun 2011 20:37:30 GMT

(This part covers project 18 in the source code.)

As I mentioned in earlier parts, the "awaiting" part of async methods is in no way limited to tasks. So long as we have a suitable GetAwaiter() method which returns a value of a type which in turn has suitable methods on it, the compiler doesn't really care what's going on. It's time to exploit that to implement some form of coroutines in C#.

Introduction to coroutines

The fundamental idea of coroutines is to have multiple methods executing cooperatively, each of them maintaining their position within the coroutine when they yield to another. You can almost think of them as executing in multiple threads, with only one thread actually running at a time, and signalling between the different threads to control flow. However, we don't really need multiple threads once we've got continuations - we can have a single thread with a complex flow of continuations, and still only a very short "real" stack. (The control flow is stored in normal collections instead of being implicit on the thread's stack.)

Coroutines were already feasible in C# through the use of iterator blocks, but the async feature of C# allows a slightly more natural way of expressing them, in my view. (The linked Wikipedia page gives a sketch of how coroutines can be built on top of generators, which in the general concept that iterator blocks implement in C#.)

I have implemented various flavours of coroutines in Eduasync. It's possible that some (all?) of them shouldn't strictly be called coroutines, but they're close enough to the real thing in feeling. This is far from an exhaustive set of approaches. Once you've got the basic idea of what I'm doing, you may well want to experiment with your own implementations.

I'm not going to claim that the use of coroutines in any of my examples really makes any sense in terms of making real tasks easier. This is purely for the sake of interest and twisting the async feature for fun.

Round-robin independent coroutines

Our first implementation of coroutines is relatively simple. A coordinator effectively "schedules" the coroutines it's set up with in a round-robin fashion: when one of the coroutines yields control to the coordinator, the coordinator remembers where the coroutine had got to, and then starts the next one. When each coroutine has executed its first piece of code and yielded control, the coordinator will go back to the first coroutine to continue execution, and so on until all coroutines have completed.

The coroutines don't know about each other, and no data is being passed between them.

Hopefully it's reasonably obvious that the coordinator contains all the smarts here - the coroutines themselves can be relatively dumb. Let's look at what the client code looks like (along with the results) before we get to the coordinator code.

Client code

The sample code contains three coroutines, all of which take a Coordinator parameter and have a void return type. These are passed to a new coordinator using a collection initializer and method group conversions; the coordinator is then started. Here's the entry point code for this:

private static void Main(string[] args)
{
    var coordinator = new Coordinator {
        FirstCoroutine,
        SecondCoroutine,
        ThirdCoroutine
    };
    coordinator.Start();
}

When each coroutine is initially started, the coordinator passes a reference to itself as the argument to the coroutine. That's how we solve the chicken-and-egg problem of the coroutine and coordinator having to know about each other. The way a coroutine yields control is simply by awaiting the coordinator. The result type of this await expression is void - it's just a way of "pausing" the coroutine.

We're not doing anything interesting in the actual coroutines - just tracing the execution flow. Of course we could do anything we wanted, within reason. We could even await a genuinely asynchronous task such as fetching a web page asynchronously. In that case the whole coroutine collection would be "paused" until the fetch returned.

Here's the code for the first coroutine - the second and third ones are similar, but use different indentation for clarity. The third coroutine is also shorter, just for fun - it only awaits the coordinator once.

private static async void FirstCoroutine(Coordinator coordinator)
{
    Console.WriteLine("Starting FirstCoroutine");
    Console.WriteLine("Yielding from FirstCoroutine...");

    await coordinator;

    Console.WriteLine("Returned to FirstCoroutine");
    Console.WriteLine("Yielding from FirstCoroutine again...");

    await coordinator;

    Console.WriteLine("Returned to FirstCoroutine again");
    Console.WriteLine("Finished FirstCoroutine");
}

And here's the output...

Starting FirstCoroutine
Yielding from FirstCoroutine...
    Starting SecondCoroutine
    Yielding from SecondCoroutine...
        Starting ThirdCoroutine
        Yielding from ThirdCoroutine...
Returned to FirstCoroutine
Yielding from FirstCoroutine again...
    Returned to SecondCoroutine
    Yielding from SecondCoroutine again...
        Returned to ThirdCoroutine
        Finished ThirdCoroutine...
Returned to FirstCoroutine again
Finished FirstCoroutine
    Returned to SecondCoroutine again
    Finished SecondCoroutine

Hopefully that's the output you expected, given the earlier description. Again it may help if you think of the coroutines as running in separate pseudo-threads: the execution within each pseudo-thread is just linear, and the timing is controlled by our explicit "await" expressions. All of this would actually be pretty easy to implement using multiple threads which really did just block on each await expression - but the fun part is keeping it all in one real thread. Let's have a look at the coordinator.

The Coordinator class

Some of the later coroutine examples end up being slightly brainbusting, at least for me. This one is relatively straightforward though, once you've got the basic idea. All we need is a queue of actions to execute. After initialization, we want our queue to contain the coroutine starting points.

When a coroutine yields control, we just need to add the remainder of it to the end of the queue, and move on to the next item. Obviously the async infrastructure will provide "the remainder of the coroutine" as a continuation via the OnContinue method.

When a coroutine just returns, we continue with the next item in the queue as before - it's just that we won't add a continuation to the end of the queue. Eventually (well, hopefully) we'll end up with an empty queue, at which point we can stop.

Initialization and a choice of data structures

We'll represent our queue using Queue<T> where the T is a delegate type. We have two choices here though, because we have two kinds of delegate - one which takes the Coordinator as a parameter (for the initial coroutine setup) and one which has no parameters (for the continuations). Fortunately we can convert between the two in either direction very simply, bearing in mind that all of this is within the context of a coordinator. For example:

// If we're given a coroutine and want a plain Action
Action<Coordinator> coroutine = ...;
Action action = () => coroutine(this);

// If we're given a plain Action and want an Action<Continuation>:
Action continuation = ...;
Action<Coordinator> coroutine = ignored => continuation();

I've arbitrarily chosen to use the first option, so there's a Queue<Action> internally.

Now we need to get the collection initializer working. The C# compiler requires an appropriate Add method (which is easy) and also checks that the type implements IEnumerable. We don't really need to be able to iterate over the queue of actions, so I've use explicit interface implementation to reduce the likelihood of GetEnumerator() being called inappropriately, and made the method throw an exception for good measure. That gives us the skeleton of the class required for setting up:

public sealed class Coordinator : IEnumerable
{
    private readonly Queue<Action> actions = new Queue<Action>();

    // Used by collection initializer to specify the coroutines to run
    public void Add(Action<Coordinator> coroutine)
    {
        actions.Enqueue(() => coroutine(this));
    }

    // Required for collection initializers, but we don't really want
    // to expose anything.
    IEnumerator IEnumerable.GetEnumerator()
    {
        throw new NotSupportedException("IEnumerable only supported to enable collection initializers");
    }
}

(Note that I haven't used XML documentation anywhere here - it's great for real code, but adds clutter in blog posts.)

For production code I'd probably prevent Add from being called after the coordinator had been started, but there's no need to do it in our well-behaved sample code. We're only going to add extra actions to the queue via continuations, which will be added due to await expressions.

The main execution loop and async infrastructure

So far we've got code to register coroutines in the queue - so now we need to execute them. Bearing in mind that the actions themselves will be responsible for adding continuations, the main loop of the coordinator is embarrassingly simple:

// Execute actions in the queue until it's empty. Actions add *more*
// actions (continuations) to the queue by awaiting this coordinator.
public void Start()
{
    while (actions.Count > 0)
    {
        actions.Dequeue().Invoke();
    }
}

Of course, the interesting bit is the code which supports the async methods and await expressions. We know we need to provide a GetAwaiter() method, but what should that return? Well, we're just going to use the awaiter to add a continuation to the coordinator's queue. It's got no other state than that - so we might as well return the coordinator itself, and put the other infrastructure methods directly in the coordinator.

Again, this is slightly ugly, as the extra methods don't really make sense on the coordinator - we wouldn't want to call them directly from client code, for example. However, they're fairly irrelevant - we could always create a nested type which just had a reference to its "parent" coordinator if we wanted to. For simplicity, I haven't bothered with this - I've just implemented GetAwaiter() trivially:

// Used by await expressions to get an awaiter
public Coordinator GetAwaiter()
{
return this;
}

So, that leaves just three members still to implement: IsCompleted, OnCompleted and GetResult. We always want the IsCompleted property to return false, as otherwise the coroutine will just continue executing immediately without returning to cede control; the await expression would be pointless. OnCompleted just needs to add the continuation to the end of the queue - we don't need to attach it to a task, or anything like that. Finally, GetResult is a no-op - we have no results, no exceptions, and basically nothing to do. You might want to add a bit of logging here, if you were so inclined, but there's no real need.

So, here are the final three members of Coordinator:

// Force await to yield control
public bool IsCompleted { get { return false; } }

public void OnCompleted(Action continuation)
{
    // Put the continuation at the end of the queue, ready to
    // execute when the other coroutines have had a go.
    actions.Enqueue(continuation);
}

public void GetResult()
{
    // Our await expressions are void, and we never need to throw
    // an exception, so this is a no-op.
}

And that's it! Fewer than 50 lines of code required, and nothing complicated at all. The interesting behaviour is all due to the way the C# compiler uses the coordinator when awaiting it.

We need AsyncVoidMethodBuilder as before, as we have some async void methods - but that doesn't need to do anything significant. That's basically all the code required to implement these basic round-robin coroutines.

Conclusion

Our first foray into the weird and wonderful world of coroutines was relatively tame. The basic idea of a coordinator keeping track of the state of all the different coroutines in one sense or another will keep coming back to us, but with different ways of controlling the execution flow.

Next time we'll see some coroutines which can pass data to each other.

Eduasync part 12: Observing all exceptions

skeet — Wed, 22 Jun 2011 06:04:20 GMT

(This post covers projects 16 and 17 in the source code.)

Last time we looked at unwrapping an AggregateException when we await a result. While there are potentially other interesting things we could look at with respect to exceptions (particularly around cancellation) I'm just going to touch on one extra twist that the async CTP implements before I move on to some weird ways of using async.

TPL and unobserved exceptions

The Task Parallel Library (TPL) on which the async support is based has some interesting behaviour around exceptions. Just as it’s entirely possible for more than one thing to go wrong with a particular task, it's also quite easy to miss some errors, if you're not careful.

Here's a simple example of an async method in C# 5 where we create two tasks, both of which will throw exceptions:

private static async Task<int> CauseTwoFailures()
{
    Task<int> firstTask = Task<int>.Factory.StartNew(() => {
        throw new InvalidOperationException();
    });
    Task<int> secondTask = Task<int>.Factory.StartNew(() => {
        throw new InvalidOperationException();
    });

    int firstValue = await firstTask;
    int secondValue = await secondTask;

    return firstValue + secondValue;
}

Now the timing of the two tasks is actually irrelevant here. The first task will always throw an exception, which means we're never going to await the second task. That means there's never any code which asks for the second task's result, or adds a continuation to it. It's alone and unloved in a cruel world, with no-one to observe the exception it throws.

If we call this method from the Eduasync code we've got at the moment, and wait for long enough (I've got a call to GC.WaitForPendingFinalizers in the same code) the program will abort, with this error:

Unhandled Exception: System.AggregateException: A Task's exception(s) were not observed either by Waiting on the Task or accessing its Exception property. As a result, the unobserved exception was rethrown by the finalizer thread. ---> System.InvalidOperationException: Operation is not valid due to the current state of the object.

Ouch. The TPL takes a hard line on unobserved exceptions. They indicate failures (presumably) which you'll never find out about until you start caring about the result of a task. Basically there are various ways of "observing" a task's failure, whether by performing some act which causes it to be thrown (usually as part of an AggregateException) or just asking for the exception for a task which is known to be faulted. An unobserved exception will throw an InvalidOperationException in its finalizer, usually causing the process to exit.

That works well in "normal" TPL code, where you're explicitly managing tasks - but it's not so handy in async, where perfectly reasonable looking code which starts a few tasks and then awaits them one at a time (possibly doing some processing in between) might hide an unobserved exception.

Observing all exceptions

Fortunately TPL provides a way of us to get out of the normal task behaviour. There's an event TaskScheduler.UnobservedTaskException which is fired by the finalizer before it goes bang. The handlers of the event are allowed to observe the exception using UnobservedTaskExceptionEventArgs.SetObserved and can also check whether it's already been observed.

So all we have to do is add a handler for the event and our program doesn't crash any more:

TaskScheduler.UnobservedTaskException += (sender, e) =>
{
    Console.WriteLine("Saving the day! This exception would have been unobserved: {0}",
                      e.Exception);
    e.SetObserved();
};

In Eduasync this is currently only performed explicitly, in project 17. In the async CTP something like this is performed as part of the type initializer for AsyncTaskMethodBuilder<T>, which you can unfortunately tell because that type initializer crashes when running under medium trust. (That issue will be fixed before the final release.)

Global changes

This approach has a very significant effect: it changes the global behaviour of the system. If you have a system which uses the TPL and you want the existing .NET 4 behaviour of the process terminating when you have unobserved exceptions, you basically can't use async at all - and if you use any code which does, you'll see the more permissive behaviour.

You could potentially add your own event handler which aborted the application forcibly, but that's not terribly nice either. You should quite possibly add a handler to at least log these exceptions, so you can find out what's been going wrong that you haven't noticed.

Of course, this only affects unobserved exceptions - anything you're already observing will not be affected. Still, it's a pretty big change. I wouldn't be surprised if this aspect of the behaviour of async in C# 5 changed before release; it feels to me like it isn't quite right yet. Admittedly I'm not sure how I would suggest changing it, but effectively reversing the existing behaviour goes against Microsoft's past behaviour when it comes to backwards compatibility. Watch this space.

Conclusion

It's worth pondering this whole issue yourself (and the one from last time), and making your feelings known to the team. I think it's symptomatic of a wider problem in software engineering: we're not really very good at handling errors. Java's approach of checked exceptions didn't turn out too well in my view, but the "anything goes" approach of C# has problems too... and introducing alternative models like the one in TPL makes things even more complicated. I don't have any smart answers here - just that it's something I'd like wiser people than myself to think about further.

Next, we're going to move a bit further away from the "normal" ways of using async, into the realm of coroutines. This series is going to get increasingly obscure and silly, all in the name of really getting a feeling for the underlying execution model of async, before it possibly returns to more sensible topics such as task composition.

Eduasync part 11: More sophisticated (but lossy) exception handling

skeet — Wed, 22 Jun 2011 05:25:46 GMT

(This post covers projects 13-15 in the source code.)

Long-time readers of this blog may not learn much from this post - it's mostly going over what I've covered before. Still, it's new to Eduasync.

Why isn't my exception being caught properly?

Exceptions are inherently problematic in C# 5. There are two conflicting aspects:

The point of the async feature in C# 5 is that you can write code which mostly looks like its synchronous code. We expect to be able to catch specific exception types as normal.
Asynchronous code may potentially have multiple exceptions "at the same time". The language simply isn't designed to deal with that in the synchronous case.

Now if the language had been designed for asynchrony to start with, perhaps exception flow would have been designed differently - but we are where we are, and we all expect exceptions to work in a certain way.

Let's make all of this concrete with a sample:

private static void Main(string[] args)
{
    Task<int> task = FetchOrDefaultAsync();
    Console.WriteLine("Result: {0}", task.Result);
}

private static async Task<int> FetchOrDefaultAsync()
{
    // Nothing special about IOException here
    try
    {
        Task<int> fetcher = Task<int>.Factory.StartNew(() => { throw new IOException(); });
        return await fetcher;
    }
    catch (IOException e)
    {
        Console.WriteLine("Caught IOException: {0}", e);
        return 5;
    }
    catch (Exception e)
    {
        Console.WriteLine("Caught arbitrary exception: {0}", e);
        return 10;
    }
}

Here we have a task which will throw an IOException, and some code which awaits that task - and has a catch block for IOException.

So, what would you expect this to print? With the code we've got in Eduasync so far, we get this:

Caught arbitrary exception: System.AggregateException: One or more errors occurred. ---> System.IO.IOException: I/O error occurred.
...
Result: 10

If you run the same code against the async CTP, you get this:

Caught IOException: System.IO.IOException: I/O error occurred.
...
Result: 5

Hmm... we're not behaving as per the CTP, and we're not behaving as we'd really expect the normal synchronous code to behave.

The first thing to work out is which boundary we should be fixing. In this case, the problem is between the async method and the task we're awaiting, so the code we need to fix is TaskAwaiter<T>.

Handling AggregateException in TaskAwaiter<T>

Before we can fix it, we need to work out what's going on. The stack trace I hid before actually show this reasonably clearly, with this section:

at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceled Exceptions)
at System.Threading.Tasks.Task`1.get_Result()
at Eduasync.TaskAwaiter`1.GetResult() ...\TaskAwaiter.cs:line 40
at Eduasync.Program.<FetchOrDefaultAsync>d__2.MoveNext() ...\Program.cs:line 37

So AggregateException is being thrown by Task<T>.Result, which we're calling from TaskAwaiter<T>.GetResult(). The documentation for Task<T>.Result isn't actually terribly revealing here, but the fact that Task<T>.Exception is of type AggregateException is fairly revealing.

Basically, the Task Parallel Library is built with the idea of multiple exceptions in mind - whereas our async method isn't.

Now the team in Microsoft could have decided that really you should catch AggregateException and iterate over all the exceptions contained inside the exception, handling each of them separately. However, in most cases that isn't really practical - because in most cases there will only be one exception (if any) and all that looping is relatively painful. They decided to simply extract the first exception from the AggregateException within a task, and throw that instead.

We can do that ourselves in TaskAwaiter<T>, like this:

try
{
    return task.Result;
}
catch (AggregateException aggregate)
{
    if (aggregate.InnerExceptions.Count > 0)
    {
        // Loses the proper stack trace. Oops. For workarounds, see
        // See http://bradwilson.typepad.com/blog/2008/04/small-decisions.html
        throw aggregate.InnerExceptions[0];
    }
    else
    {
        // Nothing better to do, really...
        throw;
    }
}

As you can tell from the comment, we end up losing the stack trace using this code. I don't know exactly how the stack trace is preserved in the real Async CTP, but it is. I suspect this is done in a relatively obscure way at the moment - it's possible that for .NET 5, there'll be a cleaner way that all code can take advantage of.

This code is also pretty ugly, catching the exception only to rethrow it. We can check whether or not the task has faulted using Task<T>.Status and extract the AggregateException using Task<T>.Exception instead of forcing it to be thrown and catching it. We'll see an example of that in a minute.

With our new code in place, we can catch the IOException in our async code very easily.

What if I want all the exceptions?

In certain circumstances it really makes sense to collect multiple exceptions. This is particularly true when you're waiting for multiple tasks to complete, e.g. with TaskEx.WhenAll in the Async CTP. This caused me a certain amount of concern for a while, but when Mads came to visit in late 2010 and we talked it over, we realized we could use the compositional nature of Task<T> and the convenience of TaskCompletionSource to implement an extension method preserving all the exceptions.

As we've seen, when a task is awaited, its AggregateException is unwrapped, and the first exception rethrown. So if we create a new task which adds an extra layer of wrapping and make our code await that instead, only the extra layer will be unwrapped by the task awaiter, leaving the original AggregateException. To come up with a new task which "looks like" an existing task, we can simply create a TaskCompletionSource, add a continuation to the original task, and return the completion source's task as the wrapper. When the continuation fires we'll set the appropriate result on the completion source - cancellation, an exception, or the successful result.

You may expect that we'd have to create a new AggregateException ourselves - but TaskCompletionSource.SetException will already do this for us. This makes it looks like the code below isn't performing any wrapping at all, but remember that Task<T>.Exception is already an AggregateException, and calling TaskCompletionSource.SetException will wrap it in another AggregateException. Here's the extension method in question:

public static Task<T> WithAllExceptions<T>(this Task<T> task)
{
    TaskCompletionSource<T> tcs = new TaskCompletionSource<T>();

    task.ContinueWith(ignored =>
    {
        switch (task.Status)
        {
            case TaskStatus.Canceled:
                tcs.SetCanceled();
                break;
            case TaskStatus.RanToCompletion:
                tcs.SetResult(task.Result);
                break;
            case TaskStatus.Faulted:
                // SetException will automatically wrap the original AggregateException
                // in another one. The new wrapper will be removed in TaskAwaiter, leaving
                // the original intact.
                tcs.SetException(task.Exception);
                break;
            default:
                tcs.SetException(new InvalidOperationException("Continuation called illegally."));
                break;
        }
    });

    return tcs.Task;
}

Here you can see the cleaner way of reacting to a task's status - we don't just try to fetch the result and catch any exceptions; we handle each status individually.

I don't know offhand what task scheduler is used for this continuation - it may be that we'd really want to specify the current task scheduler for a production-ready version of this code. However, the core idea is sound.

It's easy to use this extension method within an async method, as shown here:

private static async Task<int> AwaitMultipleFailures()
{
    try
    {
        await CauseMultipleFailures().WithAllExceptions();
    }
    catch (AggregateException e)
    {
        Console.WriteLine("Caught arbitrary exception: {0}", e);
        return e.InnerExceptions.Count;
    }
    // Nothing went wrong, remarkably!
    return 0;
}

private static Task<int> CauseMultipleFailures()
{
    // Simplest way of inducing multiple exceptions
    Exception[] exceptions = { new IOException(), new ArgumentException() };
    TaskCompletionSource<int> tcs = new TaskCompletionSource<int>();
    tcs.SetException(exceptions);
    return tcs.Task;
}

Note that this will work perfectly well with the Async CTP and should be fine with the full release as well. I wouldn't be entirely surprised to find something similar provided by the framework itself by release time, too.

Conclusion

It's worth being aware of the impedance mismatch between the TPL and async methods in C# 5, as well as how this mismatch is handled. I dislike the idea of data loss, but I can see why it's being handled in this way. It's very much in line with the approach of trying to make asynchronous methods look like synchronous ones as far as possible.

We'll probably look at the compositional nature of tasks again later in the series, but this was one simple example of how transparent it can be - a simple extension method can change the behaviour to avoid the risk of losing exception information when you're expecting that multiple things can go wrong.

It's worth remembering that this behaviour is very specific to Task and Task<T>, and the awaiter types associated with them. If you're awaiting other types of expressions, they may behave differently with respect to exceptions.

Before we leave the topic of exceptions, there's one other aspect we need to look at - what happens when an exception isn't observed...

Eduasync part 10: CTP bug - don't combine multiple awaits in one statement...

skeet — Fri, 10 Jun 2011 21:28:46 GMT

(This post covers project 12 in the source code.)

Last time, we saw what happens when we have multiple await expressions: we end up with multiple potential states in our state machine. That's all fine, but there's a known bug in the current CTP which affects the code generated when you have multiple awaits in the same statement. (Lucian Wischik calls these "nested awaits" although I personally don't really think of one as being nested inside an another.)

This post is mostly a cautionary tale for those using the CTP and deploying it to production - but it's also interesting to see the kind of thing that can go wrong. I'm sure this will all be fixed well before release, and the team are already very aware of it. (I expect it's been fixed internally for a while. It's not the kind of bug you'd hold off fixing.)

Just as a reminder, here's the code we used last time to demonstrate multiple await expressions:

// Code from part 9
private static async Task<int> Sum3ValuesAsyncWithAssistance()
{
    Task<int> task1 = Task.Factory.StartNew(() => 1);
    Task<int> task2 = Task.Factory.StartNew(() => 2);
    Task<int> task3 = Task.Factory.StartNew(() => 3);

    int value1 = await task1;
    int value2 = await task2;
    int value3 = await task3;

    return value1 + value2 + value3;
}

To simplify things a bit, let's reduce it to two tasks. Then as a refactoring, I'm going to perform the awaiting within the summation expression:

private static async Task<int> Sum2ValuesAsyncWithAssistance()
{
    Task<int> task1 = Task.Factory.StartNew(() => 1);
    Task<int> task2 = Task.Factory.StartNew(() => 2);

    return await task1 + await task2;
}

So, with the local variables value1 and value2 gone, we might expect that in the generated state machine, we'd have lost the corresponding fields. But should we, really?

Think what happens if task1 has completed (and we've fetched the results), but task2 hasn't completed by the time we await it. So we call OnContinue and exit as normal, then keep going when the continuation is invoked.

At that point we should be ready to return... but we've got to use the value returned from task1. We need the result to be stored somewhere, and fields are basically all we've got.

Unfortunately, in the current async CTP we don't have any of these - we just have two local awaiter result variables. Here's the decompiled code, stripped of the outer skeleton as normal:

int awaitResult1 = 0;
int awaitResult2 = 0;
switch (state)
{
      case 1:
          break;

      case 2:
          goto Label_Awaiter2Continuation;

      default:
          if (state != -1)
          {
              task1 = Task.Factory.StartNew(() => 1);
              task2 = Task.Factory.StartNew(() => 2);

              awaiter1 = task1.GetAwaiter();
              if (awaiter1.IsCompleted)
              {
                  goto Label_GetAwaiter1Result;
              }
              state = 1;
              doFinallyBodies = false;
              awaiter1.OnCompleted(moveNextDelegate);
          }
          return;
}
state = 0;
Label_GetAwaiter1Result:
awaitResult1 = awaiter1.GetResult();
awaiter1 = default(TaskAwaiter<int>());

awaiter2 = task2.GetAwaiter();
if (awaiter2.IsCompleted)
{
      goto Label_GetAwaiter2Result;
}
state = 2;
doFinallyBodies = false;
awaiter2.OnCompleted(moveNextDelegate);
return;
Label_Awaiter2Continuation:
state = 0;
Label_GetAwaiter2Result:
awaitResult2 = awaiter2.GetResult();
awaiter2 = default(TaskAwaiter<int>());

result = awaitResult1 + awaitResult2;

The first half of the code looks like last time (with the natural adjustments for only having two tasks) - but look at what happens when we've fetched the result of task1. We fetch the result into awaitResult1, which is a local variable. We then don't touch awaitResult1 again unless we reach the end of the code - which will not happen if awaiter2.IsCompleted returns false. When the method returns, the local variable's value is lost forever... although it will be reset to 0 next time we re-enter, and nothing in the generated code will detect a problem.

So depending on the timing of the two tasks, the final result can be 3 (if the second task has finished by the time it's checked), or 2 (if we need to add a continuation to task2 instead). This is easy to verify by forcing the tasks to sleep before they return.

Conclusion

The moral of this post is threefold:

In general, treat CTP and beta-quality code appropriately: I happened to run into this bug, but I'm sure there are others. This is in no way a criticism of the team. It's just a natural part of developing a complex feature.
To avoid this specific bug, all you have to do is make sure that all relevant state is safely stored in local variables before any logical occurrence of "await".
If you're ever writing a code generator which needs to store state, remember that that state can include expressions which have only been half evaluated so far.

For the moment, this is all I've got on the generated code. Next time, we'll start looking at how exceptions are handled, and in particular how the CTP behaviour differs from the implementation we've seen so far. After a few posts on exceptions, I'll start covering some daft and evil things I've done with async.

Eduasync part 9: generated code for multiple awaits

skeet — Mon, 30 May 2011 20:06:10 GMT

Last time we looked at a complex async method with nested loops and a single await. This post is the exact opposite - the method is going to look simple, but it will have three await expressions in. If you're glancing down this post and feel put off by the amount of code, don't worry - once you've got the hang of the pattern, it's really pretty simple.

Here's the async method we're going to analyze:

private static async Task<int> Sum3ValuesAsyncWithAssistance()
{
    Task<int> task1 = Task.Factory.StartNew(() => 1);
    Task<int> task2 = Task.Factory.StartNew(() => 2);
    Task<int> task3 = Task.Factory.StartNew(() => 3);

    int value1 = await task1;
    int value2 = await task2;
    int value3 = await task3;

    return value1 + value2 + value3;
}

Nice and straightforward: start three tasks, then sum their results. Before we look at the decompiled code, it's worth noting that writing it this way allows the three (admittedly trivial tasks) to run in parallel. If we'd written it this way instead:

private static async Task<int> Sum3ValuesAsyncWithAssistance()
{
    // No parallelism...
    int value1 = await Task.Factory.StartNew(() => 1);
    int value2 = await Task.Factory.StartNew(() => 2);
    int value3 = await Task.Factory.StartNew(() => 3);

    return value1 + value2 + value3;
}

... then we'd have waited for the first task to finish before starting the second one, then waited for the second one to complete before we started the third one. That's appropriate when there are dependencies between your tasks (i.e. you need the result of the first as an input to the second) and it would still have been asynchronous but when you can start multiple independent tasks together, that's generally what you want to do. Don't forget that this doesn't just extend to CPU-bound tasks - you might want to launch tasks making multiple web service calls in parallel, before collecting the results.

As before, I'll just show the heart of the generated code, without its boiler-plate skeleton. Again, the full code is available online.

Generated code

switch (state)
{
      case 1:
          break;

      case 2:
          goto Label_Awaiter2Continuation;

      case 3:
          goto Label_Awaiter3Continuation;

      default:
          if (state != -1)
          {
              task1 = Task.Factory.StartNew(() => 1);
              task2 = Task.Factory.StartNew(() => 2);
              task3 = Task.Factory.StartNew(() => 3);

              awaiter1 = task1.GetAwaiter();
              if (awaiter1.IsCompleted)
              {
                  goto Label_GetAwaiter1Result;
              }
              state = 1;
              doFinallyBodies = false;
              awaiter1.OnCompleted(moveNextDelegate);
          }
          return;
}
state = 0;
Label_GetAwaiter1Result:
int awaitResult1 = awaiter1.GetResult();
awaiter1 = new TaskAwaiter<int>();
value1 = awaitResult1;

awaiter2 = task2.GetAwaiter();
if (awaiter2.IsCompleted)
{
      goto Label_GetAwaiter2Result;
}
state = 2;
doFinallyBodies = false;
awaiter2.OnCompleted(moveNextDelegate);
return;
Label_Awaiter2Continuation:
state = 0;
Label_GetAwaiter2Result:
int awaitResult2 = awaiter2.GetResult();
awaiter2 = new TaskAwaiter<int>();
value2 = awaitResult2;

awaiter3 = task3.GetAwaiter();
if (awaiter3.IsCompleted)
{
      goto Label_GetAwaiter3Result;
}
state = 3;
doFinallyBodies = false;
awaiter3.OnCompleted(moveNextDelegate);
return;
Label_Awaiter3Continuation:
state = 0;
Label_GetAwaiter3Result:
int awaitResult3 = awaiter3.GetResult();
awaiter3 = new TaskAwaiter<int>();
value3 = awaitResult3;

result = value1 + value2 + value3;

Like last time, I'll just go through the interesting points this raises, rather than examining it line by line.

Switch instead of if

In all our previous async methods, we've only had three possible states: -1 (finished), 0 (normal), 1 (return from continuation). The generated code always looked something like this to start with:

if (state != 1)
{
    if (state == -1)
    {
        return;
    }
    // Execute code before the first await, loop or try
}

The comment is somewhat brief here, but the basic idea is that all the code which will only ever execute the first time (with no continuations) can go here. If there's a loop that contains an await, then a continuation would have to jump back into that loop, so that code couldn't be contained within this initial block. (A loop which didn't have any awaits in could though.)

Anyway, this time we don't have an "if" statement like that - we have a switch. It's the same idea, but we could be in any of three different states when a continuation is called, depending on which await expression we're at. The switch statement efficiently branches to the right place for a continuation, and executes the initial code otherwise. The "branch" for state 1 is just to exit the switch statement and continue from there.

It's possible that the generated code actually has more levels of indirection than it needs; I don't know about the details of what's allowed within an IL switch, but it seems odd to effectively have an "On X goto Y" where Y immediately performs a "goto Z". If the switch statement could branch immediately to the right label, we'd end up with IL which probably wouldn't have a hope of being decompiled to C#, but which might be slightly more efficient. It's quite likely that the JIT can sort all of that out, of course.

I tend to actually think about all of this as if the code that's really in the "default" case for the switch statement appeared after the switch, and the default case just contained a goto statement to jump to it. The effect would be exactly the same, of course - but it means I have a mental model of the method consisting of a "jump to the right place" phase before an "execute the code" phase. Just because I think of it that way doesn't mean you have to, of course :)

Multiple awaiters and await results

There's room for a bit more optimization in this specific case. We have three awaiters, but they're all of the same type (TaskAwaiter<int>). Likewise, we have three await results, but they're all int. (It would be possible to have different awaiter types with the same result type, of course.)

In the CTP (at least without optimization enabled) we end up with an awaiter / awaitResult pair of variables for each await expression. There's never more than one await "active" at any one time, so the C# compiler could generate one awaiter variable per awaiter type, and one result variable per result type. In the common situation where the result is being directly assigned to a "local" variable of the same type within the method, we don't really need the result variable at all. On the other hand, it's only a local variable (unlike the awaiter) and it's quite possible that the JIT can optimize this instead.

Ultimately it's entirely reasonable for the C# compiler to be generating suboptimal code at this point in the development cycle. After all, it could be quite easy to introduce bugs due to inappropriate code generation... as we'll see next time.

Conclusion

Other than the different way of getting to the right place on entry (using a switch instead of an if statement), async methods with multiple await expressions aren't that hard to follow. Of course when you combine multiple awaits with loops, try/catch, try/finally blocks and any number of other things you might use to complicate the async method, things become tricky - but with the fundamentals covered in this blog series, hopefully you'd be able to cope with the generated code in any reasonable situation. Of course, it's rare that you'll need (or want) to look at the generated code in anything like the detail we have here - but now we've looked at it, you don't need to wonder where the magic happens.

The next post will be the last one involving the decompiled code, at least for the moment. I'd like to demonstrate a bug in the CTP - mostly to show you how a small change to the async method can trigger the wrong results. I'm absolutely positive it will be fixed before release - probably for the next CTP or beta - but I think it's interesting to see the sort of situation which can cause problems.

After that, we're going to look at exception handling, before we move into a few odd way of using async - in particular, implementing coroutines and "COMEFROM".

Eduasync part 8: generated code from a complex async method

skeet — Sun, 29 May 2011 18:30:12 GMT

Last time we had a really simple async method, and looked at the generated code. Even that took a little while... but in this post I'll demonstrate some of the intricacies that get involved when the async method is more complex. In particular, this time we have:

A parameter to the async method
A "using" statement, so we'll need a finally block
Two loops
The possibility of catching an exception generated from await and retrying

I've not made it as complicated as I could have done, mind you - we've still only got one await expression. Here's the async method we'll be investigating:

private static async Task<int> WriteValuesAsyncWithAssistance(int loopCount)
{
    using (TextWriter writer = File.CreateText("output.txt"))
    {
        int sum = 0;
        for (int i = 0; i < loopCount; i++)
        {
            Task<int> valueFetcher = Task.Factory.StartNew(() => 1);

            for (int j = 0; j < 3; j++)
            {
                try
                {
                    Console.WriteLine("Awaiting...");
                    int value = await valueFetcher;
                    writer.WriteLine("Got value {0}", value);
                    sum += value;
                    break; // Break out of the inner for loop
                }
                catch (Exception)
                {
                    Console.WriteLine("Oops... retrying");
                }
            }
        }
        return sum;
    }
}

Okay, so it's already reasonably complicated even as a normal method. But it gets nastier when we look at the code generated for it. In particular, this time the decompiled code actually isn't quite valid C#. I'll get to that in a minute. I've split the rest of this post into several smallish chunks to try to help keep several concerns separated. They're not particularly pleasantly balanced in terms of size, segue etc... but hopefully they'll make things simpler.

The decompiled code

I'm going to ignore the standard skeleton - basically everything you see here is within a try/catch block so that we can call SetException on the builder if necessary, and if we reach the end of the method normally, we'll call SetResult on the builder using the "result" variable. If you want to see the complete method, it's in the source repository. Here's the cut down version:

bool doFinallyBodies = true;
int tmpState = state;

// Effectively a three way switch: 1, -1, 0
if (tmpState != 1)
{
    if (state == -1)
    {
        return;
    }
    this.writer = File.CreateText("output.txt");
}
try
{
    tmpState = state;
    if (tmpState == 1)
    {
        goto Label_ResumePoint;
    }
    sum = 0;
    i = 0;
Label_ResumePoint: // This shouldn't quite be here... see below
    while (i < loopCount)
    {
        // Not in generated code:
        if (state == 1)
        {
            goto Label_ResumePoint2;
        }
        // Back to generated code

        valueFetcher = Task.Factory.StartNew(() => 1);
        j = 0;

        // Still not in the generated code, and still not quite right... we don't want the j test here
      Label_ResumePoint2:
        // Back to generated code again...
        while (j < 3)
        {
            try
            {
                // We want Label_ResumePoint to be here really
                tmpState = state;
                if (tmpState != 1)
                {
                    Console.WriteLine("Awaiting...");
                    awaiter = valueFetcher.GetAwaiter();
                    if (!awaiter.IsCompleted)
                    {
                        state = 1;
                        doFinallyBodies = false;
                        awaiter.OnCompleted(moveNextDelegate);
                        return;
                    }
                }
                else
                {
                    state = 0;
                }
                int awaitResult = awaiter.GetResult();
                awaiter = new TaskAwaiter<int>();
                value = awaitResult;
                writer.WriteLine("Got value {0}", value);
                sum += value;
                break;
            }
            catch (Exception)
            {
                Console.WriteLine("Oops... retrying");
            }
            j++;
        }
        i++;
    }
    result = sum;
}
finally
{
    if (doFinallyBodies && writer != null)
    {
        writer.Dispose();
    }
}

Now that's a fair bit of code... but if you were comfortable with the last post, there shouldn't be much which is surprising. I'm not going to go over every aspect of it - just the surprising twists introduced by the extra complexities of the async method.

State and tmpState

Okay, let's start from a point of ignorance: I don't know why a local variable (tmpState) is introduced to take a copy of the state. My guess is that in some really complex scenarios it either becomes relevant when we want to change the state but also know what the state was, or if we're testing the state several times without changing it, it may be quicker to do so having copied it to a local variable. I think we can pretty much ignore it - I've only included it here for completeness.

To do finally bodies or not to do finally bodies, that is the question

Last time the doFinallyBodies variable was pointless because we didn't have any finally blocks - this time we do (albeit just the one). It's implicit due to the "using" statement in the original code, but that's really just a try/finally. The CreateText call occurs before we really enter into the using statement, which is why it's before the try block in the decompiled code.

Broadly speaking, a try/finally block in an async method corresponds to the same try/finally block in the generated code, with one big difference: if we're leaving the method because we're just returning having attached a continuation to an awaiter, we don't want to execute the finally body. After all, the method is effectively "paused" in the block rather than really exiting the block. So, any time you see an OnCompleted call, you'll see a "doFinallyBodies = false;" statement... and that flag will be set to true at the start of the method, so that at any other time, exiting the try block will cause the finally body to be executed. We still want the body to execute if there's an exception, for example. This is the same pattern used by the generated code for iterator blocks. (That's extremely unsurprising at the moment, as the code generators share a lot of the code. It's one bit which will have to be preserved if the two generators are split though.)

Go to where exactly?

As I mentioned earlier, we can't write C# code which exactly represented the generated IL. That's because when we enter the method in state 1 (i.e. for a continuation) the actual generated IL performs a goto straight into the start of the try block. You can't do that in straight C#, which is why I've got an ugly double-goto in my representation. It doesn't make it a 100% accurate representation of the IL, but it's close enough for our purposes. If you build the async method and look at the generated code in Reflector, you'll see that it basically shows a jump to the final comment above, just inside the try block. (Reflector actually shows the label as occurring before the try block, but the branch instruction in the IL really does refer to the first instruction in the try block.)

Now, you may be wondering why the generated code only jumps to the start of the try block. Why doesn't it just jump straight to the "await" part? Last time it jumped straight to the "state = 0;" statement, after all... so we might expect it to do the same thing here.

It turns out that although IL has somewhat more freedom than C# in terms of goto (branch) statements, it's not entirely free of restrictions. In this case, I believe it's running up against section 12.4.2.8.2.7 of ECMA-335 (the CLI specification) which states:

If any target of the branch is within a protected block, except the first instruction of that protected block, the source shall be within the same protected block.

[...]

[Note: Code can branch to the first instruction of a protected block, but not into the middle of one. end note]

For "protected block" we can read "try block" (at least in this case).

I really shouldn't enjoy the fact that even the C# compiler can't do exactly what it would like to here, but somehow it makes me feel better about the way that the C# language restricts us. It's possible that anthropomorphizing the C# compiler at all is a bad sign for my mental state, but never mind. It gets even worse if an await expression occurs in a nested try block - another "if" is required for each level involved.

The point is that by hook or by crook, when a continuation is invoked, we need to reach the assignment back to state 0, followed by the call to awaiter.GetResult().

Why do we go back to state 0?

You may have noticed that when we return from a continuation, we set the state to 0 before we call GetResult(). We only ever set to the state to a non-zero value just adding a continuation, or when we've finished (at which point we set it to -1). Therefore it will actually be 0 whenever we call GetResult() on an awaiter. But why? Shouldn't state 0 mean "we're just starting the method"? We know that state -1 means "the method has finished" and a positive state is used to indicate where we need to come back to when the continuation is called... so what sense does it make to reuse state 0?

This bit took me a while to work out. In fact, it had me so stumped that I asked a question on Stack Overflow about it. Various theories were presented, but none was entirely satisfactory. At the same time, I mailed Mads Torgersen and Lucian Wischik about it. After a while, Lucian got back to me and assured me that this wasn't a bug - it was very deliberate, and absolutely required. He didn't say why though - he challenged me to work it out, just in the knowledge that it was required. The example I eventually came up with was a slightly simpler version of the async method in this post.

The key is to think of states slightly differently:

-1: finished
0: running normally; no special action required
Anything else: Waiting to get back to a continuation point (or in the process of getting there)

So why must we go back to the "running normally" state as soon as we've reached the right point when running a continuation? Well, let's imagine the flow in a situation like the async method shown, but where instead of just creating a trivial task, we'd started one which could fail, and does... but hasn't finished before we first await it. Ignoring the outer loop and the using statement, we have something like this:

The task is created
We enter the inner loop
We write out "Awaiting..." on the console
We get an awaiter for the task, and check whether it's completed: it hasn't, so we add a continuation and return
The task completes, so we call GetResult() on the awaiter
GetResult() throws an exception, so we log it in the catch block
We go back round the inner loop, and want to re-enter step 3

Now look at the flow control within the decompiled method again, and imagine if we hadn't set the state to 0. We'd still be in state 1 when we reached the catch block, so when we got back to the top of the loop, we'd ignore the Console.WriteLine call and not even get a fresh awaiter - we'd end up reusing the same awaiter again. In other words, we'd try to get back to the continuation point.

To think of it another way: by the time we get back to the call to awaiter.GetResult(), the two possible paths (executing immediately vs adding a continuation, based on the value of awaiter.IsCompleted) have merged and we want to behave exactly the same way from that point onwards. So naturally we'd want to be in the same state... which is 0 for the synchronous path, as we haven't had any reason to change state.

Conclusion

All this complexity, and just with one await expression! As is so often true in computing, it all seems a bit simpler when you can look past it being a wall of code, and break down each lump separately. Hopefully I've taken some of the surprises out of the generated code here, although please do leave a comment if anything is still unclear - I'll try to update the post if so.

Next time we'll look at an async method which is generally simpler, but which has three await expressions.

Eduasync part 7: generated code from a simple async method

skeet — Fri, 20 May 2011 18:21:02 GMT

In part 6 we tried to come up with a "manual" translation of a very simple async method. This time we'll see what the compiler really generates.

As a reminder, here's the async method in question:

private static async Task<int> ReturnValueAsyncWithAssistance()
{
Task<int> task = Task<int>.Factory.StartNew(() => 5);

return await task;
}

Whereas our manual code still ended up with a single method (admittedly involving a lambda expression), the generated code is split into the method itself (which ends up being quite small), and a whole extra class representing a state machine.

The "stub" method

We looked at a similar translation before, but didn't really explain it. This time let's take a closer look:

private static Task<int> ReturnValueAsyncWithStateMachine()
{
    StateMachine stateMachine = new StateMachine(0);
    stateMachine.moveNextDelegate = stateMachine.MoveNext;
    stateMachine.builder = AsyncTaskMethodBuilder<int>.Create();
    stateMachine.MoveNext();
    return stateMachine.builder.Task;
}

Note that I've renamed all the variables and the generated type itself to be valid C# identifiers with useful names.

This is pretty much a minimal stub from an async method. If the async method took parameters, each parameter value would end up being copied into a field in the state machine - but other than that, this is what all async methods end up looking like. (Void async methods just won't return the builder's task at the end.)

The method doesn't have much work to do:

It creates a new instance of the state machine, passing in an initial state of 0. (I'd expect this constructor parameter to be removed by release - it'll always be 0, after all.)
It creates a new Action delegate instance corresponding to the MoveNext method in the state machine. This is always passed to awaiters as the continuation - by using a field in the state machine, we can avoid creating a new Action instance each time we call OnCompleted.
It creates a new AsyncTaskMethodBuilder which the state machine uses to set the result and/or exception, and which exposes a task we can return to the caller.
It calls MoveNext() to synchronously: this will execute at least as far as the first "await" before returning, and may get further than that if the awaited tasks have completed before the await expression.
It returns the Task<int> from the builder. By this point there may already be a result, if all the await expressions had already completed. In real async methods, most of the time the returned Task or Task<T> will represent an ongoing task which the state machine will keep track of.

Obviously, all the smarts are in the state machine...

The state machine (completed)

Here's the complete code of the generated state machine, as of the second CTP of the async feature.

[CompilerGenerated]
private sealed class StateMachine
{
    // Fields representing local variables
    public Task<int> task;

    // Fields representing awaiters
    private TaskAwaiter<int> awaiter;

    // Fields common to all async state machines
    public AsyncTaskMethodBuilder<int> builder;
    private int state;
    public Action moveNextDelegate;

    public StateMachine(int state)
    {
        // Pointless: will always be 0. Expect this to be removed from later builds.
        this.state = state;
    }

    public void MoveNext()
    {
        int result;
        try
        {
#pragma warning disable 0219 // doFinallyBodies is never used
            bool doFinallyBodies = true;
#pragma warning restore
            if (state != 1)
            {
                if (state != -1)
                {
                    task = Task<int>.Factory.StartNew(() => 5);
                    awaiter = task.GetAwaiter();
                    if (awaiter.IsCompleted)
                    {
                        goto Label_GetResult;
                    }
                    state = 1;
                    doFinallyBodies = false;
                    awaiter.OnCompleted(moveNextDelegate);
                }
                return;
            }
            state = 0;
          Label_GetResult:
            int awaitResult = awaiter.GetResult();
            awaiter = new TaskAwaiter<int>();
            result = awaitResult;
        }
        catch (Exception e)
        {
            state = -1;
            builder.SetException(e);
            return;
        }
        state = -1;
        builder.SetResult(result);
    }

    // Obsolete: will be removed from later builds.
#pragma warning disable 0414
    private bool disposing;
#pragma warning restore

    [DebuggerHidden]
    public void Dispose()
    {
        disposing = true;
        MoveNext();
        state = -1;
    }
}

Yes, it's pretty complicated. Let's break it down a bit...

Unused code

Some of the code here is pointless. It's a side-effect of the fact that for the CTP, which uses the same code for iterator blocks and async methods - that's why the primary method within the state machine is called MoveNext. So everything below the comment line starting "Obsolete" can be ignored.

Additionally, in this case we don't have any finally blocks - so the doFinallyBodies local variable is unused too. Later on we'll see an example which does use it, so don't worry about it for now.

Fields and initialization

I've explained all the fields in the comments, to some extent. Broadly speaking, there are three types of field in async method translations:

Variables for the infrastructure:
- The state number, used to work out where to jump back into the method when the continuation is called (or on our first call from the stub method)
- The builder, used to communicate with the task returned to the caller, via SetResult and SetException
- The moveNextDelegate, always passed to the OnCompleted method when an awaiter indicates an incomplete task
Awaiter variables: each await expression (currently) has its own awaiter variable. (This may be optimized in a later build.) Each one is set back to the type's default value (e.g. null for a reference type) after use, but we need to maintain this as a field so we can get the result of an awaiter when we return from a continuation.
"User" variables representing the local variables of the async method. If the method uses language features such as collection initializers, there may be more of these than expected, but basically they're whatever you'd see as local variables in a normal method. This also includes parameters for the async method, which are copied into the state machine by the stub method.

Standard skeleton

All async methods (well, those returning results) have a standard skeleton for the MoveNext() method. Void async methods differ in the obvious way. You may recognise this as being extremely similar to part of our manually-written method, too:

public void MoveNext()
{
    int result; // Type varies based on result type
    try
    {
        // Method-specific code
    }
    catch (Exception e)
    {
        state = -1;
        builder.SetException(e);
        return;
    }
    state = -1;
    builder.SetResult(result);
}

Hopefully this doesn't need much explanation: while catching Exception is normally a bad idea, it's appropriate here, so that it can be rethrown in the caller's context (or detected without being rethrown). Obviously the method-specific code in the middle needs to make sure that we only reach the bottom of the method in the case where we're really done - if it ever calls OnCompleted, it needs to return directly rather than falling through to this code.

Actual method-specific code...

Phew, we've finally got to the bit which varies depending on the async method we're dealing with. Fortunately now that we've removed all the extraneous code, it's easier to see what's going on. I've added some comments for clarity:

if (state != 1)
{
      if (state != -1)
      {
          // Code before the first await expression
          task = Task<int>.Factory.StartNew(() => 5);

          // Boiler-plate for the first part of an await
          awaiter = task.GetAwaiter();
          if (awaiter.IsCompleted)
          {
              goto Label_GetResult;
          }
          state = 1;
          doFinallyBodies = false;
          awaiter.OnCompleted(moveNextDelegate);
      }
      return;
}
// Boiler-plate for the second part of an await
state = 0;
Label_GetResult:
int awaitResult = awaiter.GetResult();
awaiter = new TaskAwaiter<int>();

// Code after the await (return, basically)
result = awaitResult;

Now the state machine for our async method can be in any of three states at the start/end of MoveNext():

-1: the method has already completed (possibly with an exception). (It's unclear how we'd end up getting back into the state machine at this point, but the generated code makes sure it just returns immediately.)
0: The initial state - this is also used just before calling GetResult() for situations where we've not needed to use a continuation. I'll go into this slight oddity in another post :)
1: We're waiting to be called back by the awaiter. When the continuation is executed, we want to jump straight to "Label_GetResult" so we can fetch the result and continue with the rest of the method.

I suggest you look at the code path taken by the above code if you start in each of those states - convince yourself that when we exit the method, we'll be in an appropriate state, whether it's normally or via a continuation.

Even within this code, we can see boilerplate material: any time you reach an await expression in an async method, the compiler will generate code like the middle section, to get the awaiter, then either skip to the "get result" part or add a continuation. The fiddly bit is really getting your head round how the code flow works, but hopefully as there are relatively few states in play here, it's not too bad.

Conclusion

This was a very long post for such a short async method. However, now that we've sorted out how it all hangs together, we should be able to look at more complicated async methods without looking at things like the obsolete code and the stub method. Next time we'll see what happens when we introduce loops, catch blocks and finally blocks.

Eduasync part 6: using the infrastructure manually

skeet — Fri, 20 May 2011 13:36:28 GMT

Now that we've got the infrastructure for both returning a task from an async method, and awaiting a Task<T>, we're going to look at what the compiler does for us. I always find that the best way of appreciating a feature like this is to consider what we'd have to do without it. Later on we're going to see what the C# compiler does in terms of decompiling the actual generated code, but first we'll look at what we might do to achieve the same end result, using the same infrastructure.

Of course, if we were doing all of this manually we might not choose to use the same infrastructure to start with, but I'm not sure that it would end up being much cleaner anyway.

A simple async method

First, let's look at the C# 5 async method we want to emulate. It's pretty simple:

private static async Task<int> ReturnValueAsyncWithAssistance()
{
Task<int> task = Task<int>.Factory.StartNew(() => 5);

return await task;
}

Now obviously we could just return the task here to start with - the async method isn't providing any real benefit. But the simplicity is helpful for showing an equivalent. You can always imagine that there's more going on between awaiting the result of the task and returning our actual result.

What do we need to do?

Before I include the actual code, let's consider what we know about the async method execution model:

We'll create an AsyncTaskMethodBuilder to allow us to return a Task<int>.
The method will start synchronously, running all the code before the first (and in this case only) await expression in the normal way.
The awaitable expression will be evaluated, and GetAwaiter() called to get a suitable awaiter value.
We'll ask the awaiter whether the awaitable has already completed, using the IsCompleted property
- If it's already completed, we get the result by calling GetResult(). We can set the result on the builder, and return the builder's task.
- Otherwise, create a continuation delegate representing "the rest of the method". We then return the builder's task.
  - The continuation has to fetch the result by calling GetResult() on the same awaiter, set the result on the builder, and return. (It's not returning a Task<T> - the continuation is just an Action.)
If an exception occurs at any point, whether in the original synchronous call or the continuation, we need to set that exception in the builder, and return (either returning the builder's task or nothing, depending on the context).

Even describing it makes it sound significantly worse than the async method. Let's take a look.

Bring on the code!

Here's the manual method in all its glory:

private static Task<int> ReturnValueAsyncWithoutAssistance()
{
    AsyncTaskMethodBuilder<int> builder = AsyncTaskMethodBuilder<int>.Create();

    try
    {
        Task<int> task = Task<int>.Factory.StartNew(() => 5);

        TaskAwaiter<int> awaiter = task.GetAwaiter();
        if (!awaiter.IsCompleted)
        {
            // Result wasn't available. Add a continuation, and return the builder.
            awaiter.OnCompleted(() =>
            {
                try
                {
                    builder.SetResult(awaiter.GetResult());
                }
                catch (Exception e)
                {
                    builder.SetException(e);
                }
            });
            return builder.Task;
        }

        // Result was already available: proceed synchronously
        builder.SetResult(awaiter.GetResult());
    }
    catch (Exception e)
    {
        builder.SetException(e);
    }
    return builder.Task;
}

Ugly, isn't it? Even so, it's actually somewhere simpler than the real autogenerated code, which we'll see in the next part. Hopefully if you read the code and comments bearing in mind the bullet points above, it should be reasonably easy to see how it works. However:

Currently we're building a continuation delegate specific to this particular await. Imagine if we had several await expressions. Each would need "the rest of the method" specified as its own delegate, and we'd end up building multiple delegate instances over time.
It would be a nightmare to handle loops in this way.
We've got repeated code - not a lot in this case, but anything we had between the "await" and the "return" would need to be in both flows.
You don't really want to be writing this code by hand anyway, if you can help it.

Having said all of that, it works. Just don't expect me to write any more complicated async methods this way :)

Conclusion

The steps given here are still going to be valid with the compiler-generated code, but they'll take a slightly different form. Next time we'll decompile the generated code for the exact same async method, before we move on to look at more complicated cases.

Eduasync part 5: making Task awaitable

skeet — Tue, 17 May 2011 19:47:45 GMT

In part 3 we looked at what the C# 5 compiler required for you to "await" something. The sample used a class which actually had an instance method called GetAwaiter, but I mentioned that it could also be an extension method.

In this post, we'll use that ability to make Task<T> awaitable - at which point we have everything we need to actually see some asynchronous behaviour. Just like the last part, the code here is pretty plain - but by the end of the post we'll have a full demonstration of asynchrony. I should make it clear that this isn't absolutely everything we'll want, but it's a good start.

TaskAwaiter<T>

As it happens, I'm using the same type name as the async CTP does for "something which can await a task" - TaskAwaiter. It's in a different namespace though, and we could rename it with no ill-effects (unlike AsyncTaskMethodBuilder, for example). Indeed, in an old version of similar code I called this type Garçon - so GetAwaiter would return a waiter, so to speak. (Yes, you can use non-ASCII characters in identifiers in C#. I wouldn't really advise it though.)

All the task awaiter needs is a reference to the task that it's awaiting - it doesn't need to know anything about the async method which is waiting for it, for example. Task<T> provides everything we need: a property to find out whether it's completed, another to fetch the result, and the ContinueWith method to specify a continuation. The last part is particularly important - without that, we'd really have a hard time implementing this efficiently.

The extension method on Task<T> is trivial:

public static class TaskExtensions
{
    public static TaskAwaiter<T> GetAwaiter<T>(this Task<T> task)
    {
        return new TaskAwaiter<T>(task);
    }
}

The TaskAwaiter<T> type itself is slightly less so, but only slightly:

public struct TaskAwaiter<T>
{
    private readonly Task<T> task;

    internal TaskAwaiter(Task<T> task)
    {
        this.task = task;
    }

    public bool IsCompleted { get { return task.IsCompleted; } }

    public void OnCompleted(Action action)
    {
        SynchronizationContext context = SynchronizationContext.Current;
        TaskScheduler scheduler = context == null ? TaskScheduler.Current
            : TaskScheduler.FromCurrentSynchronizationContext();
        task.ContinueWith(ignored => action(), scheduler);
    }

    public T GetResult()
    {
        return task.Result;
    }
}

IsCompleted is obviously trivial - Task<T> provides us exactly what we need to know. It's just worth noting that IsCompleted will return true if the task is cancelled, faulted or completed normally - it's not the same as checking for success. However, it represents exactly what we want to know here.

OnCompleted has two very small aspects of interest:

ContinueWith takes an Action<Task<T>> or an Action<Task>, not just an Action. That means we have to create a new delegate to wrap the original continuation. I can't currently think of any way round this with the current specification, but it's slightly annoying. If the compiler could work with an OnCompleted(Action<object>) method then we could pass that into Task<T>.ContinueWith due to contravariance of Action<T>. The compiler could generate an appropriate MoveNext(object) method which just called MoveNext() and stash an Action<object> field instead of an Action field... and do so only if the async method actually required it. I'll email the team with this as a suggestion - they've made other changes with performance in mind, so this is a possibility. Other alternatives:
- In .NET 5, Task<T> could have ContinueWith overloads accepting Action as a continuation. That would be simpler from the language perspective, but the overload list would become pretty huge.
- I would expect Task<T> to have a "real" GetAwaiter method in .NET 5 rather than the extension method; it could quite easily just return "this", possibly with some explicitly implemented IAwaiter<T> interface to avoid polluting the normal API. That could then handle the situation more natively.
We're using the current synchronization context if there is one to schedule the new task. This is the bit that lets continuations keep going on the UI thread for WPF and WinForms apps. If there isn't a synchronization context, we just use the current scheduler. For months this was incorrect in Eduasync; I was using TaskScheduler.Current in all cases. It's a subtle difference which has a huge effect on correctness; apologies for the previous inaccuracy. Even the current code is a lot cruder than it could be, but it should be better than it was...

GetResult looks and is utterly trivial - it works fine for success cases, but it doesn't do what we really want if the task has been faulted or cancelled. We'll improve it in a later part.

Let's see it in action!

Between this and the AsyncTaskMethodBuilder we wrote last time, we're ready to see an end-to-end asynchronous method demo. Here's the full code - it's not as trivial as it might be, as I've included some diagnostics so we can see what's going on:

internal class Program
{
    private static readonly DateTimeOffset StartTime = DateTimeOffset.UtcNow;

    private static void Main(string[] args)
    {
        Log("In Main, before SumAsync call");
        Task<int> task = SumAsync();
        Log("In Main, after SumAsync returned");

        int result = task.Result;
        Log("Final result: " + result);
    }

    private static async Task<int> SumAsync()
    {
        Task<int> task1 = Task.Factory.StartNew(() => { Thread.Sleep(500); return 10; });
        Task<int> task2 = Task.Factory.StartNew(() => { Thread.Sleep(750); return 5; });

        Log("In SumAsync, before awaits");

        int value1 = await task1;
        int value2 = await task2;

        Log("In SumAsync, after awaits");

        return value1 + value2;
    }

    private static void Log(string text)
    {
        Console.WriteLine("Thread={0}. Time={1}ms. Message={2}",
                          Thread.CurrentThread.ManagedThreadId,
                          (long)(DateTimeOffset.UtcNow - StartTime).TotalMilliseconds,
                          text);
    }
}

And here's the result of one run:

Thread=1. Time=12ms. Message=In Main, before SumAsync call
Thread=1. Time=51ms. Message=In SumAsync, before awaits
Thread=1. Time=55ms. Message=In Main, after SumAsync returned
Thread=4. Time=802ms. Message=In SumAsync, after awaits
Thread=1. Time=802ms. Message=Final result: 15

So what's going on?

We initially log before we even start the async method. We can see that the thread running Main has ID 1.
Within SumAsync, we start two tasks using Task.Factory.StartNew. Each task just has to sleep for a bit, then return a value. Everything's hard-coded.
We log before we await anything: this occurs still on thread 1, because async methods run synchronously at least as far as the first await.
We hit the first await, and because the first task hasn't completed yet, we register a continuation on it, and immediately return to Main.
We log that we're in Main, still in thread 1.
When the first await completes, a thread from the thread pool will execute the continuation. (This may well be the thread which executed the first task; I don't know the behaviour of the task scheduler used in console apps off the top of my head.) This will then hit the second await, which also won't have finished - so the first continuation completes, having registered a second continuation, this time on the second task. If we changed the Sleep calls within the tasks, we could observe this second await actually not needing to wait for anything.
When the second continuation fires, we log that fact. Two things to notice:
- It's almost exactly 750ms after the earlier log messages. That proves that the two tasks has genuinely been executing in parallel.
- It's on thread 4.
The final log statement occurs immediately after we return from the async method - thread 1 has been blocked on the task.Result property fetch, but when the async method completes, it unblocks and shows the result.

I think you'll agree that for the very small amount of code we've had to write, this is pretty nifty.

Conclusion

We've now implemented enough of the functionality which is usually in AsyncCtpLibrary.dll to investigate what the compiler's really doing for us. Next time I'll include a program showing one option for using the same types within hand-written code... and point out how nasty it is. Then for the next few parts, we'll look at what the C# 5 compiler does when we let it loose on code like the above... and show why I didn't just have "int value = await task1 + await task2;" in the sample program.

If you've skimmed through this post reasonably quickly, now would be a good time to go back and make sure you're really comfortable with where in this sample our AsyncTaskMethodBuilder is being used, and where TaskAwaiter is being used. We've got Task<T> as the core type at both boundaries, but that's slightly coincidental - the boundaries are still very different, and it's worth making sure you understand them before you try to wrap your head round the compiler-generated code.

Eduasync part 4: Filling in AsyncTaskMethodBuilder

skeet — Sun, 15 May 2011 19:21:00 GMT

In part 2, I introduced AsyncVoidMethodBuilder, AsyncTaskMethodBuilder and AsyncTaskMethodBuilder<T>. I showed all the signatures, but the implementations were basically trivial and non-functional. Today, I'm going to change all of that... at least a bit.

In particular, by the end of this post we'll be able to make the following code actually print out 10 as we'd expect:

private static void Main(string[] args)
{
    Task<int> task = Return10Async();
    Console.WriteLine(task.Result);
}

private static async Task<int> Return10Async()
{
    return 10;
}

Note that there are no await statements - remember that AsyncTaskMethodBuilder<T> is about the boundary between the caller and the async method. We won't be actually doing anything genuinely asynchronous today - just filling in a piece of the infrastructure.

I'm not going to implement AsyncVoidMethodBuilder or AsyncTaskMethodBuilder, because once you understand how AsyncTaskMethodBuilder<T> works, the others are basically trivial to implement. (Certainly the non-generic AsyncTaskMethodBuilder is too similar to warrant going into; I may potentially revisit AsyncVoidMethodBuilder to investigate its behaviour on exceptions.)

Just as a reminder, this is what we need our AsyncTaskMethodBuilder<T> struct to look like in terms of its interface:

public struct AsyncTaskMethodBuilder<T>
{
    public static AsyncTaskMethodBuilder<T> Create();
    public void SetException(Exception e);
    public void SetResult(T result);
    public Task<T> Task { get; }
}

Frankly, this would be a bit of a pain to implement ourselves... especially if we wanted to do it in a neat way which didn't introduce a new thread unnecessarily. Fortunately, Parallel Extensions makes it pretty easy.

TaskCompletionSource to the rescue!

The TaskCompletionSource<TResult> type in the framework makes it almost trivial to implement AsyncTaskMethodBuilder<T>. It provides us everything we need - a task, and the ability to specify how it completes, either with an exception or a value. Our initial implementation is really just going to wrap TaskCompletionSource. I suspect the real CTP does slightly more work, but you can get a remarkably long way with simple wrapping:

public struct AsyncTaskMethodBuilder<T>
{
    private readonly TaskCompletionSource<T> source;

    private AsyncTaskMethodBuilder(TaskCompletionSource<T> source)
    {
        this.source = source;
    }

    public static AsyncTaskMethodBuilder<T> Create()
    {
        return new AsyncTaskMethodBuilder<T>(new TaskCompletionSource<T>());
    }

    public void SetException(Exception e)
    {
        source.SetException(e);
    }

    public void SetResult(T result)
    {
        source.SetResult(result);
    }

    public Task<T> Task { get { return source.Task; } }
}

We'll see how far that takes us - we may need to tweak it a bit, probably around the exception handling. For the moment though, it certainly does everything we need it to. The program runs, and prints out 10 as we'd expect.

This part of the code is now ready for asynchrony - even though nothing we've done so far actually creates any different threads. If we call SetResult from one thread while another thread is waiting on the task's result, the waiting thread will be unblocked as we'd expect.

Conclusion

In some ways I hope you're disappointed by this, in the same way that looking at the LINQ to Objects implementations can be slightly disappointing. It feels like so much is being given to us for free - the framework already had all this power, so what's C# 5 really adding to the mix. Well, we're nearly at the point where I can show you the details of what the compiler's doing under the hood. That's where the real power lies, and is why I'm so excited by the feature.

First though, we need to implement the awaiter pattern for Task<T>. That's the task of our next post, and then we can complete the picture by decompiling the generated code for some programs using genuine asynchrony.

Eduasync part 3: the shape of the async method / awaitable boundary

skeet — Fri, 13 May 2011 21:56:14 GMT

Last time we looked into the boundary between the caller of an async method and the method itself.

This time I'm going to show the same sort of "skeleton API" but for awaitable types. This is at the heart of C# 5's async feature: within an async method, you can include "await" expressions. You have to await a value - which could be the return value from a method call, or a property, or the value of a variable. The important thing is that the compile-time type of that expression has the right shape.

Definition time

Consider this code snippet from an async method (separated into two statements for clarity):

Task<string> t = new WebClient().DownloadStringTaskAsync(url);
string text = await t;

I wish to define the following terms:

The task of the await expression is the expression which comes after "await"; in this case it's just "t".
The awaitable type is the compile-time type of the task. In this case it's Task<string>.
The awaiter type is the compile-time type of the result of calling GetAwaiter() on the task.
The result type is the compile-time return type of the awaiter type's GetResult method. Here it would be string, when using the CTP or any sensible implementation.

(Of these, "task" and "awaiter type" are part of the draft specification. "Awaitable type" and "result type" aren't.)

Now, the first two are fairly clear from the code above, but the awaiter and result types aren't. The compiler validates that calling (foo).GetAwaiter() for a task expression "foo" is valid. It doesn't have to be an instance method - it can be an extension method, and indeed that's how it's implemented in the CTP. Unlike the weird things you can do with LINQ query expressions, GetAwaiter() really does have to be a method call, and the expression can't just be a type name. Unless you're trying to do weird stuff, this is very unlikely to be an issue for you. It's only oddballs like me who try to push the envelope.

With the current CTP the awaitable type can't be "dynamic", but the aim is for C# 5 to support awaiting dynamic values. I imagine this won't affect the execution flow significantly though.

The awaitable type only has to support the GetAwaiter() method, as we've said. The awaiter type has to have the following accessible members (which can't be extension methods):

bool IsCompleted { get; }
void OnCompleted(Action continuation)
X GetResult() where X defines the result type for the awaiter, and may be void

The overall type of the await expression is the result type - and it has to be appropriate for any surrounding context. For example, the assignment to the "text" variable will only work in our sample if the result type is "string" or a type with an implicit conversion to string.

A simple (non-functional) awaiter

The sample code for this post comes from project 5 in the source solution (SimpleInt32Awaitable); project 4 is similar but with a void result type. Neither project makes any attempt to do any real awaiting:

// Within some other class
static async void SimpleWaitAsync()
{
    SimpleInt32Awaitable awaitable = new SimpleInt32Awaitable();
    int result = await awaitable;
}

...

public struct SimpleInt32Awaitable
{
    public SimpleInt32Awaiter GetAwaiter()
    {
        return new SimpleInt32Awaiter();
    }
}

public struct SimpleInt32Awaiter
{
    public bool IsCompleted { get { return true; } }

    public void OnCompleted(Action continuation)
    {
    }

    public int GetResult()
    {
        return 5;
    }
}

Note that here both the awaitable type and the awaiter type are structs rather than classes; this is not a requirement by any means.

What does it all mean?

I don't want to leave this post without any clue of what all of this is used for.

Fairly obviously, GetAwaiter() is called to get an awaiter (clever name, huh?) on which some members are called:

IsCompleted is called to determine whether the task has already completed. If it has, we can skip over the next step.
If the task hasn't already completed, we want to return very soon rather than waiting - so OnCompleted is calling with a delegate representing the continuation of the method. The awaiter promises to ensure that the continuation is called when (and only when) the task has completed, possibly with an exception.
When we know the task has completed (either synchronously or via the continuation), GetResult is called to retrieve the result. Note that even if the method has a void return type, it still has to be called, as the "result" may be that an exception is thrown.

Conclusion

Awaiters are pretty flexible - which means they can be abused in evil, horrible ways, as we'll see later on.

So far we've seen the basic shape of both boundaries, and now we can start actually implementing them with real, useful code.

Eduasync part 2: the shape of the caller / async method boundary

skeet — Fri, 13 May 2011 19:35:37 GMT

For the first few parts of this blog series, we’re not going to write code which actually does anything useful. We’re looking at the API more than the implementation - the bare necessities to get the code to compile. Remember that we’re working without AsyncCtpLibrary.dll - which of course supplies all the necessary types normally.

In this part, we’ll look at the boundary between the caller and the async method itself. Our test code will have a method which does nothing but return - but which is marked with the “async” modifier. The fact that we don’t have an “await” expression in the method causes a compiler warning - which is entirely reasonable, but irrelevant to our particular situation.

Three return types, one basic idea

Async methods are limited to three different return types:

void
Task (non-generic)
Task<T> (generic for any type T)

Each of these return types needs a very slightly different form of compiler support... but the general principle is the same. I’ll show each of the three different types, but the test code itself remains exactly the same except for the return type of the method and the return statement. (If you’re going to return Task<int>, you need to return an int, etc.)

namespace Eduasync
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            DoNothingAsync();
        }

// Warning CS1998 is about a method with no awaits in... exactly what we're trying to
// achieve!
#pragma warning disable 1998
        // Return type of void, Task or Task<int>
        private static async void DoNothingAsync()
        {
            // For Task<int> insert return 0; here
        }
#pragma warning restore 1998
    }
}

If you try to compile this code without anything else, the compiler gives an error like this:

Test.cs(14,30): error CS0656: Missing compiler required member 'System.Runtime.CompilerServices.AsyncVoidMethodBuilder.Create'
Test.cs(14,30): error CS1993: Cannot find Task-related types. Are you missing a reference to 'AsyncCtpLibrary.dll' ?

While you’d get a similar error for the async method / awaitable boundary, this one is more demanding: you have to have exactly the right types available. It isn’t just a matter of any library providing support for your current situation (in the same way that Edulinq can live in its own namespace and be an alternative for LINQ to Objects without the compiler caring). These are exact types that the compiler relies on. They're an implementation detail: they can vary between compilers (so Mono could have a different set of types for example, although I doubt that it will, for the sake of binary compatibility) and they're not part of the language specification... at least at the moment.

There are three types required: AsyncVoidMethodBuilder, AsyncTaskMethodBuilder and AsyncTaskMethodBuilder<T>. They correspond (fairly obviously) to the return types of void, Task, and Task<T> respectively.

They all look largely the same, but here are my initial "non-implementations":

using System.Threading.Tasks;

namespace System.Runtime.CompilerServices
{
    public struct AsyncVoidMethodBuilder
    {
        public static AsyncVoidMethodBuilder Create()
        {
            return new AsyncVoidMethodBuilder();
        }

        public void SetException(Exception e) {}
        public void SetResult() {}
    }

    public struct AsyncTaskMethodBuilder
    {
        public static AsyncTaskMethodBuilder Create()
        {
            return new AsyncTaskMethodBuilder();
        }

        public void SetException(Exception e) {}
        public void SetResult() {}
        public Task Task { get { return null; } }
    }

    public struct AsyncTaskMethodBuilder<T>
    {
        public static AsyncTaskMethodBuilder<T> Create()
        {
            return new AsyncTaskMethodBuilder<T>();
        }

        public void SetException(Exception e) {}
        public void SetResult(T result) {}
        public Task<T> Task { get { return null; } }
    }
}

(In the Eduasync project these are in separate files and have diagnostic statements in the SetException/SetResult methods. I didn't want to take up too much space here.)

Note that these must be structs. If they're not, the compiler will complain. All the methods have to have the exact signatures specified, as far as I can tell - except that everything can be internal if you want it to be (so long as your async methods are in the same assembly of course). In practice these are going to be public, and they are public everywhere in Eduasync.

With these implementations, you can compile and even run async methods - but you will, of course, end up with a null reference returned from any async method with a return type of Task or Task<T>.

In case you're wondering, the code below shows a little taste of how these types are used. It's from the version of DoNothingAsync which returns a non-generic Task. This is what the compiler replaces our code with:

private static Task DoNothingAsync()
{
    <DoNothingAsync>d__0 d__ = new <DoNothingAsync>d__0(0);
    d__.<>t__MoveNextDelegate = new Action(d__.MoveNext);
    d__.$builder = AsyncTaskMethodBuilder.Create();
    d__.MoveNext();
    return d__.$builder.Task;
}

Obviously <DoNothingAsync>d__0 is a compiler-generated type (hence the unspeakable name). I'm not going into the details just yet - that will the topic of several posts in a little while. The main point of this post was just to show you the shape of what the compiler requires. The Create() method and the Task property are used within the rewritten method; the SetResult() and SetException() methods are used within the generated type (the state machine) to indicate the async method completing.

Before too long we'll implement these types properly (which is reasonably straightforward, given help from the BCL).

Conclusion

The caller / async method boundary is relatively inflexible. The compiler relies on particular types, and you simply can't make an async method return a different kind of value, such as your own Future<T> type. The guts of how it works are all hidden from you, unless you try to compile without the right libraries around: while the valid return types are part of the specification, the types used by the compiler aren't. While this is perhaps a little unfortunate in a purist sense, it's not really a big deal. The "consuming" part of the async method (the boundary between the async method and whatever it's awaiting) is much more flexible, and more interesting. That's what we're going to look at next.

Eduasync part 1: introduction

skeet — Sun, 08 May 2011 20:05:32 GMT

I’ve been waiting to start this blog series for a couple of months. It’s nice to finally get cracking.

Hopefully some of you have already read some of my thoughts around C# 5’s async feature, mostly written last year. Since that initial flurry of posts, I’ve been pretty quiet, but I’m still really excited about it. Really, really excited. I’ve given a few talks on it, and I have a few more still to give - and this blog series will partly be complementary to those talks. In particular, there’s a DevExpress webcast which covers most of the same ground, with similar code. (It was before the CTP refresh, and also before my laptop was stolen in a burglary, so the code here is a rewrite.)

Async from a compiler’s point of view

Most of this blog series (at least the bits I anticipate at the moment) will deal with what the compiler does with async methods. (I haven’t used async delegates much at all, but I can’t imagine that the machinery is particularly different.)

As far as I’ve seen, most of the coverage on the web so far has dealt with using async. That’s natural, logical and entirely proper. Oh, and a bit boring after a while. I like knowing how a feature works before I go too far using it. This is a personal idiosyncrasy, and if you’re happy just using async with no “under the hood” details, that’s absolutely fine. It’s probably worth unsubscribing from my blog for a little while, that’s all.

This can all be seen as pretty similar to my Edulinq series of posts, which is why I’ve called it Eduasync this time.

My plan is to walk you through what the C# compiler relies on - the types which are currently part of AsyncCtpLibrary.dll, and how it interacts with Task / Task<T> from .NET 4. We’ll then look at the code generated by the compiler - essentially a state machine - and some of the less obvious aspects of it. I’ll give examples of any bugs I’ve found in the CTP, just for the heck of it - and as a way of checking whether they’re fixed in later versions. (Obviously I’ve let the C#/VB team know about these as I’ve come across them.)

I’ll assume that you know the basics of using async - so if you don’t, now would be a good time to look at the numerous resources on the Visual Studio Async home page. There are loads of videos, specs (including the C# spec changes, most importantly from my point of view)

Get the source now

There’s already quite a bit of source code (everything I’m currently planning on writing about, which is almost inevitably less than I’ll actually end up writing about) on the Google Code Eduasync project. This takes a different approach from Edulinq - instead of just a couple of projects (production and tests, basically) I’ve got a separate project for each topic I want to talk about, with pretty minimal code for that topic. The reason for this is to show the evolution of the code - starting off with almost nothing, and progressing until we’ve got an implementation which achieves at least the bare bones important bits of an async system.

I’ve numbered the projects within the solution, although the assemblies themselves don’t have the same numbers. They all use a default namespace of just Eduasync, and they don’t refer to each other. Each is meant to be self-contained - oh, and there are no references to AsyncCtpLibrary.dll. The whole point is to reimplement that library :) Of course, you'll still need the CTP installed to get the compiler changes.

The Google Code repository will also contain the blog posts eventually, including any diagrams I need to create (such as the one in a minute).

The three blocks and two boundaries

One of the things I've found important to think about in async is the various parts involved. I've ended up with a mental model like this:

The bits in blue and red are the ones we're focusing on here: the contents of the async method, and the boundaries between that and the code that calls it, and the tasks (or other awaitable types) that it awaits.

For most of this series we're not really going to care much about what the caller does with the result, or how the awaitable object behaves other than in terms of the methods and properties used by the C# 5 compiler. I'll discuss the flexibility afforded though - and how it doesn't extend to the "caller/async" boundary, only the "async/awaitable" boundary.

Just to give an explicit example of all of this, here's a simple little program to asynchronously determine the size of the Stack Overflow home page:

using System;
using System.Net;
using System.Threading.Tasks;

class Program
{
    // Caller (block 1)
    static void Main()
    {
        Task<int> sizeTask = DownloadSizeAsync("http://stackoverflow.com");
        Console.WriteLine("In Main, after async method call...");
        Console.WriteLine("Size: {0}", sizeTask.Result);
    }

    // Async method (block 2)
    static async Task<int> DownloadSizeAsync(string url)
    {
        var client = new WebClient();
        // Awaitable (block 3)
        var awaitable = client.DownloadDataTaskAsync(url);

        Console.WriteLine("Starting await...");
        byte[] data = await awaitable;
        Console.WriteLine("Finished awaiting...");

        return data.Length;
    }
}

The comments should make it reasonably clear what the blocks in the diagram mean. It's not ideal in that the first two blocks are basically methods, whereas the third block is an object - but I've found that it still makes sense when we're thinking about the interactions involved at the boundaries. Notably:

How does the async method create an appropriate value to return to the caller?
How does the async method interact with the awaitable when it hits an "await" expression?

We can (and we're going to) look at these boundaries very separately. We'll start off with the first bullet, in part two, which will hopefully follow in the next few days.