Batch asynchronous tasks with coroutines

This topic describes how to batch multiple asynchronous tasks with coroutines. "Batching" means executing asynchronous operations as a group and waiting for the group to finish.

You can batch asynchronous tasks in parallel or serially.

In a parallel batch, all tasks are launched at once; the caller is notified when they all finish or one of them fails. Use the DevForce Parallel Coroutine to create and manage a parallel batch.

In a serial batch, the asynchronous operations are executed in order. The first operation must complete before the second operation can start; the third operation has to wait for the second operation to complete; so it goes until the last of the operations delivers the final result. Use the DevForce Serial Coroutine to manage a serial batch.

Serial asynchronous batching background

In the absence of coroutines, the obvious approach to sequencing a batch of asynchronous tasks is to build a chain of callbacks. You arrange for the first operation's callback to start the second operation ... whose callback starts the third operation ... until your final operation delivers the result in its callback.

It works but the code is difficult to read and maintain. There are callbacks inside of callbacks inside of callbacks. They look like deeply nested "if ... else" statements. It's usually worse when you insert logic at every step to account for errors that might be thrown if one of the operations fails.

Coroutines are a cleaner way to approach this problem. With Coroutines, we can write down a sequence of asynchronous tasks as if they were going to execute synchronously. They will actually run asynchronously but they will appear to be a series of synchronous calls, each of them blocking until it is finished ... as we'll see.

Jeremy Likness wrote a nice blog post on the problem which he describes as follows:

We have asynchronous calls that we want to process sequentially without having to chain dozens of methods or resort to complex lambda expressions. How do we do it?

He goes on to explain Coroutines and how they address this problem. Finally, he points to several available Coroutine implementations.

We looked at those implementations. We didn't love with any of them. Either they were too complicated or too closely coupled with particular UI technologies. Caliburn's "IResult", for example, depends upon an ActionExecutionContext with eight members, two of which are WPF/Silverlight specific (FrameworkElement, DependencyObject). That makes sense if you confine your use of Coroutines to the XAML UI client technologies. We wanted to support asynchronous programming in non-UI contexts and we wanted to rely on interfaces that were as small as possible.

The DevForce Coroutine Class

DevForce version 6.0.6 introduces the Coroutine  class. We think you'll prefer our implementation for most - if not all - of your asynchronous programming needs.

The Coroutine class resides in EntityModel for this release. It actually has no dependency on Entities; it lives here temporarily and will relocate to IdeaBlade.Core in a future release. Don't worry ... your code won't break. We'll use "type forwarding" so your original references will continue to work.

Let's start with a simple example.

Imagine that user must make calls to some "top customers". The sales manager prepared a list of "top customers" for each sales rep to call first thing in the morning. When the sales rep launches the application, the list of top customers should be on screen.

Due to an unfortunate design decision, each "top customers" list has CustomerIDs but no relationship to Customers. You won't be able to get the customer information in a single query. You'll have to get the sales rep's "top customers" first, fish out the CustomerIDs, and use them in a second query to get the full Customer entities and the information to present on screen. That's two queries and the second one can't start until the first one finishes.

You'll want to encapsulate the sequence in a single asynchronous method. Here is that method:

Clearly LoadTopCustomers involves an asynchronous operation. You cannot completely escape the asynchronous nature of the process.

On the bright side, there is exactly one asynchronous operation, the result of the static Coroutine.Start method. That method takes a companion coroutine that we wrote to handle the sequence of queries necessary to deliver the top customers. We'll get to that coroutine in a moment.

Before we do, we note that when the coroutine completes ...

• if it completed successfully, we take its Result, cast it as a list of Customers, and give that to the ShowTopCustomers method which knows how to put those customers on screen.
• if the coroutine failed, we let the host's HandleError method figure out how to present the failure to the user.

Move on to the coroutine, LoadTopCustomerCoroutine. This is the heart of the Coroutine business.

Concentrate on the body of the method. There are only seven statements:

1. Get the sales rep's User ID from a static so we can select the top customers for this particular rep
2. Issue an async projection query for this sales rep's top customers. The "Select" clause tells you its a projection that will return a list of Customer IDs
3. Yield the query operation object from the ExecuteAsync() call.
4. Pour the query results (the Customer IDs) into a list variable, salesRepsTopCustIDs
5. Issue a second async query, one that returns every customer whose ID is in the salesRepsTopCustIDs list.
6. Yield the query operation object from the ExecuteAsync() call.
7. Yield (for the last time) the results of the sales rep's customer query.

Reads like a normal procedure, don't you agree? Except for the yield return keywords.

The linear flow, from top to bottom, is what we're after. If you had to add another asynchronous operation somewhere in the mix ... you'd have no trouble doing so. You'd simply follow the pattern you see here:

• write synchronously until you come to an asynchronous method.
• yield return the operation object result of each asynchronous DevForce method.
• at the very end, write: "yield return Coroutine.Return(the_final_result);"

Passing Context into the Coroutine

The co-routine iterator seems fully self-sufficient in our example. Look closer. Notice that it requires a userId to filter "top customers" and it needs an EntityManager. The userId comes from the static "CurrentUser" class and it gets an EntityManager from a convenient, ambient "Manager" property.

What if you are not so fortunate? What if you think passing parameters via statics is (ahem) not the right architecture for you? You want the caller to provide "context" to the iterator from the outside, freeing the iterator from unnecessary dependencies.

Let's rewrite the skeleton of our example to show how we might do that:

The key is to (A) re-define the coroutine method to accept the context and (B) wrap the coroutine call in an "Action Lambda" before passing that into the Coroutine.Start(). Here are the two lines of interest.

A) IEnumerable<INotifyCompleted> LoadTopCustomersCoroutine(ModelEntityManager manager, int userId)
B) Coroutine.Start( () => LoadTopCustomersCoroutine(manager, userId) );

Do not go wild with the parameters. If your iterator seems to need too many context parameters, the code is telling you to bundle them into a single construct - and a single concept. Figure out what it is. Make a type to carry that context and build an instance of it before you even get to the LoadTopCustomers method. Then pass this context object along the call chain.

How does the DevForce Coroutine work?

Many of you will recognize that this coroutine is a .NET Iterator. Iterators return IEnumerable<T> and consist of a sequence of statements punctuated by "yield return" statements ... just as we see here.

For an introduction to the serial async task problem, iterators, and how DevForce Coroutines work, see The Serial Async Challenge topic. The Serial Coroutine Flow Diagram topic offers a visual rendition of the DevForce Coroutine behavior.

Good-bye, AsyncSerialTask and AsyncParallelTask

The DevForce AsyncSerialTask class was an earlier attempt to address these asynchronous batching problems. It has been deprecated in favor of the much easier Coroutine class.

The same goes for the AsyncParallelTask which handled a related scenario. With AsyncParallelTask you could run several asynchronous tasks simultaneously and wait for a signal indicating that they had all finished.  This capability is now subsumed by the Coroutine class; see its StartParallel method.

We intend to remove both the AsyncSerialTask and the AsyncParallelTask classes from DevForce in the first half of 2011; both classes will be available in source code should you wish to continue using them ... and maintain them yourself.