Peter Ritchie's MVP Blog : DevCenterPost

IDisposable and Class Hierarchies

PeterRitchie — Mon, 21 Jan 2013 17:11:00 GMT

In my previous post, I showed how the Dispose Pattern is effectively obsolete. But, there’s one area that I didn’t really cover. What do you do when you want to create a class that implements IDisposable, doesn’t implement the Dispose Pattern, and will be derived from classes that will also implement disposal?

The Dispose Pattern covered this by coincidence. Since something that derives from a class that implements the Dispose Pattern simply overrides the Dispose(bool) method, you effectively have a way to chain disposal from the sub to the base. There’s a lot of unrelated chaff that comes along with Dispose Pattern if that’s all you need. What if you want to design a base class that implements IDisposable and support sub classes that might want to dispose of managed resources? Well, you’re not screwed.

You can simply make your IDisposable.Dispose method virtual and a sub can override it before calling the base. For example:

	public class Base : IDisposable
	{
		private IDisposable managedResource;
		//...
		virtual public void Dispose()
		{
			if(managedResource != null) managedResource.Dispose();
		}
	}
 
	public class Sub : Base
	{
		private IDisposable managedResource;
		public override void Dispose()
		{
			if (managedResource != null) managedResource.Dispose();
			base.Dispose();
		}
	}

If you don’t implement a virtual Dispose and you don’t implement the Dispose Pattern, you should use the sealed modifier on your class because you’ve effectively made it impossible for base class to dispose of both their resources and the base’s resources in all circumstances. In the case of a variable declared as the base class type that holds an instance of a subclassed type (e.g. Base base = new Sub()) only the base Dispose will get invoked (all other cases, the sub Dispose will get called).

Caveat

If you do have a base class that implements IDisposable and doesn’t implement a virtual Dispose or implement the Dispose Pattern (e.g. outside of your control) then you’re basically screwed in terms of inheritance. In this case, I would prefer composition over inheritance. The type that would have been the base simply becomes a member of the new class and is treated just like any other disposable member (dealt with in the IDisposable.Dispose implementation). For example:

	public class Base : IDisposable
	{
		//...
		public void Dispose()
		{
			//...
		}
	}
 
	public class Sub : IDisposable
	{
		private Base theBase;
		//...
 
		public void Dispose()
		{
			theBase.Dispose();
		}
	}

This, of course, means you need to either mirror the interface that the previously-base-class provides, or provide a sub-set of wrapped functionality so the composed object can be used in the same ways it could have been had it been a base class.

This is why it’s important to design consciously—you need to understand the ramifications and side-effects of certain design choices.

The Dispose Pattern as an anti-pattern

PeterRitchie — Sun, 20 Jan 2013 22:16:00 GMT

When .NET first came out, the framework only had abstractions for what seemed like a handful of Windows features. Developers were required to write their own abstractions around the Windows features that did not have abstractions. Working with these features required you to work with unmanaged resources in many instances. Unmanaged resources, as the name suggests, are not managed in any way by the .NET Framework. If you don’t free those unmanaged resources when you’re done with them, they’ll leak. Unmanaged resources need attention and they need it differently from managed resources. Managed resources, by definition, are managed by the .NET Framework and their resources will be freed automatically a great proportion of the time when they’re no longer in use. The Garbage Collector (GC) knows (or is “told”) what objects are in use and what objects are not in use.

The GC frees managed resources when it gets its timeslice(s) to tidy up memory—which will be some time *after* the resource stop being used. The IDisposable interface was created so that managed resources can be deterministically freed. I say “managed resources” because interfaces can do nothing with destructors and thus the interface inherently can’t do anything specifically to help with unmanaged resources.

“Unmanaged resources” generally means dealing with a handle and freeing that handle when no longer in use. “Support” for Windows features in .NET abstractions generally involved freeing those handles when not in use. Much like managed resources, to deterministically free them you had to implement IDisposable and free them in the call to Dispose. The problem with this was if you forgot to wrap the object in a using block or otherwise didn’t call Dispose. The managed resources would be detected as being unused (unreferenced) and be freed automatically at the next collection, unmanaged resources would not. Unmanaged resources would leak and could cause potential issues with Windows in various ways (handles are a finite resource, for one, so an application could “run out”). So, those unmanaged resources must be freed during finalization of the object (the automatic cleanup of the object during collection by the GC) had they not already been freed during dispose. Since finalization and Dispose are intrinsically linked, the Dispose Pattern was created to make this process easier and consistent.

I won’t get into much detail about the Dispose Pattern, but what this means is that to implement the Dispose Pattern, you must implement a destructor that calls Dispose(bool) with a false argument. Destructors that do no work force an entry to be made in the finalize queue for each instance of that type. This forces the type to use its memory until the GC has a chance to collect and run finalizers. This impacts performance (needless finalization) as well as adds stress to the garbage collector (extra work, more things to keep track of, extra resources, etc.). [1] If you have no unmanaged resources to free, you have no reason to have a destructor and thus have no reason to implement the Dispose Pattern. Some might say it’s handy “just in case”; but those cases are really rare.

.NET has evolved quite a bit from version 1.x, it how has rich support for many of the Windows features that people need to be able to use. Most of the type handles are hidden in these feature abstractions and the developer doesn’t need to do anything special other than recognize a type implements IDisposable and deterministically call Dispose in some way. Of the features that didn’t have abstractions, lower-level abstractions like SafeHandle (which SafeHandleZeroOrMinusOneIsInvalid and SafeHandleMinuesOneIsInvalid etc. derive from)—which implement IDisposable and makes every native handle a “managed resource”—means there is very little reason to write a destructor.

The most recent perpetuation of the anti-pattern is in a Resharper extension called R2P (refactoring to patterns). Let’s analyze the example R2P IDisposable code:

As we can see from this code, the Dispose pattern has been implemented and a destructor with a Dispose(false). If we look at Dispose(bool), Dispose(bool) does nothing if a false argument is passed to it. So, effectively we could simply remove Dispose(false) and get the same result. This also means we could completely remove the destructor. Now we’re left with Dispose(true) in Dispose() and Dispose(bool). Since Dispose(bool) is now only ever called with a true argument, there’s no reason to have this method. We can take the contents of the if(disposing) block, move it to Dispose (replacing the Dispose(true)) and have exactly the same result as before without the Dispose Pattern. Except now, we’re reduced the stress on the GC *and* we’ve made our code much less complex. Also, since we no longer have a destructor there will be no finalizer, so there’s no need to call SuppressFinalize. Not implementing the Dispose Pattern would result in better code in this case:

	public class Person : IDisposable
	{
		public void Dispose()
		{
			Photo.Dispose();
		}
 
		public Bitmap Photo { get; set; }
	}

Of course, when you’re deriving from a class that implements the Dispose Pattern and your class needs to dispose of managed resources, then you need to make use of Dispose(bool). For example:

	public class FantasicalControl : System.Windows.Forms.Control
	{
		protected override void Dispose(bool disposing)
		{
			if(disposing)
			{
				Photo.Dispose();
			}
			base.Dispose(disposing);
		}
		public Bitmap Photo { get; set; }
	}

Patterns are great, they help document code by providing consistent terminology and recognizable implementation (code). But, when they’re not used in the right place at the right time, they make code confusing and harder to understand and become Anti-Patterns.

[1] http://bit.ly/YbBDAR

Thread synchronization of non-atomic invariants in .NET 4.5

PeterRitchie — Tue, 11 Sep 2012 19:04:02 GMT

Now that we’ve seen how a singular x86-x64 focus might affect how we can synchronize atomic invariants, let’s look at non-atomic invariants.

While an atomic invariant really doesn’t need much in the way of guarding, non-atomic invariants often do. The rules by which the invariant is correct are often much more complex. Ensuring an atomic invariant like int, for example is pretty easy: you can’t set it to an invalid value, you just need to make sure the value is visible. Non-atomic invariants involve data that can’t natively be modified atomically. The typical case is more than one variable, but can include intrinsic types that are not guaranteed to be modified atomically (like long and decimal). There is also the fact that not all operations on an atomic type are performed atomically.

For example, let’s say I want to deal with a latitude longitude pair. That pair of floating-point values is an invariant, I need to model accesses to that pair as an atomic operation. If I write to latitude, that value shouldn’t be “seen” until I also write to longitude. The following code does not guard that invariant in a concurrent context:

latitude = 39.73;

longitude = -86.27;

If somewhere else I changed these values, for example I wanted to change from the location of Indianapolis, IN to Ottawa, ON:

   1: latitude = 45.4112;

   2: longitude = -75.6981;

Another thread reading latitude/longitude if the thread was executing the above code was between line 1 and 2, would read a lat/long for some place near Newark instead of Ottawa or Indianapolis (the two lat/longs being written). Making these write operations volatile does nothing to help make the operation atomic and thread-safe. For example, the following is still not thread-safe:

   1: Thread.VolatileWrite(ref latitude, 45.4112);

   2: Thread.VolatileWrite(ref longitude, -75.6981);

A thread can still read latitude or longitude after line 1 executes on another thread and before line 2. Given two variables that are publicly visible, the only way to make an operation on both “atomic” is to use lock or use a synchronization class like Monitor, Semaphore, Mutex, etc. For example:

lock(latLongLock)

    latitude = 45.4112;

    longitude = -75.6981;

Considering latitude and longitude “volatile”, doesn’t help us at all in this situation—we have to use lock. And once we use lock, there’s no need to consider the variables volatile, no two threads can be in the same critical region at the same time, and any side-effect resulting from executing that critical region are guaranteed to be visible as soon as the lock is released. (as well any potentially visible side-effects from other threads are guaranteed to be visible as soon as the lock is acquired).

There are circumstances where you can have loads/stores to different addresses that get reordered in relation to each other (a load can be reordered with older stores to a different memory address). So, conceptually given two threads executing on different cores/CPUS executing the following code at the same time:

x = 1;    |    y = 1;

r1 = y;   |    r2 = x;

This could result in r1 == 0 and r2 == 0 (as described in section 8.2.3.2 of Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 3A) assuming r1 and r2 access was optimized by the compiler to be an register access. The only way to avoid this would be to force a memory barrier. The use of volatile, as we’ve seen the prior post, is not enough to ensure a memory fence is invoked under all circumstances. This can be done manually through the use of Thread.MemoryBarrier, or through the use of lock. Thread.MemoryBarrier is less understood by a wide variety of developers, so using lock is almost always what should be used prior to any micro-optimizations. For example:

lock(lockObject)

  x = 1;

  r1 = y;

and

lock(lockObject)

  y = 1;

  r2 = x;

This basically assumes x and y are involved in a particular invariant and that invariant needs to be guaranteed through atomic access to the pair of variables—which is done by creating a critical regions of code where only one region can be executing at a time across threads.

Revisiting the volatile keyword

The first post in this series could have came of as suggesting that volatile is always a good thing. As we’ve seen in the above, that’s not true. Let me be clear: using volatile in what I described previously is an optimization. It should be a micro-optimization that should be used very, very carefully. What is an isn’t an atomic invariant isn’t always cut and dry. Not every operation on an atomic type is an atomic operation.

Let’s look at some of the problems of volatile:

The first, and arguably the most discussed problem, is that volatile decorates a variable not the use of that variable. With non-atomic operations on an atomic variable, volatile can give you a false sense of security. You may think volatile gives you thread-safe code in all accesses to that variable, but it does not. For example:

private volatile int counter;

private void DoSomething()

    //...

    counter++;

    //...

Although many processors have a single instruction to increment an integer, “there is no guarantee of atomic read-modify-write, such as in the case of increment or decrement” [1]. Despite counter being volatile, there’s no guarantee this operation will be atomic and thus there’s no guarantee that it will be thread-safe. In the general case, not every type you can use operator++ on is atomic—looking strictly at “counter++;”, you can’t tell if that’s thread-safe.. If counter were of type long, access to counter is no longer atomic and a single instruction to increment it is only possible on some processors (regardless of lock of guarantees that it will be used). If counter were an atomic type, you’d have to check the declaration of the variable to see if it was volatile or not before deciding if it’s potentially thread-safe. To make incrementing a variable thread-safe, the Interlocked class should be used for supported types:

private int counter;

private void DoSomething()

    //...

    System.Threading.Interlocked.Increment(ref counter);

    //...

Non-atomic types like long, ulong (i.e. not supported by volatile) are supported by Interlocked. For non-atomic types not supported by Interlocked, lock is recommended until you’ve verified another method is “better” and works:

private decimal counter;

private readonly object lockObject = new object();

private void DoSomething()

    //...

    lock(lockObject)

        counter++;

    //...

That is volatile is problematic because it can only be applied to member fields and only to certain types of member fields.

The general consensus is that because volatile doesn’t decorate the operations that are potentially performed in a concurrent context, and doesn’t consistently lead to more efficient code in all circumstances, and passing a volatile field by ref circumvents the fields volatility, and would fail if used with non-atomic invariants, and lack of consistency with correctly guarded non-atomic operations, etc.; that the volatile operations should be explicit through the use of Interlocked, Thread.VolatileRead, Thread.VolatileWrite, or the use of lock and not through the use of the volatile keyword.

Conclusion

Concurrent and multithreaded programming is not trivial. It involves dealing with non-sequential operations through the writing of sequential code. It’s prone to error and you really have to know the intent of your code in order to decide not only what might be used in a concurrent context as well as what is thread-safe. i.e. “thread-safe” is application specific.

Despite only really having support for x86/x64 “out of the box” in .NET 4.5 (i.e. Visual Studio 2012), the potential side-effects of assuming an x86/x64 memory model just muddies the waters. I don’t think there is any benefit to writing to a x86/x64 memory model over writing to the .NET memory model. Nothing I’ve shown really affects existing guidance on writing thread-safe and concurrent code—some of which are detailed in Visual Studio 2010 Best Practices.

Knowing what’s going on at lower levels in any particular situation is good, and anything you do in light of any side-effects should be considered micro-optimizations that should be well scrutinized.

[1] C# Language Specification § 5.5 Atomicity of variable references

Thread synchronization of atomic invariants in .NET 4.5 clarifications

PeterRitchie — Mon, 10 Sep 2012 19:26:00 GMT

In Thread synchronization of atomic invariants in .NET 4.5 I’m presenting my observations of what the compiler does in very narrow context of only on Intel x86 and Intel x64 with a particular version of .NET. You can install SDKs that give you access to compilers to other processors. For example, if you write something for Windows Phone or Windows Store, you’ll get compilers for other processors (e.g. ARM) with memory models looser than x86 and x64. That post was only observations in the context of x86 and x64.

I believe more knowledge is always better; but you have to use that knowledge responsibly. If you know you’re only ever going to target x86 or x64 (and you don’t if you use AnyCPU even in VS 2012 because some yet-to-be-created processor might be supported in a future version or update to .NET) and you do want to micro-optimize your code, then that post might give you enough knowledge to do that. Otherwise, take it with a grain of salt. I’ll get into a little bit more detail in part 2: Thread synchronization of non-atomic invariants in .NET 4.5 at a future date—which will include more specific guidance and recommendations.

In the case were I used a really awkwardly placed lock:

   1: var lockObject = new object();

   2: while (!complete)

   3: {

   4:     lock(lockObject)

   5:     {

   6:         toggle = !toggle;

   7:     }

   8: }

It’s important to point out the degree of implicit side-effects that this code depends on. One, it assumes that the compiler is smart enough to know that a while loop is the equivalent of a series of sequential statements. e.g. this is effectively equivalent to:

   1: var lockObject = new object();

   2: if (complete == false) return;

   3: lock (lockObject)

   4: {

   5:     toggle = !toggle;

   6: }

   7: if (complete == false) return;

   8: lock (lockObject)

   9: {

  10:     toggle = !toggle;

  11: }

  12: //...

That is, there is the implicit volatile read (e.g. a memory fence, from the Monitor.Enter implementation detail) at the start of the lock block and an implicit volatile write (e.g. a memory fence, from the Monitor.Exit implementation detail).

In case it wasn’t obvious, you should never write code like this, it’s simply an example—and as I pointed out in the original post, it’s confusing to anyone else reading it: lockObject can’t be shared amongst threads and the lock block really isn’t protecting toggle and can/likely to get “maintained” into something that no longer works.

In the same grain, the same can be said for the original example of this code:

   1: static void Main()

   2: {

   3:   bool complete = false;

   4:   var t = new Thread (() =>

   5:   {

   6:     bool toggle = false;

   7:     while (!complete)

   8:     {

   9:         Thread.MemoryBarrier();

  10:         toggle = !toggle;

  11:     }

  12:   });

  13:   t.Start();

  14:   Thread.Sleep (1000);

  15:   complete = true;

  16:   t.Join();

  17: }

While this code works, it’s not apparently clear that the Thread.MemoryBarrier() is there so that our read of complete (and not toggle) isn’t optimized into a registry read. Regardless of the degree you might be able to depend on the compiler continuing to do this is up to you. The code is equally as valid and more clear if written to use Thread.VolatileRead, except for the fact that Thread.VolatileRead does not support the Boolean type. It can be re-written using Int32 instead. For example:

   1: static void Main(string[] args)

   2: {

   3:   int complete = 0;

   4:   var t = new Thread (() =>

   5:   {

   6:     bool toggle = false;

   7:     while (Thread.VolatileRead(ref complete) == 0)

   8:     {

   9:         toggle = !toggle;

  10:     }

  11:   });

  12:   t.Start();

  13:   Thread.Sleep (1000);

  14:   complete = 1; // CORRECTION from 0

  15:   t.Join();

  16: }

Which is more clear and shows your intent more explicitly.

Thread synchronization of atomic invariants in .NET 4.5

PeterRitchie — Sun, 09 Sep 2012 16:52:00 GMT

I've written before about multi-threaded programming in .NET (C#). Spinning up threads and executing code on another thread isn't really the hard part. The hard part is synchronization of data between threads.

Most of what I've written about is from a processor agnostic point of view. It's written from the historical point of view: that .NET supports many processors with varying memory models. The stance has generally been that you're programming for the .NET memory model and not a particular processor memory model.

But, that's no longer entirely true. In 2010 Microsoft basically dropped support for Itanium in both Windows Server and in Visual Studio (http://blogs.technet.com/b/windowsserver/archive/2010/04/02/windows-server-2008-r2-to-phase-out-itanium.aspx). In VS 2012 there is no “Itanium” choice in the project Build options. As far as I can tell, Windows 2008 R2 is the only Windows operating system, still in support, that supports Itanium. And even Windows 2008 R2 for Itanium is not supported for .NET 4.5 (http://msdn.microsoft.com/en-us/library/8z6watww.aspx)

So, what does this mean to really only have the context of running only x86/x64? Well, if you really read the documentation and research the Intel x86 and x64 memory model this could have an impact in how you write multi-threaded code with regard to shared data synchronization. x86 and x64 memory models include guarantees like “In a multiple-processor system…Writes by a single processor are observed in the same order by all processors.” but does and also includes guarantees like “Loads May Be Reordered with Earlier Stores to Different Locations”. What this really means is that a store or a load to a single location won’t be reordered with regard to a load or a store to the same location across processors. That is we don’t need fences to ensure a store to a single memory location is “seen” by all threads or that a load from memory loads the “most recent” value stored. But, it does mean that in order for multiple stores to multiple locations to be viewed by other threads in the same order, a fence is necessary (or the group of store operations is invoked as an atomic action through the user of synchronization primitives like Monitor.Enter/Exit, lock, Semaphore, etc.) (See section 8.2 Memory Ordering of the Intel Software Developer’s Manual Volume 3A found here). But, that deals with non-atomic invariants which I’ll detail in another post.

To be clear, you could develop to just x86 or just x64 prior to .NET 4.5 and have all the issues I’m about to detail.

Prior to .NET 4.5 you really programmed to the .NET memory model. This has changed over time since ECMA defined it around .NET 2.0; but that model was meant to be a “supermodel” to deal with the fact that .NET could be deployed to different CPUs with disparate memory models. Most notably was the Itanium memory model. This model is much looser than the Intel x86 memory model and allowed things like a store without a release fence and a load without an acquire fence. This meant that a load or a store might be done only in one CPU’s memory cache and wouldn’t be flushed to memory until a fence. This also meant that other CPUs (e.g. other threads) may not see the store or may not get the "latest" value with a load. You can explicitly cause release and acquire fences with .NET with things like Monitor.Enter/Exit (lock), Interlocked methods, Thread.MemoryBarrier, Thread.VolatileRead/VolatileWrite, etc. So, it wasn't a big issue for .NET programmers to write code that would work on an Itanium. For the most part, if you simply guarded all your shared data with a lock, you were fine. lock is expensive, so you could optimize things with Thread.VolatileRead/VolatileWriter if your shared data was inherently atomic (like a single int, a single Object, etc) or you could use the volatile keyword (in C#). The conventional wisdom has been to use Thread.VolatileRead/VolatileWrite rather than decorate a field with volatile because you may not need every access to be volatile and you don’t want to take the performance hit when it doesn’t need to be volatile.

For example (borrowed from Jeffrey Richter, but slightly modified) shows synchronizing a static member variable with Thread.VolatileRead/VolatileWrite:

   1: public static class Program {

   2:   private static int s_stopworker;

   3:   public static void Main() {

   4:     Console.WriteLine("Main: letting worker run for 5 seconds");

   5:     Thread t = new Thread(Worker);

   6:     t.Start();

   7:     Thread.Sleep(5000);

   8:     Thread.VolatileWrite(ref s_stopworker, 1);

   9:     Console.WriteLine("Main: waiting for worker to stop");

  10:     t.Join();

  11:   }

12:

  13:   public static void Worker(object o) {

  14:     Int32 x = 0;

  15:     while(Thread.VolatileRead(ref s_stopworker) == 0)

  16:     {

  17:       x++;

  18:     }

  19:   }

  20: }

Without the call to Thread.VolatileWrite the processor could reorder the write of 1 to s_stopworker to after the read (assuming we’re not developing to on particular processor memory model and we’re including Itanium). In terms of the compiler, without Thread.VolatileRead it could cache the value being read from s_stopworker in to a register. For example, removing the Thread.VolatileRead, the compiler optimizes the comparison of s_stopworker to 0 in the while to single register (on x86):

00000000  push        ebp

00000001  mov         ebp,esp

00000003  mov         eax,dword ptr ds:[00213360h]

00000008  test        eax,eax

0000000a  jne         00000010

0000000c  test        eax,eax

0000000e  je          0000000C

00000010  pop         ebp

00000011  ret

The loop is 0000000c to 0000000e (really just testing that the eax register is 0). Using Thread.VolatileRead, we’d always get a value from a physical memory location:

00000000  push        ebp

00000001  mov         ebp,esp

00000003  lea         ecx,ds:[00193360h]

00000009  call        71070480

0000000e  test        eax,eax

00000010  jne         00000021

00000012  lea         ecx,ds:[00193360h]

00000018  call        71070480

0000001d  test        eax,eax

0000001f  je          00000012

00000021  pop         ebp

00000022  ret

The loop is now 00000012 to 0000001f, which shows calling Thread.VolatileRead each iteration (location 00000018). But, as we’ve seen from the Intel documentation and guidance, we don’t really need to call VolatileRead, we just don’t want the compiler to optimize the memory access away into a register access. This code works, but we take the hit of calling VolatileRead which forces a memory fence through a call to Thread.MemoryBarrier after reading the value. For example, the following code is equivalent:

while(s_stopworker == 0)

  Thread.MemoryBarrier();

  x++;

And this works equally as well as using Thread.VolatileRead, and compiles down to:

00000000  push        ebp

00000001  mov         ebp,esp

00000003  cmp         dword ptr ds:[002A3360h],0

0000000a  jne         0000001A

0000000c  lock or     dword ptr [esp],0

00000011  cmp         dword ptr ds:[002A3360h],0

00000018  je          0000000C

0000001a  pop         ebp

0000001b  ret

The loop is now is 0000000c to 00000018. As we can see at 0000000c we have an extra “lock or” instruction—which is what the compiler optimizes a call to Thread.MemoryBarrier to. This instruction really just or’s 0 with what esp is pointing to (i.e. “nothing”, zero or’ed with something else does not change the value). But the lock prefix forces a fence and is less expensive than instructions like mfence. But, based on what we know of the x86/x64 memory model, we’re only dealing with a single memory location and we don’t need that lock prefix—the inherent memory guarantees of the processor means that our thread can see any and all writes to that memory location without this extra fence. So, what can we do to get rid of it? Well, using volatile actually results in code that doesn’t generate that lock or instruction. For example, if we change our code to make s_stopworker volatile:

   1: public static class Program {

   2:   private static volatile int s_stopworker;

   3:   public static void Main() {

   4:     Console.WriteLine("Main: letting worker run for 5 seconds");

   5:     Thread t = new Thread(Worker);

   6:     t.Start();

   7:     Thread.Sleep(5000);

   8:     s_stopworker = 1;

   9:     Console.WriteLine("Main: waiting for worker to stop");

  10:     t.Join();

  11:   }

12:

  13:   public static void Worker(object o) {

  14:     Int32 x = 0;

  15:     while(s_stopworker == 0)

  16:     {

  17:       x++;

  18:     }

  19:   }

  20: }

We tell the compiler that we don’t want accesses to s_stopworker optimized. This then compiles down to:

00000000  push        ebp

00000001  mov         ebp,esp

00000003  cmp         dword ptr ds:[00163360h],0

0000000a  jne         00000015

0000000c  cmp         dword ptr ds:[00163360h],0

00000013  je          0000000C

00000015  pop         ebp

00000016  ret

The loop is now 0000000c to 00000013. Notice that we’re simply getting the value from memory on each iteration and comparing it to 0. There’s no lock or. One less instruction and no extra memory fence. Although in many cases it doesn’t matter (i.e. you might only do this once, in which case an extra few milliseconds won’t hurt and this might be a premature optimization), but using lock or with the register optimization is about 992% slower when measured on my computer (or volatile is 91% faster than using Thread.MemoryBarrier and probably a bit faster still than use Thread.VolatileRead). This is actually contradictory to conventional wisdom with respect to a .NET memory model that supports Itanium. If you want to support Itanium, every access to a volatile field would be tantamount to Thread.VolatileRead or Thread.VolatileWrite, in which case, yes, in scenarios where you don’t really need the field to be volatile, you take a performance hit.

In .NET 4.5 where Itanium is out of the picture, you might be thinking “volatile all the time then!”. But, hold on a minute, let’s look at another example:

   1: static void Main()

   2: {

   3:   bool complete = false;

   4:   var t = new Thread (() =>

   5:   {

   6:     bool toggle = false;

   7:     while (!complete)

   8:     {

   9:         Thread.MemoryBarrier();

  10:         toggle = !toggle;

  11:     }

  12:   });

  13:   t.Start();

  14:   Thread.Sleep (1000);

  15:   complete = true;

  16:   t.Join();

  17: }

This code (borrowed from Joe Albahari) will block indefinitely at the call to Thread.Join (line 16) without the call to Thread.MemoryBarrier() (at line 9).

This code blocks indefinitely without Thread.MemoryBarrier() on both x86 and x64; but this is due to compiler optimizations, not because of the processor’s memory model. We can see this in the disassembly of what the JIT produces for the thread lambda (on x64):

00000000  push        ebp

00000001  mov         ebp,esp

00000003  movzx       eax,byte ptr [ecx+4]

00000007  test        eax,eax

00000009  jne         0000000F

0000000b  test        eax,eax

0000000d  je          0000000B

0000000f  pop         ebp

00000010  ret

Notice the loop (0000000b to 0000000d), the compiler has optimized access to the variable toggle into a register and doesn’t update that register from memory—identical to what we saw with the member field above. If we see the disassembly when using MemoryBarrier:

00000000  movzx       eax,byte ptr [rcx+8]

00000004  test        eax,eax

00000006  jne         0000000000000020

00000008  nop         dword ptr [rax+rax+00000000h]

00000010  lock or     dword ptr [rsp],0

00000015  movzx       eax,byte ptr [rcx+8]

00000019  test        eax,eax

0000001b  je          0000000000000010

0000001d  nop         dword ptr [rax]

00000020  rep ret

We see that loop testing toggle (instructions from 00000010 to 0000001b) grabs the memory value into eax then tests eax until it’s true (or non-zero). MemoryBarrier has been optimized to “lock or” here as well.

What we’re dealing with here is a local variable and can’t use the volatile keyword. We could use the lock keyword to get a fence, it couldn’t be around the comparison (the while) because that would enclose the entire while block and would never exit the lock to get the memory fence and thus the compiler believes reads of toggle aren’t guarded by lock’s implicit fences. We’d have to wrap the assignment to toggle to get the release fence before and the acquire fence after, ala:

var lockObject = new object();

while (!complete)

    lock(lockObject)

        toggle = !toggle;

Clearly this lock block isn’t really a critical section because the lockObject instance can’t be shared amongst threads. Anyone reading this code is likely going to think “WTF?”. But, we do get our fences, and the compiler will not optimize access to toggle to only a register and our code will no longer block at the call to Thread.Join. It’s apparent that Thread.MemoryBarrier is the better choice in this scenario, it’s just more readable and doesn’t appear to be poorly written code (i.e. code that only depends on side effects).

But, you still take the performance hit on “lock or”. If you want to avoid that, then refactor the local toggle variable to a field and decorate it with volatile.

Although some of this seems like micro-optimizations, but it’s not. You have to be careful to “synchronize” shared atomic data with respect to compiler optimizations, so you might as well pick the best way that works.

In the next post I’ll get into synchronizing non-atomic invariants shared amongst threads.

Visual Studio 2010 Best Practices published

PeterRitchie — Sat, 25 Aug 2012 22:12:00 GMT

Most of my spare time in the last few months has been taken up by writing Visual Studio 2010 Best Practices. This has now been published and is available through publisher (no longer pre-order) at http://bit.ly/Px43Pw. The pre-order price is still available for a limited time. Amazon still has it out of stock; but $39.99 at http://amzn.to/QDDmF7.

The title of the book really doesn't do the content justice. Least of which is "Best Practices". Anyone who knows me should know I don't really like that term. But, hopefully those looking for best practices will read the book and learn from chapter one why "best practice" has problems.

While it's called "Visual Studio 2010 Best Practices" it isn't limited to the UI of Visual Studio (or Visual Studio 2010 really, for that matter). It's really a set of generally accepted recommended practices based on expertise and experience for any and all developers of .NET (it assumes they use Visual Studio--but many practices deal with general design/development that can be applied almost anywhere). There are some specifics in there about the Visual Studio UI like optimizing Visual Studio settings/configuration, useful features, the correct way to use certain features, etc. But, that's mostly limited to one chapter. Other chapters include recommended practices regarding C#, SCC, deployment, testing, parallelization/multithreading, distributed applications and web services. From the book overview:

Learning source code control
Practices for advanced C# syntax
Asynchronous programming in C#
Learn tips for architecting decoupled systems
Practices for designing multi-threaded and parallel systems
Practices for designing distributed systems
Learn better ways of developing web services with WCF
Learn faster ways to design automated tests
Tips and tricks to test complex systems
Understanding proven ways of deploying software systems in Windows

Kind of a mixed bag; but, you have to work within the bounds you've been given :). It was limited to about 200 pages; so, of course, I couldn’t go into every recommended practice or every useful tidbit that everyone could use…

I'd like to thank a few people for helping-out outside of the publisher's review channel (i.e. they're not mentioned in the book): Joe Miller, Amir Barylko, and of course all those that offered…

Automated Testing Isn’t Just for Business Logic

PeterRitchie — Fri, 25 May 2012 17:51:00 GMT

I had a conversation with Kelly Sommers the other day that was partially a short support group session on the annoying tendencies of development teams to completely lose focus on the architecture and design principles of a system and let the code base devolve into a ball of muddy spaghetti.

One particular area that we discussed, and it’s one area I’ve detailed elsewhere, has to do with layers. Our gripe was that developers seem to completely ignore layering principles once they start coding and introduce cycles, put things in the wrong layer, etc. A brief recap of layering principles: Types in one layer can only access types in the adjacent lower layer. That’s it. Types that access types in a layer above are violating layering (or aren’t layer) and types that access types in a layer lower than the adjacent lower level (e.g. two layers down) are violating layering.

I’ve blogged about Visual Studio and layers (and validation) before; but not everyone uses the part of Visual Studio or doesn’t have that edition of Visual Studio. I mentioned in our conversation it’s fairly easy to write unit tests to make these verifications. I’ve written tests like this before, but the assumption was that “layers” were in different assemblies. The verification for this scenario is quite a bit simpler; so, I thought I’d tackle a test that verifies layering within a single assembly where namespaces are the scope of layers.

My initial code used Enumerable.Any to see if any types from a lower layer not adjacent to the current layer where used in this layer or whether any types from any layers above the current layer where used in this layer. This did the validation, but basically left the dev with a “test failed and I’m not giving you any details” message because we couldn’t tell where the violation was and what violated it—which isn’t too friendly. So, I expanded it out to detail all the violations. I came up with a utility method ValidateLayerRelationships would be used as follows:


public enum Layer {
    // Order is important!
    Data,
    Domain,
    UI
}
 
[TestMethod]
public void ValidateLayerUsage()
{
    var relatedNamespaces = new[] { "PRI.Data", "PRI.Domain", "PRI.FrontEnd", "PRI.ViewModels" };
 
    var levelMap = new Dictionary<string, Layer> {
                    {relatedNamespaces[0], Layer.Data},
                    {relatedNamespaces[1], Layer.Domain},
                    {relatedNamespaces[2], Layer.UI},
                    {relatedNamespaces[3], Layer.UI},
                    };
 
    var assemblyFileName = "ClassLibrary.dll";
    ValidateLayerRelationships(levelMap, assemblyFileName);
}

In this example I have two namespaces in one layer (the UI layer) FrontEnd and ViewModels and two other layers with just one namespace in each (Data with Data and Domain with Domain). mostly to show you can have more than one namespace per layer… We define a layer map, and the filename of the assembly we want to validate and call ValidateLayerRelationships. ValidateLayerRelationships is as follows:


private static void ValidateLayerRelationships(Dictionary<string, Layer> levelMap, string assemblyFileName) {
    // can't use ReflectionOnlyLoadFrom because we want to peek at attributes
    var groups = from t in Assembly.LoadFrom(assemblyFileName).GetTypes()
                    where levelMap.Keys.Contains(t.Namespace)
                    group t by t.Namespace
                    into g
                    orderby levelMap[g.Key]
                    select g;
 
    var levelsWithClasses = groups.Count();
    Assert.IsTrue(levelsWithClasses > 1, "Need more than two layers to validate relationships.");
 
    var errors = new List<string>();
    foreach (var g in groups){
        var layer = levelMap[g.Key];
        // verify this level only accesses things from the adjacent lower layer (or layers)
        var offLimitSubsets = from g1 in groups where !new[] {layer - 1, layer}.Contains(levelMap[g1.Key]) select g1;
        var offLimitTypes = offLimitSubsets.SelectMany(x => x).ToList();
        foreach (Type t in g){
            foreach (MethodInfo m in t.GetAllMethods()){
                var methodBody = m.GetMethodBody();
                if (methodBody != null)
                    foreach (LocalVariableInfo v in methodBody
                        .LocalVariables
                        .Where(v => offLimitTypes
                                        .Contains(v.LocalType)))
                    {
                        errors.Add(
                            string.Format(
                                "Method \"{0}\" has local variable of type {1} from a layer it shouldn't.",
                                m.Name,
                                v.LocalType.FullName));
                    }
                foreach (ParameterInfo p in m
                    .GetParameters()
                    .Where(p => offLimitTypes
                                    .Contains(p.ParameterType)))
                {
                    errors.Add(
                        string.Format(
                            "Method \"{0}\" parameter {2} uses parameter type {1} from a layer it shouldn't.",
                            m.Name,
                            p.ParameterType.FullName,
                            p.Name));
                }
                if (offLimitTypes.Contains(m.ReturnType)){
                    errors.Add(
                        string.Format(
                            "Method \"{0}\" uses return type {1} from a layer it shouldn't.",
                            m.Name,
                            m.ReturnType.FullName));
                }
            }
            foreach (PropertyInfo p in t
                .GetAllProperties()
                .Where(p => offLimitTypes.Contains(p.PropertyType)))
            {
                errors.Add(
                    string.Format(
                        "Type \"{0}\" has a property \"{1}\" of type {2} from a layer it shouldn't.",
                        t.FullName,
                        p.Name,
                        p.PropertyType.FullName));
            }
            foreach(FieldInfo f in t.GetAllFields().Where(f=>offLimitTypes.Contains(f.FieldType)))
            {
                errors.Add(
                    string.Format(
                        "Type \"{0}\" has a field \"{1}\" of type {2} from a layer it shouldn't.",
                        t.FullName,
                        f.Name,
                        f.FieldType.FullName));
            }
        }
    }
    if (errors.Count > 0)
        Assert.Fail(String.Join(Environment.NewLine, new[] {"Layering violation."}.Concat(errors)));
}

This method groups types within a layer, then goes through any layers that layer shouldn’t have access to (i.e. any layer that isn’t the lower adjacent layer, or “layer – 1, layer” where we create offLimitSubsets). For each type we look at return values, parameter values, fields, properties, and methods for any types they use. If any of those types are one of the off limit types, we add an error to our error collection. At the end, if there’s any errors we assert and format a nice message with all the violations.

This is a helper method that you’d use somewhere (maybe a helper static class, within the existing test class, whatever).

This uses some extension classes to make it a bit more readable, which are here:


public static class TypeExceptions {
    public static IEnumerable<MethodInfo> GetAllMethods(this Type type) {
        if (type == null) throw new ArgumentNullException("type");
        return
            type.GetMethods(BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Static | BindingFlags.Public).Where(
                m => !m.GetCustomAttributes(true).Any(a => a is CompilerGeneratedAttribute));
    }
    public static IEnumerable<FieldInfo> GetAllFields(this Type type) {
        if (type == null) throw new ArgumentNullException("type");
        return type.GetFields(BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Static | BindingFlags.Public)
            .Where(f => !f.GetCustomAttributes(true).Any(a => a is CompilerGeneratedAttribute));
    }
    public static IEnumerable<PropertyInfo> GetAllProperties(this Type type) {
        if (type == null) throw new ArgumentNullException("type");
        return
            type.GetProperties(BindingFlags.Instance | BindingFlags.NonPublic | BindingFlags.Static | BindingFlags.Public).Where
                (p => !p.GetCustomAttributes(true).Any(a => a is CompilerGeneratedAttribute));
    }
}

Because the compiler generates fields for auto properties and methods for properties, we want to filter out any compiler-generated stuff (what caused the compiler to generate the code will raise a violation) so we don’t get any duplicate violations (and violations the user can’t do anything about). (which is what the call to GetCustomAttributes is doing)

I wasn’t expecting this to be that long; so, in future blog entries I’ll try to detail some other unit tests that validate or verify specific infrastructural details. If you have any specific details you’re interested in, leave a comment.

Dispose Pattern and “Set large fields to null”

PeterRitchie — Fri, 27 Apr 2012 02:25:12 GMT

I was involved in a short side discussion about “should” fields be set to null in the Dispose method(s). I’m not sure what the impetus of the question was; but, if you read through the dispose pattern MSDN documentation (in most versions I believe) there’s a comment // Set large fields to null. in the implementation of the virtual Dispose method within the if(!disposed) block and after the if(disposing) block. But, that’s the only reference to setting fields to null during dispose. There’s nothing else that I’ve been able to find in MSDN with regard to setting fields to null.

At face value, setting a field to null means that the referenced object is now unrooted from the class that owns the field and, if that was the last root of that reference, the Garbage Collector (GC) is now free to release the memory used by the object that was referenced by that field. Although advanced, this seems all very academic because the amount of time between unrooting the reference and the return from Dispose (and thus the unrooting of the parent object) would seem like a very short amount of time. Even if the amount of time between these two actions is small, setting a single field to null (i.e. a single assignment) seems like such a minor bit of code to provide no adverse affects. The prevalent opinion seems to be that the GC “handles” this case and does what is best for you without setting the field to null.

The GC is pretty smart. There’s a lot of bright people who have worked on the GC over the years; and it improves every release of .NET. But, that doesn’t answer the question; is there benefit to setting a field to null in the Dispose method? Considering there isn’t much guidance on the topic; I’d though I’d set aside any faith I have in the GC and throw some science at the problem: take my theory, create some experiments, make observations, and collect some evidence.

What I did was to create two classes, identical except that the Dispose method doesn’t set a reference field to null. The classes contain an field that could reference a “large” or “small” object: I would experiment with large objects and small objects and observe the differences. The following are the classes:

	public class First : IDisposable {
		int[] arr = new int[Constants.ArraySize];
		public int[] GetArray() {
			return arr;
		}
		public void Dispose() {
			arr = null;
		}
	}
 
	public class Second : IDisposable {
		int[] arr = new int[Constants.ArraySize];
		public int[] GetArray() {
			return arr;
		}
 
		public void Dispose() {
		}
	}

I would vary Constants.ArraySize constant to make the arr reference a “large” object or a “small” object. I then created a loop that created several thousand instances of one of these classes then forced a garbage collection at the end; keeping track of the start time and the end time via Stopwatch:

	public class Program {
		private const int Iterations = 10000;
 
		static void Main(string[] args)
		{
			var stopwatch = Stopwatch.StartNew();
			for (int i = 0; i < Iterations; ++i)
			{
				using (var f = new First())
				{
					ConsumeValue(f.GetArray().Length);
				}
			}
			GC.Collect();
			stopwatch.Stop();
			Trace.WriteLine(String.Format("{0} {1}", stopwatch.Elapsed, stopwatch.ElapsedTicks));
			stopwatch = Stopwatch.StartNew();
			for (int i = 0; i < Iterations; ++i)
			{
				using (var s = new Second())
				{
					ConsumeValue(s.GetArray().Length);
				}
			}
			GC.Collect();
			stopwatch.Stop();
			Trace.WriteLine(String.Format("{0} {1}", stopwatch.Elapsed, stopwatch.ElapsedTicks));
		}
 
		static void ConsumeValue(int x) {
		}
	}

I wanted to make sure instanced got optimized away so the GetArray method makes sure the arr field sticks around and the ConsumeValue makes sure the First/Second instances stick around (more a knit-picker circumvention measure :). Results are the 2nd result from running the application 2 times.

As it turns out, the results were very interesting (at least to me :). The results are as follows:

Iterations: 10000 ArraySize: 85000 Debug: yes Elapsed time: 00:00:00.0759408 Ticks: 170186.
Iterations: 10000 ArraySize: 85000 Debug: yes Elapsed time: 00:00:00.7449450 Ticks: 1669448.

Iterations: 10000 ArraySize: 85000 Debug: no Elapsed time: 00:00:00.0714526 Ticks: 160128.
Iterations: 10000 ArraySize: 85000 Debug: no Elapsed time: 00:00:00.0753187 Ticks: 168792.

Iterations: 10000 ArraySize: 1 Debug: yes Elapsed time: 00:00:00.0009410 Ticks: 2109.
Iterations: 10000 ArraySize: 1 Debug: yes Elapsed time: 00:00:00.0007179 Ticks: 1609.

Iterations: 10000 ArraySize: 1 Debug: no Elapsed time: 00:00:00.0005225 Ticks: 1171.
Iterations: 10000 ArraySize: 1 Debug: no Elapsed time: 00:00:00.0003908 Ticks: 876.

Iterations: 10000 ArraySize: 1000 Debug: yes Elapsed time: 00:00:00.0088454 Ticks: 19823.
Iterations: 10000 ArraySize: 1000 Debug: yes Elapsed time: 00:00:00.0062082 Ticks: 13913.

Iterations: 10000 ArraySize: 1000 Debug: no Elapsed time: 00:00:00.0096442 Ticks: 21613.
Iterations: 10000 ArraySize: 1000 Debug: no Elapsed time: 00:00:00.0058977 Ticks: 13217.

Iterations: 10000 ArraySize: 10000 Debug: yes Elapsed time: 00:00:00.0527439 Ticks: 118201.
Iterations: 10000 ArraySize: 10000 Debug: yes Elapsed time: 00:00:00.0528719 Ticks: 118488.

Iterations: 10000 ArraySize: 10000 Debug: no Elapsed time: 00:00:00.0478136 Ticks: 107152.
Iterations: 10000 ArraySize: 10000 Debug: no Elapsed time: 00:00:00.0524012 Ticks: 117433.

Iterations: 10000 ArraySize: 40000 Debug: yes Elapsed time: 00:00:00.0491652 Ticks: 110181.
Iterations: 10000 ArraySize: 40000 Debug: yes Elapsed time: 00:00:00.3580011 Ticks: 802293.

Iterations: 10000 ArraySize: 40000 Debug: no Elapsed time: 00:00:00.0467649 Ticks: 104802.
Iterations: 10000 ArraySize: 40000 Debug: no Elapsed time: 00:00:00.0487685 Ticks: 109292.

Iterations: 10000 ArraySize: 30000 Debug: yes Elapsed time: 00:00:00.0446106 Ticks: 99974.
Iterations: 10000 ArraySize: 30000 Debug: yes Elapsed time: 00:00:00.2748007 Ticks: 615838.

Iterations: 10000 ArraySize: 30000 Debug: no Elapsed time: 00:00:00.0411109 Ticks: 92131.
Iterations: 10000 ArraySize: 30000 Debug: no Elapsed time: 00:00:00.0381225 Ticks: 85434.

For the most part, results in debug mode are meaningless. There’s no point in making design/coding decisions based on perceived benefits in debug mode; so, I don’t the results other than to document them above.

The numbers could go either way, if we look at percentages; release mode, setting a field to null seems to be slower 50% of the time and faster 50% of the time. When setting a field to null is faster it’s insignificantly faster (5.41%, 9.59%, and 4.28% faster) when it’s slower it’s insignificantly slower but more slow than it is fast (133.68%, 163.52%, and 107.84% slower). Neither seems to make a whole lot of difference (like 10281 ticks over 10000 iterations in the biggest difference for about 1 tick per iteration—1000 byte array at 10000 iterations). If we look at just the time values, we start to see that setting a field starts to approach being faster (when it’s slower it’s slower by 295, 8396, and 6697 ticks; when it’s faster it’s faster by 8664, 10281, 4490). Oddly though, setting “large” fields to null isn’t the biggest of faster setting field to null values. But, of course, I don’t know what the documentation means by “large”; it could be large-heap objects or some other arbitrary size.

Of course there’s other variables that could affect things here that I haven’t accounted for (server GC, client GC, GC not occurring at specific time, better sample size, better sample range, etc.); so, take the results with a grain of salt.

What should you do with this evidence? It’s up to you. I suggest not taking it as gospel and making a decision that is best for you own code based on experimentation and gathered metrics in the circumstances unique to your application and its usage.. i.e. setting a field to null in Dispose is neither bad nor good in the general case.

If You’re Using “#if DEBUG”, You’re Doing it Wrong

PeterRitchie — Thu, 24 Nov 2011 14:50:00 GMT

I was going through some legacy code the other day, refactoring it all over the place and I ran into many blocks of code wrapped in “#if DEBUG”. Of course, after a bit of refactoring in a RELEASE configuration these blocks of code were quickly out of date (and by out of date, I mean no longer compiling). A huge PITA.

For example, take the following code:

	public class MyCommand
	{
		public DateTime DateAndTimeOfTransaction;
	}
 
	public class Test
	{
		//...
		public void ProcessCommand(MyCommand myCommand)
		{
#if DEBUG
			if (myCommand.DateAndTimeOfTransaction > DateTime.Now)
				throw new InvalidOperationException("DateTime expected to be in the past");
#endif
			// do more stuff with myCommand...
		}
		//...
	}

If, while my active configuration is RELEASE, I rename-refactor DateAndTimeOfTransaction to TransactionDateTime then the block of code in the #if DEBUG is now invalid—DateAndTimeOfTransaction will not be renamed within this block. I will now get compile errors if I switch to DEBUG configuration (or worse still, if I check in an my continuous integration environment does a debug build).

I would have run into the same problem had I been in a DEBUG configuration with all the “#if RELEASE” blocks. Yes, I could create a configuration and define DEBUG and RELEASE and do my work in there. Yeah, no, not going down that twisted path… i.e. try doing it without manually adding a conditional compilation symbol.

I got to thinking about a better way. It dawned on me that #if blocks are effectively comments in other configurations (assuming DEBUG or RELEASE; but true with any symbol) and that comments are apologies, and it quickly became clear to me how to fix this.

Enter Extract Method refactoring and Conditional Methods. If you’ve been living under a rock or have simply forgotten, there exists a type in the BCL: System.Diagnostics.ConditionalAttribute. You put this attribute on methods that[whose calls] may or may not be included in the resulting binary (IL). But, they are still compiled (and thus syntax checked). So, if I followed the basic tenet for code comment smells and performed an Extract Method refactoring and applied the ConditionalAttribute to the resulting method, I’d end up with something like this:

		//...
		public void ProcessCommand(MyCommand myCommand)
		{
			CheckMyCommandPreconditions(myCommand);
			// do more stuff with myCommand...
		}
 
		[Conditional("DEBUG")]
		private static void CheckMyCommandPreconditions(MyCommand myCommand)
		{
			if (myCommand.DateAndTimeOfTransaction > DateTime.Now)
				throw new InvalidOperationException("DateTime expected to be in the past");
		}

Now, if I perform a rename refactoring on MyCommand.DateAndTimeOfTransaction, all usages of DateAndTimeOfTransaction get renamed and I no longer introduce compile errors.

Code Contracts

If you look closely at the name I chose for the extracted method you’ll notice that what this DEBUG code is actually doing is asserting method preconditions. This type of thing is actually supported in Code Contracts. Code Contracts implement a concept called Design by Contract (DbC). One benefit to DbC is that these checks are effectively proved at compile time so there’s no reason to perform this check at runtime or even write unit tests to check them because code to violate them becomes a “compile error” with Code Contracts. But, for my example I didn’t use DbC despite potentially being a better way of implementing this particular code.

Anyone know what we call a refactoring that introduces unwanted side-effects like compile errors?

[UPDATE: small correction to the description of conditional methods and what does/doesn't get generated to IL based on comment from Motti Shaked]

Windows 8, What Does It Mean to Me?

PeterRitchie — Tue, 20 Sep 2011 00:07:14 GMT

Well, unless you’ve been hiding under a rock, you’ve probably heard about Microsoft Build conference that happened in Anaheim last week. It was during this conference that Microsoft finally released details about the successor to Windows 7. This event is solely developer-focused and, in my opinion, tablet-specific. It went into a lot of detail about the added APIs and usability changes in Windows 8 to better support tablet and touch-based computers.

First off, the new touch-first usability changes are called “Metro” and applications written for touch are called “Metro-style apps”.

The term “touch-first” is used because it’s not touch-only. Metro will work fine with a mouse and a keyboard; but the user experience is obviously designed for a multi-touch device and that you can “get by” adequately with a mouse and a keyboard. This was done for various reasons, one being that touch isn’t good for entering reams of text and that you want to dock your tablet and use a keyboard and mouse when you want to do something that would be better served by a keyboard or a mouse. e.g. tablet/touch-based UX is better for consuming information, whereas keyboard is better for producing information.

So, with all the focus was on tablet and touch, does that mean Windows 8 is all about Metro? The focus of the conference was to show off (likely in answer to other vendors producing tablet-based hardware and systems) tablet- and touch-based user experiences. In fact, paying attendees received a custom-made Samsung tablet computer with a version of Windows 8 (called Developer Preview) and the next version of Visual Studio (called Dev11, for now) pre-installed. This puts the focus of Developer Preview almost solely on touch. But, does this mean that mouse/keyboard was ignored or that we need a touch screen on our computers to be able to use Windows 8 when it is released? No, of course not. Discussing various things with numerous Softies, the message time and time again was that if something isn’t mentioned, or doesn’t have focus, that doesn’t mean it won’t exist in the release version or in the near future. It was clear that this version of Windows 8 was produced strictly for this event, and since it was effectively deployed on tablet, it had a tablet edition of the OS. I.e. if you by a desktop without a multi-touch monitor after Windows 8 comes out, the edition of Windows 8 that will be deployed will not likely have Metro on by default. But, that’s my opinion.

We now have Metro apps, and they’re XAML-based and very similar to Silverlight. You initialize the runtime slightly differently and do some other things slightly differently; but if you’re a Silverlight developer you’ll be right at home in Metro. Does that mean all Windows 8 development is Metro? Definitely not. It’s not C++ based and it is still managed code backed by the CLR (they shows using VB and C#; but F# support should be there for RTM). The CLR backing Metro apps is the same CLR that backs “Desktop” apps written as WinForms (yes, they’re still supporting that) traditional Silverlight, Console apps, Windows services apps, etc. etc. etc.

Does this mean Silverlight and .NET is dead? Well, no; but you could interpret it that way. Metro apps are driven by a new Windows API called WinRT—which is basically a object-oriented alternative for the Win32 API and removes blocking methods leaving, or replacing them with, asynchronous methods. This and the CLR that backs it are inherent parts of Windows 8. So, it’s technically not it’s own product and simply just another part of Windows. So, you could say that .NET and Silverlight are just bits of Windows 8 and thus no longer exist as “.NET” and “Silverlight”. Just like Hydra or Terminal Services really doesn’t exist any more because it’s baked right into Windows. But, that doesn’t mean they don’t exist in this new form.

Some discussion has occurred as to whether WinRT is truly CLI-compliant because it doesn’t support all the types and methods that .NET does now. This isn’t true. We had these same discussions when Silverlight and Windows Phone came out. WinRT is just another “profile”. It has a subset .NET API. Just like how Silverlight doesn’t include classes like System.Windows.Forms.Form and Windows Phone doesn’t include classes like System.Windows.Browser.HtmlDocument. The depth of the .NET classes that are available to WinRT developers is restricted, just restricted differently than Silverlight and Windows Phone.

Just how restricted is the WinRT profile? Well, that remains to be seen at this point; but, from the looks of it it is minimally CLI compliant. This means it doesn’t include things like the reflection namespaces. WinRT is, after all, for producing touch-based applications that may run on very minimal processors and hardware.

One thing of note, to finish, is that the theme of WinRT is asynchronous. Anything that could take a noticeable amount of time (“blocking”) is not available in the WinRT profile; so, get used to asynchronous programming.

Developing Windows Services in Visual Studio

PeterRitchie — Thu, 10 Feb 2011 00:20:00 GMT

Inevitably distributed systems often need a Windows service or two for certain tasks. The creation of a Windows service project and hooking up a project installer to the service is fairly straightforward; so, I’m not going to get into much detail about that stuff.

The stuff that I find that isn’t well understood is how to debug and deploy these services in a development environment.

First off, debugging. A service is just an executable that is started up by the Windows service manager. From a native application standpoint, it has some entry points in it that the service manager looks for and invokes. From a managed standpoint, it’s basically an object that is instantiated and given to System.ServiceProcess.ServiceBase.Run to get the service running. Of course, the actual execution of service must be in the context of the service manager in order for ServiceBase to hook into it an tell it what to do. So, simply running a service EXE will spit out a message about not being able to be run from the command-line or a debugger. Ugh, shot down pretty quickly.

In it’s default state, you can simply install the service, start it, and attach a debugger to it (http://lynk.at/hIZu9w). This, of course, is a pain if what you want to debug doesn’t need to be running in the service manager context. What I’ve been doing with services for quite a while now is use a simple trick of deciding in Main whether to call ServiceBase or call my OnStart override directly. This is based on tips like http://www.codeproject.com/KB/dotnet/DebugWinServices.aspx or http://stackoverflow.com/questions/125964/easier-way-to-start-debugging-a-windows-service-in-c

This little trick is basically adding a couple of lines to Main, and a couple of methods to your ServiceBase-derived class and you’re done! In your ServiceBase class, add two methods: InteractiveStart and InteractiveStop. Each of these methods simply call OnStart and OnStop, respectively. OnStart and OnStop are protected overrides, so the InteractiveStart and InteractiveStop let us gain access to them from outside the class. For example:

public partial class MonitoringService : ServiceBase {
	public MonitoringService() {
		InitializeComponent();
	}
 
	protected override void OnStart(string[] args) {
		// complex multi-threaded code removed for clarity ;)
	}
 
	protected override void OnStop() {
	}
 
	public void InteractiveStart() {
		OnStart(null);
	}
 
	public void InteractiveStop() {
		OnStop();
	}
}

In Main, we simply check Environment.UserInteractive and call InteractiveStart and InteractiveStop. For example:

static void Main() {
	var service = new MonitoringService();
	if (Environment.UserInteractive) {
		service.InteractiveStart();
		Console.WriteLine("service running.  Press enter to exit");
		Console.ReadLine();		
		service.InteractiveStop();
		return;
	}
	ServiceBase.Run(service);
}

Probably a net gain of about seven lines of code.

To get this to work, you have to change your Windows service project’s settings from Windows Application to Console Application. You can now set a breakpoint in your service code and press F5.

This technique requires none of your service code to change, and you can wire off the code with a simple #if defined(…) if you like for Release builds.

Now that life’s easier with the ability to easily debug a Windows service; what about where you want to debug something else that uses the service in a running state? You could simply run the EXE so that it runs in our interactive mode and run the other application so it can interact with it. But, sometimes you actually need to be running it as a service (e.g. you need the app to be running in a certain user context). You can use your deployment project and install the service, type in username and password information and then run your other application. But, having to do this more than a couple of times becomes tedious very fast. Luckily, the Framework’s InstallUtil has you covered there. You may be familiar with InstallUtil’s ability to install and uninstall services from the command line (“InstallUtil MonitoringService1.exe” to install and “InstallUtil /u MonitoringService.exe” to uninstall). But, With the help of ServiceProcessInstaller added by the designer, InstallUtil also let’s you provide a username and password for when your services is configured in the service process installer for ServiceAccount.User. It’s a simple matter of using username and password service parameters. For example:

installutil /username=".\MonitorServiceAccount" /password=”SuperSecretP@55w0rd” MonitoringService.exe

Try “installUtil MyService.exe /?” sometime to see all the options available to you, including password and username, ala Additional Installer Options at http://lynk.at/hNsK1R

Of course, you have to run these from a command line in Administrative mode if you’re using Vista/7/2008 and need to make sure the account has rights to run as a service.

You can even create a batch file to install and start the service. For example:

installutil /username=".\MonitorServiceAccount" /password="SuperSecretP@55w0rd" WindowsService1.exe
net start MonitoringService

Hope this helps!

[EDIT: added the Console.ReadLine mentioned by Stefan]

Unbind a Visual Studio 2010 Solution from Source Code Control

PeterRitchie — Sat, 11 Dec 2010 19:21:51 GMT

I was working with a solution that I got from someone else the other day. When I loaded it up, Visual Studio 2010 informed me that it could not connect to a TFS server at some URL and will open the solution in offline mode (or something to that effect). Of course, I have no access to this TFS server, so, I’m going to get this message every time I open this solution. That’s going to get annoying pretty fast.

So, I had a quick search on the Internet about removing source code control from a Visual Studio 2010. I found all sorts of information about editing the sln file, removing various sections of an sln file and what-not. None of this information can from Microsoft. While in theory doing this would likely do what I wanted, manually editing sln or project files never seems like a good idea. I don’t know if there are side-effects from this that Visual Studio will telescope into some future version of the sln file and end up causing me problems. e.g. Visual Studio 201x is released and I try to upgrade this sln file—what if this future version of VS fails or even worse, corrupts my file.

No, I didn’t want to get chicken blood on my keyboard.

I was somewhat surprised that my search didn’t immediately show something that wasn’t obvious ritualistic magic; so, I went looking in the File menu at the Source Control entry.

In there, when my solution was loaded, I simply selected Change Source Control. As soon as I try to open that dialog VS told me again that it was working offline and wanted to know if I wanted to completely remove the Source Control bindings from my solution. I simply chose yes and all the SCC bindings were removed from my solution.

Saving and reloading the SLN file had no source control offline message, and my keyboard is no dirtier than when I started.

The fact that I couldn’t readily find this with a web search made me think this information would be useful for someone else; so, I’m posting it here in the hopes it helps someone else.

Testing Strategies Involving Async Functions

PeterRitchie — Thu, 04 Nov 2010 16:26:00 GMT

Some things to keep in mind when writing units tests for code that use async methods: You're not trying to test the framework's "awaitability" and you're not trying to test framework methods that are "awaitable". You want to test your code in certain isolation contexts. One context, of course, is independent of asynchronicity--do individual units of code that don't depend on asynchronous invocation "work"... e.g. "Task<string> MyMethodAsync()", you want to have a unit test that make sure this method does what it's supposed to do (one being it returns a "valid" Task<string> object, the other being that individual side-effects occur, if any). Another context is dependent on asynchronicity--do individual units of code that do depend on asynchronous invocation (or depend on something being invoked asynchronously) "work".

It's the second context that seems to be the hardest to grasp for most people to grasp and action. Let's take this example code:

	private async void startButton_Click(object sender, EventArgs e)
	{
		try
		{
			startButton.Enabled = false;
			var webRequest = WebRequest.Create("http://google.ca");
			using (var response = await webRequest.GetResponseAsync())
			using (var stream = response.GetResponseStream())
			{
				if (stream == null) return;
				using (var reader = new StreamReader(stream))
				{
					textBox.Text = await reader.ReadToEndAsync();
				}
			}
		}
		finally
		{
			startButton.Enabled = true;
		}
	}

A couple invariants that we want to test might be that the button is disabled during the asynchronous operations and enabled after the asynchronous operations. It's not immediately obvious what to in order to validate these invariants.

When the compiler encounters the await operator, it actually goes searching for method that can act upon the type of the object being returned by the method used with the await operator. In our first use of await (on GetResponseAsync()) the return type is Task<T> (Task<WebResponse> specifically, but for our example I'll use Task<T>). There's various criteria the computer uses to search for this method, one method is to search for extension methods that match the name and return an "awaiter" type that has the following methods: BeginAwait and EndAwait (more details can be found in the Aync CTP documentation). The compiler doesn't allow us to inject an awaiter type in the normal run-time dependency injection semantics; but it does allow us to implement an awaiter that does support dependency injection at run-time. To do this I would create an ITaskAwaiter interface like this:

	public interface ITaskAwaiter<out T>
	{
		bool BeginAwait(Action continuation);
		T EndAwait();
	}

I then need to create a GetAwaiter(this Task<T>) extension method to return an implementation of this type. The built-in System.Runtime.CompilerServices.TaskAwaiter<T> is public; but, unfortunately it's constructor (and the ability to pass in a Task<T>) object is internal. So, we can't simply wrap the built-in awaiter and delegate to it as a default. We actually have to write an awaiter that mimics what the built-in one does. For example:

	public class TaskAwaiter<T> : ITaskAwaiter<T>
	{
		private readonly Task<T> task;
 
		public TaskAwaiter(Task<T> task)
		{
			this.task = task;
		}
 
		public bool BeginAwait(Action continuation)
		{
			if (task.IsCompleted) return false;
			var synchronizationContext = SynchronizationContext.Current;
			Action<Task> action = theTask => 
			                      	{
			                      		if (synchronizationContext != null)
			                      		{
			                      			synchronizationContext.Post(state => continuation(), null);
			                      		}
			                      		else
			                      		{
			                      			continuation();
			                      		}
			                      	};
 
			task.ContinueWith(action, CancellationToken.None, TaskContinuationOptions.ExecuteSynchronously, TaskScheduler.Current);
			return true;
		}
 
		public T EndAwait()
		{
			return task.Result;
		}
	}

This class is given the Task<T> in question, it implements a BeginWait method that sets up an Action delegate to invoke the continuation given to it in a specific synchronization context, then tells the task to use that new action as it's continuation via a call to ContinueWith. When the task is completed EndAwait will be called and we simply return the result of the task.

Now, in order to add the ability to inject a customer awaiter, we need to provide the GetAwaiter extension method. For example:

	namespace PRI.Extensions
	{
		static class TaskExtensions
		{
			static public ITaskAwaiter<TResult> GetAwaiter<TResult>(this Task<TResult> task)
			{
				return TaskAwaiterFactory<TResult>.CreateTaskAwaiter != null
					? TaskAwaiterFactory<TResult>.CreateTaskAwaiter(task) : new TaskAwaiter<TResult>(task);
			}
		}
	}

This extension method invokes a factory method delegate to create the ITaskAwaiter<T> object (or simply creates our new default awaiter). This factory method looks like this:

	internal static class TaskAwaiterFactory<TResult>
	{
		private static Func<Task<TResult>, ITaskAwaiter<TResult>> createTaskAwaiter = task =>
		                                                                              	{
		                                                                              		Current = new TaskAwaiter<TResult>(task);
		                                                                              		return Current;
		                                                                              	};
		public static Func<Task<TResult>, ITaskAwaiter<TResult>> CreateTaskAwaiter
		{
			get { return createTaskAwaiter; }
			set
			{
				createTaskAwaiter = task =>
				                    	{
				                    		Current = value(task);
				                    		return Current;
				                    	};
			}
		}
		public static ITaskAwaiter<TResult> Current { get; private set; }
	}

We can override this factory method and give a delegate to code that creates another type of ITaskAwaiter<T> implementation. We also decorate the call to that provided delegate with something that keeps track of the Current awaiter (more on that later). Now, we simply need to add a using statement within our namespace for the compiler to user our GetAwaiter when it compiles an await operator and use our factory. For example:

	namespace PRI.MainApplication
	{
		using PRI.Extensions;
 
		//...

We're now all set to inject a new awaiter and be able to isolate our code from the framework and validate some invariants. In our case, we'd like to check to make sure that the button is disabled during invocation of asynchronous operations and enabled when done. So, we can create a spy awaiter that basically does nothing and sends back a fake or dummy object. In our example we actually have two different types of Task<T> objects being awaited; so, we need to create a couple of awaiter spys. One is generic and one is specific. For example:

	public class WebResponseTaskAwaiterSpy : ITaskAwaiter<WebResponse>
	{
		private class FakeWebResponse : WebResponse
		{
			public override Stream GetResponseStream()
			{
				return new MemoryStream(System.Text.Encoding.ASCII.GetBytes("got some text"));
			}
		}
		private readonly MainForm form;
		public bool ButtonEnabledState { get; private set; }
		public WebResponseTaskAwaiterSpy(MainForm form)
		{
			this.form = form;
		}
 
		public bool BeginAwait(Action continuation)
		{
			return false;
		}
 
		public WebResponse EndAwait()
		{
			ButtonEnabledState = form.startButton.Enabled;
			return new FakeWebResponse();
		}
	}
 
	public class TaskAwaiterSpy<T> : ITaskAwaiter<T>
	{
		private readonly MainForm form;
		public bool ButtonEnabledState { get; private set; }
		public TaskAwaiterSpy(MainForm form)
		{
			this.form = form;
		}
 
		public bool BeginAwait(Action continuation)
		{
			return false;
		}
 
		public T EndAwait()
		{
			ButtonEnabledState = form.startButton.Enabled;
			return default(T);
		}
	}

Both classes don't actually execute the task because we know task execution works and we know it works asynchronously--we're not trying to test that. The clases return false from BeginAwait to tell the generated code that the asynchronous code "completed" (we lie). The generated code the calls the EndAwait method to get the result of the so-called asynchronous operation. In both cases, we get the state of the button in EndAwait. In the case of the generic spy, we simply return a dummy--the default value for that type parameter. In the case of the WebResponse spy, we can't send back a dummy because that would be a null value and we'd get NullReferenceEexceptions that would abort our test. So, we create a FakeWebResponse object and send that back that basically creates a canned response stream in memory and sends it back and that will be used in our second await.

Now, we can write a unit test for startButton_Click method. For example:

	MainForm form = CreateForm();
	TaskAwaiterFactory<WebResponse>.CreateTaskAwaiter = task => new WebResponseTaskAwaiterSpy(form);
	TaskAwaiterFactory<String>.CreateTaskAwaiter = task => new TaskAwaiterSpy<String>(form);
 
	form.startButton_Click(null, EventArgs.Empty);
 
	var spy1 = TaskAwaiterFactory<WebResponse>.Current as WebResponseTaskAwaiterSpy;
	var spy2 = TaskAwaiterFactory<String>.Current as TaskAwaiterSpy<String>;
 
	Assert.IsTrue((spy2 != null && !spy2.ButtonEnabledState) && (spy1 != null && !spy1.ButtonEnabledState));
	Assert.IsTrue(form.startButton.Enabled);

This code injects our two types of awaiters (our spies), invokes the method, then checks the spies for the values we're looking for as well as the current state of our button (we assume the current state at this point is the same state immediately after the asynchronous operation completed) to validate our invariants.

Deep Dive on Closure Pitfalls

PeterRitchie — Wed, 03 Nov 2010 13:23:00 GMT

I've blogged about closures in C# and their pitfalls before. I keep seeing problems with closures--more now that lambdas expressions and statements ("lambdas") are becoming more widespread--even with experienced developers. So, I'd thought i'd post about some of the details surrounding where the C# compiler generates closures in the hopes that people will recognize more where they write code that creates a closure and its context.

The C# language spec does not refer specifically to "closures", with regard to capturing state for anonymous methods (including lambdas)--it refers to "outer variables" and "captured outer variables". The captured outer variables for a specific anonymous method are the "closure".

The compiler captures the state of variables within the local scope of the anonymous method in a compiler-generated class. The compiler effectively also captures the declaration scope of the state within the closure. If a variable is declared outside the local scope, that is echoed in the use of that closure. If the variable is declared within the local scope, that is captured in use of the closure. So, an anonymous method that uses state (a variable) declared outside the local scope of the anonymous method, the compiler will generate a closure use that closure so that variable is only instantiated once. For example:

	ICollection<Func<string>> Funcs = new List<Func<string>>(); 
	String[] texts = new String[] { "one", "two", "three" };
	foreach (String text in texts)
	{
		Funcs.Add(delegate { return text; });
	}

We have an anonymous method used within the local scope of the body of the foreach loop. The anonymous method uses the text variable declared outside the scope of the foreach loop. The compiler generates a class to contain the state and the code that uses that state (i.e. the anonymous method)--otherwise the highlighted code--and make use of that generated class. For example:

	Closure closure = new Closure();
	for (int i = 0; i < texts.Length; ++i)
	{
		closure.text = texts[i];
		Funcs.Add(new Func<string>(closure.Func));
	}

The compiler-generated class may look something like this:

	internal class Closure
	{
		public string text;
		public string Func()
		{
			return text;
		}
	}

The compiler instantiates that class outside the generated for loop (because there is no "foreach" loop in .NET IL--what we see here for generated code is a C# view of IL) to match the single declaration of the text variable (as the body of the foreach loop sees it). So, as we can tell from this code, each of these Func delegates are accessing the same variable. So, if we invoked each of those delegate, they would all return "three". While the code looks like each Func delegate is dealing with a different value ("one", then "two", then "three"); it's not and you've probably got a logic error with this code.

So, as I pointed out in a previous post, we get around the problem by bringing the value from the outer scope into the inner scope of the anonymous method. For example:

	foreach (String text in texts)
	{
		var temp = text;
		Funcs.Add(delegate { return temp; });
	}

Now, the compiler needs to generate a class to model this local scope--or the highlighted code. This local scope includes the creation of an object and assigning it a specific value. So, what the compiler generates now would be similar to the following:

	for (int i = 0; i < texts.Length; ++i)
	{
		Closure closure = new Closure();
		closure.temp = texts[i];
		Funcs.Add(new Func<string>(closure.Func));
	}

With a closure class similar to:

	private class Closure
	{
		public string temp;
		public string Func()
		{
			return temp;
		}
	}

As we can see here, the compiler is now instantiating a Closure class once for each iteration of the loop, assigning a specific value to the temp field, then passing a delegate to the Func instance method to a new Func<T> delegate. Now, if we invoke each of our Func delegates we get what we expect "one", then "two", then "three".

I've specifically chosen deferred execution in these examples to separate parallel processing and what's going on. The same issue occurs when we use anonymous delegates with multiple threads. For example:

	var texts = new[] { "one", "two", "three" };
	foreach (var text in texts)
	{
		ThreadPool.QueueUserWorkItem(v => Trace.WriteLine(text));
	}

This anonymous method (as a lambda) would generate code similar to:

	Closure closure = new Closure();
	for (int i = 0; i < texts.Length; ++i)
	{
		closure.text = texts[i];
		ThreadPool.QueueUserWorkItem(new WaitCallBack(closure.WaitCallback));
	}

As you can see, each thread is actually dealing with a single variable (Closure.text). At worst case when each thread runs, it outputs "three"--otherwise it's undefined what you'll see. We fix that the same way we did previously by adding a variable into the local scope:

	foreach (var text in texts)
	{
		var temp = text;
		ThreadPool.QueueUserWorkItem(v => Trace.WriteLine(temp));
	}

This then generates something like this:

	for (int i = 0; i < texts.Length; ++i)
	{
		Closure closure = new Closure();
		closure.temp = texts[i];
		ThreadPool.QueueUserWorkItem(new WaitCallback(closure.WaitCallback));
	}

This means each thread is operating on an independent variable (the Closure.temp field of each instance of Closure that was instantiated).

I hope this makes it more clear what the compiler is doing when you use anonymous methods (including Lambdas).

More on Async Functions

PeterRitchie — Fri, 29 Oct 2010 20:12:00 GMT

In my last post I showed .Net 1.1 and .NET 2.0 code that performed some asychronous operations. I then showed the new syntax with "async" and "await" that did the same thing.

But, I didn't detail what's really going on in the new syntax.

If you want to know more about the details of what's going on, read on. If you just trust me about the previous code, you don't have to read on :)

When the Click handler is executed it basically executes everything up to the first await and returns. This allows the UI to be responsive. The compiler generates some code that takes the Task<T> object that is returned from the async method and waits for it. If there is code that needs to execute after the await, then the compiler effectively generates a continuation. With regard to Task<T>, that's basically a call to Task<T>.ContinueWith. Any other await statements are collected in that continuation and the same thing is done, the continuation is executed up to the first await and returns...

The underlying architecture knows about synchronization contexts and makes sure "synchronous" code within the Click handler is executed on the UI thread. So, as you saw in the example, we didn't have to manually deal with marshaling back to the UI thread via Control.Invoke().

My original example was overly simplistic and didn't include anything to deal with disposal. To a certain extent I've done async functions a disservice because async functions make disposal much, much easier. For example, my .NET 4.0 code would do something like this:

private void button_Click(object sender, EventArgs e)
{
button.Enabled = false;
var webRequest = WebRequest.Create("http://google.ca");
webRequest.BeginGetResponse(asyncResult =>
{
  var response = webRequest.EndGetResponse(asyncResult);
  var stream = response.GetResponseStream();
  if (stream != null)
  {
   var reader = new StreamReader(stream);
   var text = reader.ReadToEnd();
   BeginInvoke((MethodInvoker) (() =>
   {
    textBox.Text = text;
    button.Enabled = true;
    reader.Dispose();
    stream.Dispose();
    ((IDisposable) response).Dispose();
   }));
  }
  else
  {
   ((IDisposable)response).Dispose();
  }
}, null);
}

As you can see, we can't use the using statement because that assumes the block in which using block resides is run asynchronously, so we're forced to manually call Dispose (and in the case of WebResponse, cast to IDisposable first because it explicitly implements IDisposable).

With await, we can write a block of code with bits of it being asynchronous and still use using statements. So, with async functions, our code can now dispose more cleanly; for example:

private async void button1_Click(object sender, EventArgs e)
{
try
{
  button.Enabled = false;
  var webRequest = WebRequest.Create("http://google.ca");
  using (var response = await webRequest.GetResponseAsync())
  using (var stream = response.GetResponseStream())
  {
   if (stream == null) return;
   using (var reader = new StreamReader(stream))
   {
    textBox.Text = await reader.ReadToEndAsync();
   }
  }
}
finally
{
  button.Enabled = true;
}
}

C# 3 and lambda expressions make the act of using asynchronous methods much easier than .NET 1.x; the next version of C# with async functions takes it that final step forward.

A New Asynchronicity Awaits You

PeterRitchie — Thu, 28 Oct 2010 05:52:00 GMT

The languages team at Microsoft have just announced that both VB and C# are giving first-class citizenship to asynchronous operations.

At long last we can cleanly program for asynchronous operations without cluttering up the code with imperative artefacts relating to how the asynchronous operation is being performed.

Let’s have a quick look at how we had you might perform an asynchronous operation in .NET 1.x:

	byte[] readbuffer = new byte[1024];
 
	public void Button1_Click()
	{
		WebRequest webRequest = WebRequest.Create("http://msdn.com");
		webRequest.BeginGetResponse(new AsyncCallback(BeginGetResponseCallback), webRequest);
	}
 
	private void BeginGetResponseCallback(IAsyncResult asyncResult)
	{
		WebRequest webRequest = (WebRequest)asyncResult.AsyncState;
		WebResponse webResponse = webRequest.EndGetResponse(asyncResult);
 
		Stream stream = webResponse.GetResponseStream();
		stream.BeginRead(readbuffer, 0, readbuffer.Length, new AsyncCallback(BeginReadCallback), stream);
	}
 
	private delegate void TakeString(string text);
 
	private void BeginReadCallback(IAsyncResult asyncResult)
	{
		Stream stream = (Stream)asyncResult.AsyncState;
		int read = stream.EndRead(asyncResult);
		this.Invoke(new TakeString(SetTextBoxText), new object[] {Encoding.ASCII.GetString(readbuffer, 0, read);
	}
 
	private void SetTextBoxText(String text)
	{
		textBox.Text = text;
	}

In .NET 1.x, in order to go get data from a web page in response to a button click, we had to create two methods that acted as callback for the asynchronous get web request and asynchronous read response. As well, we had to create a method to marshal data back to the UI thread via Control.Invoke so that modification of controls (setting of Text property) is done only on the UI thread.

In .NET 2.0 this got much easier. We could use anonymous methods and get rid of our callback methods altogether to get something like this:

	public void Button1_Click(object snder, EventArgs e)
	{
		WebRequest webRequest = WebRequest.Create("http://msdn.com");
		webRequest.BeginGetResponse(delegate(IAsyncResult asyncResult)
		{
			WebRequest webRequest1 = (WebRequest) asyncResult.AsyncState;
			WebResponse webResponse = webRequest1.EndGetResponse(asyncResult);
 
			Stream stream = webResponse.GetResponseStream();
			stream.BeginRead(readbuffer, 0, readbuffer.Length, asyncResult1 =>
         {
             Stream stream1 =
                 (Stream) asyncResult1.AsyncState;
             int read =
                 stream1.EndRead(asyncResult1);
             this.Invoke(
                 new TakeString(
                     delegate(string text)
                         {
                             textBox.Text =
                              Encoding.ASCII.GetString(
                              stream1, 0, read);
                         }));
         }, stream);
		}, webRequest);
	}

But, the details of anonymous methods and closures really make this code complex and hard to understand.

In some future version of VB and C#, it will become even easier with the new await/Await and async/Async keywords. In conjunction with the Task Parallel Library, you will now capable of declaring an asynchronous method (“asynch”) and inform the compiler which part of the method is asynchronous (“await”). For example:

	private async void Button1_Click(object sender, EventArgs e)
	{
		var request = WebRequest.Create("http://msdn.com");
		var response = await request.GetResponseAsync();
		var reader = new StreamReader(response.GetResponseStream());
 
		textBox.Text = await
		reader.ReadToEndAsync();
	}

Much, much simpler and easier to read. “async” tells the compiler that a method uses the await keyword. The await keyword tells the compiler that the method being called returns some sort of Task<> object, the type parameter signifying the left-hand-side type of the method call and that the method uses the Task Parallel library to spin up that Task<> object to perform the operation asynchronously.

As you might imagine, there’s a plethora of new methods in the BCL that will now return a Task<> object (and have the postfix of “Async”) that can be used with the await keyword.

Visual Studio 2010, Enhance your Jedi Skillz

PeterRitchie — Tue, 15 Jun 2010 23:21:00 GMT

I’ve blogged about becoming a Jedi in Visual Studio 2008 before. Being a Jedi in Visual Studio means you focus more on adding value to the software you’re working with and less on the process of the IDE you’re doing your work in.

Visual Studio 2010 has some great features to allow you to do just that. So much so, in fact, that I can’t possibly do them justice in a single post. I’ll start with a few here and continue with a few posts on ways to get Visual Studio 2010 to let you write software faster.

All of what I’ve posted about VS 2010 still applies, like Auto Hide Solution Explorer, Toolbox, etc, single custom toolbar, etc.

Use multiple monitors. VS 2010 has inherent support for multiple monitors. You could previously get VS to display on multiple monitors; but now the tool windows in VS 2010 can be moved outside the main VS window, and dragged onto another monitor. This is particularly handy (in my opinion) with debugging. You can have your watch window on one monitor, the main editor on another monitor ( and if you’re lucky enough to have a 3rd monitor) the application you’re debugging on another monitor.

Use Navigate To. I’ve been a user of Resharper for many years. One of it’s features is the ability to Go To Declaration—which allows you to navigate to the declaration of a type (if it’s in your solution, metadata if it isn’t). Now Visual Studio 2010 has a very similar feature: Navigate To. This feature can be invoked with Ctrl+, and will pre-populate the form with the type at the caret. The form lists all the project files and types within the solution that match the text, including config files, CS files, classes, interfaces, etc. Previously you had to to perform a Find In Files and wade through the false-positives to go right to the declaration of a type. For example, if I press Ctrl+, and type “Main”, I’ll see the following Navigate To form:

I can then simply press return to go to the declaration of the Main method.

If you are a TDD practitioner, switch to Suggestion Mode. Pressing Ctrl+Alt+Space toggles between Suggestion mode and Completion mode. Completion mode is the IntelliSense mode that was in version 2008 and prior. 2010 has a new Suggestion mode that doesn’t auto-complete types when you press space, period, semicolon, etc. They same types show up in the IntelliSense drop-down; but you can still auto-complete the type it suggested by pressing the down arrow then space, period, semicolon, etc. For example, if I’m typing test code to test a type named Text, in Completion mode it may auto-complete with “TextBox” when I press space. In Suggestion mode it will simply accept what I typed when I press space and not auto complete. What’s nice in 2010 is that IntelliSense now knows about “Text” as a type even though I haven’t declared it, so when I continue and type “text = new T”, “Text” now shows up in the IntelliSense drop-down. I can have it complete it with “Text” by pressing down-arrow and continue typing.

Use Call Hierarchy to see what code is calling a particular method. Call Hierarchy is new feature in Visual Studio 2010 that shows you in a tree structure what methods call a particular method as well as what methods a method calls. For example, if I right-click a call to the UnityContainer constructor and select View Call Hierarchy, the Call Hierarchy form is displayed which shows what calls UnityContainer and what UnityContainer calls. For each item in a node, you can expand to see what that method calls and what method calls it. For example, InitializeContainer calls the UnityContainer constructor, if I expand the InitializeContainer entry in the tree I can see what it calls and where it is called from. If I expand it’s Calls From ‘InitializeContainer’ I can see what other methods InitializeContainer calls. For example:

Since I last blogged on improving your Visual Studio skills, Scott Cate and Steve Andrews have taken up the Visual Studio Tips and Tricks torch from Sara Ford and are hosting vstricks.com and vstips.com, respectively.

Using the dynamic Keyword in C# to Improve Object Orientation – A Follow-up

PeterRitchie — Thu, 27 May 2010 16:37:30 GMT

Based on some feedback, some clarification is warranted with regard to my previous post titled “Using the dynamic Keyword in C# to Improve Object Orientation”.

As Jarek Kowalski correctly pointed out, the example code that I provided could have used the Visitor pattern instead to get the same result. My impetus for using the dynamic keyword the way I did was slightly different from how I described my example—which was meant to be easier to read.

I think it’s worthwhile describing the Visitor Pattern. The Visitor pattern is a pattern used to separate the responsibility of an algorithm from the class that the algorithm should operate upon. Essentially, a Visitor class is created to contain the algorithm that would be invoked through its Visit method and the classes that the algorithm operates upon some sort of method that accepts a reference to a Visitor object (which would simply invoke the Visit method). The example that Jarek provided was very similar to:

 	public interface IShapeVisitor

 		void Visit(Circle square);

 		void Visit(Rectangle square);

 	public class Circle : Shape

 		public override void Accept(IShapeVisitor shapeVisitor)

 			shapeVisitor.Visit(this);

 		//...

And an implementation of IShapeVisitor would be provided like this:

 	public  class  DrawingVisitor  : IShapeVisitor

 		public  void  Visit(Circle circle)

 			// draw circle here

 		public  void  Visit(Rectangle rectangle)

 			// draw rectangle here

In our case, the bodies of Visit methods would call a Draw method contained in another class like a View implementation. For example:

 	public  class  DrawingVisitor  : IShapeVisitor

 		private  IView  view;

 		public  DrawingVisitor(IView  view)

 			this.view = view;

 		public  void  Visit(Circle  circle)

 			view.Draw(circle);

 		public  void  Visit(Rectangle  rectangle)

 			view.Draw(rectangle);

This of course, adds another level of indirection that allows our target (a Shape implementation) to invoke a virtual method that ends up delegating to the Visitor. An implementation of Double Dispatch. This allows objects whose type isn’t known until run-time to properly invoke overloaded methods of another type.

As you can see, this is very powerful but adds a bit more complexity.

Now, in my original scenario (before I wrote the example for the post), I was dealing with invoking particular series of static overloads of System.Convert. Because I was dealing both with a class outside of my control and dealing with static overloads, the Visitor pattern could not be applied.

Using the dynamic Keyword in C# to Improve Object-Orientation

PeterRitchie — Mon, 24 May 2010 16:18:00 GMT

With polymorphism, object-oriented languages allow "...different data types to be handled using a uniform interface". Ad-hoc polymorphism is when you declare multiple methods of the same name but differ by the type of an argument. For example:

 private  static  void  Draw(Circle  circle)

 	//...

 private  static  void  Draw(Square  square)

 	//...

These are usually referred to as method overloads or method overloading. Which Draw method that gets invoked would be decided upon at compile-time based on the type of the parameter passed to it.

This is great, there are many situations where this is useful; but what about situations where you don't know the exact type of the object until run-time? Many programmers instinctly lean towards using the GetType method available on all objects in .NET and perform a manual type comparison. For example:

 public  void  ProcessShape(Shape  shape)

 	Type  type = shape.GetType();

 	if  (type == typeof (Square ))

 		Draw((Square )shape);

 	else  if  (type == typeof (Circle ))

 		Draw((Circle )shape);

 	//...

There are other variants of this that make it a little harder to detect (and often gets around static code analysis warnings):

 public  void  ProcessShape(Shape  shape)

 	Square  square = shape as  Square ;

 	if  (square != null )

 		Draw(square);

 		return ;

 	Circle  circle = shape as  Circle ;

 	if  (circle != null )

 		Draw(circle);

 		return ;

 	//...

This code is a problem because we lose the benefits of C#s polymorphism (this code is coupled to Circle and Shape, if either is removed, this code would have to be changed and re-tested. If another Shape derivative were added, the code would again have to be changed and re-tested). There are multiple circumstances where this might happen over and above the so-far specious examples above. For example, deserializing an object from a Stream often results in not knowing the exact type of the object that has been deserialized. This is common with type hierarchies. For example:

 private  static  Shape  DeserializeShape(Stream  stream)

 	IFormatter  formatter = new  BinaryFormatter ();

 	return  formatter.Deserialize(stream) as  Shape ;

With the above code, we've re-hydrated either a Circle of a Square object into a Shape object (and ignored the appropriate checks to make sure a Shape object was loaded). We're now unable to use all the other polymorphism features of C#: method overloading won't work, C# will always only use the overload with the Shape argument and generics suffers from the same problem, it will always resolve to the type parameter of type Shape. For example, if we try to call one of our two Draw methods with our re-hydrated Shape object:

 			Draw(shape);

...we simply get a CS1502 warning because there is now Draw(Shape shape) method.

When programmers simply compare types, they're admitting that polymorphism has been circumvented and they have to resort to introducing the Conditional Complexity Code Smell or the Switch Statement Code Smell (C# doesn't support Type in a switch statement, so some programmers go ahead and create a integer type code--or even worse, use String in the switch statement)

So, what's a programmer to do to keep our code object-oriented and use polymorphism?

Well, if you read the title of this post, you've probably figured it out: use the dynamic keyword. The dynamic keyword bypasses static (compile-time) type checking and performs the type-checking dynamically (at run-time). So, for us to use our Draw overloads with our Shape object, we simply assign it to a dynamic object and use that object instead. For example:

 			Shape  shape = DeserializeShape(stream);

 			dynamic  dynShape = shape;

 			Draw(dynShape);

Now the runtime will check to see the actual type of dynShape object and invoke the corresponding Draw overload.

In our example, we've used inheritance to derive our two shapes from a single base. But, given our current use of Circle and Square objects, we don't need them to inherit from the same base class. We could simply view these shapes as an Object when we want to use dynamic. For example:

 private  static  Object  DeserializeShape(Stream  stream)

 	IFormatter  formatter = new  BinaryFormatter ();

 	return  formatter.Deserialize(stream);

 			//...

 			dynamic  shape = DeserializeShape(stream);

 			Draw(shape);

With the above, we’d have the same result without deriving both Circle and Square from Shape.

Layers in Visual Studio 2010

PeterRitchie — Sat, 08 May 2010 20:32:57 GMT

Visual Studio 2010 has a new featured called Layer Diagrams. In the Ultimate edition you can create layer diagrams that model the logical layers in your software system.

What is a Layer?

I’m glad you asked what a layer is. A layer is a logical grouping of types with similar external assembly dependencies. Dependencies between layers occur only in one direction from a higher-level layer to a lower-level layer. I.e. a higher-level layer can use types in a lower-level layer, but not vice versa. A canonical example is the Data Access Layer (or DAL). The DAL contains all the types that directly use types responsible for data-access.

Using the Layer Diagram

Within the Layer Diagram user interface, it can be unclear what a layer is. The Layer Diagram is really just a dependency diagram that allows creation of a logical layer that can be linked to physical artefacts.

Creating a layer diagram in Visual Studio Ultimate is easy, select New Diagram from the Architecture menu and the Add New Diagram dialog will be displayed.

In the Add New Diagram dialog, select the Layer Diagram entry and enter the name of the layer diagram file.

If your solution does not currently contain a Modeling project, one will be created automatically when you press OK as indicated by “Create a new modelling project…” in the Add to modeling project combo box.

Once the modeling project is created and the new layer diagram is added to that project, you will be presented with the Layer Diagram design surface. You have many options to creating a layer in the diagram, you can use entries in the toolbox to create layers, dependencies, and comments:

Or, you can drag artefacts from the Solution Explorer or the Architecture Explorer to create layers in various ways. Normal dragging of a single artefact onto the design service creates a new layer with a one-to-one mapping (a “link” in Layer Diagrams) from layer to artefact. This is especially useful for projects from the solution explorer. It’s not uncommon for logical layers to be implemented as tiers (or physical assemblies). Or, you can create a one-to-many mapping from layer to artefacts by normal dragging of artefacts onto the design surface. This is useful when you’re modeling layers but haven’t modeled them as tiers and multiple layers exist in a single assembly or project. Via the Architecture Explorer and the Solution Explorer you can link assemblies (projects), types, namespaces, methods, properties, fields, and files to a layer.

Regardless of how the layer was created on the diagram, you can add links to it by dragging them from the Solution Explorer or Architecture Explorer onto a layer to create a link to it. Oddly, although you can link to classes and namespaces, you can’t drag classes or namespaces from the Class View onto the Layer Diagram design surface.

The following diagram was created by dragging three projects from the Solution Explorer to the design surface and invoking Generate Dependencies on all three of the created layers:

Once a layer model has been defined, you can then ask Visual Studio to validate the solution against that logical architecture. Within the layer diagram you can have a one-to-one mapping of layer to links, or a one-to-many mapping. i.e. a layer can consist of one or more links (at least in terms of validation—you can have zero links in a layer if you want; but then VS has no information with which to validate anything). If, for whatever reason, the current solution doesn’t follow the logical architecture, an error will result. You can also configure the solution to automatically validate the solution against the layer diagram during a build. This build-time validate can occur in both the Ultimate and Premiums editions of Visual Studio 2010.

Why is it really just a Dependency Diagram?

Another really good question! The Layer Diagram is really just a dependency diagram because there’s no inherent recognition for layer levels. Because there’s no way of defining a level, there’s no way to group layers within a specific level. Layer diagrams also allow bi-directional dependencies; a definite no-no with two layers at different levels. It’s also really difficult to really enforce only a particular layer having references to particular external references. You can declare forbidden namespaces for a particular layer, but it’s a cumbersome method of enforcing references to particular components only occur in a certain layer. For example, if I had a DAL and I wanted to make sure only data-access occurred in that layer, I could add “System.Data” to the Forbidden Namespace Dependencies for all layers expect the DAL and incur an error with Layer Validation should any types from System.Data be used in any of those Layers. But, things like DataGridView in Windows Forms are used primarily with types in System.Data so it becomes difficult to manage these dependencies.