Peter Ritchie's MVP Blog : .NET 2.0

DevTeach 2009 Vancouver

PeterRitchie — Thu, 26 Mar 2009 16:33:00 GMT

The schedule for DevTeach 2009 Vancouver has been announced (http://www.devteach.com/). There’s lots of great software development sessions from some of the leaders in our industry.

If you’re planning on improving yourself, this is the conference to go to. Not only can you attend excellent sessions; but you can hob-knob with the presenters and pick their brains.

If you have a friend or co-worker who’s interested, there’s a limited-time two-for-one offer for an even better price: http://www.devteach.com/Register.aspx

ITSWITCH #1: Answer

PeterRitchie — Mon, 28 Jul 2008 19:00:00 GMT

Last post I detailed some code that may or may not have something wrong in it. If you thought InitializeOne and IntializeTwo are semantically identical (e.g. they differ only by performance), you'd be wrong.

If you simply ran the code, you'd be able to guess where the problem is. To understand what's causing the problem. Let's look at how C# effectively implements the two loops.

InitializeOne is essentially equivalent to

        private class PrivateDelegateHelper

            public String Value { get; set; }

            public void Method()

                TestClass.ProcessText(Value);

        public void InitializeThree(String[] strings)

            delegates = new List<MethodInvoker>(strings.Length);

            MethodInvoker cachedAnonymousDelegate = null;

            PrivateDelegateHelper privateDelegateHelper = new PrivateDelegateHelper();

            String[] copyOfStrings = strings;

            for(int i = 0; i < copyOfStrings.Length; ++i)

                privateDelegateHelper.Value = copyOfStrings;

                if (cachedAnonymousDelegate == null)

                    cachedAnonymousDelegate = new MethodInvoker(privateDelegateHelper.Method);

                delegates.Add(cachedAnonymousDelegate);

Now it's obvious, right?

For those you don't want to read all the code, the problem is that only one PrivateDelegateHelper object is instantiated and its value property is set in each iteration of the loop. Because the delegates aren't run until sometime after the loop, they're all run with the last value of the string array as their argument.

The technical term for what we've implemented here is a closure. If you're using Resharper 4.x, you would have noticed a warning "Access to modified closure":

...which is attempting to tell you that the closure (the delegate and cached bound variables) has changed (in this case one of the bound variables has changed between the creation of a closure and another and out expected output is effected).

By the way, you can get the same thing with C# 3+ with lambdas (i.e. you can also write closures with lambdas):

        public void InitializeOne(String[] strings)

            delegates = new List<MethodInvoker>(strings.Length);

            for (int i = 0; i < strings.Length; ++i)

                String value = strings;

                delegates.Add(() => ProcessText(value));

        public void InitializeTwo(String[] strings)

            delegates = new List<MethodInvoker>(strings.Length);

            foreach(String value in strings)

                delegates.Add(() => ProcessText(value));

ITSWITCH: #1

PeterRitchie — Fri, 25 Jul 2008 18:58:00 GMT

Is There Something Wrong In This Code Here

UPDATE: as several readers pointed out there was compile error in the code what was being displayed. The line "String value = string" was showing up as "String value = string". I'm not sure why; but throwing some spaces between the i and the square brackets seems to have solved it.

In the following class, is there something wrong with either InitializeOne or InitializeTwo (hint, it has nothing to do with compile errors and I'm ignoring performance differences)?

    public class TestClass

        private List<MethodInvoker> delegates;

        private static void ProcessText(String text)

            Trace.WriteLine(text);

        public void Run()

            foreach(MethodInvoker invoker in delegates)

                invoker();

        public void InitializeOne(String[] strings)

            delegates = new List<MethodInvoker>(strings.Length);

            for (int i = 0; i < strings.Length; ++i)

                String value = strings[ i ];

                delegates.Add(delegate { ProcessText(value); });

        public void InitializeTwo(String[] strings)

            delegates = new List<MethodInvoker>(strings.Length);

            foreach(String value in strings)

                delegates.Add(delegate { ProcessText(value); });

Answer on Monday.

Formatting/parsing for a specific culture redux

PeterRitchie — Tue, 22 Jan 2008 23:35:00 GMT

In recent blog post I detailed how creating a culture via the CultureInfo constructor could actually create a user-overridden culture--which could be completely different than the culture that you've requested by name. Fortunately there's a way of overriding the user override (apologies for overloading "override") by supplying the boolean value "false" in a CultureInfo overload.

As Greg Beech commented, there's another method to create a culture--System.Globalization.CultureInfo.CreateSpecificCulture. This sounds like it does exactly what you might expect and creates a "specific" culture. Unfortunately, this method too violates the principle of least astonishment and creates a culture that uses the user-overridden values when the culture name matches that of the user's current culture.

CreateSpecificCulture does not, as far as I can tell, have an overload/alternative to you allow to to force a "specific" culture, so the problem is much worse with CreateSpecificCulture. I've gone ahead and logged a bug for it on Connect: https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=321241

In case you're wondering why this is more serious, consider the following following block of code that formats a date value as text:

System.Globalization.CultureInfo ci;

String text;

ci = System.Globalization.CultureInfo.CreateSpecificCulture("en-us");

text = String.Format(ci, "{0:d}", DateTime.Now);

And that text is transmitted to another application (or another session of the same application, e.g. serialization), potentially in another locale, to be parsed with the following block of code:

System.Globalization.CultureInfo ci;

ci = System.Globalization.CultureInfo.CreateSpecificCulture("en-us");

String text = ReadRecord();

DateTime now = DateTime.Parse(text, ci);

If either (or) the user's current culture is set to "English (United States)" and they've overridden the currency format the short date (say from "M/d/yyyy" to "d/M/yyyy") will randomly result in the wrong date.

Using Exceptions For Normal Logic Flow

PeterRitchie — Fri, 12 Oct 2007 17:13:00 GMT

The generally accepted wisdom is that you shouldn't use Exceptions for normal logic flow. Normal logic flow is a bit subjective; but anything that must happen at least once in all known scenarios is normal logic flow.

Enter XML Serialization in the framework. The framework actually dynamically creates types that perform the actual serialization of a given type and caches that assembly. The next time that type needs to be serialized it reuses that generated type, reflection is minimized and things happen pretty quickly.

Here's the rub. The framework decides that it must generate the type when Assembly.Load generates an exception. Something like:

        Assembly assembly = null;
        try
        {
            assembly = Assembly.Load(assemblyName);
        }
        catch (Exception)
        {
            assembly = GenerateTempAssembly();
        }

Which means exceptions are being used for normal logic flow, because when the code is first run the exception is thrown. Now, there's some folks out there (including folks at Microsoft) that will say "what's wrong with that, it gets the job done doesn't it?". Yes, it does get the job done. The problem is there's at least once that Assembly.Load is going to generate an exception (if you modify the type being serialized it will happen again). Again, some may say, "So what?". Well, in Visual Studio there is a BindingFailure Managed Debugging Assistant (MDA) that sits there looking for Assembly.Load failures. This will get kicked off and force a break the first time you try to serialize your type and you're debugging. When you're using XmlSerializer to serialize many different types and you turn on this MDA (after all, it's there and does do some good) then you'll spend a whole bunch of time wading through the BindingFailure MDA.

This has been occurring since XmlSerializer was created. I say "including folks at Microsoft", because up until a couple of days ago I thought someone was on this problem (although I can't find my sources anymore) at Microsoft. But, the problem still occurs in Ocras Beta 2; which means in the past 3-4 years no one has done a thing about this problem. So, I logged a bug about it on Connect (https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=304095) and it was quickly closed as "By Design".

Keep in mind, the "fix" were talking about is simply checking to see if a file exists before trying to load it, something like:

        Assembly assembly;
        try
        {
            Debug.Assert(assemblyName.CodeBase.StartsWith("file:///", true, System.Globalization.CultureInfo.InvariantCulture));
 
            String path = assemblyName.CodeBase.Substring(8).Replace('/', '\\');
            if (!System.IO.File.Exists(path))
            {
                assembly = GenerateTempAssembly();
            }
            else
            {
                assembly = Assembly.Load(assemblyName);
            }
        }
        catch (Exception)
        {
            assembly = GenerateTempAssembly();
        }

...not too tricky...

Who's Referencing Whom?

PeterRitchie — Fri, 12 Oct 2007 15:15:00 GMT

When developing any sort of application, debugging in inevitable. Sometimes, part of that debugging means trying to figure out why objects haven't been collected and therefore figuring out what object is referencing the object that has yet to be collected.

There's many reasons why you'd want to find out what object is referencing, like suspected memory "leaks".

With Visual Studio (and MDbg) you can use a tool called SOS (or Son Of Strike). This is included in the .NET installation. To use SOS you first need to enable unmanaged debugging in your project (Project\Properties, Debug tab, check "Enable unmanaged code debugging" in the "Enable Debuggers" section). Once unmanaged debugging is enabled you can then debug your application. To use SOS once debugging, you need to load the extension (every time a new debugging session is started). Once a breakpoint has been hit, open the Immediate Window and type .load sos which should result in the following:

.load sos
extension C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\sos.dll loaded

With SOS loaded, you can find out if any objects of a particular type are currently in memory with the "dumpheap -type" command, for example:

!dumpheap -type NamespaceName.TypeName
Address       MT     Size
012b2db8 009159c4      328
total 1 objects
Statistics:
      MT    Count    TotalSize Class Name
009159c4        1          328 NamespaceName.TypeName
Total 1 objects

This lists all objects of the requested type, their address, their MethodTable (MT), and the count of each object per MethodTable.

Once you have the object's address you can then find out what objects are referencing that particular instance. This is done with the gcroot command:

!gcroot 012b2db8
Note: Roots found on stacks may be false positives. Run "!help gcroot" for
more info.
Error during command: warning! Extension is using a feature which Visual Studio does not implement.

Scan Thread 4756 OSTHread 1294
ESP:12f0dc:Root:012b2db8(WindowsApplication1.Form1)->
012bb104(WindowsApplication1.Form2)->
012bb254(System.Collections.Generic.List`1[[NamespaceName.TypeName, WindowsApplication1]])->
012bc178(System.Object[])->
012bc16c(NamespaceName.TypeName)
Scan Thread 3496 OSTHread da8

In this particular example, the above tells us that our object (012bc16c(NamespaceName.TypeName)) is referenced by an Object array (012bc178(System.Object[])), which is referenced by a List<T> object (012bb254(System.Collections.Generic.List`1[[NamespaceName.TypeName, WindowsApplication1]])), which is referenced by Form2 (012bb104(WindowsApplication1.Form2)), which is referenced by a Form1 object(ESP:12f0dc:Root:012b2db8(WindowsApplication1.Form1)).

Thread.Abort is a Sign of a Poorly Designed Program

PeterRitchie — Wed, 22 Aug 2007 19:13:00 GMT

Continuing the theme of Thead.Sleep is a sign of a poorly designed program, I've been meaning to provide similar detail on Thread.Abort and not just allude to it in other posts like 'System.Threading.Thread.Suspend() is obsolete: 'Thread.Suspend has been deprecated....

Many of the concepts I've discussed regarding Thread.Suspend also apply to Thread.Abort, and in much the same way that the ability to terminate a thread has existed for so long that the concept has remained ubiquitous when dealing with threads and it just keeps getting implemented without thought. Thread.Abort is far more unsafe than Thread.Suspend; but, unfortunately Thread.Abort has yet to be deprecated.

Unlike Thread.Suspend, Thread.Abort stops your thread and it can't continue afterwards. In the same vein as Thread.Suspend, Thread.Abort doesn't know anything about your thread, and if your thread isn't in a try block that handles ThreadAbortException, it just stops it where it is, which may be in the middle of updating a complex invariant. This means the thread can't continue where the left off like Thread.Suspend and Thread.Resume and can never get that invariant out of that corrupt state (and nothing else can, it has no idea where that thread left off).

Thread.Abort isn't a full-blown terminate; it does cause all finally blocks that it knows about to execute before your thread is stopped and won't terminate your thread while it's in a catch or finally block.

There's several concepts that go along with Thread.Abort. One I've already mentioned, the ThreadAbortException exception. Catching ThreadAbortException is conceptually cooperative thread termination. My issue with catching ThreadAbortException is that you're essentially using exceptions for normal flow of logic; which I don't agree with. The other concept is critical regions. You might think that wrapping critical code with Thread.BeginCritcalRegion and Thread.EndCriticalRegion will mean your thread won't be aborted while it's executing that code. Unfortunately, the .NET Framework doesn't really us Begin/EndCriticalRegion for anything. But, that's also not the intention, the intention is to signify to the runtime (currently only SQL Server vNext, I believe, uses that information) that should a Thread.Abort be called on that thread, or an unhandled exception occur, corruption has occurred and it should simply unload the thread's AppDomain.

But, the concept of interrupting the thread in the middle of something is really why you should never use Thread.Abort. It's one thing to interrupt you own code, but Thread.Abort actually as the potential to interrupt lock statements and cause locks not to be unlocked. As Joe Duffy points out, there's a check when running code on the x86 that makes sure Thread.Abort can't interrupt between a call to Monitor.Enter when it is adjacent to a protected region. But, on current 64-bit JITs this check does not exist and a thread can be aborted between the call to Monitor.Enter and the start of the protected region.

Unfortunately, the x86 is not entirely immune. In the C# compiler, when optimizations are turned off, it emits no-ops in various places. I can't really confirm why, but the prevalent theory is to allow the Visual Debugger to put breakpoints on source code that otherwise wouldn't have corresponding IL. (like the start brace, for example). Unfortunately there's a bug in the generation of that IL that means Thread.Abort can cause locks not to be unreleased.

Eric Lippert recently provided some of the detail of the bug on his blog. But, essentially, this is what the C# compiler generates for IL code for the lock statement:
            call void [mscorlib]System.Threading.Monitor::Enter(object)
Hole:       nop
StartTry:   nop
            ldstr "in lock"
            call void [mscorlib]System.Console::WriteLine(string)
            nop
            nop
            leave.s EndFinally
EndTry:     ldloc.0
            call void [mscorlib]System.Threading.Monitor::Exit(object)
            nop
            endfinally
EndFinally: nop
            ret
    .try StartTry to EntTry finally handler EndTry to EndFinally

Which is emitted (from both .NET 2.0 and .NET 3.5 Beta 2) for the compilation of the following:

lock (locker)

{

Console.WriteLine("in lock");

}

This means the lock is acquired (call to Monitor.Enter) and a nop (at Hole) is executed before the start of the protected region (try block). This means there's an instruction after the acquisition of the lock that isn't in the try block. It's at that point if Thread.Abort were called, the finally block would never get executed and the lock would not be released. Likely this would result in deadlocks because nothing can ever release that lock now.

If you never use Thread.Abort, this issue doesn't affect you. But, it really just provides another good reason why people keep saying you should never use Thread.Abort.

In 'System.Threading.Thread.Suspend() is obsolete: 'Thread.Suspend has been deprecated.... I provide an example of cooperatively terminating a thread, if you're interested in terminating your thread without using Thread.Abort.

Like catching Exception, there is an instance when Thread.Abort can be safely used: when you're terminating your application. If you're terminating your application and you know you have a foreground thread running and it's not responding, Thread.Abort can safely be used to ensure your application terminates bcause you're shutting down and not using any existing invariants or locks.

Performance Implications of try/catch/finally, Part Two

PeterRitchie — Fri, 13 Jul 2007 02:30:00 GMT

In a previous blog entry Performance Implications of try/catch/finally I outlined that the conventional wisdom that there are no performance implications to try blocks unless an exception is thrown is false. I have some clarifications and details to add.

My original tests used academic sample code like this:

public int Method( )

{

int i = 10;

try

{

i = 20;

i = 30;

Thread.Sleep(i);

}

finally

{

Thread.Sleep(i);

}

return i;

}

As written, the optimized instructions generated by the JIT (Just-In-Time compiler) was showing that no optimizations were occurring in the try block in that the only obvious optimization--the assignment of 20 to i--wasn't optimized away.

But, I've loaded-the-deck, so to speak. Let's consider what exception handling needs to be able to do. In this case, I've explicitly said I need to do something in the event of an exception in the block of code I've wrapped with try{}. The JIT takes me at my word and assumes every line in that block could throw an exception. In order for the finally block to get accurate values for the current state--in the event of that exception--it can't optimize any variables that are used in or past the finally block. In this case 10 must first be assigned to i, then 20 must be assigned to i then 30 must be assigned to i in the sequence specified in the source code; because if those assignments threw an exception, the finally block would need one of the values: 10, 20, or 30.

As you might be able to tell, I've loaded-the-deck because I don't have much outside the try block and everything in the try block is used outside of it as well. I haven't given the JIT much to work with. So, the "no optimizations are performed in try blocks" isn't entirely accurate. Obviously, in this case, the assignment of 10 to i wasn't optimized as well. But, the JIT is smart enough to attempt to optimize within the try block. If we change the sample slightly to this:

public int Method( )

{

int i = 10;

try

{

int c = 2;

c = 3;

i = 20;

i = 30;

Thread.Sleep(c);

}

finally

{

Thread.Sleep(i);

}

return i;

}

...the JIT knows that c is not used outside the try block and proceeds to optimize it away and converts the Sleep statement into Sleep(3). Essentially it's if I had written:

public int Method( )

{

int i = 10;

try

{

i = 20;

i = 30;

Thread.Sleep(3);

}

finally

{

Thread.Sleep(i);

}

return i;

}

Okay, things aren't that dire; at least the JIT is doing what it can within the try block. But what about outside the try block; how drastic are the lack of optimizations there? Again, the JIT is smart enough to do what it can. If we change our original sample slightly again:

public int Method( )

{

int i = 10;

int b = 9;

b = 7;

try

{

i = 20;

i = 30;

Thread.Sleep(i);

}

finally

{

Thread.Sleep(i);

}

return b;

}

We see that the JIT is smart enough to attempt to optimize what's outside the try block, and essentially results in this:

public int Method( )

{

int i = 10;

int b = 7;

try

{

i = 20;

i = 30;

Thread.Sleep(i);

}

finally

{

Thread.Sleep(i);

}

return b;

}

Better, it's coalesced the writes (7 and 9) to b into a single write (7); but for some reason it still forces the assignment of 7 to b before the try block. There's nothing that could see b until the return, so it should be able to optimize it a bit further to this:

public int Method( )

{

int i = 10;

try

{

i = 20;

i = 30;

Thread.Sleep(i);

}

finally

{

Thread.Sleep(i);

}

return 7;

}

So, while my previous statement wasn't entirely accurate, I think what the JIT can optimize is neither better nor worse than "no optimizations are performed in try blocks", just different. You should still pay close attention to what you're doing in and around try blocks.

Also, an apples-to-oranges comparison; but methods with try/catch/finally blocks won't be inlined.

Having described what the JIT appears to be doing, and the implication that optimizations to variables used before and after will result in only coalesce-ations prior to the try block (i.e. we've observed that it won't optimize away a variable declared before a try block that is not used within the try or the finally); you should not rely on those side-effects. As far a I'm concerned the JIT is free to perform that last optimization I described, and probably many others.

Performance Implications of try/catch/finally

PeterRitchie — Fri, 22 Jun 2007 17:38:00 GMT

The accepted wisdom regarding performance of try/catch|finally in C# has normally been: try has no performance side-effects unless an exception is thrown.

A discussion I was involved in recently caused me to discover some performance implications of try/catch blocks. The discussion revolved around protecting the volatility of certain members and the cross-thread memory ordering rules during run-time and during just-in-time compilation. As it turns out, Microsoft’s x86 .NET 2.0 just-in-time compiler disables optimizations that would affect the CIL order of read/writes in protected blocks (AKA “protected regions”, “guarded blocks”, or “try blocks”). As a result no optimizations are performed in try blocks.

As it turns out, there are performance side-effects to try/catch even if no exceptions are thrown. In the following academic example:

    int count = 1;
    SomeMethod();
    count++;
    SomeOtherMethod();
    count++;
    Console.WriteLine(count);

The x86 just-in-time compiler will effectively optimize this as follows:

    SomeMethod();
    SomeOtherMethod();
    Console.WriteLine(3);

The end-result is the same but the side-effects (that would normally be only visible in a debugger) are different (e.g. count never has the value 2). The just-in-time compiler can do this because it knows no other code can see count when it has the value 2.

Now, wrapping this in a try block, as follows:

    int count = 1;
    try
    {
        SomeMethod();
        count++;
        SomeOtherMethod();
        count++;
    }
    finally
    {
        Console.WriteLine(count);
    }

...is not optimized. count will be created, it will have the value 1, it will have the value 2 after the call to SomeMethod, it will have the value 3 after SomeOther method, and will have the value 3 when Console.WriteLine is called. There are two increments that therefore must be performed that are a waste of cycles.

So, be careful how you write code in try blocks.

.NET Framework 2.0 Service Pack 1

PeterRitchie — Tue, 22 May 2007 17:45:00 GMT

I noticed mention of .NET Framework 2.0 Service Pack 1 on a Microsoft site today.

The BCL Team's latest blog entry http://blogs.msdn.com/bclteam/archive/2007/05/21/the-regexoptions-compiled-flag-and-slow-performance-on-64-bit-net-framework-2-0-josh-free.aspx mentions a fix that will make it into the .NET Framework 2.0 SP 1. Seems an inevitable release of SP1 for .NET Framework 2.0 is pretty official.

Changing TextBox Text as an Undo-able Action

PeterRitchie — Mon, 11 Sep 2006 01:23:00 GMT

The TextBox class supports undoing the last action--inherited from TextBoxBase. Normally the user does this by pressing the undo key (Ctrl-Z if your keyboard doesn't have a specific Undo key) or by selecting "Undo" from the context menu. The last action can also be undone programmatically by calling TextBoxBase.Undo() (after calling CanUndo() to see if Undo() will work).

Changing the text in a TextBox so that the change can be undone is not so obvious though. Changing the Text property or the SelectedText property is not undo-able. .NET 2.0 added TextBox.Paste(String) (not inherited from TextBoxBase, it's inherent to TextBox) that replaces the currently selected text and is undoable. If you want to replace all the text while allowing the user to undo it, simply call TextBoxBase.SelectAll() before Paste(Sting). For example:

textBox.SelectAll();
textBox.Paste(text);

This side-effect, unfortunately, is not documented--which is why it "is not so obvious".

Protecting intellectual properties in .NET, Part 1.

PeterRitchie — Sat, 09 Sep 2006 16:17:00 GMT

One thing that bothers many people and organizations about .NET is the ease of which IL code can be re-hydrated into source code (C#/VB/etc.). While this has always been a problem with binaries, IL code is a much smaller set of instructions compared to the instruction sets of today's processors and is designed around accommodating high-level-language usage patterns--making it easier to translate into high-level source code. Native binaries could always be disassembled and the assembler code be reassembled into another, new, application. But, it was assembler code, and optimized--nearly impossible to translate into a high-level-language, let alone similar to the original code . Everyone seems fine with this because it doesn't really look like the original code. .NET IL an be re-hydrated into source that is almost identical to the original--sans comments.

One thing you can do with .NET is to force a method to be precompiled to native code. This can be done by attributing the method with a little-known, not well documented attribute: System.Runtime.CompilerServices.MethodImpl and setting the MethodCodeType property to System.Runtime.CompilerServices.MethodCodeType.Native.

Some caveats

Seems to only work in a FullTrust environment.
Since native methods can't be reflected these methods cannot be uses as event handlers
May cause some grief for many debuggers.
The Runtime severely restricts where methods tagged as Native can be loaded.