Jon Skeet: Coding Blog : C#

"Magic" null argument testing

skeet — Wed, 09 Dec 2009 18:04:47 GMT

Warning: here be dragons. I don't think this is the right way to check for null arguments, but it was an intriguing idea.

Today on Stack Overflow, I answered a question about checking null arguments. The questioner was already using an extension similar to my own one in MiscUtil, allowing code like this:

public void DoSomething(string name)
{
name.ThrowIfNull("name");

// Normal code here
}

That's all very well, but it's annoying to have to repeat the name part. Now in an ideal world, I'd say it would be nice to add an attribute to the parameter and have the check performed automatically (and when PostSharp works with .NET 4.0, I'm going to give that a go, mixing Code Contracts and AOP…) – but for the moment, how far can we go with extension methods?

I stand by my answer from that question – the code above is the simplest way to achieve the goal for the moment… but another answer raised the interesting prospect of combining anonymous types, extension methods, generics, reflection and manually-created expression trees. Now that's a recipe for hideous code… but it actually works.

The idea is to allow code like this:

public void DoSomething(string name, string canBeNull, int foo, Stream input)
{
new { name, input }.CheckNotNull();

// Normal code here
}

That should check name and input, in that order, and throw an appropriate ArgumentNullException - including parameter name - if one of them is null. It uses the fact that projection initializers in anonymous types use the primary expression's name as the property name in the generated type, and the value of that expression ends up in the instance. Therefore, given an instance of the anonymous type initializer like the above, we have both the name and value despite having only typed it in once.

Now obviously this could be done with normal reflection – but that we be slow as heck. No, we want to effectively find the properties once, and generate strongly typed delegates to perform the property access. That sounds like a job for Delegate.CreateDelegate, but it's not quite that simple… to create the delegate, we'd need to know (at compile time) what the property type is. We could do that with another generic type, but we can do better than that. All we really need to know about the value is whether or not it's null. So given a "container" type T, we'd like a bunch of delegates, one for each property, returning whether that property is null for a specified instance – i.e. a Func<T, bool>. And how do we build delegates at execution time with custom logic? We use expression trees…

I've now implemented this, along with a brief set of unit tests. The irony is that the tests took longer than the implementation (which isn't very unusual) – and so did writing it up in this blog post. I'm not saying that it couldn't be improved (and indeed in .NET 4.0 I could probably make the delegate throw the relevant exception itself) but it works! I haven't benchmarked it, but I'd expect it to be nearly as fast as manual tests – insignificant in methods that do real work. (The same wouldn't be true using reflection every time, of course.)

The full project including test cases is now available, but here's the (almost completely uncommented) "production" code.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Reflection;
using System.Linq.Expressions;

public static class Extensions
{
    public static void CheckNotNull<T>(this T container) where T : class
    {
        if (container == null)
        {
            throw new ArgumentNullException("container");
        }
        NullChecker<T>.Check(container);
    }

    private static class NullChecker<T> where T : class
    {
        private static readonly List<Func<T, bool>> checkers;
        private static readonly List<string> names;

        static NullChecker()
        {
            checkers = new List<Func<T, bool>>();
            names = new List<string>();
            // We can't rely on the order of the properties, but we
            // can rely on the order of the constructor parameters
            // in an anonymous type - and that there'll only be
            // one constructor.
            foreach (string name in typeof(T).GetConstructors()[0]
                                             .GetParameters()
                                             .Select(p => p.Name))
            {
                names.Add(name);
                PropertyInfo property = typeof(T).GetProperty(name);
                // I've omitted a lot of error checking, but here's
                // at least one bit...
                if (property.PropertyType.IsValueType)
                {
                    throw new ArgumentException
                        ("Property " + property + " is a value type");
                }
                ParameterExpression param = Expression.Parameter(typeof(T), "container");
                Expression propertyAccess = Expression.Property(param, property);
                Expression nullValue = Expression.Constant(null, property.PropertyType);
                Expression equality = Expression.Equal(propertyAccess, nullValue);
                var lambda = Expression.Lambda<Func<T, bool>>(equality, param);
                checkers.Add(lambda.Compile());
            }
        }

        internal static void Check(T item)
        {
            for (int i = 0; i < checkers.Count; i++)
            {
                if (checkers[i](item))
                {
                    throw new ArgumentNullException(names[i]);
                }
            }
        }
    }
}

Oh, and just as a miracle – the expression tree worked first time. I'm no Marc Gravell, but I'm clearly improving :)

Update: Marc Gravell pointed out that the order of the results of Type.GetProperties isn't guaranteed - something I should have remembered myself. However, the order of the constructor parameters will be the same as in the anonymous type initialization expression, so I've updated the code above to reflect that. Marc also showed how it could almost all be put into a single expression tree which returns either null (for no error) or the name of the "failing" parameter. Very clever :)

Custom value types are like buses

skeet — Sat, 28 Nov 2009 09:44:02 GMT

You wait years to write one… and then six of them come along at once.

(Cross-posted to the Noda Time blog and my coding blog as it's relevant to both.)

When we started converting Joda Time to .NET, there was always going to be the possibility of using custom value types (structs) – an opportunity which isn't available in Java. This has meant reducing the type hierarchy a fair amount, but that's actually made things simpler. However, I didn't realise quite how many we'd end up with – or how many would basically just wrap a long.

So far, we have 4 value types whose only field is a long. They are:

Instant: an instant on the theoretical timeline, which doesn't know about days of the week, time zones etc. It has a single reference point – the Unix epoch – but that's only so that we can represent it in a concrete fashion. The long represents the number of ticks since the Unix epoch.
LocalInstant: this is a tricky one to explain, and I'm still looking for the right way of doing so. The basic idea is that it represents a day and a time within that day, but without reference to a time zone or calendar system. So if I'm talking to someone in a different time zone and an Islamic calendar, we can agree on the idea of "3pm tomorrow" even if we have different ideas of what month that's in and when it starts. A LocalInstant is effectively the instant at which that date/time would occur if you were considering it in UTC… but importantly it's not a genuine instant, in that it doesn't unambiguously represent a point in time.
Duration: a number of ticks, which can be added to an instant to get another instant. This is a pure number of ticks – again, it doesn't really know anything about days, months etc (although you can find the duration for the number of ticks in a standard day – that's not the same as adding one day to a date and time within a time zone though, due to daylight savings).
Offset: very much like a duration, but only used to represent the offset due to a time zone. This is possibly the most unusual of the value types in Noda, because of the operators using it. You can add an offset to an instant to get a local instant, or you can subtract an offset from a local instant to get an instant… but you can't do those things the other way round.

The part about the odd nature of the operators using Offset really gets to the heart of what I like about Joda Time and what makes Noda Time even better. You see, Joda Time already has a lot of types for different concepts, where .NET only has DateTime/DateTimeOffset/TimeSpan – having these different types and limiting what you can do with them helps to lead developers into the pit of success; the type system helps you naturally get things right.

However, the Joda Time API uses long internally to represent all of these, presumably for the sake of performance: Java doesn't have custom value types, so you'd have to create a whole new object every time you manipulated anything. This could be quite significant in some cases. Using the types above has made the code a lot simpler and more obviously correct – except for a couple of cases where the code I've been porting appears to do some very odd things, which I've only noticed are odd because of the type system. James Keesey, who's been porting the time zone compiler, has had similar experiences: since introducing the offset type with its asymmetric operators, found that he had a bug in some of his ported code – which immediately caused a compile-time error when he'd converted to using offsets.

When I first saw the C# spec, I was dubious about the value of user-defined value types and operator overloading. Indeed I still suspect that both features are overused… but when they're used appropriately, they're beautiful.

Noda Time is still a long way from being a useful API, but I'm extremely pleased with how it's shaping up.

Where do you benefit from dynamic typing?

skeet — Tue, 17 Nov 2009 07:31:48 GMT

Disclaimer: I don't want this to become a flame war in the comments. I'm coming from a position of ignorance, and well aware of it. While I'd like this post to provoke thought, it's not meant to be provocative in the common use of the term.

Chapter 14 of C# in Depth is about dynamic typing in C#. A couple of reviewers have justifiably said that I'm fairly keen on the mantra of "don't use dynamic typing unless you need it" – and that possibly I'm doing dynamic typing a disservice by not pointing out more of its positive aspects. I completely agree, and I'd love to be more positive – but the problem is that I'm not (yet) convinced about why dynamic typing is something I would want to embrace.

Now I want to start off by making something clear: this is meant to be about dynamic typing. Often advocates for dynamically typed languages will mention:

REPL (read-eval-print-loop) abilities which allow for a very fast feedback loop while experimenting
Terseness – the lack of type names everywhere makes code shorter
Code evaluated at execution time (so config files can be scripts etc)

I don't count any of these as benefits of dynamic typing per se. They're benefits which often come alongside dynamic typing, but they're not dependent on dynamic typing. The terseness argument is the one most closely tied to their dynamic nature, but various languages with powerful type inference show that being statically typed doesn't mean having to specify type names everywhere. (C#'s var keyword is a very restricted form of type inference, compared with – say – that of F#.)

What I'm talking about is binding being performed at execution time and only at execution time. That allows for:

Duck typing
Dynamic reaction to previously undeclared messages
Other parts of dynamic typing I'm unaware of (how could there not be any?)

What I'm interested in is how often these are used within real world (rather than demonstration) code. It may well be that I'm suffering from Blub's paradox – that I can't see the valid uses of these features simply because I haven't used them enough. Just to be clear, I'm not saying that I never encounter problems where I would welcome dynamic typing – but I don't run into them every day, whereas I get help from the compiler every day.

Just as an indicator of how set in my statically typed ways I am, at the recent Stack Overflow DevDays event in London, Michael Sparks went through Peter Norvig's spelling corrector. It's a neat piece of code (and yes, I'll finish that port some time) but I kept finding it hard to understand simply because the types weren't spelled out. Terseness can certainly be beneficial, but in this case I would personally have found it simpler if the variable and method types had been explicitly declared.

So, for the dynamic typing fans (and I'm sure several readers will come into that category):

How often do you take advantage of dynamic typing in a way that really wouldn't be feasible (or would be very clunky) in a statically typed language?
Is it usually the same single problem which crops up regularly, or do you find a wide variety of problems benefit from dynamic typing?
When you declare a variable (or first assign a value to a variable, if your language doesn't use explicit declarations) how often do you really either not know its type or want to use some aspect of it which wouldn't typically have been available in a statically typed environment?
What balance do you find in your use of duck typing (the same method/member/message has already been declared on multiple types, but there's no common type or interface) vs truly dynamic reaction based on introspection of the message within code (e.g. building a query based on the name of the method, such as FindBooksByAuthor("Josh Bloch"))?
What aspects of dynamic typing do I appear to be completely unaware of?

Hopefully someone will be able to turn the light bulb on for me, so I can be more genuinely enthusiastic about dynamic typing, and perhaps even diversify from my comfort zone of C#…

Just how spiky is your traffic?

skeet — Mon, 16 Nov 2009 19:48:19 GMT

No, this isn't the post about dynamic languages I promise. That will come soon. This is just a quick interlude. This afternoon, while answering a question on Stack Overflow¹ about the difference between using an array and a Dictionary<string, string> (where each string was actually the string representation of an integer) I posted the usual spiel about preferring readable code to micro-optimisation. The response in a comment - about the performance aspect - was:

Well that's not so easily said for a .com where performance on a site that receives about 1 million hits a month relies on every little ounce of efficiency gains you can give it.

A million hits a month, eh? That sounds quite impressive, until you actually break it down. Let's take a month of 30 days - that has 30 * 24 * 60 * 60 = 2,592,000 seconds². In other words, a million hits a month is less than one hit every two seconds. Not so impressive. At Google we tend to measure traffic in QPS (queries per second, even if they're not really queries - the search terminology becomes pervasive) so this is around 0.39 QPS. Astonished that someone would make such a claim in favour of micro-optimisation at that traffic level, I tweeted about it. Several of the replies were along the lines of "yeah, but traffic's not evenly distributed." That's entirely true. Let's see how high we can make the traffic without going absurd though.

Let's suppose this is a site which is only relevant on weekdays - that cuts us down to 20 days in the month. Now let's suppose it's only relevant for one hour per day - it's something people look at when they get to work, and most of the users are in one time zone. That's a pretty massive way of spiking. We've gone down from 30 full days of traffic to 20 hours - or 20 * 60 * 60 = 72000 seconds, giving 14 QPS. Heck, let's say the peak of the spike is double that - a whopping 28 QPS.

Three points about this:

28 QPS is still not a huge amount of traffic.
If you're really interested in handling peak traffic of ~28 QPS without latency becoming huge, it's worth quoting that figure rather than "a million hits a month" because the latter is somewhat irrelevant, and causes us to make wild (and probably wildly inaccurate) guesses about your load distribution.
If you're going to bring the phrase "a .com" into the picture, attempting to make it sound particularly important, you really shouldn't be thinking about hosting your web site on one server - so the QPS gets diluted again.
Even at 28 QPS, the sort of difference that would be made here is tiny. A quick microbenchmark (with all the associated caveats) showed that on my laptop (hardly a server-class machine) I could build the dictionary and index into it 3 times 2.8 million times in about 5 seconds. If every request needed to do that 100 times, then the cost of doing it 28 requests per second on my laptop would still only be 0.5% of that second - not a really significant benefit, despite the hugely exaggerated estimates of how often we needed to do that.

There are various other ways in which it's not a great piece of code, but the charge against premature optimization still stands. You don't need to get every little ounce of efficiency out of your code. Chances are, if you start guessing at where you can get efficiency, you're going to be wrong. Measure, measure, measure - profile, profile, profile. Once you've done all of that and proved that a change reducing clarity has a significant benefit, go for it - but until then, write the most readable code you can. Likewise work out your performance goals in a meaningful fashion before you worry too much - and hits per months isn't a meaningful figure.

Performance is important - too important to be guessed about instead of measured.

¹ I'm not linking to it because the Streisand effect would render this question more important than it really is. I'm sure you can find it if you really want to, but that's not the point of the post.

² Anyone who wants to nitpick and talk about months which are a bit longer or shorter than that due to daylight saving time changes (despite still being 30 days) can implement that logic for me in Noda Time.

Noda Time gets its own blog

skeet — Fri, 13 Nov 2009 19:54:48 GMT

I've decided it's probably not a good idea to make general Noda Time posts on my personal blog. I'll still post anything that's particularly interesting in a "general coding" kind of way here, even if I discover it in Noda Time, but I thought it would be good for the project to have a blog of its very own, which other team members can post to.

I still have plenty of things I want to blog about here. Next up is likely to be a request for help: I want someone to tell me why I should love the "dynamic" bit of dynamic languages. Stay tuned for more details :)

Noda Time is born

skeet — Fri, 06 Nov 2009 18:02:31 GMT

There was an amazing response to yesterday's post – not only did readers come up with plenty of names, but lots of people volunteered to help. As a result, I'm feeling under a certain amount of pressure for this project to actually take shape.

The final name chosen is Noda Time. We now have a Google Code Project and a Google Group (/mailing list). Now we just need some code…

I figured it would be worth explaining a bit more about my vision for the project. Obviously I'm only one contributor, and I'm expecting everyone to add there own views, but this can act as a starting point.

I want this project to be more than just a way of getting better date and time handling on .NET. I want it to be a shining example of how to build, maintain and deploy an open source .NET library. As some of you know, I have a few other open source projects on the go, and they have different levels of polish. Some have downloadable binaries, some don't. They all have just-about-enough-to-get-started documentation, but not nearly enough, really. They have widely varying levels of test coverage. Some are easier to build than others, depending on what platform you're using.

In some ways, I'm expecting the code to be the easy part of Noda Time. After all, the implementation is there already – we'll have plenty of interesting design decisions to make in order to marry the concepts of Joda Time with the conventions of .NET, but that shouldn't be too hard. Here are the trickier things, which need discussion, investigation and so forth:

What platforms do we support? Here's my personal suggested list:

.NET 4.0
.NET 3.5
.NET 2.0SP1 (require the service pack for DateTimeOffset)
Mono (versions TBD)
Silverlight 2, 3 and 4
Compact Framework 2.0 and 3.5

What do we ship, and how do we handle different platforms? For example, can we somehow use Code Contracts to give developers a better experience on .NET 4.0 without making it really hard to build for other versions of .NET? Can we take advantage of the availability of TimeZoneInfo in .NET 3.5 and still build fairly easily for earlier versions? Do developers want debug or release binaries? Can we build against the client profile of .NET 3.5/4.0?
What should we use to build? I've previously used NAnt for the overall build process and MSBuild for the code building part. While this has worked quite well, I'm nervous of the dependency on NAnt-Contrib library for the <msbuild> task, and generally being dependent on a build project whose last release was a beta nearly two years ago. Are there better alternatives?
How should documentation be created and distributed?

Is Sandcastle the best way of building docs? How easy is it to get it running so that any developer can build the docs at any time? (I've previously tried a couple of times, and failed miserable.)
Would Monodoc be a better approach?
How should non-API documentation be handled? Is the wiki which comes with the Google Code project good enough? Do we need to somehow suck the wiki into an offline format for distribution with the binaries?

What do we need to do in order to work in low-trust environments, and how easily can we test that?
What do we do about signing? Ship with a "public" snk file which anyone can build with, but have a private version which the team uses to validate a "known good" release? Or just have the private key and use deferred signing?
While the library itself will support i18n for things like date/time formatting, do we need to apply it to "developer only" messages such as exceptions?
I'm used to testing with NUnit and Rhino.Mocks, but they're not the last word in testing on .NET – what should we use, and why? What about coverage?
Do we need any dependencies (e.g. logging)? If so, how do we handle versioning of those dependencies? How are we affected by various licences?

These are all interesting topics, but they're not really specific to Noda Time. Information about them is available all over the place, but that's just the problem – it's all over the place. I would like there to be some sort of documentation saying, "These are the decisions you need to think about, here are the options we chose for Noda Time, and this is why we did so." I don't know what form that documentation will take yet, but I'm considering an ebook.

As you can tell, I'm aiming pretty high with this project – especially as I won't even be using Google's 20% time on it. However, there's little urgency in it for me personally. I want to work out how to do things right rather than how to do them quickly. If it takes me a bit of time to document various decisions, and the code itself ships later, so be it… it'll make the next project that much speedier.

I'm expecting a lot of discussion in the group, and no doubt some significant disagreements. I'm expecting to have to ask a bunch of questions on Stack Overflow, revealing just how ignorant I am on a lot of the topics above (and more). I think it'll be worth it though. I think it's worth setting a goal:

In one year, I want this to be a first-class project which is the natural choice for any developers wanting to do anything more than the simplest of date/time handling on .NET. In one year, I want to have a guide to developing open source class libraries on .NET which tells you everything you need to know other than how to write the code itself.

A year may seem like a long time, but I'm sure everyone who has expressed an interest in the project has significant other commitments – I know I do. Getting there in a year is going to be a stretch – but I'm expecting it to be a very enlightening journey.

What's in a name (again)?

skeet — Thu, 05 Nov 2009 20:01:05 GMT

I have possibly foolishly decided to stop resisting the urge to port Joda Time to .NET. For those of you who are unaware, "use Joda Time" is almost always the best answer to any question involving "how do I achieve X with java.util.Date/Calendar?" It's a Java library for handling dates and times, and it rocks. There is a plan to include a somewhat redesigned version in some future edition of Java (JSR-310) but it's uncertain whether this will ever happen.

Now, .NET only gained the ability to work with time zones other than UTC and the local time zone (using only managed code) – it has a bit of catching up to do. It's generally easier to work with the .NET BCL than the Java built-in libraries, but it's still not a brilliant position to be in. I think .NET deserves good date/time support, and as no-one else appears to be porting Joda Time, I'm going to do it. (A few people have already volunteered to help. I don't know how easily we'll be able to divvy up the work, but we'll see. I suspect the core may need to be done first, and then people can jump in to implement different chronologies etc. As a side-effect, I may try to use this project as a sort of case in terms of porting, managing an open source project, and properly implementing a .NET library with useful versioning etc.)

The first two problems, however, are to do with naming. First, the project name. Contenders include:

Joda Time.NET (sounds like it would be an absolutely direct port; while I intend to port all the tricky bits directly, it's going to be an idiomatic port with appropriate .NET bits. It's also a bit of a mouthful.)
Noda Time (as suggested in the comments and in email)
TonyTime (after Tony the Pony)
CoffeeTime
TeaTime
A progression of BreakfastTime, CoffeeTime, LunchTime, TeaTime, DinnerTime and SupperTime for different versions (not a serious contender)
ParsleySageRosemaryAndThyme (not a serious contender)
A few other silly ones too

I suspect I'm going to go for CoffeeTime, but we'll see.

The second problem is going to prove more awkward. I want to mostly copy the names given in Joda Time – aside from anything else, it'll make it familiar to anyone who uses Joda Time in Java (such as me). Now one of the most commonly used classes in Joda is "DateTime". Using that name in my port would be a Bad Idea. Shadowing a name in the System namespace is likely to lead to very disgruntled users who may prove hard to regruntle before they abandon the library.

So what do I do? Go for the subtly different DateAndTime? Tie it to the library with CoffeeDateTime? Change it to Instant? (It'll derive from AbstractInstant anyway – assuming I keep the same hierarchy instead of moving to a composition model and value types.)

Obviously this is a decision which the "team" can make, when we've got one… but it feels like a decision which is lurking round the corner in a hostile way.

What I find interesting is that these are two very different naming problems: one is trying to name something in a relatively arbitrary way – I know I want something reasonably short and memorable for the overall name, but beyond that it doesn't matter too much. The other is trying to nail a very specific name which really has to convey its meaning clearly… but where the obvious name is already taken. Also interestingly, neither is a particularly good example of my most common issue with naming: attempting to come up with a two or three word noun for something that actually needs a whole sentence to describe it adequately.

Oh well – we'll see what happens. In another blog post I'll suggest some of the goals I have in terms of what I'm hoping to learn from the project, and how I'd like it to progress. In other words, expect a work of complete fiction…

If you're interested in helping out with the project, please mail me directly (rather than adding comments here) and as soon as I've set the project up, I'll invite you to the mailing list.

UPDATE: I've already got a few interested names, which is great. Rather than be dictatorial about this, I'll put it to a vote of the people who are willing to help out on it.

Revisiting randomness

skeet — Wed, 04 Nov 2009 07:40:15 GMT

Almost every Stack Overflow question which includes the words "random" and "repeated" has the same basic answer. It's one of the most common "gotchas" in .NET, Java, and no doubt other platforms: creating a new random number generator without specifying a seed will depend on the current instant of time. The current time as measured by the computer doesn't change very often compared with how often you can create and use a random number generator – so code which repeatedly creates a new instance of Random and uses it once will end up showing a lot of repetition.

One common solution is to use a static field to store a single instance of Random and reuse it. That's okay in Java (where Random is thread-safe) but it's not so good in .NET – if you use the same instance repeatedly from .NET, you can corrupt the internal data structures.

A long time ago, I created a StaticRandom class in MiscUtil – essentially, it was just a bunch of static methods (to mirror the instance methods found in Random) wrapping a single instance of Random and locking appropriately. This allows you to just call StaticRandom.Next(1, 7) to roll a die, for example. However, it has a couple of problems:

It doesn't scale well in a multi-threaded environment. When I originally wrote it, I benchmarked an alternative approach using [ThreadStatic] and at the time, locking won (at least on my computer, which may well have only had a single core).
It doesn't provide any way of getting at an instance of Random, other than by using new Random(StaticRandom.Next()).

The latter point is mostly a problem because it encourages a style of coding where you just use StaticRandom.Next(…) any time you want a random number. This is undoubtedly convenient in some situations, but it goes against the idea of treating a source of randomness as a service or dependency. It makes it harder to get repeatability and to see what needs that dependency.

I could have just added a method generating a new instance into the existing class, but I decided to play with a different approach – going back to per-thread instances, but this time using the ThreadLocal<T> class introduced in .NET 4.0. Here's the resulting code:

using System;
using System.Threading;

namespace RandomDemo
{
    /// <summary>
    /// Convenience class for dealing with randomness.
    /// </summary>
    public static class ThreadLocalRandom
    {
        /// <summary>
        /// Random number generator used to generate seeds,
        /// which are then used to create new random number
        /// generators on a per-thread basis.
        /// </summary>
        private static readonly Random globalRandom = new Random();
        private static readonly object globalLock = new object();

        /// <summary>
        /// Random number generator
        /// </summary>
        private static readonly ThreadLocal<Random> threadRandom = new ThreadLocal<Random>(NewRandom);

        /// <summary>
        /// Creates a new instance of Random. The seed is derived
        /// from a global (static) instance of Random, rather
        /// than time.
        /// </summary>
        public static Random NewRandom()
        {
            lock (globalLock)
            {
                return new Random(globalRandom.Next());
            }
        }

        /// <summary>
        /// Returns an instance of Random which can be used freely
        /// within the current thread.
        /// </summary>
        public static Random Instance { get { return threadRandom.Value; } }

        /// <summary>See <see cref="Random.Next()" /></summary>
        public static int Next()
        {
            return Instance.Next();
        }

        /// <summary>See <see cref="Random.Next(int)" /></summary>
        public static int Next(int maxValue)
        {
            return Instance.Next(maxValue);
        }

        /// <summary>See <see cref="Random.Next(int, int)" /></summary>
        public static int Next(int minValue, int maxValue)
        {
            return Instance.Next(minValue, maxValue);
        }

        /// <summary>See <see cref="Random.NextDouble()" /></summary>
        public static double NextDouble()
        {
            return Instance.NextDouble();
        }

        /// <summary>See <see cref="Random.NextBytes(byte[])" /></summary>
        public static void NextBytes(byte[] buffer)
        {
            Instance.NextBytes(buffer);
        }
    }
}

The user can still call the static Next(…) methods if they want, but they can also get at the thread-local instance of Random by calling ThreadLocalRandom.Instance – or easily create a new instance with ThreadLocalRandom.NewRandom(). (The fact that NewRandom uses the global instance rather than the thread-local one is an implementation detail really; it happens to be convenient from the point of view of the ThreadLocal<T> constructor. It wouldn't be terribly hard to change this.)

Now it's easy to write a method which needs randomness (e.g. to shuffle a deck of cards) and give it a Random parameter, then call it using the thread-local instance:

public void Shuffle(Random rng)
{
...
}
...
deck.Shuffle(ThreadLocalRandom.Instance);

The Shuffle method is then easier to test and debug, and expresses its dependency explicitly.

Performance

I tested the "old" and "new" implementations in a very simple way – for varying numbers of threads, I called Next() a fixed number of times (from each thread) and timed how long it took for all the threads to finish. I've also tried a .NET-3.5-compatible version using ThreadStatic instead of ThreadLocal<T>, and running the original code and the ThreadStatic version on .NET 3.5 as well.

Threads	StaticRandom (4.0b2)	ThreadLocalRandom (4.0b2)	ThreadStaticRandom (4.0b2)	StaticRandom(3.5)	ThreadStaticRandom (3.5)
1	11582	6016	10150	10373	16049
2	24667	7214	9043	25062	17257
3	38095	10295	14771	36827	25625
4	49402	13435	19116	47882	34415

A few things to take away from this:

The numbers were slightly erratic; somehow it was quicker to do twice the work with ThreadStaticRandom on .NET 4.0b2! This isn't the perfect benchmarking machine; we're interested in trends rather than absolute figures.
Locking hasn't changed much in performance between framework versions
ThreadStatic has improved massively between .NET 3.5 and 4.0
Even on 3.5, ThreadStatic wins over a global lock as soon as there's contention

The only downside to the ThreadLocal<T> version is that it requires .NET 4.0 - but the ThreadStatic version works pretty well too.

It's worth bearing in mind that of course this is testing the highly unusual situation where there's a lot of contention in the global lock version. The performance difference in the single-threaded version where the lock is always uncontended is still present, but very small.

Update

After reading the comments and thinking further, I would indeed get rid of the static methods elsewhere. Also, for the purposes of dependency injection, I agree that it's a good idea to have a factory interface where that's not overkill. The factory implementation could use either the ThreadLocal or ThreadStatic implementations, or effectively use the global lock version (by having its own instance of Random and a lock). In many cases I'd regard that as overkill, however.

One other interesting option would be to create a thread-safe instance of Random to start with, which delegated to thread-local "normal" implementations. That would be very useful from a DI standpoint.

Contract classes and nested types within interfaces

skeet — Sat, 31 Oct 2009 22:04:54 GMT

I've just been going through some feedback for the draft copy of the second edition of C# in Depth. In the contracts section, I have an example like this:

[ContractClass(typeof(ICaseConverterContracts))]
public interface ICaseConverter
{
    string Convert(string text);
}

[ContractClassFor(typeof(ICaseConverter))]
internal class ICaseConverterContracts : ICaseConverter
{
    string ICaseConverter.Convert(string text)
    {
        Contract.Requires(text != null);
        Contract.Ensures(Contract.Result<string>() != null);
        return default(string);
    }

    private ICaseConverterContracts() {}
}

public class InvariantUpperCaseFormatter : ICaseConverter
{
    public string Convert(string text)
    {
        return text.ToUpperInvariant();
    }
}

The point is to demonstrate how contracts can be specified for interfaces, and then applied automatically to implementations. In this case, ICaseConverter is the interface, ICaseConverterContracts is the contract class which specifies the contract for the interface, and InvariantUpperCaseFormatter is the real implementation. The binary rewriter effectively copies the contract into each implementation, so you don't need to duplicate the contract in the source code.

The reader feedback asked where the contract class code should live - should it go in the same file as the interface itself, or in a separate file as normal? Now normally, I'm firmly of the "one top-level type per file" persuasion, but in this case I think it makes sense to keep the contract class with the interface. It has no meaning without reference to the interface, after all - it's not a real implementation to be used in the normal way. It's essentially metadata. This does, however, leave me feeling a little bit dirty. What I'd really like to be able to do is nest the contract class inside the interface, just like I do with other classes which are tightly coupled to an "owner" type. Then the code would look like this:

[ContractClass(typeof(ICaseConverterContracts))]
public interface ICaseConverter
{
    string Convert(string text);

    [ContractClassFor(typeof(ICaseConverter))]
    internal class ICaseConverterContracts : ICaseConverter
    {
        string ICaseConverter.Convert(string text)
        {
            Contract.Requires(text != null);
            Contract.Ensures(Contract.Result<string>() != null);
            return default(string);
        }

        private ICaseConverterContracts() {}
    }
}

public class InvariantUpperCaseFormatter : ICaseConverter
{
    public string Convert(string text)
    {
        return text.ToUpperInvariant();
    }
}

That would make me feel happier - all the information to do with the interface would be specified within the interface type's code. It's possible that with that as a convention, the Code Contracts tooling could cope without the attributes - if interface IFoo contains a nested class IFooContracts which implements IFoo, assume it's a contract class and handle it appropriately. That would be sweet.

You know the really galling thing? I'm pretty sure VB does allow nested types in interfaces...

Migrating from Visual Studio 2010 beta 1 to beta 2 – solution file change required

skeet — Mon, 26 Oct 2009 10:46:36 GMT

Having installed Visual Studio 2010 beta 2 on my freshly-reinstalled netbook (now with Windows 7 and and SSD – yummy) I found that my solution file from Visual Studio 2010 beta 1 wasn’t recognised properly: double-clicking on the file didn’t do anything. Opening the solution file manually was absolutely fine, but slightly less convenient than being able to double-click.

After a bit of investigation, I’ve found the solution. Manually edit the solution file, and change the first few lines from this:

Microsoft Visual Studio Solution File, Format Version 11.00
# Visual Studio 10

to this:

Microsoft Visual Studio Solution File, Format Version 11.00
# Visual Studio 2010

It's just a case of changing "10" to "2010".

Hopefully between this and the linked SuperUser post, this should avoid others feeling the same level of bafflement :)

Iterating atomically

skeet — Fri, 23 Oct 2009 21:20:00 GMT

The IEnumerable<T> and IEnumerator<T> interfaces in .NET are interesting. They crop up an awful lot, but hardly anyone ever calls them directly - you almost always use a foreach loop to iterate over the collection. That hides all the calls to GetEnumerator(), MoveNext() and Current. Likewise iterator blocks hide the details when you want to implement the interfaces. However, sometimes details matter - such as for this recent Stack Overflow question. The question asks how to create a thread-safe iterator - one that can be called from multiple threads. This is not about iterating over a collection n times independently on n different threads - this is about iterating over a collection once without skipping or duplicating. Imagine it's some set of jobs that we have to complete. We assume that the iterator itself is thread-safe to the extent that calls from different threads at different times, with intervening locks will be handled reasonably. This is reasonable - basically, so long as it isn't going out of its way to be thread-hostile, we should be okay. We also assume that no-one is trying to write to the collection at the same time.

Sounds easy, right? Well, no... because the IEnumerator<T> interface has two members which we effectively want to call atomically. In particular, we don't want the collection { "a", "b" } to be iterated like this:

Thread 1	Thread 2
MoveNext()
	MoveNext()
Current
	Current

That way we'll end up not processing the first item at all, and the second item twice.

There are two ways of approaching this problem. In both cases I've started with IEnumerable<T> for consistency, but in fact it's IEnumerator<T> which is the interesting bit. In particular, we're not going to be able to iterate over our result anyway, as each thread needs to have the same IEnumerator<T> - which it won't do if each of them uses foreach (which calls GetEnumerator() to start with).

Fix the interface

First we'll try to fix the interface to look how it should have looked to start with, at least from the point of view of atomicity. Here are the new interfaces:

public interface IAtomicEnumerable<T>
{
IAtomicEnumerator<T> GetEnumerator();
}

public interface IAtomicEnumerator<T>
{
bool TryMoveNext(out T nextValue);
}

One thing you may notice is that we're not implementing IDisposable. That's basically because it's a pain to do so when you think about a multi-threaded environment. Indeed, it's possibly one of the biggest arguments against something of this nature. At what point do you dispose? Just because one thread finished doesn't mean that the rest of them have... don't forget that "finish" might mean "an exception was thrown while processing the job, I'm bailing out". You'd need some sort of co-ordinator to make sure that everyone is finished before you actually do any clean-up. Anyway, the nice thing about this being a blog post is we can ignore that little thorny issue :)

The important point is that we now have a single method in IAtomicEnumerator<T> - TryMoveNext, which works the way you'd expect it to. It atomically attempts to move to the next item, returns whether or not it succeeded, and sets an out parameter with the next value if it did succeed. Now there's no chance of two threads using the method and stomping on each other's values (unless they're silly and use the same variable for the out parameter).

It's reasonably easy to wrap the standard interfaces in order to implement this interface:

/// <summary>
/// Wraps a normal IEnumerable[T] up to implement IAtomicEnumerable[T].
/// </summary>
public sealed class AtomicEnumerable<T> : IAtomicEnumerable<T>
{
    private readonly IEnumerable<T> original;

    public AtomicEnumerable(IEnumerable<T> original)
    {
        this.original = original;
    }

    public IAtomicEnumerator<T> GetEnumerator()
    {
        return new AtomicEnumerator(original.GetEnumerator());
    }

    /// <summary>
    /// Implementation of IAtomicEnumerator[T] to wrap IEnumerator[T].
    /// </summary>
    private sealed class AtomicEnumerator : IAtomicEnumerator<T>
    {
        private readonly IEnumerator<T> original;
        private readonly object padlock = new object();

        internal AtomicEnumerator(IEnumerator<T> original)
        {
            this.original = original;
        }

        public bool TryMoveNext(out T value)
        {
            lock (padlock)
            {
                bool hadNext = original.MoveNext();
                value = hadNext ? original.Current : default(T);
                return hadNext;
            }
        }
    }
}

Just ignore the fact that I never dispose of the original IEnumerator<T> :)

We use a simple lock to make sure that MoveNext() and Current always happen together - that nothing else is going to call MoveNext() between our TryMoveNext() calling it, and it fetching the current value.

Obviously you'd need to write your own code to actually use this sort of iterator, but it would be quite simple:

T value;
while (iterator.TryMoveNext(out value))
{
// Use value
}

However, you may already have code which wants to use an IEnumerator<T>. Let's see what else we can do.

Using thread local variables to fake it

.NET 4.0 has a very useful type called ThreadLocal<T>. It does basically what you'd expect it to, with nice features such as being able to supply a delegate to be executed on each thread to provide the initial value. We can use a thread local to make sure that so long as we call both MoveNext() and Current atomically when we're asked to move to the next element, we can get back the right value for Current later on. It has to be thread local because we're sharing a single IEnumerator<T> across multiple threads - each needs its own separate storage.

This is also the approach we'd use if we wanted to wrap an IAtomicEnumerator<T> in an IEnumerator<T>, by the way. Here's the code to do it:

public class ThreadSafeEnumerable<T> : IEnumerable<T>
{
    private readonly IEnumerable<T> original;

    public ThreadSafeEnumerable(IEnumerable<T> original)
    {
        this.original = original;
    }

    public IEnumerator<T> GetEnumerator()
    {
        return new ThreadSafeEnumerator(original.GetEnumerator());
    }

    IEnumerator IEnumerable.GetEnumerator()
    {
        return GetEnumerator();
    }

    private sealed class ThreadSafeEnumerator : IEnumerator<T>
    {
        private readonly IEnumerator<T> original;
        private readonly object padlock = new object();
        private readonly ThreadLocal<T> current = new ThreadLocal<T>();

        internal ThreadSafeEnumerator(IEnumerator<T> original)
        {
            this.original = original;
        }

        public bool MoveNext()
        {
            lock (padlock)
            {
                bool ret = original.MoveNext();
                if (ret)
                {
                    current.Value = original.Current;
                }
                return ret;
            }
        }

        public T Current
        {
            get { return current.Value; }
        }

        public void Dispose()
        {
            original.Dispose();
            current.Dispose();
        }

        object IEnumerator.Current
        {
            get { return Current; }
        }

        public void Reset()
        {
            throw new NotSupportedException();
        }
    }
}

I'm going to say it one last time - we're broken when it comes to disposal. There's no way of safely disposing of the original iterator at "just the right time" when everyone's finished with it. Oh well.

Other than that, it's quite simple. This code has the serendipitous property of actually implementing IEnumerator<T> slightly better than C#-compiler-generated implementations from iterator blocks - if you call the Current property without having called MoveNext(), this will throw an InvalidOperationException, just as the documentation says it should. (It doesn't do the same at the end, admittedly, but that's fixable if we really wanted to be pedantic.

Conclusion

I found this an intriguing little problem. I think there are better ways of solving the bigger picture - a co-ordinator which takes care of disposing exactly once, and which possibly mediates the original iterator etc is probably the way forward... but I enjoyed thinking about the nitty gritty.

Generally speaking, I prefer the first of these approaches. Thread local variables always feel like a bit of a grotty hack to me - they can be useful, but it's better to avoid them if you can. It's interesting to see how an interface can be inherently thread-friendly or not.

One last word of warning - this code is completely untested. It builds, and I can't immediately see why it wouldn't work, but I'm making no guarantees...

Generic collections - relegate to an appendix?

skeet — Fri, 02 Oct 2009 21:00:59 GMT

(I tweeted a brief version of this suggestion and the results have been overwhelmingly positive so far, but I thought it would be worth fleshing out anyway.)

I'm currently editing chapter 3 of C# in Depth. In the first edition, it's nearly 48 pages long - the longest in the book, and longer than I want it to be.

One of the sections in there (only 6 pages, admittedly) is a description of various .NET 2.0 collections. However, it's mostly comparing them with the nongeneric collections from .NET 1.0, which probably isn't relevant any more. I suspect my readership has now moved on from "I only know C# 1" to "I've used C# 2 and I'm reasonably familiar with the framework, but I want to know the details of the language."

I propose moving the collections into an appendix. This will mean:

I'll cover all versions of .NET, not just 2.0
It will all be done in a fairly summary form, like the current appendix. (An appendix doesn't need as much of a narrative structure as a main chapter, IMO.)
I'll cover the interfaces as well as the classes - possibly even with pictures (type hierarchies)!
Chapter 3 can be a bit slimmer (although I've been adding a little bit here and there, so I'm not going to save a massive amount)
It will be easier to find as a quick reference (and I'll write it in a way which makes it easy to use as a reference too, hopefully)
I don't have to edit it right now :)

Does this sound like a plan? I don't know why I didn't think of it before, but I think it's the right move. In particular, it's in-keeping with the LINQ operator coverage in the existing appendix.

MVP no more

skeet — Thu, 01 Oct 2009 05:21:27 GMT

It's with some sadness that I have to announce that as of the start of October, I'm no longer a Microsoft MVP.

As renewal time came round again, I asked my employer whether it was okay for me to renew, and was advised not to do so. As a result, while I enjoyed being awarded as an MVP, I've asked not to be considered for renewal this year.

This doesn't mean I'm turning my back on that side of software development, of course. I'm still going to be an active member of the C# community. I'm still writing the second edition of C# in Depth. I'm still going to post on Stack Overflow. I'm still going to blog here about whatever interesting and wacky topics crop up.

I just won't be doing so as an MVP.

Thanks to all the friends I've made in the MVP community and Microsoft over the last 6 years, and I wish you all the best.

Keep in touch.

An object lesson in blogging and accuracy; was: Efficient "vote counting" with LINQ to Objects - and the value of nothing

skeet — Sun, 20 Sep 2009 20:48:00 GMT

Well, this is embarrassing.

Yesterday evening, I excitedly wrote a blog post about an interesting little idea for making a particular type of LINQ query (basically vote counting) efficient. It was an idea that had occurred to me a few months back, but I hadn't got round to blogging about it.

The basic idea was to take a completely empty struct, and use that as the element type in the results of a grouping query - as the struct was empty, it would take no space, therefore "huge" arrays could be created for no cost beyond the fixed array overhead, etc. I carefully checked that the type used for grouping did in fact implement ICollection<T> so that the Count method would be efficient; I wrote sample code which made sure my queries were valid... but I failed to check that the empty struct really took up no memory.

Fortunately, I have smart readers, a number of whom pointed out my mistake in very kind terms.

Ben Voigt gave the reason for the size being 1 in a comment:

The object identity rules require a unique address for each instance... identity can be shared with super- or sub- class objects (Empty Base Optimization) but the total size of the instance has to be at least 1.

This makes perfect sense - it's just a shame I didn't realise it before.

Live and learn, I guess - but apologies for the poorly researched post. I'll attempt to be more careful next time.

API design: choosing between non-ideal options

skeet — Mon, 14 Sep 2009 16:38:05 GMT

So, UnconstrainedMelody is coming on quite nicely. It now has quite a few useful options for flags enums, "normal enums" and delegates. However, there are two conflicting limitations which leave a couple of options. (Other related answers on Stack Overflow have suggested alternative approaches, basically.)

Currently, most of the enums code is in two classes: Flags and Enums. Both are non-generic: the methods within them are generic methods, so they have type parameters (and constraints). The main benefit of this is that generic type inference only applies to generic methods, and I definitely want that for extension methods and anywhere else it makes sense.

The drawback is that properties can't be generic. That means my API is entirely expressed in terms of methods, which can be a pain. The option to work around this is to have a generic type which properties in. This adds confusion and guesswork - what call is where?

To recap, the options are:

// Option 1 (current): all methods in a nongeneric class:
// Some calls which are logically properties end up
// as methods...
IList<Foo> foos = Enums.GetValues<Foo>();
// Type infererence for extenion methods
// Note that we couldn't have a Description property
// as we don't have extension properties
string firstDescription = foos[0].GetDescription();

// Option 2: Use just a generic type:
// Now we can use a property...
IList<Foo> foos = Enums<Foo>.Values;
// But we can't use type inference
string firstDescription = Enums<Foo>.GetDescription(foos[0]);

// Option 3: Use a mixture (Enums and Enums<T>):
IList<Foo> foos = Enums<Foo>.Values;
// All looks good...
string firstDescription = foos[0].GetDescription();
// ... but the user has to know when to use which class

All of these are somewhat annoying. If we only put extension methods into the nongeneric class, then I guess users would never need to really think about that - they'd pretty much always be calling the methods via the extension method syntactic sugar anyway. It still feels like a pretty arbitrary split though.

Any thoughts? Which is more important - conceptual complexity, or the idiomatic client code you end up with once that complexity has been mastered? Is it reasonable to make design decisions like this around what is essentially a single piece of syntactic sugar (extension methods)?

(By the way, if anyone ever wanted justification for extension properties, I think this is a good example... Description feels like it really should be a property.)

Generic constraints for enums and delegates

skeet — Thu, 10 Sep 2009 20:09:00 GMT

As most readers probably know, C# prohibits generic type constraints from referring to System.Object, System.Enum, System.Array, System.Delegate and System.ValueType. In other words, this method declaration is illegal:

public static T[]GetValues<T>() where T : struct, System.Enum
{
return (T[]) Enum.GetValues(typeof(T));
}

This is a pity, as such a method could be useful. (In fact there are better things we can do... such as returning a read-only collection. That way we don't have to create a new array each time the method is called.) As far as I can tell, there is no reason why this should be prohibited. Eric Lippert has stated that he believes the CLR doesn't support this - but I think he's wrong. I can't remember the last time I had cause to believe Eric to be wrong about something, and I'm somewhat nervous of even mentioning it, but section 10.1.7 of the CLI spec (ECMA-335) partition II (p40) specifically gives examples of type parameter constraints involving System.Delegate and System.Enum. It introduces the table with "The following table shows the valid combinations of type and special constraints for a representative set of types." It was only due to reading this table that I realized that the value type constraint on the above is required (or a constructor constraint would do equally well) - otherwise System.Enum itself satisfies the constraint, which would be a Bad Thing.

It's possible (but unlikely) that the CLI doesn't fully implement this part of the CLR spec. I'm hoping that Eric's just wrong on this occasion, and that actually there's nothing to stop the C# language from allowing such constraints in the future. (It would be nice to get keyword support, such that a constraint of "T : enum" would be equivalent to the above, but hey...)

The good news is that ilasm/ildasm have no problem with this. The better news is that if you add a reference to a library which uses those constraints, the C# compiler applies them sensibly, as far as I can tell...

Introducing UnconstrainedMelody

(Okay, the name will almost surely have to change. But I like the idea of it removing the constraints of C# around which constraints are valid... and yet still being in the key of C#. Better suggestions welcome.)

I have a plan - I want to write a utility library which does useful things for enums and delegates (and arrays if I can think of anything sensible to do with them). It will be written in C#, with methods like this:

public static T[]GetValues<T>() where T : struct, IEnumConstraint
{
return (T[]) Enum.GetValues(typeof(T));
}

(IEnumConstraint has to be an interface of course, as otherwise the constraint would be invalid.)

As a post-build step, I will:

Run ildasm on the resulting binary
Replace every constraint using EnumConstraint with System.Enum
Run ilasm to build the binary again

If anyone has a simple binary rewriter (I've looked at PostSharp and CCI; both look way more complicated than the above) which would do this, that would be great. Otherwise ildasm/ilasm will be fine. It's not like consumers will need to perform this step.

As soon as the name is finalized I'll add a project on Google Code. Once the infrastructure is in place, adding utility methods should be very straightforward. Suggestions for utility methods would be useful, or just join the project when it's up and running.

Am I being silly? Have I overlooked something?

A couple of hours later...

Okay, I decided not to wait for a better name. The first cut - which does basically nothing but validate the idea, and the fact that I can still unit test it - is in. The UnconstrainedMelody Google Code project is live!

Recent activities

skeet — Thu, 03 Sep 2009 23:08:00 GMT

It's been a little while since I've blogged, and quite a lot has been going on. In fact, there are a few things I'd have blogged about already if it weren't for "things" getting in the way.

Rather than writing a whole series of very short blog posts, I thought I'd wrap them all up here...

C# in Depth: next MEAP drop available soon - Code Contracts

Thanks to everyone who gave feedback on my writing dilemma. For the moment, the plan is to have a whole chapter about Code Contracts, but not include a chapter about Parallel Extensions. My argument for making this decision is that Code Contracts really change the feel of the code, making it almost like a language feature - and its applicability is almost ubiquitous, unlike PFX.

I may write a PFX chapter as a separate download, but I'm sensitive to those who (like me) appreciate slim books. I don't want to "bulk out" the book with extra topics.

The Code Contracts chapter is in the final stages before becoming available to MEAP subscribers. (It's been "nearly ready" for a couple of weeks, but I've been on holiday, amongst other things.) After that, I'm going back to the existing chapters and revising them.

Talking in Dublin - C# 4 and Parallel Extensions

Last week I gave two talks in Dublin at Epicenter. One was on C# 4, and the other on Code Contracts and Parallel Extensions. Both are now available in a slightly odd form on the Talks page of the C# in Depth web site. I no longer write "formal" PowerPoint slides, so the downloads are for simple bullet points of text, along with silly hand-drawn slides. No code yet - I want to tidy it up a bit before including it.

Podcasting with The Connected Show

I recently recorded a podcast episode with The Connected Show. I'm "on" for the second 2/3 of the show - about an hour of me blathering on about the new features of C# 4. If you can understand generic variance just by listening to me talking about it, you're a smart cookie ;)

(Oh, and if you like it, please express your amusement on Digg / DZone / Shout / Kicks.)

Finishing up with Functional Programming for the Real World

Well, this hasn't been taking much of my time recently (I bowed out of all the indexing etc!) but Functional Programming for the Real World is nearly ready to go. Hard copy should be available in the next couple of months... it'll be really nice to see how it fares. Much kudos to Tomas for all his hard work - I've really just been helping out a little.

Starting on Groovy in Action, 2nd edition

No sooner does one book finish than another one starts. The second edition of Groovy in Action is in the works, which should prove interesting. To be honest, I haven't played with Groovy much since the first edition of the book was finished, so it'll be interesting to see what's happened to the language in the meantime. I'll be applying the same sort of spit and polish that I did in the first edition, and asking appropriately ignorant questions of the other authors.

Tech Reviewing C# 4.0 in a Nutshell

I liked C# 3.0 in a Nutshell, and I feel honoured that Joe asked me to be a tech reviewer for the next edition, which promises to be even better. There's not a lot more I can say about it at the moment, other than it'll be out in 2010 - and I still feel that C# in Depth is a good companion book.

MoreLINQ now at 1.0 beta

A while ago I started the MoreLINQ project, and it gained some developers with more time than I've got available :) Basically the idea is to add some more useful LINQ extension methods to LINQ to Object. Thanks to Atif Aziz, the first beta version has been released. This doesn't mean we're "done" though - just that we think we've got something useful. Any suggestions for other operators would be welcome.

Manning Pop Quiz and discounts

While I'm plugging books etc, it's worth mentioning the Manning Pop Quiz - multiple choice questions on a wide variety of topics. Fabulous prizes available, as well as one-day discounts:

Monday, Sept 7th: 50% of all print books (code: pop0907)
Monday, Sept 14: 50% off all ebooks (code: pop0914)
Thursday, Sept 17: $25 for C# in Depth, 2nd Edition MEAP print version (code: pop0917) + C# Pop Quiz question
Monday, Sept 21: 50% off all books (code: pop0921)
Thursday, Sept 24: $12 for C# in Depth, 2nd Edition MEAP ebook (code: pop0924) + another C# Pop Quiz question

Future speaking engagements

On September 16th I'm going to be speaking to Edge UG (formerly Vista Squad) in London about Code Contracts and Parallel Extensions. I'm already very much looking forward to the Stack Overflow DevDays London conference on October 28th, at which I'll be talking about how humanity has screwed up computing.

Future potential blog posts

Some day I may get round to writing about:

Revisiting StaticRandom with ThreadLocal<T>
Volatile doesn't mean what I thought it did

There's a lot more writing than coding in that list... I'd like to spend some more time on MiniBench at some point, but you know what deadlines are like.

Anyway, that's what I've been up to and what I'll be doing for a little while...

The "dream book" for C# and .NET

skeet — Thu, 20 Aug 2009 16:57:40 GMT

This morning I showed my hand a little on Twitter. I've had a dream for a long time about the ultimate C# book. It's a dream based on Effective Java, which is my favourite Java book, along with my experiences of writing C# in Depth.

Effective Java is written by Josh Bloch, who is an absolute giant in the Java world... and that's both the problem and the opportunity. There's no-one of quite the equivalent stature in the .NET world. Instead, there are many very smart people, a lot of whom blog and some of whom have their own books.

There are "best practices" books, of course: Microsoft's own Framework Design Guidelines, and Bill Wagner's Effective C# and More Effective C# being the most obvious examples. I'm in no way trying to knock these books, but I feel we could do even better. The Framework Design Guidelines (also available free to browse on MSDN) are really about how to create a good API - which is important, but not the be-all-and-end-all for many application developers who aren't trying to ship a reusable class library and may well have different concerns. They want to know how to use the language most effectively, as well as the core types within the framework.

Bill's books - and many others which cover the core framework, such as CLR via C#, Accelerated C# 2008 and C# 3.0 in a Nutshell - give plenty of advice, but often I've felt it's a little one-sided. Each of these books is the work of a single person (or brothers in the case of Nutshell). Reading them, I've often wanted to give present a different point of view - or alternatively, to give a hearty "hear, hear." I believe that a book giving guidance would benefit greatly from being more of a conversation: where the authors all agree on something, that's great; where they differ, it would be good to hear about the pros and cons of various approaches. The reader can then weigh up those factors as they apply to each particular real-world scenario.

Scope

So what would such a book contain? Opinions will vary of course, but I would like to see:

Effective ways of using language features such as lambda expressions, generic type inference (and indeed generics in general), optional parameters, named arguments and extension methods. Assume that the reader knows roughly what C# does, but give some extra details around things like iterator blocks and anonymous functions.
Guidance around class design (in a similar fashion to the FDG, but with more input from others in the community)
Core framework topics (again, assume the basics are understood):

Resource management (disposal etc)
Exceptions
Collections (including LINQ fundamentals)
Streams
Text (including internationalization)
Numeric types
Time-related APIs
Concurrency
Contracts
AppDomains
Security
Performance

I would prefer to avoid anything around the periphery of .NET (WPF, WinForms, ASP.NET, WCF) - I believe those are better handled in different topics.

Obstacles and format

There's one big problem with this idea, but I think it may be a saving grace too. Many of the leading authors work for different publishers. Clearly no single publisher is going to attract all the best minds in the C# and .NET world. So how could this work in practice? Well...

Imagine a web site for the book, paid for jointly by all interested publishers. The web site would be the foremost delivery mechanism for the content, both to browse and probably to download in formats appropriate for offline reading (PDF etc). The content would be edited in a collaborative style obviously, but exactly how that would work is a detail to be thrashed out. If you've read the annotated C# or CLI specifications, they have about the right feel - opinions can be attributed in places, but not everything has a label.

Any contributing publisher could also take the material and publish it as hard copy if they so wished. Quite how this would work - with potentially multiple hard copy editions of the same content - would be interesting to see. There's another reason against hard copy ever appearing though, which is that it would be immovable. I'd like to see this work evolve as new features appear and as more best practices are discovered. Publishers could monetize the web site via adverts, possibly according to how much they're kicking into the site.

I don't know how the authors would get paid, admittedly, and that's another problem. Would this cannibalize the sales of the books listed earlier? It wouldn't make them redundant - certainly not for the Nutshell type of book, which teaches the basics as well as giving guidance. It would hit Effective C# harder, I suspect - and I apologise to Bill Wagner in advance; if this ever takes off and it hurts his bottom line, I'm very sorry - I think it's in a good cause though.

Dream Team

So who would contribute to this? Part of me would like to say "anyone and everyone" in a Wikipedia kind of approach - but I think that practically, it makes sense for industry experts to take their places. (A good feedback/comments mechanism for anyone to use would be crucial, however.) Here's a list which isn't meant to be exhaustive, but would make me happy - please don't take offence if your name isn't on here but should be, and I wouldn't expect all of these people to be interested anyway.

Anders Hejlsberg
Eric Lippert
Mads Torgersen
Don Box
Brad Abrams
Krzysztof Cwalina
Joe Duffy
Vance Morrison
Rico Mariani
Erik Meijer
Don Symes
Wes Dyer
Jeff Richter
Joe and Ben Albahari
Andrew Troelsen
Bill Wagner
Trey Nash
Mark Michaelis
Jon Skeet (yeah, I want to contribute if I can)

I imagine "principal" authors for specific topics (e.g. Joe Duffy for concurrency) but with all the authors dropping in comments in other places too.

Dream or reality?

I have no idea whether this will ever happen or not. I'd dearly love it to, and I've spoken to a few people before today who've been encouraging about the idea. I haven't been putting any work into getting it off the ground - don't worry, it's not been delaying the second edition of C# in Depth. One day though, one day...

Am I being hopelessly naïve to even consider such a venture? Is the scope too broad? Is the content valuable but not money-making? We'll see.

Tricky decisions... Code Contracts and Parallel Extensions in C# in Depth 2nd edition

skeet — Wed, 05 Aug 2009 16:16:15 GMT

I'd like some feedback from readers, and I suspect my blog is the simplest way to get it.

I'm currently writing chapter 15 of C# in Depth, tentatively about Code Contracts and Parallel Extensions. The problem is that I'm 15 pages in, and I haven't finished Code Contracts yet. I suspect that with a typesetter moving the listings around a little it can be shortened a little bit, but I'm still concerned. With the amount I've still got to write, Code Contracts is going to end up at 20 pages and I expect Parallel Extensions may be 25. That makes for a pretty monstrous chapter for non-language features.

I'd like to present a few options:

Keep going as I am, and take the hit of having a big chapter. I'm not going into huge amounts of detail anyway, but the bigger point is to demonstrate how code isn't what it used to be. We're no longer writing a simple series of statements to be executed in order. Code Contracts changes this dramatically with the binary rewriter, and Parallel Extensions adjusts the parallelism, and ironically makes it easier to write asynchronous code as if it were executed sequentially.
Try to whittle the material down to my original target of around 35 pages. This means it'll be a really cursory glance at each of the technologies - I'm unsure of how useful it would be at all at that point.
Don't even claim to give enough information to really get people going with the new technologies, but possibly introduce extra ones as well, such as PostSharp. Build the theme of "you're not writing C# 1 any more" in a stronger sense - zoom back to show the bigger picture while ignoring the details.
Separate them into different chapters. At this point half the new chapters would be non-language features, which isn't great for the focus of the book... but at least they'd be a more reasonable size.
Ditch the chapters from the book completely, possibly writing them as separate chapters to be available as a mini-ebook companion to the book. (We could possibly include them in the ebook version.) This would make the second edition more focused again and possibly give me a bit more space when revising earlier chapters. However, it does mean there'd only be two full-size new chapters for the second edition. (There'll be a new "wrapping up" chapter as well for a sense of closure, but I'm not generally counting that.)

Other suggestions are welcome, of course. I'm not going to claim that we'll end up doing whatever is suggested here, but I'm sure that popular opinion will influence the final decision.

Thoughts?

Evil Code of the Day: variance and overloading

skeet — Mon, 13 Jul 2009 16:10:00 GMT

(Note that this kind of breakage was mentioned a long time ago in Eric Lippert's blog, although not in this exact form.)

Whenever a conversion becomes available where it wasn't before, overload resolution can change its behaviour. From C# 1 to C# 2 this happened due to delegate variance with method group conversions - now the same thing is true for generic variance for interfaces.

What does the following code print?

using System;
using System.Collections.Generic;

class Base
{
    public void Foo(IEnumerable<string> strings)
    {
        Console.WriteLine("Strings");
    }
}

class Derived : Base
{
    public void Foo(IEnumerable<object> objects)
    {
        Console.WriteLine("Objects");
    }
}

class Test
{
    static void Main()
    {
        List<string> strings = new List<string>();
        new Derived().Foo(strings);
    }
}

The correct answer is "it depends on which version of C# and .NET framework you're using."

If you're using C# 4.0 and .NET 4.0, then IEnumerable<T> is covariant: there's an implicit conversion from IEnumerable<string> to IEnumerable<object>, so the derived overload is used.

If you're using C# 4.0 but .NET 3.5 or earlier then the compiler still knows about variance in general, but the interface in the framework doesn't have the appropriate metadata to indicate it, so there's no conversion available, and the base class overload is used.

If you're using C# 3.0 or earlier then the compiler doesn't know about generic variance at all, so again the base class overload is used.

So, this is a breaking change, and a fairly subtle one at that - and unlike the method group conversion in .NET 2.0, the compiler in .NET 4.0 beta 1 doesn't issue a warning about it. I'll edit this post when there's an appropriate Connect ticket about it...

In general though, I'd say it's worth avoiding overloading a method declared in a base class unless you really have to. In particular, overloading it using the same number of parameters but more general ones seems to be a recipe for unreadable code.