Why boxing doesn't keep me awake at nights

re: Why boxing doesn't keep me awake at nights

Konrad — Tue, 04 Nov 2008 20:54:18 GMT

Where's the upvote button? ;-)

This is the kind of benchmarks that I like to read: actually comparing different factors that might have an influence. This can definitely serve as a reference for future discussions.

re: Why boxing doesn't keep me awake at nights

skeet — Mon, 03 Nov 2008 07:48:41 GMT

@Anon: Generics has *far* bigger benefits in terms of exressiveness than the benefit of avoiding boxing. (Note that Java introduced generics without the benefits that C# has in terms of performance.)

I doubt that many developers *really* hit a "roadblock" due to boxing. They may have hit a conceptual roadblock, but rarely a performance one. Yes, it hurts performance - but rarely enough to actually stop you making progress.

re: Why boxing doesn't keep me awake at nights

Anon — Mon, 03 Nov 2008 01:42:18 GMT

Apparently though C# language designers felt that boxing was a big enough issue to introduce generics. Maybe boxing doesn't keep you awake at night (anymore) but it used to, at least a little.

Boxing is important, and many developers hit a roadblock because they don't understand it.

String concatenations instead of string builders still keep me up at night too.

re: Why boxing doesn't keep me awake at nights

Matt — Mon, 20 Oct 2008 13:40:27 GMT

Hello all,

I wrote a performance test to see what kind of results I would get. On my machine I find the overhead of boxing to be negligable (on average 20ms when performing 1 million boxing operations).

I thought I'd share my code listing and open myself up to any comments... I'd also like to know if anyone knows if there are any compiler performance optimisations being enforced that are skewing my results.

Let me know what you guys think. :)

Thanks,

Matt

---------------

//define how many boxing operation we will perform

const int LOOPS = 1000000;

//an object to box to

static object testObject;

//an object to not box to

static double testDouble;

//a random generator

static Random random = new Random();

/// <summary>

/// Gets a list of random numbers, so we can use random numbers but move their

/// overhead to a non critical execution time.

/// </summary>

private static List<double> getRandomNumbers()

{

//create a list to store the results in

List<double> result = new List<double>();

//create enough random numbers for the box test

for (int loop = 0; loop < LOOPS; loop++)

{

//record the numbers

result.Add(random.NextDouble());

}

//return our random numbers

return result;

}

/// <summary>

/// Performs a test to determine the overhead caused by boxing.

/// </summary>

static double boxTest()

{

//get a list of random numbers

var randomNumbers = getRandomNumbers();

//record the start time

var start = DateTime.Now;

//go through all the random numbers

foreach(var randomNumber in randomNumbers)

{

//perform an assignment without boxing

testDouble = randomNumber;

}

//record the duration of the "non boxing" example

var nonBoxDuration = DateTime.Now - start;

//reset the start time

start = DateTime.Now;

//go through all the random numbers

foreach (var randomNumber in randomNumbers)

{

//perform an assignment with boxing

testObject = (object)randomNumber;

}

//record the duration of the boxing example

var boxDuration = DateTime.Now - start;

//return the difference in milliseconds

return boxDuration.TotalMilliseconds - nonBoxDuration.TotalMilliseconds;

}

/// <summary>

/// Performs the box test multiple times and performs a list of the results in milliseconds.

/// </summary>

private static List<double> getBoxTestResults()

{

//make a list to record the results

List<double> results = new List<double>();

//perform the box test 1000 times and record the result

for (int loop = 0; loop < 100; loop++)

{

//add the box test result to our list of results

results.Add(boxTest());

}

//return the results

return results;

}

/// <summary>

/// The Main entry point of the program.

/// </summary>

static void Main(string[] args)

{

//get the box test results

var boxTestResults = getBoxTestResults();

//get the maximum difference (ms)

var max = boxTestResults.Max();

//get the minimum difference (ms)

var min = boxTestResults.Min();

//get the average difference (ms)

var average = boxTestResults.Average();

//output the results

Console.WriteLine("Max: {0}, Min: {1}, Avg: {2}", max, min, average);

}

re: Why boxing doesn't keep me awake at nights

Matthew Jones — Mon, 20 Oct 2008 08:47:35 GMT

@Granville, I personally do not have time to read many books. I tend to spend time skimming topics online, and then interrogating ones I find of interest further. If I do manage to find time to read an educational text I certainly wouldn't then seek to read another 5 on exatly the same subject just to see if they all correlate. When one's time is at such a premium it is simply not possible to validate every morcel of information. When I attend a seminar or purchase an educational text I am paying for a service that I expect to deliver accurate information.

I do concur that readers should try to take things with a pinch of salt when reading around a subject. When dealing with a topic as subjective as performance the author should provide some quantative information to back up his or her claims. This will aid the reader by providing a context against which an opinion can be formed.

Why should I use List and not ArrayList

Helper Code — Fri, 10 Oct 2008 11:34:14 GMT

I came upon a questing at StackOverflow : The system I work on here was written before .net 2.0 and didn't

re: Why boxing doesn't keep me awake at nights

skeet — Thu, 09 Oct 2008 22:20:16 GMT

Ben: Absolutely. It should have been called "SINGLE_BOX" or something like that. I noticed it after I'd already done everything else, I'm afraid :(

re: Why boxing doesn't keep me awake at nights

Ben Voigt [C++ MVP] — Thu, 09 Oct 2008 21:59:23 GMT

Ummm, I think the _NO_BOXING version is misnamed. Surely this line:

object o = i;

involves boxing?

Of course the new C++0x variable template typeargs will put an end to this discussion for a while (because all the ToString calls inside Format can be statically bound to the correct version in the C++0x version and even inlined). This will give the first version the performance of the last one. Well, maybe only the FORMAT_STRINGS version, still over 3x faster.

re: Why boxing doesn't keep me awake at nights

Granville Barnett — Thu, 09 Oct 2008 17:25:37 GMT

@David - I look at books as being a catalyst for further investigation.

Maybe my approach is this because the books I tend to read are not as clear cut like most programming books are. This is just the way I feel about them in general.

re: Why boxing doesn't keep me awake at nights

David Nelson — Thu, 09 Oct 2008 14:42:10 GMT

@Chris, Granville

Of course you cannot take everything you read in a book as 100% gospel; even good authors sometimes make mistakes and are sometimes just plain wrong.

But at the same time, part of the reason that developers read books is that there is way more out there to learn than they could ever possibly learn on their own through experimentation. The software world is expanding far too quickly, no more so than in the .NET space. So developers read books by people who are supposedly already experienced in a particular topic as an efficient way of assimilating new information. If the author is wrong, or is exaggerating, then the reader is assimilating WRONG information, which undermines the point of reading the book in the first place. Sure, the reader could go out and verify everything he reads for himself, but then what is the point of reading the book?

That is why it is so important for authors to make every effort to be as accurate as possible; like it or not, what they say is often the first and possibly the last impression that a developer will get on a particular subject, and if they care at all about being positive contributors to the development community, they need to do their best to make sure that impression is the right one.

re: Why boxing doesn't keep me awake at nights

skeet — Thu, 09 Oct 2008 13:24:03 GMT

Yup, that's the book I'll be getting when it's available in the US. I think they've changed the title though - it used to include the word "Annotated" right there.

I already have the 2.0 edition, signed by Anders, Scott and Peter :)

re: Why boxing doesn't keep me awake at nights

Granville Barnett — Thu, 09 Oct 2008 13:16:12 GMT

I've just realised that its not the the 10th of October yet ;-( Sorry, shows how out of touch I am with days/dates at the moment.

So, Safari has it right now (electronic of course) available but Amazon (US, print) says its not out until the 20th of this month.

re: Why boxing doesn't keep me awake at nights

Granville Barnett — Thu, 09 Oct 2008 13:06:44 GMT

Its called: The C# Programming Language, Third Edition. Was released on October 10th according to Safari. Amazon (US) - www.amazon.com/.../0321562992

The good thing about this edition is that it has annotations in it like the framework design guidelines book. For the most part it seems the same as the other editions bar the new stuff and annotations.

'Annotated C# 3.0 spec' - are we on about the same book here?

re: Why boxing doesn't keep me awake at nights

skeet — Thu, 09 Oct 2008 12:27:41 GMT

@Granville: Which book do you mean by "the updated Anders one"? I'm hoping to get hold of the Annotated C# 3.0 spec when it comes out, but I haven't got it yet.

re: Why boxing doesn't keep me awake at nights

Granville Barnett — Thu, 09 Oct 2008 11:58:19 GMT

> I think there's room for movement on both sides.

Again, I agree. The problem here is that if you did take the advice of what an author said and it was hideously/slightly wrong and when prompted to explain why you did that you simply stated "Well, <INSERT AUTHOR> said that was the best way to go" The onus is surely on the reader to verify what they are reading is correct.

One should always read several books on the same subject if only to gain a more rounded view of the topic in question. Reading one book and formulating an opinion based on one book is not really a valid opinion unless you have a vast amount of knowledge on one of the other subject areas it touches upon.

I know that you have reviewed now several books on C# 3.0 (have you read the updated Anders one? Its pretty good, I find the annotations a nice touch) and I would suggest that your readers read as many of them as they can lay their hands on if only to gain that rounded perspective.

> Obviously it's nigh-on impossible to be perfect, but we can aim high :)

Indeed.

re: Why boxing doesn't keep me awake at nights

skeet — Thu, 09 Oct 2008 11:00:11 GMT

I think there's room for movement on both sides. Unfortunately in this case, there are plenty of books which *don't* contradict each other but all overstate the importance of boxing when it comes to performance (without enough caveats about context, profiling, micro-optimisation etc).

Readers should be alert and ready to check things which sound wrong - but there are quite a few things which don't actually sound wrong but *are* wrong. (For instance, a writer describing some C# behaviour as if it matched Java when it didn't - that would sound right to a reader familiar with Java, so they may readily believe it, even if reality is different.)

In terms of efficiency, I think it makes a lot more sense for the author to get it right though - that's only one person (or the author + tech reviewers) checking, rather than potentially thousands of readers. Obviously it's nigh-on impossible to be perfect, but we can aim high :)

re: Why boxing doesn't keep me awake at nights

Granville Barnett — Thu, 09 Oct 2008 10:49:42 GMT

I totally agree that authors have a responsibility for being accurate, totally agree BUT maybe it is the reader who ought to change their stance then?

Throughout I have always viewed books as merely an aid, I don't take it word for word. I like to verify things myself (much like you have done here). I think the reason I take this stance, particularly with programming books is that many contradict one another.

I've always felt that the burden is on the reader. Sure you can appreciate what someone else is telling you but its best to come to your own conclusions.

That's my two pence on the subject ;-)

re: Why boxing doesn't keep me awake at nights

skeet — Thu, 09 Oct 2008 09:45:46 GMT

Well, my post is aimed at a number of things - people searching for data, people interested in books and accuracy, and also as a place I can direct people if they start referring to books with examples of boxing as if they're gospel truth.

And yes, I'll certainly be contacting Jeff before my review (which will be a while anyway). The errata sheet is going to be relatively bare though, if it keeps going in its current form...

re: Why boxing doesn't keep me awake at nights

Chris Nahr — Thu, 09 Oct 2008 09:28:23 GMT

So your benchmarks were aimed at people who might find this page through Google when they are searching for "C# boxing performance" or the like? That makes much more sense, I didn't realize your intention here.

Regardless of the general confusion about .NET performance issues, I still don't think there's any great danger of readers misunderstanding "CLR via C#" in particular, since it's aimed at an advanced audience that would likely understand the context and intention of the example. But perhaps Richter would agree to insert a disclaimer like the one you suggest in the next edition -- I presume you'll contact him, as usual?

re: Why boxing doesn't keep me awake at nights

skeet — Thu, 09 Oct 2008 08:12:25 GMT

@Chris: You could certainly blame the reader for naivety, but I think there's a duty for the author to pick examples which actually make their point accurately.

Even if it's easy to see what was meant, there can be little doubt that the book is *just plain wrong* when it claims that "The second version executes much faster."

Further, slips like this reinforce the micro-optimisation mentality which can be so damaging to readability. Suppose the book had made the point like this: "The first version boxes three times, which is wasteful - but probably irrelevant compared with the string formatting and console output. Be aware of performance, but keep a sense of context." That would have been great! But no, instead a frankly alarmist tone was taken. I probably wouldn't have gone to the trouble of benchmarking it if I hadn't seen the same tone and attitude expressed in numerous forums. People really *don't* take context into account nearly enough. If, by writing this blog post, just a handful of developers adopt a more contextual view of performance, I'll be very happy. (Obviously I'll never know, but there we go...)

I agree that Jeff Richter is bound to be aware of the dominant factors here - so the question remains as to why he picked an example which didn't actually demonstrate his point.

I do want to stress how good the rest of the book is, btw. This is a relatively isolated incident *in this book* but it's a symptom of a wider problem, and it happened to be a particularly simple claim to disprove. I started off just with the two options presented in the book - but then became interested in exactly where the rest of the time was going, hence the other options.