Jon Skeet: Coding Blog

Just how spiky is your traffic?

No, this isn't the post about dynamic languages I promise. That will come soon. This is just a quick interlude. This afternoon, while answering a question on Stack Overflow¹ about the difference between using an array and a Dictionary<string, string> (where each string was actually the string representation of an integer) I posted the usual spiel about preferring readable code to micro-optimisation. The response in a comment - about the performance aspect - was:

Well that's not so easily said for a .com where performance on a site that receives about 1 million hits a month relies on every little ounce of efficiency gains you can give it.

A million hits a month, eh? That sounds quite impressive, until you actually break it down. Let's take a month of 30 days - that has 30 * 24 * 60 * 60 = 2,592,000 seconds². In other words, a million hits a month is less than one hit every two seconds. Not so impressive. At Google we tend to measure traffic in QPS (queries per second, even if they're not really queries - the search terminology becomes pervasive) so this is around 0.39 QPS. Astonished that someone would make such a claim in favour of micro-optimisation at that traffic level, I tweeted about it. Several of the replies were along the lines of "yeah, but traffic's not evenly distributed." That's entirely true. Let's see how high we can make the traffic without going absurd though.

Let's suppose this is a site which is only relevant on weekdays - that cuts us down to 20 days in the month. Now let's suppose it's only relevant for one hour per day - it's something people look at when they get to work, and most of the users are in one time zone. That's a pretty massive way of spiking. We've gone down from 30 full days of traffic to 20 hours - or 20 * 60 * 60 = 72000 seconds, giving 14 QPS. Heck, let's say the peak of the spike is double that - a whopping 28 QPS.

Three points about this:

• 28 QPS is still not a huge amount of traffic.
• If you're really interested in handling peak traffic of ~28 QPS without latency becoming huge, it's worth quoting that figure rather than "a million hits a month" because the latter is somewhat irrelevant, and causes us to make wild (and probably wildly inaccurate) guesses about your load distribution.
• If you're going to bring the phrase "a .com" into the picture, attempting to make it sound particularly important, you really shouldn't be thinking about hosting your web site on one server - so the QPS gets diluted again.
• Even at 28 QPS, the sort of difference that would be made here is tiny. A quick microbenchmark (with all the associated caveats) showed that on my laptop (hardly a server-class machine) I could build the dictionary and index into it 3 times 2.8 million times in about 5 seconds. If every request needed to do that 100 times, then the cost of doing it 28 requests per second on my laptop would still only be 0.5% of that second - not a really significant benefit, despite the hugely exaggerated estimates of how often we needed to do that.

There are various other ways in which it's not a great piece of code, but the charge against premature optimization still stands. You don't need to get every little ounce of efficiency out of your code. Chances are, if you start guessing at where you can get efficiency, you're going to be wrong. Measure, measure, measure - profile, profile, profile. Once you've done all of that and proved that a change reducing clarity has a significant benefit, go for it - but until then, write the most readable code you can. Likewise work out your performance goals in a meaningful fashion before you worry too much - and hits per months isn't a meaningful figure.

Performance is important - too important to be guessed about instead of measured.

¹ I'm not linking to it because the Streisand effect would render this question more important than it really is. I'm sure you can find it if you really want to, but that's not the point of the post.

² Anyone who wants to nitpick and talk about months which are a bit longer or shorter than that due to daylight saving time changes (despite still being 30 days) can implement that logic for me in Noda Time.