I may have joined the wrong side : ASP.NET

What's wrong with ASP.NET? Cultures

calinoiu — Fri, 30 Jun 2006 19:30:00 GMT

The problem

The ability offered by .NET to set a thread-level culture then automatically format and select localizable resources using that culture's settings is wonderful stuff. Unfortunately, it's an approach that plays out quite a bit better in a client-side application than in a server-based application. The reason for this lies in the nature of the work one performs in a server-based application: some formatting and/or rendering is intended for consumption by client applications, but some (e.g.: log entries) is intended for consumption on the server.

Things tend to muddle along just fine as long as both the client and the server use the same or similar cultures. This is helped along by the fact that most servers are unlikely to have additional .NET language packs installed, so any exceptions logged on the server are likely to contain English error messages. However, there are plenty of cases in which it might be necessary to install additional language packs on the server (e.g.: server and/or application administrators don't speak English), so it might be a wee bit naïve for a software vendor to assume that the only .NET language supported on a server is one that will be understood by folks who read text generated for comsumption on the server.

In case this all sounds a bit odd, let's take a look at some concrete examples. Our base scenario is a localizable ASP.NET application that, as is typical, sets the current thread's CurrentCulture and CurrentUICulture properties to match the preferred culture of the client user. Amongst other things, this application happens to create a log entry every time an invoice is created via its UI. e.g.:

	
private void LogOrder(int invoiceId, double price)
{
	string logText = string.Format("Invoice {0:N0} for {1:C} created at {2:G}.",
		invoiceId, price, DateTime.Now);

	// Text written to log here...
}

If the application has set the current thread's CurrentCulture property to the client's preferred culture, here is what will be written to the server-side log for various values of the client culture:

Client culture	Logged text
en-US	Invoice 21,456 for $5,000.00 created at 5/6/2006 9:30:00 AM.
en-CA	Invoice 21,456 for $5,000.00 created at 06/05/2006 9:30:00 AM.
en-GB	Invoice 21,456 for £5,000.00 created at 06/05/2006 09:30:00.
fr-FR	Invoice 21 456 for 5 000,00 € created at 06/05/2006 09:30:00.
fr-CA	Invoice 21 456 for 5 000,00 $ created at 2006-05-06 09:30:00.
es-ES	Invoice 21.456 for 5.000,00 € created at 06/05/2006 9:30:00.
es-MX	Invoice 21,456 for $5,000.00 created at 06/05/2006 09:30:00 a.m..
ja-JP	Invoice 21,456 for ¥5,000 created at 2006/05/06 9:30:00.

The above table shows some of the sorts of trouble that one can get into applying client-side culture options to formatting of server-side texts, and things will only get worse when/if digit substitution is fully implemented. The problem is compounded by the fact that the client-specified culture used to apply formatting is no longer known at the time one reads the server-side text. Of course, formatting as in the above example is a mistake, and there are ways to avoid the problem, but more on that after we look at something a little more difficult to work around.

Like many applications built upon it, the .NET framework emits a variety of texts based on localized resources. The examples with which many developers will likely be most familiar are exception messages. In fact, in most applications, a very large majority of exception messages will have been generated within the .NET Framework base class libraries. Let's say that your application happens to be running on a server with the Hungarian language pack, and an exception is logged while a Hungarian client is using the application. Would you have any idea what the exception message "Nem megfelelő a bemeneti karakterlánc formátuma" means? (It happens to be the Hungarian version of "Input string was not in a correct format", but BabelFish won't exactly be helping you figure that one out...)

To add to the fun, some Framework-generated texts (including exception messages) are automatically formatted from resource strings containing placeholders. Since you cannot control the formatting applied in these instances, you may run into interpretation issues even if you don't have any additional language packs installed. For example, if you see an exception message containing the number "2,345", how can you tell if this was a 2345 written under an American culture option or a 2.345 written under a French Canadian setting? Sometimes context may help, but other times you'll just plain have to guess.

The workaround

So what can we do to avoid the problem? The easy answer is to not use the thread culture as a store for the client's preferred culture. This, of course, means that we would then need to perform explicit formatting of any non-string values before passing them to UI elements for rendering. While somewhat burdensome, particularly for folks who like living the RAD life, this isn't exactly the end of the world. There's also a bit of good news: FxCop can help you identify at least those cases of explicit formatting that are defaulting to using the thread culture.

The first bit of rather bad news with this approach is that you can't use ASP.NET's auto-magical client culture detection since it pops the detected culture into the current thread properties. Things might be a bit different if the ASP.NET team had chosen to make the HttpApplication.SetCulture method virtual, but it's not, so we're stuck implementing our own mechanism from scratch if we want to use an alternate store for the client culture.

The second bit of bad news is that there are several ASP.NET controls that are rather aggressive about using the thread culture internally. For example, when using the CompareValidator with ValidationDataType.Double, it's possible to use culture-specific parsing¹. However, there's no way to specify which culture one wants to use for parsing. Unless you choose to use culture-invariant parsing, the thread culture will be used. That means that a French Canadian user's 5,123 would theoretically be interpreted as 5123 rather than 5.123 if the thread culture is set to en-US because that's the appropriate setting for server-side use. However, things are even worse in practice than in theory, and our 5,123 will fail validation since the CompareValidator does not permit use of thousands group separators and will reject the comma if the thread culture is en-US.

For a fully functional workaround, you'll need to implement at least four sets of changes:

Use an alternate store for the client culture,
Create a mechanism for populating this new store with the client-preferred culture (rather than setting Culture and UICulture attributes to "auto"),
Where possible, pre-format non-string values that will be rendered by controls, and
Implement custom versions of control that take the rendering culture as a property rather than using the thread culture exclusively.

The solution

The workarounds are a pack of trouble, and far more work than most folks would be willing to put in unless they've already been bitten by client vs. server culture bugs. The real solution here is for the ASP.NET platform to properly accomodate separate tracking of client and server cultures. Obviously, items 1 and 2 from the workarounds list above would be a start. In addition, it would probably be a great idea if controls defaulted to using the client culture but allowed easy overriding to use either the server culture or a custom assigned culture.

¹For some bizarre reason, CompareValidator is strictly culture-invariant for some data types (e.g.: ValidationDataType.Integer), which presents its own set of potential problems. However, this isn't directly relevant to the client vs. server culture issue, so I'll drop it for now. However, if you want to complain, connect.microsoft.com is the place.

What's wrong with ASP.NET? HTML encoding

calinoiu — Tue, 13 Jun 2006 19:58:00 GMT

The problem

Back when ASP.NET was first introduced, I had pretty high hopes that the new controls would offer support for automatic HTML encoding. Unfortunately, there was very little of this, and most of it was more than a bit lukewarm (more on this later). In some ways, things have improved a bit in v. 2.0, but they're considerably worse in others.

Before you read any further, you might want to ask yourself which ASP.NET controls perform HTML encoding for you and under what circumstances this is done. If the answer doesn't leap to mind, you've perhaps got a first inkling that there might be a little problem with API consistency and/or the documentation. Then again, maybe you've never worried about HTML encoding in your web applications, in which case I'd strongly recommend that you read up on HTML injection and cross-site scripting. A good starting point might be CERT Advisory CA-2000-02.

We'll look at which controls perform HTML encoding soon. First, we're going to need to nail down some conceptual stuff because not all encoding is created equal. You may already be aware that HTML, URLs, and client-side script use different encodings. For the sake of simplicity, the remainder of this post will refer mainly to HTML encoding, although the other two forms of encoding do merit consideration as well.

There is more than one flavour of HTML encoding, even within ASP.NET. The first is exposed via the System.Web.HttpUtility.HtmlEncode methods. These encode the characters >, <, &, ", as well as any characters with codes between 160 and 255, inclusive. The other main encoding flavour used by ASP.NET is "attribute" encoding, which is exposed via the System.Web.HttpUtility.HtmlAttributeEncode methods. In ASP.NET 1.1., these encode the & and " characters only. In ASP.NET 2.0, these encode the characters <, &, and ".

Attribute encoding ought to be a superset of full HTML encoding that also encodes the single quote character in case that's what happens to be wrapping the attribute. However, as you may have noticed from the above, the ASP.NET version of attribute encoding is even wimpier than its full encoding brother. To make matters worse, the full HTML encoding implemented by ASP.NET is no great shakes in the first place. Security isn't the only reason for HTML encoding, and failure to encode everything outside the low ASCII range can impact on page readability when client browsers don't apply the correct code page (which happens more often than you might think, whether it's the client's or the server's fault).

Now that we know what kinds of HTML encoding are available in ASP.NET, let's take a look at the encoding support offered by the built-in ASP.NET controls. The following table covers some of the more commonly used controls and properties. (There are, of course, many other controls and properties that one might wish to see encoded, but I've tried to keep the list down to things that most folks are likely to use reasonably frequently.)

Control	ASP.NET 1.1	ASP.NET 2.0
Literal	None	None by default. HTML encoded if Mode property is set to LiteralMode.Encode.
Label	None
Button	Text is attribute encoded.
LinkButton	None
ImageButton	Image URL is attribute encoded.	Image URL is URL path encoded then attribute encoded.
HyperLink	Text is not encoded. NavigateUrl is attribute encoded.	Text is not encoded. NavigateUrl is URL path encoded (unless it uses the BLOCKED SCRIPT protocol) then attribute encoded.
TextBox	Single-line text box (input type="text") is attribute encoded. Multi-line text box (textarea) is HTML encoded.
DropDownList and ListBox	Option values are attribute encoded. Option display texts are HTML encoded.
CheckBox and CheckBoxList	Value is not used. Display text is not encoded.
RadioButton and RadioButtonList	Value is attribute encoded. Display text is not encoded.
Table	None
DataGrid	None for text columns. Hyperlink columns follow the pattern for HyperLink controls.
Validators (BaseValidator subclasses) and ValidationSummary	Validator display text is not encoded. For client script, the validator error message and validation summary header text are attribute encoded. When rendering "populated" validators and the validation summary controls from the server, no encoding is applied.	Validator display text is not encoded. For client script, the validator error message and validation summary header text are javascript encoded (blacklisting approach). When rendering "populated" validators and the validation summary controls from the server, no encoding is applied.
HiddenField	N/A	Value is attribute encoded.
GridView and DetailsView	N/A	Text fields HTML encode if their HtmlEncode property is set to true. (This is the default, which is also used for auto-generated columns.) However, the null display text for text fields is not encoded even if the field's HtmlEncode is set to true. Hyperlink fields follow the pattern for HyperLink controls.

Assuming you've actually taken the time to read the above, you might have noticed that there are five basic patterns of encoding usage:

No encoding ever applied.
HTML and/or attribute encoding, as appropriate (with a bit of additional URL and/or javascript encoding applied when appropriate), applied all the time.
Attribute encoding applied for attributes, but no encoding applied for other text.
Optional encoding set via a boolean property, defaulting to applying the encoding.
Optional encoding set via an enumerated property, defaulting to not applying the encoding.

If this strikes you as perhaps a wee bit inconsistent, you wouldn't be alone. Wouldn't it be great to see a consistent approach that telegraphs well and acts as a pit of success? If all the controls performed HTML encoding by default but allowed overriding when necessary (preferably via a single approach), the vast majority of developers writing for ASP.NET would end up generating a more secure, more reliable applications with considerably less effort.

Workarounds

While we're all waiting around for the ASP.NET team to eventually provide reasonable built-in support for HTML encoding, what can we do to ensure that our apps are both protected from HTML injection and character mis-rendering? A good starting point would be to fully encode all data (i.e.: anything not 100% known at compile time, and even some stuff that is) that will be pushed to the client browser. Unfortunately, as was already mentioned above, the built-in encoding scheme leaves a little something to be desired. Luckily, the ACE team folks at Microsoft have been working on a couple of tools that take a more robust approach to HTML (and URL and script) encoding. Rather than blacklisting a fixed set of potentially problematic characters for encoding, they whitelist a set of known safe characters (low ASCII a-z, A-Z, 0-9, space, period, comma, dash, and underscore for HTML encoding) and encode everything else. This quite nicely takes care of both security and appearance issues, and you may wish to seriously consider using this approach rather than calling System.Web.HttpUtility.HtmlEncode to perform your HTML encoding.

Regardless of which HTML encoding approach you select, you're quickly going to run into a bit of trouble with double encoding if you simply start assigning pre-encoded text to control properties (e.g.: someTextBox.Text = HttpUtility.HtmlEncode(someString)). When dealing with malicious input, this is pretty much a non-issue. However, not all data that ought to be encoded is malicious, and you usually wouldn't want users seeing stuff like a > b rather than a > b. Unfortunately, if we want to avoid double encoding in the set of controls that perform non-overrideable encoding (including attribute encoding), we need to use custom controls. To make matters worse, it can require rather a lot of work to subclass most of the controls in order to override the encoding behaviour. In quite a few cases, simply starting from scratch would probably make more sense than trying to subclass the built-in controls. Also, even for those controls where double encoding wouldn't be an issue (e.g.: Label, CheckBox), it's probably worth considering using custom controls anyway since the pain of authoring the custom control isn't likely to outweigh the cumulative effort of all the manual encoding calls you might make across all your projects.

Don't like these workarounds? Maybe it's time to start complaining...

What’s wrong with ASP.NET? Validation

calinoiu — Fri, 09 Jun 2006 16:20:00 GMT

ASP.NET introduced a fancy new user input validation framework that, at least at first glance, appears to be a great advance over the complete lack of built-in validation support in ASP.OLD. Declarative validation is certainly wonderful stuff, and getting client-side validation with no additional effort (at least if your clients are using supported browsers) isn’t too shabby either. Overall, using the built-in validation controls certainly seems like a good idea, particularly for those folks who wouldn’t be performing any validation otherwise because of the amount of work involved.

But what about those of us who had been performing validation all along? Do the ASP.NET validation controls really offer equivalent protection to our former "manual" validation? Unfortunately, for many of us, the answer is probably "no". The main problem lies with the validation paradigm chosen for ASP.NET, which has the following properties:

Each validation control operates independently of the others.
The validation process does not output a validated value.

So what’s wrong with that, you ask? Well, each of the validation controls needs to re-read the target control value, as does the code that will ultimately process that value. To make the problem a bit more concrete, let’s look at the example of validating a birth date provided via a text box:

	Validation rule	Validation control
1	Value is required.	RequiredFieldValidator
2	Value must be parseable to a valid date value without any time portion.	CompareValidator
3	Date may not be in the future or more than 130 years in the past.	CompareValidator
4	If the value indicates an age of less than 10 years or greater than 90 years, the user must provide confirmation that the value is correct (as opposed to a data entry error).	Custom control for soft validation

In the above example, each validation control reads the text box value independently, which means that they cannot build upon parsing and restriction steps performed by the other validators. In addition, when our code needs to process the birth date, it must go back to the text box and read the string value again. That mean that there are at least 5 reads and 4 separate parsing operations from the original text box value, each of which represents an opportunity to parse inconsistently with respect to culture and/or format. In addition, there’s no guarantee that each of those 5 reads is even accessing the same value since it’s possible for the text box’s Text property to be altered between the reads (even if this is somewhat unlikely in an ASP.NET app).

How can we address this problem? There are plenty of workarounds, including using only custom validation controls, but that means quite a bit more work for developers. A much more reliable approach would involve reading the "raw" value from the text box only once. It would then be passed from one validator to another in a pre-defined order and would only require a single parsing. In the birth date example, the validator at step 2 would output a strongly typed DateTime value (assuming, of course, that step 2 validation passed), and that’s the value that would be passed to the validator at step 3. The output from step 5 would be the value that our code would read, rather than going back to the text box to read its Text property.

It’s entirely feasible to develop such a framework, but it’s also quite a bit of a lot of work, particularly when one considers addition of appropriate design-time support. It’s obviously not the sort of thing on which most shops would want their developers spending time. Unfortunately, it’s also not the sort of thing that many shops would be willing to pay for as a third-party product, particularly given that most developers probably perceive themselves as getting by just fine with the existing ASP.NET validation controls.

On the other hand, it would probably be a reasonably small project on the Microsoft scale of things, and I suspect that I’m not the only one who would sleep better if the built-in tools would help the Morts of this world develop more reliable and secure applications out of the box, particularly given that most of us use applications built on those tools sooner on a more or less regular basis.

What’s wrong with ASP.NET?

calinoiu — Fri, 09 Jun 2006 16:18:00 GMT

For quite some time now, I’ve been harbouring an increasing bit of frustration with ASP.NET. Overall, I like the platform, and I think that it’s a great advance over ASP.OLD. Unfortunately, there are a few areas in which I can’t help but feel that the design team missed the boat by just a wee bit too much, and compensating for these lacunae can mean a ridiculous amount of work for the individual developer.

There are three main areas that have been grating on my nerves of late:

User input validation
HTML encoding
Culture usage

I’d love to see the ASP.NET team address fundamentals like these in upcoming versions, but new features generally garner considerably more interest amongst users than fixing things that very few people ever realized were broken. In an attempt to perhaps rouse a bit of interest in the latter, I’m starting a short series of posts on the above topics, starting with What’s wrong with ASP.NET? Validation.