Tales from the Crypto

So I deleted it without reading it.

A golden rule of performance improvement

The Rule: Performance optimizations are not worth making for anything less than 10% improvement in speed.

Corollary: Performance optimizations must be measured before and after, and changes reverted if they do not cause significant performance improvement.

Converse: If you are pushing back on implementing a feature “because it will make the app unbearably slow”, particularly if that feature is deemed a security requirement, you had better be able to demonstrate that loss in performance, and it had better be significant.

I come about this insight through years of work as a developer, in which I’ve seen far too many mistakes introduced through people either being “clever” or taking shortcuts – and the chief reason given for both of these behaviours is that the developer was “trying to make the program faster” (or leaner, or smaller).

I have also seen developers disable security features, or insist that they shouldn’t implement security measures (SSL is a common example) because they “will make the application slower”.

I have yet to see a project complete a proper implementation of SSL for security that significantly slowed the application. In many cases, the performance testing that was done to ensure that SSL had no significant effect demonstrated that the bottleneck was already somewhere else.

“Someone should invent a machine to do that”

Lately, I find myself saying this a whole lot.

The conversations in which I’m saying this follow the pattern of these examples:

Me: “Let me understand your problem – tell me why you don’t want to have expiring passwords on this service account?”

Other guy: “Because when we change passwords, we’d have to change the password at the domain controller, then log on to each machine that runs the service, change the passwords, and stop and restart the service on those machines. Then we’d have to take note of which machines had issues with the process, so that we could work on troubleshooting those.”

Me: “Someone should invent a machine to do that … they could call it something like … a komputor. I’m sure they’d sell millions.”

Yes, it’s a little sarcastic when I put it that way, and for the most part that last piece is in my head. It usually comes out as “that could be done by a simple script”, or “can we justify the cost of buying one of the many programs out there that manages that entire process automatically?”

When you take a boring and mundane manual process and turn it into an automated process, whether through scripting or purpose-built applications, you significantly reduce the likelihood of error, and you can take your staff and turn them to other tasks that require human thought and are interesting.

Ten key truths

In the spirit of "ten unavoidable security truths", and numerous other top-ten lists, here's a list of ten key truths that apply to public / private key pairs:

1. Your private key has to be private to you. It cannot be created by anyone else.
2. Anyone who has your private key is you, for the purposes to which that key is applied.
3. If you have a private key that was generated by your employer, then that key identifies you as a part of the employer. It cannot be used to uniquely identify you, because the key was generated under your employer's control.
4. Keys associated with expired or revoked certificates are not always useless - you can use them to decrypt a file that they encrypted a long time ago; you can also verify the time-stamped signature of a document, if the certificate was valid at the time of the signature.
5. A key is a number - it cannot expire, it is the associated certificate that expires. Similarly, the certificate, not the key, is what is revoked after exposure.
6. A key is not a certificate. A certificate is a statement (usually of ownership) about a key pair, and contains the public key.
7. Too short of a key is no key at all, and an exposed private key is no key at all.
8. Protecting a password or key by encrypting it may do nothing more than extend the problem by a level – now you have to protect the key used to encrypt the password / key.
9. Revocation of a certificate applies only to trusting parties who actually bother to check for revocation.
10. A key derived from a password has the same strength as the password, no matter how long the key.

Note that this list describes what happens when cryptography is working perfectly. There are other key facts that apply to broken cryptography and broken process:

1. You cannot increase entropy by repeating bits, or by padding with constant or predictable values.
2. If someone creates a private key for you, they can become you using that key.
3. Protecting anything by using a password or key to feed a yes/no gate requires the attacker to only fool the yes/no gate. Encrypted USB drive manufacturers found this out recently to their embarrassment.
4. Inventing your own crypto – or the processes around it – is always a bad idea. Research others’ ideas and use them, particularly published standards.
5. Even the longest strongest key in the world can be defeated by a big enough bribe to the key-holder.
   1. If the price of a card number on the black market is $1, then access to a million card numbers is equivalent to a million dollars. Who do you pay well enough that they can ignore that value?
6. A key or password that is never updated as it passes through many hands is vulnerable to abuse by each of those hands.
7. Broken cryptographic algorithms cannot be made better by increasing the length of the key or by applying the algorithm more times.
8. All cryptographic algorithms become broken in time. The trick is to choose one that lasts longer than your need to protect your data.
9. When you lose the private key through insufficient key management, you have lost the data it protected.
10. Even a good cryptographic algorithm can be destroyed by a poor key-generation algorithm.

Note that these lists are rather arbitrarily scoped to ten – there may be more important truths I’ve forgotten, or items I’ve included that aren’t really so important.

Corrections to Thierry Zoller’s Whitepaper

Thanks to Thierry Zoller for mentioning me in the FTP section of his whitepaper summary of the TLS renegotiation attacks on various protocols. I’m glad he also spells my name right – you’d be surprised how many people get that wrong, although I’m sure Thierry gets his own share of people unable to spell his name.

The whitepaper itself contains some really nice and simple documentation of the SSL MITM renegotiation attack, and how it works. It’s well worth reading if you’re looking for some insight into how this works.

First, though, a couple of corrections to Thierry’s summary – while he’s working on revising his whitepaper, I’ll post them here:

• The name of my FTP server is “WFTPD Server”, or “WFTPD Pro Server”, as you will see from http://www.wftpd.com. Of the two, only the WFTPD Pro Server has FTP over TLS capabilities.
• SFTP is not “FTP over SSH”. SFTP is a completely different protocol – a sub-protocol of SSH. For example, where FTP uses commands of three or four letters, SFTP uses binary single byte instructions. There is no one-to-one mapping between SFTP commands and FTP commands, and there are many other differences that aren’t really worth going into.
• I don’t see the use of CCC as a reason to “use SFTP over FTPS” – I see it as a reason to not use FTPS clients that require, or FTPS servers that support, the CCC command. There are better ways to surmount the NAT problem in FTPS – the best is IPv6, but even in IPv4, the use of block-mode FTP over the default data channel removes NATs from being a problem for FTPS.
• The language in the “Client Certificate Authentication” FTPS section appears to be an incomplete sentence. What I think he is trying to say is that when authentication is performed in FTPS, it resets the command state, so that commands entered prior to authentication, if they are executed at all, are executed in the context that existed at that time, rather than the newly-authenticated context.

I think that where FTPS has problems that are thrown into sharp relief with the SSL MITM renegotiation attacks I’ve been discussing for a while now, it has had those problems before. If an attacker can monitor and modify the FTP control channel (because the client requested CCC and the server allowed it), the attacker can easily upload whatever data they like in place of the client’s bona fide upload.

The renegotiation attack simply makes it easier for the attacker to hide the attack. It’s the use of CCC which facilitates the MITM attack, far more than the renegotiation does.

To address one further comment I’ve heard with regard to SSL MITM attacks, I hear “yeah, but getting to be a man-in-the-middle is so difficult anyway, that even a really simple attack is unlikely”. That’s a true comment – for the most part, there is little chance of a man-in-the-middle attack occurring on the general Internet in a bulk situation. The ‘last mile’ of home wireless, coffee bars and other public wireless hangouts, or the possibility of DNS hijacking, HOSTS file editing, broadband router hacking, or just plain viruses and worms, are the place where most man-in-the-middle entry points exist.

However, if you’re going to assert that it’s truly unlikely that an attacker can insert himself into your network stream, you basically have no reason whatever to use SSL / TLS – without a potential for that interception and modification of your traffic, there’s really no need to authenticate it, encrypt it, or monitor its integrity along the path.

The fact that a protocol or application uses SSL /TLS means that it tacitly assumes the existence of a man in the middle. If SSL / TLS allows a man-in-the-middle attack at all, it fails in its basic raison d’etre.

Next post, I promise something other than SSL renegotiation attacks.

My take on the SSL MITM Attacks – part 3 – the FTPS attacks

[Note - for previous parts in this series, see Part 1 and Part 2.]

FTP, and FTP over SSL, are my specialist subject, having written one of the first FTP servers for Windows to support FTP over SSL (and the first standalone FTP server for Windows!)

Rescorla and others have concentrated on the SSL MITM attacks and their effects on HTTPS, declining to discuss other protocols about which they know relatively far less. OK, time to step up and assume the mantle of expert, so that someone with more imagination can shoot me down.

FTPS is not vulnerable to this attack.

No, that’s plainly rubbish. If you start thinking along those lines in the security world, you’ve lost it. You might as well throw in the security towel and go into a job where you can assume everybody loves you and will do nothing to harm you. Be a developer of web-based applications, say. :-)

FTPS has a number of possible vulnerabilities

And they are all dependent on the features, design and implementation of your individual FTPS server and/or client. That’s why I say “possible”.

Attack 1 – renegotiation with client certificates

The obvious attack – renegotiation for client certificates – is likely to fail, because FTPS starts its TLS sessions in a different way from HTTPS.

In HTTPS, you open an unauthenticated SSL session, request a protected resource, and the server prompts for your client certificate.

In FTPS, when you connect to the control channel, you provide your credentials at the first SSL negotiation or not at all. There’s no need to renegotiate, and certainly there’s no language in the FTPS standard that allows the server to query for more credentials part way into the transaction. The best the server can do is refuse a request and say you need different or better credentials.

Attack 2 – unsolicited renegotiation without credentials

A renegotiation attack on the control channel that doesn’t rely on making the server ask for client credentials is similarly unlikely to succeed – when the TLS session is started with an AUTH TLS command, the server puts the connection into the ‘reinitialised’ state, waiting for a USER and PASS command to supply credentials. Request splitting across the renegotiation boundary might get the user name, but the password wouldn’t be put into anywhere the attacker could get to.

Attack 3 – renegotiating the data connection

At first sight, the data connection, too, is difficult or impossible to attack – an attacker would have to guess which transaction was an upload in order to be able to prepend his own content to the upload.

But that’s betting without the effect that NATs had on the FTP protocol.

Because the PORT and PASV commands involve sending an IP address across the control channel, and because NAT devices have to modify these commands and their responses, in many implementations of FTPS, after credentials have been negotiated on the control channel, the client issues a “CCC” command, to drop the control channel back into clear-text mode.

Yes, that’s right, after negotiating SSL with the server, the client may throw away the protection on the control channel, so the MitM attacker can easily see what files are going to be accessed over what ports and IP addresses, and if the server supports SSL renegotiation, the attacker can put his data in at the start of the upload before renegotiating to hand off to the legitimate client. Because the client thinks everything is fine, and the server just assumes a renegotiation is fine, there’s no reason for either one to doubt the quality of the file that’s been uploaded.

How could this be abused? Imagine that you are uploading an EXE file, and the hacker prepends it with his own code. That’s how I wrote code for a ‘dongle’ check in a program I worked on over twenty years ago, and the same trick could still work easily today. Instant Trojan.

There are many formats of file that would allow abuse by prepending data. CSV files, most exploitable buffer overflow graphic formats, etc.

Attack 3.5 – truncation attacks

While I’m on FTP over SSL implementations and the data connection, there’s also the issue that most clients don’t properly terminate the SSL connection in FTPS data transfers.

As a result, the server can’t afford to report as an error when a MitM closes the TCP connection underneath them with an unexpected TCP FIN.

That’s bad – but combine it with FTP’s ability to resume a transfer from part-way into a file, and you realize that an MitM could actually stuff data into the middle of a file by allowing the upload to start, interrupting it after a few segments, and then when the client resumed, interjecting the data using the renegotiation attack.

The attacker wouldn’t even need to be able to insert the FIN at exactly the byte mark he wanted – after all, the client will be sending the REST command in clear-text thanks to the CCC command. That means the attacker can modify it, to pick where his data is going to sit.

Not as earth-shattering as the HTTPS attacks, but worth considering if you rely on FTPS for data security.

How does WFTPD Pro get around these attacks?

1. I never bothered implementing SSL / TLS renegotiation – didn’t see it as necessary; never had the feature requested. Implementing unnecessary complexity is often cause for a security failure.

2. I didn’t like the CCC command, and so I didn’t implement that, either. I prefer to push people towards using Block instead of Stream mode to get around NAT restrictions.

I know, it’s merely fortunate that I made those decisions, rather than that I had any particular foresight, but it’s nice to be able to say that my software is not vulnerable to the obvious attacks.

I’ve yet to run this by other SSL and FTP experts to see whether I’m still vulnerable to something I haven’t thought of, but my thinking so far makes me happy – and makes me wonder what other FTPS developers have done.

I wanted to contact one or two to see if they’ve thought of attacks that I haven’t considered, or that I haven’t covered. So far, however, I’ve either received no response, or I’ve discovered that they are no longer working on their FTPS software.

Let me know if you have any input of your own on this issue.

My take on the SSL MitM Attacks – part 2 – clarifications

Since the last post I made on the topic of SSL renegotiation attacks, I’ve had a few questions in email. Let’s see how well I can answer them:

Q. Some stories talk about SSL, others about TLS, what’s the difference?

A. For trademark reasons, when SSL became an open standard, it had to change its name from SSL to TLS. TLS 1.0 is essentially SSL 3.1 – it even claims to be version “3.1” in its communication. I’ll just call it SSL from here on out to remind you that it’s a problem with SSL and TLS both.

Q. All the press coverage seems to be talking about HTTPS – is this limited to HTTPS?

A. No, this isn’t an HTTPS-only attack, although it is true that most people’s exposure to SSL is through HTTPS. There are many other protocols that use SSL to protect their connections and traffic, and they each may be vulnerable in their own special ways.

Q. I’ve seen some posts saying that SSH and SFTP are not vulnerable – how did they manage that?

A. Simply by being “not SSL”. SFTP is a protocol on top of SSH, and SSH is not related to SSL. That’s why it’s not affected by this issue. Of course, if there’s a vulnerability discovered in SSH, it’ll affect SSH and SFTP, but won’t affect SSL or SSL-based protocols such as HTTPS and FTPS.

Q. Is it OK to disable SSL renegotiation to fix this bug?

A. Obviously, if SSL didn’t need renegotiation at all, it wouldn’t be there. So, in some respects, if you disable SSL renegotiation, you may be killing functionality. There are a few reasons that you might be using SSL renegotiation:

1. Because that’s how client authentication works – while you can do client authentication without renegotiation, most HTTPS implementations use renegotiation to request the client certificate. Disabling renegotiation will generally prevent most clients from authenticating with client authentication.
2. After 10 hours, renegotiation is required, so as to refresh the session key. Do you have SSL connections lasting 10 hours? You probably should be looking at some disconnect/reconnect scenario instead.
3. Because you can’t disable SSL renegotiation in all cases. In OpenSSL, you can only disable renegotiation if you download and install the new version, and in other SSL implementations, there is no way to disable renegotiation outside of modifying the application.

Q. Since this attack requires the attacker to become a man-in-the-middle, doesn’t that make it fundamentally difficult, esoteric, or close to impossible?

A. If becoming a man-in-the-middle (MitM) was impossible or difficult, there would be little-to-no need for SSL in the first place. SSL is designed specifically to protect against MitM attacks by authenticating and encrypting the channel. If a MitM can alter traffic and make it seem as if everything’s secure between client and server over SSL, then there’s a failure in SSL’s basic goal of protecting against men-in-the-middle.

Once you assume that an attacker can intercept, read, and modify (but not decrypt) the SSL traffic, this attack is actually relatively easy. There are demonstration programs available already to show how to exploit it.

I was asked earlier today how someone could become a man-in-the-middle, and off the top of my head I came up with six ways that are either recently or frequently used to do just that.

Q. Am I safe at a coffee shop using the wifi?

A. No, not really – over wifi is the easiest way for an attacker to insert himself into your stream.

When using a public wifi spot, always connect as soon as possible to a secured VPN. Ironically, of course, most VPNs are SSL-based, these days, and so you’re relying on SSL to protect you against possible attacks that might lead to SSL issues. This is not nearly as daft as it sounds.

Q. Is this really the most important vulnerability we face right now?

A. No, it just happens to be one that I understood quickly and can blather on about. I think it’s under-discussed, and I don’t think we’ve seen the last entertaining use of it. I’d like to make sure developers of SSL-dependent applications are at least thinking about what attacks can be performed against them using this step, and how they can prevent these attacks. I know I’m working to do something with WFTPD Pro.

Q. Isn’t the solution to avoid executing commands outside the encrypted tunnel?

A. Very nearly, yes. The answer is to avoid executing commands sent across two encrypted sessions, and to deal harshly with those connections who try to send part of their content in one session and the rest in a differently negotiated session.

In testing WFTPD Pro out against FTPS clients, I found that some would send two encrypted packets for each command – one containing the command itself, the other containing the carriage return and linefeed. This is bad in itself, but if the two packets straddle either side of a renegotiation, disconnect the client. That should prevent the HTTPS Request-Splitting using renegotiation.

One key behaviour HTTPS has is that when you request a protected resource, it will ask for authentication and then hand you the resource. What it should probably be doing is to ask for authentication and then wait for you to re-request the resource. That action alone would have prevented the client-certificate attacks discussed so far.

Q. What is the proposed solution?

A. The proposed solution, as I understand it, is for client and server to state in their renegotiation handshake what the last negotiated session state was. That way, an interloper cannot hand off a previously negotiated session to the victim client without the client noticing.

Note that, because this is implemented as a TLS handshake extension, it cannot be implemented in SSLv3. Those of you who just got done with mandating SSLv2 removal throughout your organisations, prepare for the future requirement that SSLv3 be similarly disabled.

Q. Can we apply the solution today?

A. It’s not been ratified as a standard yet, and there needs to be some discussion to avoid rushing into a solution that might, in retrospect, turn out to be no better – or perhaps worse – than the problem it’s trying to solve.

Even when the solution is made available, consider that PCI auditors are still working hard to persuade their customers to stop using SSLv2, which was deprecated over twelve years ago. I keep thinking that this is rather akin to debating whether we should disable the Latin language portion of our web pages.

However, it does demonstrate that users and server operators alike do not like to change their existing systems. No doubt IDS and IPS vendors will step up and provide modules that can disconnect unwarranted renegotiations.

Update: Read Part 3 for a discussion of the possible threats to FTPS.

My take on the SSL MITM Attacks – part 1 – the HTTPS attack

If you’re in the security world, you’ve probably heard a lot lately about new and deadly flaws in the SSL and TLS protocols – so-called “Man in the Middle” attacks (aka MITM).

These aren’t the same as old-style MITM attacks, which relied on the attacker somehow pretending strongly to be the secure site being connected to – those attacks allowed the attacker to get the entire content of the transmission, but they required the attacker to already have some significant level of access. The access required included that the attacker had to be able to intercept and change the network traffic as it passed through him, and also that the attacker had to provide a completely trusted certificate representing himself as the secure server. [Note – you can always perform a man-in-the-middle attack if you own a trusted certificate authority.]

The current SSL MITM attack follows a different pattern, because of the way HTTPS authentication works in practice. This means it has more limited effect, but requires less in the way of access. You gain some security advantage, you lose some. The attacker still needs to be able to intercept and modify the traffic between client and server, but does not get to see the content of traffic between client and server. All the attacker gets to do is to submit data to the server before the client gets its turn.

Imagine you’re ordering a pizza over the phone. Normally, the procedure is that you call and tell them what the pizza order is (type of pizza, delivery address), and they ask you for your credit card number as verification. Sometimes, though, the phone operator asks for your credit card number first, and then takes your order. So, you’re comfortable working either way.

Now, suppose an attacker can hijack your call to the pizza restaurant and mimic your voice. While playing you a ringing tone to keep you on the line, he talks to the phone operator, specifying the pizza he wants and the address to which it is to be delivered. Immediately after that, he connects you to your pizza restaurant, you’re asked for your credit card number, which you supply, and then you place your pizza order.

Computers are as dumb as a bag of rocks. Not very smart rocks at that. So, imagine that this phone operator isn’t smart enough to say “what, another pizza? You just ordered one.”

That’s a rough, non-technical description of the HTTPS attack. There’s another subtle variation, in which the caller states his pizza order, then says “oh, and ignore my attempt to order a pizza in a few seconds”. The computer is dumb enough to accept that, too.

For a more technical description, go see Eric Rescorla’s summary at Understanding the TLS Renegotiation Attack, or Marsh Ray’s original report.

Let’s call these the HTTPS client-auth attack and the HTTPS request-splitting attack. That’s a basic description of what they do.

HTTPS client-authentication attack

The client-authentication attack is getting the biggest press, because it allows the attacker one go (per try) at persuading the server to perform an action in the context of the authenticated user. From ordering a pizza to pretty any activity that can be caused in a single request to a web site can be achieved with this attack.

Preventing the attack at the server.

Servers have been poorly designed in this respect – but out of some necessity. Eric Rescorla explains this in the SSL and TLS bible, “SSL and TLS” [Subtitle: Designing and Building Secure Systems] on page 322, section 9.18.

“The commonly used approach is for the server to negotiate an ordinary SSL connection for all clients. Then, once the request has been received, the server determines whether client authentication is required… If it is required, the server requests a rehandshake using HelloRequest. In this second handshake, the server requests client authentication.”

How does HTTP handle other authentication, such as Forms, Digest, Basic, Windows Integrated, etc? Is it different from the above description?

A client can provide credentials along with its original request using the WWW-Authenticate header, or the server can refuse an unauthorised (anonymous) request with a 401 error code indicating that authentication is necessary (and listing WWW-Authenticate headers containing appropriate challenges). In the latter case, the client resends the request with the appropriate WWW-Authenticate header.

HTTPS Mutual Authentication (another term for client authentication) doesn’t do this. Why on earth not? I’m not sure, but I think it’s probably because SSL already has a mostly unwarranted reputation for being slow, and this would add another turnaround to the process.

Whatever the reason, a sudden dose of unexpected ‘401’ errors would lead to clients failing, because they aren’t coded to re-request the page with mutual auth in place.

So, we can’t redesign from scratch to fix this immediately – how do we fix what’s in place?

The best way is to realise what the attack can do, and make sure that the effects are as limited as possible. The attack can make the client engage in one action – the first action it performs after authenticating – using the credentials sent immediately after requesting the action to be performed.

A change of application design is warranted, then, to ensure that the first thing your secure application does on authenticating with a client certificate is to display a welcome screen, and not to perform an action. Reject any action requested prior to authentication having been received.

Sadly, while this is technically possible using SSL if you’ve written your own server to go along with the application, or can tie into information about the underlying SSL connection, it’s likely that most HTTPS servers operate on the principle that HTTP is stateless, and the app should have no knowledge of the SSL state beyond “have I been authenticated or not”.

Doubtless web server vendors are going to be coming out with workarounds, advice and fixes – and you should, of course, be looking to their advice on how to fix this behaviour.

The best defence against the client-authentication attack, of course, is to not use client authentication.

Preventing the attack at the client

Not much you can do here, I’m afraid – the client can’t tell if the server has already received a request. Perhaps it would work to not provide client certificates to a server unless you already have an existing SSL connection, but that would kill functionality to perfectly good web sites that are operating properly. Assuming that most web sites operate in the mode of “accept a no-client-auth connection before requesting authentication”, you could rework your client to insist on this happening all the time. Prepare for failures to be reported.

Again, the best defence is not to use client authentication right now. Perhaps split your time between browsers – one with client certificates built in for those few occasions when you need them, and the other without client certs, for your main browsing. That will, at least, limit your exposure.

HTTPS Request-splitting attack

Preventing the attack at the server

The HTTPS Request-splitting attack is technically a little easier to block at the server, if you write the server’s SSL interface – there should be absolutely no reason for an HTTP Request to be split across an SSL renegotiation. So, an HTTPS server should be able to discard any connection state, including headers already sent, when renegotiation happens. Again, consult with your web server developer / vendor for their recommendations.

Preventing the attack at the client?

Again, you’re pretty much out of luck here – even sending a double carriage return to terminate any previous request would cause the attacker’s request to succeed.

The long term approach – fix the protocol

As you can imagine, there are some changes that can be made to TLS to fix all of this. The basic thought is to have client and server add a little information in the renegotiation handshake that checks that client and server both agree about what has already come before in their communication. This allows client and server both to tell when an interloper has added his own communication before the renegotiation has taken place.

Details of the current plan can be found at draft-rescorla-tls-renegotiate.txt

Final thoughts

Yeah, this is a significant attack against SSL, or particularly HTTPS. There are few, if any, options for protecting yourself as a client, and not very many for protecting yourself as a server.

Considering how long it’s taken some places to get around to ditching SSLv2 after its own security flaws were found and patched 14 years ago with the development of SSLv3 and TLS, it seems like we’ll be trying to cope with these issues for many years to come.

Like it or not, though, the long-term approach of revising TLS is our best protection, and it’s important as users that we consider keeping our software up-to-date with changes in the security / threat landscape.

Update: read Part 2 of this discussion for answers to a number of questions.

Update: read Part 3 for some details on FTPS and the potential for attacks.

Why .NET apps keep crashing on your Tablet PC

I’ve been struggling with this issue for some time.

I have a small, simple .NET application I wrote in Visual C# a few months ago – I’ve tentatively titled it “iFetch”, because it fetches radio shows from the BBC iPlayer.

It really is very little more than a simple data grid view that displays the details of the shows and allows users to select them for downloading and later listening.

Despite that, I’ve had some terrible trouble with it. Sometimes it’ll work perfectly, other times it’ll just suddenly crash, and apparently without warning and for different reasons – sometimes when I click on a row, other times when I select to sort on a column heading.

The crash seems to be intermittent, but doesn’t reproduce on other computers; even computers of the same configuration.

For those who want technical details, here we go – the crash is a System.StackOverflowException error, and appears to be due to an unchecked infinite recursion in System.Windows.Forms.dll!System.Windows.Forms.DataGridViewRow.DataGridViewRowAccessibleObject.Bounds.get().

The clue here is that this is a “DataGridViewRowAccessibleObject” – not a mere DataGridViewRow. These “AccessibleObject” versions of common .NET components only come into existence and spread their effect when an “accessibility application” is active on the system. Apparently, in addition to text-to-speech readers, braille devices, etc, a Tablet – whether external like mine, or internal like those in a Tablet PC – classifies as an accessibility application.

That’s why this bug was intermittent for me – sometimes I had my external graphics tablet plugged in, other times I didn’t. To make matters worse, it seems to only trigger when one or more rows in the DataGrid are hidden.

If you get this error, first try checking to see if Microsoft have fixed the flaw – check for .NET service packs – and then, if there is no direct fix for the flaw, try either unplugging your tablet, if you can, or temporarily stop the Tablet PC Input Service, while running the program.

So far, I have received no feedback from Microsoft about when this will be fixed.

Why changing passwords should be done regularly

A little birdie sent me a copy of today’s SANS ISC diary entry. That’s a good thing, because I’m at home sick with alleged piggy flu, and I’m not able to keep up with a whole lot.

The diary entry argues that regular changes of passwords are often done for no other reason than “because we’ve always done it that way”.

Apparently, people responsible for security policy have “read somewhere” that you’re supposed to change passwords every ninety days, and having no other basis on which to proceed, that’s the policy carved in stone.

When asked why this policy is the way it is, the usual response is “good security practice” – and in such environments it’s difficult to give a good response to someone who pushes back, arguing that changing passwords in their application is ‘difficult’ or, more often, ‘expensive’. This is, after all, business, and if one side pleads “expense”, while the other side pleads “good thing to do”, the latter side will lose.

So, why is it best practice?

One reason is that you have to recognise that for all that we tell users not to share their passwords, not to use the same password on multiple sites (aka “share their passwords”), etc, very often users will do exactly that. So, every ninety days, you change your password and you cut off everyone with whom you previously shared your password (to an extent).

Another reason is to allow changes in password policy to propagate out to new passwords. If you suddenly realise that passwords can be easily hacked if they are only six characters, you change the password policy to require punctuation as well, and then you realise that because no one has to change their password, the new policy will never be applied.

Those are the common arguments for regular password changes, and there are a few others, but there’s one I rarely hear being made.

What about when you do get an exposure?

In my professional career, I have seen, or heard of, a number of cases of exposure of password information. Sometimes it’s as simple as a departing employee who knows far too much information and may not be trusted, or as mind-boggling as a team sharing a list of important passwords, and one of the team members losing the list. Other times it’s more complex.

Each time, the response from security is the same – if the existing passwords are in danger of being used because of such exposure, then those passwords need to be changed.

Most times, the response from the business is the same – that the passwords haven’t been changed in so long, and they’re spread through so many different applications, that they have no idea what will be affected if they change the password.

Once you hit that scenario, it can be months before you get the password changed. Yes, months. And all during that time, the account may be compromised.

How do you prevent this?

Think of your disaster recovery drills – when there’s a process that needs to be followed quickly and correctly in an emergency situation, you achieve that by meticulous planning and regular exercise. You create the process and test it regularly, updating the process as you find there’s a need.

If you don’t change passwords on these high-value accounts once every 90 days (or so), how do you know that you’ll be able to change those passwords after an exposure or compromise? How will you guarantee that your password change procedures are current, without testing them? How will you enforce changes being documented if you don’t check the documentation against reality once in a while?

White House moves to Open Source

Subtitle: Media posts uninformed rubbish as commentary

From the MSNBC story “White House opens Web site coding to public”:

"Security is fundamentally built into the development process because the community is made up of people from all across the world, and they look at the source code from the very start of the process until it's deployed and after," said Terri Molini of Open Source for America, an interest group that has pushed for more such programs.

Expecting Open Source to be more secure because the general public contributes to and reviews it is like expecting a televised football match to be safer, because the folks at home are engaged in crowd control and looking for pickpockets.

While you might luck out in finding a few talented, devoted, and dare I say it, obsessed individuals who will call the police every time they see an infraction on screen, most of the people tuning in are going to be watching the game; and those that are trying to help are often clueless about how the security in the grounds works, and you’ll get many calls from people who see the security guards searching bags on entry as pickpockets.

Lots more to pick on

There’s lots more to pick on in the article – for instance, the inability to determine the difference between a content management system and the web site it serves (akin to not knowing the difference between a story and the typewriter on which it was written), which itself significantly reduces the need for this one Open Source product to be secure.

The news article barely hints at some of the true advantages of Open Source – that others can drop additional components in at their pleasure, and that you can pick up whichever of those components you need. [Of course, the same is true of closed source products with good published interface specifications, so perhaps this is only an advantage in the extreme case that the provided interfaces are incomplete.]

Is Open Source more or less secure?

There are plenty of reasons to believe that Open Source offers security advantages – it’s possible, for instance, to do your own deep security investigations and fix problems when you become aware of them. Of course, that’s rather like saying an advantage of buying an old car is that you get to do your own services – great if you’re a mechanic, not so good if you have to check the owner’s manual to remember which end to put petrol into.

Software is more secure because it is written by good, dedicated, experienced programmers, reviewed by other good, dedicated, experienced programmers, analysed by tools and experienced programmers looking for security flaws, and tested pretty much to destruction.

Don’t forget, as well, that there is little perceivable difference between secure software, lucky software, and uninteresting software. All will appear to be unhacked – until luck runs out, or the software becomes interesting to an attacker.

I don’t claim to be able to determine that all Open Source is more or less secure than all Closed Source.

Just that the “more eyeballs” line doesn’t remotely provide anything close to an explanation.

Phishing at Hotmail, GMail, Yahoo! Mail, etc.

Recent password exposures at a number of online email services remind me to give a little advice on passwords.

Definitely use this as a reminder to do something about your passwords – but don’t do the obvious thing. Don’t rush round and change all your passwords right away.

Don’t change your passwords, change your password habits.

1. Don’t use the same password everywhere.
   If your password gets exposed, or the service owner is malicious (or has a malicious staff member), you’ll be exposed everywhere.
   Many times, of course, you will be unable to use the same password everywhere, because one site will require a symbol, and another will not allow that symbol. It is better to cope with this than to have to try and synchronise all your passwords.
2. Write down some of your passwords.
   What, seriously? Yep. Write down those passwords you don’t frequently use, and lock them away. Or store them in a password-protected (encrypted) file, whether that’s a Word file, Excel spreadsheet or any number of other storage mechanisms that will allow you to encrypt your passwords and store them away. Now you have replaced multiple passwords to remember with one.
   See point 1, though, make sure the password encrypting your password store is one you don’t share with any other sites.
   There are products out there which will protect your passwords for you – whether they are called password safes, vaults, strongboxes etc, they all do basically the same kind of thing.
3. Consider what passwords should be accessible to others.
   This may sound like bad security – and in a managed environment where others can always exert administrative rights to access files and systems that your passwords were used for, you should generally not be sharing your passwords.
   But think on this – a friend of mine received a traumatic train/brain injury, and though his recovery borders on the miraculous, there are many things he has forgotten. Passwords seem to be the hardest for him to hang on to, and he has had to recover through other means – sometimes simply wiping and recreating the system.
   Just as you have a will to direct people how to continue after your passing, store safely away account details and passwords so that your affairs can be brought into control if you are interrupted like my friend, or more permanently disconnected from the Internet. And make sure someone trustworthy and reliable can find that store when necessary.
4. Plan to change your passwords.
   If you don’t occasionally change your passwords, you will not know how to change them when it comes time to do so in a hurry.
   At several times in my professional career, I’ve had to deal with accounts whose passwords might have been exposed, whether through departing employees, lost password sheets, and at some of those occasions the natural security response of ‘change the passwords as soon as possible’ results in major push-back, by teams who have never changed their passwords, don’t know how to achieve it quickly, and aren’t sure what other applications depend on those passwords.
   If you don’t regularly change your important passwords, you’ll be flummoxed and panicked when it’s actually necessary to do so, and you may break something that depends on those passwords being synchronised.
5. Change your passwords often enough, but not too often.
   How often is too often?
   How often is enough?
   Difficult questions – often enough that you can remember changes to the systems to figure out why a password change caused some difficulty, and often enough to cover departing employees or others who might have had legitimate access once, but shouldn’t have access any more.
   Too often is when you get so tired of changing your passwords that you start regretting the process entirely.

There are no doubt dozens more things that could be suggested as good password practice, but these five will stand you in good stead.

Windows 7 – what it’s missing

Unless you’ve been living under a rock, you’ll be aware that today was the release of Microsoft’s latest operating system version, Windows 7.

So, everyone else has their own ideas of what’s missing in Windows 7, here’s my list, and it’s not the same petty focus that everyone else seems to have. Mine is based on what I want, rather than what’s remotely close to being reasonably achievable.

1. Media Center devices to provide support for DirecTV.
2. Trimmable transparent screen overlays supporting multi-touch input.
3. IPv6 support from my home ISP.
4. A web browser that opens quickly enough that I don’t forget what I was about to browse to.
5. A tool to answer “why is the system so slow right now?” – especially on those occasions when the CPU is not being over-taxed.
6. A free Zune HD. (Why not, since I’m dreaming here.)
7. Simple facilities to allow electronic commerce to operate on ‘zero knowledge’ principles, so that I would share my credit card account number only with my credit card provider, rather than with every merchant I might do business with. (Maybe Infocard or something like it could come close to fulfilling this wish)
8. An “Expert” mode, where menus are visible, files and file extensions are not hidden in Explorer. (For that matter, file extensions should not be hidden in Explorer. Ever.)
9. MSN – excuse me – Windows Live Messenger that works in a somewhat rational way, back in the system tray, rather than as a minimised icon.

So, what are the things in your twisted imaginings that would turn Windows 7 from this kind of Seven:

into this kind?

[Note: Having said all of this, it should be clear by now that I think Windows Seven is well worth having. But I still want more!]

SAL-like code annotations for Java

http://types.cs.washington.edu/jsr308/ seems to be talking about a set of type annotations for Java that are similar to those provided in Microsoft Visual C++ by SAL, the Standard Annotation Language.

One thing that the Java annotations have going for them over the SAL is that these annotations are going to be a part of the Java 7 standard, so it’s something that will come with the language no matter who implements it, whereas the C++ SAL extensions are specific to Microsoft. Of course, when I say “no matter who implements it”, I’m not aware of any significant currently supported implementations of Java outside of Sun, so it’s possible that such a statement is necessarily limited.

[Note that the SAL extensions can be included in C++ code that is compiled with other compilers, you just won’t see any benefit from them when using other compilers.]

What do annotations do?

As explained in the blog post that Michael Howard put out when SAL was first made available, these code annotations add something to object and function prototypes. What they add is the ability to turn run-time issues into compile-time errors.

So, for instance, a null pointer dereference, that would be an instant denial of service on your application, is trapped at compile time, because you declared in your function or class prototypes that you expect the pointer not to be null.

Adding these annotations to your code can certainly be a time-consuming task, since you have to revisit old code and add them in by had, recapturing assumptions that you had originally made about objects you reference. That in itself can be a learning experience, of course, and because you will capture a number of outstanding reliability, quality and security bugs, it’s far from being an empty investment.

What do the Java annotations capture?

The first checker that the Java annotations implement is the Nullness checker. This allows you to declare whether you are expecting a reference or value to be null or not. This generally avoids you seeing exceptions through dereferencing null.

The Interning checker prevents you from seeing poor results when comparing two objects (such as two instances of “Integer(2)”) using “==” or “!=”. Without the Interning checker, using “==” to compare two Integer variables each containing an instance of Integer(2) will produce the result ‘false’. This can cause logical processing errors, which the Interning checker will address.

The Mutability (IGJ or Javari) checker allows you to specify that an object reference should not be used to modify the contents of that object.

The Lock checker prevents locking errors by allowing you to declare that objects can only be accessed when guarded by a lock, or to declare that a function can only be called when holding a particular lock.

The Tainted checker allows you to mark an object as coming from an untrusted source (think “user”). Marking some functions as expecting Untainted data will prevent them from being fed Tainted data, and will ensure that other developers accessing such a function will call checking routines to convert the data from Tainted to Untainted before passing it to your functions.

The Linear checker prevents your code from holding more than one reference to objects marked as Linear. Since Java, like C#, likes to copy references to objects, rather than the objects themselves, this checker can prevent you from finding unexpected side-effects from objects being modified through aliases you weren’t expecting.

Further checkers appear to allow you to write your own checkers, but I haven’t got the Java programming chops to really play with these.

Should I include these annotations in my Java projects?

Definitely. Or something like them. Using annotations to define to the compiler some of the expectations you make of your code (a hyped-up interface contract, if you like) allows the compiler to check more deeply into whether those assumptions can hold true throughout your code.

My own experience of SAL-annotated code is that it has allowed me to discover some relatively subtle bugs in my programs. Or bugs that weren’t quite so subtle, but just weren’t jumping out at me.

A quick look at these Java annotations suggests that they will do the same for Java projects. Frankly, the more help you can get from these static analysis tools, the better. Analysis tools don’t catch all problems, and they aren’t a substitute for good programming, but they do provide a second check on your own assumptions that can be very useful.

Previous articles on SAL:

• SAL - pipped at the post by Michael Howard
• Okay, scratch what I said about SAL
• Forget that I asked you to ignore what I said.

Google bans MVP

Google certainly sounds like it’s a nice place to work. Table football, free lunches, that whole “don’t be evil” mantra, and the requirement to spend 20% of your time on projects that aren’t specifically to do with any particular company goal (with the obvious intent that some of those projects will result in interesting discoveries and/or personal development that the company can use).

But I can’t say that I’ll be applying there, at least until they publicly state that they are permanently reversing a decision they made in the last few weeks.

What did Google do that was, to my mind, so very close to evil?

Google told Jon Skeet that he shouldn’t accept the Microsoft MVP Award that was offered to him, despite the fact that he’s been awarded for the sixth year in a row.

Remember, this is a retrospective award – it is a recognition of what you have done for the community of Microsoft’s users, not a request or obligation to do anything in the future or act in a particular way.

Jon’s award stems from his frequent, continued and voluntary assistance to other C# developers.

As far as I can tell, Google has not told Jon to stop helping C# developers, and certainly his blog is still up, his support pages and FAQs are still up, and he’s still posting helpful advice on C#.

I could even understand if they said “don’t go to the annual MVP Summit”, or “don’t advocate Microsoft products” (although that’s not expected of MVPs, who bristle at the slightest suggestion of being ‘evangelists’).

So, Google’s not angry with his behaviour – they are angry with his being recognised and rewarded, by Microsoft, for that helpful volunteer behaviour.

My own C# projects have benefitted on numerous occasions from finding an article Jon has posted on the Internet. I’d like to thank him for that, and I hope he isn’t required by Google to reject my gratitude as well.

Thanks, Jon Skeet, for all you do for the C# developer community.

Thanks, Google, for reminding us that there’s a line that divides “evil” from simply “really, really bad”.

[Note: Please feel free to pass this posting on. I’d like to see Google feel very very ashamed for this, and to recant. If only because Jon will be eligible every quarter for the MVP award, and he’s going to get awfully tired of refusing it over and over. News coverage would be great, but I don’t see any at the time of writing on Bing’s News search.]

Sometimes It Seems Like Unix(*) Needs to Learn from Windows

(*) By “Unix”, I mean Linux, Unix, AIX, OS/X, and similar flavours.

Way back when, about twenty or so years ago, I was a Unix admin, and a Unix developer. I had to be both, because I was the only person in the company who could spell Unix.

My favourite game was to go along to presentations for Microsoft Windows ‘new features’ and say “Oh, but hasn’t Unix had that for the last twenty years?”

Sure enough, there were countless things that Windows users and developers were just discovering (TCP/IP, shared libraries, multiple sessions on the same computer) that had been in Unix for some time. Linux was yet to make a mention, but as I’ve moved firmly into the Windows world, and left Unix behind, I’ve pretty much assumed that technologically speaking, if Windows has it, Unix and the like must also have the same functionality.

As I re-engage with Unix and Linux developers and IT professionals in recent months, though, I can see that there are some areas – particularly in security - where Windows is far ahead of the *x operating systems. Here’s a few:

Where’s my EFS?

EFS, the Encrypting File System, is one of Windows’ best-kept secrets. It’s not really a secret, of course, but it acts like one – there are so few people willing to use it, and mostly because they’re scared of or don’t understand it.

EFS allows users (under administrative control and with appropriate recovery measures in place) to choose files to encrypt, and to declare which other users can access the encrypted files.

EFS-encrypted files are encrypted on disk, and the keys cannot be broken simply by mounting an offline attack, because the key for each file is encrypted with users’ public keys, and the private keys are held securely in the users’ certificate store.

What does *x have in response? Whole disk encryption by third-party products (OK, Windows has Bitlocker and any number of third-party products). EFS protects individual files, and is far more fine-grained than the ‘all or nothing’ access of WDE (or FDE, Full Disk Encryption, if you prefer).

Single Certificate Store

This isn’t really a “single” store so much as a predictable location for the certificate store. If you want to read a user’s certificates and keys, you know where to find them (although you generally only have access if you are the user in question. Private keys from the certificate store are protected using the DPAPI, appropriately protecting them (apart from some key recovery scenarios, you have to log in using the password associated with the keys).

Similarly, certificates and keys belonging to the system and its service accounts are also in predictable locations.

This makes life easy for tools that need to scan for certificates due to expire.

Where are certificates and keys stored in *x? All over the place. Generally in “PEM” files, usually (but not always) in the same directory in which the application that installs them is.

How are these private keys protected in *x? There’s sometimes a password to open up the private key from the PEM file, and usually the PEM file has a restrictive access mask on it. [Read further for more problems with this]

Single SSL Library

It’s not uncommon to see several instances of OpenSSL installed on any particular system, whether it’s *x or Windows, if the system runs applications that use OpenSSL.

Windows developers, of course, can simply use the SSL API built in to Windows (CryptoAPI, CAPI and SChannel), and not have to worry about shipping an SSL library with their application, or keeping up with new versions as they come out, or tracking down customers and notifying them of updates to address security flaws (such as the Debian Linux key generation flaw I posted about a while ago).

Single SSL Configuration

If I want to disable SSL v2, or ciphers with fewer than 128 bits, on Windows I can change a few registry settings and know that I’ve fixed every application that uses SChannel. I can even do that remotely, with remote registry editing from a script or group policy tattooing the registry.

To do the same for OpenSSL, it seems that I have to find every application that uses OpenSSL and change the configuration files there. 

Data Protection API and configuration file protection

This is the one that really started me on this article.

How do you store a password in a configuration file?

Yes, the ‘right’ security answer is “you don’t”, but that’s naive. The fact is that there are many instances wherein you have to store a password – to access and authenticate to a remote application, or (if you’re using OpenSSL) to open a password-protected PEM or PFX file in order to read out the private key.

On Windows, the Patterns and Practices team have documented how to do this – basically, you use the DPAPI to encrypt the password into the config file, and again to decrypt it back out – and your DPAPI keys are encrypted by your master key, which is derived from your password. The end result is that you can’t get those DPAPI keys without the password.

What do the *x platforms have?

”Put the password in plain text, and protect it with a restrictive access mask”, is what I’m told. And in a search, I couldn’t find anything better being recommended. OK, one person recommended encoding the password with base64, but that’s hardly a security measure.

Jesper brought up the excellent question of “how is it different?” – in the *x system, the password is marked as only being accessible to the correct user. I was about to answer him when Steve F spoke up for me, and noted that in the DPAPI case, you have to read the file, and then an API has to be called to decrypt the password; in the *x case, you simply have to read the file. There are many many more exploits that allow the reading of a file under privileged rights than there are exploits that allow the execution of code.

Patch Management and Group Policy

Microsoft has done a really good job of implementing enterprise-level management features into their operating systems, from Group Policy and WMI to WSUS and other update management tools.

The *x systems I’ve seen seem to be built from the perspective that each system has its own attendant administrator, who is only too happy to manually deploy patches or tweak settings in line with some policy on a scrap of paper or post-it.

Maybe I’m missing some huge advances, and maybe some of these issues are resolved with a third-party tool – but then, maybe that’s part of the problem too. All of the above are a part of the operating system in Windows, and can be relied on to exist by developers, and their use by applications can be expected by IT professionals.

[Disclaimer: Yes, I know there are still areas where Microsoft needs to learn from Unix and Linux, and perhaps it’d be good if you’d educate me on those, too. This isn’t a “Windows is better than *X” debate, it’s a “hey, even if you think *X is better than Windows, here are some areas *X needs improving in”.]

Edit: There have been some excellent comments posted overnight in response to this article, and as I had hoped, I am mostly still 'in the dark' about what Linux and Unix-like systems offers. I'll be looking at these as I have time, and responding when I can. For now, just let me say that I am impressed to see so much technical content in the responses, and so little of the "fanboy" behaviour that often characterises these discussions.

Zune HD – but not mine

A friend of mine ordered a Platinum Zune HD recently (that’s the 32GB model), and because he was unable to receive the shipment, asked for me to open it for him and check on its functionality to make sure he hadn’t been shipped a lemon.

Since I’ve previously commented on the Zune 30 that my wife bought for my birthday, I thought I’d have a quick look and see what I like about it.

The demonstration video is stunning, and shows off the display impressively. The display is wonderfully bright, and fulfils every bit of the promise of OLED technology. Light-weight, thin, amazingly bright and detailed.

Installing the new Zune software from http://www.zune.net/setup went smoothly, although when the player was plugged in, the Zune software immediately insisted on a Player update. The Zune needs to be updated from 4.0 to 4.1 already.

This may come as a surprise, but really it’s not too shocking. There’s a considerable gap between preparing a bunch of hardware for simultaneous shipping and the actual delivery, during which time there may be some interesting bugs discovered. Possibly this time, the bug is that the charge indicator doesn’t light in version 4.0, but does light up in version 4.1. At least, that’s a change I noticed.

So, have any of my previous complaints been addressed? Given the timing of my last post, close to the end of the Zune HD’s development, I doubt that Microsoft had a chance to fix the problems I noted, and I seem to be correct about that.

You can still put MP3 files into your Podcast folder and give them a genre of “Podcast” in order to make them work like Podcasts (i.e. remembering their position while you go do other things), but the images tied into the MP3 files are still not displayed along with those podcasted MP3s. And they still don't play ordered by track number, preferring instead to use some bizarre combination of date and textual sort, with some apparent randomness thrown in.

It appears so far that all of the other issues I’ve encountered are still there, so I’m still waiting for someone at Microsoft to address those and deliver a Zune (updated firmware, software, or hardware) that is absolutely perfect. If they could make it cheaper, too, it would be easier to justify a purchase.

But man, I love that bright display on the new Zune HD. I just wish I didn’t have to part with this one so soon. I guess I’d better save my Amazon gift cards…

Would you behave differently in a shared office?

How would you change your behaviour at work if you knew the person seated one desk over worked for a competitor?

How would your behaviour change if you knew the person one cubicle over was about to work for a competitor?

What if you knew that your cubicle neighbour was going to lose her job (be fired or laid off) in the next six months? Do you think she’d be looking to work in a different industry, or the one where she had the most recent experience?

What if the economic situation was such that you just couldn’t be sure who in your office would still be with you a year from now?

How would you protect your data then?

My point is less about pointing out that the current economic situation seems very like this harsh threatening landscape, but to ask you to consider that the answer to this question is actually the answer you should give all the time.

A recent study from Ponemon stated that six out of ten departing employees will take data with them as they leave, whether that’s customer data or business intelligence. Why do they do this? Well, we could get into the whole motivation of why, but the real answer is simple:

Because they can, and because they think they can benefit from doing so. Not because they won’t get caught – because, really, what are you going to do, fire them?

Behave (and design!) as if you’re in an open environment.

Design your data and processes around the idea that important, private, or proprietary data should only rest with individuals or in stores for as long as it is needed to do the job at hand.

After that, then what?

If you no longer need it, or can reconstruct or re-collect it when you next need it, why not just destroy the data?

If you need it, return it to a secure data store, from which it can’t be fetched again without business need, and appropriate authorisation.

If you never needed it in the first place, why collect it at all?

Protecting systems, networks, applications – that’s just resiliency and protection of a few thousand dollars of assets. The real money – and the real requirement for security protection – is in the data.

Act (and architect!) like the data is, AND isn’t, yours.

I used to say that people should “act like the data isn’t yours in the first place” – makes logical sense, doesn’t it?

Sure, if you think that way – if you think that you should be careful with other people’s possessions that they’ve loaned to you.

Over several jobs and several years, I’ve come to realise that we aren’t all of the same species of thought. Some of us are careless with other people’s possessions, and are only concerned with taking care of what’s ours.

So, my explanation has changed – now, the explanation is still that the data doesn’t belong to us, but we have possession of it, and therefore we, as application designers and architects, have a double requirement to be careful with it. We must protect it because it isn’t ours, and we must protect it because it is in our care. To be loose with other people’s data would be to cause them damage, and to be loose with data in our care would be to cause our business damage by reducing the value that we get from holding that data.

How FTP Data Connections Work Part 2 (OR: Fun With Port 20)

As we mentioned in the 1st part of this series, FTP is a more complex protocol than many, using one control connection and one data connection.

A recap of the first post…

In typical Stream Mode operation, a new data connection is opened and closed for each data transfer, whether that’s an upload, a download, or a directory listing. To avoid confusion between different data connections, and as a recognition of the fact that networks may have old packets shuttling around for some time, these connections need to be distinguishable from one another.

In the previous article, we noted that two network sockets are distinguished by the five elements of “Local Address”, “Local Port”, “Protocol”, “Remote Address”, and “Remote Port”. For a data connection associated with any particular request, the local and remote addresses are fixed, as the addresses of the client and server. The protocol is TCP, and only the two ports are variable.

For a PASV, or passive data connection, the client-side port is chosen randomly by the client, and the server-side port is similarly chosen randomly by the server. The client connects to the server.

For a PORT, or active data connection, the client-side port is chosen randomly by the client, and the server-side port is set to port 20. The server connects to the client.

All of these work through firewalls and NAT routers, because firewalls and NAT routers contain an Application Layer Gateway (ALG) that watches for PORT and PASV commands, and modifies the control (in the case of a NAT) and/or uses the values provided to open up a firewall hole.

Isn’t there a totally predictable data connection?

For the default data connection (what happens if no PORT or PASV command is sent before the first data transfer command), the client-side port is predictable (it’s the same as the source port the client used when connecting the control channel), and the server-side port is 20. Again, the server connects to the client.

Because firewalls and NATs open up a ‘reverse’ hole for TCP sockets, the default data port works with firewalls and NATs that aren’t running an ALG, or whose ALG cannot scan for PORT and PASV commands.

Why would an ALG stop scanning for PORT and PASV commands?

There are a couple of reasons – the first is that it doesn’t know that the service connected to is running the FTP protocol. This is common if the server is running on a port other than the usual port 21.

The second reason is that the FTP control connection doesn’t look like it contains FTP commands – usually because the connection is encrypted. This can happen because you’re tunneling the FTP control connection through an encrypted tunnel such as SSH (don’t laugh – it does happen!), or hopefully it’s because you’re running FTP over SSL, so that the control and data connections can be encrypted, and you can authenticate the identity of the FTP server.

So how do you get FTP over SSL to work through a firewall?

In the words of Deep Thought: “Hmm… tricky”.

There are a couple of classic solutions:

1. Allow PASV data connections, select a wide range of ports, and open that range for incoming traffic from all external addresses in your firewall configuration; hope that your FTP server can be configured to use only that range of ports (WFTPD Pro can), and that it has protections against traffic stealing attacks (again, WFTPD Pro has). Still, this option seems really risky.
2. Block all PASV connections, and make the clients responsible for opening up holes in their firewalls. If you’re convinced the risk is too great to do this on your server, how does it look to convince your users that they should accept that risk?
3. After you’ve authenticated the server and provided your username and password in the encrypted control connection, issue the “CCC” (Clear Control Channel) command, to switch the control connection back into clear-text. I dislike this as a solution, because it requires the ALG pay attention to a lot of SSL traffic in the hope that there might be clear-text coming up, and because you may want the control channel to remain encrypted.

Awright, clever clogs, you solve the problem.

The astute reader can probably see where I’m going with this.

The default data port is predictable – if the client connects from port U to port L at the server (L is usually 21), then the default data port will be opened from port L-1 at the server to port U at the client.

The default data port doesn’t need the firewall to do anything other than allow reverse connections back along the port that initiated the connection. You don’t need to open huge ranges at the server’s firewall (in fact you should be able to simply open port 21 inbound to your server).

The default data port is required to be supported by FTP servers going back a long way- at least a couple of decades. Yes, really, that long.

If it’s that simple, why isn’t everyone doing it?

Good point, that, and a great sentence to use whenever you wish to halt innovation in its tracks.

Okay, it’s obvious that there are some drawbacks:

• In stream mode, the data transfer is ended by closing the stream. This means that you have to open a new control connection. Not good, given the number of round-trips you need for a logon, and the work needed to start an SSL connection.
• Most FTP clients view the default data connection as, at best, a fail-over in case the PORT or PASV commands fail to work. Obviously, that means it’s not likely to be a well-tested or favoured solution on these clients.

Even with those drawbacks, there are still further solutions to apply – the first being to use Block-mode instead of Stream-mode. In Stream-mode, each data transfer requires opening and closing the data connection; in Block-mode, which is a little like HTTP’s chunked mode, blocks of data are sent, and followed by an “EOF” marker (End of File), so that the data connection doesn’t need to be closed. If you can convince your FTP client to request Block-mode with the default data connection, and your FTP server supports it (WFTPD Pro has done so for several years), you can achieve FTP over SSL through NATs and firewalls simply by opening port 21.

For the second problem, it’s worth noting that many FTP client authors implemented default data connections out of a sense of robustness, so default data connections will often work if you can convince the PORT and PASV commands to fail – by, for instance, putting restrictive firewalls or NATs in the way, or perhaps by preventing the FTP server from accepting PORT or PASV commands in some way.

Clearly, since Microsoft’s IIS 7.5 downloadable FTP Server supports FTPS in block mode with the default data port, there has been some consideration given to my whispers to them that this could solve the FTP over SSL through firewall problem.

Other than my own WFTPD Explorer, I am not aware of any particular clients that support the explicit use of FTP over SSL with Block-mode on the default data connection – I’d love to hear of your experiments with this mode of operation, to see if it works as well for you as it does for me.

Dreaming of the future...

Here are some technologies I just can't wait for:

• OLEDs:
  • for room lighting - ambient light from ceiling-tile sized light panels [those of us that suffer migraines want an alternative to fluorescent lights, compact or otherwise]
  • for either back-lighting of LCD screens, or for the screen itself - I didn't know until after I bought it that my laptop uses a fluorescent bulb for the backlight. When the battery gets low, I can see it flicker, and I have to turn it off or risk another migraine.
  • Are they green? They're any colour you want, baby!
  • Oh, you mean are they ecologically sound? Far more so than incandescent, fluorescent, effervescent, evanescent or putrescent. Incandescent bulbs burn way more power; fluorescent bulbs have mercury - and, surprisingly, burn way more power for the amount of light they put out than equivalent LEDs or OLEDs.
  • OLEDs are cool to the touch - perhaps in some climates this means you'll have to run your heater more, but really, you don't think a light-bulb is an efficient heat generator, do you?
• Multi-touch support, including fingertip and stylus support.
  • Windows 7 (which I thought would be called "Viista") will feature multi-touch support, where users will grab objects with a couple of fingers, to more naturally twist and scale them.
  • Stylus support would allow drawing and writing - I wish I had an excuse to get a Tablet PC, but I just can't afford to sacrifice power in order to get that capability. Maybe I'll buy a cheap USB tablet to plug in at the side.
• Single sign-on through the use of federated identity.
  • Okay, that one probably needs some explanation.
  • I'm tired - so tired - of one password here, a different password there, here I'm "alunj", there I'm "aljones", another place I'm ma7amj, yet another place AMJ10.
  • I want to enter one user name, one password, and be able to authenticate to everywhere.
  • Of course, that would mean everywhere would have to trust the one user name and one password - and if that isn't carefully monitored, you'll see people tying their bank accounts and nuclear secrets to a one character password. This requires some thought.
• Transflective displays.
  • Tra-wha?
  • Put simply, if it's light enough to read a piece of paper, I want to be able to use my laptop. And if it's really, really bright, I want to be able to use my laptop.
  • No backlights - I want the screen to be like coloured paper, reflecting ambient light.
  • That'll cut down on weight, battery consumption, and probably also frame rate in games. Can't have everything :)
• Wi-tricity
  • Wireless electricity.
  • Sure, it's going to bombard me with electrons, but only if I'm resonant. Otherwise, it'll power my technology without requiring that it all be tethered to the wall.
• Wide-spread adoption of IPv6
  • Heck, even though Microsoft installs IPv6 by default in Vista and Server 2008, there still isn't an IPv6-based Microsoft "front page".
  • www.ipv6.microsoft.com has been dead for months.
  • Akamai, which hosts www.microsoft.com, doesn't appear to know about IPv6.
  • IPv6 brings us back to the way that nature intended the Internet to be - everyone's a peer node; everyone can be a server. Firewalls are firewalls, and NATs are non-existent.