No, you cannot call Server.Transfer on an ASP.NET AJAX enabled page
Even though ASP.NET AJAX has been here for some time, this question still pops up on the forums. So I thought I should put a post here and redirect future questions on the subject to it.
Well, to be precise, what you cannot do is call Server.Transfer (or other transfer method - btw, if you're using IIS 7, there is a new TransferRequest method that might interest you) if you're on a partial postback (ie, you have an UpdatePanel on your page and, for instance, you're handling the click event of an ASP.NET button that you've dropped inside the UpdatePanel). The reason for this is simple: a partial request is always started by the client PageRequestManager object. This object expects to receive a text response from the server which must follow a specific format (if you're interested in this kind of thing, then just use fiddler to see what's going on) so that it can parse it and change the page DOM according to the changes that have been made on the server side.
Now, if you're calling Server.Transfer, you're really executing the new page (by using the Server.Execute method) and stopping the execution of the current page(ie, the one that has called the Transfer method). This means that the server side of the ASP.NET AJAX platform won't be able to "intercept" the response and change it so that it is sent back on the specific text format that the PageRequestManager object understands. So, in the client side you'll end up getting the HTML generated by the second page. When the PageRequestManager gets the response, it simply doesn't know what to do with it and it ends up generating a parsing exception.
If you need to transfer the user for another page, you can use the Response.Redirect method since this can be intercepted by the server side of the plaform during a partial postback. When that happens, the "redirect" will be tranformed on a text message that can be understood by the client side, meaning that you'll get your page navigation, instead of the parsing exception you see when you use the Server.Transfer method.