Omar AL Zabir blog on ASP.NET Ajax and .NET 3.5 : asp.net performance

Prevent Denial of Service (DOS) attacks in your web application

omar — Sat, 24 Mar 2007 13:37:00 GMT

Web services are the most attractive target for hackers because even a pre-school hacker can bring down a server by repeatedly calling a web service which does expensive work. Ajax Start Pages like Pageflakes are the best target for such DOS attack because if you just visit the homepage repeatedly without preserving cookie, every hit is producing a brand new user, new page setup, new widgets and what not. The first visit experience is the most expensive one. Nonetheless, it’s the easiest one to exploit and bring down the site. You can try this yourself. Just write a simple code like this:

   1: for( int i = 0; i < 100000; i ++ )

   2: {

   3:    WebClient client = new WebClient();

   4:    client.DownloadString("http://www.pageflakes.com/default.aspx");

   5: }

In your great surprise, you will notice that, after a couple of call, you don't get valid response. It’s not that you have succeeded in bringing down the server. It’s that your requests are being rejected. You are happy that you no longer get any service, thus you achieve Denial of Service (for yourself). I am happy to Deny You of Service (DYOS).

The trick I have in my sleeve is an inexpensive way to remember how many requests are coming from a particular IP. When the number of request exceeds the threshold, deny further request for some duartion. The idea is to remember caller’s IP in Asp.net Cache and maintain a count of request per IP. When the count exceeds a predefined limit, reject further request for some specific duration like 10 mins. After 10 mins, again allow requests from that IP.

I have a class named ActionValidator which maintains a count of specific actions like First Visit, Revisit, Asynchrnous postbacks, Add New widget, Add New Page etc. It checks whether the count for such specific action for a specific IP exceeds the threshold value or not.

   1: public static class ActionValidator

   2: {
   3:      private const int DURATION = 10; // 10 min period

4:

   5:     public enum ActionTypeEnum

   6:     {

   7:         FirstVisit = 100, // The most expensive one, choose the valu wisely.

   8:         ReVisit = 1000, // Welcome to revisit as many times as use likes

   9:         Postback = 5000,   // Not must of a problem for us

  10:         AddNewWidget = 100,

  11:         AddNewPage = 100,

  12:     }

The enumeration contains the type of actions to check for and their threshold value for a specific duration – 10 mins.

A static method named IsValid does the check. It returns true if the request limit is not passed, false if the request needs to be denied. Once you get false, you can call Request.End() and prevent Asp.net from proceeding further. You can also switch to a page which shows “Congratulations! You have succeeded in Denial of Service Attack.”

   1: 
public static bool IsValid( ActionTypeEnum actionType )

   2: {

   3: HttpContext context  = HttpContext.Current;

   4: 
if( context.Request.Browser.Crawler ) return false;

5:

   6: string key = actionType.ToString()  + context.Request.UserHostAddress;

7:

   8: HitInfo hit  = (HitInfo)(context.Cache[key] ?? new HitInfo());

9:

  10: if( hit.Hits > (int)actionType ) return false;

  11: else hit.Hits ++;

12:

  13: if( hit.Hits == 1 )

  14:     context.Cache.Add(key, hit, null, DateTime.Now.AddMinutes(DURATION),

  15:       System.Web.Caching.Cache.NoSlidingExpiration, System.Web.Caching.CacheItemPriority.Normal, null);

16:

  17: return true;

  18: }

The cache key is built with a combination of action type and client IP address. First it checks if there’s any entry for the action and the client IP in Cache or not. If not, start the count and store remember the count for the IP in cache for the specific duration. The absolute expiration on cache item ensures after the duration, the cache item will be cleared and the count will restart. When there’s already an entry in the cache, get the last hit count, and check if the limit is exceeded or not. If not exceeded, increase the counter. There is no need to store the updated value in the cache again by doing: Cache[url]=hit; because the hit object is by reference and changing it means it gets changed in the cache as well. In fact, if you do put it again in the cache, the cache expiration counter will restart and fail the logic of restarting count after specific duration.

The usage is very simple:

   1: 
protected override void OnInit(EventArgs e)

   2: {

   3:   base.OnInit(e);

4:

   5:   // Check if revisit is valid or not

   6:   if(  !base.IsPostBack )

   7:   {

   8:     // Block cookie less visit attempts

   9:     if( Profile.IsFirstVisit )

  10:     {

  11:        if( !ActionValidator.IsValid(ActionValidator.ActionTypeEnum.FirstVisit) Response.End();

  12:     }

  13:    else

  14:     {

  15:      if( !ActionValidator.IsValid(ActionValidator.ActionTypeEnum.ReVisit) ) Response.End();

  16:     }

  17:   }

  18:  else

  19:   {

  20:    // Limit number of postbacks

  21:     if( !ActionValidator.IsValid(ActionValidator.ActionTypeEnum.Postback)  Response.End();

  22:   }

  23: }

Here I am checking specific scenario like First Visit, re-visit, postbacks etc.

Of course you can put in some Cisco firewall and prevent DOS attack. You will get guaranty from your hosting provider that their entire network is immune to DOS and DDOS (Distributed DOS) attacks. What they guaranty is network level attack like TCP SYN attacks or malformed packet floods etc. There is no way they can analyze the packet and find out a particular IP is trying to load the site too many times without supporting cookie or trying to add too many widgets. These are called application level DOS attack which hardware cannot prevent. It must be implemented in your own code.

There are very few websites out their which take such precaution for application level DOS attacks. Thus it’s quite easy to make servers go mad by writing a simple loop and hitting expensive pages or web services continuously from your home broadband connection. I hope this small but effective class will help you implement DOS attack in your own web applications.

Update

Here's the code of the full ActionValidator class:

// Copyright (c) Omar AL Zabir. All rights reserved.
// For continued development and updates, visit http://msmvps.com/omar

using System;
using System.Data;
using System.Configuration;
using System.Web;
using System.Web.Security;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using System.Web.UI.HtmlControls;

/// <summary>
/// Summary description for ActionValidator
/// </summary>
namespace Dropthings.Web.Util
{
public static class ActionValidator
{
private const int DURATION = 10; // 10 min period

/*
* Type of actions and their maximum value per period
*
*/
public enum ActionTypeEnum
{
None = 0,
FirstVisit = 100, // The most expensive one, choose the value wisely.
Revisit = 1000, // Welcome to revisit as many times as user likes
Postback = 5000, // Not must of a problem for us
AddNewWidget = 100,
AddNewPage = 100,
}

private class HitInfo
{
public int Hits;
private DateTime _ExpiresAt = DateTime.Now.AddMinutes(DURATION);
public DateTime ExpiresAt { get { return _ExpiresAt; } set { _ExpiresAt = value; } }
}

public static bool IsValid( ActionTypeEnum actionType )
{
HttpContext context = HttpContext.Current;
if( context.Request.Browser.Crawler ) return false;

string key = actionType.ToString() + context.Request.UserHostAddress;

HitInfo hit = (HitInfo)(context.Cache[key] ?? new HitInfo());

if( hit.Hits > (int)actionType ) return false;
else hit.Hits ++;

if( hit.Hits == 1 )
context.Cache.Add(key, hit, null, DateTime.Now.AddMinutes(DURATION),
System.Web.Caching.Cache.NoSlidingExpiration, System.Web.Caching.CacheItemPriority.Normal, null);

return true;
}
}
}

Things you must do for ASP.NET 2.0 Membership Provider before going live

omar — Sun, 27 Aug 2006 09:43:00 GMT

Some tweakings you should do on your web.config if you are using ASP.NET 2.0 Membership Provider before you go live on your production server:

Add "applicationname" attribute in Profile Provider. IF you do not add a specific name here, Profile provider will use a GUID. So, on your local machine you will have one GUID and on production server you will have another GUID. If you copy your local DB to production server, you won't be able to reuse the records available in your local DB and ASP.NET will create a new application on production. Here's where you need to add it:

< profile enabled ="true" > < providers > < clear /> < add name ="..." type ="System.Web.Profile.SqlProfileProvider" connectionStringName ="..." applicationName ="YourApplicationName" description ="..." /> </ providers >
Profile provider will automatically save the profile whenever a page request completes. So, this might result in unnecessary UPDATE on your DB which has significant performance penalty. So, turn off automatic save and do it explicitly from your code using Profile.Save();

< profile enabled ="true" automaticSaveEnabled ="false" >
Role Manager always queries database in order to get the user roles. This has significant performance penalty. You can avoid this by letting Role Manager cache role information on cookie. But this will work for users who do not have a lot of roles assigned to them which exceeds the 2 KB limit of Cookie. But it's not a common scenario. So, you can safely store role info on cookie and save one DB rountrip on every request to .aspx and .asmx.

< roleManager enabled ="true" cacheRolesInCookie ="true" >
See my previous post on optimizing two important SPs used by Profile Provider.

Optimize ASP.NET 2.0 Profile Provider before you go live

omar — Thu, 17 Aug 2006 17:41:00 GMT

Do you know there are two important stored procedures in ASP.NET 2.0 Profile Provider that you can optimize significantly? If you use them without doing the necessary optimization, your servers will sink taking your business down with you during heavy load. Here're a story:

During March, Pageflakes was shown on MIX 2006. We were having a glamorous time back then. We were right on Showcase of Atlas web site. The first company, not the second or third. Number of visits per day were rising sky high. One day we noticed, the server was no more. We restarted the server, brought it back, again it died within an hour. After doing a lot of postmortem analysis on the remaining of server's body parts, we found that it was having 100% CPU and super high IO usage. The hard drives were over heated and turned themselves off in order to save themselves. This was quite surprising to us because we were very intelligent back then and we profiled every single web service function. So, we went through hundreds of megabytes of logs hoping to find which webservice function was taking the time. We suspected one. It was the first function that loads a user's page setup. We broke it up into smaller parts in order to see which part is taking most of the time.


    
private


GetPageflake(
string

source, 
string

pageID, 
string

userUniqueName
)


    {

      if( Profile.IsAnonymous ) {


     
using

(
new


TimedLog
(Profile.UserName,
"GetPageflake"
))

You see, the entire function body is timed. If you want to learn how this timing works, wait until I make a post on it. We also timed smaller parts which we suspected were taking the most resource. But we could find not a single place in our code which was taking any significant time. Our codebase is always super optimized (after all, you know who is reviewing it, me).

Meanwhile, users were shouting, management was screaming, support staff was complaining on the phone. Developers were fainting one by one. Remaining ones were furiously sweating and blood vessels on their forehead were coming out etc. Nothing special, just a typical situation we have couple of times every month.

Now you must be shouting, "You could have used SQL Profiler, you idiot!" We were using SQL Server workgroup edition. It does not have SQL Profiler. So, we had to hack our way through to get it running on a server somehow. Don't ask how. After running the SQL Profiler, boy, were we surprised! The name of the honorable SP which were giving us so much pleasure was none but the famous and the most pretigious one, which we all know by name due to its incredible contribution to the development of human civilization. Please welcome, the great stored procedure "dbo.aspnet_Profile_GetProfiles"!

We used (and still use) Profile provider extensively.

Here's the SP:


CREATE PROCEDURE
        
[dbo].[aspnet_Profile_GetProfiles]


    @ApplicationName   
                
nvarchar
(256),


    @ProfileAuthOptions   
        
int
,


    @PageIndex   
                      

                int
,


    @PageSize   
                       

                int
,


    @UserNameToMatch   
                
nvarchar
(256)
= 
NULL
,


    @InactiveSinceDate   
              
datetime


  
                    = 
NULL

AS


BEGIN


    
DECLARE

@ApplicationId


                        uniqueidentifier


    
SELECT

@ApplicationId
= 
NULL


    
SELECT

@ApplicationId
= ApplicationId
                
FROM

aspnet_Applications
                        

                    
WHERE
LOWER
(@ApplicationName)
                            = LoweredApplicationName


    
IF

(@ApplicationId

IS
NULL
)


       
        
RETURN

 

   
        
--
Set the page bounds


    
DECLARE

@PageLowerBound

int


    
DECLARE

@PageUpperBound

int


    
DECLARE

@TotalRecords
  
int


    
SET

@PageLowerBound
= @PageSize
                    * @PageIndex


    
SET

@PageUpperBound
= @PageSize
                    - 1 + @PageLowerBound


    
--
Create a temp table TO store the select results


    
CREATE
TABLE 
#PageIndexForUsers


       
            IndexId 
int
IDENTITY 
(0,
1) 
NOT
NULL
,


       
            UserId 
uniqueidentifier


    
--
Insert into our temp table


    
INSERT
INTO 
#PageIndexForUsers
                    (UserId)


       
        
SELECT

u.UserId


       
        
FROM


  dbo.aspnet_Users
                u, dbo.aspnet_Profile p


       
        
WHERE


 ApplicationId = @ApplicationId


       
                
AND

u.UserId
                    = p.UserId


       
                
AND

(@InactiveSinceDate
                
IS
NULL OR 
LastActivityDate
                        <= @InactiveSinceDate)


       
                
AND

(
  
                    (@ProfileAuthOptions = 2)


       
                       

                OR 
(@ProfileAuthOptions
= 0 

                        AND 
IsAnonymous
= 1)


       
                       

                OR 
(@ProfileAuthOptions
= 1 

                        AND 
IsAnonymous
= 0)


       
                
AND

(@UserNameToMatch
                
IS
NULL OR 
LoweredUserName
                    
LIKE
LOWER
(@UserNameToMatch))


       
        
ORDER
BY 
UserName


    
SELECT

u.UserName,
u.IsAnonymous, u.LastActivityDate,
                    p.LastUpdatedDate,


       
                
DATALENGTH
(p.PropertyNames)
                    + 
DATALENGTH
(p.PropertyValuesString)

                            + 
DATALENGTH
(p.PropertyValuesBinary)


    
FROM


  dbo.aspnet_Users
                    u, dbo.aspnet_Profile p, #PageIndexForUsers
i


    
WHERE


 u.UserId = p.UserId 
AND

p.UserId
= i.UserId 
AND
                                

                            
i.IndexId
>= @PageLowerBound 

                                    AND 
i.IndexId
<= @PageUpperBound


    
DROP
TABLE 
#PageIndexForUsers

END

END

First it looks up for ApplicationID.


    
DECLARE

@ApplicationId


                    uniqueidentifier


    
SELECT

@ApplicationId
= 
NULL


    
SELECT

@ApplicationId
= ApplicationId
            
FROM

aspnet_Applications
                    

                
WHERE
LOWER
(@ApplicationName)
                        = LoweredApplicationName


    
IF

(@ApplicationId

IS
NULL
)


       
    
RETURN

Then it creates a temporary table (why not table data type?) in order to store profiles of users.

-- Create a temp table TO store the select results


    
CREATE
TABLE 
#PageIndexForUsers


        IndexId 

        int IDENTITY 
(0,
1) 
NOT
NULL
,


        UserId 

        uniqueidentifier


    
--
Insert into
        our temp table


    
INSERT
INTO 
#PageIndexForUsers
(UserId)

If it gets called very frequently, there will be too high IO due to the temporary table creation. It also runs through 2 very big tables - aspnet_Users and aspnet_Profile. The SP is written in such a way that if one user has multiple profiles, it will return all profiles of the user. But normally we store one profile per user. So, there's no need for creating a temporary table. Moreover, there's no need for doing LIKE LOWER (@UserNameToMatch). It is always called with a full user name which we can match directly using the equal operator.

So, we opened up the stored proc and did a open heart bypass surgery like this:


IF 
@UserNameToMatch

IS
NOT NULL


   
        
BEGIN


   
        
SELECT

u.UserName,
u.IsAnonymous, u.LastActivityDate, p.LastUpdatedDate,


   
                    
DATALENGTH
(p.PropertyNames)
                    + 
DATALENGTH
(p.PropertyValuesString)
+ 
DATALENGTH
(p.PropertyValuesBinary)


   
        
FROM


  dbo.aspnet_Users u


   
        
INNER
JOIN 
dbo.aspnet_Profile
p 
ON

u.UserId
= p.UserId


   
        
WHERE


   
            u.LoweredUserName = 
LOWER
(@UserNameToMatch)


   
        
SELECT

@@ROWCOUNT

END


   
        
ELSE


    
BEGIN

-- Do the original bad
things

It ran fine locally. Now it was time to run it on the server. This is an important SP which is used by the ASP.NET 2.0 Profile Provider, heart of ASP.NET framework. If we do something wrong here, we might not be able to see the problem immediately, but may be after one month we will realize users profile is mixed up and there's no way to get it back. So, it was a pretty hard decision to run this on a live production server directly without doing enough test. We did not have time to do enough test anyway. We are already down. So, we all gathered, said our prayer and hit the "Execute" button on SQL Server Management Studio.

The SP ran fine. On the server we noticed from 100% CPU usage it came down to 30% CPU usage. IO usage also came down to 40%.

We went live again.

Here's another SP which gets called on every page load and webservice call on our site because we use Profile provider extensively.


CREATE PROCEDURE 
[dbo].[aspnet_Profile_GetProperties]


    @ApplicationName      
nvarchar
(256),


    @UserName           

nvarchar
(256),


    @CurrentTimeUtc       
datetime

AS


BEGIN


    
DECLARE

@ApplicationId

uniqueidentifier


    
SELECT

@ApplicationId
= 
NULL


    
SELECT

@ApplicationId
= ApplicationId 

                FROM 
dbo.aspnet_Applications


                        WHERE LOWER
(@ApplicationName)
= LoweredApplicationName


    
IF

(@ApplicationId

IS
NULL
)


        

        RETURN


    
DECLARE

@UserId

uniqueidentifier


    
SELECT

@UserId
= 
NULL


    
SELECT

@UserId
= UserId


    
FROM


 dbo.aspnet_Users


    
WHERE

ApplicationId
= @ApplicationId 

                AND 
LoweredUserName
= 

                        LOWER
(@UserName)


    
IF

(@UserId

IS
NULL
)


        

        RETURN


    
SELECT
TOP 
1
PropertyNames, PropertyValuesString, PropertyValuesBinary


    
FROM


       dbo.aspnet_Profile


    
WHERE


      UserId = @UserId


    
IF

(@@ROWCOUNT
> 0)


    
BEGIN


        

        UPDATE 
dbo.aspnet_Users


        

        SET 

  LastActivityDate=@CurrentTimeUtc


        

        WHERE 
UserId
= @UserId

END

END

When you run the SP, see the statistics:


Table 
'aspnet_Applications'
.
Scan 
count

1,
logical reads 2, physical reads 0, 

                        read
-ahead
reads 0, lob logical reads 0, lob physical
                            reads 0, lob 
read
-ahead
reads
                                    0.


(1 row(s) affected)


Table 
'aspnet_Users'
.
Scan 
count

1,
logical reads 4, physical reads 0, 

                        read
-ahead
reads 0, lob logical reads 0, lob physical
                            reads 0, lob 
read
-ahead
reads
                                    0.


(1 row(s) affected)


(1 row(s) affected)


Table 
'aspnet_Profile'
.
Scan 
count

0,
logical reads 3, physical reads 0, 

                        read
-ahead
reads 0, lob logical reads 0, lob physical
                            reads 0, lob 
read
-ahead
reads
                                    0.


(1 row(s) affected)


Table 
'aspnet_Users'
.
Scan 
count

0,
logical reads 27, physical reads 0, 

                        read
-ahead
reads 0, lob logical reads 0, lob physical
                            reads 0, lob 
read
-ahead
reads
                                    0.


(1 row(s) affected)


(1 row(s) affected)

This stored proc is called whenever you access some property on Profile object.

First it does a SELECT on aspnet_application to find out the application ID from application name. You can easily replace this with a hard coded application ID inside the SP and save some effort. Normally we run only one application on our production server. So, there's no need to lookup application ID on every single call.

From client statistics, you can see how expensive it is:

Then look at the last block where aspnet_users table is updated with LastActivityDate. This is the most expensive one.

This is done in order to ensure Profile provider remembers when was the last time a user's profile was accessed. We do not need to do this on every single page load and web service call. May be we can do it when user first logs in and logs out. In our case, a lot of web service is called while user is on the page. There's only one page anyway. So, we can easily remove this in order to save a costly update on the giant aspnet_users table on every single web service call.

Client side Page Fragment Output cache, reduce page download time significantly

omar — Thu, 10 Aug 2006 06:34:00 GMT

When you have a page full of lots of HTML, it will be best if the whole page can be cached on the browser. You can do this using HTTP Response caching header either by injecting them manually or by using @OutputCache tag directive on ASPX pages.

<%
@

OutputCache 

Location
="Client"

Duration
="86400"

VaryByParam
="*"

VaryByHeader
="*"

%>

But if part of the page is dynamic and part of the page is static (like header, left side menu, foother, right side menu, bottom part) etc where static parts of the page occupy a significant amount of html, then if you could cache those parts on the browser, you could save a lot of bytes that gets downloaded every time the page downloads. On most of the websites, you will find the header, navigation menu, footer, bottom part, advertisements are mostly static and thus easily cacheable. If the whole page size is say 50KB, at least 20KB is static and 30KB might be dynamic. If you can use client side caching of page fragments (not ASP.NET’s server side page output cache), you can save 40% download time easily.

ASP.NET offers you page fragment caching using @Outputcache which is good, but that caching is on server side. It cache the output of user controls and serves them from server side cache. You cannot eliminate the download of those costly bytes. It just saves some CPU process. Nothing much for users in it.

The only way to cache part of page is by allowing browser to download those parts separately and making those parts cacheable just like images/CSS/javascripts get cached. So, we need to download page fragments separately and make them cached on the browser’s cache. IFRAME is an easy way to do this but IFRAME makes the page heavy and thus not follow CSS of the page. There are many reasons why IFRAME can’t work. So, we have a better way, we will use Javascript to render content of the page and javascript will get cached on the browser’s cache.

So, here's the idea:

We will split the whole page into multiple parts
We will generate page content using Javascript. Each cacheable part comes from javascript and javascript renders the HTML of it.
The parts which are cachable gets cached by the browser and thus never downloaded again (until you want them to be). But those parts which are non-cachable and highly dynamic, does not get cached by browser.

So, let's think of a page setup like this:

Logo	Header
Left navigation Menu	Dynamic part of the page
Footer

Here only one part is dynamic and the rest is fully cacheable. So, the Default.aspx which renders this whole page looks like this:

<%@PageLanguage="VB"AutoEventWireup="false"%>

<%@OutputCacheNoStore="true"Location="None"%>

<!DOCTYPEhtmlPUBLIC"-//W3C//DTDXHTML 1.0 Transitional//EN""http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

<htmlxmlns="http://www.w3.org/1999/xhtml">

<headrunat="server">

    <title>MyBig Fat Page</title>

</head>

<body>

<formid="form1"runat="server">

<tablewidth="100%"border="1">

<tr>

<td>Somelogo here</td>

<td><scriptid="Script1"src="Header.aspx"type="text/javascript"></script></td>

</tr>

<tr>

<td><scriptid="LeftMenu"src="LeftMenu.aspx"type="text/javascript"></script></td>

<tdbgcolor="lightgrey"><div>

This is the dynamic part which gets changed on every load. Checkout the time when

it was generated: <%=DateTime.Now %></div></td>

</tr>

<tr>

<tdcolspan="2"><scriptid="Footer"src="Footer.aspx"type="text/javascript"></script></td>

</tr>

</table>

</form>

</body>

</html>

The page looks like this:

You see, the cached parts are 30 mins older. Browser has not downloaded those parts at all and thus saved a significant amount of data transfer. The only part that was downloaded was the dynamic part.

When you load the page first time, all 4 files are downloaded. But the last 3 files get cached and never downloaded until browser’s cache expires. So, on second visit, only one file downloaded and thus saves a significant amount of data transfer.

Let’s look at one of the files Header.aspx which gets cached. Nothing fancy here, it’s a regular ASPX page:

The interesting thing here is the “ContentType” which I have set to “text/html/javascript”. This is not something built-in, I have introduced this type.

When you put an ASPX inside a Script tag, it surely does not work because < script id ="Script1" src ="Header.aspx" type ="text/javascript"> expects javascript output, not html output. If html output is provided, browsers simply ignores it. So, we need to convert the output of Header.aspx into Javascript which when downloaded and executed by the browser, emits the original html that was generated when ASP.NET executed the page.

We use HTTP Module to intercept all .aspx calls and when the page is about to be written to the output, we check if the content type is “text/html/javascript”. If it is, this is our cue to convert the page output to javascript representation.

If you want to know details about HTTP Module and how to use Response Filter to modify page output, please read this wonderful article:

http://www.aspnetresources.com/articles/HttpFilters.aspx

It really explains all the things. I would recommend you read this article first and then continue with the rest.

We have made a response filter named Html2JSPageFilter.js (available in the code download), which overrides the Write method of Stream and converts the entire HTML of the page to javascript representation:

    publicoverridevoidWrite(byte[]buffer, intoffset, intcount)

        stringstrBuffer = System.Text.UTF8Encoding.UTF8.GetString(buffer, offset, count);

        //---------------------------------

        //Wait for the closing </html> tag

        //---------------------------------

        Regexeof = newRegex("</html>",RegexOptions.IgnoreCase);

        if(!eof.IsMatch (strBuffer))

            responseHtml.Append (strBuffer);

        else

            responseHtml.Append (strBuffer);

            string finalHtml = responseHtml.ToString ();

            //extract only the content inside the form tag tag ASP.NET generatesin all .aspx

            intformTagStart = finalHtml.IndexOf("<form");

            intformTagStartEnd = finalHtml.IndexOf('>',formTagStart);

            intformTagEnd = finalHtml.LastIndexOf("</form>");

            stringpageContentInsideFormTag = finalHtml.Substring(formTagStartEnd + 1,formTagEnd - formTagStartEnd - 1);

First we get the entire page output, then we get only what is inside the <form> tag that ASP.NET generates for all .aspx pages.

Next step is to remove the viewstate hidden field because this will conflict with the view state on the default.aspx.

            //Remove the __VIEWSTATE tag because page fragments don't needviewstate

            //Note this will make all ASP.NET controls in the page fragments gomad which

            //needs viewstate to do their work.

            Regexre = newRegex("(<input.*?__VIEWSTATE.*?/>)",RegexOptions.IgnoreCase);

            pageContentInsideFormTag =re.Replace(pageContentInsideFormTag, string.Empty);

Now we convert the entire html output to javascript string format:

            ///Convert the HTML to javascript string representation

            stringjavascript2Html =

                pageContentInsideFormTag.Replace("\r","")

                .Replace("\n","")

                .Replace("   ","")

                .Replace(" ","")

                .Replace("  ","")

                .Replace("\\","\\\\")

                .Replace("'","\\'");

Final touch is to put that javascript string inside a “document.write(‘...’);” call. When you call document.write to emit html, it gets part of the page html:

            //Generate the document.write('...') which adds the content in thedocument

            stringpageOutput = "document.write('"+ javascript2Html + "');";

This is basically the trick. Use a Response filter to get the .aspx output, and then convert it to Javascript representation.

For convenience, I have used a HttpModule to hook into ASP.NET pipeline and wait for .aspx files which try to emit content type of “text/html/javascript”. Again this content type is nothing special, you could use “text/Omar Al Zabir”.

    voidIHttpModule.Init(HttpApplicationcontext)

        context.ReleaseRequestState += newEventHandler(InstallResponseFilter);

    privatevoidInstallResponseFilter(objectsender, EventArgse)

     HttpResponseresponse = HttpContext.Current.Response;

     if(response.ContentType == "text/html/javascript")

         response.ContentType = "text/javascript";

         response.Filter = newHtml2JSPageFilter(response.Filter);

And finally in web.config, we have to register the HttpModule so that it gets called:

        <httpModules>

            <addname="Html2JSModule"type="Html2JavascriptModule"/>

        </httpModules>

The entire source code is available in this URL:

Download Source code of: Client side Page Fragment Output cache, reduce page download time significantly

Enjoy. Use this approach in your aspx and html files and save significant amount of download time on users end. Although it slightly increases first time visit download time (200+ms for each script tag), but it makes second time visit a breeze. See the performance difference yourself. First visit www.pageflakes.com. Then close your browser, open it again and enter www.pageflakes.com. See how fast it loads. If you use a HTTP debugger to monitor how much data is transferred, you will see it's only 200 bytes!