Fast page loading by moving ASP.NET AJAX scripts after visible content

ASP.NET ScriptManager control has a property LoadScriptsBeforeUI, when set to false, should load all AJAX framework scripts after the content of the page. But it does not effectively push down all scripts after the content. Some framework scripts, extender scripts and other scripts registered by Ajax Control Toolkit still load before the page content loads. The following screen taken from www.dropthings.com shows several script tags are still added at the beginning of <form> which forces them to download first before the page content is loaded and displayed on the page. Script tags pause rendering on several browsers especially in IE until the scripts download and execute. As a result, it gives user a slow loading impression as user stares at a white screen for some time until the scripts before the content download and execute completely. If browser could render the html before it downloads any script, user would see the page content immediately after visiting the site and not see a white screen. This will give user an impression that the website is blazingly fast (just like Google homepage) because user will ideally see the page content, if it's not too large, immediately after hitting the URL.

image
Figure: Script blocks being delivered before the content

From the above screen shot you see there are some scripts from ASP.NET AJAX framework and some scripts from Ajax Control Toolkit that are added before the content of the page. Until these scripts download, browser don't see anything on the UI and thus you get a pause in rendering giving user a slow load feeling. Each script to external URL adds about 200ms avg network roundtrip delay outside USA while it tries to fetch the script. So, user basically stares at a white screen for at least 1.5 sec no matter how fast internet connection he/she has.

These scripts are rendered at the beginning of form tag because they are registered using Page.ClientScript.RegisterClientScriptBlock. Inside Page class of System.Web, there's a method BeginFormRender which renders the client script blocks immediately after the form tag.

   1: internal void BeginFormRender(HtmlTextWriter writer, string formUniqueID)
   2: {
   3:     ...
   4:         this.ClientScript.RenderHiddenFields(writer);
   5:         this.RenderViewStateFields(writer);
   6:         ...
   7:         if (this.ClientSupportsJavaScript)
   8:         {
   9:             ...
  10:             if (this._fRequirePostBackScript)
  11:             {
  12:                 this.RenderPostBackScript(writer, formUniqueID);
  13:             }
  14:             if (this._fRequireWebFormsScript)
  15:             {
  16:                 this.RenderWebFormsScript(writer);
  17:             }
  18:         }
  19:         this.ClientScript.RenderClientScriptBlocks(writer);
  20: }

Figure: Decompiled code from System.Web.Page class

Here you see several script blocks including scripts registered by calling ClientScript.RegisterClientScriptBlock are rendered right after form tag starts.

There's no easy work around to override the BeginFormRender method and defer rendering of these scripts. These rendering functions are buried inside System.Web and none of these are overridable. So, the only solution seems to be using a Response Filter to capture the html being written and suppress rendering the script blocks until it's the end of the body tag. When the </body> tag is about to be rendered, we can safely assume page content has been successfully delivered and now all suppressed script blocks can be rendered at once.

In ASP.NET 2.0, you to create Response Filter which is an implementation of a Stream. You can replace default Response.Filter with your own stream and then ASP.NET will use your filter to write the final rendered HTML. When Response.Write is called or Page's Render method fires, the response is written to the output stream via the filter. So, you can intercept every byte that's going to be sent to the client (browser) and modify it the way you like. Response Filters can be used in variety ways to optimize Page output like stripping off all white spaces or doing some formatting on the generated content, or manipulating the characters being sent to the browser and so on.

I have created a Response filter which captures all characters being sent to the browser. It it finds that script blocks are being rendered, instead of rendering it to the Response.OutputStream, it will extract the script blocks out of the buffer being written and render the rest of the content. It stores all script blocks, both internal and external, in a string buffer. When it detects </body> tag is about to be written to the response, it flushes all the captured script blocks from the string buffer.

   1: public class ScriptDeferFilter : Stream
   2: {
   3:     Stream responseStream;
   4:     long position;
   5:  
   6:     /// <summary>
   7:     /// When this is true, script blocks are suppressed and captured for 
   8:     /// later rendering
   9:     /// </summary>
  10:     bool captureScripts;
  11:  
  12:     /// <summary>
  13:     /// Holds all script blocks that are injected by the controls
  14:     /// The script blocks will be moved after the form tag renders
  15:     /// </summary>
  16:     StringBuilder scriptBlocks;
  17:     
  18:     Encoding encoding;
  19:  
  20:     public ScriptDeferFilter(Stream inputStream, HttpResponse response)
  21:     {
  22:         this.encoding = response.Output.Encoding;
  23:         this.responseStream = response.Filter;
  24:  
  25:         this.scriptBlocks = new StringBuilder(5000);
  26:         // When this is on, script blocks are captured and not written to output
  27:         this.captureScripts = true;
  28:     }

Here's the beginning of the Filter class. When it initializes, it takes the original Response Filter. Then it overrides the Write method of the Stream so that it can capture the buffers being written and do it's own processing.

   1: public override void Write(byte[] buffer, int offset, int count)
   2: {
   3:     // If we are not capturing script blocks anymore, just redirect to response stream
   4:     if (!this.captureScripts)
   5:     {
   6:         this.responseStream.Write(buffer, offset, count);
   7:         return;
   8:     }
   9:  
  10:     /* 
  11:      * Script and HTML can be in one of the following combinations in the specified buffer:          
  12:      * .....<script ....>.....</script>.....
  13:      * <script ....>.....</script>.....
  14:      * <script ....>.....</script>
  15:      * <script ....>.....</script> .....
  16:      * ....<script ....>..... 
  17:      * <script ....>..... 
  18:      * .....</script>.....
  19:      * .....</script>
  20:      * <script>.....
  21:      * .... </script>
  22:      * ......
  23:      * Here, "...." means html content between and outside script tags
  24:     */
  25:  
  26:     char[] content = this.encoding.GetChars(buffer, offset, count);
  27:  
  28:     int scriptTagStart = 0;
  29:     int lastScriptTagEnd = 0;
  30:     bool scriptTagStarted = false;
  31:  
  32:     for (int pos = 0; pos < content.Length; pos++)
  33:     {
  34:         // See if tag start
  35:         char c = content[pos];
  36:         if (c == '<')
  37:         {
  38:             int tagStart = pos;
  39:             // Check if it's a tag ending
  40:             if (content[pos+1] == '/')
  41:             {
  42:                 pos+=2; // go past the </ 
  43:  
  44:                 // See if script tag is ending
  45:                 if (isScriptTag(content, pos))
  46:                 {
  47:                     /// Script tag just ended. Get the whole script
  48:                     /// and store in buffer
  49:                     pos = pos + "script>".Length;
  50:                     scriptBlocks.Append(content, scriptTagStart, pos - scriptTagStart);
  51:                     scriptBlocks.Append(Environment.NewLine);
  52:                     lastScriptTagEnd = pos;
  53:  
  54:                     scriptTagStarted = false;
  55:                     continue;
  56:                 }
  57:                 else if (isBodyTag(content, pos))
  58:                 {
  59:                     /// body tag has just end. Time for rendering all the script
  60:                     /// blocks we have suppressed so far and stop capturing script blocks
  61:  
  62:                     if (this.scriptBlocks.Length > 0)
  63:                     {
  64:                         // Render all pending html output till now
  65:                         this.WriteOutput(content, lastScriptTagEnd, tagStart - lastScriptTagEnd);
  66:  
  67:                         // Render the script blocks
  68:                         byte[] scriptBytes = this.encoding.GetBytes(this.scriptBlocks.ToString());
  69:                         this.responseStream.Write(scriptBytes, 0, scriptBytes.Length);
  70:  
  71:                         // Stop capturing for script blocks
  72:                         this.captureScripts = false;
  73:  
  74:                         // Write from the body tag start to the end of the inut buffer and return
  75:                         // from the function. We are done.
  76:                         this.WriteOutput(content, tagStart, content.Length - tagStart);
  77:                         return;
  78:                     }
  79:                 }
  80:                 else
  81:                 {
  82:                     // some other tag's closing. safely skip one character as smallest
  83:                     // html tag is one character e.g. <b>. just an optimization to save one loop
  84:                     pos++;
  85:                 }
  86:             }
  87:             else
  88:             {
  89:                 if (isScriptTag(content, pos+1))
  90:                 {
  91:                     /// Script tag started. Record the position as we will 
  92:                     /// capture the whole script tag including its content
  93:                     /// and store in an internal buffer.
  94:                     scriptTagStart = pos;
  95:  
  96:                     // Write html content since last script tag closing upto this script tag 
  97:                     this.WriteOutput(content, lastScriptTagEnd, scriptTagStart - lastScriptTagEnd);
  98:  
  99:                     // Skip the tag start to save some loops
 100:                     pos += "<script".Length;
 101:  
 102:                     scriptTagStarted = true;
 103:                 }
 104:                 else
 105:                 {
 106:                     // some other tag started
 107:                     // safely skip 2 character because the smallest tag is one character e.g. <b>
 108:                     // just an optimization to eliminate one loop 
 109:                     pos++;
 110:                 }
 111:             }
 112:         }
 113:     }
 114:     
 115:     // If a script tag is partially sent to buffer, then the remaining content
 116:     // is part of the last script block
 117:     if (scriptTagStarted)
 118:     {
 119:         
 120:         this.scriptBlocks.Append(content, scriptTagStart, content.Length - scriptTagStart);
 121:     }
 122:     else
 123:     {
 124:         /// Render the characters since the last script tag ending
 125:         this.WriteOutput(content, lastScriptTagEnd, content.Length - lastScriptTagEnd);
 126:     }
 127: }

There are several situations to consider here. The Write method is called several times during the Page render process because the generated HTML can be quite big. So, it will contain partial HTML. So, it's possible the first Write call contains a start of a script block, but no ending script tag. The following Write call may or may not have the ending script block. So, we need to preserve state to make sure we don't overlook any script block. Each Write call can have several script block in the buffer as well. It can also have no script block and only page content.

The idea here is to go through each character and see if there's any starting script tag. If there is, remember the start position of the script tag. If script end tag is found within the buffer, then extract out the whole script block from the buffer and render the remaining html. If there's no ending tag found but a script tag did start within the buffer, then suppress output and capture the remaining content within the script buffer so that next call to Write method can grab the remaining script and extract it out from the output.

There are two other private functions that are basically helper functions and does not do anything interesting:

   1: private void WriteOutput(char[] content, int pos, int length)
   2: {
   3:     if (length == 0) return;
   4:  
   5:     byte[] buffer = this.encoding.GetBytes(content, pos, length);
   6:     this.responseStream.Write(buffer, 0, buffer.Length);
   7: }
   8:  
   9: private bool isScriptTag(char[] content, int pos)
  10: {
  11:     if (pos + 5 < content.Length)
  12:         return ((content[pos] == 's' || content[pos] == 'S')
  13:             && (content[pos + 1] == 'c' || content[pos + 1] == 'C')
  14:             && (content[pos + 2] == 'r' || content[pos + 2] == 'R')
  15:             && (content[pos + 3] == 'i' || content[pos + 3] == 'I')
  16:             && (content[pos + 4] == 'p' || content[pos + 4] == 'P')
  17:             && (content[pos + 5] == 't' || content[pos + 5] == 'T'));
  18:     else
  19:         return false;
  20:  
  21: }
  22:  
  23: private bool isBodyTag(char[] content, int pos)
  24: {
  25:     if (pos + 3 < content.Length)
  26:         return ((content[pos] == 'b' || content[pos] == 'B')
  27:             && (content[pos + 1] == 'o' || content[pos + 1] == 'O')
  28:             && (content[pos + 2] == 'd' || content[pos + 2] == 'D')
  29:             && (content[pos + 3] == 'y' || content[pos + 3] == 'Y'));
  30:     else
  31:         return false;
  32: }

The isScriptTag and isBodyTag functions may look weird. The reason for such weird code is pure performance. Instead of doing fancy checks like taking a part of the array out and doing string comparison, this is the fastest way of doing the check. Best thing about .NET IL is that it's optimized, if any of the condition in the && pairs fail, it won't even execute the rest. So, this is  as best as it can get to check for certain characters.

There are some corner cases that are not handled here. For example, what if the buffer contains a partial script tag declaration. For example, "....<scr" and that's it. The remaining characters did not finish in the buffer instead next buffer is sent with the remaining characters like "ipt src="..." >.....</scrip". In such case, the script tag won't be taken out. One way to handle this would be to make sure you always have enough characters left in the buffer to do a complete tag name check. If not found, store the half finished buffer somewhere and on next call to Write, combine it with the new buffer sent and do the processing.

In order to install the Filter, you need to hook it in in the Global.asax BeginRequest or some other event that's fired before the Response is generated.

   1: protected void Application_BeginRequest(object sender, EventArgs e)
   2: {
   3:     if (Request.HttpMethod == "GET")
   4:     {
   5:         if (Request.AppRelativeCurrentExecutionFilePath.EndsWith(".aspx"))
   6:         {
   7:             Response.Filter = new ScriptDeferFilter(Response);
   8:         }
   9:     }
  10: }

Here I am hooking the Filter only for GET calls to .aspx pages. You can hook it to POST calls as well. But asynchronous postbacks are regular POST and I do not want to do any change in the generated JSON or html fragment. Another way is to hook the filter only when ContentType is text/html.

When this filter is installed, www.dropthings.com defers all script loading after the <form> tag completes.

image
Figure: Script tags are moved after the <form> tag when the filter is used

You can grab the Filter class from the App_Code\ScriptDeferFilter.cs of the Dropthings project. Go to CodePlex site and download the latest code for the latest filter.

Published Sun, Apr 6 2008 15:49 by omar

Comments

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Monday, April 07, 2008 5:46 AM by borko

That's really a great idea! Thank you for sharing this with us.

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Monday, April 07, 2008 2:27 PM by borko

How can you measure the difference between this two solutions? Is there any way to measure the time which is passed between the request for the page and the moment when the page starts to render?

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Monday, April 07, 2008 2:36 PM by omar

I haven't found any tool that can hook on browser and see when it starts rendering. So, I rely on my eye for now. As my goal is to improve perceived speed, not the actual speed, I have to look at the site and see how fast it 'feels'.

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Tuesday, April 08, 2008 12:05 PM by Sameeh

Good Idea Omar, but I think that you are serving the client at the expense of the server.

You added so much processing for filtering each page request that may lead to performance issue on server during heavy load.

Did you captured the time consumed by your new filter approach and compared it with the 1.5 sec?

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Tuesday, April 08, 2008 4:10 PM by Anil

Ican see the difference on your site. thanks for sharing

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Tuesday, April 08, 2008 11:03 PM by omar

On my laptop, it takes about an avg 40ms more to filter out 20KB of html output. It's negiligble compared to the client side benefit we get. There can be 20% reduction on total RPS on the server, but we can always throw in more server power or optimize server side code to compensate for that small loss in RPS.

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Wednesday, April 09, 2008 3:50 AM by Adeel

if you use HttpWatch Professional, it shows all active connections used by the browser client. It wont show you exactly when the form tag will begin rendering, but you will be able to see the page download first and begin loading page elements such as images or javascripts. In the realtime transcript, it becomes quite clear that loading all js at the end makes for very fast page loading.

At coroflot.com, we had to do something about multiple css files called at the start of the page. The same techniques / filter can be used to consolidate all css into one single file, compress and then serve. We do it via an offline process, but it can be done automatically with this technique.

Lots of high profile system (such as communityserver.com) can get a nice boost with this.

Thanks for sharing all your web dev. IQ

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Tuesday, April 15, 2008 2:30 PM by satish

Omar,

I was trying to implement this conecept on my application i have a auto complete text search box and the user types in by the time the page renders how to handle this scenarious and sometimes the output also behaves weirdly. Please update

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Thursday, May 01, 2008 3:13 AM by sanjay kr gupta

I was trying to implement this conecept on my application thank u

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Friday, May 23, 2008 4:18 PM by Muhammad Yousaf Sulahria

Adsense for ScriptDeferFilter v2:

// See if script tag is ending

if (isScriptTag(content, pos))

{

   /// Script tag just ended. Get the whole script

   /// and store in buffer

   pos = pos + "script>".Length;

   string temp=new string(content, scriptTagStart, pos - scriptTagStart);

   if ((temp.IndexOf("google_ad_client")==-1) && (temp.IndexOf("googlesyndication") == -1))

   {

       scriptBlocks.Append(content, scriptTagStart, pos - scriptTagStart);

       scriptBlocks.Append(Environment.NewLine);

   }

   else

   {

       this.WriteOutput(content, scriptTagStart, pos - scriptTagStart);

   }

   lastScriptTagEnd = pos;

   scriptTagStarted = false;

   pos--; // continue will increase pos by one again

   continue;

}

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Sunday, June 08, 2008 6:15 PM by Sayeed Ahmed

Omar,

It is really very good article.  Just need to know how can I implement “ScriptDeferFilter.cs” on VS 2005 project .

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Thursday, June 12, 2008 12:38 PM by Yousaf

There is a problem with scriptdeferfilter.cs my ajax rating is not working with it.

can u post the fix.

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Monday, August 18, 2008 6:32 PM by Mohammed El Baz

Thanks very much buddy, I was looking for this many days ago

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Friday, September 26, 2008 11:59 AM by roncansan

Hi Omar,

Thanks for the article. I'm using it in one of my projects, but I have found something strange

"I have a couple of pages that only have the google analytics JS, at the end of the page, when I applied the ScriptDeferFilter the google analytics JS disappear"

Any insights?

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Friday, September 26, 2008 12:13 PM by roncansan

Hi Omar "again"

If I put the Google Analytics JS before the "head tag" ends it works grate, but if I put it after the "body tag" ends the ScriptDeferFilter eliminates it.

In summary,

if you use ScriptDeferFilter you must place at least one script before the "head tag" ends. If you manually place them after the "body tag" ends the ScriptDeferFilter eliminates it.

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Friday, September 26, 2008 12:34 PM by omar

Please use the "pin" on the google analytics script tag to let ScriptDeferFilter ignore the script tags. See my article how to use the "pin" feature.

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Friday, September 26, 2008 1:05 PM by roncansan

Where can I find the article, i did to see it?

Thanks

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Tuesday, February 03, 2009 12:03 AM by Pandiya

Hey it doesn't work for me.... I ve added ScriptDeferFilter to my app code folder... And in global ascx file it doesnt seems to work

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Monday, February 09, 2009 11:18 AM by darshan shah

i could'nt understand the example at all.

can you please give me the working example of aspx page so i can understand and incorporate in my site.

my id is darshan43shah@gmail.com

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Thursday, February 19, 2009 10:01 AM by Faiyaz

Asak,

I have incorporated ur sample but the last part which is combining script files into one url I didnt understand. You have mentioned some configuration file changes required for combining scripts into one URL.

can you please give me the working sample so I can incorporate the same.

faiyazmumtaz@yahoo.com

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Saturday, March 07, 2009 6:25 AM by Frano Hartman

Hello, I really like this approach, you can look at my site www.adriaticglobal.net how it works.

Also I have adapted it to work with ASP NET MVC control, look at this code...

protected override void OnActionExecuting(ActionExecutingContext filterContext)

       {

           Response.Filter = new AdriaticGlobal.MVC.Utilty.ScriptDeferFilter(Response);

           base.OnActionExecuting(filterContext);

       }

just change in code HttpResponse to HttpResponseBase.

You can see live example at:

www.dubrovnik-accommodation.biz (beta site).

# re: Fast page loading by moving ASP.NET AJAX scripts after visible content

Friday, January 08, 2010 2:49 AM by Joe Nganga

Awesome.

But is there a VB version of this.

email me: euniqjoe@gmail.com

Leave a Comment

(required) 
(required) 
(optional)
(required)