Omar AL Zabir blog on ASP.NET Ajax and .NET 3.5 : javascript

I am no more

omar — Sun, 25 Apr 2010 05:53:00 GMT

I have moved my blog to http://omaralzabir.com and I am no longer blogging here. Please update your readers and subscribe to the feed on the new site.

Fast Streaming Ajax Proxy with GET PUT POST DELETE

omar — Sun, 21 Feb 2010 18:23:00 GMT

I have enhanced my streaming Ajax Proxy with POST, PUT and DELETE features. Previously it supported only GET. Now it supports all 4 popular methods for complete REST support. Using this proxy, you can call REST API on external domain directly from your website’s javascript code. You can test the proxy from this link:

labs.omaralzabir.com/ajaxstreamingproxy/GetPutDeleteTest.aspx

The latest source code for the Ajax Proxy is available here:

http://code.google.com/p/fastajaxproxy/

You can find a detail CodeProject article that explains how the streaming asynchronous aspect of this proxy works:

Fast, Scalable, Streaming AJAX Proxy - continuously deliver data from across domains

Here’s how the test UI looks like where you can test POST, PUT and DELETE:

If you want to run the sample source code on your local IIS, make sure you allow the POST, PUT, and DELETE headers on .ashx extension from IIS properties:

The sample project shows how you can use the proxy to make calls to external domains. You can directly hit any external URL and perform POST or DELETE from your javascript code:

var proxyUrl = "StreamingProxy.ashx";
function download(method, proxyUrl, contentUrl, isJson, bodyContent, completeCallback) {
    var request = new Sys.Net.WebRequest();
    
    if (method == "POST" || method == "PUT")
        request.set_httpVerb("POST");
    else
        request.set_httpVerb("GET");

    var url = proxyUrl + "?m=" + method + 
  (isJson ? "&t=" + escape("application/json") : "") + "&u=" + escape(contentUrl);

    request.set_url(url);

    if (bodyContent.length > 0) {
        request.set_body(bodyContent);
        request.get_headers()["Content-Length"] = bodyContent.length;
    }

    var startTime = new Date().getTime();

    request.add_completed(function(executor) {
        if (executor.get_responseAvailable()) {
            var content = executor.get_responseData();
            var endTime = new Date().getTime();

            var statistics =
            "Duration: " + (endTime - startTime) + "ms" + '\n' +
            "Length: " + content.length + " bytes" + '\n' +
            "Status Code: " + executor.get_statusCode() + " " + '\n' +
            "Status: [" + executor.get_statusText() + "]" + '\n';
            appendStat(statistics);

            $get('resultContent').value = content;
            completeCallback();
        }
    });

    var executor = new Sys.Net.XMLHttpExecutor();
    request.set_executor(executor);
    executor.executeRequest();
}

I am using MS AJAX here. You can use jQuery to perform the same test as well. All you need to do is hit the URL of the StreamingProxy.ashx and pass the actual URL in query string parameter “u” and pass the type of the http method in query string parameter “m”. That’s it!

7 tips for for loading Javascript rich Web 2.0-like sites significantly faster

omar — Fri, 25 Sep 2009 14:46:19 GMT

Introduction

When you create rich Ajax application, you use external JavaScript frameworks and you have your own homemade code that drives your application. The problem with well known JavaScript framework is, they offer rich set of features which are not always necessary in its entirety. You may end up using only 30% of jQuery but you still download the full jQuery framework. So, you are downloading 70% unnecessary scripts. Similarly, you might have written your own javascripts which are not always used. There might be features which are not used when the site loads for the first time, resulting in unnecessary download during initial load. Initial loading time is crucial – it can make or break your website. We did some analysis and found that every 500ms we added to initial loading, we lost approx 30% traffic who never wait for the whole page to load and just close browser or go away. So, saving initial loading time, even by couple of hundred milliseconds, is crucial for survival of a startup, especially if it’s a Rich AJAX website.

You must have noticed Microsoft’s new tool Doloto which helps solve the following problem:

Modern Web 2.0 applications, such as GMail, Live Maps, Facebook and many others, use a combination of Dynamic HTML, JavaScript and other Web browser technologies commonly referred as AJAX to push page generation and content manipulation to the client web browser. This improves the responsiveness of these network-bound applications, but the shift of application execution from a back-end server to the client also often dramatically increases the amount of code that must first be downloaded to the browser. This creates an unfortunate Catch-22: to create responsive distributed Web 2.0 applications developers move code to the client, but for an application to be responsive, the code must first be transferred there, which takes time.

Microsoft Research looked at this problem and published this research paper in 2008, where they showed how much improvement can be achieved on initial loading if there was a way to split the javascripts frameworks into two parts – one primary part which is absolutely essential for initial rendering of the page and one auxiliary part which is not essential for initial load and can be downloaded later or on-demand when user does some action. They looked at my earlier startup Pageflakes and reported:

2.2.2 Dynamic Loading: Pageflakes
A contrast to Bunny Hunt is the Pageflakes application, an
industrial-strength mashup page providing portal-like functionality.
While the download size for Pageflakes is over 1 MB, its initial
execution time appears to be quite fast. Examining network activity
reveals that Pageflakes downloads only a small stub of code
with the initial page, and loads the rest of its code dynamically in
the background. As illustrated by Pageflakes, developers today can
use dynamic code loading to improve their web application’s performance.
However, designing an application architecture that is
amenable to dynamic code loading requires careful consideration
of JavaScript language issues such as function closures, scoping,
etc. Moreover, an optimal decomposition of code into dynamically
loaded components often requires developers to set aside the semantic
groupings of code and instead primarily consider the execution
order of functions. Of course, evolving code and changing
user workloads make both of these issues a software maintenance
nightmare.

Back in 2007, I was looking at ways to improve the initial load time and reduce user dropout. The number of users who would not wait for the page to load and go away was growing day by day as we introduced new and cool features. It was a surprise. We thought new features will keep more users on our site but the opposite happened. Analysis concluded it was the initial loading time that caused more dropout than it retained users. So, all our hard work was essentially going to drain and we had to come up with something ground breaking to solve the problem. Of course we had already tried all the basic stuffs – IIS compression, browser caching, on-demand loading of JavaScript, css and html when user does something, deferred JavaScript execution – but nothing helped. The frameworks and our own hand coded framework was just too large. So, the idea tricked me, what if we could load functions inside a class in two steps. First step will load the class with absolutely essential functions and second step will inject more functions to the existing classes.

I published a codeproject article which shows you 7 tricks to significantly improve page load time even if you have large amount of Javascript used on the page.

7 Tips for Loading JavaScript Rich Web 2.0-like Sites Significantly Faster

Use Doloto
Split a Class into Multiple JavaScript Files
Stub the Functions Which Aren't Called During Initial Load
JavaScript Code in Text
Break UI Loading into Multiple Stages
Always Grow Content from Top to Bottom, Never Shrink or Jump
Deliver Browser Specific Script from Server

If you like these tricks, please vote for me!

Web 2.0 AJAX Portal using jQuery, ASP.NET 3.5, Silverlight, Linq to SQL, WF and Unity

omar — Wed, 08 Apr 2009 18:16:22 GMT

Dropthings – my open source Web 2.0 Ajax Portal has gone through a technology overhauling. Previously it was built using ASP.NET AJAX, a little bit of Workflow Foundation and Linq to SQL. Now Dropthings boasts full jQuery front-end combined with ASP.NET AJAX UpdatePanel, Silverlight widget, full Workflow Foundation implementation on the business layer, 100% Linq to SQL Compiled Queries on the data access layer, Dependency Injection and Inversion of Control (IoC) using Microsoft Enterprise Library 4.1 and Unity. It also has a ASP.NET AJAX Web Test framework that makes it real easy to write Web Tests that simulates real user actions on AJAX web pages. This article will walk you through the challenges in getting these new technologies to work in an ASP.NET website and how performance, scalability, extensibility and maintainability has significantly improved by the new technologies. Dropthings has been licensed for commercial use by prominent companies including BT Business, Intel, Microsoft IS, Denmark Government portal for Citizens; Startups like Limead and many more. So, this is serious stuff! There’s a very cool open source implementation of Dropthings framework available at National University of Singapore portal.

Visit: http://dropthings.omaralzabir.com

I have published a new article on this on CodeProject:

http://www.codeproject.com/KB/ajax/Web20Portal.aspx

Get the source code

Latest source code is hosted at Google code:

http://code.google.com/p/dropthings

There’s a CodePlex site for documentation and issue tracking:

http://www.codeplex.com/dropthings

You will need Visual Studio 2008 Team Suite with Service Pack 1 and Silverlight 2 SDK in order to run all the projects. If you have only Visual Studio 2008 Professional, then you will have to remove the Dropthings.Test project.

New features introduced

Dropthings new release has the following features:

Template users – you can define a user who’s pages and widgets are used as a template for new users. Whatever you put in that template user’s pages, it will be copied for every new user. Thus this is an easier way to define the default pages and widgets for new users. Similarly you can do the same for a registered user. The template users can be defined in the web.config.
Widget-to-Widget communication – Widgets can send message to each other. Widgets can subscribe to an Event Broker and exchange messages using a Pub-Sub pattern.
WidgetZone – you can create any number of zones in any shape on the page. You can have widgets laid in horizontal layout, you can have zones on different places on the page and so on. With this zone model, you are no longer limited to the Page-Column model where you could only have N vertical columns.
Role based widgets – now widgets are mapped to roles so that you can allow different users to see different widget list using ManageWidgetPersmission.aspx.
Role based page setup – you can define page setup for different roles. For ex, Managers see different pages and widgets than Employees.
Widget maximize – you can maximize a widget to take full screen. Handy for widgets with lots of content.
Free form resize – you can freely resize widgets vertically.
Silverlight Widgets – You can now make widgets in Silverlight!

Why the technology overhauling

Performance, Scalability, Maintainability and Extensibility – four key reasons for the overhauling. Each new technology solved one of more of these problems.

First, jQuery was used to replace my personal hand-coded large amount of Javascript code that offered the client side drag & drop and other UI effects. jQuery already has a rich set of library for Drag & Drop, Animations, Event handling, cross browser javascript framework and so on. So, using jQuery means opening the door to thousands of jQuery plugins to be offered on Dropthings. This made Dropthings highly extensible on the client side. Moreover, jQuery is very light. Unlike AJAX Control Toolkit jumbo sized framework and heavy control extenders, jQuery is very lean. So, total javascript size decreased significantly resulting in improved page load time. In total, the jQuery framework, AJAX basic framework, all my stuffs are total 395KB, sweet! Performance is key; it makes or breaks a product.

Secondly, Linq to SQL queries are replaced with Compiled Queries. Dropthings did not survive a load test when regular lambda expressions were used to query database. I could only reach up to 12 Req/Sec using 20 concurrent users without burning up web server CPU on a Quad Core DELL server.

Thirdly, Workflow Foundation is used to build operations that require multiple Data Access Classes to perform together in a single transaction. Instead of writing large functions with many if…else conditions, for…loops, it’s better to write them in a Workflow because you can visually see the flow of execution and you can reuse Activities among different Workflows. Best of all, architects can design workflows and developers can fill-in code inside Activities. So, I could design a complex operations in a workflow without writing the real code inside Activities and then ask someone else to implement each Activity. It is like handing over a design document to developers to implement each unit module, only that here everything is strongly typed and verified by compiler. If you strictly follow Single Responsibility Principle for your Activities, which is a smart way of saying one Activity does only one and very simple task, you end up with a highly reusable and maintainable business layer and a very clean code that’s easily extensible.

Fourthly, Unity Dependency Injection (DI) framework is used to pave the path for unit testing and dependency injection. It offers Inversion of Control (IoC), which enables testing individual classes in isolation. Moreover, it has a handy feature to control lifetime of objects. Instead of creating instance of commonly used classes several times within the same request, you can make instances thread level, which means only one instance is created per thread and subsequent calls reuse the same instance. Are these going over your head? No worries, continue reading, I will explain later on.

Fifthly, enabling API for Silverlight widgets allows more interactive widgets to be built using Silverlight. HTML and Javascripts still have limitations on smooth graphics and continuous transmission of data from web server. Silverlight solves all of these problems.

Read the article for details on how all these improvements were done and how all these hot techs play together in a very useful open source project for enterprises.

http://www.codeproject.com/KB/ajax/Web20Portal.aspx

Don’t forget to vote for me if you like it.

ensure - Ensure relevant Javascript and HTML are loaded before using them

omar — Mon, 09 Jun 2008 06:08:00 GMT

ensure allows you to load Javascript, HTML and CSS on-demand, whenever they are needed. It saves you from writing a gigantic Javascript framework up front so that you can ensure all functions are available whenever they are needed. It also saves you from delivering all possible html on your default page (e.g. default.aspx) hoping that they might some day be needed on some user action. Delivering Javascript, html fragments, CSS during initial loading that is not immediately used on first view makes initial loading slow. Moreover, browser operations get slower as there are lots of stuff on the browser DOM to deal with. So, ensure saves you from delivering unnecessary javascript, html and CSS up front, instead load them on-demand. Javascripts, html and CSS loaded by ensure remain in the browser and next time when ensure is called with the same Javascript, CSS or HTML, it does not reload them and thus saves from repeated downloads.

Ensure supports jQuery, Microsoft ASP.NET AJAX and Prototype framework. This means you can use it on any html, ASP.NET, PHP, JSP page that uses any of the above framework.

For example, you can use ensure to download Javascript on demand:

ensure( { js: "Some.js" }, function()
{
    SomeJS(); // The function SomeJS is available in Some.js only
});

The above code ensures Some.js is available before executing the code. If the SomeJS.js has already been loaded, it executes the function write away. Otherwise it downloads Some.js, waits until it is properly loaded and only then it executes the function. Thus it saves you from deliverying Some.js upfront when you only need it upon some user action.

Similarly you can wait for some HTML fragment to be available, say a popup dialog box. There's no need for you to deliver HTML for all possible popup boxes that you will ever show to user on your default web page. You can fetch the HTML whenever you need them.

ensure( {html: "Popup.html"}, function()
{
    // The element "Popup" is available only in Popup.html
    document.getElementById("Popup").style.display = "";    
});

The above code downloads the html from "Popup.html" and adds it into the body of the document and then fires the function. So, you code can safely use the UI element from that html.

You can mix match Javascript, html and CSS altogether in one ensure call. For example,

ensure( { js: "popup.js", html: "popup.html", css: "popup.css" }, function()
{
    PopupManager.show();
});

You can also specify multiple Javascripts, html or CSS files to ensure all of them are made available before executing the code:

ensure( { js: ["blockUI.js","popup.js"], html: ["popup.html", "blockUI.html"], css: ["blockUI.css", "popup.css"] }, function()
{
    BlockUI.show();
    PopupManager.show();
});

You might think you are going to end up writing a lot of ensure code all over your Javascript code and result in a larger Javascript file than before. In order to save you javascript size, you can define shorthands for commonly used files:

var JQUERY = { js: "jquery.js" };
var POPUP = { js: ["blockUI.js","popup.js"], html: ["popup.html", "blockUI.html"], css: ["blockUI.css", "popup.css"] };
...
...
ensure( JQUERY, POPUP, function() {
    $("DeleteConfirmPopupDIV").show();
});
...
...
ensure( POPUP, function()
{
    $("SaveConfirmationDIV").show();
);

While loading html, you can specify a container element where ensure can inject the loaded HTML. For example, you can say load HtmlSnippet.html and then inject the content inside a DIV named "exampleDiv"

ensure( { html: ["popup.html", "blockUI.html"], parent: "exampleDiv"}, function(){});

You can also specify Javascript and CSS that will be loaded along with the html.

How ensure works

The following CodeProject article explains in detail how ensure it built. Be prepared for a high dose of Javascript techniques:

http://www.codeproject.com/KB/ajax/ensure.aspx

If you find ensure useful, please vote for me.

Download Code

Download latest source code from CodePlex: www.codeplex.com/ensure

Share this post :

UFrame: goodness of UpdatePanel and IFRAME combined

omar — Sat, 24 May 2008 19:31:00 GMT

UFrame combines the goodness of UpdatePanel and IFRAME in a cross browser and cross platform solution. It allows a DIV to behave like an IFRAME loading content from any page either static or dynamic. It can load pages having both inline and external Javascript and CSS, just like an IFRAME. But unlike IFRAME, it loads the content within the main document and you can put any number of UFrame on your page without slowing down the browser. It supports ASP.NET postback nicely and you can have DataGrid or any other complex ASP.NET control within a UFrame. UFrame works perfectly with ASP.NET MVC making it an replacement for UpdatePanel. Best of all, UFrame is implemented 100% in Javascript making it a cross platform solution. As a result, you can use UFrame on ASP.NET, PHP, JSP or any other platform.

<div class="UFrame" id="UFrame1" src="SomePage.aspx?ID=UFrame1" >
  <p>This should get replaced with content from Somepage.aspx</p>
</div>

Response from SomePage.aspx is rendered directly inside the UFrame. Here you see two UFrame's are used to load the same SomePage.aspx as if they are loaded inside IFRAME. Another UFrame is used to load AnotherPage.aspx that shows photos from Flickr.

See it in action!

You can test UFrame from:

http://labs.omaralzabir.com/UFrame2005 - Visual Studio 2005 version, .NET 2.0 implementation showing regular ASP.NET 2.0 controls work as usual
http://labs.omaralzabir.com/UFrameMvc - Visual Studio 2008 version shows ASP.NET MVC works fine making UFrame an ultimate replacement for UpdatePanel

What is UFrame?

UFrame can load and host a page (ASP.NET, PHP or regular html) inside a DIV. Unlike IFRAME which loads the content inside a browser frame that has no relation with the main document, UFrame loads the content within the same document. Thus all the Javascripts, CSS on the main document flows through the loaded content. It's just like UpdatePanel with IFRAME's src attribute.

The above UFrames are declared like this:

<div id="UFrame1" src="SomePage.aspx" >
    <p>This should get replaced with content from Somepage.aspx</p>
</div>

The features of UFrame are:

You can build regular ASP.NET/PHP/JSP/HTML page and make them behave as if they are fully AJAX enabled! Simple regular postback will work as if it's an UpdatePanel, or simple hyperlinks will behave as if content is being loaded using AJAX.
Load any URL inside a DIV. It can be a PHP, ASP.NET, JSP or regular HTML page.
Just like IFRAME, you can set src property of DIVs and they are converted to UFrames when UFrame library loads.
Unlike IFRAME, it loads the content within the main document. So, main document's CSS and Javascripts are available to the loaded content.
It allows you to build parts of a page as multiple fully independent pages.
Each page is built as standalone page. You can build, test and debug each small page independently and put them together on the main page using UFrames.
It loads and executes both inline and external scripts from loaded page. You can also render different scripts during UFrame postback.
All external scripts are loaded before the body content is set. And all inline scripts are executed when both external scripts and body has been loaded. This way the inline scripts execute when the body content is already available.
It loads both inline and external CSS.
It handles duplicates nicely. It does not load the same external Javascript or CSS twice.

Download the code

You can download latest version of UFrame along with the VS 2005 and VS 2008 (MVC) example projects from CodePlex:

www.codeplex.com/uframe

Please go to the "Source Code" tab for the latest version. You are invited to join the project and improve it or fix bugs.

Read the article about UFrame

I have published an article about UFrame at CodeProject:

http://www.codeproject.com/KB/aspnet/uframe.aspx

The article explains in details how the UFrame is built. Be prepared for a big dose of Javascript code.

If you find UFrame or the article useful, please vote for me at CodeProject.

Fast ASP.NET web page loading by downloading multiple javascripts in batch

omar — Sat, 10 May 2008 00:31:00 GMT

A web page can load a lot faster and feel faster if the javascripts on the page can be loaded after the visible content has been loaded and multiple javascripts can be batched into one download. Browsers download one external script at a time and sometimes pause rendering while a script is being downloaded and executed. This makes web pages load and render slow when there are multiple javascripts on the page. For every javascript reference, browser stops downloading and processing of any other content on the page and some browsers (like IE6) pause rendering while it processes the javascript. This gives a slow loading experience and the web page kind of gets 'stuck' frequently. As a result, a web page can only load fast when there are small number of external scripts on the page and the scripts are loaded after the visible content of the page has loaded.

Here's an example, when you visit http://dropthings.omaralzabir.com, you see a lot of Javascripts downloading. Majority of these are from the ASP.NET AJAX framework and the ASP.NET AJAX Control Toolkit project.

Figure: Many scripts downloaded on a typical ASP.NET AJAX page having ASP.NET AJAX Control Toolkit

As you see, browser gets stuck for 15 times as it downloads and processes external scripts. This makes page loading "feel" slower. The actual loading time is also pretty bad because these 15 http requests waste 15*100ms = 1500ms on the network latency inside USA. Outside USA, the latency is even higher. Asia gets about 270ms and Australia gets about 380ms latency from any server in USA. So, users outside USA wastes 4 to 6 seconds on network latency where no data is being downloaded. This is an unacceptable performance for any website.

You pay for such high number of script downloads only because you use two extenders from AJAX Control Toolkit and the UpdatePanel of ASP.NET AJAX.

If we can batch the multiple individual script calls into one call like Scripts.ashx as shown in the picture below and download several scripts together in one shot using an HTTP Handler, it saves us a lot of http connection which could be spent doing other valuable work like downloading CSS for the page to show content properly or downloading images on the page that is visible to user.

Figure: Download several javascripts over one connection and save call and latency

The Scripts.ashx handler can not only download multiple scripts in one shot, but also has a very short URL form. For example:

/scripts.ashx?initial=a,b,c,d,e&/

Compared to conventional ASP.NET ScriptResource URLs like:

/ScriptResource.axd?d=WzuUYZ-Ggi7-B0tkhjPDTmMmgb5FPLmciWEXQLdjNjt
bmek2jgmm3QETspZjKLvHue5em5kVYJGEuf4kofrcKNL9z6AiMhCe3SrJrcBel_c1
&t=633454272919375000

The benefits of downloading multiple Javascript over one http call are:

Saves expensive network roundtrip latency where neither browser nor the origin server is doing anything, not even a single byte is being transmitted during the latency
Create less "pause" moments for the browser. So, browser can fluently render the content of the page and thus give user a fast loading feel
Give browser move time and free http connections to download visible artifacts of the page and thus give user a "something's happening" feel
When IIS compression is enabled, the total size of individually compressed files is greater than multiple files compressed after they are combined. This is because each compressed byte stream has compression header in order to decompress the content.
This reduces the size of the page html as there are only a few handful of script tag. So, you can easily saves hundreds of bytes from the page html. Especially when ASP.NET AJAX produces gigantic WebResource.axd and ScriptResource.axd URLs that have very large query parameter

The solution is to dynamically parse the response of a page before it is sent to the browser and find out what script references are being sent to the browser. I have built an http module which can parse the generated html of a page and find out what are the script blocks being sent. It then parses those script blocks and find the scripts that can be combined. Then it takes out those individual script tags from the response and adds one script tag that generates the combined response of multiple script tags.

For example, the homepage of Dropthings.com produces the following script tags:

<script type="text/javascript">
...
//]]>
</script>
<script src="/Dropthings/WebResource.axd?d=_w65Lg0FVE-htJvl4_zmXw2&amp;t=633403939286875000" 
type="text/javascript"></script>
...
<script src="Widgets/FastFlickrWidget.js" type="text/javascript"></script>
<script src="Widgets/FastRssWidget.js" type="text/javascript"></script>
<script src="/Dropthings/ScriptResource.axd?d=WzuUYZ-Ggi7-B0tkhjPDTmMmgb5FPLmciWEXQLdj
Njtbmek2jgmm3QETspZjKLvHue5em5kVYJGEuf4kofrcKNL9z6AiMhCe3SrJrcBel_c1
&amp;t=633454272919375000" type="text/javascript"></script>
<script type="text/javascript">
//<![CDATA[
...
</script>
<script src="/Dropthings/ScriptResource.axd?d=WzuUYZ-Ggi7-B0tkhjPDTmMmgb5FPLmciWEXQLdjNjtbmek2j
gmm3QETspZjKLvHIbaYWwsewvr_eclXZRGNKzWlaVj44lDEdg9CT2tyH-Yo9jFoQij_XIWxZNETQkZ90
&amp;t=633454272919375000" type="text/javascript"></script>
<script type="text/javascript">
...
</script>
<script type="text/javascript">
...
</script>
<script type="text/javascript" charset="utf-8">
...
</script>
<script src="Myframework.js" type="text/javascript"></script>
<script type="text/javascript">
...
</script>
<script type="text/javascript">if( typeof Proxy == "undefined" ) Proxy = ProxyAsync;</script>
<script type="text/javascript">
...
</script>
<script src="/Dropthings/ScriptResource.axd?d=WzuUYZ-Ggi7-B0tkhjPDTmMmgb5FPLmciWEXQLdjN
jtbmek2jgmm3QETspZjKLvH-H5JQeA1OWzBaqnbKRQWwc2hxzZ5M8vtSrMhytbB-Oc1
&amp;t=633454272919375000" type="text/javascript"></script>
<script src="/Dropthings/ScriptResource.axd?d=BXpG1T2rClCdn7QzWc-HrzQ2ECeqBhG6oiVakhRAk
RY6YSaFJsnzqttheoUJJXE4jMUal_1CAxRvbSZ_4_ikAw2
&amp;t=633454540450468750" type="text/javascript"></script>
<script src="/Dropthings/ScriptResource.axd?d=BXpG1T2rClCdn7QzWc-HrzQ2ECeqBhG6oiVakhRA
kRYRhsy_ZxsfsH4NaPtFtpdDEJ8oZaV5wKE16ikC-hinpw2
&amp;t=633454540450468750" type="text/javascript"></script>
<script src="/Dropthings/ScriptResource.axd?d=BXpG1T2rClCdn7QzWc-HrzQ2ECeqBhG6oiVakhRAk
RZbimFWogKpiYN4SVreNyf57osSvFc_f24oloxX4RTFfnfj5QsvJGQanl-pbbMbPf01
&amp;t=633454540450468750" type="text/javascript"></script>
...

<script type="text/javascript">
...
</script>
</body>
</html>

As you see, there are lots of large script tags, in total 15 of them. The solution I will show here will combine the script links and replace with two script links that download 13 of the individual scripts. I have left two scripts out that are related to ASP.NET AJAX Timer extender.

<script type="text/javascript">
...
</script>



<script type="text/javascript" src="Scripts.ashx?initial=a,b,c,d,e,f&/dropthings/"></script>

<script type="text/javascript">
...
</script>

<script type="text/javascript">
...
</script>
<script type="text/javascript">
...
</script>

<script type="text/javascript">
...
</script>

<script type="text/javascript">if( typeof Proxy == "undefined" ) Proxy = ProxyAsync;</script>
<script type="text/javascript">
...
</script>
<script src="/Dropthings/ScriptResource.axd?d=WzuUYZ-..." type="text/javascript"></script>
<script src="/Dropthings/ScriptResource.axd?d=BXpG1T2..." type="text/javascript"></script>

<script type="text/javascript" src="Scripts.ashx?post=C,D,E,F,G,H,I,J&/dropthings/"></script>

<script type="text/javascript">
...
</script>

As you see, 13 of the script links have been combined into two script links. The URL is also smaller than majority of the script references.

There are two steps involved here:

Find out all the script tags being emitted inside generated response HTML and collect them in a buffer. Move them after the visible artifacts in the HTML, especially the <form> tag that contains the generated output of all ASP.NET controls on the page
Parse the buffer and see which script references can be combined into one set. The sets are defined in a configuration file. Replace the individual script references with the combined set reference.

The whole solution is explained in this CodeProject article:

Fast ASP.NET web page loading by downloading multiple javascripts after visible content and in batch
http://www.codeproject.com/KB/aspnet/fastload.aspx

You should be able to use this approach in any ASP.NET (even better if AJAX) application and give your site a big performance boost.

If you like the idea, please vote for me.

Fast page loading by moving ASP.NET AJAX scripts after visible content

omar — Sun, 06 Apr 2008 14:49:00 GMT

ASP.NET ScriptManager control has a property LoadScriptsBeforeUI, when set to false, should load all AJAX framework scripts after the content of the page. But it does not effectively push down all scripts after the content. Some framework scripts, extender scripts and other scripts registered by Ajax Control Toolkit still load before the page content loads. The following screen taken from www.dropthings.com shows several script tags are still added at the beginning of <form> which forces them to download first before the page content is loaded and displayed on the page. Script tags pause rendering on several browsers especially in IE until the scripts download and execute. As a result, it gives user a slow loading impression as user stares at a white screen for some time until the scripts before the content download and execute completely. If browser could render the html before it downloads any script, user would see the page content immediately after visiting the site and not see a white screen. This will give user an impression that the website is blazingly fast (just like Google homepage) because user will ideally see the page content, if it's not too large, immediately after hitting the URL.

Figure: Script blocks being delivered before the content

From the above screen shot you see there are some scripts from ASP.NET AJAX framework and some scripts from Ajax Control Toolkit that are added before the content of the page. Until these scripts download, browser don't see anything on the UI and thus you get a pause in rendering giving user a slow load feeling. Each script to external URL adds about 200ms avg network roundtrip delay outside USA while it tries to fetch the script. So, user basically stares at a white screen for at least 1.5 sec no matter how fast internet connection he/she has.

These scripts are rendered at the beginning of form tag because they are registered using Page.ClientScript.RegisterClientScriptBlock. Inside Page class of System.Web, there's a method BeginFormRender which renders the client script blocks immediately after the form tag.

   1: internal void BeginFormRender(HtmlTextWriter writer, string formUniqueID)

   2: {

   3:     ...

   4:         this.ClientScript.RenderHiddenFields(writer);

   5:         this.RenderViewStateFields(writer);

   6:         ...

   7:         if (this.ClientSupportsJavaScript)

   8:         {

   9:             ...

  10:             if (this._fRequirePostBackScript)

  11:             {

  12:                 this.RenderPostBackScript(writer, formUniqueID);

  13:             }

  14:             if (this._fRequireWebFormsScript)

  15:             {

  16:                 this.RenderWebFormsScript(writer);

  17:             }

  18:         }

  19:         this.ClientScript.RenderClientScriptBlocks(writer);

  20: }

Figure: Decompiled code from System.Web.Page class

Here you see several script blocks including scripts registered by calling ClientScript.RegisterClientScriptBlock are rendered right after form tag starts.

There's no easy work around to override the BeginFormRender method and defer rendering of these scripts. These rendering functions are buried inside System.Web and none of these are overridable. So, the only solution seems to be using a Response Filter to capture the html being written and suppress rendering the script blocks until it's the end of the body tag. When the </body> tag is about to be rendered, we can safely assume page content has been successfully delivered and now all suppressed script blocks can be rendered at once.

In ASP.NET 2.0, you to create Response Filter which is an implementation of a Stream. You can replace default Response.Filter with your own stream and then ASP.NET will use your filter to write the final rendered HTML. When Response.Write is called or Page's Render method fires, the response is written to the output stream via the filter. So, you can intercept every byte that's going to be sent to the client (browser) and modify it the way you like. Response Filters can be used in variety ways to optimize Page output like stripping off all white spaces or doing some formatting on the generated content, or manipulating the characters being sent to the browser and so on.

I have created a Response filter which captures all characters being sent to the browser. It it finds that script blocks are being rendered, instead of rendering it to the Response.OutputStream, it will extract the script blocks out of the buffer being written and render the rest of the content. It stores all script blocks, both internal and external, in a string buffer. When it detects </body> tag is about to be written to the response, it flushes all the captured script blocks from the string buffer.

   1: public class ScriptDeferFilter : Stream

   2: {

   3:     Stream responseStream;

   4:     long position;

5:

   6:     /// <summary>

   7:     /// When this is true, script blocks are suppressed and captured for

   8:     /// later rendering

   9:     /// </summary>

  10:     bool captureScripts;

11:

  12:     /// <summary>

  13:     /// Holds all script blocks that are injected by the controls

  14:     /// The script blocks will be moved after the form tag renders

  15:     /// </summary>

  16:     StringBuilder scriptBlocks;

17:

  18:     Encoding encoding;

19:

  20:     public ScriptDeferFilter(Stream inputStream, HttpResponse response)

  21:     {

  22:         this.encoding = response.Output.Encoding;

  23:         this.responseStream = response.Filter;

24:

  25:         this.scriptBlocks = new StringBuilder(5000);

  26:         // When this is on, script blocks are captured and not written to output

  27:         this.captureScripts = true;

  28:     }

Here's the beginning of the Filter class. When it initializes, it takes the original Response Filter. Then it overrides the Write method of the Stream so that it can capture the buffers being written and do it's own processing.

   1: public override void Write(byte[] buffer, int offset, int count)

   2: {

   3:     // If we are not capturing script blocks anymore, just redirect to response stream

   4:     if (!this.captureScripts)

   5:     {

   6:         this.responseStream.Write(buffer, offset, count);

   7:         return;

   8:     }

9:

  10:     /*

  11:      * Script and HTML can be in one of the following combinations in the specified buffer:

  12:      * .....<script ....>.....</script>.....

  13:      * <script ....>.....</script>.....

  14:      * <script ....>.....</script>

  15:      * <script ....>.....</script> .....

  16:      * ....<script ....>.....

  17:      * <script ....>.....

  18:      * .....</script>.....

  19:      * .....</script>

  20:      * <script>.....

  21:      * .... </script>

  22:      * ......

  23:      * Here, "...." means html content between and outside script tags

  24:     */

25:

  26:     char[] content = this.encoding.GetChars(buffer, offset, count);

27:

  28:     int scriptTagStart = 0;

  29:     int lastScriptTagEnd = 0;

  30:     bool scriptTagStarted = false;

31:

  32:     for (int pos = 0; pos < content.Length; pos++)

  33:     {

  34:         // See if tag start

  35:         char c = content[pos];

  36:         if (c == '<')

  37:         {

  38:             int tagStart = pos;

  39:             // Check if it's a tag ending

  40:             if (content[pos+1] == '/')

  41:             {

  42:                 pos+=2; // go past the </

43:

  44:                 // See if script tag is ending

  45:                 if (isScriptTag(content, pos))

  46:                 {

  47:                     /// Script tag just ended. Get the whole script

  48:                     /// and store in buffer

  49:                     pos = pos + "script>".Length;

  50:                     scriptBlocks.Append(content, scriptTagStart, pos - scriptTagStart);

  51:                     scriptBlocks.Append(Environment.NewLine);

  52:                     lastScriptTagEnd = pos;

53:

  54:                     scriptTagStarted = false;

  55:                     continue;

  56:                 }

  57:                 else if (isBodyTag(content, pos))

  58:                 {

  59:                     /// body tag has just end. Time for rendering all the script

  60:                     /// blocks we have suppressed so far and stop capturing script blocks

61:

  62:                     if (this.scriptBlocks.Length > 0)

  63:                     {

  64:                         // Render all pending html output till now

  65:                         this.WriteOutput(content, lastScriptTagEnd, tagStart - lastScriptTagEnd);

66:

  67:                         // Render the script blocks

  68:                         byte[] scriptBytes = this.encoding.GetBytes(this.scriptBlocks.ToString());

  69:                         this.responseStream.Write(scriptBytes, 0, scriptBytes.Length);

70:

  71:                         // Stop capturing for script blocks

  72:                         this.captureScripts = false;

73:

  74:                         // Write from the body tag start to the end of the inut buffer and return

  75:                         // from the function. We are done.

  76:                         this.WriteOutput(content, tagStart, content.Length - tagStart);

  77:                         return;

  78:                     }

  79:                 }

  80:                 else

  81:                 {

  82:                     // some other tag's closing. safely skip one character as smallest

  83:                     // html tag is one character e.g. <b>. just an optimization to save one loop

  84:                     pos++;

  85:                 }

  86:             }

  87:             else

  88:             {

  89:                 if (isScriptTag(content, pos+1))

  90:                 {

  91:                     /// Script tag started. Record the position as we will

  92:                     /// capture the whole script tag including its content

  93:                     /// and store in an internal buffer.

  94:                     scriptTagStart = pos;

95:

  96:                     // Write html content since last script tag closing upto this script tag

  97:                     this.WriteOutput(content, lastScriptTagEnd, scriptTagStart - lastScriptTagEnd);

98:

  99:                     // Skip the tag start to save some loops

 100:                     pos += "<script".Length;

 101:

 102:                     scriptTagStarted = true;

 103:                 }

 104:                 else

 105:                 {

 106:                     // some other tag started

 107:                     // safely skip 2 character because the smallest tag is one character e.g. <b>

 108:                     // just an optimization to eliminate one loop

 109:                     pos++;

 110:                 }

 111:             }

 112:         }

 113:     }

 114:

 115:     // If a script tag is partially sent to buffer, then the remaining content

 116:     // is part of the last script block

 117:     if (scriptTagStarted)

 118:     {

 119:

 120:         this.scriptBlocks.Append(content, scriptTagStart, content.Length - scriptTagStart);

 121:     }

 122:     else

 123:     {

 124:         /// Render the characters since the last script tag ending

 125:         this.WriteOutput(content, lastScriptTagEnd, content.Length - lastScriptTagEnd);

 126:     }

 127: }

There are several situations to consider here. The Write method is called several times during the Page render process because the generated HTML can be quite big. So, it will contain partial HTML. So, it's possible the first Write call contains a start of a script block, but no ending script tag. The following Write call may or may not have the ending script block. So, we need to preserve state to make sure we don't overlook any script block. Each Write call can have several script block in the buffer as well. It can also have no script block and only page content.

The idea here is to go through each character and see if there's any starting script tag. If there is, remember the start position of the script tag. If script end tag is found within the buffer, then extract out the whole script block from the buffer and render the remaining html. If there's no ending tag found but a script tag did start within the buffer, then suppress output and capture the remaining content within the script buffer so that next call to Write method can grab the remaining script and extract it out from the output.

There are two other private functions that are basically helper functions and does not do anything interesting:

   1: private void WriteOutput(char[] content, int pos, int length)

   2: {

   3:     if (length == 0) return;

4:

   5:     byte[] buffer = this.encoding.GetBytes(content, pos, length);

   6:     this.responseStream.Write(buffer, 0, buffer.Length);

   7: }

8:

   9: private bool isScriptTag(char[] content, int pos)

  10: {

  11:     if (pos + 5 < content.Length)

  12:         return ((content[pos] == 's' || content[pos] == 'S')

  13:             && (content[pos + 1] == 'c' || content[pos + 1] == 'C')

  14:             && (content[pos + 2] == 'r' || content[pos + 2] == 'R')

  15:             && (content[pos + 3] == 'i' || content[pos + 3] == 'I')

  16:             && (content[pos + 4] == 'p' || content[pos + 4] == 'P')

  17:             && (content[pos + 5] == 't' || content[pos + 5] == 'T'));

  18:     else

  19:         return false;

20:

  21: }

22:

  23: private bool isBodyTag(char[] content, int pos)

  24: {

  25:     if (pos + 3 < content.Length)

  26:         return ((content[pos] == 'b' || content[pos] == 'B')

  27:             && (content[pos + 1] == 'o' || content[pos + 1] == 'O')

  28:             && (content[pos + 2] == 'd' || content[pos + 2] == 'D')

  29:             && (content[pos + 3] == 'y' || content[pos + 3] == 'Y'));

  30:     else

  31:         return false;

  32: }

The isScriptTag and isBodyTag functions may look weird. The reason for such weird code is pure performance. Instead of doing fancy checks like taking a part of the array out and doing string comparison, this is the fastest way of doing the check. Best thing about .NET IL is that it's optimized, if any of the condition in the && pairs fail, it won't even execute the rest. So, this is as best as it can get to check for certain characters.

There are some corner cases that are not handled here. For example, what if the buffer contains a partial script tag declaration. For example, "....<scr" and that's it. The remaining characters did not finish in the buffer instead next buffer is sent with the remaining characters like "ipt src="..." >.....</scrip". In such case, the script tag won't be taken out. One way to handle this would be to make sure you always have enough characters left in the buffer to do a complete tag name check. If not found, store the half finished buffer somewhere and on next call to Write, combine it with the new buffer sent and do the processing.

In order to install the Filter, you need to hook it in in the Global.asax BeginRequest or some other event that's fired before the Response is generated.

   1: protected void Application_BeginRequest(object sender, EventArgs e)

   2: {

   3:     if (Request.HttpMethod == "GET")

   4:     {

   5:         if (Request.AppRelativeCurrentExecutionFilePath.EndsWith(".aspx"))

   6:         {

   7:             Response.Filter = new ScriptDeferFilter(Response);

   8:         }

   9:     }

  10: }

Here I am hooking the Filter only for GET calls to .aspx pages. You can hook it to POST calls as well. But asynchronous postbacks are regular POST and I do not want to do any change in the generated JSON or html fragment. Another way is to hook the filter only when ContentType is text/html.

When this filter is installed, www.dropthings.com defers all script loading after the <form> tag completes.

Figure: Script tags are moved after the <form> tag when the filter is used

You can grab the Filter class from the App_Code\ScriptDeferFilter.cs of the Dropthings project. Go to CodePlex site and download the latest code for the latest filter.

On-demand UI loading on AJAX websites

omar — Fri, 09 Nov 2007 09:01:00 GMT

AJAX websites are all about loading as many features as possible into the browser without having any postback. If you look at the Start Pages like Pageflakes, it's only one single page that gives you all the features of the whole application with zero postback. A quick and dirty approach for doing this is to deliver every possible html snippets inside hidden divs during page load and then make those divs visible when needed. But this makes first time loading way too long and browser gives sluggish performance as too much stuff is on the DOM. So, a better approach is to load html snippet and necessary javascript on-demand. In my dropthings project, I have shown an example how this is done.

When you click on the "help" link, it loads the content of the help dynamically. This html is not delivered as part of the default.aspx that renders the first page. Thus the giant html and graphics related to the help section has no effect on site load performance. It is only loaded when user clicks the "help" link. Moreover, it gets cached on the browser and thus loads only once. When user clicks the "help" link again, it's served directly from the browser cache, instead of fetching from the origin server again.

The principle is making an XMLHTTP call to an .aspx page, get the response html, put that response html inside a container DIV, make that DIV visible.

AJAX framework has a Sys.Net.WebRequest class which you can use to make regular HTTP calls. You can define http method, URI, headers and the body of the call. It’s kind of a low level function for making direct calls via XMLHTTP. Once you construct a web request, you can execute it using Sys.Net.XMLHttpExecutor.

   1: function showHelp()

   2: {

   3:     var request = 
new Sys.Net.WebRequest();

   4:     request.set_httpVerb(
"GET");

   5:     request.set_url(
'help.aspx');

   6:     request.add_completed(
function( executor )

   7:     {

   8:         
if (executor.get_responseAvailable())

   9:         {

  10:             var helpDiv = $get(
'HelpDiv');

  11:             var helpLink = $get(
'HelpLink');

12:

  13:             var helpLinkBounds =
Sys.UI.DomElement.getBounds(helpLink);

14:

  15:             helpDiv.style.top =
(helpLinkBounds.y + helpLinkBounds.height) + 
"px";

16:

  17:             var content =
executor.get_responseData();

  18:             helpDiv.innerHTML =
content;

  19:             helpDiv.style.display =

"block";

  20:         }

  21:     });

22:

  23:     var executor = 
new Sys.Net.XMLHttpExecutor();

  24:     request.set_executor(executor);

  25:     executor.executeRequest();

  26: }

The example shows how the help section is loaded by hitting help.aspx and injecting its response inside the HelpDiv. The response can be cached by the output cache directive set on help.aspx. So, next time when user clicks on the link, the UI pops up immediately. The help.aspx file has no <html> block, only the content that is set inside the DIV.

   1: 
<%@ Page Language="C#" AutoEventWireup="true"
CodeFile="Help.aspx.cs" Inherits="Help" %>

   2: 
<%@ OutputCache Location="ServerAndClient"
Duration="604800" VaryByParam="none" %>

   3: 
<
div 
class
="helpContent"
>

   4: 
<
div 
id
="lipsum"
>

   5: 
<
p
>

   6: Lorem ipsum dolor sit amet,
consectetuer adipiscing elit. Duis lorem

   7: eros, volutpat sit amet, venenatis
vitae, condimentum at, dolor. Nunc

   8: porttitor eleifend tellus. Praesent
vitae neque ut mi rutrum cursus.

Using this approach, you can break the UI into smaller .aspx files. Although these .aspx files cannot have JavaScript or stylesheet blocks, but they can contain large amount of html that you need to show on the UI on-demand. Thus you can keep initial download to absolute minimum just for loading the basic stuffs. When user explores new features on the site, load those areas incrementally.

10 cool web development related articles in 2007

omar — Tue, 02 Oct 2007 17:26:00 GMT

Here's a list of 10 cool ASP.NET, AJAX and web development related articles and blog posts that I have written this year that you might want to take a look:

13 disasters for production website and their solutions
Talks about 13 production disasters that can happen to any website any time and bring down your business.

Build Google IG like Ajax Start Page in 7 days using ASP.NET Ajax and .NET 3.0
This block buster article shows how ASP.NET AJAX, Linq to XML, Linq to SQL and Workflow Foundation can be used to create a Google IG like start page in just 7 nights. Learn how to put together such hot technologies into one project and make a production site out of it.

Serve extensionless URL from ASP.NET without using ISAPI module or IIS 6 Wildcard mapping
Currently there are only two ways to service extentionless URL like www.pageflakes.com/omar that hits something besides the default document - use a custom ISAPI module or use IIS 6 wildcard mapping. Both has performance and scalability problems because both intercepts each and every hit. Learn how you can solve it by using a custom 404 handler.

Request format is unrecognized for URL unexpectedly ending in /SomeWebServiceMethod
Since ASP.NET AJAX 1.0 release, Microsoft prevented JSON hijacking by adding a special content type header. But this caused us some trouble.

Cleanup inactive anonymous users from ASP.NET Membership Tables
When you store anonymous user profile using ASP.NET Membership provider and Anonymous Identification provider, you soon end up with lots of idle anonymous user data where those users never come back. We (Pageflakes) went through a lot of difficulty keeping our database size down as we allow anonymous users to do almost everything that a registered user can do. This introduces scalability challenge. See how we solved this problem.

Prevent Denial of Service (DOS) attacks in your web application
Web applications can be brought down to its knees by hitting the site repeatedly or by calling expensive webservices randomly. Anyone can write a simple loop that hits a webserver very frequently from a high bandwidth connectivity and bring your production server down. See how to prevent such application level DOS attacks.

ASP.NET Ajax Extender for multi-column widget drag & drop
It's an ASP.NET AJAX extender that allows Pageflakes style drag & drop functionality between columns and rows.

ASP.NET Ajax in-depth performance analysis
While building an open source start page using ASP.NET AJAX, I have done a lot of performance analysis on AJAX framework in order to improve first time load and perceived speed of javascript rich pages. Check out my analysis.

Think you know how to write UPDATE statement? Think again.
Learn how to optimize common UPDATE statements

Make a surveillance application which captures desktop and then emails you as attachment
Some time back I needed to capture a certain computers desktop in order to find out what that user is doing every day. So, I made a .NET 2.0 Winforms Application which stays on system tray (optional) and capture the desktop in given time interval (say every 60 secs) and emailed the captured images to me as message attachment (say every 30 mins).

Today I received MVP award for the 3rd time on Visual C#. Thanks to Microsoft for the award and setting up my new blog. I will continue both my MVPS Blog and this blog from now on.