Omar AL Zabir blog on ASP.NET Ajax and .NET 3.5 : asp.net ajax

Fast, Streaming AJAX proxy - continuously download from cross domain

omar — Mon, 14 Apr 2008 07:29:00 GMT

Due to browser's prohibition on cross domain XMLHTTP call, all AJAX websites must have server side proxy to fetch content from external domain like Flickr or Digg. From client side javascript code, an XMLHTTP call goes to the server side proxy hosted on the same domain and then the proxy downloads the content from the external server and sends back to the browser. In general, all AJAX websites on the Internet that are showing content from external domains are following this proxy approach except some rare ones who are using JSONP. Such a proxy gets a very large number of hits when a lot of component on the website are downloading content from external domains. So, it becomes a scalability issue when the proxy starts getting millions of hits. Moreover, web page's overall load performance largely depends on the performance of the proxy as it delivers content to the page. In this article, we will take a look how we can take a conventional AJAX Proxy and make it faster, asynchronous, continuously stream content and thus make it more scalable.

You can see such a proxy in action when you go to Pageflakes.com. You will see flakes (widgets) loading many different content like weather feed, flickr photo, youtube videos, RSS from many different external domains. All these are done via a Content Proxy. Content Proxy served about 42.3 million URLs last month which is quite an engineering challenge for us to make it both fast and scalable. Sometimes Content Proxy serves megabytes of data, which poses even greater engineering challenge. As such proxy gets large number of hits, if we can save on an average 100ms from each call, we can save 4.23 million seconds of download/upload/processing time every month. That's about 1175 man hours wasted throughout the world by millions of people staring at browser waiting for content to download.

Such a content proxy takes an external server's URL as a query parameter. It downloads the content from the URL and then writes the content as response back to browser.

Figure: Content Proxy working as a middleman between browser and external domain

The above timeline shows how request goes to the server and then server makes a request to external server, downloads the response and then transmits back to the browser. The response arrow from proxy to browser is larger than the response arrow from external server to proxy because generally proxy server's hosting environment has better download speed than the user's Internet connectivity.

Such a content proxy is also available in my open source Ajax Web Portal Dropthings.com. You can see from its code how such a proxy is implemented.

The following is a very simple synchronous, non-streaming, blocking Proxy:

[WebMethod]
[ScriptMethod(UseHttpGet=true)]
public string GetString(string url)
{
        using (WebClient client = new WebClient())
        {
            string response = client.DownloadString(url);
            return response;
        }
    }
}

Although it shows the general principle, but it's no where close to a real proxy because:

It's a synchronous proxy and thus not scalable. Every call to this web method causes the ASP.NET thread to wait until the call to the external URL completes.
It's non streaming. It first downloads the entire content on the server, storing it in a string and then uploading that entire content to the browser. If you pass MSDN feed URL, it will download that gigantic 220 KB RSS XML on the server and store it on a 220 KB long string (actually double the size as .NET strings are all Unicode string) and then write 220 KB to ASP.NET Response buffer, consuming another 220 KB UTF8 byte array in memory. Then that 220 KB will be passed to IIS in chunks so that it can transmit it to the browser.
It does not produce proper response header to cache the response on the server. Nor does it deliver important headers like Content-Type from the source.
If external URL is providing gzipped content, it decompresses the content into a string representation and thus wastes server memory.
It does not cache the content on the server. So, repeated call to the same external URL within the same second or minute will download content from the external URL and thus waste bandwidth on your server.

So, we need an asynchronous streaming proxy that transmits the content to the browser while it downloads from the external domain server. So, it will download bytes from external URL in small chunks and immediately transmit that to the browser. As a result, browser will see a continuous transmission of bytes right after calling the web service. There will be no delay while the content is fully downloaded on the server.

Before I show you the complex streaming proxy code, let's take an evolutionary approach. Let's build a better Content Proxy that the one shown above, which is synchronous, non-streaming but does not have the other problems mentioned above. We will build a HTTP Handler named RegularProxy.ashx which will take url as a query parameter. It will also take cache as a query parameter which it will use to produce proper response headers in order to cache the content on the browser. Thus it will save browser from downloading the same content again and again.

<%@ WebHandler Language="C#" Class="RegularProxy" %>

using System;
using System.Web;
using System.Web.Caching;
using System.Net;
using ProxyHelpers;
public class RegularProxy : IHttpHandler {
        
    public void ProcessRequest (HttpContext context) {
        string url = context.Request["url"];
        int cacheDuration = Convert.ToInt32(context.Request["cache"]?? "0");
        string contentType = context.Request["type"];

        // We don't want to buffer because we want to save memory
        context.Response.Buffer = false;
            
        // Serve from cache if available
        if (context.Cache[url] != null)
        {
            context.Response.BinaryWrite(context.Cache[url] as byte[]);
            context.Response.Flush();
            return;
        }
        using (WebClient client = new WebClient())
        {
            if (!string.IsNullOrEmpty(contentType))
                client.Headers["Content-Type"] = contentType;
            
            client.Headers["Accept-Encoding"] = "gzip";
            client.Headers["Accept"] = "*/*";
            client.Headers["Accept-Language"] = "en-US";
            client.Headers["User-Agent"] = 
"Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6";
            
            byte[] data = client.DownloadData(url);

            context.Cache.Insert(url, data, null,
                        Cache.NoAbsoluteExpiration,
                        TimeSpan.FromMinutes(cacheDuration),
                        CacheItemPriority.Normal, null); 
            
            if (!context.Response.IsClientConnected) return;

            
            // Deliver content type, encoding and length as it is received from the external URL
            context.Response.ContentType = client.ResponseHeaders["Content-Type"];
            string contentEncoding = client.ResponseHeaders["Content-Encoding"];
            string contentLength = client.ResponseHeaders["Content-Length"];

            if (!string.IsNullOrEmpty(contentEncoding))
                context.Response.AppendHeader("Content-Encoding", contentEncoding);
            if (!string.IsNullOrEmpty(contentLength))
                context.Response.AppendHeader("Content-Length", contentLength);

            if (cacheDuration > 0)
                HttpHelper.CacheResponse(context, cacheDuration);
            
            // Transmit the exact bytes downloaded
            context.Response.BinaryWrite(data);
        }
    }
 
    public bool IsReusable {
        get {
            return false;
        }
    }

}

There are two enhancements in this proxy:

It allows server side caching of content. Same URL requested by a different browser within a time period will not be downloaded on server again, instead it will be served from cache.
It generates proper response cache header so that the content can be cached on browser.
It does not decompress the downloaded content in memory. It keeps the original byte stream intact. This saves memory allocation.
It transmits the data in non-buffered fashion, which means ASP.NET Response object does not buffer the response and thus saves memory

However, this is a blocking proxy. We need to make a streaming asynchronous proxy for better performance. Here's why:

Figure: Continuous streaming proxy

As you see, when data is transmitted from server to browser while server downloads the content, the delay for server side download is eliminated. So, if server takes 300ms to download something from external source, and then 700ms to send it back to browser, you can save up to 300ms Network Latency between server and browser. The situation gets even better when the external server that serves the content is slow and takes quite some time to deliver the content. The slower external site is, the more saving you get in this continuous streaming approach. This is significantly faster than blocking approach when the external server is in Asia or Australia and your server is in USA.

The approach for continuous proxy is:

Read bytes from external server in chunks of 8KB from a separate thread (Reader thread) so that it's not blocked
Store the chunks in an in-memory Queue
Write the chunks to ASP.NET Response from that same queue
If the queue is finished, wait until more bytes are downloaded by the reader thread

The Pipe Stream needs to be thread safe and it needs to support blocking Read. By blocking read it means, if a thread tries to read a chunk from it and the stream is empty, it will suspend that thread until another thread writes something on the stream. Once a write happens, it will resume the reader thread and allow it to read. I have taken the code of PipeStream from CodeProject article by James Kolpack and extended it to make sure it's high performance, supports chunks of bytes to be stored instead of single bytes, support timeout on waits and so on.

A did some comparison between Regular proxy (blocking, synchronous, download all then deliver) and Streaming Proxy (continuous transmission from external server to browser). Both proxy downloads the MSDN feed and delivers it to the browser. The time taken here shows the total duration of browser making the request to the proxy and then getting the entire response.

Figure: Time taken by Streaming Proxy vs Regular Proxy while downloading MSDN feed

Not a very scientific graph and response time varies on the link speed between the browser and the proxy server and then from proxy server to the external server. But it shows that most of the time, Streaming Proxy outperformed Regular proxy.

Figure: Test client to compare between Regular Proxy and Streaming Proxy

You can also test both proxy's response time by going to http://labs.dropthings.com/AjaxStreamingProxy. Put your URL and hit Regular/Stream button and see the "Statistics" text box for the total duration. You can turn on "Cache response" and hit a URL from one browser. Then go to another browser and hit the URL to see the response coming from server cache directly. Also if you hit the URL again on the same browser, you will see response comes instantly without ever making call to the server. That's browser cache at work.

Learn more about Http Response caching from my blog post:
Making best use of cache for high performance website

A Visual Studio Web Test run inside a Load Test shows a better picture:

Figure: Regular Proxy load test result shows Average Requests/Sec 0.79 and Avg Response Time 2.5 sec

Figure: Streaming Proxy load test result shows Avg Req/Sec is 1.08 and Avg Response Time 1.8 sec.

From the above load test results, Streaming Proxy is 26% better Request/Sec and Average Response Time is 29% better. The numbers may sound small, but at Pageflakes, 29% better response time means 1.29 million seconds saved per month for all the users on the website. So, we are effectively saving 353 man hours per month which was wasted staring at browser screen while it downloads content.

Building the Streaming Proxy

The details how the Streaming Proxy is built is quite long and not suitable for a blog post. So, I have written a CodeProject article:

Fast, Scalable, Streaming AJAX Proxy - continuously deliver data from cross domain

Please read the article and please vote for me if your find it useful.

Request format is unrecognized for URL unexpectedly ending in /SomeWebServiceMethod

omar — Wed, 19 Sep 2007 05:08:19 GMT

At Pageflakes, our webservers; event logs are being flooded with this error. In ASP.NET AJAX 1.0 version, Microsoft added a check for all web service calls to have Content-Type: application/json in the request headers. Unless this request header is present, ASMX handler fires an exception. This exception is raised directly from the ScriptHandler which handles all web service calls made via ASP.NET AJAX.

This is done for security reason. This prevents anyone from feeding off from your webservices. For example, you might have a webservice that returns some useful information that others might be interested to consume. So, any one could just add a <script> tag pointing to that web service URL and get the json. If that webservice is a very expensive webservice in terms of I/O and/or CPU, then other websites feeding off from your webservice could easily bog down your server.

Now, this back fires when you have HTTP GET supported webservice calls that produce response headers to cache the response. For example, you might have a webmethod that returns Stock Quotes. You have used response caching so that browser caches the response of that webmethod and repeated visit do not produce repeated calls to that I/O costly webservice. What will happen is proxy gateways or proxy servers will see that their client users are making this request frequently and it can be cached. So, they will make periodic calls to that webservice and try to precache them on behalf of their client users. However, during precache, they won't send the Content-Type: application/json header. That's what I have seen for several proxy servers. As a result, it produces exception and you get your event log flooded with this exception.

The reason why MS might have not seen this or anyone else is because there's no way to make HTTP GET response cacheable on browser from webservice calls unless you do the hack I mentioned in earlier blog post.

However, one thing that puzzles me is we get this error so frequently and the request headers look so legitimate that I am not 100% sure whether it's the proxies not producing the content-type header properly all the time. Somehow the error patterns look like an registered user is trying to use the site and failing. It is possible that some firewall might see this content-type as invalid content type and filter out the header.

ASP.NET Ajax Extender for multi-column widget drag & drop

omar — Thu, 22 Mar 2007 20:03:00 GMT

My open source Ajax Start Page www.dropthings.com has an ASP.NET Ajax Extender which provides multi-column drag & drop for widgets. It allows reordering of widgets on the same column and also drag & drop between column. It also supports client side notification so that you can call web service and store the position of the widgets behind the scene without (async) postback.

I first thought of going for a plain vanilla Javascript based solution for drag & drop. It requires less code, less architectural complexity and provides better speed. Another reason was the high learning curve for making Extenders in proper way in Asp.net Ajax given that there’s hardly any documentation available on the web (during the time of writing this blog). However, writing a proper extender which pushes Asp.net Ajax to the limit is a very good way to learn under-the-hood secrets of Asp.net Ajax framework itself. So, the two extenders I will introduce here will tell you almost everything you need to know about Asp.net Ajax Extenders.

Ajax Control Toolkit comes with a DragPanel extender which I could use to provide drag & drop support to panels. It also has a ReorderList control which I could use to provide reordering of items in a single list. Widgets are basically panels that flow vertically in each column. So, it might be possible that I could create reorder list in each column and use the DragPanel to drag the widgets. But I could not use ReorderList because:

ReorderList strictly uses Html Table to render its items in a column. I have no table inside the columns. Only one Panel is there inside an UpdatePanel.
ReorderList takes a Drag Handle template and creates a drag handle for each item at runtime. I already have drag handle created inside a Widget which is the widget header. So, I cannot allow ReorderList to create another drag handle.
I need client side callback on drag & drop so that I can make Ajax calls and persist the widget positions. The callback must give me the Panel where the widget is dropped, which is dropped and at what position.

Next challenge is with the DragPanel extender. The default implement of Drag & Drop in Ajax Control Toolkit has some problems:

When you start dragging, the item becomes absolutely positioned, but when you drop it, it does not become static positioned. There's a small hack needed for restoring the original position to "static".
It does not put the dragging item on top of all items. As a result, when you start dragging, you see the item being dragged below other items which makes the drag get stuck especially when there's an IFRAME.

So, I have made a CustomDragDropExtender and a CustomFloatingExtender. CustomDragDropExtender is for the column containers where widgets are placed. It provides the reordering support. You can attach this extender to any Panel control.

Here's how you can attach this extender to any Panel and make that Panel support drag & drop of Widgets:

   1: 
<
asp:Panel 
ID
="LeftPanel" 
runat
="server"  
class
="widget_holder" 
columnNo
="0"
>

   2:         
<
div 
id
="DropCue1" 
class
="widget_dropcue"
>

   3:         
</
div
>

   4: 
</
asp:Panel
>

5:

   6: 
<
cdd:CustomDragDropExtender 
ID
="CustomDragDropExtender1"

   7:     
runat
="server"

   8:     
TargetControlID
="LeftPanel"

   9:     
DragItemClass
="widget"

  10:     
DragItemHandleClass
="widget_header"

  11:     
DropCueID
="DropCue1"

  12:     
OnClientDrop
="onDrop" 
/>

<cdd:CustomDragDropExtender> offers the following properties:

TargetControlID – ID of the Panel which becomes the Drop zone
DragItemClass – All child elements inside the Panel having this class will become draggable. E.g. Widget DIV has this class so that it can become draggable.
DragItemHandleClass – Any child element having this class inside the draggable elements will become the drag handle for the draggable element. E.g. Widget Header area has this class, so it acts as the drag handle for the Widget.
DropCueID – ID of an element inside the Panel which acts as Drop Cue.
OnClientDrop – Name of a Javascript function which is called when widget is dropped on the Panel.

LeftPanel becomes a widget container which allows widgets to be dropped on it and reordered. The DragItemClass attribute on the extender defines the items which can be ordered. This prevents from non-widget Html Divs from getting ordered. Only the DIVs with the class "widget" are ordered. Say there are 5 DIVs with the class named "widget". It will allow reordering of only these five divs:

   1: 
<
div 
id
="LeftPanel" 
class
="widget_holder" 
>

   2:         
<
div 
class
="widget"
> ... 
</
div
>

   3:         
<
div 
class
="widget"
> ... 
</
div
>

4:

   5:         
<
div 
class
="widget"
> ... 
</
div
>

   6:         
<
div 
class
="widget"
> ... 
</
div
>

   7:         
<
div 
class
="widget"
> ... 
</
div
>

8:

   9:         
<
div
>This DIV will not move
</
div
>

  10:         
<
div 
id
="DropCue1" 
class
="widget_dropcue"
></
div
>

  11: 
</
div
>

When a widget is dropped on the panel, the extender fires the function specified in OnClientDrop. It offers standard Ajax Events. But unlike basic Ajax events where you have to programmatically bind to events, you can define property and specify the function name to call. So, instead of doing this:

   1: 
function pageLoad( sender, e ) {

2:

   3:   
var extender1 =
$get(‘CustomDragDropExtender1’);

   4:   extender1.add_onDrop( onDrop );

5:

   6: }

You can do this:

   1: 
<
cdd:CustomDragDropExtender 
ID
="CustomDragDropExtender1"

   2:     
runat
="server"

   3:     
OnClientDrop
="onDrop" 
/>

When the event is raised, the function named onDrop gets fired. This is done with the help of some handy library available in ACT project.

When the event is fired, it sends the container, the widget and the position of the widget where the widget is dropped.

   1: 
function onDrop( sender, e )

   2: {

   3:     
var container = e.get_container();

   4:     
var item = e.get_droppedItem();

   5:     
var position = e.get_position();

6:

   7:     
//alert( String.format( "Container: {0}, Item:
{1}, Position: {2}", container.id, item.id, position ) );

8:

   9:     
var instanceId =
parseInt(item.getAttribute(
"InstanceId"));

  10:     
var columnNo =
parseInt(container.getAttribute(
"columnNo"));

  11:     
var row = position;

12:

  13:    
WidgetService.MoveWidgetInstance( instanceId, columnNo, row );

  14: }

The widget location is updated by calling the WidgetService.MoveWidgetInstance.

CustomDragDropExtender has 3 files:

CustomDragDropExtender.cs – The server side extender implementation
CustomDragDropDesigner.cs – Designer class for the extender
CustomDragDropExtender.js – Client side scriptfor the extender

Server side class CustomDragDropExtender.cs has the following code:

   1: [assembly:
System.Web.UI.WebResource(
"CustomDragDrop.CustomDragDropBehavior.js",

"text/javascript")]

2:

   3: 
namespace CustomDragDrop

   4: {

   5:     [Designer(
typeof(CustomDragDropDesigner))]

   6:     [ClientScriptResource(
"CustomDragDrop.CustomDragDropBehavior", 
"CustomDragDrop.CustomDragDropBehavior.js")]

   7:     [TargetControlType(
typeof(WebControl))]

   8:     [RequiredScript(
typeof(CustomFloatingBehaviorScript))]

   9:     [RequiredScript(
typeof(DragDropScripts))]

  10:     
public 
class CustomDragDropExtender :
ExtenderControlBase

  11:     {

  12:         
// TODO: Add your property accessors here.

  13:         
//

  14:         [ExtenderControlProperty]

  15:         
public 
string DragItemClass

  16:         {

  17:             get

  18:             {

  19:                 
return GetPropertyValue<String>(
"DragItemClass", 
string.Empty);

  20:             }

  21:             set

  22:             {

  23:                
SetPropertyValue<String>(
"DragItemClass", 
value);

  24:             }

  25:         }

26:

  27:         [ExtenderControlProperty]

  28:         
public 
string DragItemHandleClass

  29:         {

  30:             get

  31:             {

  32:                 
return GetPropertyValue<String>(
"DragItemHandleClass", 
string.Empty);

  33:             }

  34:             set

  35:             {

  36:                
SetPropertyValue<String>(
"DragItemHandleClass", 
value);

  37:             }

  38:         }

39:

  40:         [ExtenderControlProperty]

  41:         [IDReferenceProperty(
typeof(WebControl))]

  42:         
public 
string DropCueID

  43:         {

  44:             get

  45:             {

  46:                 
return GetPropertyValue<String>(
"DropCueID", 
string.Empty);

  47:             }

  48:             set

  49:             {

  50:                
SetPropertyValue<String>(
"DropCueID", 
value);

  51:             }

  52:         }

53:

  54:         [ExtenderControlProperty()]

  55:         [DefaultValue(
"")]

  56:         [ClientPropertyName(
"onDrop")]

  57:         
public 
string OnClientDrop

  58:         {

  59:             get

  60:             {

  61:                 
return GetPropertyValue<String>(
"OnClientDrop", 
string.Empty);

  62:             }

  63:             set

  64:             {

  65:                
SetPropertyValue<String>(
"OnClientDrop", 
value);

  66:             }

  67:         }

68:

  69:     }

  70: }

Most of the code in the extender defines the property. The important part is the declaration of the class:

   1: [assembly:
System.Web.UI.WebResource(
"CustomDragDrop.CustomDragDropBehavior.js",

"text/javascript")]

2:

   3: 
namespace CustomDragDrop

   4: {

   5:     [Designer(
typeof(CustomDragDropDesigner))]

   6:     [ClientScriptResource(
"CustomDragDrop.CustomDragDropBehavior", 
"CustomDragDrop.CustomDragDropBehavior.js")]

   7:     [TargetControlType(
typeof(WebControl))]

   8:     [RequiredScript(
typeof(CustomFloatingBehaviorScript))]

   9:     [RequiredScript(
typeof(DragDropScripts))]

  10:     
public 
class CustomDragDropExtender :
ExtenderControlBase

  11:     {

The extender class inherits from ExtenderControlBase defined in ACT project. This base class has additional features over Ajax runtime provided Extender base class. It allows you to define RequiredScript attribute, which makes sure all the required scripts are downloaded before the extender script is downloaded and initialized. This extender has dependency over another extender named CustomFloatingBehavior. It also depends on ACT’s DragDropManager. So, the RequiredScript attribute makes sure those are downloaded before this extender’s script is downloaded. The ExtenderControlBase is a pretty big class and does a lot of work for us. It contains default implementations for discovering all the script files for the extender and rendering them properly.

The [assembly:System.Web.UI.WebResource] attribute defines the script file containing the script for extender. The script file is an embedded resource file.

[ClientScriptResource] attribute defines the scripts required for the extender. This class is also defined in ACT. ExtenderControlBase uses this attribute to find out which .js files are working for the extender and renders them properly.

The challenge is to make the client side javascript for the extender. On the js file, there’s a Javascript pseudo class:

   1: Type.registerNamespace(
'CustomDragDrop');

2:

   3:
CustomDragDrop.CustomDragDropBehavior = 
function(element) {

4:

   5:    
CustomDragDrop.CustomDragDropBehavior.initializeBase(
this, [element]);

6:

   7:     
this._DragItemClassValue = 
null;

   8:     
this._DragItemHandleClassValue = 
null;

   9:     
this._DropCueIDValue = 
null;

  10:     
this._dropCue = 
null;

  11:     
this._floatingBehaviors = [];

  12: }

During initialize, it hooks on the Panel it is attached to and the drop cue to show while drag & drop is going on over the Panel:

   1:
CustomDragDrop.CustomDragDropBehavior.prototype = {

2:

   3:     initialize : 
function() {

   4:         
// Register ourselves as a drop target.

   5:        
AjaxControlToolkit.DragDropManager.registerDropTarget(
this);

   6:         
//Sys.Preview.UI.DragDropManager.registerDropTarget(this);

7:

   8:         
// Initialize drag behavior after a while

   9:         window.setTimeout(
Function.createDelegate( 
this, 
this._initializeDraggableItems ), 3000 );

10:

  11:         
this._dropCue = $get(
this.get_DropCueID());

  12:     },

After initializing the DragDropManager and marking the Panel as a drop target, it starts a timer to discover the dragable items inside the panel and create Floating behavior for them. Floating behavior is the one which makes a DIV draggable.

FloatingBehavior makes one DIV freely draggable on the page. But it does not offer drop functionality. DragDropBehavior offers the drop functionality which allows a freely moving DIV to rest on a fixed position.

Discovering and initializing floating behavior for the dragable items is the challenging work:

   1: 
// Find all items with the drag item class and
make each item

   2: 
// draggable

   3: _initializeDraggableItems : 
function()

   4: {

   5:     
this._clearFloatingBehaviors();

6:

   7:     
var el = 
this.get_element();

8:

   9:     
var child = el.firstChild;

  10:     
while( child != 
null )

  11:     {

  12:         
if( child.className == 
this._DragItemClassValue && child
!= 
this._dropCue)

  13:         {

  14:             
var handle = 
this._findChildByClass(child, 
this._DragItemHandleClassValue);

  15:             
if( handle )

  16:             {

  17:                 
var handleId = handle.id;

  18:                 
var behaviorId = child.id + 
"_WidgetFloatingBehavior";

19:

  20:                 
// make the item draggable by adding floating
behaviour to it

  21:                 
var floatingBehavior =
$create(CustomDragDrop.CustomFloatingBehavior,

  22:                         {
"DragHandleID":handleId, 
"id":behaviorId, 
"name": behaviorId}, {}, {}, child);

23:

  24:                 Array.add( 
this._floatingBehaviors, floatingBehavior
);

  25:             }

  26:         }

  27:         child = child.nextSibling;

  28:     }

  29: },

Here’s the algorithm:

Run through all immediate child elements of the control where the extender is attached to
If the child item has the class for draggable item, then:
- Find any element under the child item which has the class for Drag handle
- If such item found, then attach a CustomFloatingBehavior with the child item

The _findChildByClass function recursively iterates through all the child elements and looks for an element which has the defined class. It’s an expensive function. So, it is important that the drag handle is very close to the dragable element.

   1: _findChildByClass : 
function(item, className)

   2: {

   3:     
// First check all immediate child items

   4:     
var child = item.firstChild;

   5:     
while( child != 
null )

   6:     {

   7:         
if( child.className == className ) 
return child;

   8:         child = child.nextSibling;

   9:     }

10:

  11:     
// Not found, recursively check all child
items

  12:     child = item.firstChild;

  13:     
while( child != 
null )

  14:     {

  15:         
var found = 
this._findChildByClass( child, className
);

  16:         
if( found != 
null ) 
return found;

  17:         child = child.nextSibling;

  18:     }

  19: },

When user drags an item over the Panel where the extender is attached to, DragDropManager fires the following events:

   1: onDragEnterTarget : 
function(dragMode, type, data) {

   2:     
this._showDropCue(data);

   3: },

4:

   5: onDragLeaveTarget : 
function(dragMode, type, data) {

   6:     
this._hideDropCue(data);

   7: },

8:

   9: onDragInTarget : 
function(dragMode, type, data) {

  10:     
this._repositionDropCue(data);

  11: },

Here we deal with the drop cue. The challenging work is to find out the right position for the drop cue.

We need to find out where we should show the drop cue based on where user is dragging the item. The idea is to find out the widget which is immediately below the dragged item. The item is pushed down by one position and the drop cue takes its place. While dragging, the position of the drag item can be found easily. Based on that, I locate the widget below the drag item:

   1: _findItemAt : 
function(x,y, item)

   2:     {

   3:         
var el = 
this.get_element();

4:

   5:         
var child = el.firstChild;

   6:         
while( child != 
null )

   7:         {

   8:             
if( child.className == 
this._DragItemClassValue && child
!= 
this._dropCue && child != item )

   9:             {

  10:                 
var pos =
Sys.UI.DomElement.getLocation(child);

11:

  12:                 
if( y <= pos.y )

  13:                 {

  14:                     
return child;

  15:                 }

  16:             }

  17:             child =
child.nextSibling;

  18:         }

19:

  20:         
return 
null;

  21:     },

This function returns the widget which is immediately under the dragged item. Now I add the drop cue immediately above the widget:

   1: _repositionDropCue : 
function(data)

   2:     {

   3:         
var location =
Sys.UI.DomElement.getLocation(data.item);

   4:         
var nearestChild = 
this._findItemAt(location.x, location.y,
data.item);

5:

   6:         
var el = 
this.get_element();

7:

   8:         
if( 
null == nearestChild )

   9:         {

  10:             
if( el.lastChild != 
this._dropCue )

  11:             {

  12:                 el.removeChild(
this._dropCue);

  13:                 el.appendChild(
this._dropCue);

  14:             }

  15:         }

  16:         
else

  17:         {

  18:             
if( nearestChild.previousSibling != 
this._dropCue )

  19:             {

  20:                 el.removeChild(
this._dropCue);

  21:                 el.insertBefore(
this._dropCue, nearestChild);

  22:             }

  23:         }

  24:     },

One exception to consider here is that there can be no widget immediately below the dragged item. It happens when user is trying to drop the widget at the bottom of column. In that case, the drop cue is shown at the bottom of the column.

When user releases the widget, it drops right on top of drop cue and the drop cue disappears. After the drop the onDrop event is raised to notify where the widget is dropped.

   1: _placeItem : 
function(data)

   2:     {

   3:         
var el = 
this.get_element();

4:

   5:        
data.item.parentNode.removeChild( data.item );

   6:         el.insertBefore( data.item,

this._dropCue );

7:

   8:         
// Find the position of the dropped item

   9:         
var position = 0;

  10:         
var item = el.firstChild;

  11:         
while( item != data.item )

  12:         {

  13:             
if( item.className == 
this._DragItemClassValue ) position++;

  14:             item =
item.nextSibling;

  15:         }

  16:         
this._raiseDropEvent( 
/* Container */ el, 
/* droped item */ data.item, 
/* position */ position );

  17:     }

Generally you can make events in extenders by adding two functions in the extender:

   1: add_onDrop : 
function(handler) {

   2:     
this.get_events().addHandler(
"onDrop", handler);

   3: },

4:

   5: remove_onDrop : 
function(handler) {

   6:     
this.get_events().removeHandler(
"onDrop", handler);

   7: },

But this does not give you the support for defining the event listener name in the ASP.NET declaration:

   1: 
<
cdd:CustomDragDropExtender 
ID
="CustomDragDropExtender1"

   2:     
runat
="server"

   3:     
OnClientDrop
="onDrop" 
/>

The declaration only allows properties. In order to support such declarative assignment of events, we need to first introduce a property named OnClientDrop in the extender. Then during assignment of the property, we need to find out the function specified there and attach event notification on that function. The discovery of the function from its name is done by CommonToolkitScripts.resolveFunction which is available in ACT project.

   1: 
// onDrop property maps to onDrop event

   2:     get_onDrop : 
function() {

   3:         
return 
this.get_events().getHandler(
"onDrop");

   4:     },

5:

   6:     set_onDrop : 
function(value) {

   7:         
if (value && (0 <
value.length)) {

   8:             
var func =
CommonToolkitScripts.resolveFunction(value);

   9:             
if (func) {

  10:                 
this.add_onDrop(func);

  11:             } 
else {

  12:                 
throw Error.argumentType(
'value', 
typeof(value), 
'Function', 
'resize handler not a function, function name, or
function text.');

  13:             }

  14:         }

  15:     },

Raising the event is same as basic Ajax events:

   1: _raiseEvent : 
function( eventName, eventArgs ) {

   2:         
var handler = 
this.get_events().getHandler(eventName);

   3:         
if( handler ) {

   4:             
if( !eventArgs ) eventArgs =
Sys.EventArgs.Empty;

   5:             handler(
this, eventArgs);

   6:         }

   7:     },

This is all about the CustomDragDropExtender. Next challenge is to make the CustomFloatingBehavior. The server side class is declared as:

   1: [assembly:
System.Web.UI.WebResource(
"CustomDragDrop.CustomFloatingBehavior.js",

"text/javascript")]

2:

   3: 
namespace CustomDragDrop

   4: {

   5:     [Designer(
typeof(CustomFloatingBehaviorDesigner))]

   6:     [ClientScriptResource(
"CustomDragDrop.CustomFloatingBehavior", 
"CustomDragDrop.CustomFloatingBehavior.js")]

   7:     [TargetControlType(
typeof(WebControl))]

   8:     [RequiredScript(
typeof(DragDropScripts))]

   9:     
public 
class CustomFloatingBehaviorExtender :
ExtenderControlBase

  10:     {

  11:         [ExtenderControlProperty]

  12:         [IDReferenceProperty(
typeof(WebControl))]

  13:         
public 
string DragHandleID

  14:         {

  15:             get

  16:             {

  17:                 
return GetPropertyValue<String>(
"DragHandleID", 
string.Empty);

  18:             }

  19:             set

  20:             {

  21:                
SetPropertyValue<String>(
"DragHandleID", 
value);

  22:             }

  23:         }

  24:     }

  25: }

There’s only one property – DragHandleID. Widget’s header works as the drag handle. So, the header ID is specified here.

This extender has dependency on DragDropManager so the [RequiredScript(typeof(DragDropScripts))] attribute is there.

Besides the designer class, there’s one more class which CustomDragDropExtender need in order to specify its dependency over this floating behavior:

   1: [ClientScriptResource(
null, 
"CustomDragDrop.CustomFloatingBehavior.js")]

   2:   
public 
static 
class CustomFloatingBehaviorScript

   3:   {

   4:   }

This class can be used inside RequiredScript attribute. It only defines which script file contains the client side code for the extender.

The client side Javascript is same as FloatingBehavior that comes with ACT. The only difference is some hack when drag starts. DragDropManager does not return the item being dragged to static position once it makes it absolute. It also does not increase the zIndex of the item. If the drag item does not become the top most item, while dragging it goes below other elements on the page. So, I have made some changes in the mouseDownHandler of the behavior to add these features:

   1: 
function mouseDownHandler(ev) {

   2:     window._event = ev;

   3:     
var el = 
this.get_element();

4:

   5:     
if (!
this.checkCanDrag(ev.target)) 
return;

6:

   7:     
// Get the location before making the element
absolute

   8:     _location =
Sys.UI.DomElement.getLocation(el);

9:

  10:     
// Make the element absolute

  11:     el.style.width = el.offsetWidth
+ 
"px";

  12:     el.style.height =
el.offsetHeight + 
"px";

  13:    
Sys.UI.DomElement.setLocation(el, _location.x, _location.y);

14:

  15:     _dragStartLocation =
Sys.UI.DomElement.getLocation(el);

16:

  17:     ev.preventDefault();

18:

  19:     
this.startDragDrop(el);

20:

  21:     
// Hack for restoring position to static

  22:     el.originalPosition = 
"static";

  23:     el.originalZIndex =
el.style.zIndex;

  24:     el.style.zIndex = 
"60000";

  25: }

Setting el.originalPosition = static fixes the bug in DragDropManager. It incorrectly stores absolute has the originalPosition when startDragDrop is called. So, after calling this function, I set it to correct originalPosition which is “static”.

When drag completes, the original zIndex is restored and left, top, width and height is cleared. DragDropManager makes the item position static, but it does not clear the left, top, width and height attributes. This moves the element away from the place where it is dropped. This bug is fixed in the onDragEnd event:

   1: 
this.onDragEnd = 
function(canceled) {

   2:     
if (!canceled) {

   3:         
var handler = 
this.get_events().getHandler(
'move');

   4:         
if(handler) {

   5:             
var cancelArgs = 
new Sys.CancelEventArgs();

   6:             handler(
this, cancelArgs);

   7:             canceled =
cancelArgs.get_cancel();

   8:         }

   9:     }

10:

  11:     
var el = 
this.get_element();

  12:     el.style.width =
el.style.height = el.style.left = el.style.top = 
"";

  13:     el.style.zIndex =
el.originalZIndex;

  14: }

That's all folks!

You can get the code for the extenders here.

ASP.NET Ajax in-depth performance analysis

omar — Fri, 16 Mar 2007 09:24:00 GMT

Let's do some ASP.NET runtime analysis on www.dropthings.com. Those who don't know what it is, it's an open source start page I made using ASP.NET Ajax, .NET 3.0 and Linq.

ASP.NET Ajax has a pretty big runtime which consists of Core Framework, scripts for UpdatePanel, Preview Script required for Drag & Drop. Additionally I need Ajax Control Toolkit (ACT). All these add up to a staggering 564 KB of download in 12 script references on the page. The download size mostly depends on the usage of extenders and Ajax features. A moderate use of extender results in the following download:

Here’s a simulation of 256kbps internet speed on 200ms network latency. I use a tool named Charles ( www.xk72.com/charles) to capture traffic and simulate slow internet speed by throttling data transfer speed. From the durations, we see almost 20 sec is spent on downloading the runtime over a 256kbps line!

Let’s explain which script in the above list does what. I will be identifying the scripts using their file size in order of their appearance:

21.64 KB – Handy script for Postbacks
83.38 KB – Microsoft Ajax Core runtime
30.16 KB - UpdatePanel, partial update, asynchronous postback scripts.
136.38 KB – Preview version of Ajax which allows Drag & Drop script
36.02 KB – The actual drag & drop script in Preview library
45.25 KB – Ajax Control Toolkit
4.08 KB – Timer script
140.86 KB – ACT Animation framework
18.05 KB – ACT Behavior base implementation. Required for Behaviors provided in the Ajax Control Toolkit project.
16.48 KB – ACT Animation Behavior
7.32 KB – My Custom Drag Drop behavior
9.73 KB – My Custom Floating Behavior

The total payload for only the runtime is too high to accept because we cannot make a user wait for 20 sec just to download Ajax scripts before user can actually start interacting on the page. So, I took several approaches to reduce the size of the download:

Eliminate Preview version of Ajax completely and use ACT for Drag & Drop
Use IIS 6 compression to deliver compressed scripts from the client
Combine multiple script files into one file

ACT comes with its own DragDropManager which we need for Drag & Drop. So far I thought of using Sys.Preview.UI.DragDropManager in the ASP.NET Ajax January CTP. But just for the DragDropManager, I need to add nearly 180 KB of scripts for the entire Preview Library runtime. If I use ACT’s DrgaDropManager, I can get rid of the Preview runtime.

Without the preview scripts, the scripts downloaded are:

The following downloads are missing here:

136.38 KB – Preview version of Ajax which allows Drag & Drop script
36.02 KB – The actual drag & drop script in Preview library

I saved about 180 KB, nearly 7 sec from total download. So, user gets to work on the page 7 secs earlier than before.

When I enabled IIS 6 compression, the situation improved dramatically. Here’s the data transfers when compression is enabled:

The total download comes down from 448 KB to 163 KB! Total 64% download bytes reduction! This results in 3 times faster page download.

The scripts are downloaded in two steps – first the core runtimes download and then ACT and other scripts download. The content is displayed after the core runtime is downloaded. So, the time it takes to show the content on the browser reduces significantly because now it takes only 50KB to download before something is visible on the screen compared to 130 KB on non-compressed mode.

ScriptManager control has LoadScriptsBeforeUI property which you can set to “False” in order to postpone several script downloads after the content is downloaded. This adds the script references end of the <body> tag. As a result, you see the content first and then the additional scripts, exteders, ACT scripts get downloaded and initialized.

< asp:ScriptManager ID ="ScriptManager1" runat ="server" EnablePartialRendering ="true" LoadScriptsBeforeUI ="false" ScriptMode ="Release" > ... </ asp:ScriptManager >

You can explicitly set ScriptMode=false in order to emit release mode highly optimized Ajax runtime scripts during local debugging.

Article of the month - vote for me

omar — Tue, 13 Feb 2007 18:29:45 GMT

My article "Build Google IG like Ajax Start Page in 7 days using ASP.NET Ajax and .NET 3.0" is now on the nomination list of Codeproject. If you found this article helpful, please give me a vote here:

http://www.codeproject.com/script/survey/survey.asp?survey=647

If you have not read it yet, I would request you to take a look at it. You can learn about ASP.NET Ajax, ASP.NET 2.0 tricks, Workflow Foundation, real use of WF from ASP.NET and finally how to make real use of Dlinq, XLinq and Linq.

Build Google IG like AJAX Start Page in 7 days using ASP.NET AJAX and .NET 3.0

omar — Thu, 04 Jan 2007 08:38:00 GMT

I will show you how I built a start page similar to Google IG in 7 nights using ASP.Net Ajax, .NET 3.0, Linq, DLinq and XLinq. I have logged my day to day development experience in this article and documented all the technical challenges, interesting discoveries and important design & architectural decisions. You will find the implementation quite close to actual Google IG. It has drag & drop enabled widgets, complete personalization of the pages, multi page feature and so on. It's not just a prototype or a sample project. It's a real living and breathing open source start page running at http://www.dropthings.com/ which you can use everyday. You are welcome to participate in the development and make widgets for the project.

You can find the article on CodeProject:

Build Google IG like AJAX Start Page in 7 days using ASP.NET AJAX and .NET 3.0

If you like it, please vote for me.

ASP.net Ajax under the hood secrets

omar — Fri, 01 Dec 2006 16:48:00 GMT

I have published a new article in codeproject:

ASP.net Ajax under the hood secrets

You will find the following tricks on it:

Batch call advantages and disadvantages
Implement auto retry in all web method call by modifying Ajax core framework
Implement queue based web method call in order to prevent browser from getting stuck with too many Ajax calls
Caching web method response, fix bug in ASP.net 2.0 cache header problem
How slow is Http Post, capture data transfer over the wire and see

Some of the techniques may be known to you if you have read my earlier posts. But I have converted all code to the latest ASP.net Ajax Beta 2 version which is not available in the blog posts.

Please rate the article if you like it at codeproject.

Also please digg this article.

Atlas 7: Caching web service response on browser and save bandwidth significantly

omar — Sat, 23 Sep 2006 17:04:00 GMT

Browser can cache images, javascripts, css files on users hard drive and it can also cache Xml Http calls if the call is a Http Get. The cache is based on Url. If it's the same Url and it's cached on the computer then the response is loaded from cache, not from the server when it is requested again. Basically, browser can cache any Http Get call and return cached data based on Url. If you make a Xml Http call as Http GET and server returns some special header which informs the browser to cache the response; on future calls, the response will be immediately returned from the cache and thus save the delay of network roundtrip and download time.

At Pageflakes, we cache user's state so that when user visits again the following day, user gets a cached page which loads instantly from browser cache, not from the server. Thus second time load becomes very fast. We also cache several small parts of the page which appears on users action. When user does the same action again, a cached result is loaded immediately from local cache and thus saves the network roundtrip time. User gets a fast loading site and a very responsive site. The perceived speed increases dramatically.

The idea is to make Http Get calls while making Atlas web service calls and return some specific Http Response headers which tells the browser to cache the response for some specific duration. If you return "Expires" header during the response, browser will cache the Xml Http response. There are 2 headers that you need to return with response which will instruct browser to cache the response:

HTTP/ 1.1 200 OK Expires: Fri , 1 Jan 2030 Cache-Control: public

This will instruct browser to cache the response till Jan 2030. As long as you make the same Xml Http call with the same parameters, you will get cached response from the computer and no call will go to the server. There are more advanced ways to get further control over response caching. For example, here is a header which will instruct browser to cache for 60 seconds but do contact server and get a fresh response after 60 seconds. It will also prevent proxies from returning cached response when browser local cache expires after 60 seconds.

HTTP/ 1.1 200 OK Cache-Control: private , must-revalidate , proxy-revalidate , max-age = 60

Let's try to produce such response headers from ASP.NET web service call:

This will result in the following response headers:

Expires header is set properly. But the problem is with Cache-Control. It is showing "max-age" is set to zero which will prevent browser from doing any kind of caching. If you seriously want to prevent caching, you should emit such cache-control header. Looks like exactly the opposite thing happened.

There's a bug in ASP.NET 2.0 that you cannot change "max-age" header. As max-age is set to zero, ASP.NET 2.0 sets Cache-Control to private because max-age = 0 means no cache needed. So, there's no way you can make ASP.NET 2.0 return proper headers which caches the response.

Time for a hack. After decompiling the code of HttpCachePolicy class (Context.Response.Cache object's class), I found the following code:

Somehow, this._maxAge is getting set to zero and the check: "if (!this._isMaxAgeSet || (delta < this._maxAge))" is preventing it from getting set to a bigger value. Due to this problem, we need to bypass the SetMaxAge function and set the value of the _maxAge field directly using Reflection.

This will return the following headers:

Now max-age is set to 60 and thus browser will cache the response for 60 seconds. If you make the same call again within 60 seconds, it will return the same response. Here's a test output which shows the date time returned from the server:

The client side code is like this:

function cachedHttpGet() { WebService.CachedGet( { useGetMethod: true , onMethodComplete: function (result) { debug.dump(result); } } ); }

Here you see, the response is cached for 60 seconds and after the time elapsed, there was a server call made and new date was returned. That response was again cached for 60 seconds.

Atlas 6: When 'this' is not really 'this'

omar — Sat, 23 Sep 2006 16:46:00 GMT

Atlas callbacks are not executed on the same context where they are called. For ex, if you are making a Page method call from a javascript class like this:

function SampleClass() { this .id = 1 ; this .call = function () { PageMethods.DoSomething( " Hi " , function (result) { debug.dump( this .id ); } ); } }

What happens when you call the "call" method? Do you get "1" on the debug console? No, you get "null" on the debug console because "this" is no longer the instance of the class. This is a common mistake everyone makes. As this is not yet documented in Atlas documentations, I have seen many developers spend time finding out what's wrong.

Here's the reason. We know whenever Javascript events are raised "this" refers to the html element which produced the event. So, if you do this:

function SampleClass() { this .id = 1 ; this .call = function () { PageMethods.DoSomething( " Hi " , function (result) { debug.dump( this .id ); } ); } }

If you click the button, you see "ButtonID" instead of "1". The reason is that, the button is making the call. So, the call is made within button object's context and thus "this" maps to the button object.

Similarly, when Xml Http raises the event onreadystatechanged which Atlas traps and fires the callback, the code execution is still on the Xml Http's context. It's Xml Http object which raises the event. As a result, "this" refer to the Xml Http object, not to your own class where the callback is declared.

In order to make the callback fire on the context of the instance of the class so that "this" refers to the instance of the class, you need to make the following change:

function SampleClass() { this .id = 1 ; this .call = function () { PageMethods.DoSomething( " Hi " , Function.createDelegate( this , function (result) { debug.dump( this .id ); } ) ); } }

Here, the Function.createDelegate is used to create a delegate which calls the given function under the "this" context. Function.createDelegate is defined in AtlasRuntime:

Function.createDelegate = function (instance, method) { return function () { return method.apply(instance, arguments); } }

Atlas 4: Only 2 calls at a time and don't expect any order

omar — Sat, 23 Sep 2006 16:37:00 GMT

Browsers make 2 concurrent AJAX calls at a time to a domain. If you make 5 AJAX calls, browser is going to make 2 calls first, then wait for any one of them to complete and then make another call until all remaining 4 calls are complete. Moreover, you cannot expect calls to execute in the same order as you make the calls. Here's why:

Here you see, call 3's response download is quite big and thus takes longer than Call 5. So, Call 5 actually gets executed before Call 3.

So, the world of HTTP is unpredictable.

Atlas 5: Bad calls make good calls timeout

omar — Sat, 23 Sep 2006 16:31:00 GMT

If 2 http calls somehow get stuck for too long, those two bad calls are going to make some good calls expire too which in the meantime got queued. Here's a nice example:

function timeoutTest()

{

PageMethods.Timeout( { timeoutInterval : 3000, onMethodTimeout: function() { debug.dump("Call 1 timed out"); } } );

PageMethods.Timeout( { timeoutInterval : 3000, onMethodTimeout: function() { debug.dump("Call 2 timed out"); } } );

PageMethods.DoSomething( 'Call 1', { timeoutInterval : 3000, onMethodTimeout: function() { debug.dump("DoSomething 1 timed out"); } } );

PageMethods.DoSomething( 'Call 2', { timeoutInterval : 3000, onMethodTimeout: function() { debug.dump("DoSomething 2 timed out"); } } );

PageMethods.DoSomething( 'Call 3', { timeoutInterval : 3000, onMethodTimeout: function() { debug.dump("DoSomething 3 timed out"); } } );

}

I am calling a method named "Timeout" on the server which does nothing but to wait for a long time so that the call gets timed out. After that I am calling a method which does not timeout. But guess what the output is:

Only one call succeeded "Do Something 1". Try again and you might see this:

Now two calls succeeded. So, if at any moment, browser's two connections get jammed, then you can expect other waiting calls are going to timeout also.

In Pageflakes, we used to get nearly 400 to 600 timeout error reports from users' browsers. We could never find out how this can happen. First we suspected slow internet connection. But that cannot happen for so many users. Then we suspected something is wrong with the hosting providers network. We did a lot of network analysis to find out whether there's any problem on the network. But we could not detect any. We used SQL Profiler to see whether there's any long running query which times out ASP.NET request execution time. But no luck. We finally discovered that, it mostly happened due to some bad calls which got stuck and made the good calls expire too. So, we modified the Atlas Runtime and introduce automatic retry on it and the problem disappeared completely. However, this auto retry requires a sophisticated open heart bypass surgery on Atlas Runtime javascript code which you have to perform again and again whenever Microsoft releases newer version of Atlas Runtime. You also can no longer use the <atlas:scriptmanager> tag which produces Atlas runtime references instead you have to manually put links to Atlas runtime and compatibility javascript files. So, you better do auto retry yourself in your own code from Day 1. On the onMethodTimeout method, just make one retry all the time to be on the safe side.

Atlas 3: Atlas batch calls are not always faster

omar — Sat, 23 Sep 2006 15:59:00 GMT

Atlas provides you batch call feature which combines multiple web service calls into one call. It works transparently, you won't notice anything nor do you need not write any special code. Once you turn on the Batch feature, all web service calls made within a duration gets batched into one call. Thus saves roundtrip time and total response time.

The actual response time might be reduced but the perceived delay is higher. If 3 web service calls are batched, the 1st call does not finish first. All 3 calls finish at the same time. If you are doing some UI updates upon completion of each WS calls, it does not happen one by one. All of the calls complete in one shot and then the UI gets updated in one shot. As a result, you do not see incremental updates on the UI, instead a long delay before the UI updates. If any of the call, say the 3rd call downloads a lot of data, user sees nothing happening until all 3 calls complete. So, the duration of the 1st call becomes nearly the duration of the sum of all 3 calls. Although actual total duration is reduced, but the perceived duration is higher. Batch calls are handy when each call is transmitting small amount of data. Thus 3 small calls gets executed in one roundtrip.

Let's work on a scenario where 3 calls are made one by one. Here's how the calls actually get executed.

The second call takes a bit time to reach the server because first call is eating up bandwidth. The same reason it takes longer to download. Browsers open 2 simultaneous connections to the server. So at a time, only 2 calls are made. Once the second/first call completes, the third call is made.

When these 3 calls are batched into one:

Here the total download time is reduced (if IIS compression enabled) and there's only one network latency overhead. All 3 calls get executed on the server in one shot and the combined response is downloaded in one call. But to the user, the perceived speed is slower because all the UI update happens after the entire batch call completes. The total duration the batch call will take to complete will always be higher than 2 calls. Moreover, if you do a lot of UI update one after another, Internet Explorer freezes for a while giving user a bad impression. Sometimes expensive update on the UI makes the browser screen go blank and white. But Firefox and Opera does not have this problem.

Batch call has some advantages too. Total download time is less than downloading individual call responses because if you use gzip compression in IIS, the total result is compressed instead of individually compressing each result. So, generally batch call is better for small calls. But if a call is going to send a large amount of data or is going to return say 20KB of response, then it's better not to use batch. Another problem with batch call is, say 2 calls are very small but the 3rd call is quite big. If these 3 call gets batched, the smaller calls are going to suffer from long delay due to the 3rd larger call.

Atlas 2: HTTP POST is slower and it's default in Atlas

omar — Fri, 22 Sep 2006 18:11:00 GMT

Atlas by default makes HTTP POST for all AJAX calls. Http POST is more expensive than Http GET. It transmits more bytes over the wire, thus taking precious network time and it also makes ASP.NET do extra processing on the server end. So, you should use Http Get as much as possible. However, Http Get does not allow you to pass objects as parameters. You can pass numeric, string and date only. When you make a Http Get call, Atlas builds an encoded url and makes a hit to that url. So, you must not pass too much content which makes the url become larger than 2048 chars. As far as I know, that's what is the max length of any url.

Another evil thing about http post is, it's actually 2 calls. First browser sends the http post headers and server replies with "HTTP 100 Continue". When browser receives this, it sends the actual body. Here's what the headers look like:

Request:

POST / Atlas1 / Default . aspx HTTP / 1.1 __serviceMethodName = Timeout&__serviceMethodParams = {}&__VIEWSTATE =/ wEPDwUJMTY3NTY1MjM2ZGSlWDXIdYj44hhTUd0z8yyp1q %2 bUtw %3 d %3 d Response: HTTP / 1.1 100 Continue Server: ASP . NET Development Server / 8.0 . 0.0 Date : Mon , 11 Sep 2006 15 : 04 : 13 GMT Content-Length: 0

After getting this clearance from the server, the actual body is sent. This is done in order to prevent long posts which are not going to succeed anyway if the server cannot accept it. But it comes at the cost of network latency.

So, Http Get should be at least twice faster than Http Post.

Here's how you make Http GET calls in Atlas:

Step 1: Decorate web method with attribute

First you need to put a special attribute on the web method:

[WebMethod] [WebOperation( true )] public string HelloWorld() { return " Hello World " ; }

Step 2: Set useGetMethod=true while making the call

Here's how you call the web method using Http Get:

WebService.HelloWorld( { useGetMethod: true , onMethodComplete: function (result) { debug.dump(result); } } );

Note: It needs to be a web service (.asmx). It will not work if you use it on page methods.

Please see this page in Atlas Quickstart for further info.

Atlas 1: Try not to use page methods

omar — Fri, 22 Sep 2006 18:08:00 GMT

One of the easiest thing in Atlas is to use the Page Method feature. If you use Atlas on your web page say Default.aspx, you can directly call public methods on Default.aspx from javascript. Just put a [WebMethod] attribute on a public method in Default.aspx and then you can them from Javascript using PageMethods.MethodName().

public partial class _Default : System.Web.UI.Page { protected void Page_Load( object sender, EventArgs e) { } [WebMethod] public string DoSomething( string param) { return param; } }

On the client side you can call this method like this:

PageMethods.DoSomething( ' Hi ' , function (result) { alert(result); } );

Here's the catch, Page method calls are always HTTP POST calls. You can never make HTTP GET call to the Page methods but you can make HTTP GET calls to Web service method. So, on later stage of your project when you will need Http response caching in order to save roundtrips, you will have to refactor all page methods to web service methods and for this you will have to move all public methods and related code from default.aspx to some web service. So, try not to use Page methods from Day 1.

Beginning Atlas series: Why Atlas?

omar — Mon, 18 Sep 2006 05:51:06 GMT

This is the first question everyone asks me when they see Pageflakes. Why not Protopage or Dojo library? Microsoft Atlas is a very promising AJAX library. They are putting a lot of effort on Atlas, making lots of reusable components that can really save you a lot of time and give your web application a complete face lift at reasonably low effort on changes. It integrated with ASP.NET v very well and it is compatible with ASP.NET Membership and Profile provider.

When we first started developing Pageflakes, Atlas was in infant stage. We were only able to use the Page Method and Webservice Method call feature of Atlas. We had to make our own drag & drop, component architecture, popups, collapse/expand features etc. But now you can have all these from Atlas and thus save a lot of development time. The web service proxy feature of Atlas is a marvel. You can point a <script> tag to a .asmx file and you get a javascript class generated right out of the web service definition. The Javascript class contains the exact methods that you have on the web service class. This makes it really easy to add/remove new webservices, add/remove methods in webservices which does not require any changes on the client side. It also offers a lot of control over the AJAX calls and provides rich exception trapping feature on the javascript. Server side exceptions are nicely thrown to client side javascript code and you can trap it and show nicely formatted error messages to the user. Atlas works really well with ASP.NET 2.0 eliminating the integration problem completely. You need not worry about authentication and authorization on page methods and web service methods. So, you save a lot of code on the client side (of course Atlas Runtime is huge for this reason) and you can concentrate more on your own code then building up all these framework related codes.

Recent version of Atlas works nicely with ASP.NET Membership and Profile services giving you login/logout features from Javascript without requiring page postbacks and you can read/write Profile object directly from Javascript. This comes very handy when you heavily use ASP.NET membership and profile providers in your web application which we do at Pageflakes.

On earlier versions of Atlas, there was no way to make HTTP GET calls. All calls were HTTP POST and thus quite expensive calls. Now you can say which calls should be HTTP GET. Once you have HTTP GET, you can utilize Http response caching features which I will explain soon.

I will be writing about lots of Atlas tips and tricks. I am assuming you are familiar with Atlas and you have already tried some quick start tutorials and you know the concepts of Page Method, Web service Proxy, Script Manager etc.

String and Array extensions in Atlas Runtime - a StringBuilder too!

omar — Wed, 02 Aug 2006 12:40:00 GMT

Atlas runtime extends the String class of Javascript with the following handy extensions:

String.prototype.endsWith=function(suffix){
String.prototype.startsWith=function(prefix){
String.prototype.lTrim=String.prototype.trimLeft=function(){
String.prototype.rTrim=String.prototype.trimRight=function(){
String.prototype.trim=function(){
String.format=function(format){

Format one is pretty powerful. You can build complex strings very efficiently:

var result = String.format("Hello {0} My name {1} {2}", param1, param2, param3);

It also has a StringBuilder just like .NET. If you are building a string of HTML anywhere which has more than 10 concatenation, use StringBuilder. Concatenations are like:

Html = "...." + "...." + "...." + "....";

Html += ".....";

If you are doing things like this in loops, you must use StringBuilder. Javascript string concatenation is very very slow.

StringBuilder is defined as:

Sys.StringBuilder=function(initialText){
this.append=function(text){
this.appendLine=function(text){
this.clear=function(){
this.toString=function(delimiter){

Here's how you use it:

var builder = new Sys.StringBuilder("Initial text, or leave blank");
builder.append("more text");
var result = builder.toString();

The Array class is pretty rich. It has the following public methods:

Array.prototype.add=function(item)
Array.prototype.addRange=function(items)
Array.prototype.clear=function()
Array.prototype.clone=function()
Array.prototype.contains=Array.prototype.exists=function(item)
Array.prototype.dequeue=function()
Array.prototype.indexOf=function(item,startIndex)
Array.prototype.insert=function(index,item)
Array.prototype.remove=function(item)
Array.prototype.removeAt=function(index)
Array.parse=function(value)

All these functions are equivalent to .NET Array and String class. So, I am not explaining in details what they do.