March 2007 - Posts

Cleanup inactive anonymous users from ASP.NET Membership Tables

ASP.NET 2.0 Websites that allow anonymous visit and anonymous user profile have a unique challenge to cleanup unused data which is generated by anonymous users who never come back. Every first visit is creating one anonymous user, page setup, and other user specific content. If the user never comes back, it still remains in the database permanently. It is possible user might come back within a day, or a week or a month. But there’s no guaranty if user will ever come back or not. Generally sticky users are max 30% of the total users who come to most websites. So, you end up with 70% unused data which are never needed. All these requires cleanup, otherwise the database keeps growing uncontrollably and gets slower and slower. This cleanup operation is humongous for busy websites. Think about deleting millions of rows from several tables, one after another while maintaining foreign key constraints. Also the cleanup operation needs to run while the site is running, without hampering site's overall performance. The whole operation results in heavily fragmented index and space in the MDF file. The log file also becomes enormous in order to keep track of the transactions. Hard drives get really hot and start sweating furiously. While the CPU keeps smiling having nothing to do with it, it’s really painful to watch SQL Server go through this every day. Unless you clean up the database and maintain its size under control; you can't keep up with SQL Server’s RAM and Disk IO requirement. 

When a user visits the site, Asp.net Membership Provider updates the LastActivityDate of aspnet_users table. From this field, I can find out how long the user has been idle. The IsAnonymous bit field tells me whether the user account is anonymous or registered. If it is registered, no need to worry. But if it is anonymous and more than 30 days old, I can be sure that the user will never come back because the cookie has already expired. If you repeatedly logout from your start page, all cookie related to the site gets cleared. That means you are producing one new anonymous user record during each log out. That anonymous record is never used because you will soon log in to have your customized pages back and then you will log out again. This will result in another anonymous user account which again becomes useless as soon as you log in.

Here’s how the whole cleanup process works:

  • Find out the users which are old enough to be discarded
  • Find out the pages user has
  • Delete all the widget instances on those pages
  • Then delete those pages
  • Remove rows from child tables related to aspnet_users like aspnet_profile, aspnet_UsersInRoles, aspnet_PersonalizationPerUser. Remove rows for users to be deleted
  • Remove the users from aspnet_users
  • Pray that you did not accidentally remove any good user

Here’s the giant DB script which does it all. I have put enough inline comment so that you can understand what the script is doing:

 

   1: -- Number of days after which we give users 'bye bye'
   2: DECLARE @Days int
   3: SET @Days = 14
   4:  
   5: -- No of users to delete per run. If it's too high, database will get stuck
   6: -- for a long time. If it's too low, you will end up having more trash than
   7: -- you can cleanup
   8: DECLARE @NoOfUsersToDelete int
   9: SET @NoOfUsersToDelete = 1000
  10:  
  11: -- Create temporary tables which holds the users and pages to delete
  12: -- As the user and the page is used to find out other tables, instead
  13: -- of running SELECT ID FORM ... repeatedly, it's better to have them
  14: -- in a temp table
  15: IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[PagesToDelete]') AND type in (N'U'))
  16: DROP TABLE [dbo].[PagesToDelete]
  17: IF  EXISTS (SELECT * FROM sys.objects WHERE object_id = OBJECT_ID(N'[dbo].[aspnetUsersToDelete]') AND type in (N'U'))
  18: DROP TABLE [dbo].[AspnetUsersToDelete]
  19:  
  20: create table PagesToDelete (PageID int NOT NULL PRIMARY KEY)
  21: create table AspnetUsersToDelete (UserID uniqueidentifier NOT NULL PRIMARY KEY)
  22:  
  23: -- Find out inactive anonymous users and store the UserID in the temporary
  24: -- table
  25: insert into AspnetUsersToDelete
  26: select top(@NoOfUsersToDelete) UserID from aspnet_Users where
  27: (isAnonymous = 1) and (LastActivityDate < (getDate()-@Days))
  28: order by UserID -- Saves SQL Server from sorting in clustered index again
  29:  
  30: print 'Users to delete: ' + convert(varchar(255),@@ROWCOUNT)
  31: GO
  32:  
  33: -- Get the pages of the users which will be deleted
  34: insert into PagesToDelete 
  35: select ID from Page where UserID in
  36: (
  37: select UserID from AspnetUsersToDelete
  38: )
  39:  
  40: print 'Pages to delete: ' + convert(varchar(255),@@ROWCOUNT)
  41: GO
  42:  
  43: -- Delete all Widget instances on the pages to be deleted
  44: delete from WidgetInstance where PageID IN
  45: ( SELECT PageID FROM PagesToDelete )
  46:  
  47: print 'Widget Instances deleted: ' + convert(varchar(255), @@ROWCOUNT)
  48: GO
  49:  
  50: -- Delete the pages
  51: delete from Page where ID IN
  52: ( SELECT PageID FROM PagesToDelete )
  53: GO
  54:  
  55: -- Delete User Setting
  56: delete from UserSetting WHERE UserID IN
  57: ( SELECT UserID FROm AspnetUsersToDelete )
  58: GO
  59:  
  60: -- Delete profile of users
  61: delete from aspnet_Profile WHERE UserID IN
  62: ( SELECT UserID FROm AspnetUsersToDelete )
  63: GO
  64:  
  65: -- Delete from aspnet_UsersInRoles
  66: delete from aspnet_UsersInRoles WHERE UserID IN
  67: ( SELECT UserID FROm AspnetUsersToDelete )
  68: GO
  69:  
  70: -- Delete from aspnet_PersonalizationPerUser
  71: delete from aspnet_PersonalizationPerUser WHERE UserID IN
  72: ( SELECT UserID FROm AspnetUsersToDelete )
  73: GO
  74:  
  75: -- Delete the users
  76: delete from aspnet_users where userID IN
  77: ( SELECT UserID FROm AspnetUsersToDelete )
  78:  
  79: PRINT 'Users deleted: ' + convert(varchar(255), @@ROWCOUNT)
  80: GO
  81:  
  82:  
  83: drop table PagesToDelete
  84: drop table AspnetUsersToDelete
  85: GO


Now the question comes, when can I run this script? It depends on several factors:

  •     The lowest traffic period. For example, USA midnight time when everyone in USA is sleeping if your majority users are from USA
  •     The period when there’s no other maintenance tasks running like Index Defrag or Database Bakup. If by any chance any other maintenance task conflicts with this enormous delete operation, SQL Server is dead.
  •     The operation will take from 10 mins to hours depending on the volume of trash to cleanup. So, consider the duration of running this script and plan other maintenance jobs accordingly.
  •     It’s best to run 30 mins before INDEX DEFRAG jobs run. After the script completes, the tables will be heavily fragmented. So, you need to defrag the indexes.

Before running this script, there are some preparations to take:

  • Make sure you have turned of AutoShrink from Database Property. Database size will reduce after the cleanup and if SQL Server tried to shrink the database, there will be a big IO activity. Turn off auto shrink because the database will grow again.
  • Make sure the LOG file’s initial size is big enough to hold such enormous transactions. You can specify 1/3rd of the MDF size as LDF’s Initial Size. Also make sure the log file is not shrunk. Let it occupy HD space. It saves SQL Server from expanding the file and shrinking the file. Both of these require high Disk IO.

Once the cleanup job runs and the INDEX DEFRAG runs, the database performance will improve significantly. The tables are now smaller. That means the indexes are now smaller. SQL Server need not to run through large indexes anymore. Future INDEX DEFRAGs take shorter time because there’s not much data left to optimize. SQL Server also takes less RAM because it has to work with much less amount of data. Database backup size also reduces because the MDF size does not keep increasing indefinitely. As a result, the significant overhead of this cleanup operation is quite acceptable when compared to all the benefits.

Note: I will be posting some stuffs from my old blog to new blog. Please ignore if you have read them before.
Posted by omar with 31 comment(s)
Filed under:

Prevent Denial of Service (DOS) attacks in your web application

Web services are the most attractive target for hackers because even a pre-school hacker can bring down a server by repeatedly calling a web service which does expensive work. Ajax Start Pages like Pageflakes are the best target for such DOS attack because if you just visit the homepage repeatedly without preserving cookie, every hit is producing a brand new user, new page setup, new widgets and what not. The first visit experience is the most expensive one. Nonetheless, it’s the easiest one to exploit and bring down the site. You can try this yourself. Just write a simple code like this:

   1: for( int i = 0; i < 100000; i ++ )
   2: {
   3:    WebClient client = new WebClient();
   4:    client.DownloadString("http://www.pageflakes.com/default.aspx");
   5: }

In your great surprise, you will notice that, after a couple of call, you don't get valid response. It’s not that you have succeeded in bringing down the server. It’s that your requests are being rejected. You are happy that you no longer get any service, thus you achieve Denial of Service (for yourself). I am happy to Deny You of Service (DYOS).

The trick I have in my sleeve is an inexpensive way to remember how many requests are coming from a particular IP. When the number of request exceeds the threshold, deny further request for some duartion. The idea is to remember caller’s IP in Asp.net Cache and maintain a count of request per IP. When the count exceeds a predefined limit, reject further request for some specific duration like 10 mins. After 10 mins, again allow requests from that IP.

I have a class named ActionValidator which maintains a count of specific actions like First Visit, Revisit, Asynchrnous postbacks, Add New widget, Add New Page etc. It checks whether the count for such specific action for a specific IP exceeds the threshold value or not.

   1: public static class ActionValidator
   2: {
3: private const int DURATION = 10; // 10 min period
   4:  
   5:     public enum ActionTypeEnum
   6:     {
   7:         FirstVisit = 100, // The most expensive one, choose the valu wisely. 
   8:         ReVisit = 1000, // Welcome to revisit as many times as use likes
   9:         Postback = 5000,   // Not must of a problem for us
  10:         AddNewWidget = 100, 
  11:         AddNewPage = 100,
  12:     }

The enumeration contains the type of actions to check for and their threshold value for a specific duration – 10 mins.

A static method named IsValid does the check. It returns true if the request limit is not passed, false if the request needs to be denied. Once you get false, you can call Request.End() and prevent Asp.net from proceeding further. You can also switch to a page which shows “Congratulations! You have succeeded in Denial of Service Attack.”

 

   1: 
public static bool IsValid( ActionTypeEnum actionType )
   2: {
   3: HttpContext context  = HttpContext.Current;
   4: 
if( context.Request.Browser.Crawler ) return false;
   5:  
   6: string key = actionType.ToString()  + context.Request.UserHostAddress;
   7:  
   8: HitInfo hit  = (HitInfo)(context.Cache[key] ?? new HitInfo());
   9:  
  10: if( hit.Hits > (int)actionType ) return false;
  11: else hit.Hits ++;
  12:  
  13: if( hit.Hits == 1 )
  14:     context.Cache.Add(key, hit, null, DateTime.Now.AddMinutes(DURATION), 
  15:       System.Web.Caching.Cache.NoSlidingExpiration, System.Web.Caching.CacheItemPriority.Normal, null);
  16:  
  17: return true;
  18: }

The cache key is built with a combination of action type and client IP address. First it checks if there’s any entry for the action and the client IP in Cache or not. If not, start the count and store remember the count for the IP in cache for the specific duration. The absolute expiration on cache item ensures after the duration, the cache item will be cleared and the count will restart. When there’s already an entry in the cache, get the last hit count, and check if the limit is exceeded or not. If not exceeded, increase the counter. There is no need to store the updated value in the cache again by doing: Cache[url]=hit; because the hit object is by reference and changing it means it gets changed in the cache as well. In fact, if you do put it again in the cache, the cache expiration counter will restart and fail the logic of restarting count after specific duration.

The usage is very simple:

   1: 
protected override void OnInit(EventArgs e)
   2: {
   3:   base.OnInit(e);
   4:  
   5:   // Check if revisit is valid or not
   6:   if(  !base.IsPostBack ) 
   7:   {
   8:     // Block cookie less visit attempts
   9:     if( Profile.IsFirstVisit )
  10:     {
  11:        if( !ActionValidator.IsValid(ActionValidator.ActionTypeEnum.FirstVisit) Response.End();
  12:     }
  13:    else
  14:     {
  15:      if( !ActionValidator.IsValid(ActionValidator.ActionTypeEnum.ReVisit) ) Response.End();
  16:     }
  17:   }
  18:  else
  19:   {
  20:    // Limit number of postbacks
  21:     if( !ActionValidator.IsValid(ActionValidator.ActionTypeEnum.Postback)  Response.End();
  22:   }
  23: }

Here I am checking specific scenario like First Visit, re-visit, postbacks etc.

Of course you can put in some Cisco firewall and prevent DOS attack. You will get guaranty from your hosting provider that their entire network is immune to DOS and DDOS (Distributed DOS) attacks. What they guaranty is network level attack like TCP SYN attacks or malformed packet floods etc. There is no way they can analyze the packet and find out a particular IP is trying to load the site too many times without supporting cookie or trying to add too many widgets. These are called application level DOS attack which hardware cannot prevent. It must be implemented in your own code.

There are very few websites out their which take such precaution for application level DOS attacks. Thus it’s quite easy to make servers go mad by writing a simple loop and hitting expensive pages or web services continuously from your home broadband connection. I hope this small but effective class will help you implement DOS attack in your own web applications.

 

Update

Here's the code of the full ActionValidator class:

// Copyright (c) Omar AL Zabir. All rights reserved.
// For continued development and updates, visit http://msmvps.com/omar

using System;
using System.Data;
using System.Configuration;
using System.Web;
using System.Web.Security;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Web.UI.WebControls.WebParts;
using System.Web.UI.HtmlControls;

/// <summary>
/// Summary description for ActionValidator
/// </summary>
namespace Dropthings.Web.Util
{
  public static class ActionValidator
  {
  private const int DURATION = 10; // 10 min period
   
  /*
  * Type of actions and their maximum value per period
  *
  */
  public enum ActionTypeEnum
  {
  None = 0,
  FirstVisit = 100, // The most expensive one, choose the value wisely.
  Revisit = 1000, // Welcome to revisit as many times as user likes
  Postback = 5000, // Not must of a problem for us
  AddNewWidget = 100,
  AddNewPage = 100,
  }

  private class HitInfo
  {
  public int Hits;
  private DateTime _ExpiresAt = DateTime.Now.AddMinutes(DURATION);
  public DateTime ExpiresAt { get { return _ExpiresAt; } set { _ExpiresAt = value; } }
  }

  public static bool IsValid( ActionTypeEnum actionType )
  {
  HttpContext context = HttpContext.Current;
  if( context.Request.Browser.Crawler ) return false;

  string key = actionType.ToString() + context.Request.UserHostAddress;

  HitInfo hit = (HitInfo)(context.Cache[key] ?? new HitInfo());

  if( hit.Hits > (int)actionType ) return false;
  else hit.Hits ++;

  if( hit.Hits == 1 )
  context.Cache.Add(key, hit, null, DateTime.Now.AddMinutes(DURATION),
  System.Web.Caching.Cache.NoSlidingExpiration, System.Web.Caching.CacheItemPriority.Normal, null);
   
  return true;
  }
  }
}

 

Posted by omar with 43 comment(s)
Filed under:

ASP.NET Ajax Extender for multi-column widget drag & drop

My open source Ajax Start Page www.dropthings.com has an ASP.NET Ajax Extender which provides multi-column drag & drop for widgets. It allows reordering of widgets on the same column and also drag & drop between column. It also supports client side notification so that you can call web service and store the position of the widgets behind the scene without (async) postback.

I first thought of going for a plain vanilla Javascript based solution for drag & drop. It requires less code, less architectural complexity and provides better speed. Another reason was the high learning curve for making Extenders in proper way in Asp.net Ajax given that there’s hardly any documentation available on the web (during the time of writing this blog). However, writing a proper extender which pushes Asp.net Ajax to the limit is a very good way to learn under-the-hood secrets of Asp.net Ajax framework itself. So, the two extenders I will introduce here will tell you almost everything you need to know about Asp.net Ajax Extenders.

Ajax Control Toolkit comes with a DragPanel extender which I could use to provide drag & drop support to panels. It also has a ReorderList control which I could use to provide reordering of items in a single list. Widgets are basically panels that flow vertically in each column. So, it might be possible that I could create reorder list in each column and use the DragPanel to drag the widgets. But I could not use ReorderList because:

  • ReorderList strictly uses Html Table to render its items in a column. I have no table inside the columns. Only one Panel is there inside an UpdatePanel.
  • ReorderList takes a Drag Handle template and creates a drag handle for each item at runtime. I already have drag handle created inside a Widget which is the widget header. So, I cannot allow ReorderList to create another drag handle.
  • I need client side callback on drag & drop so that I can make Ajax calls and persist the widget positions. The callback must give me the Panel where the widget is dropped, which is dropped and at what position.

 

Next challenge is with the DragPanel extender. The default implement of Drag & Drop in Ajax Control Toolkit has some problems:

  • When you start dragging, the item becomes absolutely positioned, but when you drop it, it does not become static positioned. There's a small hack needed for restoring the original position to "static".
  • It does not put the dragging item on top of all items. As a result, when you start dragging, you see the item being dragged below other items which makes the drag get stuck especially when there's an IFRAME.

So, I have made a CustomDragDropExtender and a CustomFloatingExtender. CustomDragDropExtender is for the column containers where widgets are placed. It provides the reordering support. You can attach this extender to any Panel control.

Here's how you can attach this extender to any Panel and make that Panel support drag & drop of Widgets:  

   1: 
<
asp:Panel 
ID
="LeftPanel" 
runat
="server"  
class
="widget_holder" 
columnNo
="0"
>
   2:         
<
div 
id
="DropCue1" 
class
="widget_dropcue"
>
   3:         
</
div
>
   4: 
</
asp:Panel
>
   5:  
   6: 
<
cdd:CustomDragDropExtender 
ID
="CustomDragDropExtender1" 
   7:     
runat
="server" 
   8:     
TargetControlID
="LeftPanel"
   9:     
DragItemClass
="widget" 
  10:     
DragItemHandleClass
="widget_header" 
  11:     
DropCueID
="DropCue1"
  12:     
OnClientDrop
="onDrop" 
/>

<cdd:CustomDragDropExtender> offers the following properties:

  • TargetControlID – ID of the Panel which becomes the Drop zone
  • DragItemClass – All child elements inside the Panel having this class will become draggable. E.g. Widget DIV has this class so that it can become draggable.
  • DragItemHandleClass – Any child element having this class inside the draggable elements will become the drag handle for the draggable element. E.g. Widget Header area has this class, so it acts as the drag handle for the Widget.
  • DropCueID – ID of an element inside the Panel which acts as Drop Cue.
  • OnClientDrop – Name of a Javascript function which is called when widget is dropped on the Panel.

LeftPanel becomes a widget container which allows widgets to be dropped on it and reordered. The DragItemClass attribute on the extender defines the items which can be ordered. This prevents from non-widget Html Divs from getting ordered. Only the DIVs with the class "widget" are ordered. Say there are 5 DIVs with the class named "widget". It will allow reordering of only these five divs: 

   1: 
<
div 
id
="LeftPanel" 
class
="widget_holder" 
>
   2:         
<
div 
class
="widget"
> ... 
</
div
>
   3:         
<
div 
class
="widget"
> ... 
</
div
>
   4:  
   5:         
<
div 
class
="widget"
> ... 
</
div
>
   6:         
<
div 
class
="widget"
> ... 
</
div
>
   7:         
<
div 
class
="widget"
> ... 
</
div
>
   8:  
   9:         
<
div
>This DIV will not move
</
div
>
  10:         
<
div 
id
="DropCue1" 
class
="widget_dropcue"
></
div
>
  11: 
</
div
>

When a widget is dropped on the panel, the extender fires the function specified in OnClientDrop. It offers standard Ajax Events. But unlike basic Ajax events where you have to programmatically bind to events, you can define property and specify the function name to call. So, instead of doing this:

   1: 
function pageLoad( sender, e ) {
   2:  
   3:   
var extender1 =
$get(‘CustomDragDropExtender1’);
   4:   extender1.add_onDrop( onDrop );
   5:   
   6: }

You can do this:

   1: 
<
cdd:CustomDragDropExtender 
ID
="CustomDragDropExtender1" 
   2:     
runat
="server" 
   3:     
OnClientDrop
="onDrop" 
/>

When the event is raised, the function named onDrop gets fired. This is done with the help of some handy library available in ACT project.

When the event is fired, it sends the container, the widget and the position of the widget where the widget is dropped.

   1: 
function onDrop( sender, e )
   2: {
   3:     
var container = e.get_container();
   4:     
var item = e.get_droppedItem();
   5:     
var position = e.get_position();
   6:     
   7:     
//alert( String.format( "Container: {0}, Item:
{1}, Position: {2}", container.id, item.id, position ) );
   8:     
   9:     
var instanceId =
parseInt(item.getAttribute(
"InstanceId"));
  10:     
var columnNo =
parseInt(container.getAttribute(
"columnNo"));
  11:     
var row = position;
  12:     
  13:    
WidgetService.MoveWidgetInstance( instanceId, columnNo, row );
  14: }

 

The widget location is updated by calling the WidgetService.MoveWidgetInstance.

CustomDragDropExtender has 3 files:

  • CustomDragDropExtender.cs – The server side extender implementation
  • CustomDragDropDesigner.cs – Designer class for the extender
  • CustomDragDropExtender.js – Client side scriptfor the extender

Server side class CustomDragDropExtender.cs has the following code:

   1: [assembly:
System.Web.UI.WebResource(
"CustomDragDrop.CustomDragDropBehavior.js",

"text/javascript")]
   2:  
   3: 
namespace CustomDragDrop
   4: {
   5:     [Designer(
typeof(CustomDragDropDesigner))]
   6:     [ClientScriptResource(
"CustomDragDrop.CustomDragDropBehavior", 
"CustomDragDrop.CustomDragDropBehavior.js")]
   7:     [TargetControlType(
typeof(WebControl))]
   8:     [RequiredScript(
typeof(CustomFloatingBehaviorScript))]
   9:     [RequiredScript(
typeof(DragDropScripts))]
  10:     
public 
class CustomDragDropExtender :
ExtenderControlBase
  11:     {
  12:         
// TODO: Add your property accessors here.
  13:         
//
  14:         [ExtenderControlProperty]
  15:         
public 
string DragItemClass
  16:         {
  17:             get
  18:             {
  19:                 
return GetPropertyValue<String>(
"DragItemClass", 
string.Empty);
  20:             }
  21:             set
  22:             {
  23:                
SetPropertyValue<String>(
"DragItemClass", 
value);
  24:             }
  25:         }
  26:  
  27:         [ExtenderControlProperty]
  28:         
public 
string DragItemHandleClass
  29:         {
  30:             get
  31:             {
  32:                 
return GetPropertyValue<String>(
"DragItemHandleClass", 
string.Empty);
  33:             }
  34:             set
  35:             {
  36:                
SetPropertyValue<String>(
"DragItemHandleClass", 
value);
  37:             }
  38:         }
  39:  
  40:         [ExtenderControlProperty]
  41:         [IDReferenceProperty(
typeof(WebControl))]
  42:         
public 
string DropCueID
  43:         {
  44:             get
  45:             {
  46:                 
return GetPropertyValue<String>(
"DropCueID", 
string.Empty);
  47:             }
  48:             set
  49:             {
  50:                
SetPropertyValue<String>(
"DropCueID", 
value);
  51:             }
  52:         }
  53:  
  54:         [ExtenderControlProperty()]
  55:         [DefaultValue(
"")]
  56:         [ClientPropertyName(
"onDrop")]
  57:         
public 
string OnClientDrop
  58:         {
  59:             get
  60:             {
  61:                 
return GetPropertyValue<String>(
"OnClientDrop", 
string.Empty);
  62:             }
  63:             set
  64:             {
  65:                
SetPropertyValue<String>(
"OnClientDrop", 
value);
  66:             }
  67:         }
  68:  
  69:     }
  70: }

Most of the code in the extender defines the property. The important part is the declaration of the class:

   1: [assembly:
System.Web.UI.WebResource(
"CustomDragDrop.CustomDragDropBehavior.js",

"text/javascript")]
   2:  
   3: 
namespace CustomDragDrop
   4: {
   5:     [Designer(
typeof(CustomDragDropDesigner))]
   6:     [ClientScriptResource(
"CustomDragDrop.CustomDragDropBehavior", 
"CustomDragDrop.CustomDragDropBehavior.js")]
   7:     [TargetControlType(
typeof(WebControl))]
   8:     [RequiredScript(
typeof(CustomFloatingBehaviorScript))]
   9:     [RequiredScript(
typeof(DragDropScripts))]
  10:     
public 
class CustomDragDropExtender :
ExtenderControlBase
  11:     {

The extender class inherits from ExtenderControlBase defined in ACT project. This base class has additional features over Ajax runtime provided Extender base class. It allows you to define RequiredScript attribute, which makes sure all the required scripts are downloaded before the extender script is downloaded and initialized. This extender has dependency over another extender named CustomFloatingBehavior. It also depends on ACT’s DragDropManager. So, the RequiredScript attribute makes sure those are downloaded before this extender’s script is downloaded. The ExtenderControlBase is a pretty big class and does a lot of work for us. It contains default implementations for discovering all the script files for the extender and rendering them properly.

The [assembly:System.Web.UI.WebResource] attribute defines the script file containing the script for extender. The script file is an embedded resource file.

[ClientScriptResource] attribute defines the scripts required for the extender. This class is also defined in ACT. ExtenderControlBase uses this attribute to find out which .js files are working for the extender and renders them properly.

The challenge is to make the client side javascript for the extender. On the js file, there’s a Javascript pseudo class:

   1: Type.registerNamespace(
'CustomDragDrop');
   2:  
   3:
CustomDragDrop.CustomDragDropBehavior = 
function(element) {
   4:  
   5:    
CustomDragDrop.CustomDragDropBehavior.initializeBase(
this, [element]);
   6:     
   7:     
this._DragItemClassValue = 
null;    
   8:     
this._DragItemHandleClassValue = 
null;
   9:     
this._DropCueIDValue = 
null;
  10:     
this._dropCue = 
null;
  11:     
this._floatingBehaviors = [];
  12: }

During initialize, it hooks on the Panel it is attached to and the drop cue to show while drag & drop is going on over the Panel:

   1:
CustomDragDrop.CustomDragDropBehavior.prototype = {
   2:     
   3:     initialize : 
function() {
   4:         
// Register ourselves as a drop target.
   5:        
AjaxControlToolkit.DragDropManager.registerDropTarget(
this); 
   6:         
//Sys.Preview.UI.DragDropManager.registerDropTarget(this);
   7:         
   8:         
// Initialize drag behavior after a while
   9:         window.setTimeout(
Function.createDelegate( 
this, 
this._initializeDraggableItems ), 3000 );
  10:         
  11:         
this._dropCue = $get(
this.get_DropCueID());
  12:     },

After initializing the DragDropManager and marking the Panel as a drop target, it starts a timer to discover the dragable items inside the panel and create Floating behavior for them. Floating behavior is the one which makes a DIV draggable.

FloatingBehavior makes one DIV freely draggable on the page. But it does not offer drop functionality. DragDropBehavior offers the drop functionality which allows a freely moving DIV to rest on a fixed position.

Discovering and initializing floating behavior for the dragable items is the challenging work:

   1: 
// Find all items with the drag item class and
make each item
   2: 
// draggable        
   3: _initializeDraggableItems : 
function() 
   4: {
   5:     
this._clearFloatingBehaviors();
   6:     
   7:     
var el = 
this.get_element();
   8:     
   9:     
var child = el.firstChild;
  10:     
while( child != 
null )
  11:     {
  12:         
if( child.className == 
this._DragItemClassValue && child
!= 
this._dropCue)
  13:         {
  14:             
var handle = 
this._findChildByClass(child, 
this._DragItemHandleClassValue);
  15:             
if( handle )
  16:             {
  17:                 
var handleId = handle.id;
  18:                 
var behaviorId = child.id + 
"_WidgetFloatingBehavior";
  19:                 
  20:                 
// make the item draggable by adding floating
behaviour to it                    
  21:                 
var floatingBehavior =
$create(CustomDragDrop.CustomFloatingBehavior, 
  22:                         {
"DragHandleID":handleId, 
"id":behaviorId, 
"name": behaviorId}, {}, {}, child);
  23:                 
  24:                 Array.add( 
this._floatingBehaviors, floatingBehavior
);
  25:             }
  26:         }            
  27:         child = child.nextSibling;
  28:     }
  29: },

Here’s the algorithm:

  • Run through all immediate child elements of the control where the extender is attached to
  • If the child item has the class for draggable item, then:
    • Find any element under the child item which has the class for Drag handle
    • If such item found, then attach a CustomFloatingBehavior with the child item

The _findChildByClass function recursively iterates through all the child elements and looks for an element which has the defined class. It’s an expensive function. So, it is important that the drag handle is very close to the dragable element.

   1: _findChildByClass : 
function(item, className)
   2: {
   3:     
// First check all immediate child items
   4:     
var child = item.firstChild;
   5:     
while( child != 
null )
   6:     {
   7:         
if( child.className == className ) 
return child;
   8:         child = child.nextSibling;
   9:     }
  10:     
  11:     
// Not found, recursively check all child
items
  12:     child = item.firstChild;
  13:     
while( child != 
null )
  14:     {
  15:         
var found = 
this._findChildByClass( child, className
);
  16:         
if( found != 
null ) 
return found;
  17:         child = child.nextSibling;
  18:     }
  19: },

When user drags an item over the Panel where the extender is attached to, DragDropManager fires the following events:

   1: onDragEnterTarget : 
function(dragMode, type, data) {
   2:     
this._showDropCue(data);    
   3: },
   4:  
   5: onDragLeaveTarget : 
function(dragMode, type, data) {
   6:     
this._hideDropCue(data);
   7: },
   8:  
   9: onDragInTarget : 
function(dragMode, type, data) {
  10:     
this._repositionDropCue(data);
  11: },

Here we deal with the drop cue. The challenging work is to find out the right position for the drop cue.

We need to find out where we should show the drop cue based on where user is dragging the item. The idea is to find out the widget which is immediately below the dragged item. The item is pushed down by one position and the drop cue takes its place. While dragging, the position of the drag item can be found easily. Based on that, I locate the widget below the drag item:

   1: _findItemAt : 
function(x,y, item)
   2:     {
   3:         
var el = 
this.get_element();
   4:         
   5:         
var child = el.firstChild;
   6:         
while( child != 
null )
   7:         {
   8:             
if( child.className == 
this._DragItemClassValue && child
!= 
this._dropCue && child != item )
   9:             {
  10:                 
var pos =
Sys.UI.DomElement.getLocation(child);
  11:                 
  12:                 
if( y <= pos.y )
  13:                 {
  14:                     
return child;
  15:                 }
  16:             }
  17:             child =
child.nextSibling;
  18:         }
  19:         
  20:         
return 
null;
  21:     },

This function returns the widget which is immediately under the dragged item. Now I add the drop cue immediately above the widget:

   1: _repositionDropCue : 
function(data)
   2:     {
   3:         
var location =
Sys.UI.DomElement.getLocation(data.item);
   4:         
var nearestChild = 
this._findItemAt(location.x, location.y,
data.item);
   5:         
   6:         
var el = 
this.get_element();        
   7:             
   8:         
if( 
null == nearestChild )
   9:         {
  10:             
if( el.lastChild != 
this._dropCue )
  11:             {
  12:                 el.removeChild(
this._dropCue);
  13:                 el.appendChild(
this._dropCue);
  14:             }
  15:         }
  16:         
else
  17:         {
  18:             
if( nearestChild.previousSibling != 
this._dropCue )
  19:             {
  20:                 el.removeChild(
this._dropCue);
  21:                 el.insertBefore(
this._dropCue, nearestChild);            
  22:             }            
  23:         }
  24:     },

One exception to consider here is that there can be no widget immediately below the dragged item. It happens when user is trying to drop the widget at the bottom of column. In that case, the drop cue is shown at the bottom of the column.

When user releases the widget, it drops right on top of drop cue and the drop cue disappears. After the drop the onDrop event is raised to notify where the widget is dropped.

   1: _placeItem : 
function(data)
   2:     {
   3:         
var el = 
this.get_element();
   4:                 
   5:        
data.item.parentNode.removeChild( data.item );
   6:         el.insertBefore( data.item,

this._dropCue );
   7:         
   8:         
// Find the position of the dropped item
   9:         
var position = 0;
  10:         
var item = el.firstChild;
  11:         
while( item != data.item )
  12:         {  
  13:             
if( item.className == 
this._DragItemClassValue ) position++; 
  14:             item =
item.nextSibling; 
  15:         }
  16:         
this._raiseDropEvent( 
/* Container */ el, 
/* droped item */ data.item, 
/* position */ position );
  17:     }
 
Generally you can make events in extenders by adding two functions in the extender:
 
   1: add_onDrop : 
function(handler) {
   2:     
this.get_events().addHandler(
"onDrop", handler);
   3: },
   4:  
   5: remove_onDrop : 
function(handler) {
   6:     
this.get_events().removeHandler(
"onDrop", handler);
   7: },
 
But this does not give you the support for defining the event listener name in the ASP.NET declaration:
   1: 
<
cdd:CustomDragDropExtender 
ID
="CustomDragDropExtender1" 
   2:     
runat
="server" 
   3:     
OnClientDrop
="onDrop" 
/>

The declaration only allows properties. In order to support such declarative assignment of events, we need to first introduce a property named OnClientDrop in the extender. Then during assignment of the property, we need to find out the function specified there and attach event notification on that function. The discovery of the function from its name is done by CommonToolkitScripts.resolveFunction which is available in ACT project.

   1: 
// onDrop property maps to onDrop event
   2:     get_onDrop : 
function() {
   3:         
return 
this.get_events().getHandler(
"onDrop");
   4:     },
   5:  
   6:     set_onDrop : 
function(value) {
   7:         
if (value && (0 <
value.length)) {
   8:             
var func =
CommonToolkitScripts.resolveFunction(value);
   9:             
if (func) { 
  10:                 
this.add_onDrop(func);
  11:             } 
else {
  12:                 
throw Error.argumentType(
'value', 
typeof(value), 
'Function', 
'resize handler not a function, function name, or
function text.');
  13:             }
  14:         }
  15:     },
 
Raising the event is same as basic Ajax events:
 
 
   1: _raiseEvent : 
function( eventName, eventArgs ) {
   2:         
var handler = 
this.get_events().getHandler(eventName);
   3:         
if( handler ) {
   4:             
if( !eventArgs ) eventArgs =
Sys.EventArgs.Empty;
   5:             handler(
this, eventArgs);
   6:         }
   7:     },
 
This is all about the CustomDragDropExtender. Next challenge is to make the CustomFloatingBehavior. The server side class is declared as:
   1: [assembly:
System.Web.UI.WebResource(
"CustomDragDrop.CustomFloatingBehavior.js",

"text/javascript")]
   2:  
   3: 
namespace CustomDragDrop
   4: {
   5:     [Designer(
typeof(CustomFloatingBehaviorDesigner))]
   6:     [ClientScriptResource(
"CustomDragDrop.CustomFloatingBehavior", 
"CustomDragDrop.CustomFloatingBehavior.js")]
   7:     [TargetControlType(
typeof(WebControl))]
   8:     [RequiredScript(
typeof(DragDropScripts))]
   9:     
public 
class CustomFloatingBehaviorExtender :
ExtenderControlBase
  10:     {
  11:         [ExtenderControlProperty]
  12:         [IDReferenceProperty(
typeof(WebControl))]
  13:         
public 
string DragHandleID
  14:         {
  15:             get
  16:             {
  17:                 
return GetPropertyValue<String>(
"DragHandleID", 
string.Empty);
  18:             }
  19:             set
  20:             {
  21:                
SetPropertyValue<String>(
"DragHandleID", 
value);
  22:             }
  23:         }
  24:     }
  25: }

There’s only one property – DragHandleID. Widget’s header works as the drag handle. So, the header ID is specified here.

This extender has dependency on DragDropManager so the [RequiredScript(typeof(DragDropScripts))] attribute is there.

Besides the designer class, there’s one more class which CustomDragDropExtender need in order to specify its dependency over this floating behavior:

   1: [ClientScriptResource(
null, 
"CustomDragDrop.CustomFloatingBehavior.js")]
   2:   
public 
static 
class CustomFloatingBehaviorScript
   3:   {
   4:   }

This class can be used inside RequiredScript attribute. It only defines which script file contains the client side code for the extender.

The client side Javascript is same as FloatingBehavior that comes with ACT. The only difference is some hack when drag starts. DragDropManager does not return the item being dragged to static position once it makes it absolute. It also does not increase the zIndex of the item. If the drag item does not become the top most item, while dragging it goes below other elements on the page. So, I have made some changes in the mouseDownHandler of the behavior to add these features:

   1: 
function mouseDownHandler(ev) {
   2:     window._event = ev;
   3:     
var el = 
this.get_element();
   4:     
   5:     
if (!
this.checkCanDrag(ev.target)) 
return;
   6:     
   7:     
// Get the location before making the element
absolute
   8:     _location =
Sys.UI.DomElement.getLocation(el);
   9:  
  10:     
// Make the element absolute 
  11:     el.style.width = el.offsetWidth
+ 
"px";
  12:     el.style.height =
el.offsetHeight + 
"px";            
  13:    
Sys.UI.DomElement.setLocation(el, _location.x, _location.y);       
       
  14:     
  15:     _dragStartLocation =
Sys.UI.DomElement.getLocation(el);        
  16:     
  17:     ev.preventDefault();
  18:     
  19:     
this.startDragDrop(el);
  20:     
  21:     
// Hack for restoring position to static
  22:     el.originalPosition = 
"static";
  23:     el.originalZIndex =
el.style.zIndex;
  24:     el.style.zIndex = 
"60000";
  25: }

Setting el.originalPosition = static fixes the bug in DragDropManager. It incorrectly stores absolute has the originalPosition when startDragDrop is called. So, after calling this function, I set it to correct originalPosition which is “static”.

When drag completes, the original zIndex is restored and left, top, width and height is cleared. DragDropManager makes the item position static, but it does not clear the left, top, width and height attributes. This moves the element away from the place where it is dropped. This bug is fixed in the onDragEnd event:

   1: 
this.onDragEnd = 
function(canceled) {
   2:     
if (!canceled) {
   3:         
var handler = 
this.get_events().getHandler(
'move');
   4:         
if(handler) {
   5:             
var cancelArgs = 
new Sys.CancelEventArgs();
   6:             handler(
this, cancelArgs);
   7:             canceled =
cancelArgs.get_cancel();
   8:         }            
   9:     }
  10:     
  11:     
var el = 
this.get_element();
  12:     el.style.width =
el.style.height = el.style.left = el.style.top = 
"";
  13:     el.style.zIndex =
el.originalZIndex;
  14: }

That's all folks!

You can get the code for the extenders here.

Posted by omar with 142 comment(s)
Filed under:

ASP.NET Ajax in-depth performance analysis

Let's do some ASP.NET runtime analysis on www.dropthings.com. Those who don't know what it is, it's an open source start page I made using ASP.NET Ajax, .NET 3.0 and Linq.

ASP.NET Ajax has a pretty big runtime which consists of Core Framework, scripts for UpdatePanel, Preview Script required for Drag & Drop. Additionally I need Ajax Control Toolkit (ACT). All these add up to a staggering 564 KB of download in 12 script references on the page. The download size mostly depends on the usage of extenders and Ajax features. A moderate use of extender results in the following download:

Here’s a simulation of 256kbps internet speed on 200ms network latency. I use a tool named Charles ( www.xk72.com/charles) to capture traffic and simulate slow internet speed by throttling data transfer speed. From the durations, we see almost 20 sec is spent on downloading the runtime over a 256kbps line!

Let’s explain which script in the above list does what. I will be identifying the scripts using their file size in order of their appearance:

  • 21.64 KB – Handy script for Postbacks
  • 83.38 KB – Microsoft Ajax Core runtime
  • 30.16 KB - UpdatePanel, partial update, asynchronous postback scripts.
  • 136.38 KB – Preview version of Ajax which allows Drag & Drop script
  • 36.02 KB – The actual drag & drop script in Preview library
  • 45.25 KB – Ajax Control Toolkit
  • 4.08 KB – Timer script
  • 140.86 KB – ACT Animation framework
  • 18.05 KB – ACT Behavior base implementation. Required for Behaviors provided in the Ajax Control Toolkit project.
  • 16.48 KB – ACT Animation Behavior
  • 7.32 KB – My Custom Drag Drop behavior
  • 9.73 KB – My Custom Floating Behavior

The total payload for only the runtime is too high to accept because we cannot make a user wait for 20 sec just to download Ajax scripts before user can actually start interacting on the page. So, I took several approaches to reduce the size of the download:

  • Eliminate Preview version of Ajax completely and use ACT for Drag & Drop
  • Use IIS 6 compression to deliver compressed scripts from the client
  • Combine multiple script files into one file

ACT comes with its own DragDropManager which we need for Drag & Drop. So far I thought of using Sys.Preview.UI.DragDropManager in the ASP.NET Ajax January CTP. But just for the DragDropManager, I need to add nearly 180 KB of scripts for the entire Preview Library runtime. If I use ACT’s DrgaDropManager, I can get rid of the Preview runtime.

Without the preview scripts, the scripts downloaded are:

The following downloads are missing here:

  • 136.38 KB – Preview version of Ajax which allows Drag & Drop script
  • 36.02 KB – The actual drag & drop script in Preview library

I saved about 180 KB, nearly 7 sec from total download. So, user gets to work on the page 7 secs earlier than before.

When I enabled IIS 6 compression, the situation improved dramatically. Here’s the data transfers when compression is enabled:

The total download comes down from 448 KB to 163 KB! Total 64% download bytes reduction! This results in 3 times faster page download.

The scripts are downloaded in two steps – first the core runtimes download and then ACT and other scripts download. The content is displayed after the core runtime is downloaded. So, the time it takes to show the content on the browser reduces significantly because now it takes only 50KB to download before something is visible on the screen compared to 130 KB on non-compressed mode.

ScriptManager control has LoadScriptsBeforeUI property which you can set to “False” in order to postpone several script downloads after the content is downloaded. This adds the script references end of the <body> tag. As a result, you see the content first and then the additional scripts, exteders, ACT scripts get downloaded and initialized.


< asp:ScriptManager ID ="ScriptManager1" runat ="server" EnablePartialRendering ="true" LoadScriptsBeforeUI ="false" ScriptMode ="Release" > ... </ asp:ScriptManager >

You can explicitly set ScriptMode=false in order to emit release mode highly optimized Ajax runtime scripts during local debugging.

Posted by omar with 21 comment(s)
Filed under:

Live OneCare - don't try this at home

I honestly and sincerely tried to use Windows Live OneCare for a month. It made my computer crawl to death, I still insisted on using it. It made Visual Studio Build process 5x slower, I still kept it on. It made my every day email check at least 40 mins longer than before, I still stubbornly kept using it for my sincere love for Microsoft Products. But no more. I had enough of it. This is the most crap product Microsoft has ever released and will ever release (I bet, otherwise I am switching to Linux/Java platform). I am so shockingly surprised how Microsoft can produce such a low quality mass consumer product, release it and keep shouting every where about its praise. If you go to Microsoft forums, you will be surprised how badly everyone has rejected this product. If Microsoft is really honest about their products, they should have withdrawn OneCare from the market after so much negative feedback. 

Here are some of the major problems I have faced:

  • I had to spend at least 30 to 40 mins more in Outlook 2007 since I installed it. Every single click, swtching folders, downloading emails was so slow that it felt like I am using a 386 PC.
  • Almost whole day whenever I look at CPU meter, it shows MsMpEng is taking 50% CPU. I have dual core, so it's occupying one core at 100%.
  • Visual Studio build was like hell. I can make a cup of tea, drink it, chat with my friends while my solution builds.
  • It will not cancel Virus scan no matter how many times I tell it to stop. It will again start within a minute. There is simply no way to return to workable condition until you disable the Virus & Spyware scanning feature.
  • Every morning, I tried to tolerate it for an hour or so, then I disable the Virus & Spyware scanning and go back to normal working mode. OneCare leaves no room for anyone to work on PC when it's installed.

Every single person I have seen so far installing OneCare, has gone back to some other product. There was not a single person who could continue with it. So, unless Microsoft makes major fix in this product and releases it under a different name, I will never use it. I strongly suggest you also never use it.

Posted by omar with 8 comment(s)
Filed under: