November 2007 - Posts

Creating a Performance Benchmark Framework for Your .NET Codes

Performance is a one of the basic requirements in the most of the current trend of web applications. While designing software architecture, there are several technologies kept available for the designers. Among these available options, some of techniques may includes multiple solutions, combining of those can create a case, where the basic performance output can't be determined based on the common sense and/or in depth knowledge on the corresponding topics. In short, based upon different options the software architect needs to determine the appropriate object model which fits best with the current application requirement.

The Object Model

There are three layers of our test framework object model, as follows:

The Program Class: This is the starter class, which creates the requires instances to start the test operation. The corresponding caller method initializes the Runner class along with the number of iterations to be performed for each test case.

The Runner Class: The runner class includes one or multiple methods, which calls a bunch of methods of executor class, each of which contains required functionalities to call the test cases appropriately.

    • The Runner Base Class: One important issue with respect to the runner class is, we need to have a way to keep track to execution time for each run and finally we need to determine the average execution time. For this purpose, to perform required initialization and support common reporting feature of test cases, we considered a base class, which includes all tracking data to isolate individual test cases and reporting functionalities. We have used C# delegate to have a very structured and reusable object model regarding this common functionalities. The methods of the inherited class uses this features accordingly.

The Executor Class: The executor class contains the very basic codes with respect to a particular test case.

ClassDiagram

The Sample Codes

The  Program Class

class Program

    {

        static void Main(string[] args)

        {

            Output.WriteLine("Test starts: " + DateTime.Now.ToString());

            //----------------------------------------------------------------

            SampleRunner s1 = new SampleRunner(10);

            s1.RunBenchmarks();

            //----------------------------------------------------------------

            Output.WriteLine("Test ends: " + DateTime.Now.ToString());

            Console.ReadLine();

        }

    }

The Runner Base Class

public class SampleRunnerBase

    {

        protected delegate void DoWork(string text);

 

        long _Start;

        long _Stop;

        double _Total = 0;

        double _TotalSecs;

        double _TotalAvgSecs;

        int _Repeats;

        protected int _RepeatsToDo;

 

        public SampleRunnerBase(int repeats)

        {

            _RepeatsToDo = repeats;

        }

 

        private void Init()

        {

            _Total = 0;

            _TotalSecs = 0;

            _TotalAvgSecs = 0;

            _Repeats = _RepeatsToDo;

        }

 

        private void ReportResults()

        {

            _TotalSecs = _Total / 10000000;

            _TotalAvgSecs = _Total / 10000000 / _RepeatsToDo;

 

            Output.WriteLine("Avg: " + _TotalAvgSecs + " s");

        }

 

        protected void RunBenchmark(DoWork func, string text)

        {

            if (null != func)

            {

                Init();

                while (_Repeats-- > 0)

                {

                    _Start = DateTime.Now.Ticks;

                    func(text);

                    _Stop = DateTime.Now.Ticks;

                    _Total += _Stop - _Start;

                }

                ReportResults();

            }

        }

    }

The Runner Class

public sealed class SampleRunner : SampleRunnerBase

    {

        public SampleRunner(int repeats)

            : base(repeats)

        {

        }

 

        public void RunBenchmarks()

        {

 

            Output.WriteLine("Running Simple Sample Benchmark");

            Output.WriteLine("--------------------------------");

            RunSample1();

            RunSample2();

        }

 

        private void RunSample1()

        {

            Output.WriteLine("Running Sample 1");

            SampleExecutor runner = new SampleExecutor();

            RunBenchmark(new DoWork (runner.DoSample1), "");

        }

 

        private void RunSample2()

        {

            Output.WriteLine("Running Sample 2");

            SampleExecutor runner = new SampleExecutor();

            RunBenchmark(new DoWork(runner.DoSample2), "");

        }

 

    }

The Executor Class

public class SampleExecutor

    {

        public void DoSample1(string text)

        {

            //do something

        }

 

        public void DoSample2(string text)

        {

            //do something

        }

    }

 

Performing Long Running Operation with Asynchronous Page in ASP.NET 2.0

fire

There are few cases in current web applications, where it involves long running operations. For instance, a system admin uploads 1 gig of data in the system, which will be processed by the system for 30-40 minutes to be splitted and putted properly in the database system, as well as notifying other system user regarding the availability if data.

In the traditional approach of web application, the user has to wait for the whole time until the whole process needs to be completed. The modern age of web application includes a technique where the user doesn't required to be wait to complete the whole process with the current page, where as they can move on other page (or even can close the browser) or if they resides on that page the system will show a simple progress bar , which stops when the data processing (or the long running operation) completes. The corresponding technique that implies this usability feature is using an asynchronous model.

Before the asp.net 2.0 ages, creating an asynchronous model was very complex and time consuming along with several over-heads. ASP.NET 2.0 comes with a new nice feature, where we can declare and use a page easily in asynchronous style. Besides the easy implementation model, ASP.NET asynchronous page utilizes ASP.NET threads efficiently. While calling a long running database query or web service, at the moment the request reaches at the asynchronous point, the corresponding asynchronous operation starts without keeping the current ASP.NET allocated, and thus lets the thread to be returned back to the thread pool. As soon as the asynchronous process completes, another ASP.NET thread get allocated and continues accordingly. This model really improves the utilization of ASP.NET thread pool and thus improves the scalability of the corresponding web applications. More can be found here.

To make a web page asynchronous, we just have to put a property in the page directive as follows:

<%@ Page Async="true" ...

There are two different implementation model for ASP.NET 2.0 asynchronous pages, with respect to database and web-service call.

Implementation model for database

The implementation model for database query requires a "PageAsyncTask" class to be set with few event handler methods such as "Begin Event Handler", "End Event Handler", "Timeout Event Handler" etc. This task should be registered as an Asynchronous tasks to continue and complete the asynchronous scenario properly.

public partial class AsyncPageTask : System.Web.UI.Page

{

    private SqlConnection _connection;

    private SqlCommand _command;

    private SqlDataReader _reader;

 

    IAsyncResult BeginAsyncOperation(object sender, EventArgs e, AsyncCallback cb, object state)

    {

        Response.Write("Starts at: " + DateTime.Now.ToString());

 

        _command = DAL.GetCommandForInsertRecord();

        _connection = _command.Connection;

        return _command.BeginExecuteReader(cb, state);

    }

 

    void EndAsyncOperation(IAsyncResult ar)

    {

        _reader = _command.EndExecuteReader(ar);

        Response.Write(" Ends at: " + DateTime.Now.ToString());

    }

 

    void TimeoutAsyncOperation(IAsyncResult ar)

    {

 

    }

 

    public override void Dispose()

    {

        if (_connection != null)

            _connection.Close();

        base.Dispose();

    }

 

    protected void btnClick_Click(object sender, EventArgs e)

    {

        Response.Write("Hello ");

        PageAsyncTask task = new PageAsyncTask(

            new BeginEventHandler(BeginAsyncOperation),

            new EndEventHandler(EndAsyncOperation),

            new EndEventHandler(TimeoutAsyncOperation),

            null

        );

        RegisterAsyncTask(task);

    }

 

}

Implementation model for web service

The implementation model for web service call is much more simpler than the database query. Ii just includes creating an web service instance, followed by calling the MethodName"Async" (for instance, if the web method name is "HelloWorld" then its asynchronous method name is "HelloWorldAsync". Besides the calling the asynchronous method for a web service method, we can register it's complete event handler to perform some post completion tasks.

protected void btnClick_Click(object sender, EventArgs e)

    {

        Response.Write("Starts at: " + DateTime.Now.ToString());

        MyWebService.WebService MyWS = new MyWebService.WebService();

        MyWS.HelloWorldCompleted += new MyWebService.HelloWorldCompletedEventHandler(MyWS_HelloWorldCompleted);

        MyWS.HelloWorldAsync();

    }

 

    void MyWS_HelloWorldCompleted(object sender, MyWebService.HelloWorldCompletedEventArgs e)

    {

        Response.Write(" Ends at: " + DateTime.Now.ToString());

    }

Performance Benchmark on Database Column and Row Expansion

speed

What performs better – Expanding DB column or expanding db row? My benchmark test shows expanding DB column performs better, on the following machine configuration:

Processor: Intel® Pentium® 4; 2.66 GHz;

Memory: 960 RAM.

Operating System: Windows XP, Version 2002, Service Pack 2.

Expanding Columns

A table having 4 columns with nchar(10) data type with 20,112 rows, query executes:

SQL Server Execution Times: CPU time = 47 ms, elapsed time = 125 ms.

Scan count 1, logical reads 232, physical reads 0, read-ahead reads 0.

Expanding Rows

A table having 1 column with nchar(10) data type with 80,017 rows, query executes:

SQL Server Execution Times: CPU time = 63 ms, elapsed time = 253 ms.

Scan count 1, logical reads 301, physical reads 0, read-ahead reads 0.

Efficient Batch Operation From ASP.NET 2.0 Using SQL Server 2005 XML Data Type

One of the good practice while developing high performance web application is to reduce the database roundtrip as much as possible. In the current trend of web applications, its a very common to enable users to delete multiple items in a list user interface at a time.

list

To perform this type of batch delete operation, one common mistake that happens by beginner developers are to calling the data access method repetitively for each item with in a loop as below:

For Each row As GridViewRow In gvEmployees.Rows
            ' Access the CheckBox
            Dim cb As CheckBox = CType(row.FindControl("chkEmployeeSelector"), CheckBox)
            If cb IsNot Nothing AndAlso cb.Checked Then
                ' First, get the EmployeeID for the selected row
                Dim employeeId As SqlInt32 = CType(gvEmployees.DataKeys(row.RowIndex).Value, SqlInt32)

                ' Deleting a employee...
                DAL.Employee.Delete.(employeeId)

            End If
Next

A good approach regarding this issue is to send all of the employee id's at a time to the database stored procedure, so that all the deletion operation can be handled at a time, and thus reducing the cost with respect to database roundtrip! BUT one basic problem with this approach is SQL Server doesn't support array, and thus we can't pass the list of ID's as an array! Then?

Oh yes! We have XML data type available is SQL Server 2005, well we can utilize that concept easily. How? Easy! Just to create the list as an xml and then pass to the corresponding stored procedure. The stored procedure will extract the xml accordingly and perform required operation individually on each item.

Application End:

The method below converts a .net generic list to a xml representation of list of Id's which is passed to stored procedure to perform required batch operation: 

public static string FormatXMLForIdArray(System.Collections.Generic.List<SqlInt32> idsToList)
        {
            //converting the list to xml first
            StringBuilder xmlString = new StringBuilder();
            for (int i = 0; i < idsToList.Count; i++)
            {
                xmlString.AppendFormat("<Id>{0}</Id>", idsToList [ i ] );
            }

            return xmlString.ToString();
        }

Stored Procedure End:

ALTER PROCEDURE spr_Employee_DeleteEmployeeList
@EmployeeSystemUserIdList xml
AS

DELETE App_Employee
WHERE EmployeeSystemUserId in
(
    SELECT ParamValues.EmployeeSystemUserId.value('.','Int') as Id
    FROM @EmployeeSystemUserIdList.nodes('/EmployeeSystemUserId') as ParamValues(EmployeeSystemUserId)
)

 

We have reached 100: ASP.NET Developers, Bangladesh

image_thumb2

We have reached 100 members recently in our user group ASP.NET Developers, Bangladesh. Its a good news really. Having our own user group help us to move forward with respect to the current technology trend in faster and professional manner. Hope we will grow faster. Cheers to all group members!

5 Things About Me

Thanks to Ferdous, as he has tagged me! Here are 5 things about me.

1. Although I am writing computer programs now by profession, once upon a time, I wrote literature and news in local newspapers. My parent really didn't like it, as being a "Bengali writer" will not give me a bright future. Well, I got admitted to computer science under-grad course, where I first leaned how to use computers, by the year 2000! Primarily it was a bit confusing area for me, but very soon programming became a great choice for me, found that it is not less interesting than writing literature, lolz. Any way’s, still I have the same passion on writing literature, since it creates a wonderful world in my mind! Hoping on the coming days, I will get time to concentrate on that as well as, by going parallel with my program writing.

2. Besides programming and literature, my other favorite subjects are philosophy, psychology and space science. I believe, everybody should have own philosophy, regarding his own integrity, life style and surroundings. In my life, the best philosophy is Socrates' "Know thyself". On the other hand, Psychology is really an interesting area, where we can know how many varieties can reside in different peoples, even in one person! Space science? Huh! When I learn something related to it, I find how tiny we are and get lost in the endless universe. It's really a strange feeling.

I don't feel any interest about sports most of the time. Eventually I am not good at sports at all :D

3. Honesty and respect are two basic thumbs in relation for me. Personally, I try to be that best to my friends, relatives. I have very few friends at all, but when I find lack of honesty and respect in my friends, it really hearts me. I have suffered a lot in past for that. Any ways, recently I find that, rather continuing an un-happy relation, it is better to say "good bye". As a person, I am very emotional and my only expectation to my nearest persons is, to understand and respect my emotion.

My wife is Leuza. We both love each other truly and really good friend of each other.

4. Two things which change my mood better always, are music and perfume. I believe these are the gifts of god. A wonderful music or nice perfume gives me a soft touch of heaven!

5. Traveling is another favorite thing of mine, which can be thru air-plane, car, risk-shaw (a three wheeler human driven tri-cycle, which is very common in my country, Bangladesh), even though thru walking. But among those, my best ride choices are risk-shaw and rocket! Regarding place, natural beauty is my first choice however!

While traveling alone, my thoughts get spread on sky, and it's really helpful for me to think deeply to solve complex problems.

Here are my favorite tagged blogs:

Nikhil Omar Shahed Jeff Scott
A Quick View to the Capabilities Which Can Make You a Great Developer

keyboardsmall 

Regarding my previous experience in software development team, as developer and project manager, I’ve found some capabilities that are required to be a good developer to put a valuable effort in the development team. As mainly I have worked with web based database driven applications, these capabilities are closely or loosely relevant to that context.

Of course you can consider some other capabilities regarding the software development context in broader sense, but here I consider the very basic things, that came in my mind on the fly.

  • Punctuality: This is a basic virtue that helps project managers to meet the estimated timeline in a more specific and sharp way.
  • Discipline: This is a basic virtue that is required to maintain a developers work efforts effectively, that includes obeying team/organization hierarchy, along with other rules and regulations
  • Team Player: This is focused on how a developer shares his knowledge with his following/sibling developers/team members
  • Planning: Involves a developer capability that how s/he is planning his/her assigned tasks to be completed on time, along with the integration with other modules or code segment
  • Problem Solving: This capability is a major factor to solve, code, design complex business/computational programming logics.
  • Coding Quality (.NET): Includes the coding quality, as putting proper comment, following coding standard, way to utilize important API methods (such as DB, UI etc), using powerful language features etc. Also the capability to write high quality code that is manageable, understandable, along with utilizing best practices with available built-in and external programming resources.
  • Coding Quality (Java-script): Includes the capability to write codes that creates high-end user end experience.
  • Coding Quality (Robustness): Includes the capability to write error/bug free codes
  • Learning Capability: Capability to learn required technology quickly
  • Eagerness to Learn: Although learning capability is one of the main virtues to be able to be productive based on application requirements, but eagerness to learn is another factor that plays an important issue, besides the capability to learn.
  • Communication (Written): Capability to document code, communicating with software development stake holders via IM, mail.
  • Communication (Verbal): Capability to understand and explain problem domain, communication with team members and project manager to explain accomplished works/codes.
  • Technical Concept (OOP): Includes a clear understanding of Object Oriented Programming and Technology (encapsulation, polymorphism, inheritance, design patterns etc), along with the ability to utilize them in proper context.
  • Technical Concept (Database): Includes the capability to normalize, de-normalize data relations in most optimized but requirement specific way.
  • Technical Concept (Web): Includes a clear understanding to web based technology along with the best practices and standards (html, xhtml, web 2.0, div, SEO friendly code, CSS, browser independence etc)
  • Technical Concept (Software Engineering Best Practices): A solid understanding of Software Development Life Cycle, Unified Process etc)
Brainstorm with Marbles!

When I was the 2nd year student in my computer science under-grad course, solving logical and computational problem was my favorite hobby. Some of those were directly related to programming and some were just brainstorming. When not engaged with study, I passed my time either using pen and pages to draw diagrams or just seen in the sky whether I can find any solutions there!

12 Marbles Problem

Among all of the problems that I found and solved, the most interesting problem to me was "12 marbles problem". One of my senior course mate gave it to me. When I first hear it, it didn't seemed much complex to me. He smiled to me to see it, and told, "well lets solve it first!". Ah, very soon I got understood why he smiled at me! The problem started to take my hours after hours, days after days! I really forgot my daily life, by this time, couldn't study well, couldn't eat, couldn't sleep...12 marbles were moving and playing around my world! After 3-4 days, I got really embarrassed, and thought there might be no solution for it, so I shouldn't try to solve it more. Or at least I should drop it for few days, coz I have lots of other stuffs to do! Sometimes, even a simple problem may turn to too complex, by continuously thinking of it. Leaving it for few time will create some new views to solve the  problem. Can't remember how many days I left the problem, but after few days again, I started to think about it, with new energy! Ah, it was a cool feelings for me when I got the solution! And couldn't believe first, I have solved it! Also got lost soon, as the solution itself is not that much simple.

Finally I grabbed the total solution, by more careful and patient thought! And felt that, this is the only solution for it that I just found! It was a matter of great joy for me, as so far I learned only a few number of person have solved it! But the interesting thing was, when I have heard their solution, I found those are separate to each others and even separate from my solution! Well, lets see the problem and see whether you can find your own solution !

You have 12 marbles. All the marbles have the same look and same weight, but there is a defective one, which weight is a bit more or less than others. Using a balance just three times, you have to find the defective marble!

400 Marbles Problem !

Once I solved it, I was given another problem :P By this time I have 400 marbles and I can use a weight machine  only one time! Huh! Kidding? 400 marbles and can take weight only once? I didn't want to spent hundreds of "MAD" days for just to solve one problem, since 12 marbles took my few days to solve! But once I surrendered, when I heard the solution, I really wanted to kick at my ass! Ah, how easy the solution is! Basically although it was about 400 marbles, but it is much easier than before. So found a moral on problem solution: "Sometimes solving complex problems made our thoughts complex. You can find a solution easily, just by understanding the problem clearly, and then find your best way to solve it!" 

Any way, lets see the interesting problem, and check which one you find easier!

You have 20 boxes, each of the boxes has 20 marbles, each of the marbles is weighted 20 gram. There is only one box which has 20 marbles, but each of which marbles has 1 gram less or more than 20 gram. You have a weight machine, which can be used only once, to find which box contains less or more valued box.