/bill's House O Insomnia : Coding Techniques

Looking for some .NET Developers

William — Sat, 07 Jun 2008 02:19:26 GMT

A client of mine located in the Aiken/Columbia South Carolina Area has asked me if I had any friends who are .NET Developers and looking for a job. They are looking for at least one Junior Developer, on Mid-Level Developer and several Senior developers. I know these folks quite well and I can vouch for the following:

They pay extremely well
Generous benefits package
There's plenty of cool people on their staff
They are growing at an explosive rate so there's plenty of room for upward mobility. There's also a very strong focus on learning/teaching and career development.
They are an Agile/SCRUM shop
Currently developing with the following technologies:

Visual Studio Team System 2008 / .NET 3.5 Framework
Windows Communication Foundation
Windows Presentation Foundation
ASP.NET
LINQ
Silverlight
Windows Mobile 5/ Windows Mobile 6/ .NET Compact Framework
Sql Server 2005 / Sql Server 2008
Windows Installer for XML
Sandcastle

If you're in the area and looking for a job... if you know someone who looking for a job.... or if you're willing to move, please drop me a line using the Contact option on my blog or emailing me directly at WilliamRyan@gmail.com and I'll put you in contact with them. They are looking to hire folks immediately so I can probably get you an interview within a few days.

Junior Developer:

Ideal candidate will be a recent college graduate with a Computer Science of MIS degree. No experience is necessary for this position. The main soft-skill qualifications for this position are:

A true love for development
Strong desire to learn
A desire to stand out among one's peers

The hard-skill qualifications include a good theoretical understanding of at least two of the following:

Object Oriented Design and Analysis
Relational Database theory
Design Patterns
Familiarity with at least two of the following: C, C++, Java, C#, VB.NET

This is essentially a dream job for recent college grads b/c the job is being offered "no experience required". Since they are a SCRUM/Agile shop, you'll get to work extensively with their Senior and Mid level devs via Pair Programming. You'll be able to pick an area to specialize in which you will decide on for yourself. Additionally, you'll be using some of the most sought after Microsoft technologies available. Go to Monster.com or Dice.com and see for yourself how much demand there is for the above technologies and how much those jobs pay. You'll get to work with these on day 1 and essentially carve out a path for yourself using whichever technologies you like the best. If you want to be a "UI" expert, that's where you'll be put. If you want to be a Web Services dev, again, that's what you can focus on. If you want to be an expert buildmaster who specializes in installers, that's what you'll get to do. The company is looking for some excellent candidates and willing to offer a really fun and great paying job with plenty of training and career development available.

Mid-Level Developer:

The ideal candidate will have a computer science or MIS degree and 1-3 years of development experience. The ideal candidate will have two or more the following abilities:

Solid command of OOP/OOD using C++, C#, Java or VB.NET
Test Driven Design
Service Oriented Architecture
Familiarity with MySql, Sql Server (T-SQL), Oracle (PL/SQL)
Basic understanding of both Winforms and ASP.NET
Basic familiarity with Web Services, MSMQ, Enterprise Services/COM+, .NET Remoting, DCOM, J2EE, Java Beans

Just like the Junior development positions, these ones will provide a great deal of flexibility to learn and specialize the area of your choice. These are considered 'fast track' positions to move into the Senior Developer or Architect roles

Senior Developer:

The ideal candidate will be someone with at least 4 years experience. This person will love challenges and love coming up with solutions to problems that other people run from. They will be people that love figuring out how to do things that everyone else said can't be done. They will be the stereotypical Alpha-Geek, Someone that spends their spare time answering quiz questions like these:

All of the requirements for the Mid-Level Developer as well as several of the following skills:

Proficiency with T-Sql or PL/SQL. You should be able to write a 5 table join on your own. You should have written several stored procedures that were > 100 lines long.
Strong command of query tuning and ability to easily spot design flaws
Ability to design 'real' objects.

Someone who pays meticulous attention to the scope modifiers of their classes/functions/properties
Someone who understands the problems associated with coupling and strives to avoid it.
Someone who can write and use events in their classes without thinking twice about it.
Someone with a solid understanding of thread safety and thread synchronization
Someone who is comfortable using several different design patterns
Someone who is comfortable implementing several different interfaces in one object, using combinations of interface implementation and inheritance and someone who can use inheritance responsibly

Keeps up with technology, reads computer books regularly, has favorite blogs in multiple disciplines and/or blogs themselves, attends or speaks at user's groups, is active in one or more online forum or newsgroup
Someone who answers "Which is better, C# or VB.NET?" with "Either can be better depending on the circumstances but I'm comfortable writing in either of them" and then "But I've been learning F# lately and it's really been growing on me"
Experience with .NET Remoting, Web Services & Web Services Enhancements, MSMQ, Enterprise Services and/or WCF
Ability to create syntactically correct UML diagrams (meaning they can do more than write the letters "UML" on their resume) and if not, they are willing to learn to do this.
Understands data structures solidly and could easily implement their own HashTable or LinkedList
Someone who's always looking to come up with a newer, cooler, faster, and more secure way to do implement something
Experienced with either Biztalk server or Sharepoint
Takes pride in their code without falling in love with it or getting all defensive about it. Ability to acknowledge the fact that everyone makes mistakes and can talk about their mistakes/bugs without getting defensive.
Someone who is the first to raise their hand whenever the boss says "I need someone to learn ____________, do I have any volunteers"

Basically, if you love development, love creating new cutting edge solutions all the while making a lot of money and having a lot of fun you definitely should talk to these folks. If you're interested...... Please drop me a line using the Contact option on my blog or emailing me directly at WilliamRyan@gmail.com and I'll put you in contact with them.

Filtering data ADO.NET Synchronization Services

William — Wed, 14 Nov 2007 02:20:57 GMT

If you dabble in Orcas or are a data aficionado, then you have probably at least heard of Synchronization Services. If you haven't heard of it, let me try to pique your interest.

The business case for Sync Services is simple. You have data in one place and you want to sync it with Data in another place. This is an old school problem that has been dealt with many times before by many different people. That should clue you in to the realization that such problems should probably be approached using design patterns and Data Oriented Design Patterns in particular. My memory is a little foggy but I don't specifically remember reading any 'data movement patterns' in the seminal design pattern book, Design Patterns: Elements of Reusable Object-Oriented Software by the Gang of Four. (As a slight digression, I've found quite a few superb design pattern books and a few stinkers. The GOF book is by far the best one I've come across in terms of being understandable and easy to follow and can say with comfort that it won't disappoint).

So you can roll your own code, you can use an existing mechanism like Replication, or you can go new school and try SS. I don't claim to be an expert at SS but so far, I've found it shocking easily to use and learn. Of all the zillions of new MS products rolling out of Microsoft, SS is one of the easiest I've come across, at least in terms of getting up and running.

Now, chances are that if you have the need to sync data back and forth (for instance, moving data from a database server to your salesforce's Tablet PC's), you will have a lot more data on your server than you'd need or be able to use on the remote machine. Even if you wanted all of that data, in many cases you won't be able to get it all b/c moving it would take forever and the target machine is resource constrained. To that end, the out of the box demo code that ships with SS will need some modification.

The first point I'd make is that Yes, it is possible to sync only portions of the data. This came up recently in one of the newsgroups I frequent and it's an excellent question. Just to get a conceptual feel for this, think of the same scenario I mentioned earlier. You have a db server and a mobile salesforce (or nursing force, or delivery force, or you're a large law firm, consulting group, accounting practice...you get the idea) and you only need to sync each target machine with a subset of data. In an ideal world remote users could see everything that users at the home base could, but moving the data could take forever in many large companies. Moreover, you probably wouldn't need most of it. If you're a nurse that makes house calls, you probably wouldn't need information for people that have passed on, or that have moved or even that you're not visiting. If your device was a PDA, you need to economize on every resource possible. But even if you had a laptop or tablet you still have some resource limits. Hence, envision a case where you might just want to sync only the data related to each person's active clients (and probably augment the query's restriction with a date filter of some sort and perhaps an active account indicator) that have engaged in at least one transaction in the last year.

You could give each person some sort of UI tool which they could initiate that queries the db and writes the data to a file, which they in turn copy over to their machine.
You could do the same thing, but run it as a job so it's done automatically
You could expose the query results as a web service and let them retrieve data as they needed it
You could export the data to Excel or Access (the latter of which I've seen done more times than I care to remember)

Each of these approaches is easy enough to implement but has major suckiness associated with it. The first approach is one of the simpler ones and fairly fool-proof. Writing out the file is something you could automate via a schedule but not copying it to the device, unless of course you were positive the device would always be available when the job ran. With the web service, you're depending on internet connectivity which, even in this day where everyone has mobile broadband, there are still a lot of coverage holes. If you had any sort of substantial dataset, pulling it down over a mobile broadband connection would be a doubleplusbad. Copying it to Access or Excel would have the same issues and shortcomings that the first approach, after all it really is the same approach but simply uses a different file format.

Referring to the SyncServices_CSharp_DownloadOnly sample included in the Sync Services install, refer to the SampleServerSyncProvider method of the SampleServerSyncProvider.cs or SampleServerSyncProvider.vb file, you see the following code (the Author was kind enough to include copious comments which explain each portion - I've removed them for readability but would encourage you to refer to them if you're unfamiliar with Sync Service):

public class SampleServerSyncProvider : DbServerSyncProvider
{

     public SampleServerSyncProvider()
      {

        this.Connection = new SqlConnection(Properties.Settings.Default.ServerConnString);

string awNewAnchorVariable = "@" + SyncSession.SyncNewReceivedAnchor;
SqlCommand selectNewAnchorCommand = new SqlCommand();
selectNewAnchorCommand.CommandText = "SELECT " + awNewAnchorVariable + " = @@DBTS";
selectNewAnchorCommand.Parameters.Add(awNewAnchorVariable, SqlDbType.Timestamp);
selectNewAnchorCommand.Parameters[awNewAnchorVariable].Size = 8;
selectNewAnchorCommand.Parameters[awNewAnchorVariable].Direction = ParameterDirection.Output;
selectNewAnchorCommand.Connection = (SqlConnection)this.Connection;
this.SelectNewAnchorCommand = selectNewAnchorCommand;

SqlSyncAdapterBuilder builder = new SqlSyncAdapterBuilder((SqlConnection)this.Connection);

builder.TableName = "Sales.Customer";
builder.TombstoneTableName = builder.TableName + "_Tombstone";
builder.TombstoneFilterClause = filterCommand;

builder.DataColumns.Add("CustomerId");
builder.DataColumns.Add("CustomerName");
builder.DataColumns.Add("SalesPerson");

builder.CreationTrackingColumn = "InsertTimestamp";
builder.UpdateTrackingColumn = "UpdateTimestamp";
builder.DeletionTrackingColumn = "DeleteTimestamp";

SyncAdapter customerSyncAdapter = builder.ToSyncAdapter();
customerSyncAdapter.TableName = "Customer";
this.SyncAdapters.Add(customerSyncAdapter);
}

}

[Note: Even with the comments, some of this code may be hard to follow when you first read it. One thing I did which was very enlightening and always a useful thing to do in such situations is fire up good ole' Sql Profiler, then run the application and see exactly what happens, what objects are being used and what commands are being sent to the db].

I show some of the relevant results from the trace below (I've color coded the results to the portion of the code that which caused it to happen):

declare @p3 binary(8)
set @p3=0x0000000000000FA0
exec sp_executesql N'SELECT @sync_new_received_anchor = @@DBTS',N'@sync_new_received_anchor timestamp
output',@sync_new_received_anchor=@p3 output
select @p3

exec sp_executesql N'SELECT [CustomerId], [CustomerName], [SalesPerson] FROM Sales.Customer WHERE
(InsertTimestamp > @sync_last_received_anchor AND InsertTimestamp <=
@sync_new_received_anchor)',N'@sync_last_received_anchor binary(8000),@sync_new_received_anchor
binary(8)',@sync_last_received_anchor=0x00,@sync_new_received_anchor=0x0000000000000FA0

exec sp_executesql N'SELECT [CustomerId], [CustomerName], [SalesPerson] FROM Sales.Customer WHERE
(CustomerName=''Sharp Bikes'') AND (UpdateTimestamp > @sync_last_received_anchor AND UpdateTimestamp <=
@sync_new_received_anchor AND InsertTimestamp <= @sync_last_received_anchor)',N'@sync_last_received_anchor
timestamp,@sync_new_received_anchor
timestamp',@sync_last_received_anchor=0x0000000000000000,@sync_new_received_anchor=0x0000000000000FA0

exec sp_executesql N'SELECT [CustomerId], [CustomerName], [SalesPerson], [CustomerType], [DeleteId],
[DeleteTimestamp] FROM Sales.Customer_Tombstone WHERE (CustomerName=''Sharp Bikes'') AND (@sync_initialized = 1
AND DeleteTimestamp > @sync_last_received_anchor AND DeleteTimestamp <=
@sync_new_received_anchor)',N'@sync_initialized bit,@sync_last_received_anchor
timestamp,@sync_new_received_anchor
timestamp',@sync_initialized=0,@sync_last_received_anchor=NULL,@sync_new_received_anchor=0x0000000000000FA0

As it stands, the code above is pulling everything from the Sales.Customer table. But as I mentioned, chances are that you wouldn't want the whole resultset. Rather, you'd probably want a filtered subset. For the sake of visual simplicity, I'm going to violate one of my biggest coding Pet Peeves and concatenate a sql statement to include a parameter instead of doing it the right way, but I'll show the correct way shortly.

The SqlSyncAdapterBuilder has several properties and for filtering purposes, you can simply set the FilterClause property. Starting with the code beginning at the adapter instantiation, I modified the code to just return records that have a CustomerName value equal to the literal 'Sharp Bikes' (The relevant portions are included in Bold face):

SqlSyncAdapterBuilder builder = new SqlSyncAdapterBuilder((SqlConnection)this.Connection);

String FilterCommand = "CustomerName='Sharp Bikes'";

builder.TableName = "Sales.Customer";
builder.TombstoneTableName = builder.TableName + "_Tombstone";
builder.FilterClause = FilterCommand ;
builder.TombstoneFilterClause = FilterCommand;

builder.SyncDirection = SyncDirection.DownloadOnly;

So, it's pretty much this easy, you simply need to set the FilterClause property of the SqlSyncAdapterBuilder and absent some really compelling reason to do so, set the TombStoneFilterClause to the same value. Because you use the same statement, it's best to use a String literal so that the two FilterClause values don't get out of sync. Just as you set the FilterClause and TombstoneFilterClause properties to facilitate filtering, if you use a parameter, you need to add a SqlParameter (or SqlParameter(s)) to the FilterParameters and TombstoneFilterParameters collections. So, the below code shows the previous snippet modified to use a SqlParameter instead of concatenating the Sql text, again, the relevant portions are included in Bold face:

SqlSyncAdapterBuilder builder = new SqlSyncAdapterBuilder((SqlConnection)this.Connection);
SqlParameter CustomerNameParameter = new SqlParameter("@CustomerName", SqlDbType.NVarChar);

String FilterCommand = "CustomerName=@CustomerName";

builder.TableName = "Sales.Customer";
builder.TombstoneTableName = builder.TableName + "_Tombstone";
builder.FilterClause = FilterCommand ;
builder.TombstoneFilterClause = FilterCommand;
builder.FilterParameters.Add(CustomerNameParameter);
builder.TombstoneFilterParameters.Add(CustomerNameParameter);

Now the results from Profiler show something notably different:

exec sp_executesql N'SELECT [CustomerId], [CustomerName], [SalesPerson] FROM Sales.Customer WHERE
(CustomerName=@CustomerName) AND (InsertTimestamp > @sync_last_received_anchor AND InsertTimestamp <=
@sync_new_received_anchor)',N'@CustomerName nvarchar(11),@sync_last_received_anchor
binary(8000),@sync_new_received_anchor binary(8)',@CustomerName=N'Sharp
Bikes',@sync_last_received_anchor=0x00,@sync_new_received_anchor=0x0000000000000FA0

exec sp_executesql N'SELECT [CustomerId], [CustomerName], [SalesPerson] FROM Sales.Customer WHERE
(CustomerName=@CustomerName) AND (UpdateTimestamp > @sync_last_received_anchor AND UpdateTimestamp <=
@sync_new_received_anchor AND InsertTimestamp <= @sync_last_received_anchor)',N'@CustomerName
nvarchar(11),@sync_last_received_anchor timestamp,@sync_new_received_anchor timestamp',@CustomerName=N'Sharp
Bikes',@sync_last_received_anchor=0x0000000000000000,@sync_new_received_anchor=0x0000000000000FA0

exec sp_executesql N'SELECT [CustomerId], [CustomerName], [SalesPerson], [CustomerType], [DeleteId],
[DeleteTimestamp] FROM Sales.Customer_Tombstone WHERE (CustomerName=@CustomerName) AND (@sync_initialized = 1
AND DeleteTimestamp > @sync_last_received_anchor AND DeleteTimestamp <=
@sync_new_received_anchor)',N'@CustomerName nvarchar(11),@sync_initialized bit,@sync_last_received_anchor
timestamp,@sync_new_received_anchor timestamp',@CustomerName=N'Sharp
Bikes',@sync_initialized=0,@sync_last_received_anchor=NULL,@sync_new_received_anchor=0x0000000000000FA0

You can see here clearly that not only is the @CustomerName parameter being used, it's being set to the literal Sharp Bikes

There's one catch though when using real parameters. If you modify the code as I did above and try to run it, you'll encounter the following exception:

Microsoft.Synchronization.Data.SessionVariableException was unhandled
Message="Unable to set session parameters in DbServerSyncProvider. Cannot obtain the value for command parameter '@CustomerName'."
Source="Microsoft.Synchronization.Data.Server"
SyncSource="ServerSyncProvider"
StackTrace:
       at Microsoft.Synchronization.Data.Server.DbServerSyncProvider.SetSessionParameters(IDbCommand cmd, SyncGroupMetadata groupMetadata, SyncTableMetadata tableMetadata, SyncSession syncSession, DataColumnCollection columns, SyncStage stage)
       at Microsoft.Synchronization.Data.Server.DbServerSyncProvider.EnumerateChanges(SyncGroupMetadata groupMetadata, SyncSession syncSession, IDbTransaction transaction, EnumerateChangeType changeType)
       at Microsoft.Synchronization.Data.Server.DbServerSyncProvider.GetChanges(SyncGroupMetadata groupMetadata, SyncSession syncSession)
       at Microsoft.Synchronization.SyncAgent.DownloadChanges(SyncGroupMetadata groupMetadata)
       at Microsoft.Synchronization.SyncAgent.Synchronize()
       at Microsoft.Samples.Synchronization.MainForm.btnSynchronize_Click(Object sender, EventArgs e) in C:\SyncServicesCode\SyncServices_CSharp_DownloadOnly\Client\Forms\MainForm.cs:...

Nothing too shocking though right, after all, although we specified a parameter be used in the CommandText property, we never set the value so it's not surprising that an exception was thrown unless the parameter had a default value, which in this case it clearly did not.

But if you change the code to explicitly set the value for the parameter, for instance like this...

SqlParameter CustomerNameParameter = new SqlParameter("@CustomerName", SqlDbType.NVarChar);
CustomerNameParameter.Value = "Sharp Bikes";

you'll still get the same exception. This really threw me for a loop b/c I used the debugger and proved to myself that the value wasn't null or left out. If you step through with the debugger, you can verify for yourself that the above code is in fact setting the value of @CustomerName = "Sharp Bikes"

One point that's extremely important... Normally, as long as you modify your CommandText to refer to the SqlParameter and add a SqlParameter object to a SqlCommand object's Parameters collection, you don't need to do anything else. But as I just babbled on about, that won't cut it here... something is still missing.

As it turns out, (and makes perfect sense once I cleared my tunnel vision), the SyncAgent (Microsoft.Synchronization.SyncAgent) class is what actually does much of the heavy lifting and just because you have a Parameter referenced in code in your SqlSyncAdapterBuilder, the SyncAgent doesn't know anything about it. Since it doesn't know anything about it, it doesn't know to pass in a value or what value to pass to it, so when it tries to finish processing the value for the parameter isn't set. It didn't matter that we set it explicitly because we set it in the wrong place.

Fortunately, it's very easy to address. The code below is the SyncAgent's code, the last line in Bold is what you need to add if you're using a parameterized filter:

public class SampleSyncAgent : Microsoft.Synchronization.SyncAgent
{

     public SampleSyncAgent()
     {

         SqlCeClientSyncProvider clientSyncProvider = new SqlCeClientSyncProvider(Properties.Settings.Default.ClientConnString, true);
         this.LocalProvider = clientSyncProvider;
              clientSyncProvider.ChangesApplied += new EventHandler<ChangesAppliedEventArgs>(clientSyncProvider_ChangesApplied);

this.RemoteProvider = new SampleServerSyncProvider();

         SyncTable customerSyncTable = new SyncTable("Customer");
         customerSyncTable.CreationOption = TableCreationOption.DropExistingOrCreateNewTable;
         customerSyncTable.SyncDirection = SyncDirection.DownloadOnly;

         this.Configuration.SyncTables.Add(customerSyncTable);
         this.Configuration.SyncParameters.Add(
           new SyncParameter("@CustomerName", "Sharp Bikes"));
     }

}

(Part of this class was left out for brevity - all that is missing is the private variables and their corresponding public properties).

The corresponding code is available Here. Please note that this code is taken from the Synchronization Services samples that are included when you install Synchronization Services. I merely modified that code to support parameterized filters. I will be adding a UI Element on the MainForm object so that you can set the value dynamically instead of hard coding it as I did. Again, although this post is a little long, we really are talking about basically adding less than 10 lines of code to add filtering. You need to set the FilterClause, the TombstoneFilterClause, FilterParameters, TombstoneFilterParameters and then add a SyncParameter to the SyncAgent. As you can infer, you if you need more expressions, you can just add them accordingly to meet whatever your requirements happen to be.

Free Silverlight Training

William — Tue, 14 Aug 2007 19:53:43 GMT

My friend Brian Madsen who owns CSharpZealot has teamed up with InnerWorkings to extend a really slick offer for Free Silverlight Training for CSharpZealot members.

In addition, Shawn Wildermuth has been way ahead of the crowd when it comes to Silverlight and has just created two more Silverlight Videos. His ADO Guy site has an entire Section dedicated to Silverlight and well, Shawn is gifted in many areas but when it comes to teaching, no one does it better.

On a similar note, my company recently sponsored an internal Silverlight contest where our different offices are competing to build the best game written in Silverlight. They put there money where their mouth is so the first place prize is pretty impressive. I've had my nose in the keyboard for the past few weeks trying to help my team win. If you're interested in learning Silverlight, one great place to get started is Here. Microsoft really did the right thing with Orcas (VS 2008) and even provides ready to run (almost anyway) VPC images.

Hopefully when I get some time I can start doing some more in depth Silverlight blogging but let me make a few recommendations.

Use the VPC images for Orcas. The reasons are the same for any other alpha or beta product you'd want to install but outside of isolation and reducing risk, the time factor is huge. Using the VPC images, you can be up and running in under 5 minutes once you have them downloaded. Benefits like this are really hard to quantify but I can tell you personally that at times, getting my environment set up without VPC images was frustrating to say the least.
Although it's often prudent to learn to code things without all the crutches that a powerful IDE provide, creating Silverlight with Intellisense and a designer is probably not a very effective way to start things. Unless you are really familiar with WPF already, the syntax takes a little getting used to and there's a lot of room to mess things up with typos.
In addition to Orcas, make sure you pull down Expression Blend and Design. Blend allows you to create Graphics with XAML. I've already had a few Paint.NET and Photoshop fanatics get all over me for plugging Blend, but I think they are off base. I'm not advocating replacing Photoshop or Paint.NET with Blend. Blend however, allows one to create images with XAML. You in turn can look at the XAML that is generated and use it to help understand and learn XAML. If you did much Office automation, you more than likely 'cheated' in the sense that you just ran a macro and saw the code that was generated and used this as a mechanism to learn VBA and get past sticking points. Blend allows you to do the same thing. Sure, hardcore 'real' developers do everything in Notepad but if you want to get up to speed, using a tool to help you draw the effects you want and then being able to disassemble the XAML that was generated will get you where you want to be a LOT faster.
Pull down each of the items in the Gallery and work through the QuickStarts. These encompass many common kick a55 effects and you can reverse engineer them to see learn how to do quite a bit.
I have some pretty strong opinions on the books out there but even if a book doesn't address Silverlight specifically (most of them however have updated versions which do include Silverlight), you can still learn quite a bit because you are ultimately going to need to really understand WPF and XAML. I really want to gather my thoughts a little better before making any more book recommendations because although all of the WPF books out there are pretty good, some are a lot more helpful than others when it comes to learning Silverlight
Read Shawn Wildermuth's Silverlight posts.

Personally, since I'm not a designer and very challenged when it comes to graphics, I had a pretty rough time at the beginning of my learning curve. Little by little though, I'm getting through it and hopefully I'll have some blogworthy material to post soon. As always, if you have any specific questions, please let me know and I'll be glad to answer what I can.

Theory matters after all...

William — Wed, 08 Aug 2007 03:53:00 GMT

There are many discussions developers have with each other that always make for interesting discussions. One of the best ones is "How much does a Computer Science degree matter?" If you argue that it's really important, you'll invariably have your argument countered with anecdotes of 'some idiot who was totally Ivorty Tower" and examples of people who were self taught and complete bad asses.

I definitely know many many examples of the latter. I'm pretty skeptical about the former which I'll get to in a minute. I don't consider myself a bad ass or guru in any sense, but I've had a pretty successful career and been able to excel in every job I've held. I'm kind of a hybrid though b/c I have taken 30 credit hours of CS even though most of what I know is self taught. However had I not had the education that I did, I know there's more than a few subjects I would have had a lot more trouble with. Outside of the formal education, there's a lot of vicarious learning thrust upon you while taking college courses and at least in my case, I'm pretty sure a good bit of my success is attributable directly to my education. I work/have worked with more than a few folks though who had little or no formal CS education and are far better developers than I am. Experience and learning are ultimately what matter and at the end of the day, if you learn to create effective classes for instance, it doesn't much matter if you learned to creat them in a class room or at work. Ultimately, the reason this always makes for such a good discussion is precisely because there are so many excellent examples to support either side of the argument.

One thing I hear a lot though are people that criticize people that 'think they are still in college', 'all theory' 'too much textbook' or 'ivory tower'. I have yet to meet or work with someone who's so well educated that they are poor at development. Like most things, I think good theory makes good practice. What I have come across however is posers who hide behind big words. In every single case where I've been told someone is 'all theory' and had the opportunity to work with this person, what I've found is that they aren't all theory at all. They are all 'use words and phrases they don't understand to compensate for their disturbing level of ignorance'. In most cases, these folks learned just enough to sound like they know what they are talking about (to others that aren't familiar with the theory they claim to espouse). A few notable examples come to mind, in most cases, I've run into and/or heard from friends about this more times than I care to remember:

Tonight though, I was flipping through Wired and came across an article which proves that without a firm command of theory, you're going to be in over your head in some cases. Even cooler than the article is the source code they posted with it . I'm not very familiar with the .jpg file format, but this article really got me thinking. I mean, I would never have even thought to look for such things. The code isn't really all that complex, but understanding what it's doing and why is something you aren't just going to figure out on your own. At least I know I wouldn't. Not surprising, Neal Krawetz the guy who performed this analysis has a Ph.D in Computer Science. Read the article, look at the source code, hit his web site - seems to me this is true Ivory Tower stuff and it's pretty darned impressive if you ask me.

Coding Arguments - aka How to Start Religious Wars

William — Mon, 23 Jul 2007 04:33:00 GMT

I am lucky enough to work for a company that really encourages us to get together and learn/hang out etc. One way this happens is by Tech Nights. Basically, we have in office meetings which are recorded via Live Meeting, where someone picks a topic and teaches about it. Afterward, folks go to dinner and the company picks up the tab. Two weeks ago I had done the presentation and we all went out afterward. Two of our new guys were with us, and the rest of the folks there had all worked together at a particular client. Both of our new guys are really sharp, can talk their share of smack when necessary and been here long enough to feel comfortable speaking their minds if they want to.

The first discussion was regarding Frameworks. One of us mentioned that he liked Frameworks and believes in them. Being the smart a55 that I am, I asked "What in the h3ll does that mean?" . He proceeded to explain that he believed in code reuse and has build reusable code at every opportunity. Trying to have a little fun and encourage discussion, I asked him "Who doesn't believe in code reuse? But if that's the only goal, then a function gets the job done. Above functions, the whole reason for OOP is to facilitate reuse and simply building a bunch of objects that can be used sounds a lot more like an API than a Framework". The guy in question is a top notch coder and certainly understands the difference, he just had a little trouble using 'official definitions'. But asking what differentiates a Framework from an API turned out to be a pretty decent question to ask b/c neither have absolutely conclusive definitions that make them mutually exclusive. As such, it's precisely the type of argument that a bunch of nerds can argue into the ground over dinner.

I mentioned before that the 'rest of us' all worked at a certain client together. Personally, I'm pretty fond of standardization when it comes to coding. So even if I think my way is better, I'll use the predominant methodology in most cases simply b/c I think that in general, any improvements that doing it 'my' way brings will be more than offset by complications resulting from differing coding styles. If nothing else, it will usually mean that every time someone looks at it, they are going to ask "Why is it like this instead of ...". It ends up causing you to explain it over and over and have pedantic discussions with each new person that joins the team. Similarly, if you differ from what's being done, it's a virtual certainty that the implication of it will be that 'the other way is wrong'. Most of the folks I work/worked with are humble enough to acknolwedge that any given way they chose to implement something could have been better (any *real* coder will always acknolwedge it could have been done better. Anyone that says it couldn't is either egotistical or clueless). However, just because people are ok with admitting they didn't chose an optimal design methodology, that doesn't mean it's cool to rub it in their face. After all, you'll be on that side of the equation unless you never have to design stuff. Not that I recommend doing things wrong just to save people's egos, but some times, it's too easy to get so carried away with saying "this is right" or "this is wrong' that you forget to remember "It needs done". Another problem with doing things differently is that opinions are like Carrolls, everyone has one. What seems obvious as day to one person may seem utterly silly to another. Design decisions are all about tradeoffs and since we all see the value of tradeoffs differently, well you know...

Since a few of us worked at the same client, I decided to provoke a little bit. We had worked there at different times, except for me - I was there throughout the time each of the other guys was there. So one of them had really strong opinions about how well code should be formatted. So on the one side we had the following:

Upper case for Keywords and reserved words
Commas at the beginning of the line
Each column on a different line
Lining up words in the center
UPDATE Whatever SET Whatever WHERE WHATEVER
Immediately getting rid of any objects that aren't used or that were just used for testing
Using Source Control to track versioning differences.
Death to Hungarian Notation
Names should be self documenting - no acronyms, developer initials or anything that might lead to confusion about what the method should do.
Only including update information in the procedure's/parameter's documentation and leaving big changes to the source control system.

Since we didn't have a Source control system at the client (their choice not ours), part of this reasoning was problematic and I'm probably leaving a few things out,but overall, this characterizes Argument 1.

Argument 2 on the other hand was

Document everything in the procedure.
Have a dated comment showing who made the change every single time anything was changed
Leave backups and test objects in the database because they may be needed later
Minimal attention to formatting (although each person that advocated this was very careful about how their code performed. So it would be totally unfair to imply their work was sloppy - they just had more of a don't sweat the small stuff but make really sure the big stuff is right approach
Use initials for personal utility procedures so they can be found and used easily.

Personally, I am a nut about formatting. I'm really fastidious about formatting and would clearly rather have NO comments than out of date comments or too many comments - but that's not to say that I am anti-comments in any sense. I also am pretty fussy about source control, if you are going to have any configuration management, I can't stand it being done in comments inline or at the top of the code - it needs to be in a version control system. So on this one, the people that were on the project who preferred Argument 1 were in the minority. However at the table, we were in the Majority. This immediately led to a small trash talking session among friends. Little did we know the new guys were so interested in the topic. One of them seemed to be pretty opinionated but it wasn't until a week later that we realized how much so.

The thing is that while I consider myself a Data guy, my theoretical understanding of Relational Database theory is much stronger than say my DBA skills are. And although I'm admittedly a little fussy about formatting and the like, I have to admit that many of the best DBA's I've worked with are much more relaxed about formatting and commenting than I am. Many of them not being AppDev guys don't depend on Source Control and prefer to handle versioning issues inline. It's difficult to argue against competency so in the end, my utimate belief is to defer to the best guy on the project.

Have you encountered any discussions like this that you didn't really think would get people talking much but ended up getting people really involved? Personally, I think that as long as you're dealing with mature professionals, these sorts of things are great b/c folks are passionate about them, so you have smart people putting forth their best arguments, and you can't help but gain perspective and knowledge. On the other hand, if you deal with people who get defensive easy, or who have trouble admitting they don't know everything (funny how the more defensive they are , the less they actually know in most cases), well, this stuff will do nothing but cause bad blood.

So which side do you fall on? I'm guessing it's not this black and white in many cases, for instance, the comma issue could fall in either group.

What sorts of things have you come across that have stirred discussion? Any specific coding arguments, and if so, what?

Programmatic Malpractice

William — Thu, 01 Feb 2007 03:23:00 GMT

In the category of "I couldn't make something like this up even if I tried" comes the following code. Rather than try to dissect what it is that's wrong with it, it's a lot easier to find what's right with it. If the person that wrote this cashes their paycheck, they should be charged with theft. We have all been in a hurry and written code that we aren't proud of. We all are guilty of oversights. But this was done by a self proclaimed guru and expert, and when you read the comments that were attached to it, it's clear that this was done intentionally. So I'm opening the floor to pointing out what's wrong with it, or for my really advanced readers, what's right with it (God knows a lot can go wrong calling the AddNew method of a DataTable and wrapping such a dangerous operation in catch block catching System.Exception is absolutely critical): Read the code first, then read the comments associated with it. P.S. I'm warning you up front that I'm not responsible if you start barfing, become overwhelmed by the ugliness of it and gouge your eyes out as solace or if your brains succumbs to spontaneous combustion. Yep, a 'professional consultant' wrote this work of art. As a consultant, I guess I should find comfort in stuff like this b/c as long as there are people like this committing malpractice, I mean working, then I'll always have a job. On the other hand, this sort of stuff is so bad and so inexcusably terrible , that my heart goes out to this person's victim, urr, I mean client.

public static decimal EvalNumber(string strExpr, ref bool
calcSuccessful, ref string NegativeDelim)
{
calcSuccessful=false;
NegativeDelim = "-";
DataTable dt = new DataTable();
try
{
dt.Columns.Add("Expr", typeof(Decimal), strExpr);
}
catch
{
// tough decision they entered a error
// colud be as simple as text/words
// could be division by zero
return (0);
}

try
{
DataRow dr; dr = dt.NewRow();
dt.Rows.Add(dr);
}
catch
{
return (0);
}

decimal TempReturn=0;
try
{
TempReturn=Convert.ToDecimal(dt.Rows[0]["Expr"]);
calcSuccessful=true;
}
catch
{
calcSuccessful=false;
}

// expressions like (9) will resolve as positive 9 which is wrong
// expressions like (9+2*12)*20 will obviously be a real formula
if (calcSuccessful == true && strExpr.IndexOf("(") == 0)
{
string tempStr = strExpr;
tempStr = tempStr.Replace("(", "");
tempStr = tempStr.Replace(")", "");
// parentheses are stripped

decimal tempNum = 0;
try
{
tempNum = Convert.ToDecimal(tempStr);
// (9) == 9 so we must turn it to -9
if (tempNum == TempReturn)
{
TempReturn = TempReturn * -1;
NegativeDelim = "(";
}
}
catch
{
// nothing to do, all is well
}
} // end if parentheses issues

return (TempReturn);

} // end EvalNumber

private void ITDataGridView_CellParsing(object s,
DataGridViewCellParsingEventArgs e)
{
string tempMsg = "ITDataGridViewFund_CellParsing";
Logger.DGVStatusGrid(tempMsg, s, e, this);

if (CellTypeCalc(e.RowIndex, e.ColumnIndex, this) == false)
return;

// we are in a row that allows formulas
bool evalSucceeded = false;
string negativeDelim = "";
decimal decMoney = 0;
string CellValue =
this.Rows[e.RowIndex].Cells[e.ColumnIndex].EditedFormattedValue.ToString();
decMoney = Utils.EvalNumber(CellValue, ref evalSucceeded,
ref negativeDelim);
if (evalSucceeded)
{

// set format based on ( or -
if (negativeDelim == "(")

this.Rows[e.RowIndex].Cells[e.ColumnIndex].Style.Format =
"#,##0;(#,##0)"; ;
if (negativeDelim == "-")

this.Rows[e.RowIndex].Cells[e.ColumnIndex].Style.Format =
"#,##0;-#,##0";
}

} // end ITDataGridView_CellParsing

-----------------------------

Quote from the Guru that wrote this:

I tried every event to set .Value to the result of that formula. I
tried dozens of desperate acts including just trying to SendKeys the
Calc result to the cell (even that hack is impossible to get to work).
Despite printing dozens of articles from Google and reading and
empirically experimenting, days of frustrating no answer were my fate.
But its ok, they had not resolved this in months and were going to say
"no" to that user requirement before they handed it to me. They had
made a pretty good app, otherwise that is a 1:1 knockoff functionality
wise of another .net app written in 1 C# file with the Infragistics
grid and spaghetti code everywhere, extraneous code, ad-hoc SQL, etc.
whereas they wrote a fresh code base in 2 dozen sanely organized files
and using Sprocs instead. They just needed a few bugs and enhancements
fixed, and the code refactored in places for the new code and they
were quite burn out on taming the DataGridView with very little,
incomplete and confusing and contradictory docs whereas I am not burnt
out.

The two things that maybe could help but the docs/Google searches were
as clear as mud i.e.
e.FormattingApplied=true
e.ParsingApplied=true
seemed to not work well. They are normally false but if set to True
they tended to create recursive DGV events in an infinite loop and
overflow the stack. Ouch!

Finally the answer turns out to be to use
_CellParsing

and VERY IMPORTANT use these 2 lines:

e.Value = newValue;
e.ParsingApplied = true;

and never do:
this.Rows[e.RowIndex].Cells[e.ColumnIndex].Value = newValue;
e.ParsingApplied = true;

even though its basically the same thing because DataError will fire
and recursion will occur etc.

The users had 2 needs that sound simple but are quite tricky...

1) a cell that understands a formula
2) that cell if you enter -33 it should format as -33
if you enter (33) it should format as (33)
(ouch harder than it sounds)

and the Eval function I use i.e. is crude and convoluted because of
requirement 2 and the fact that when it works as designed (33)
resolves to +33 if one does not compensate since it considers
parentheses to be an algebraic operator i.e. to make (2+3)*9 resolve
without the multiplication occurring first via the order of operation
rules. I probably will redo the EvalNumber function soon to ve cleaner
and maybe with generics to support more numeric types than decimal
(int, double, etc.) but this app only needs decimal.

UPDATE: You've heard the adage 'the customer is always right' haven't you? Well, I guess the developer that wrote this has his own version "If the customer doesn't complain, then the code doesn't suck" Hysterically, since the customer was ok with the end result, it's not bad code. I guess you couldn't write code that met the customer's needs that wasn't so lame? The fact is, that a lot of code problems don't show themselves at first. It's only after stuff doesn't scale or port well, that people realize it's crap. Trust me dude, you could have written better code that they would have been ok with and rationalize it however you like, people bragging about being gurus shouldn't start writing stuff like this.

I'll be doing the presentation next Monday

William — Fri, 26 Jan 2007 00:39:00 GMT

If you're going to be in Atlanta next week, stop by the .NET User Group Presentation .

As you probably know already, I tend toward non mainstream technologies. This presentation is going to be on Windows Cardspace , Windows Communication Foundation and Windows Presentation Foundation.

I've been working diligently as a tech reviewer for Scott Klein's Professional WCF Programming so I'm pretty well versed there. I've learned a lot about WCF from Scott so hopefully I can make a good showing there.
My fascination with Cardspace borders on an obsession so I'm quite confident this part should be cool.
I have 0 UI Skills so while I know my way around WPF, well, my game is pretty weak.

I was a pretty big fan of Remoting, love Web Services and loved WSE (heck, I was one of the 4 people in the world that actually cared about WSE 3.0) so WCF is just - well, da bomb. Huge move forward here.

As far as Cardspace, I can only hope it's going to get adopted and go mainstream. Self issuing cards are a cool idea and more than anything, will make things more convenient. There's got to be some streamlined distribution of Managed cards, but once that's addressed, it's got to be huge. I'm having some server problems right now, but hopefully they'll be addressed tonight. Ryan and I are putting together some web services and I hope to have Cardspace up and running with one of them.

Everybody else in the world is blogging about WPF so I don't have much to add there.

How to be really annoying with Pocket Outlook

William — Thu, 25 Jan 2007 06:08:00 GMT

If you've been following my coding tonight, you can probably tell what I'm up to. If you call me regularly, you probably are already familiar with this little project, aka the CuckooBot. He's up to Version 4.0 and about to be released into the wild. Yes, this one is more functional and useful than the last but also a lot snarkier and more annoying. In fact, he's also a lot smarter. He has a lot better AI and text parsing for one thing, so he can make better decisions about whether or not I can be disturbed and if you are on the 'cool' list, he'll actually do some of your bidding. When I release the final version, I'll provide everyone with the list of commands he'll respond to. But enough about that for now. Part two of this project involves email. You can shoot the CuckooBot an email with a request, like "Please send ClevelandSteamer.jpg" and if you are authorized, he'll send it. The authorization piece is causing me some grief but it's not like I have anything better to be doing in the middle of the night right?

Ok, so the way to send Email messages is to first create an OutlookSession object. Instantiate it and scope it as necessary. Next, specify an EmailAccount for it. Next create an EmailMessage and at the end, just Send() it.

Now, I've mentioned many times that things like BCC, CC, Importance and Sensitivity should be outlawed and considered as tantamount to war crimes for they are pure evil. But too many people like those features so I'll have to keep my little quixotic jihad against them going in other ways.

const String Primary = "Cuckoo1";

private

{

OutlookSession MainSession = new OutlookSession()

EmailAccount

EmailAccount MainAccount = MainSession.EmailAccounts[Primary];

EmailMessage MyMessage = new EmailMessage();

//One dorks set this to High.

MyMessage.Importance = Importance.Normal;

//Again, see above.

MyMessage.Sensitivity = Sensitivity.Normal;

MyMessage.Subject = "Cleveland Steamers vs. Pink Socks on Fox - 8:00 PM EDT";

MyMessage.To = new Recipient("Knights of the Cleveland Steamers", "911@clevelandsteamers.org");

MyMessage.Attachments.Add(new Attachment(@"\eviction1.jpg"));

MyMessage.BodyText = "Cleveland Steamers aren't appropriate for community sites. However I predict they'll win tonight - 9-2");

MainAccount.Send(MyMessage);

}

Password Protection != Encryption on SqlCe

William — Thu, 25 Jan 2007 03:50:00 GMT

I know this probably seems pretty obvious, but believe it or not, it's a very common area of confusion. Take a look at the following declaration:

String ConnectionString = String.Format(@"Data Source=\BlueCuckoo.sdf;password={0}", "SomeValue");

SqlCeEngine

engine.CreateDatabase();

If I asked, "Will this database be encrypted?" you should hopefully answer "No" b/c I gave away the answer in the title. However on more than a few occassions, i've seen people assume that it was. If you don't explicitly add Encrypt = True - it isn't going to be encrypted. And just in case you're wondering "Well, what if I want to encrypt the database but not password protect it?" , well, you wouldn't seriously ask me something like that right?

SMS Messages with Windows Mobile 5.0

William — Thu, 25 Jan 2007 02:40:00 GMT

A while ago, I was playing around creating a bot, a Snarkier version of the Mobile Secretary Application . Well, it was late and I got sloppy. So basically, my little app sent out a text message to everyone on one of my company's Outlook distribution lists. I learned a lot that night.

So you've always been able to send SMS messages with Smartphone, but it's a lot easier now. Sending them isn't a big deal though. The cool thing is intercepting them. To that end, the Message Interceptor class makes it quite easy:

Create a module level variable (or whatever scope you need it).

private MessageInterceptor Interceptor;

Instantiate it in the constructor

Interceptor = new MessageInterceptor(InterceptionAction.NotifyAndDelete, false);

Specify a InterceptionAction property. (Notify or NotifyAndDelete) - those should be self explanatory [See above]
Specify if you want it to use the Form's thread (a must if you want to update the UI) [see above]
Wire up a handler.

Interceptor.MessageReceived +=new MessageInterceptorEventHandler(Interceptor_MessageReceived);

Cast the e.Message object in the handler as a SmsMessage and use accordingly

void Interceptor_MessageReceived(object sender, MessageInterceptorEventArgs e)

{

SmsMessage IncomingMessage = (e.Message as SmsMessage);

String[] ImportantStuff = new String[]{"Dog pooping",

"Vista Launch",

"Would you care to explain THIS?",

"Vicodin ES",

"Camel Toe",

"Kim",

"Mom",

"EDC"};

foreach(String s in ImportantStuff){

if(IncomingMessage.Body.IndexOf(s)> -1){

SmsMessage OutgoingMessage = new SmsMessage(e.Message.From.Address, "This is the Cuckoobot, I'm handling Bill's messages, he'll be in touch shortly");

OutgoingMessage.Send();//Yes, I know this isn't optimal - it's demo code.

}

else{

SmsMessage OutgoingMessage = new SmsMessage(e.Message.From.Address, "This is the Cuckoobot. Your message isn't important and will have to wait.");

OutgoingMessage.Send();
}
}
}

A very slick Smartphone Emulator feature

William — Thu, 25 Jan 2007 01:25:00 GMT

I'm finishing up some SmartPhone articles I should have posted soon but figured I could discuss this one very quickly. Any professional application should be thoroughly tested on a physical device that it will be run on. However that's not practical for incremental builds throughout the day. So that's where Emulators fit in right? Anyway, one shortcoming of emulators is, rather was, the inability to duplicate functionality exclusive to the phone. How would you simulate an incoming call or SMS message? Fortunately, that's easily resolved now. Let's say you want to send a SMS message and test it. Just use 425.001.0001 and viola, it will fire a SMS message directly to your emulator. Just to show how easy it is...

private void SendSMSMessage(String phoneNumber, String message){

SmsMessage NewMessage = new SmsMessage(phoneNumber, message);

NewMessage.Send();

}

Then to fire it off...

SendSMSMessage("14250010001", "Behold the greatness of the Blue Cuckoo!");

More coming shortly, I promise!

Musings on the database formerly known as Sql Everywhere

William — Thu, 02 Nov 2006 01:53:00 GMT

The IM volume I received today about the Rollback to SqlCE issue was enough to border on overwhelming. Between PMs and some lazy SOB CRM guy trying to get me to do all his work for him, it was hard focusing on work today. (And seriously, if you are a CRM professional, telling me you don't know how to do a two table, one field inner join so i have to do it for you, makes you look like a Complete Moron. Try 'Bill, I'm really busy, can you do me a favor and write this join for me' next time, at least then my gag reflex won't kick in). But I digress.

Anyway, I've been getting a lot of email about my presentations on Everywhere. I guess I got people thinking about a few assertions I've made. In a nutshell, I made the following comments:

I love ADO.NET and in no way am I disparaging it, but in some scenarios, the disconnected model just doesn't fit. This has nothing to do with any deficiency in theory or implementation of ADO.NET. A Ferrari is a great vehicle but if you need to tow a heavy boat, it's not the right vehicle.
The one case where I don't think the ADO.NET architectue is well suited to handle is single user scenarios where you are *positive* that you'll only have one user connecting to your database.
Devices like Smartphones and PDAs proabably aren't going to be targeting multiple users simultaneously and in those cases, where you are resource constrained and pretty much assured of a single user, the database->dataset model is overly consumptive.

In case you don't understand why, let me explain. What does using a disconnected model bring to the table? Essentially it brings tremendous scability and at least as far as the server is concerned, the possiblity of improved performance. It is a Just In Time data access strategy that can greatly minimize locking and bottlenecks. Like anything, it needs to be used wisely but the fact you don't have to keep a persisent connection to the db is huge. This benefit isn't free. In exchange for it you have to pay a price in RAM b/c you're using cached copy of the data. In most cases, this tradeoff is well worth it. In some though, it makes very little sense. Think of a Smartphone. I have one user. Only one person needs to be hitting the db at once. I have the data persisted to disk and I have limited RAM. What benefit does sticking a dataset in between my UI and the backend db bring? None.

That's where the beauty of using SqlCE/SqlMobile/SqlEverywhere/SqlCE comes in. You can use the SqlCeResultset which is scrollable and updateable, and you can cut out the layer in the middle. Your logic is trimmed down b/c updates/inserts/deletes require no work on your part provided the table you're hitting is keyed.

But that has a problem. What if the data needs synchronized with a db server? Well, you can roll your own logic. Or you can use a familiar, well documented, well tested and peer reviewed mechanism. Unless you're getting paid by the hour and you have no compunction about robbing from your clients, use the latter.

So in short, on a SmartPhone or PDA, SqlCe/SqlMobile/SqlEverywhere/SqlCe is a seemlingly perfect solution for data access. Now that you can run it on a desktop, the same holds for single user scenarios there. And using a SqlCeResultset is *not* a replacement for ADO.NET. It's still ADO.NET in every sense of the word, it's just a evolution of one aspect of it. And a darned good on for single user scenarios.

Cool things you can do with Windows Mobile

William — Tue, 24 Oct 2006 00:11:00 GMT

There's a lot of possiblity to create cool stuff like this on Windows Mobile . Devices are waiting for a compelling app and stuff like this, or like Casey often does are going to drive us to finding it (my money is on Casey to actuall invent it but that's another story). Anyway, very cool and practical post.

Differentiating between "Denormalization" and excuse making.

William — Wed, 27 Sep 2006 05:47:00 GMT

I've actually got a lot to discuss about database stuff but wanted to start out explaining a very basic concept. All me to disgress for a moment. I've worked at a few places where there were some Cobol/AS 400 developers that were quite full of themselves. They knew how to handle business with COBOL/AS400 and it went straight to their heads. Along came round wheels, indoor plumbing, relational databases and object oreinted programming and they dismissed it as a fad. Then as reality repeatedly *** slapped them, they just turned into eternal pity parties lamenting how all the good jobs are moving to India and how relational databases suck. And as is often the case with people that think Relational Databases suck, they were heavy on opinion but really light on reasons to back it up. There was a perfect correlation between the degree to which they thought RDBMS's sucked and their lack of knowledge about them. How you can know something is lame without knowing anything about it [with repsect to technology] is beyond me. Sure, I can know that drinking out of a Port-O-Potty sucks without doing it, but I at least have to know what's in a Port-O-Potty to understand why drinking from it would suck.

So their criticisms came down to two things for the most part. First they claimed that joins had too much overhead. Next they claimed that Normalization was unnecessary and that in many cases, it was impossible to do. Back when I had my Apple 2e, perhaps joins would be too expensive. But in and of itself, it's a lame argument. [I won't even touch the fact that many of their Killer AS400 apps used to take 20 minutes to run even the simplest of queries]. And they would build in redundancies everywhere. They must have all surfed the internet long and hard for "Lame excuses to use when you get busted creating a really crappy db structure" and somewhere stumbled across the word http://en.wikipedia.org/wiki/Denormalization because that was their excuse for everything. One manager who fancied himself an architect was addicted to redundancies. He actually lamented the fact that the tool we were using only allowed 250 columns and we had *many* tables that used every last one. When the chickens came home to roost he decided he needed a good excuse. The president of the company decided to circumvent him and went to each of the developers asking why we were having all the problems we were experiencing. And everyone that wasn't a COBOL programmer who refused to change said that same thing - redundancies. He was in a bind. Then he called a meeting, with the president, and had a bunch of printouts about Denormalization and how that's the approach he used.

So people that don't know what they are talking about often use Denormalization and redundancy synonomously. The metaphor that comes to mind is this... If you fill your bathtub up with hot water, then find it's too hot and proceed to turn on the cold water until it becomes tolerable, that's not the same as never turning on the hot water in the first place.

This silly excuse usually shows itself with 'Historical' data, they'll claim they built in the redundancies to handle historical data. If you had a customers table with a SalesPersonID which relates to a name, they'll point out that if you change the name of the sales person, you'll affect all the records. This issue can usually be worked out with effective dates but that's another story. So I'll concede, the historical data issue is a valid place to have redundancies. However, and this is a big howerver!!!!!, it's only valid if the historical data is not to be changed in the future. If it's data that you might have to change or update, then the redundancies are just that, redundancies. And you can still use historical data and use relational modelling. If you have the SalesPersonID and Name in the Historical table, and you have cause to update it because of business requirements, it's not 'denormalization' and it's a bad way to do things.

You may think this is so obvious that it goes without saying. However I was in a situation not too long ago where a database was created that had redundancies All over the place. If you needed to change a Salesperson's name (for instance, someone gets married), you had to go into at least 12 different tables to change it. And if you missed one record, it was enough to seriously screw up the logic reporting was built on. Another wonderful feature was computed fields stored as hard coded values. Every week a new series of transactions would be imported. A few processes were run and the aggregate total of sales, fees etc were stored in multiple tables. Then the details that were used to create them were stored in another series of tables. If you changed data in one place and didn't change it everywhere else, reporting would be totally hosed. To say it was a nightmare to maintain would be putting it mildly. Yet the person that built this system claimed that it was done intentionally as a means to double check the values. So let me ask you this. If you have a table with CustomerID, Sales, Week, and you run a process to stick the aggregate of sales in one table, how is that any different than just using SUM? Rather, how can you claim it's 'better' or intentional? People like this often rationalize it with "Well if everyone did their job and just made sure they updated all the tables, there wouldn't be a problem.' True enough. But people make mistakes. If you can't force all the other changes to occur each time source data changes through triggers or integrity constraints, you can't guarantee changes will be made. Plain and simple.

When you see a nightmare database or program, it's often attributed to the ignorance of the person that created it. But pure ignorance will typically only produce bad stuff, not truly terrible stuff. To be truly terrible, it takes someone who's convinced their way is best, while being totally ignorant of the consequences. You can only make something so bad accidentally. No, the really horrendous things out there are always the result of deliberate action by someone or some group of people. And this begs the bigger question - why, in 2006, with all the books out there, internet sites etc, so people still fight normalization when using Relational Database Systems? And why or why do people that have no clue what they are doing and little to no desire to learn get to design critical systems?

I'm betting this is more common than it may seem. Anyone else still run into stuff like this?

Comment of the week - Sql Server Reporting Services vs. Crystal

William — Wed, 27 Sep 2006 05:27:00 GMT

A while ago, I posted this about how much a55 I think Crystal Reports sucks. A long time ago, back at like v.7, I thought Crystal was a decent product. And to be honest, it still is a decent product. But it comes with a lot of baggage and has a lot that just makes it lame. So I had almost total consenus on the Crystal Reports w/ .NET sucks side, I think there are two people that disagree. Ignoring the fact that SSRS is very new product and Crystal has been around forever, I think side by side it's a slam dunk. But one big mouth said this: (NOTE: I'm only posting a comment that was publicly posted on my site. So hopefully the guy won't be a cry baby about it and throw a temper tantrum over it)

"Either you guys are idiots or you have never used Crystal and don't realize the minimal functionality that SQL Reporting Services offers. I have spent the last week evaluating SSRS and am greatly dissapointed. I came up with a list of over 250 items that SSRS could not do. As for all you Access reporters, your probably the only people to ever purchase Microsoft's "Bob" product. "

So the guy claims to know enough about Crystal to have cataloged over 250 features that SSRS doesn't have. That means that including the features he'll admit it does have, he knows way more than 250 features. I think that would qualify you as a fairly advanced Crystal user if you'll pardon the oxymoron. And he spent one week with SSRS. And the only conclusion he can draw is that SSRS is definitely lacking those features and that it couldn't possibly be that he just doesn't know how to use them, right? And far be it from me to say anything positive about Access, but come on, being a hard core Crystal user still means you're the report ***, and that gives you credentials to snob no one. Certainly not Access developers and certainly not Bob users. Is he really claiming Bob was lame but Crystal Reports isn't? Pulllleeezz. But if you read the context of the comments, the people were pointing out that Access, as lame as it is, provides more effective funcionality. The fact this can even be debated speaks volumes don't you think? Is Access supposed to be first and foremost a report writer? Last time I looked it's main purpose was as a relational database (more precisely, a database that anyone can build with, that causes major headaches for the people that will inevitably be brought in to clean up the mess it made) not a report writer. Reports are one of its features but one would certainly expect that a mature product who's sole purpose is report writing ought to be a little better than a RDBMS product with reports added in.

So I offered a challenge, I asked him to post just 20 of his 250 features that CR can do that SSRS can't. I'll see if they really can't be done and post the results. If I'm wrong, I'll admit it. If he's wrong I'll point out what an a33hat he is. I actually believe there are probably 20 or so features missing, but certainly not major ones. Even assuming there were really 250 features missing, how many times do you need 250 features for a report? Typically you need Grouping, Subreporting, charts/graphs, conditional formatting, functions and the like. He didn't comment as to his evaluation of how each stacked up there - for reasons that are obvious. Anyway, I believe his 250+ number about as much as I believe in the easter bunny so although *someone* could probably post 20 things lacking, I doubt he can. One of us is very wrong here and hopefully it's not me. We'll have to wait and see.

Normalization really does matter

William — Thu, 07 Sep 2006 14:30:00 GMT

It seems like years ago, but I remember one of my professors giving us an assignment. It involved taking a database chock full of redundancies and writing the SQL Statements to make a few small updates. We thought it was an assignment to test our SQL Skills. However we were graded on how close to 'right' we got all of the changes. It wasn't about SQL at all, he could care less about SQL at this point. It was about illustrating, with much painful monotony, why "Normalization matters."

After we turned the assignment in, he asked everyone how long it took. Suffice to say it took everyone a long time. But the interesting thing was that not one single person got all of the changes correct. Many got close, but no one got all of them.

Then he explained some more why normalization matters. You see, not only did working with non-normalized data (as opposed to denormalized) take forever to do, it was very error prone. No matter how careful you are, if you do the same thing over and over, you're likely to miss something.

Afterward, he proceeded to give us a story from the 'real world'. He told us how some things in CS have great theoretical value but not much practical value. But he said that in general, good theory makes for good practice and no where is this more true than in DB Design. We were told that when we get out into the real world, we'll hear all sorts of 'Senior' folks trashing normalization as ivory tower stuff that works great in college but doesn't have value in the real world. He said that *when* we hear this, to immediately ask the person "At which level doesn't normalization make sense in the 'real world'? All of them or just certain forms" This was because, in his opinion, the only people that will trash normalization are ones who don't understand it. Granted, there were many caveats in the lectures. Sometimes it's best to denormalize (which, is something you do AFTER you've normalized - for the uninitiated). Many will design garbage and when the redundancies are pointed out will say "I totally do believe in normalization but I denormalized the design". If it wasn't normalized in the first place it couldn't have been denormalized, such statements are mere excuse making. He also pointed out that sometimes when time is of the essense and maintainability isn't a real issue, then normalization may not justify the costs. There were definitely exceptions to his rule, but one thing that always stuck with me was that folks that trash normalization almost always have never studied it very far.

So why am I babbling about this? It's simple. Some things can be totally screwed up and then fixed quickly/easily/inexpensively. I could get the color scheme of a CSS layout totally upside down and fix it pretty quickly in many cases. Other things are a little more difficult but still doable. Let's say I didn't use inheritance and coded some total redundant classes. Making the code modification won't be trivial but if my code is modular, it should be fairly straightforward considering the development tools in place these days. Other things though are deal breakers. Absolute and utter deal breakers. Database design is the epitome of a deal breaker. To those new to development, you may be asking why DB Design is so vastly different from other design issues.

Just imagine the following scenario. You have a Dog and a Cat class and they are in a running production app that's working great. You decide that you should have had a Mammal base class and decided to implement it. You guess it'll take you about 16 hours to find every instance and make the appropriate changes. So the app runs along, users are happy, you take your 16 hours, make your changes , test them (or test them, make them for many) and redeploy. No problems. Users are none the wiser and everyone's happy.

Now what if this was a production database? Well, if you have downtime on the db then you can create a replica of the data, make your changes, verify they worked correctly and then implement them at some time when no one is using the system. I've only worked in two places where such was the case. If the design flaws go out the door on a live system, you're stuck.

In the Dog Cat example, if you totally screwed something up, you can redeploy the previous version and in many cases it won't be too big of a deal (in some it obviously will but that's related to installation more than anything else). But what if you damaged a lot of data, the impact very well may (and probably will) be immediate. Moreoever, you are probably dealing with a moving target b/c the data is constantly changing. Even if you think you coded everythign correctly, new data may be introduced that you weren't counting on. And even then, even assuming you got everything right, many changes of this sort are going to cause service interruptions. Depending on what the db is used for, this could be a huge deal.

We all know that the further downstream it is that a bug is caught the more it costs to correct. We've all seen the upward trending curve. The thing is that in most case, with DB changes in a production system, that curve is very short and very steep. And if you do any of the following, you should be flogged:

Use Reserved words as object names, column names etc (they are called Reserved words for a reason)
Use inconsistent naming conventions
Use unintelligible naming conventions
Ignore data types. NVARCHAR isn't the solution to everything. Yes, I know you can cast dates held in VARCHAR fields to DATETIME Values but that's irrelevant
Fail to use keys. At a minimum you're going to have a lot of trouble getting tools to update data but that's the least of your worries
Use multiple fields to represent similar data and then using inconsistent rules for maintaining the data that they are populated with
Fail to enable integrity constraints. I once heard a developer, with a straight face, say he didn't believe in integrity contsraints b/c he was a 'real developer' and preferred to write his own. He contended that he wanted to make sure things were done right as opposed to being lazy and leaving it to Oracle to handle. You have to have the initials CC or have forgotten to take your lithium if you think you're going to do a better job writing your own integrity constraints than Oracle/Microsoft etc are. And for those of you that are going to get all pedantic and tell me of some obscure bug or situation where MS's or Oracle's constraints fail, I'd counter with this. Use their AND use your own. You can't guarantee that they only way the data will be accessed is through your killer app and it's uber integrity algorithms. So why leave it bare? If you're constraints are really that good, then it shouldn't ever be an issue anyway right?
Freestyling! Freestyling is cool as long as it's done in a Rap/BMX/Skateboard etc context. It has no place in the development world other than in R & D scenarios. RDBMS systems are well tested and rooted in very strong theoretical foundations. They have stood the test of time and they work. They aren't perfect but they are damn good and they work well when used correctly. Unless you really are a super genius, chances are that you're not going to have some superior way of doing things that the rest of the world has missed. And even if you did find one, the chances of you finding another one are slim to none. And why do these folks never seem to freestyle with other things like addition, subtraction, multiplication, division, gravity, intertia etc? What do you think would happen if you owed the IRS 10k and used your own math to compute that they actually owed you 10k? Why would it be any different with anything else? So if you're some Maverick has a ton of 'unique' ways of building your dbs - all you're doing is shoving the burden of your sloppiness onto someone else.
Failing to at a minimum follow the first two normal forms.

It's a matter of "you can get with the program or you can pay a ton of money to have someone else fix it, or spend a ton of time down the road fixing it yourself." And if this happens, you can't cry foul - you're the one that decided to break the rules.

Transferring data around with ADO.NET 2.0 - SqlBulkCopy

William — Sat, 26 Aug 2006 16:03:00 GMT

I'm working at a client on a data migration and found some results that were surprising. I expected there to be a difference but not one quite this large... Synopsis of Problem: Right now I'm working on a rather complex migration from a bunch of disparate...(read more)

Stealing code ;-(

William — Mon, 03 Jul 2006 22:45:00 GMT

I don't know why I even let it bother me but a lot of times is still does. Usually starts out with me working on something, an article, a book, a newsgroup question. I start searching around and use words that I think describe what I'm looking for. Low and behold, there's an article on X. Great. Click through, start reading it and then realize Cuckoo c = new Cuckoo(); or some other code looks really familiar. Look up at the site - nope, not mine. Look at the author - nope, not me. Look for credit, nope, none given. Look at the variable names or namespaces, "Gee, this guy decided to use KDN.xxxx as a namespace. I use KDN for KnowDotNet - small world I guess." except that it is my code. Then I start wondering, did I give anyone permisson? If anyone asks I'm normally ok with it but if I wrote it, I'd probably at least have asked for some credit.

Then being a grown up kicks in. Putting code samples on the net means some folks are going to take it and use it which is what it was up there for in the first place. But others are going to take it and try to take credit for it. It's life. Get over it. Quit whining. Write a blog post about it if you must and then shut up about it - you're too old to snivel, particularly about something like this. Then I go back to what I was doing....

Do you have any ideas about preparation for 70-529?

William — Wed, 28 Jun 2006 02:30:00 GMT

As you may know, I'm coauthoring http://www.microsoft.com/learning/exams/70-529.asp . In particular, I'm covering the following objectives: Create and configure an XML Web service method. Create a public method. Attach the WebMethodAttribute attribute...(read more)

Biztalk - What do you call a 'difficult' application

William — Thu, 15 Jun 2006 02:19:00 GMT

I think I'm going back into one of my hypomanic phases b/c everything seems so clear and I'm learning at an amazing pace. Back to Biztalk. Ok ok. Back in the day I was pretty big into Biztalk. For a while, it seemed like it was fizzling out and since...(read more)