WinDrvr : Tools

Improving the documentation

DonBurn — Tue, 28 Aug 2007 18:32:00 GMT

If you haven’t heard, Microsoft is now updating the WDK documentation monthly and is now providing a way to download these updated docs to your computer. This update, just for documentation, can be found at http://www.microsoft.com/whdc/DevTools/WDK/WDKdocs.mspx Microsoft should be congratulated for this effort to improve the documentation in a timely manner.

Currently there are some flaws to be aware of. First the install process does not take into account that many of us would just like to update the documentation we got with the WDK with the new docs. In fact, the installer does not even know about the WDK installation, and provides no way to update the documentation. I find that the installation of the WDK’s and its documentation are the least friendly parts of the product. After several attempts to install the updates, I finally pulled the doc files out of the underlying CAB file and put them in the help directory of the WDK where they belong.

The second problem is that the documentation does not have a feedback link so you can send in bug reports. This will be corrected in the next version, but for now the feedback alias is ddksurv1@microsoft.com. Having a well documented feedback alias is significant. Those who have been in the industry awhile may remember that when OSR hosted the documentation bug finding contest, the quality of the documentation went up significantly.

With all the new material in the documentation, there are errors. Now that there is a mechanism for providing feedback, and with frequent updates, we have a chance to fix these problems. Let’s roll up our sleeves and get to it. I am going to cut this entry short so I can send some mail about the technical mistakes I know of in the doc set.

Tag, you're it

DonBurn — Fri, 15 Jun 2007 23:31:00 GMT

I’ve been spending the last couple of days tracking down a bug in a driver I am writing. The effort reminded me of how great tags on memory allocations and frees can be. Also, the work reminded me that there are at least a couple of features Microsoft does not promote and I rarely see.

For the uninitiated, tags are a four character value that is passed as an argument in memory allocation calls. The tag gives you a way to identify what the memory was allocated for by having a different tag for each common structure allocated. Here is a common problem: many driver developers commonly assume that they have only one tag for their whole driver because so many of the common samples do this. There is nothing stopping you from having multiple tags, and in fact there is a strong reason to have them.

Right now I am developing a file system mini-filter that has twenty-one different tags it uses for allocations. All the major buffer types and context blocks each have a unique tag for their allocations and frees.

Yes, I said frees. Part of the reason to have multiple tags is that you can also put the tag on the free of the memory by using the call ExFreePoolWithTag. Unfortunately, this call has been described as worthless in Walter Oney’s Programming the Windows Driver Model and is incorrectly documented by Microsoft.

The value of ExFreePoolWithTag is when you combine it with a tag with the PROTECTED_POOL bit set. This bit requires that you free the memory with ExFreePoolWithTag, and the OS will bug check if the memory being freed does not have the matching tag. Unfortunately, PROTECTED_POOL is not documented except in include files, and is not used by any Microsoft sample. Using ExFreePoolWithTag with PROTECTED_POOL tags gives you an automatic check that you are freeing what you intended to.

You do have to be careful on memory you allocate that the system will be freeing, since Windows will not know what tag you are using, so the system frees everything without tags. For everything except the rare instance where the system frees the memory, using multiple tags with PROTECTED_POOL is worthwhile.

Document explorer versus useful data

DonBurn — Wed, 30 May 2007 19:21:00 GMT

There is a trend in the tools coming out of Microsoft that is driving me nuts and in my opinion significantly hurting productivity. This trend is the shrinking of the amount of data that appears on the display and requiring more mouse clicks to get there.

For driver writers this trend is most obvious in Document Explorer 8 which is used to display the Vista WDK documentation. Comparing it to HTML Help Control 5 that was used for Windows Server 2003 SP1 DDK the previous version will show you what I mean.

On opening the DDK documentation you have two major panes, the pane on the right goes from the toolbar to the bottom of the window, giving you a large amount of information, on my display (1280x1024) about 60 lines or a full page. When you open the WDK you see an extra pane on the right, called index results that takes away about 12 lines on my display or roughly 20% less useful data.

Now the second assault on data occurs when you search, with the DDK I get 36 terms on the screen, with the WDK I get 27 or a 25% drop in data. Worse yet, the search display is now a tabbed window in the data area making toggling between several elements of the search more cumbersome.

Of course index results mentioned previously also make thing more painful, since a large portion of the time you click on a kernel API in the index, instead of seeing the API in the data pane, you see some page about a use of the API, so you have to go down and click the index results to get the API. As an example go into the Vista DDK documentation, and choose ZwCreateFile in the index, I doubt most of us are looking for “TDI Kernel-Mode Client Interactions” when we look up ZwCreateFile but that is what we get.

Microsoft is doing all this at the same time that displays are going to wide screen models with 13% less vertical pixels. All of this adds up to taking longer to get the information needed to develop or check a driver, making it more likely that the implementation will be rushed and unreliable.

I don’t blame the WDK documentation group, the tool is a corporate standard. Perhaps the folks who designed and approved this trend have become so used to Power Point that more that six lines of data at a time is too much for them. Hopefully, Microsoft will wake up to the problem, at the recent WinHEC a feedback session was asked “How do you search for WDK help?” and the overwhelming answer was Google!

Bleeding edge and far from it

DonBurn — Sun, 20 May 2007 17:36:00 GMT

I am just back from WinHEC and while there I realized that many people including a number from Microsoft don’t distinguish developing for the leading edge from living there. I am known as a guy who has done a number of things that Redmond had said “Windows is not capable of doing” and technologies that Microsoft was later proud to show off once they were working.

When it comes to my tools, I am far from the bleeding edge. For instance, though I recently started using the latest Visual Studio, most of my work is still done with VS6. I like the new Studio, but I trust VS6 and that version produces code that for the most part needs no libraries other than the standard system lib’s to support it. I use Office XP and probably will not move for a long time, since I once had a terrible experience of not being able to send a customer a promised document, since the conversion back to the format the client was using did not work well.

I am not alone in this attitude; many of the developers I respect the most stay far away from the leading edge for their work environments. In this way we are like Seymour Cray, who required that the parts for his designs were in production and testable even though the systems he was designing were years off.

The challenge here is that Microsoft does not seem to understand this. The WDK team was proudly pointing out the features and fast updates with online MSDN, when most developers use the local documents. One of the new features with the MSDN stuff is a wiki capability that is supposed to be monitored by using an RSS feed. Most of the senior developers I know do not monitor this, since Microsoft’s previous generation of tools do not support RSS! Another RSS problem is that when Microsoft “improved” its blogging site, it dropped email notification of blog postings, because RSS was available in all the brand new tools.

Another example from WinHEC was the Windows Driver Testing Framework. This tool assumes you are using the latest Visual Studio to develop with managed languages using the full strength of COM. I know of almost no respected driver writers who know how to do this, and even fewer who would want to.

It has to be confusing for the WDK team because their core product is the exception that proves the rule. Over the years, the development community has found that BETA and just released versions of the DDK are excellent so we trust using these in our day to day work. Unfortunately, few other teams in Redmond inspire this confidence.

So, Microsoft, if you want us to keep developing on the bleeding edge, give us tools that do not force us to live there.

How tuned is your time machine?

DonBurn — Fri, 27 Apr 2007 17:03:00 GMT

How tuned is your time machine? No, I am not talking about DeLoreans with Flux Capacitors, but a tool almost all software development groups have but many use poorly, namely source control. A simple test of how well your source control system is doing is to see how many times a day the average developer checks in code. The sad fact is that in most development shops I encounter, the time a file is checked out is measured in weeks—as for a library book.Driver development can benefit greatly from a source control system that encourages its use. It is amazing how many times I hear “It was working, I must have changed something to break it.” If you use source control all the time, it is easy to back up to prove things worked, find the version with the set of changes that broke things, and quickly resolve the problem. For a manager, frequent source control is beneficial to scheduling since it gives you concrete data on the state of the work. Statements from developers, “I’m 80% of the way done” do not mean anything. Is this 80% coded, tested or what? Instead, “The current checked in version supports these capabilities and passes these tests” gives you a solid basis for scheduling and for knowing a project’s status.Source control gets out of tune for two major reasons, either the system is cumbersome to use, or developers ignore it. Many source control systems become cumbersome because release engineering owns them and requires additional input and checks since every check in is part of the release! Unfortunately, this approach is wrong because developers need to be able to check in often, and that includes versions with diagnostics that will never go into production.If you are a manager stuck with cumbersome source control, consider setting up your own with something like SourceSafe. Having two source control systems may mean that you as the manager need to take a stable version out of the lightweight system and check into the release system periodically, but having the history of changes I am talking about is worth it.There also needs to be an environment that encourages checking in often. The common developer model of “I’ve checked it out, I’ll check it in when all the modifications to the module work”, has to be changed. Encouraging developers to check in after every change can be a challenge. Showing the value of frequent revisions for debugging will help. For a manager, looking at what goes into a revision and encouraging incremental revisions is also needed. Finally, consider using a source control system where only one developer can have the file checked out at a time. This not only creates an internal push to finish the change and check it in, but also eliminates the error-prone merge steps in systems that allow multiple checkouts.Encouraging incremental revisions of the code, and using source control wisely is one of the best ways to improve quality and find bugs in drivers. When you are checking your code in often, it is easy to step into your time machine and track down the bugs.