Documentation with Sandcastle - a notebook

(Posted to both my main code blog and the Noda Time blog.)

I apologise in advance if this blog post becomes hard to get good information from. It's a record of trying to get Sandcastle to work for Noda Time; as much as anything it's meant to be an indication of how smooth or otherwise the process of getting started with Sandcastle is. My aim is to be completely honest. If I make stupid mistakes, they'll be documented here. If I have decisions to make, they'll be documented here.

I should point out that I considered using NDoc (it just didn't make sense to use a dead project) and docu (I'm not keen on the output style, and it threw an exception when I tried running it on Noda Time anyway). I didn't try any other options as I'm unaware of them. Hmm.

Starting point and aims

My eventual aim is to include "build the docs" as a task in the build procedure for Noda Time. I don't have much in the way of preconceived ideas of what the output should be: my guess is a CHM file and something to integrate into MSDN, as well as some static web pages. Ideally I'd like to be able to put the web pages on the Google Code project page, but I don't know how feasible that will be. If the best way forward turns out to be something completely different, that's fine.

(I've mailed Scott Hanselman and Matt Hawley about the idea of having an ASP.NET component of some form which could dynamically generate all this stuff on the fly - you'd just need to upload the .xml and .dll files, and let it do the rest. I'm not expecting that idea to be useful for Noda Time in the near future, but you never know.)

Noda Time has some documentation, but there are plenty of public members which don't have any XML documentation at all at the moment. Obviously there's a warning available for this so we'll be able to make sure that eventually everything's documented, but we also need to be able to build documentation before we reach that point.

Step 0: building the XML file

The build project doesn't currently even create the .xml file, so that's the first port of call - just a case of ticking a box and then changing the default filename slightly... because for some bizarre reason, Visual Studio defaults to creating a ".XML" file instead of ".xml". Why? Where else are capitals used in file extensions?

Rebuild the solution, gaze in fear at the 496 warnings generated, and we have everything we should need from Visual Studio. My belief is that I should now be able to close Visual Studio and not reopen it (with the Noda Time solution, anyway) during the course of this blog post.

Step 1: building Sandcastle

First real choice: which version of Sandcastle do I go for? There was a binary release on May 29th 2008, a source release on July 2nd 2008, and three commits to source control since then, the latest of which was in July 2009. Personally I like the idea of not having to actually install anything: tools which can just be run without installation are nicer for Open Source projects, particularly if you can check the binaries into source control with appropriate licence files. That way anyone can build after just fetching. On the other hand, I'm not sure how well the Sandcastle licence fits in with the Apache 2 licence we're using for Noda Time. I can investigate that later.

What the heck, let's try building it from source. It's probably easier to go from that to the installed version than the other way round. Change set 26202 downloaded and unpacked... now how do we build it, and what do we need to build? Okay, there's a solution file, which opens up in VS2008 (unsurprising and not a problem). Gosh, what a lot of projects (but no unit tests?) - still, everything builds with nary a warning. I've no idea what to do with it now, but it's a start. It looks like it's copied four executables and a bunch of DLLs into the ProductionTools directory, which is promising.

Shockingly, it's only just occurred to me to check for some documentation to see whether or not I'm doing the right thing. Looking at the Sandcastle web page, it seems I'm not missing much. Well, I was aware that this might be the case.

Step 2: Sandcastle Help File Builder

I've heard about SHFB from a few places, and it certainly sounds like it's the way to go - it even has a getting started guide and installation instructions! It looks like there's a heck of a lot of documentation for something sounds like it should be simple, but hey, let's dive in. (I know it sounds inconsistent to go from complaining about no documentation to complaining about too much - but I'm really going from complaining about no documentation to complaining about something being so complicated that it needs a lot of documentation. I'm very grateful to the SHFB team for documenting everything, even if I plan to read it on a Just-In-Time basis.)

A few notes from the requirements page:

  • It looks like I'll need to install the HTML Help Workshop if I want CHM files; the Help 2 compiler should already be part of the VS2008 SDK which I'm sure is already installed. I have no idea where Help 3 fits into this :(
  • It looks like I need a DXROOT environment variable pointing at my Sandcastle "installation". I wonder what that means in my home-built version? I'll assume it just means the Development directory containing ProductionTools and ProductionTransforms.
  • There's a further set of patches available in the Sandcastle Styles project. Helpfully, this says it includes all the updates in the July 2009 source code, and can be applied to the binary installation from May 2008. It's not clear, however, whether it can also be applied to a home-built version of Sandcastle. Given that I can get all the latest stuff in conjunction with an installed version, it sounds like it's worth installing the binary release after all. (Done, and patches installed.)
  • It sounds like I need to install the H2 Viewer and H2Reg. (I suspect that H2Reg will be something we direct our users at rather than shipping and running ourselves; I don't intend to have an MSI-style "installer" for Noda Time at the moment, although the recent CoApp announcement sounds interesting. It's too early to worry about that for the moment though.)
  • We're not documenting a web site project, so I'm not bothering with "Custom Web Code Providers". I've installed quite enough by this point, thank you very much. Oh, except I haven't installed SHFB itself yet. I'd better do that now...

Step 3: creating a Help File Builder project

This feels like it could be reasonably straightforward, so long as I don't try to do anything fancy. Let's follow (roughly) the instructions. (I'm doing it straight to Noda Time rather than using the example project.)

Open the GUI, create a new project, add a documentation source of NodaTime.csproj... and hit Build Project. Wow, this takes quite a while - and this is a pretty beefy laptop. However, it seems to work! I have a CHM file which looks like it includes all the right stuff. Hoorah! It's a pretty huge CHM file (just over 3MB) for a relatively small project, but never mind.

Let's build it again, this time with all the output enabled (Help 1, Help 2, MSHelpViewer and Website).

Hmm... no MS Help 2 compiler found. Maybe I didn't have the VS2008 SDK installed after all. After a bit of hunting, it's here. Time to install it - and make sure it doesn't mess up the Sandcastle installation, as the SHFB docs warned me about. Yikes - 109MB. Ah well.

Okay, so after the SDK installation, rebuild the help... which will take even longer of course, as it's now building four different output formats. 3 minutes 18 seconds in the end... not too bad, but not something I'll want to do after every build :)

Step 4: checking the results

  • Help 1 (CHM): looks fine, if old-fashioned :)
  • Help 2 (HxS): via H2Viewer, looks fine - I'm not sure whether I dare to integrate it with MSDN just yet though.
  • ASP.NET web site: works even in Chrome
  • Static HTML: causes Chrome to flicker, constantly reloading. Works fine in Firefox. Maybe I need to submit a bug report.

I'm not entirely sure which output option corresponds to which result here; in particular, is "Website" the static one or the ASP.NET one? What's MSHelpViewer? It's easy enough to find out of course - I'll just experiment at a later date.

Step 5: building from the command line

I can't decide whether this is crucial (as it should be part of a continuous build server) or irrelevant (as there are so many tools to install, I may never get the ability to run a CB server with everything installed). However, it would certainly be nice.

Having set SHFBROOT appropriately, running msbuild gives this error:

SHFB : error BE0067: Unable to obtain assembly name from project file '[...]' using Configuration 'Debug', Platform 'X64'

Using Debug is definitely correct, but X64 sounds wrong... I suspect I want AnyCPU instead. Fortunately, this can be set in the SHFB project file (it was previously just defaulting). Once that's been fixed, the build works with one warning: BHT0001: Unable to get executing project: Unable to obtain internal reference. Supposedly this may indicate a problem in SHFB itself... I shall report it later on. It doesn't seem to affect the ability to produce help though.

Conclusion

That wasn't quite as painful as I'd feared. I'm nearly ready to check in the SHFB project file now - but I need to work out a few other things first, and probably create a specific "XML" configuration for the main project itself. I'm somewhat alarmed at the number of extra bits and pieces that I had to install though - and the lack of any mention of Help 3 is also a bit worrying.

I've just remembered one other option that I haven't tried, too - MonoDoc. I may have another look at that at a later date, although the fact that it needs a GTK# help viewer isn't ideal.

I still think the Open Source community for .NET has left a hole at the moment. It may be that SHFB + Sandcastle is as good as it gets, given the limitations of how much needs to be installed to build MS help files. I'd still like to see a better way of providing docs for web sites though... ideally one which doesn't involve shipping hundreds of files around when only two are really required.

Published Sat, Apr 10 2010 10:17 by skeet
Filed under: , ,

Comments

# re: Documentation with Sandcastle - a notebook

> an ASP.NET component of some form which could dynamically generate all this stuff on the fly

Love the idea. That would be a very handy community tool indeed. Even better if the generated website was directly persisted by tool itself

Saturday, April 10, 2010 5:40 AM by Joannes Vermorel

# re: Documentation with Sandcastle - a notebook

@Joannes: The idea is that there would be nothing to persist. You'd just upload the XML file and the DLL, and next time someone visited the web site, they'd see the updated docs.

Saturday, April 10, 2010 7:07 AM by skeet

# re: Documentation with Sandcastle - a notebook

You might want to take a look at docu for an OSS solution.  http://docu.jagregory.com/

Saturday, April 10, 2010 11:51 PM by John Teague

# re: Documentation with Sandcastle - a notebook

@John: I tried docu - it blew up. I'm also not terribly keen on the output anyway. I do like the fact that it doesn't have any dependencies, admittedly.

Sunday, April 11, 2010 1:52 AM by skeet

# re: Documentation with Sandcastle - a notebook

@John Teague: Just for the record, both Sandcastle and Sandcastle Help File Builder are OSS.

@Jon Skeet: You said "I still think the Open Source community for .NET has left a hole at the moment". I could not agree more. You already felt it was kind of cumbersome. Have you tried writing introductory, so called conceptual documenatation? This involves writing XML files (MAML). There is no graphical editor and the XML schema is not really self explanatory.

Do not get me wrong - I like SHFB. I am also the author of an XML schema documentation plugin.

But I also think there still is a long road to create a solution that is as easy to use as other projects, such as Windows Installer XML.

Sunday, April 11, 2010 5:50 AM by Immo Landwerth

# re: Documentation with Sandcastle - a notebook

I had a similar experience when trying to generate documentation for an open source project I'm working on, using Sandcastle + SHFB. I eventually managed to make it work, but it was a painful experience.

Generating documentation for a project should be as easy as building the project... There's definitely room for improvement in that area. If I remember correctly, javadoc creates a full documentation from Java source code in HTML format, and doesn't require any external tool (admittedly, the presentation is much simpler than the typical MSDN documentation page). Perhaps Microsoft should include a similar tool along with the .NET SDK...

Sunday, April 11, 2010 4:18 PM by Thomas Levesque

# re: Documentation with Sandcastle - a notebook

I too have the preference to have build-related tools in source control alongside code. Some time ago, I figured out how to be able to run Sandcastle as well as SHFB without having them installed, so they could be checked into source control. Have a look at <a href="grantpalin.com/.../">my article</a>, perhaps you will find the details you need.

Sunday, April 11, 2010 9:35 PM by Grant Palin

# re: Documentation with Sandcastle - a notebook

A few days ago, I accidentally found one reason why running Sandcastle takes so much time, even on decent machines.

By default, all dependencies to classes in the BCL (string, int...) are resolved by integrating a link to the online MSDN (not the locally installed one) in your documentation . While this is may or may not be intended, I learned that the actual URLs used in those links are retrieved from a Microsoft server while Sandcastle is compiling your documentation. So, most of the doc build time is spent waiting for some response from a MS webservice...

If you set your documentation to refer to a locally installed MSDN, the build is much faster. You can control this by setting the "SdkLinkType" property to "Index" (instead of "MSDN") in the SHFB project settings.

Monday, April 12, 2010 2:47 AM by iTod

# re: Documentation with Sandcastle - a notebook

@iTod: Thanks, that's really useful. It sounds like it would be really, really handy to be able to periodically download a single file containing the complete index for the online links (just API would be fine), and then use that. It would be incredibly useful for various other things as well...

Monday, April 12, 2010 3:00 AM by skeet

# re: Documentation with Sandcastle - a notebook

Monday, April 12, 2010 3:21 AM by Rainer Schuster

# re: Documentation with Sandcastle - a notebook

Possible duplicate ;)

stackoverflow.com/.../generating-documentation-from-c-xml-comments

Doxygen is the de-facto standard for automatic code generation, supports a large number of languages including C#, and is open source.  Oh, it also runs very well from the command-line.

I don't see any link to the documentation you generated with Sandcastle, so I can't comment on the richness, but doxygen is quite good with dependency diagrams and call trees if you have AT&T graphviz installed.

Monday, April 12, 2010 12:03 PM by Ben Voigt [C++ MVP]

# re: Documentation with Sandcastle - a notebook

Hmm, seems my article link got mangled. I'll try again!

grantpalin.com/.../net-projects-generating-documentation-with-sandcastle

Monday, April 12, 2010 12:16 PM by Grant Palin

# re: Documentation with Sandcastle - a notebook

@Ben: I'll give it a try. I think we'd try to avoid making the XML comments themselves Doxygen-dependent, but if it can work nicely with just the normal XML, it may be a nice alternative...

Monday, April 12, 2010 12:27 PM by skeet

# re: Documentation with Sandcastle - a notebook

Tuesday, April 13, 2010 4:46 PM by Joshua Dale

# re: Documentation with Sandcastle - a notebook

@Joshua: Did you check the date on that? It was written over a year ago :(

@Ben: I've now tried doxygen, and I quite like it.  Hmm. We'll see :)

Tuesday, April 13, 2010 5:01 PM by skeet

# re: Documentation with Sandcastle - a notebook

I really like doxygen's ability go generate an index and a search facility all in standard html/js.

Wednesday, April 14, 2010 4:46 PM by Todd Stout

# re: Documentation with Sandcastle - a notebook

We use Ndoc3 at my office and it works fine for the most part.

Wednesday, April 14, 2010 7:17 PM by bashwork

# Another tool used beforehand within VS

Maybe you should also take a look into GhostDoc (submain.com/.../ghostdoc.aspx). It's a Visual Studio plug-in and helps you very much to document all your publics.

Monday, April 19, 2010 4:17 AM by Oliver

# re: Doxygen

We use Doxygen for our C# code as it supports both the ugly verbose XML syntax and the lightweight JavaDoc style syntax. Enabling AUTOBRIEF as well means your doc comments can be largely non-invasive making the code stand out more - which is how it should be.

www.stack.nl/.../docblocks.html

Friday, April 23, 2010 4:00 AM by Chris Oldwood

# re: Documentation with Sandcastle - a notebook

Thanks Jon. Yes Sandcaslte + SHFB is used by most customers genrating API documentation.

We do have an option of generating website out of Sandcastle. I have documented it here blogs.msdn.com/.../documenting-apis-with-sandcastle-using-windows-powershell.aspx

We will be releasing a new version of Sandcastle to support VS 2010 (.NET 4) help (MSHC) format (Help Viewer 1.0/Help3).

Please feel free to contact me directly at AramATmicrosoftDotcom. Cheers.

Anand..

Tuesday, April 27, 2010 7:05 PM by Anand Raman