Leaning Into Windows

As I face the upcoming release of Visual Studio 2010, I am contemplating a machine cleanup. It’s been six months, my drives tend to gather a lot of junk.

And I’ve rejected the notion of rebuilding my drive. It takes time I don’t have, and far worse, it’s scary and I’m trying to remove scary things from my life. It’s scary because it means I have to lean out on my safety line. My safety line is my backups, and yes I make sure the knot looks good. But my Dad once leaned out on a safety line in a tree stand and, well, let’s just say we’re glad he’s still here.

So, the options as I see it are what I really want – a new T400s, with dual monitor support, multi-touch, solid state 256Gb drive, etc. Well, OK, let’s get real I don’t have a spare 3 grand, and my mechanic says I need a new car – what’s with that?

The option I can live with is a new 320GB 7200RPM drive for my current T400. That’s $120 bucks. I’ll slide my current drive (same specs), into the ultra bay thingee where I yanked out the DVD (who needs a DVD?), slide the six month old drive into a SATA USB enclosure I also just ordered, and live happily ever after. When I did this six months ago, it was to finally move up from my 5400RPM drive. This time, it’s

just to save some time and hassle.

There’s a catch though. It’s a bigger waste/carbon load on the planet. Thus, I need to ride my bike to the gym every day (well, every day I go) from now until the snow flies to earn back my karma points.

Anyway, with the Visual Studio 2010 launch a few weeks away, you should decide now if you’re going to cycle drives or rebuild your drive in time to get it ordered.

And yes, I’m working on a bluegrass song with the title of this post.

I’m writing this partly because I think there must be a happy solution and I just can’t find it.

I love immutability. Not all the time, but many times. I’d like to identify values which either because problems when they are changed, or they just do nothing. Of course reference variables generally need to be immutability, but they can also generally be initialized without any data – for example you can set a collection variable to an empty list in the declaration or the constructor.

The problem occurs with data happy classes – classes with strings or dates or integers or strings – where it is either dangerous or useless to set the values.

In traditional coding where you explicitly call a constructor, all is happy because you can create read only variables. You can assign to these variables in the constructor and they can’t be assigned to again.

But enter composition, MEF, IoC, whatever. Now, how do you manage those immutable values?

I’ve come up with four solutions:

a) Provide an Initialize method with positional parameters

b) Provide an Initialize method with optional named parameters

c) Write ugly code that tracks values and issues runtime errors

d) Provide a second interface that allows the sets

e) Use a singleton factory strongly bound to the actual class

f) Forget immutability

I’ll veto c for my work. Not only is it extra stupid code everywhere, but it throws runtime, not compile time errors.

If you need things to be “kind of, sort of” immutable, the interface trick would work. In some cases you might be able to limit visibility of the interface. But it’s not immutable.

I hate initialize methods. You have to remember to call them and its extra thinking. Without optional/named parameters they are extremely fragile to parameter list changes. And, it would be so weird to call Initialize twice on an object that I’m OK with the runtime error for that, and it can refuse to reset the values so you have nearly true immutability.

It’s not true immutability because the variables must be writable from within the class – for example, you could use private setters. This means within the class, the values can be changed rendering them less than fully immutable.

You could solve this problem if you created custom factories for every class you would instantiate items of. Instead of retrieving the new value via composition/MEF, retrieve the singleton factory. The factory could be hard wired to the actual class and thus directly call the constructor. A method like “Instantiate” could take as many parameters as needed, and could use optional and named parameters to avoid parameter list fragility issues.

This leaves me less than enamored with PartInitializer. It encourages us to skip immutability or to take shortcuts that leave our classes less than fully immutable. I’m not enamored with custom factories either. They are only a few lines of code, but they barf an implementation detail all over our code. Significant ceremony.

I’d beat up on the MEF team if I could figure out a way they could solve this with minimal ceremony.

Anyone have a great solution?

I’m not convinced this error is nearly as catastrophic as Silverlight made it out to be, but I thought I’d share anyway. I received the following error while working before breakfast, I’m blaming it on the lack of Wheaties in my system.

? e.ExceptionObject
{System.Exception}
    Data: {System.Collections.ListDictionaryInternal}
    InnerException: Nothing
    Message: "Catastrophic failure (Exception from HRESULT: 0x8000FFFF (E_UNEXPECTED))"
    StackTrace: "   at MS.Internal.XcpImports.GetClassFullName(String coreClassName)    at System.Windows.DependencyProperty.LookupAttachedCoreProperty(String propertyName)    at System.Windows.PropertyPathParser.GetDpFromName(String name, Boolean namespaceMappingAvailable)    at System.Windows.PropertyPathParser.ReadAttachedPropertyStepDescriptor(Boolean calledFromParser)    at System.Windows.PropertyPathParser.ReadStepDescriptor(Boolean calledFromParser)    at System.Windows.PropertyPathParser.Parse()    at System.Windows.Data.Binding..ctor(String path, Boolean calledFromParser)    at MS.Internal.FrameworkCallbacks.CreateBindingExtension(String constructorArgument, IntPtr& nativeOutValue)"

From searching the web I found a number of things that might cause this. Apparently it’s Siverlight 3’s way of saying “you confused me”. I didn’t find any issues relating to my problem, which happened to be on a date picker.

I had this code:

<controls:DatePicker Grid.Row="1" Grid.Column="0"
                     Name="StartDate"
                     DisplayDate="{Binding Values(0), Converter={StaticResource ObjectToDateConverter}}"/>

Did you catch it?

It was early, it was a VB project, I didn’t notice that I had reflexively used parentheses instead of square brackets on the binding to the Enumerable member.

I was trying to debug a Silverlight application when I encountered the error

“The debugger cannot continue to run the process. Process was terminated.”

Happily I found a thread that discussed this issue because it was certainly not what I was expecting or looking for.

I was touching a property which called an initialization process in its Get accessor (did I mention this was a really, really bad idea). I had added code that touched the property in an overriding method that was called from the Initialize method.

Unexpected recursion.

I can’t say whether there are other things that can cause this error, but if it occurs I suggest that you check carefully for recursion.

One of the reasons this error can behave bizarrely is that it may occur because of recursion as one of the automatic watch windows such as Locals is displayed.

Many mixed feelings about posting this on the first day of the MVP summit. I got this email from Bruce McKinney this morning:

We haven’t talked for a while. I thought I’d let you know that I just posted a version of our discussion of VB.NET on my web site. Check out:

www.pobox.com/HardcoreVB/vbnet2.htm

How far out-of-date is this description of VB.NET in 2008?

It’s bizarre to think that most of the people on the VB Team have probably never heard of Bruce McKinney. I’m going to post this now because I received it now, and delaying it would be as much editorializing someone else’s post as deliberately timing it would have been. I did not. It came out of the blue this morning. Had I wished this to be main topics at the summit, I would have tried to get these things into our conversations weeks ago. I do not.

Obviously many of you do not know who Bruce McKinney is. Perhaps you don’t know who Dan Appleman is either. Perhaps you’re a C# programmer and think you just don’t care. But merging business programming and platform power started with VB (we’re comparing with Dbase and C++ in this time frame). The success of this merge is part of the history of C#, part of why the market existed in 2000. Bruce McKinney and Dan Appleman (and a few others) are the ones who proved that power mattered in such languages. They proved that business programmers could do great things and that they needed a powerful language to do it. Thankfully VB.NET/C# moved us an order of magnitude closer to the metal and allowed us to do the amazing things we do today.

That’s what Bruce meant to our collective history. Do you care what Bruce meant to me? As I look at the future and the work I’m doing to provide easier development and the thinking I’m doing to about the next level of abstractions, Bruce and Dan remain prophets. We can’t separate from the metal – which is why my current mantra is that we need to abstract application writing while not abstracting debugging and not hiding what’s really going on. Code still matters. Dropping down from an abstraction to the metal still matters. Anyone for in-line DSL? In line 3GL in a DSL? I believe understanding 3GL will eventually become as unnecessary as understanding assembly is today, but we can’t get there with a declaration or in a single jump or just because we want to get there. It’s a decade. We can only get there with a long iterative process that will require we go to the metal, understand why we went to the metal, and iterate again. Unlike VB6, we sit on an amazing platform where this process will work. .

If you still can’t quite recall where you heard that name before – Bruce McKinney wrote Hardcore Visual Basic which was on the bookshelf of most of the people that pushed the edges of Visual Basic. And if you read Bruce’s piece, remember this is one of the smartest guys there was in VB6.

My debugging strategy is to develop a hypothesis or two, pick the easiest most valuable one to test, test it in the easiest manner, and go on. I’m working to disprove the hypothesis because this is far easier to do and helps avoid me getting stuck on a particular notion of what is wrong. A test is more valuable if it cuts out a greater portion of the possibilities for failure, which John Robbins calls the divide and conquer approach.

I bring this up to explain that I rarely use the reporting MEF provides to solve stable composition errors. Maybe one day I will, but its much faster for me to find the missing part and begin adding the “AllowDefault=true” parameter to the Import attribute. I’m not being chaotic or random, it just works for me.

This morning I remained a bit stumped by a stable composition error. I realized it was a stable composition error because it went away when I said “AllowDefault=true”. But, my own reporting and other tests showed that the implementation was clearly in the container. So, what was going on.

A passing hypothesis flitted through my head “what would happen if there were two?”

It was a quick test to put a break point right after composition competed and request the exports for the part. Voila there were two!

Two tips:

- Stable composition fails when cardinality is not correct. A big word say Import fails when there are zero or more than one match.

- If you have multiple discovery directories, it is very easy for common assemblies to wind up in more than one directory. This results in multiple matches, and why I had the cardinality failure.

VS 2010 is nearly out the door, so it’s time to start fantasizing about killer features in the next version of .NET and Visual Studio.

The feature I want to see is “embedded DSL.”

Like many killer features, success comes from doing in a great way something you can already do halfway. So, I’ll demonstrate this feature in relation to T4 and a MEF scenario where it’s useful today. But before I get there…

I want to see DSL embedded in a language – maybe VB since it oddly enough has become the platform for experimentation, and since we’ve already started embedding other stuff in the language (XML). DSL implies particular syntax issues, but ultimately its metadata for code generation. I want to do code generation that is under my control in a fragmented way within the scope of normal code. Here’s a potential syntactic example:

Public Sub New()
‘ Normal Code
End Sub
{PropertyPattern}
{Property Name=”DisplayName” Type=String}
{Property Name=”DataName” Type=String}
{End PropertyPattern}

‘ More normal code

In this simplistic DSL, the Property metadata is just a pretty syntax for filling data into an interface. Wait, wait, you can’t stuff data into an interface. Right. You need an implementation which you can discover in real time (MEF anyone?) and that implementation can provide defaults for all the other values! And an include pattern here should let you reuse metadata defined once in your application.

PropertyPattern refers to a generation template – like an extended version of T4 – that can output real code in response to the metadata that’s passed. It’s an extended version because it’s strongly typed to the metadata defined via the metadata/DSL interface. The pattern is also discovered (MEF anyone?), which, well just trust me on this, allows a full governance model (customizing and governance on templates is one of my specialties, but let’s not geek out on that right in the middle of a hot fantasy). In simple terms, the governance model means a fallback mechanism through project (assembly), to group, to organization, to defaults/in the box.

Before I go further, let me say this is not a replacement for application generation. Application generation and code generation aren’t the same thing. Application generation is either a closely linked set of templates where the interrelations are as important as the templates, or application generation is architecture generation which is a new and emerging field.

Back, to the fantasy, because it’s an important part of a bigger picture of changing how we write applications…

Is this fantasy just Kathleen on too much MEF?

Actually no MEF required at all this morning… let me show you how to do this today – no compiler changes, no new dependencies, you can do this right now in VS 2008 (if you download the DSL Toolkit so you have T4, which you already did anyway, right?).

Well, OK, just a little MEF to get our morning started… the scenario is a MEF scenario, although there is no MEF in the solution. When you work with MEF and you want to create a MEF friendly interface for later discovery, you create an interface that a part will later fulfill, a separate interface of composition metadata, and a custom export attribute, which allows you to define a part just by implementing the first interface and tossing on the attribute. Whew! This means there is an annoying tedious (but technically elegant and necessary) pattern that I’ve implemented perhaps a billion times. What’s worse, the pattern obscures information about the interface, which of course I should also include in XML Documentation so I’m not in the code in the first place checking out the info – but see, more internal redundancy. And oh, by the way, if you screw up the pattern, the bugs can be very hard to track. Let’s fix it…

[NOTE: The implementation jumped back and forth between VB and C#. In VB, I’d implement this as XML literals which makes a more concise syntax with way less code in the included template, but T4 barfed at the XML literals and I didn’t feel like bothering with it today. Without XML literals, C# has better syntax because of the collection initializers which don’t make it into VB until 2010. There is no relation between the syntax of the DSL and the output syntax. I happen to be outputting VB.]

I created a T4 template. If you’re in VS2010, this is a normal Text Template, not a preprocessed one. If you’re in 2008, create a text file and give it a .tt extension. Also copy in the MefInterfaceDsl.t4 file that I’ve attached into your project. (I tested in VS 2008, in VS 2010, you may need to change the extension to .tx) Yes, your DSL will be written in T4 and yes, your DSL T4 template will have a different extension than the supporting one. That’s because Visual Studio outputs code for a file with a .tt extension and does not for one with a .t4 extension. You want output only from your DSL T4.

Here’s the template, then more talk:

<#@ template debug="false" hostspecific="false" language="C#v3.5" #>
<#@ assembly name="System.Core" #>
<#@ output extension=".vb" #>
<#@ include file="MefInterfaceDsl.t4" #>
<#=
         new Interface()
         {
            Name = "ISearchModelBase",
            Scope = Scope.Public,
            CompositionInfo =
            {
               new Property() {Name="TargetType", PropertyType="Type"}
            },
            Members =
            {
               new Property() {Name="DisplayName", PropertyType="string"},
               new Property() {Name="DataName", PropertyType="string"}
            }
         }.Output()
#>

Wow, is that really a DSL? Yes, but I won’t claim it’s a very good one. It’s a hack to allow the concept to work in VS 2008 and VS 2010, in hopes that we can get an elegant syntax in VS 20Next.

The initial four lines are a necessary T4 distraction. I’ve stated that the template language is C# with 3.5 extensions (not necessary in VS 2010). I’ve included core because it makes 3.5 work. I’ve stated that output will be in .vb. The actual syntax for the output is in the include file. Remember this is a working sample, if you wish to use this, you’ll want to enhance the MefInterfaceDsl.t4, including rewriting it to output C# if that’s your current flavor preference.

The include file has a class named Interface, Property and a few others. Initializers and collection initializers build the graph for the ISearchModelBase interface which has two properties, and one composition metadata property. The Interface class has an Output method that returns a string with the code output. Visual Studio places this output in a dependent file (select Show All Files to see this in VB). I included it below so you don’t have to run a project just to see the output.

Since the artifact is generated, I don’t have to remember the pattern and I’ll never be bit by forgetting to set AllowMultiple=false. Since the DSL/metadata is in the project beside the artifact, I can find an work with it (this would not be appropriate for application generation DSL/metadata whose artifacts spanned many projects, solution, and platforms).

So why is a normal T4 template a DSL? Because a DSL is a way to define generation information (metadata) in a way that is friendly to the human, followed by artifact generation.

I want this extended to be a true embedded part of the language to avoid these limitations and supply these features (and probably a bunch more stuff I haven’t thought of):

• - Can be any part of any normal code file
  • Not limited to entire files (although the T4 output can be partials)
  • The DSL/metadata is a holistic part of the code
• - Artifact patterns (templates) are discoverable (can be anywhere by anyone)
• - DSL/metadata patterns are discoverable (default values and pattern extensions
• - Intellisense on the DSL
• - Better syntax (drop the new and other class residuals)
• - No reliance on file location (the T4 support file must be in relation to your project)
• - No ugly opening stuff about T4 unrelated to the task at hand
• - Standard extensible metadata/DSL patterns provided
• - Standard extensible artifact patterns provided

Here’s the output:

Option Strict On
Option Explicit On
Option Infer On

Imports System
Imports System.Collections.Generic
Imports System.Linq
Imports System.ComponentModel.Composition

public Interface ISearchModelBase
   Property DisplayName As string
   Property DataName As string
End Interface
public Interface ISearchModelBaseComposition
   Readonly Property TargetType As Type
End Interface

< MetadataAttribute() > _
< AttributeUsage(AttributeTargets.Class, AllowMultiple:=False) > _
public Class SearchModelBaseAttribute
   Inherits ExportAttribute
   Implements ISearchModelBaseComposition

   Public Sub New( ByVal targetType As Type)
      MyBase.New(GetType(ISearchModelBase))
      _targetType = targetType
   End Sub

   Private _targetType As Type
   Public Readonly Property TargetType As Type Implements ISearchModelBaseComposition.TargetType
      Get
         Return _targetType
      End Get
   End Property

End Class

I got bit with an issue that I’d love to spare you.

For a short time MEF Preview 8 was posted with a fairly serious bug for Silverlight programmers using PartInitializer.

The way PartInitializer should work, and works now, is that all exports within the current XAP should be discovered by default.

When MEF Preview 8 was first posted, it included a bug which resulted in the XAP not being searched for exports – no exports except those explicitly placed into the container were discovered. The incorrect version was up for a handful of days. Of course luck would have it that I downloaded during that time, and a spasm of insecurity led me to believe I’d forgotten how to use MEF.

Here’s Glenn Block’s discussion.

I demo’d code contracts in my session at DevConnections. When I did a Camtasia video of the talk I found that they were not working. This led to a frustrating few hours, as I really wanted to complete the recording. Let me explain what I think happened in hopes of saving you some time. Note that this may be entirely different at VS 2010 RTM.

First, the reason they were not working was because runtime checking was turned off. The normal way to turn runtime checking on is through a tab called Code Contracts which appears on the Project Properties dialog. But alas, I had no such tab.

There is a download for code contracts which looks like this on the code contracts page.

Perhaps you would have thought this was intended for VS 2010 Beta 2, but I did not. I also thought I had already installed this because I’ve previously played with contracts in VS 2008, but apparently this didn’t get reinstalled after a machine rebuild.

At Julie Lerman’s suggestion, I reinstalled the Code Contract development tools and the tab appeared. This allowed me to turn runtime checking back on and voila see the failure.

Code contracts are very cool and I hope they will be implemented a bit more cleanly in the final release of VS 2010.

I got the following error with Beta 2.

I do not know what is relevant, but the project back-targets .NET 3.5 and is a project that includes T4 preprocessed templates. Neither of these should matter in relation to this error:

Error    4    Unable to write to output file ‘[[full project diretory]]/\obj\x86\Debug\T4CompiledTemplates.pdb': Unspecified error     T4CompiledTemplates

(with [full project directory] replaced with the actual directory.

I tried the obvious – unload the project, unload VS, restart the system. None worked.

The solution for me was to create a brand new project (with new name and directory) copy the files, reset the references, etc.

I rarely lobby publically for feature changes. But as we get closer to Beta 2, the feature lockdown becomes tighter. MEF is in the core of the framework, so any changes need to be carefully considered. I know that. But this change is small, testable and very important. 

And Glenn has told me that its unlikely to happen, partly because I seem to be the only one asking for it. I think that’s just because you don’t know about it yet.

You need to help me make this happen.

The attributed model of MEF is the one you will almost certainly use in the VS 2010 timeframe. In this model, you adorn classes that you want in play in your container with the Export attribute. This is MEF specific attribute and a separation concern guideline is that you want to expose as little as possible about your technology, except where you need it. So, MEF Preview 6 came out with a very slick solution. The ExportAttribute class is unsealed. You can inherit from it and create your own attributes, named whatever you want. If you have a class which exports as an IProcess you can have a ProcessAttribute adorn that class. This makes your class more readable and moves the type your exporting (IProcess) into the base class where no one needs to remember it. This can be important when you have more complex derivation stacks.

This is especially valuable with custom metadata attributes. Let’s say my process needs a name, perhaps so it can be displayed in a menu. MEF metadata allows you to add this information, managed at the manifest level so your class isn’t fully loaded until needed. Custom metadata is very valuable and by adding a MetadataAttributeAttribute (yes, that’s the name) to the class derived from ExportAttribute, you give your ProcessAttribute a nice name, and the data required for MEF to do its job.

So far everything is very nice. Nick describes this briefly in his post here.

But it’s about to get very ugly.

Metadata is inserted into a dictionary for use by MEF. Your name is a string, so you expect to be access it as a string. You commonly MEF metadata via an interface, and a best practice prior to Preview 6 was to supply the same interface for your custom attribute (ProcessAttribute) and the call to retrieve the metadata.

This fails when you derive from ExportAttribute.

Let’s first look at why, then the simple fix that I think is so important to add, even if its late in the cycle.

ExportAttribute is a class that has an available attribute of AttributeUsage. AttributeUsage has a parameter called AllowMultiple. ExportAttribute quite reasonably has AllowMultiple set to true so you can export several interfaces at once.

MetadataAttributeAttribute is marking a class as supplying metadata information to MEF for us in a dictionary. This dictionary allows you to find out more information about your part before loading it – for display, filtering, custom prioritization, and other reasons. MetadataAttributeAttribute is applied to a class deriving ultimately from Attribute. It checks the AttributeUsage to see if the attribute allows multiples. If it allows multiples, MEF adds the values to the dictionary as an array of the declared type instead of the declared type.

OK, that’s a bit convoluted. Let me explain why this is really nice behavior. Let’s say you want to add metadata to category to any of your MEF parts. You create a CategoryAttribute class and give this class both the MetadataAttributeAttribute attribute and the AttributeUsage(AllowMultiple=true) attribute. Now you can supply Category as an attribute to any of your MEF parts and you can supply as many categories as you want to each part. This is very nice behavior.

However, we take these two very nice behaviors together we get a rather bizarre result. If you simply inherit from ExportAttribute the way Nick describes, you’ll place your metadata into the metadata dictionary as an array, and you will only be able to successfully retrieve the metadata with an interface that uses an array. The attribute you declare and the way you retrieve the metadata will not be able to use the same interface. This will be totally unexpected, you will get a rather cryptic error, and I hope you find this blog post so you spend somewhat less time than the 20 hours I did. You will also be able to place the attribute on multiple times – this is almost never what you want because you placed the interface you were exporting into the attribute – in other words, the ProcessAttribute is identifying things that export IProcess. You don’t want to repeat this.

Thanks for sticking with me through that. This is arcane knowledge that puts a twisty mountain pass between you and the pit of success. Instead I want it to “just work” the way you expect.

Now here’s the itty bitty fix that to solve this problem….

The MEF core needs to add a new attribute class – let’s say “ExportCustomBaseAttribute” for this discussion. This attribute inherits from ExportAttribute and does only one thing. It resets the AllowMultiple parameter to False. All you need to know is that you generally inherit from ExportCustomBaseAttribute instead of ExportAttribute. You will find this attribute adjacent to ExportAttribute in Intellisense. It gives a very nice place in the documentation to explain this problem for the folks that wish to understand it. Alternatively, this attribute could be called “ExportSingleAttribute” or a variety of other names. It might be possible to make it abstract if avoiding direct usage is desirable.

The benefit is that you do not need to understand the entire scenario described in this blog post to be able to create readable custom attributes that tuck the MEF dependency away behind the scenes.

But this isn’t going to happen unless we tell the team we want it. Comments on this site have been a bit ditzed. If you have a problem with posting a comment here (or you just want to bug Glenn instead), please place a comment here in Glenn’s Preview 6 post or here at Nick’s

I just posted this on my AppVenture blog on preprocessed T4 templates

I love it when Glenn and the MEF team make me wrong! Because that means we get something new that I didn’t think they could pull off

First it was Stable Composition.

Now it’s Open Generic Composition.

A closed generic is something like this air code: Public Class x These are the only types of generics I anticipated MEF would support, at least in .NET 4.0. I’ve said this in a number of talks, and I’m happy to say that information is not wrong..

An open generic allows you to expose Foo(Of T) and allow the specific class to be specified by the import.

Note that this is not intrinsic to MEF itself but an alternate catalog. So, it’s not only cool in and of itself that we can now do Open Generics, but it’s a hint at how powerful MEF really is as we move beyond what’s delivered in the box. I’ll let you read Glenn’s post for the details.

What blows me away is that MEF is just starting. I think that’s what amazes me most. It’s like a seed just sprouting, or a kernel we’re just beginning to imagine how to harness. I’m excited about MEF in NET 4.0, but I also think more cool things like this approach to Open Generics will unfold as more people explore MEF.

I’ll follow up with some cool uses I’ve got in mind after I work through the details another day.

You can check here for a quick description of MEF

I'm giving a half dozen MEF talks this summer and I'm frequently asked "what happens if a part isn't available". The old answer was "the system crashes, how could it do anything else?" This conversation definitely deflates the upbeat mood of a MEF talk. Recently, MEF has changed, making that answer obsolete.

MEF is a composition container which satisfies imports by tracking down associated exports. The correct number of exports to satisfy each import is called the cardinality and can be one, zero to one or zero to many. Thus an import can fail because there are too few or too many matching exports.

In the MEF previews 1-5 (inclusive) and in the Visual Studio CTPs and beta1, MEF throws an exception when a failure occurs.

MEF Preview 6, released last Monday, introduces "stable composition." With stable composition, the container can know about, but not expose parts. If a part fails on a cardinality rule, the MEF container remembers the part, but keeps it hidden. If additional composition occurs (such as through the Refresh method) additional attempts to fully compose the part occur. If its cardinality is fulfilled, the part becomes available for additional composition. You can think of this as "if a missing sub part shows up later, the containing part will become available."

There are both good and bad aspects of this, and it definitely affects how you think of and write your MEF systems. In general, it will make MEF apps more stable. If you design a plug-in model, and the creator of a plug-in fails to properly deploy (or a confused user deletes some but not all of a plug-in via File Explorer), your system will not crash. The containing part, which would probably fail if run, doesn't appear in composition. This makes your system more robust against errors that are beyond your control. It also allows the late composition strategy, although I'm not yet clear on good scenarios for it.

The down side is that you may have more challenges finding certain types of composition errors because you will not receive an exception - you need to catch the current state of the composition container. And if you don't consider this behavior when writing your app, you can get officially bad behavior.

In a plug-in design such as the directory composition model, any part in your system can fail on a cardinality (and could previously have crashed) because all parts can be made of other parts with dependencies you don't know about.

For example, consider creating a main menu as a part and that menu is made up of menu items and sub menus and one of the menu items cannot be composed due to a deeply nested cardinality failure. If each main menu item is a part and you import the menu items as a collection using the ImportMany attribute, you're fine. Your application will simply not display the failed menu item.As deep as the nesting goes, each layer that is a collection is naturally protected because cardinality failures just remove one part from the collection.

If instead you create your main menu to explicitly expect a particular menu item (such as a Tools sub menu) and that menu has a required Import for a part and so on down to a point of cardinality failure, then the failure cascades up the chain. This happens because the leaf cardinality failure means that part is not available, causing a cardinality failure at the next level. In this hypothetical case, the failure cascades all the way up to the main menu which does not now exist. The application either runs without a main menu, or the application fails because the main menu is missing.

You can avoid this by considering the intent of each import and providing appropriate protection. Some parts are optional and your application can run just fine without them. These should either be in a collection (ImportMany in recent previews) or not required. Other parts are important, but not important enough to cause their container to fail. These can be managed via asserts, and communication with support or the user. If parts are critical to the application running, then you need to check that they exist after composition and shut down the application as gracefully as possible.

I think this is a good change, but at least until more patterns emerge, you need to consider what would happen if any import is not successfully satisfied. Should the containing part also fail to be composed?

MEF is a very sharp knife. It cuts up the tomatoes and carrots really well and we can have a dandy stew. Or we can wind up in the emergency room. Understanding MEF is important to safe MEF use.

You can find more on the Preview 6 changes in this post by Nicholas Blumhardt who is a member of the MEF team. I'd like to thank Glenn Block for his discussions with me on this change.

A couple of times recently I've wanted a super short summary of what MEF is and why you care. I thought I'd share what I came up with here. MEF stands for Managed Extensibility Framework.

MEF allows you to compose parts at runtime to alter application behavior based on the current environment. The most common way to alter behavior is to include dll's in a directory. The most common way to code MEF is through attributes. Both of these decisions are flexible because MEF itself is extensible and can work in many different ways. The Visual Studio 2010 IDE uses a good deal of MEF. You can use MEF for plug-ins for your application, improve testability, and to create a system of small replaceable parts. You may find other creative ways to use MEF.

I don't know whether MEF is addicting, but composability definitely is !

A few friends have pointed out that my post discussing my disappointment with Workflow 4.0 was somewhere between difficult and incomprehensible if you have no idea what's actually changed. Maurice posted this which gives some additional background.

Well, to me it is vastly disappointing. Back in January, I said that I thought Workflow 4.0 would be the most important feature of .NET 4.0. I was completely wrong. WF 4.0 will be a rather uneventful part of the .NET 4.0 release.

My initial projection was partly because .NET 4.0 will (thankfully) be a rather quiet release at core with only limited new language features so the competition was never all that steep. But it was also because the WF in the CTP represented an important change to Workflow. Microsoft screwed up WF 3.0 and 3.5. It was excessively complex for the marketplace. They implicitly acknowledged this when deciding to scrap the entire thing and start over with WF 4.0. While some people do have deployed applications, WF 3.0 and 3.5 could not be the ubiquitous WF I think we all need in our toolbox. I was hoping WF 4.0 would be that tool.

The bad news is that WF 4.0 probably won’t be. WF 4.0 suffers from two problems, one implied by the other. The first problem is that it has no state machine. At least it’s not in the beta and I have heard no contradictory announcement. The second is that if they could not even provide basic functional parity with WF 3.0/3.5 how could they possibly have had their development act together/sufficient resources/fill in the blank to produce a great 1.0 product. Remember, with the rewrite, WF 4.0 is really a 1.0 product.

I hope I’m wrong on the second part of this and that they’ve covered an adequate range of real world scenarios that it will at least be solid. And WF 4.0 will probably be an interesting tool in the core scenarios, particularly those around WCF integration. I’m not saying it’s useless by any means. I’m just saying it’s probably not going to change your world.

I’m still angry about the lack of a state machine. Certainly WF 3.0/3.5 state machines will still work as well as it ever has, and if you can manage deployment and avoid the layout being trashed by the IDE (hint, keep your layout files ready for restore from source control or edit them to find your lost states). But I’m angry because you and I can’t give our customers new products that just ignore half the scenarios for the old product. We can’t toss out all the work our customers have done (interop is present but limited). We just can’t do stuff like this!

I know the WF team has been working hard and that there are really smart people on the team. I don’t know what went wrong, but the problems with WF 4.0 became the problems of the early adopters, and shafting early adopters (especially when two years still involves “early adoption”) is very bad business.

In fairness, they offered a flowchart solution which will on paper accommodate some of the scenarios where state machines made sense. It’s just the wrong solution, and a horrible decision to do something new while ignoring previous functionality - state machine should have had a higher priority.

The vast majority of non-trivial workflow scenarios are best solved with a state machine. It’s based on a quick analysis. Theoretically all state machines can be expressed in flowchart terms and all flowcharts can be expressed as state machines. So in theory as long as you have either you’re covered. But it doesn’t quite work out that way. Business systems have non-trivial numbers of states. As the number of states increase, the number of legal paths between them increases. The theoretical increase is huge, but unimportant because many states are illegal. However, a sufficient number of legal states remain that as systems become more complex, the number of paths in a flowchart increases very fast. Paths are added more often than states (paths are the most common unexpected side cases). Thus, the system also remains more stable if the initial analysis is against states, not paths. Finally in many business systems, thinking of the problem in terms of states simplifies analysis.

The good news is that WF isn’t the only new thing in .NET 4.0 and in my next couple of posts, I’ll talk about what I now think are some of the most important features of the next release.

I got a Twitter on MEF resources, so I wanted to at least post the resources slide for my MEF/MAF talk. My big question, is what else do you want. Do you want more technie stuff on the core, more extensibility details, more stuff on the WPF editor, more on scenarios and value proposition, to know why I think composability is the most important thing since OOP/OOD (yep, I rank it above generics), or something else entirely? I think there's at least a half dozen people trying to get the right stuff out on this, but we don't need to shotgun this.

I actually modified this a bit if you were in the talks, but it’s a start – definitely not the final word:

• http://codebetter.com/blogs/glenn.block

• www.msmvps.com/blogs/kathleen

• Ask Kathleen, www.VisualStudioMagazine.com

• April 2009 for MEF/System.ComponentModel/Composition discussion

• April 2008 for MAF/System.AddIn discussion

• Hanselminutes: www.Hanselminutes.com

• #148 About MEF –Glenn Block

• #152 March 5, 2009 – Kathleen Dollard

• #147 VS 2010 WPF editor interview

• Also, good Bob Martin interview

• DotNetRocks: (me #436 April 9, 2009 -also, #304, 171, 121, 63)

• MEF download - www.codeplex.com/MEF

• MEF provider model and other interesting Andreas/TheCodeJunkie work www.codeplex.com/MEFcontrib

• PDC Videos

• TL49 (MEF-ify BabySmash)

• TL33 (Glenn Block) at PDC

• Pipeline Builder download: www.codeplex.com/clrAddin

• Kids, don’t try MAF without this

• Channel 9 video on MAF: Daniel Moth (search for System.AddIn)

It’s not up yet. You may have noticed. And if you downloaded what we’ve currently got up at AppVenture, you probably noticed that it was nothing like what I discussed on Hanselminutes or DotNetRocks.

Why is it delayed?

Primarily because I have had a very hard time drawing a line around what should and should not be in the tool. I’m still having a hard time of that, and the first version won’t include everything I want.

Also, MEF is a new paradigm. It’s been rough at times to learn to rethink and debug differently. This has caused a few redesigns along the way, and lots of consideration for things like the large number of assemblies. These are things I think need to be pretty close to correct in the first go round. This process would have been much more difficult without Glenn Block’s help, and I really appreciate his help and friendship.

I began many parts of this project in VB. I love VB. But the open source aspect of the future of this project will involve primarily C# coders. That’s the world around me. I am happy in C#, I just went fairly far down the VB path and had to do a lot of translation. When I release it, it will be mostly C#, with some remaining VB in parts I want to rewrite (the UI) and places where VB offers benefits (XML literals).

I’ve also run into a half dozen really nasty scenarios, both in my code and the framework that took a few days to resolve.

Parts of this have been raw research. What should such a tool look like? I’m convinced it’s by far the best way to generate code, but the devil’s in the details. I think it will make it easier for you and others that I worked though these details, although as I look back at the simplicity of many parts of the solution I have plenty of self-doubt that I just started out stupid.

All these things take time, a lot of time. And I try occasionally to have a life, although the people in it would certainly argue my capacity for setting this aside.

When? I hope next week.

The purpose of MEF is to hook up parts. Parts are indentified by string contracts.

MEF (and tools like it) empower fully composable systems. I use that term to describe a system in which the system no longer knows what is going on. What you write empowers an ecosystem specific to a problem domain it does not solve the problem.

In my case, MEF empowers a harness where processes, generally code generation templates, are run in a specific manner (such as with pre and post processing). I don’t care what your process does. At one level I don’t care about what services it needs or what metadata it uses (although in another post, I’ll explain why I actually do care, the harness definitely doesn’t care). This is a tremendously powerful environment for certain problems, precisely because it becomes an ecosystem solution, not a proscribed solution.

It is also a very different environment where we’re still working out the rules as an industry. One area where the rules shift is in assembly organization. There are always two core drivers for assembly (project) organization – deployment and isolation. How does this play out in a MEF system?

Let’s assume UI isolation. MEF systems generally solve a complex but finite problem. It makes sense for them to be class libraries that are distinct from any single user interface. In the code generation case, we know we will have a graphical UI and a command line. I suspect I’ll have a kind of sucky graphical UI and someone will make a pretty one and starting in 2010 embedding in Visual Studio becomes more realistic. That’s 2 in any single deployment - one of the UI’s and a class library that is the core of the system.

A core goal of a composable system is to replace direct references with indirect references. This means interactions will generally use interfaces which belong in their own library for isolation. These interfaces will generally have a default implementation. While technically these defaults could reside in either the interface or the core library directory, putting them in either location means they will always be available and you will have to work harder for the user to replace them. Thus I think four class libraries is the minimum complexity for a real world MEF solution.

Almost all fully composed systems should go to at least one additional set of assemblies. Fully composed assemblies contain interfaces which are directly referenced and used by the system core and interfaces that do not have this direct interaction. It’s software entropy if I force you to design an interface for extracted database metadata. I’ll give you one, although the core system could care less about it. For isolation and clarifying intent, I think separation is a better approach.

You could place the default implementations for indirect services (such as metadata extraction) into the same assembly as the default implementations for services directly referenced by the core. Nothing will break. But they will always be in the same catalog and export provider, requiring the complexity of selecting the correct one. Deployment is simpler if these are separate. I also think a parallel between implementation and interface assemblies makes the system overall easier to understand.

In some cases, including the code generation harness, there are specific sets of roles parts can play. A given use of the harness may use any combination of these. If the assemblies are partitioned across these boundaries, people setting up the MEF system can employ only those items they need. Among other things, this speeds the MEF discovery process. For example, in the code generator, there are a couple of categories of metadata, output, naming and miscellaneous services.

The specific roles the parts play is defined via the interface assemblies. If these assemblies are partitioned, programmers extending the system later reference the specific set of interfaces they want. With partitioned assemblies, they might need to reference more than one, but once referenced, the complexity of what the programmer sees is less (which could also be done with interfaces).

Let’s explore the role of the extending programmer a bit more. If there is no extending programmer (you or someone else) there is no point in a composable system. This extending programmer might be part of the original team, if the purpose of composition is simply flexibility in deployment, such as whether particular modules based on customer status. In many cases, including the generation harness, the system is really being created for the convenience of the extending programmer, and it is there scenarios that should drive the application design. You may be able to design your system so certain changes can be made without MEF knowledge (new templates in the code generation case) but the reason you’re using a composable system is to support programmers creating new parts.

This extending programmer works in their own assembly, which among other things allows you to issue updates to default assemblies without breaking their work. These assemblies reference interface assemblies. Is it easier for that programmer to select well partitioned interface assemblies and use them to better understand how the system works. If you partition your implementation assemblies to parallel the interfaces, deployment is simpler and if you fully implement an assembly, you can remove the default one.

There are a few blog posts including this by Chad Myers and this by Davy Brion that speak disparagingly about solutions with many projects. I do not actually agree with either post (because solutions are convenience only features and our architectures should not reflect Visual Studio bugs) but I do think they make some good points. And the fundamental agreement is that you should understand why you have a lot of projects if you do. In this case, it’s for isolation (a core tenant of MEF) and deployment. If you want only a couple of projects in your solution, you don’t want a composable system (although you might still use MEF for other reasons).