Leaning Into Windows

Code Contract Tab Missing in Project Properties in VS 2010 Beta 2

Kathleen — Tue, 17 Nov 2009 17:22:27 GMT

I demo’d code contracts in my session at DevConnections. When I did a Camtasia video of the talk I found that they were not working. This led to a frustrating few hours, as I really wanted to complete the recording. Let me explain what I think happened in hopes of saving you some time. Note that this may be entirely different at VS 2010 RTM.

First, the reason they were not working was because runtime checking was turned off. The normal way to turn runtime checking on is through a tab called Code Contracts which appears on the Project Properties dialog. But alas, I had no such tab.

There is a download for code contracts which looks like this on the code contracts page.

Perhaps you would have thought this was intended for VS 2010 Beta 2, but I did not. I also thought I had already installed this because I’ve previously played with contracts in VS 2008, but apparently this didn’t get reinstalled after a machine rebuild.

At Julie Lerman’s suggestion, I reinstalled the Code Contract development tools and the tab appeared. This allowed me to turn runtime checking back on and voila see the failure.

Code contracts are very cool and I hope they will be implemented a bit more cleanly in the final release of VS 2010.

Visual Studio 2010 Beta 2

Kathleen — Sun, 25 Oct 2009 14:10:00 GMT

I got the following error with Beta 2.

I do not know what is relevant, but the project back-targets .NET 3.5 and is a project that includes T4 preprocessed templates. Neither of these should matter in relation to this error:

Error 4 Unable to write to output file ‘[[full project diretory]]/\obj\x86\Debug\T4CompiledTemplates.pdb': Unspecified error T4CompiledTemplates

(with [full project directory] replaced with the actual directory.

I tried the obvious – unload the project, unload VS, restart the system. None worked.

The solution for me was to create a brand new project (with new name and directory) copy the files, reset the references, etc.

A Fix We Really Need in MEF for .NET 4.0

Kathleen — Mon, 07 Sep 2009 15:36:49 GMT

I rarely lobby publically for feature changes. But as we get closer to Beta 2, the feature lockdown becomes tighter. MEF is in the core of the framework, so any changes need to be carefully considered. I know that. But this change is small, testable and very important.

And Glenn has told me that its unlikely to happen, partly because I seem to be the only one asking for it. I think that’s just because you don’t know about it yet.

You need to help me make this happen.

The attributed model of MEF is the one you will almost certainly use in the VS 2010 timeframe. In this model, you adorn classes that you want in play in your container with the Export attribute. This is MEF specific attribute and a separation concern guideline is that you want to expose as little as possible about your technology, except where you need it. So, MEF Preview 6 came out with a very slick solution. The ExportAttribute class is unsealed. You can inherit from it and create your own attributes, named whatever you want. If you have a class which exports as an IProcess you can have a ProcessAttribute adorn that class. This makes your class more readable and moves the type your exporting (IProcess) into the base class where no one needs to remember it. This can be important when you have more complex derivation stacks.

This is especially valuable with custom metadata attributes. Let’s say my process needs a name, perhaps so it can be displayed in a menu. MEF metadata allows you to add this information, managed at the manifest level so your class isn’t fully loaded until needed. Custom metadata is very valuable and by adding a MetadataAttributeAttribute (yes, that’s the name) to the class derived from ExportAttribute, you give your ProcessAttribute a nice name, and the data required for MEF to do its job.

So far everything is very nice. Nick describes this briefly in his post here.

But it’s about to get very ugly.

Metadata is inserted into a dictionary for use by MEF. Your name is a string, so you expect to be access it as a string. You commonly MEF metadata via an interface, and a best practice prior to Preview 6 was to supply the same interface for your custom attribute (ProcessAttribute) and the call to retrieve the metadata.

This fails when you derive from ExportAttribute.

Let’s first look at why, then the simple fix that I think is so important to add, even if its late in the cycle.

ExportAttribute is a class that has an available attribute of AttributeUsage. AttributeUsage has a parameter called AllowMultiple. ExportAttribute quite reasonably has AllowMultiple set to true so you can export several interfaces at once.

MetadataAttributeAttribute is marking a class as supplying metadata information to MEF for us in a dictionary. This dictionary allows you to find out more information about your part before loading it – for display, filtering, custom prioritization, and other reasons. MetadataAttributeAttribute is applied to a class deriving ultimately from Attribute. It checks the AttributeUsage to see if the attribute allows multiples. If it allows multiples, MEF adds the values to the dictionary as an array of the declared type instead of the declared type.

OK, that’s a bit convoluted. Let me explain why this is really nice behavior. Let’s say you want to add metadata to category to any of your MEF parts. You create a CategoryAttribute class and give this class both the MetadataAttributeAttribute attribute and the AttributeUsage(AllowMultiple=true) attribute. Now you can supply Category as an attribute to any of your MEF parts and you can supply as many categories as you want to each part. This is very nice behavior.

However, we take these two very nice behaviors together we get a rather bizarre result. If you simply inherit from ExportAttribute the way Nick describes, you’ll place your metadata into the metadata dictionary as an array, and you will only be able to successfully retrieve the metadata with an interface that uses an array. The attribute you declare and the way you retrieve the metadata will not be able to use the same interface. This will be totally unexpected, you will get a rather cryptic error, and I hope you find this blog post so you spend somewhat less time than the 20 hours I did. You will also be able to place the attribute on multiple times – this is almost never what you want because you placed the interface you were exporting into the attribute – in other words, the ProcessAttribute is identifying things that export IProcess. You don’t want to repeat this.

Thanks for sticking with me through that. This is arcane knowledge that puts a twisty mountain pass between you and the pit of success. Instead I want it to “just work” the way you expect.

Now here’s the itty bitty fix that to solve this problem….

The MEF core needs to add a new attribute class – let’s say “ExportCustomBaseAttribute” for this discussion. This attribute inherits from ExportAttribute and does only one thing. It resets the AllowMultiple parameter to False. All you need to know is that you generally inherit from ExportCustomBaseAttribute instead of ExportAttribute. You will find this attribute adjacent to ExportAttribute in Intellisense. It gives a very nice place in the documentation to explain this problem for the folks that wish to understand it. Alternatively, this attribute could be called “ExportSingleAttribute” or a variety of other names. It might be possible to make it abstract if avoiding direct usage is desirable.

The benefit is that you do not need to understand the entire scenario described in this blog post to be able to create readable custom attributes that tuck the MEF dependency away behind the scenes.

But this isn’t going to happen unless we tell the team we want it. Comments on this site have been a bit ditzed. If you have a problem with posting a comment here (or you just want to bug Glenn instead), please place a comment here in Glenn’s Preview 6 post or here at Nick’s

Eleven Cool Things About Preprocessed T4 Templates

Kathleen — Fri, 04 Sep 2009 19:21:04 GMT

I just posted this on my AppVenture blog on preprocessed T4 templates

MEF Can Support Open Generics!

Kathleen — Fri, 21 Aug 2009 01:29:41 GMT

I love it when Glenn and the MEF team make me wrong! Because that means we get something new that I didn’t think they could pull off

First it was Stable Composition.

Now it’s Open Generic Composition.

A closed generic is something like this air code: Public Class x These are the only types of generics I anticipated MEF would support, at least in .NET 4.0. I’ve said this in a number of talks, and I’m happy to say that information is not wrong..

An open generic allows you to expose Foo(Of T) and allow the specific class to be specified by the import.

Note that this is not intrinsic to MEF itself but an alternate catalog. So, it’s not only cool in and of itself that we can now do Open Generics, but it’s a hint at how powerful MEF really is as we move beyond what’s delivered in the box. I’ll let you read Glenn’s post for the details.

What blows me away is that MEF is just starting. I think that’s what amazes me most. It’s like a seed just sprouting, or a kernel we’re just beginning to imagine how to harness. I’m excited about MEF in NET 4.0, but I also think more cool things like this approach to Open Generics will unfold as more people explore MEF.

I’ll follow up with some cool uses I’ve got in mind after I work through the details another day.

MEF and Cardinality Composition Failures

Kathleen — Fri, 17 Jul 2009 13:44:24 GMT

You can check here for a quick description of MEF

I'm giving a half dozen MEF talks this summer and I'm frequently asked "what happens if a part isn't available". The old answer was "the system crashes, how could it do anything else?" This conversation definitely deflates the upbeat mood of a MEF talk. Recently, MEF has changed, making that answer obsolete.

MEF is a composition container which satisfies imports by tracking down associated exports. The correct number of exports to satisfy each import is called the cardinality and can be one, zero to one or zero to many. Thus an import can fail because there are too few or too many matching exports.

In the MEF previews 1-5 (inclusive) and in the Visual Studio CTPs and beta1, MEF throws an exception when a failure occurs.

MEF Preview 6, released last Monday, introduces "stable composition." With stable composition, the container can know about, but not expose parts. If a part fails on a cardinality rule, the MEF container remembers the part, but keeps it hidden. If additional composition occurs (such as through the Refresh method) additional attempts to fully compose the part occur. If its cardinality is fulfilled, the part becomes available for additional composition. You can think of this as "if a missing sub part shows up later, the containing part will become available."

There are both good and bad aspects of this, and it definitely affects how you think of and write your MEF systems. In general, it will make MEF apps more stable. If you design a plug-in model, and the creator of a plug-in fails to properly deploy (or a confused user deletes some but not all of a plug-in via File Explorer), your system will not crash. The containing part, which would probably fail if run, doesn't appear in composition. This makes your system more robust against errors that are beyond your control. It also allows the late composition strategy, although I'm not yet clear on good scenarios for it.

The down side is that you may have more challenges finding certain types of composition errors because you will not receive an exception - you need to catch the current state of the composition container. And if you don't consider this behavior when writing your app, you can get officially bad behavior.

In a plug-in design such as the directory composition model, any part in your system can fail on a cardinality (and could previously have crashed) because all parts can be made of other parts with dependencies you don't know about.

For example, consider creating a main menu as a part and that menu is made up of menu items and sub menus and one of the menu items cannot be composed due to a deeply nested cardinality failure. If each main menu item is a part and you import the menu items as a collection using the ImportMany attribute, you're fine. Your application will simply not display the failed menu item.As deep as the nesting goes, each layer that is a collection is naturally protected because cardinality failures just remove one part from the collection.

If instead you create your main menu to explicitly expect a particular menu item (such as a Tools sub menu) and that menu has a required Import for a part and so on down to a point of cardinality failure, then the failure cascades up the chain. This happens because the leaf cardinality failure means that part is not available, causing a cardinality failure at the next level. In this hypothetical case, the failure cascades all the way up to the main menu which does not now exist. The application either runs without a main menu, or the application fails because the main menu is missing.

You can avoid this by considering the intent of each import and providing appropriate protection. Some parts are optional and your application can run just fine without them. These should either be in a collection (ImportMany in recent previews) or not required. Other parts are important, but not important enough to cause their container to fail. These can be managed via asserts, and communication with support or the user. If parts are critical to the application running, then you need to check that they exist after composition and shut down the application as gracefully as possible.

I think this is a good change, but at least until more patterns emerge, you need to consider what would happen if any import is not successfully satisfied. Should the containing part also fail to be composed?

MEF is a very sharp knife. It cuts up the tomatoes and carrots really well and we can have a dandy stew. Or we can wind up in the emergency room. Understanding MEF is important to safe MEF use.

You can find more on the Preview 6 changes in this post by Nicholas Blumhardt who is a member of the MEF team. I'd like to thank Glenn Block for his discussions with me on this change.

A Quick Description of MEF

Kathleen — Fri, 17 Jul 2009 12:58:41 GMT

A couple of times recently I've wanted a super short summary of what MEF is and why you care. I thought I'd share what I came up with here. MEF stands for Managed Extensibility Framework.

MEF allows you to compose parts at runtime to alter application behavior based on the current environment. The most common way to alter behavior is to include dll's in a directory. The most common way to code MEF is through attributes. Both of these decisions are flexible because MEF itself is extensible and can work in many different ways. The Visual Studio 2010 IDE uses a good deal of MEF. You can use MEF for plug-ins for your application, improve testability, and to create a system of small replaceable parts. You may find other creative ways to use MEF.

I don't know whether MEF is addicting, but composability definitely is !

Workflow 4.0, What's Coming, What's Changed

Kathleen — Fri, 10 Jul 2009 22:07:16 GMT

A few friends have pointed out that my post discussing my disappointment with Workflow 4.0 was somewhere between difficult and incomprehensible if you have no idea what's actually changed. Maurice posted this which gives some additional background.

Workflow 4.0 Is Disappointing

Kathleen — Sun, 28 Jun 2009 18:34:08 GMT

Well, to me it is vastly disappointing. Back in January, I said that I thought Workflow 4.0 would be the most important feature of .NET 4.0. I was completely wrong. WF 4.0 will be a rather uneventful part of the .NET 4.0 release.

My initial projection was partly because .NET 4.0 will (thankfully) be a rather quiet release at core with only limited new language features so the competition was never all that steep. But it was also because the WF in the CTP represented an important change to Workflow. Microsoft screwed up WF 3.0 and 3.5. It was excessively complex for the marketplace. They implicitly acknowledged this when deciding to scrap the entire thing and start over with WF 4.0. While some people do have deployed applications, WF 3.0 and 3.5 could not be the ubiquitous WF I think we all need in our toolbox. I was hoping WF 4.0 would be that tool.

The bad news is that WF 4.0 probably won’t be. WF 4.0 suffers from two problems, one implied by the other. The first problem is that it has no state machine. At least it’s not in the beta and I have heard no contradictory announcement. The second is that if they could not even provide basic functional parity with WF 3.0/3.5 how could they possibly have had their development act together/sufficient resources/fill in the blank to produce a great 1.0 product. Remember, with the rewrite, WF 4.0 is really a 1.0 product.

I hope I’m wrong on the second part of this and that they’ve covered an adequate range of real world scenarios that it will at least be solid. And WF 4.0 will probably be an interesting tool in the core scenarios, particularly those around WCF integration. I’m not saying it’s useless by any means. I’m just saying it’s probably not going to change your world.

I’m still angry about the lack of a state machine. Certainly WF 3.0/3.5 state machines will still work as well as it ever has, and if you can manage deployment and avoid the layout being trashed by the IDE (hint, keep your layout files ready for restore from source control or edit them to find your lost states). But I’m angry because you and I can’t give our customers new products that just ignore half the scenarios for the old product. We can’t toss out all the work our customers have done (interop is present but limited). We just can’t do stuff like this!

I know the WF team has been working hard and that there are really smart people on the team. I don’t know what went wrong, but the problems with WF 4.0 became the problems of the early adopters, and shafting early adopters (especially when two years still involves “early adoption”) is very bad business.

In fairness, they offered a flowchart solution which will on paper accommodate some of the scenarios where state machines made sense. It’s just the wrong solution, and a horrible decision to do something new while ignoring previous functionality - state machine should have had a higher priority.

The vast majority of non-trivial workflow scenarios are best solved with a state machine. It’s based on a quick analysis. Theoretically all state machines can be expressed in flowchart terms and all flowcharts can be expressed as state machines. So in theory as long as you have either you’re covered. But it doesn’t quite work out that way. Business systems have non-trivial numbers of states. As the number of states increase, the number of legal paths between them increases. The theoretical increase is huge, but unimportant because many states are illegal. However, a sufficient number of legal states remain that as systems become more complex, the number of paths in a flowchart increases very fast. Paths are added more often than states (paths are the most common unexpected side cases). Thus, the system also remains more stable if the initial analysis is against states, not paths. Finally in many business systems, thinking of the problem in terms of states simplifies analysis.

The good news is that WF isn’t the only new thing in .NET 4.0 and in my next couple of posts, I’ll talk about what I now think are some of the most important features of the next release.

MEF and MAF Resources

Kathleen — Thu, 07 May 2009 13:08:11 GMT

I got a Twitter on MEF resources, so I wanted to at least post the resources slide for my MEF/MAF talk. My big question, is what else do you want. Do you want more technie stuff on the core, more extensibility details, more stuff on the WPF editor, more on scenarios and value proposition, to know why I think composability is the most important thing since OOP/OOD (yep, I rank it above generics), or something else entirely? I think there's at least a half dozen people trying to get the right stuff out on this, but we don't need to shotgun this.

I actually modified this a bit if you were in the talks, but it’s a start – definitely not the final word:

• http://codebetter.com/blogs/glenn.block

• www.msmvps.com/blogs/kathleen

• Ask Kathleen, www.VisualStudioMagazine.com

• April 2009 for MEF/System.ComponentModel/Composition discussion

• April 2008 for MAF/System.AddIn discussion

• Hanselminutes: www.Hanselminutes.com

• #148 About MEF –Glenn Block

• #152 March 5, 2009 – Kathleen Dollard

• #147 VS 2010 WPF editor interview

• Also, good Bob Martin interview

• DotNetRocks: (me #436 April 9, 2009 -also, #304, 171, 121, 63)

• MEF download - www.codeplex.com/MEF

• MEF provider model and other interesting Andreas/TheCodeJunkie work www.codeplex.com/MEFcontrib

• PDC Videos

• TL49 (MEF-ify BabySmash)

• TL33 (Glenn Block) at PDC

• Pipeline Builder download: www.codeplex.com/clrAddin

• Kids, don’t try MAF without this

• Channel 9 video on MAF: Daniel Moth (search for System.AddIn)

Where’s the MEF Harness?

Kathleen — Fri, 10 Apr 2009 14:29:23 GMT

It’s not up yet. You may have noticed. And if you downloaded what we’ve currently got up at AppVenture, you probably noticed that it was nothing like what I discussed on Hanselminutes or DotNetRocks.

Why is it delayed?

Primarily because I have had a very hard time drawing a line around what should and should not be in the tool. I’m still having a hard time of that, and the first version won’t include everything I want.

Also, MEF is a new paradigm. It’s been rough at times to learn to rethink and debug differently. This has caused a few redesigns along the way, and lots of consideration for things like the large number of assemblies. These are things I think need to be pretty close to correct in the first go round. This process would have been much more difficult without Glenn Block’s help, and I really appreciate his help and friendship.

I began many parts of this project in VB. I love VB. But the open source aspect of the future of this project will involve primarily C# coders. That’s the world around me. I am happy in C#, I just went fairly far down the VB path and had to do a lot of translation. When I release it, it will be mostly C#, with some remaining VB in parts I want to rewrite (the UI) and places where VB offers benefits (XML literals).

I’ve also run into a half dozen really nasty scenarios, both in my code and the framework that took a few days to resolve.

Parts of this have been raw research. What should such a tool look like? I’m convinced it’s by far the best way to generate code, but the devil’s in the details. I think it will make it easier for you and others that I worked though these details, although as I look back at the simplicity of many parts of the solution I have plenty of self-doubt that I just started out stupid.

All these things take time, a lot of time. And I try occasionally to have a life, although the people in it would certainly argue my capacity for setting this aside.

When? I hope next week.

Assembly Organization in MEF

Kathleen — Fri, 20 Mar 2009 17:16:49 GMT

The purpose of MEF is to hook up parts. Parts are indentified by string contracts.

MEF (and tools like it) empower fully composable systems. I use that term to describe a system in which the system no longer knows what is going on. What you write empowers an ecosystem specific to a problem domain it does not solve the problem.

In my case, MEF empowers a harness where processes, generally code generation templates, are run in a specific manner (such as with pre and post processing). I don’t care what your process does. At one level I don’t care about what services it needs or what metadata it uses (although in another post, I’ll explain why I actually do care, the harness definitely doesn’t care). This is a tremendously powerful environment for certain problems, precisely because it becomes an ecosystem solution, not a proscribed solution.

It is also a very different environment where we’re still working out the rules as an industry. One area where the rules shift is in assembly organization. There are always two core drivers for assembly (project) organization – deployment and isolation. How does this play out in a MEF system?

Let’s assume UI isolation. MEF systems generally solve a complex but finite problem. It makes sense for them to be class libraries that are distinct from any single user interface. In the code generation case, we know we will have a graphical UI and a command line. I suspect I’ll have a kind of sucky graphical UI and someone will make a pretty one and starting in 2010 embedding in Visual Studio becomes more realistic. That’s 2 in any single deployment - one of the UI’s and a class library that is the core of the system.

A core goal of a composable system is to replace direct references with indirect references. This means interactions will generally use interfaces which belong in their own library for isolation. These interfaces will generally have a default implementation. While technically these defaults could reside in either the interface or the core library directory, putting them in either location means they will always be available and you will have to work harder for the user to replace them. Thus I think four class libraries is the minimum complexity for a real world MEF solution.

Almost all fully composed systems should go to at least one additional set of assemblies. Fully composed assemblies contain interfaces which are directly referenced and used by the system core and interfaces that do not have this direct interaction. It’s software entropy if I force you to design an interface for extracted database metadata. I’ll give you one, although the core system could care less about it. For isolation and clarifying intent, I think separation is a better approach.

You could place the default implementations for indirect services (such as metadata extraction) into the same assembly as the default implementations for services directly referenced by the core. Nothing will break. But they will always be in the same catalog and export provider, requiring the complexity of selecting the correct one. Deployment is simpler if these are separate. I also think a parallel between implementation and interface assemblies makes the system overall easier to understand.

In some cases, including the code generation harness, there are specific sets of roles parts can play. A given use of the harness may use any combination of these. If the assemblies are partitioned across these boundaries, people setting up the MEF system can employ only those items they need. Among other things, this speeds the MEF discovery process. For example, in the code generator, there are a couple of categories of metadata, output, naming and miscellaneous services.

The specific roles the parts play is defined via the interface assemblies. If these assemblies are partitioned, programmers extending the system later reference the specific set of interfaces they want. With partitioned assemblies, they might need to reference more than one, but once referenced, the complexity of what the programmer sees is less (which could also be done with interfaces).

Let’s explore the role of the extending programmer a bit more. If there is no extending programmer (you or someone else) there is no point in a composable system. This extending programmer might be part of the original team, if the purpose of composition is simply flexibility in deployment, such as whether particular modules based on customer status. In many cases, including the generation harness, the system is really being created for the convenience of the extending programmer, and it is there scenarios that should drive the application design. You may be able to design your system so certain changes can be made without MEF knowledge (new templates in the code generation case) but the reason you’re using a composable system is to support programmers creating new parts.

This extending programmer works in their own assembly, which among other things allows you to issue updates to default assemblies without breaking their work. These assemblies reference interface assemblies. Is it easier for that programmer to select well partitioned interface assemblies and use them to better understand how the system works. If you partition your implementation assemblies to parallel the interfaces, deployment is simpler and if you fully implement an assembly, you can remove the default one.

There are a few blog posts including this by Chad Myers and this by Davy Brion that speak disparagingly about solutions with many projects. I do not actually agree with either post (because solutions are convenience only features and our architectures should not reflect Visual Studio bugs) but I do think they make some good points. And the fundamental agreement is that you should understand why you have a lot of projects if you do. In this case, it’s for isolation (a core tenant of MEF) and deployment. If you want only a couple of projects in your solution, you don’t want a composable system (although you might still use MEF for other reasons).

Prioritizing MEF Parts – Or When Things Are Easy

Kathleen — Wed, 18 Mar 2009 20:28:03 GMT

So much of what I do with technology turns out ten times harder than I think it should be that I’ll admit I walked very softly and fearful of the quicksand when I wanted to implement my own prioritization strategy for MEF parts. In the end, the MEF interaction was dirt simple.

For review, MEF has a prioritization strategy. When you are looking for parts, the first export provider that fulfills the request (contract and cardinality) ends the part search process (the GetExport(s) variation). However, the generation harness has more complex requirements that this allows.

To create an application, you’ll use a set of templates. Templates will have a specific purpose, such as a Select stored procedure. I’ll supply a default for this, but you may not like my default and will want to replace it.

I want to do this while maintaining complete ignorance in the template about what contracts you are working with. I can’t use the multiple export provider strategy because I want all the templates in all of the catalogs considered.

You also will probably want to replace metadata and service parts. This scenario is different because the template knows the specific contract. Because there is a specific contract request, the MEF native prioritization could potentially work, except that I want the simplicity of a single export provider to manage the templates and do not want multiple export providers/containers.

Parts, including templates, have a contract, which in MEF is a string. There will be many templates doing multiple jobs in the generation process. MEF just grabs everything that implements IProcessWrapper when it grabs templates. I want another level of selection.

All contracts involved in the prioritization strategy have an Id and a priority as MEF metadata. MEF metadata is just a dictionary of values so I can check whether an Id or priority exist in the export definition. I want to run only the highest priority template for each Id. An Id might be “SelectStoredProc” or some other descriptive name. For convenience I’ll supply constants for the common templates. My templates will have a priority of 0 and will only run if no other templates are available. I’ll include a different post about why I think we can all play nice with priorities and about how to override priorities, after I figure out how I’m going to do that.

To accomplish this…

I created a new aggregate catalog that derived from the MEF AggregateCatalog and override the GetExports method.

Imports System

Imports System.Linq

Imports System.ComponentModel.Composition

Imports System.Collections.Generic

Public Class PrioritizedPartCatalog

Inherits Hosting.AggregateCatalog

Public Overrides Function GetExports( _

ByVal definition As Primitives.ImportDefinition) _

As IEnumerable(Of TempTuple(Of Primitives.ComposablePartDefinition, Primitives.ExportDefinition))

[[ In case I confused the C# folks, VB has “partial namespaces” which means it tries to tack namespace pieces like “Primitives” onto the stated namespaces and applies a little more clarity to namespace organization. I could have alternatively imported System.ComponentModel.Composition.Primitives]]

I want to grab all of the matching parts and do some additional filtering.

Dim allExports = MyBase.GetExports(definition)

The next part involves some KinkyLinq (I should probably grab that url). When I’m doing hard stuff in Linq, I split things up into multiple queries. Linq will optimize this if it can, and my brain won’t explode from one overly complex statement. The first query creates groups by Id. The second grabs the highest priority item in each group:

Dim q1 = From y In allExports _

Group y By key = GetId(y) _

Into Group _

Select key, Group

Dim q2 = From g In q1 _

Select _

( _

From y In g.Group _

Where GetPriority(y) = _

g.Group.Max(Function(z) GetPriority(z)) _

).FirstOrDefault

Return q2.ToList()

End Function

Not all parts will opt into prioritization. This is entirely optional. Managing this with priority is relatively easy. Any part without a priority has a priority of zero. However, managing Id’s for things that opt out of prioritization is a little more complex.

Private Function GetId( _

ByVal tuple As TempTuple( _

Of Primitives.ComposablePartDefinition, _

Primitives.ExportDefinition)) _

As String

' The guid means any export that opted out of prioritization is included

Dim idObject As Object = Nothing

tuple.Second.Metadata.TryGetValue("Id", idObject)

Dim id = CStr(idObject)

If String.IsNullOrEmpty(id) Then

Return Guid.NewGuid().ToString()

End If

Return id

End Function

If the ID exists, it’s used. If the Id does not exist, a Guid takes its place for the purpose of the filter. That means anything without an Id will always be considered the sole member of a unique group and will always be included since one member is selected for each Id.

The amusing thing about this solution is looking back in retrospect at where I spent my time. I spent a lot of time figuring out what could be done with MEF. I spent some time doing something wrong with Id’s at the metadata attribute level (which does not need to be complex) and spend some time on that KinkyLinq statement. Thanks to Glenn Block, Jay Harlow, Rob Teixaira, and Andreas for their help.

What I didn’t spend much time on is actually fitting the solution into LINQ. Adding this additional filtering involved only the code here (and the helper function for GetId() which you can predict). How simple can you get? The extensibility of MEF is what really makes playing with it worthwhile. If you hit a problem, like my unique filtering needs, you can probably find a way to work around it.

What’s Wrong with T4?

Kathleen — Tue, 17 Mar 2009 16:32:41 GMT

First, hugs and kisses to the people at Microsoft that made the decision to include T4 in Visual Studio 2008 instead of having it only available within the separately downloaded DSL toolkit. This indicated an important commitment by Microsoft to T4 and was a really positive step in code generation over all. Thanks guys!

I think the option wasn’t to have T4 done right or wrong in VS 2008, but to have T4 done wrong or not to have it. I’m happy the decision to include it was made, but I also think people need to understand what’s wrong with it.

First, the T4 template engine runs code in place and puts the result in the same project as the template in a dependent location. Huh? When would you ever want this? A core purpose of templates is to reuse templates across many projects, and once you get to working with and debugging the project, the template should be in the background, not the foreground. Oh, and this design strongly implies a single output file when single file output has been pursued by no one outside Microsoft (and by the way, Microsoft, we hate single file output). The folks working on the T4 Toolbox have shown us ways around this, but they are hacks giving templates inappropriate responsibilities for organizing the generation process. Templates should have the single responsibility of defining what to output.

Second, templates do not have a mechanism for passing parameters. Huh? We always pass parameters to templates, and we are almost always looping when we do it. There are hooks to pass parameters in T4, but you must understand some pretty complex plumbing to write into these hooks and it takes a few hundred lines of ugly code and internal, on the fly code generation.

Third, templates represent two interleaved sets of code. It’s the nature of templates and in T4 this is C# or VB code embedded within the artifact definition, which is often the definition of C# or VB code. Ad in the Microsoft provided editor its all the same color (black). Huh? Did you ever notice that these are nearly unreadable when displayed in a single color?

Sure you can fix these problems on your own, but everyone doing things their own way both wastes time and increases software entropy. Software entropy is the converse of feedback systems, and I’ll be talking more about that in relation to architecture expression in the next few days.

There’s good news here. Clarius provides a colorizing editor as a community edition to get you started. If you want fancier features, their main product is just $100. We’re releasing a community generation tool (also for free) as the AppVenture Community Generation Harness. This harness is a fully composable generation harness that will initially deliver with support for T4 and class based generation (such as VB 9 XML literal generation). It’s got the basic feature list in this entry. That’s in review at the moment and I’ll post here on my blog when it’s available at www.AppVenture.com. Using these two tools together, you’ll have a very workable generation environment based on what is a really a very sweet template language in T4 or VB 9 XML literal code generation, or any other type of generation someone writes a wrapper for. And thanks to Clarius and AppVenture, it’s all free to you.

MEF Assembly Granularity

Kathleen — Sun, 15 Mar 2009 15:25:00 GMT

I’ve been contemplating how to organize MEF assemblies. I think the processing I did establishing the first cut at organization, and the shake down of that strategy, may be interesting to other people designing MEF systems.

As a quick review, MEF lets you throw parts into a MEF container and sort out how parts work together at runtime. Parts are recognized by a string identifier. I’m almost always using interfaces as the contract and the interface names as the identifiers. Parts reside within assemblies and in the common case assemblies are discovered because they are grouped into anticipated directories.

With this approach, only the part implementing the interface and the part that is using the interface need to understand the interface or explicitly reference the interface’s assembly. And since parts are discovered and loaded at an assembly level, the granularity of implementing assemblies also controls the granularity of the load. I care about the assembly granularity of contract/interface assemblies so excess stuff can be avoided and naming conflicts (resolved via namespaces) are minimized. I care about the granularity of implementation assemblies because until I attain a priority system with additional granularity, prioritization/defaults are only as granular as their containing assemblies.

At one extreme, all interfaces reside in one assembly and all implementations reside in another. It doesn’t make sense to put them into the same assembly as then hard coded references exist and ensuring isolation is difficult. At the other extreme, every interface and every implementation resides in its own assembly. I think both of these extremes are a terrible solution. That’s because this composable system (and I would think any composable system) have parts with very different roles and lineages/history. In the simplest sense for a generator - metadata providers and templates are fundamentally different and could easily be provided by different teams.

Initially I thought the primary consideration should be the implementation deployment, but Phil Spidey pointed out in the MEF discussions that the interface organization is more important, because once released to the wild it might be hard to fix.

I decided on six contract assemblies:

CommonContracts	Interfaces referenced by the template harness itself
CommonDatabaseMetadataContracts	Interfaces sharing database structure
CommonDomainMetadataContracts	Interfaces sharing business object structure
CommonNamingServiceContracts	Interfaces for a naming service
CommonOutputServiceContracts	Interfaces for outputting data, including hashing
CommonServiceContracts	Miscellaneous interfaces that don’t fit elsewhere

I’ve used a few criteria for this design:

Interfaces that are used by the system and therefore can’t easily be changed reside together in CommonContracts. The template harness also references CommonOutputServiceContracts but this is in a separate assembly because it has a distinct purpose, may evolve on a different time frame and you are far more likely to provide alternate implementations for output than for the core interfaces.

The naming service is also a separate assembly because it is a distinct purpose and some people will certainly supply alternate implementations to manage human languages other than US English. Both the output service and naming service are a few distinct interfaces that work together. I also had a few odd ball interfaces and decided to go with a grab bag of miscellaneous interfaces rather than a separate assembly for each interface. Time will tell whether that is a good decision.

I initially put the two metadata interfaces into a single assembly, but I think it’s quite likely that these interfaces will evolve separately and almost certain that they will be implemented independently.

I’d like to note that the first version of the harness, which is almost, almost done (a separate blog post) will be a CTP /alpha level release. I will take feedback on the interfaces and I do expect them to change. A core part of the composable design is that you can spin off your interfaces/implementations so while these changes will be breaking, you can uptake them at your own pace.

Code Generation on Hanselminutes

Kathleen — Mon, 09 Mar 2009 15:05:42 GMT

Scott and I sat down at the Rocky Mountain Trifecta 1 to discuss code generation and other fun things. Check out show 170 at www.Hanselminutes.com

Talking to T4 – When NOT to MEF-ify

Kathleen — Sun, 08 Mar 2009 23:09:21 GMT

If you aren’t currently creating T4 templates, skip this post as a rather geeky exploration of something you should never have to touch. Hopefully in another few days you’ll harness to take care of this ickiness.

T4 templates should know their own output file name! This is not a function of the host or the harness or anything else. We want to wrap up the responsibility of creating a template in one location (with any redirection necessary by the class to get the job done and support single responsibility).

How to do this? My first idea was way nerdy wacko cool – use MEF! It’s new, it’s cool, everyone loves something with such a cute name!

The first step to MEF-ifywas moderately easy – add a class in the T4 template that has an export (or a property, or whatever). This just involves usual ickiness of setting T4 references, imports etc. Messy, but if you are working with T4 hosts and engines, you know this stuff. Got that done.

The code for the MEF export approach might look like:

<#+

[Export(typeof(IT4Template))]

public class TemplateSupport : IT4Template

{

public string GetFileName()

{

return "Fred";

}

Then, comes the harder part – where do you deal with the MEF container? Some place in looking through this, I realized why this is NOT a good use for MEF. Two reasons actually – First I know exactly where the value should come from. I suppose I could argue for some sort of default service that uses an embedded naming pattern, but I already know that embedded naming patterns get ugly over time. XSLT will require a naming pattern (or call backs which are problematic during testing), but MEF and XML literals, etc will not. We need a single location, the template, to tell us the output file name. And it needs to run code to determine the name. I can’t predict for a pattern language everything someone will want to do, leaving people to extend a pattern service – well complexity goes up.

The second reason is that there is another way that is demonstrably less complex. To meet this bar, we need code simpler than what I showed above.

The second approach I considered was a value holder service for the host. Basically the template could request a service (which could the template itself) that supported the service and simply set the value. Then after the template ran, the engine could retrieve the filename from the host, which was holding it for everyone’s convenience:

<# (this.Host as IValueHolderService).SetValue("FileName", "Fred"); #>

That’s still pretty ugly. It occurred to me that we already have a value holder service, we just call it properties and it contains parameters to the T4 template. I’m just going to swipe a slot in the properties dictionary for my use, which by implication means you can’t use the property name I use. I decided it would not be nice to force you to avoid having a property named FileName with some other purpose than the output file name. The name needs to be exquisitely precise, because I am messing in your logical symbol space: TemplateOutputFileName does the trick for me (I’ll be happy to take other suggestions).

Now, you can just set the name of the output file in code, using whatever additional functionality you need. At the level of the T4 template code, you are just setting the value of a variable. The result is something like:

<# TemplateOutputFileName = "Fred"; #>

This fulfills my goal of a very simple way for you to set the filename inside your T4 template while allowing you to run any code you. There will be some T4 geeks that do a double take at the fact this is not a directive, but directives do not allow internal template code to run. If this gives people heartburn they can either declare the property (I won’t redeclare it) or I’ll also add support for a directive if people really want it.

Binding to a MEF ExportCollection

Kathleen — Sun, 08 Mar 2009 18:01:34 GMT

Josh said this in the MEF discussion list on CodePlex:

MEF is just a composition engine its job isn't really to offer choices for a users.

While that is true, it’s not that hard to present choices to users and it gives you a bit of insight into the very powerful strongly typed metadata attributes of MEF.

MEF is a tool for matching up parts. Its discoverability is not against the actual class, because that would require too deep a dive into the assembly. Its discoverability is based on class attributes which announce that a part fulfills a contract. In MEF, a contract ID is always a string and you’re committing to fulfill expectations someone will have about something that claims that contract. I almost always use one of two types of contracts – type contracts against interfaces that formalize my commitments and simple strings when I have a random piece of information floating around like a database or server name. To clarify how it works, MEF turns the type into a string for its internal use.

When you create an interface for MEF you have the option to create metadata to use with it. Metadata generally (always in my case) exists as two pieces – an attribute and an interface. What happens internally is that MEF creates a dictionary with the data provided by the attribute. At runtime, you can request that you only see parts that fulfill the contract and metadata by supplying contract interface and metadata interface. If you strongly type this via the generic overload you have a MetadataView property which is of the type of your metadata interface.

You can explore this more in the documentation for strongly typed attributes at the MEF site on CodePlex.

From a binding perspective, if you want to bind something you create an ExportCollection(Of IFoo, IFooMetadata). This is a collection of export objects which have a IFooMetadata strongly typed MetadataView property. With me so far?

<Import()> _

Private mDatabaseServices As _

ExportCollection(Of IDatabaseServices, _

IDatabaseServicesComposition)

And in the Load method:

providerComboBox.ItemsSource = mDatabaseServices

You can bind to this export collection (FriendlyName is a property on my metadata interface):

<ComboBox Grid.Row="1" Grid.Column="1" Name="providerComboBox"

SelectionChanged="providerComboBox_SelectionChanged">

<ComboBox.ItemTemplate>

</DataTemplate>

</ComboBox.ItemTemplate>

</ComboBox>

ExportCollection is not an ObservableCollection, so if you want those extra features, you can create one yourself. In the Load method this is simply:

availableOutputs.ItemsSource = OutputItem.GetBindableList(mOutputListeners)

where OutputItem is

Public Class OutputItem

Private mExport As Export(Of IOutputListener, IOutputListenerComposition)

Public Shared Function GetBindableList( _

ByVal list As IEnumerable(Of Export(Of IOutputListener, IOutputListenerComposition))) _

As ObservableCollection(Of OutputItem)

Dim ret = New ObservableCollection(Of OutputItem)

For Each item In list

ret.Add(New OutputItem(item))

Return ret

End Function

Private Sub New(ByVal export As Export(Of IOutputListener, IOutputListenerComposition))

mExport = export

End Sub

Public ReadOnly Property FriendlyName() As String

Get

Return mExport.MetadataView.FriendlyName

End Get

End Property

The XAML for binding this is:

<ListBox.ItemTemplate>

<CheckBox Content="{Binding Path=FriendlyName}"

IsChecked="{Binding Path=IsActive}" />

</DataTemplate>

</ListBox.ItemTemplate>

</ListBox>

Where differences other than in the binding are because I’m solving a different problem.

What Does a T4/Code Generation Harness Need to do?

Kathleen — Sun, 08 Mar 2009 17:08:27 GMT

I'm struggling to get the AppVenture Community Edition Code Generation harness into release because I can't figure out where the boundaries should be. I initially thought I could just reuse my old stuff in the area of data extraction and mapping/morphing, but too much has changed.

What's in:

Multi-UI supporting core code generation engine
Full composability via MEF
Configuration driven ordering of automatically discovered templates
Template focused generation (templates know stuff)
Multi-file output via a simple naming mechanism
Support for VB 9 XML literal templates
Support for T4 templates
A few example templates at Hello <item> level
Key service interfaces defined
T4 property extraction/value setting
Partial data extraction (tables and a few others)
Rudimentary US English pluralization service

What's delayed:

Full data extraction (views, foreign keys, etc)
Mapping across data/object impedance mismatch boundary
Full project templates (stored procs and biz layer)
Hashing output files to protect handcrafted code
Services to load stored procedures
Services to create project files
XSLT template support
More services such as a better naming service

The core of this design is that I do not want to stand between you and your templates, metadata or services. Thus if you have metadata, you can hook this up and generate anything you can write T4 or VB9 XML literal templates for. You can create any services, metadata or templates you want.

Delaying stuff is hard. I want this to fully meet the capabilities of the GenDotNet harness so I can retire it. This is a vastly superior way to approach the problem.

So, where's the boundary? For this tool to be worth a look, what does it need to do? This is V1, the rest will come. I'm trying to balance when to release V1 and I'm doing this all in my "spare" time so some patience will be needed.

Comments please!

A Quiet Conversation about DDD and Data First Design

Kathleen — Sat, 07 Mar 2009 14:36:27 GMT

At the MVP Summit I had the pleasure to sit down at a party for some one on one time with Don Smith. I’m trying to think of my blog as a nice little corner to talk, rather than a soapbox. I want to make an share something that is not shouted from the rooftops. Eeegads, I don’t want to start another debate on this.

A few months ago, the EF team started a wiki where Ward Bell and I felt quite attacked for suggesting that DDD is not always the best approach. And thus, it is with some trepidation that I touch this topic. But today’s Database Weekly has a column on it and I really feel there’s stuff worth hearing. If you’re here in my nice intimate corner, you can hear it.

The question of whether to start with a database or a domain (business object) model makes no sense. The answer is easy: start with the one most likely to bring you success, and don’t ignore the impedance mismatch problem.

A well structured application has a good domain model and a good (relational) database and a good strategy to cross the impedance mismatch boundary. That boundary exists because neither the domain nor the database should drive the structure of the other.

A database might be a more successful starting point if you have good, stubborn, or available DBA’s or if your DBA’s are good analysts. If you’re a small shop – which do you build better and have you ever tried building it the other way? Database first is also often a good starting point if you have an existing database. Even if the database is bad, it contains the existing business, and it’s my belief we should never close our eyes to a way the business has already expressed itself if we can get a hold of it (it’s not in code). While we should consider available expressions of the business, we should not blindly accept any piece without exploring also exploring its problems.

A domain might be a more successful starting point if you have good, stubborn, or available coders, or if your coders are good analysts. If you’re a small shop – which do you build better and have you ever tried building it the other way? Domain first (DDD) can also be a good starting point if you have an existing database. If you build a domain model that you constantly validate against the existing database you can base your thinking on experience while not being stuck in that experience. While we should consider available expressions of the business, we should not blindly accept any piece without exploring also exploring its problems.

If it’s an even match, consider DDD. The issues are more subtle and getting them out of the way might be helpful to your project.

The monumental disservice that resulted from the EF wiki (which has thankfully now died a formal death) is that this decision appeared to be a religious one or one that marked you in one camp, or perhaps to some even something about your level of coding. All of that is stupid.

- Do DDD or database first based on what makes sense in your specific scenario

- Whichever way you start, attention to the impedance mismatch will minimize negative consequences to the other side of the boundary

It comes down to the obvious. It’s your team, it’s your project. Make decisions based on your reality, not dogma. Learn from the debates in our industry. Don’t pick sides and follow blindly (even my side.)

So, now we can go back to the rest of the party. If this kicks off another brawl, I suggest slipping out by the side door.