Leaning Into Windows : .NET

MEF and Cardinality Composition Failures

Kathleen — Fri, 17 Jul 2009 13:44:24 GMT

You can check here for a quick description of MEF

I'm giving a half dozen MEF talks this summer and I'm frequently asked "what happens if a part isn't available". The old answer was "the system crashes, how could it do anything else?" This conversation definitely deflates the upbeat mood of a MEF talk. Recently, MEF has changed, making that answer obsolete.

MEF is a composition container which satisfies imports by tracking down associated exports. The correct number of exports to satisfy each import is called the cardinality and can be one, zero to one or zero to many. Thus an import can fail because there are too few or too many matching exports.

In the MEF previews 1-5 (inclusive) and in the Visual Studio CTPs and beta1, MEF throws an exception when a failure occurs.

MEF Preview 6, released last Monday, introduces "stable composition." With stable composition, the container can know about, but not expose parts. If a part fails on a cardinality rule, the MEF container remembers the part, but keeps it hidden. If additional composition occurs (such as through the Refresh method) additional attempts to fully compose the part occur. If its cardinality is fulfilled, the part becomes available for additional composition. You can think of this as "if a missing sub part shows up later, the containing part will become available."

There are both good and bad aspects of this, and it definitely affects how you think of and write your MEF systems. In general, it will make MEF apps more stable. If you design a plug-in model, and the creator of a plug-in fails to properly deploy (or a confused user deletes some but not all of a plug-in via File Explorer), your system will not crash. The containing part, which would probably fail if run, doesn't appear in composition. This makes your system more robust against errors that are beyond your control. It also allows the late composition strategy, although I'm not yet clear on good scenarios for it.

The down side is that you may have more challenges finding certain types of composition errors because you will not receive an exception - you need to catch the current state of the composition container. And if you don't consider this behavior when writing your app, you can get officially bad behavior.

In a plug-in design such as the directory composition model, any part in your system can fail on a cardinality (and could previously have crashed) because all parts can be made of other parts with dependencies you don't know about.

For example, consider creating a main menu as a part and that menu is made up of menu items and sub menus and one of the menu items cannot be composed due to a deeply nested cardinality failure. If each main menu item is a part and you import the menu items as a collection using the ImportMany attribute, you're fine. Your application will simply not display the failed menu item.As deep as the nesting goes, each layer that is a collection is naturally protected because cardinality failures just remove one part from the collection.

If instead you create your main menu to explicitly expect a particular menu item (such as a Tools sub menu) and that menu has a required Import for a part and so on down to a point of cardinality failure, then the failure cascades up the chain. This happens because the leaf cardinality failure means that part is not available, causing a cardinality failure at the next level. In this hypothetical case, the failure cascades all the way up to the main menu which does not now exist. The application either runs without a main menu, or the application fails because the main menu is missing.

You can avoid this by considering the intent of each import and providing appropriate protection. Some parts are optional and your application can run just fine without them. These should either be in a collection (ImportMany in recent previews) or not required. Other parts are important, but not important enough to cause their container to fail. These can be managed via asserts, and communication with support or the user. If parts are critical to the application running, then you need to check that they exist after composition and shut down the application as gracefully as possible.

I think this is a good change, but at least until more patterns emerge, you need to consider what would happen if any import is not successfully satisfied. Should the containing part also fail to be composed?

MEF is a very sharp knife. It cuts up the tomatoes and carrots really well and we can have a dandy stew. Or we can wind up in the emergency room. Understanding MEF is important to safe MEF use.

You can find more on the Preview 6 changes in this post by Nicholas Blumhardt who is a member of the MEF team. I'd like to thank Glenn Block for his discussions with me on this change.

A Quiet Conversation about DDD and Data First Design

Kathleen — Sat, 07 Mar 2009 14:36:27 GMT

At the MVP Summit I had the pleasure to sit down at a party for some one on one time with Don Smith. I’m trying to think of my blog as a nice little corner to talk, rather than a soapbox. I want to make an share something that is not shouted from the rooftops. Eeegads, I don’t want to start another debate on this.

A few months ago, the EF team started a wiki where Ward Bell and I felt quite attacked for suggesting that DDD is not always the best approach. And thus, it is with some trepidation that I touch this topic. But today’s Database Weekly has a column on it and I really feel there’s stuff worth hearing. If you’re here in my nice intimate corner, you can hear it.

The question of whether to start with a database or a domain (business object) model makes no sense. The answer is easy: start with the one most likely to bring you success, and don’t ignore the impedance mismatch problem.

A well structured application has a good domain model and a good (relational) database and a good strategy to cross the impedance mismatch boundary. That boundary exists because neither the domain nor the database should drive the structure of the other.

A database might be a more successful starting point if you have good, stubborn, or available DBA’s or if your DBA’s are good analysts. If you’re a small shop – which do you build better and have you ever tried building it the other way? Database first is also often a good starting point if you have an existing database. Even if the database is bad, it contains the existing business, and it’s my belief we should never close our eyes to a way the business has already expressed itself if we can get a hold of it (it’s not in code). While we should consider available expressions of the business, we should not blindly accept any piece without exploring also exploring its problems.

A domain might be a more successful starting point if you have good, stubborn, or available coders, or if your coders are good analysts. If you’re a small shop – which do you build better and have you ever tried building it the other way? Domain first (DDD) can also be a good starting point if you have an existing database. If you build a domain model that you constantly validate against the existing database you can base your thinking on experience while not being stuck in that experience. While we should consider available expressions of the business, we should not blindly accept any piece without exploring also exploring its problems.

If it’s an even match, consider DDD. The issues are more subtle and getting them out of the way might be helpful to your project.

The monumental disservice that resulted from the EF wiki (which has thankfully now died a formal death) is that this decision appeared to be a religious one or one that marked you in one camp, or perhaps to some even something about your level of coding. All of that is stupid.

- Do DDD or database first based on what makes sense in your specific scenario

- Whichever way you start, attention to the impedance mismatch will minimize negative consequences to the other side of the boundary

It comes down to the obvious. It’s your team, it’s your project. Make decisions based on your reality, not dogma. Learn from the debates in our industry. Don’t pick sides and follow blindly (even my side.)

So, now we can go back to the rest of the party. If this kicks off another brawl, I suggest slipping out by the side door.

Entity Framework Petition

Kathleen — Tue, 24 Jun 2008 22:44:43 GMT

I feel I need to respond to the “Vote of No Confidence” on the Entity Framework.

I have little interest in petitions. They are by nature backwards looking. To get a group of people to sign onto something they have to either understand it or be driven by the charisma of the leaders. In this case, I assume the first. The contents of the petition must be stable and old enough that everyone has worked out the details. That’s the case with all the technical petitions I can think of, although admittedly that’s just a handful like the VB6 petition.

When it comes to the appalling scenario where we have at least 14 major categories of data access strategies in use in new projects today, we need the Microsoft teams to look forward and be creative in combining the best set of techniques – NOT pick one of the existing strategies and latch on to it because it came from the group that yelled the loudest.

Entity approaches are good because they better separate the business and data sides of our middle tiers. But they are also inherently difficult and inaccessible to most programmers. Entity Framework’s goal must be to bridge this gap. That means being extremely creative in picking its battles to reach toward the real world developer – not copy a strategy that is available to that developer today and fails (the combination of NHibernate and other tools used in a specific style of development). The failure is not because NHibernate is an Open Source tool. It’s not because people don’t know about it. If it worked in the majority of shops it would burn through our industry like wildfire. Why don’t you use them? Because they do not fit your development environment! This is not an easy problem that someone’s solved and Microsoft is looking the other way. It’s an incredibly hard problem – how do I know? I’ve been working on occasionally novel solutions to the problems for 20 years.

Entity Framework has issues. This is not news. It’s not even news to the Entity Framework team.

- EF is not a failure because it doesn’t fit TDD development

- EF is not a failure because business logic goes into partial classes

- EF is not a failure because it treats data as an important part of biz objects

- EF is not a failure because it accepts that most people do data first development

- EF is not a failure because lazy loading is hard – lazy loading can destroy performance

- EF is not a failure because its design tools are 1.0 level

- EF is not a failure because it has a poor strategy for merging into source control

All of these are potentially issues, but it’s critical, essential, I cannot yell this loud enough – Entity Frameworks must not be designed for the group that is best organized and screams the loudest. This already happened once with the disastrous IPOCO attempt that helped no one and wasted a lot of manpower that could have improved mapping and provided better metadata.

But then I’m sort of caught in a corner, because an important point of the petition is correct. Be cautions with EF. Do not jump into Entity Framework because of Microsoft marketing. It’s a tough platform that will get a little easier when the current spasm of books comes out. The niche is pretty narrow and if you step off the boards, the quicksand can be pretty deep. Treat it like what it is - an amazingly large and complex project that is being released as a 1.0 product. It’s an infant. The metadata and mapping still stink. Look at it as Microsoft’s current future direction, but remember how many current future directions we’ve had over the last 15 years (around ten) and remain skeptical.

Runtime vs. Design/Compile Time

Kathleen — Thu, 14 Feb 2008 17:21:00 GMT

Chris asks:

At what point with code gen / templating do you start to think about doing all this codegen at runtime instead of compile time?

And if we were to be doing it at runtime would be be better served by using a dynamic language such as ruby to program in?

That's a good point. In a perfect world, there would be no need for code generation. We would write nothing but business/domain specific code and everything else would just happen. But for well over twenty years we've been aiming for that perfect world and we seem only a few baby steps closer than in 1987.

In an imperfect world, we have two basic choices, manage an architecture and run a lot of code at design time or manage an architecture and run a lot of code at runtime. Both require a fair amount of configuration.

And both can offer the real long term benefit of switching away from coding code – which is transitory and dying before you even finish coding. We want to switch away from coding code and toward creating metadata which is a true business representation. Of course metadata changes. But it changes at the speed the business changes – not due to artificial technology shifts.

For my effort – I want the extra code run at design time to offer the best possible runtime behavior, including performance. I can extend an architecture I’m expressing in templates far easier than I can extend an architecture I am expressing via an OR/M tool. I also want to debug directly into code specific to the problem at hand – I want to debug through generated code, not an OR/M engine.

The next round of effort at making plumbing simpler from Microsoft is also code generation – Entity Framework. When someone gets the plumbing correct and we truly never need to care, we can turn off the code gen and go direct to whatever structure, however its’ done and completely ignore the problems code gen is primarily used for today. In the long term, we shouldn’t care about anything except the business problem we’re solving.

Dynamic languages certainly change the architecture. I'm looking forward to exploring them as the overall knowledge base expands. But we've been sitting with a pretty dynamic language in our laps for many years, the majority of us have programmed in it, and with the exception of precisely one person - everyone I know varies between mild distaste and downright hatred for it. I think a lot of Javascript's problems have been related to debugging and platform issues, and I realize that there are differences between Ruby/Python and Javascript. However, if we didn't fall in love with the dynamic aspects of Javascript in the last ten years, I remain slightly skeptical about dynamic languages in the next few years. Some of the differences, and our attitudes and skills may change as a result, are that the new languages have a broader platform base and we're increasing our understanding of how to actually use them as opposed to hacking them enough to solve some trivial website detail.

But at the core, the technique for expressing metadata into a working application is not half as important as metadata at the core of the application, however its expressed.

Open Source

Kathleen — Wed, 13 Feb 2008 15:31:24 GMT

It’s occurred to me that if you are following this and my DNR TV show a logical reaction would be “OK, so that’s a lot of hot air, where do I get it?” I intend for all of this to be released Open Source, on whichever site s hot when I release it. I hope I’ll start releasing pieces in just a matter of weeks. It will help a lot if it becomes “we” instead of “me”. So, if you’re interested n this stuff and you want to help, let me know and you can get this stuff directly as its rapidly evolving too much to publically post right now.

My current expectation of the order of release:

Metadata interfaces
GenDotNet metadata load
EDMX metadata load
Template infrastructure – what’s needed for the language neutral templates
Simple harness
Activity based harness

If you’re interested, you can contact me directly at Kathleen@mvps.org

Isolating Metadata

Kathleen — Wed, 13 Feb 2008 15:26:09 GMT

In code generation, metadata is the information about your application, generally about your database and definitions to express your data as business objects. If you use Entity Framework, your metadata is the edmx file which is displayed via the designers. If you’re using CodeSmith, the metadata is more subtle. Metadata can also be about the process itself. CodeBreeze in particular has a very rich set and extensible set of information about your application.

Since metadata itself is data – information - we can store it many ways. I’ve used XML for years. CodeSmith has used a couple of mechanisms including XML. Entity Framework uses XML. Metadata can also come directly from a database, although I think this is a remarkably bad idea and one of my code generation principles is not to do that – you need a level of indirection and isolation surrounding your database.

What I haven’t talked about before how valuable it is to have another layer of indirection between your metadata storage structure – your XML schema – and your templates. In my XSLT templates I could provide this only through a common schema – you can morph your XML into my schema so that’s indirection – right?

No, that’s not really indirection. It’s great to be back in .NET classes with real tools for isolation and abstraction. Now I use a set of interfaces for common metadata constructs such as objects, properties and criteria. I can then offer any number of sets of metadata wrappers that implement these interfaces via a factory.

The template programs only against the interfaces. The template could care less whether I am using entity framework, my own metadata tools, or something entirely different. I can write the same template and use it against Entity Framework’s edmx file or any other metadata format. That’s powerful stuff. Especially since you already heard that the template will run against C# or VB. That means in my world the only reason to have more than one set of templates against an architecture like CSLA is that they are pushing the boundaries and actually doing different things.

But if you don’t like this new templating style, you can use classes based on exactly the same interfaces in CodeSmtih (at least) and again free your framework and metadata extraction. You’ll still need VB/C# versions there, but you’re metadata input can use the same interfaces.

The interfaces is expressed in sets of classes that know how to load themselves from a data source. Each set uses a different metadata source – different XML structures or other format.

Isolated metadata removes your templates from caring what the metadata source is – beyond being something that could successfully fill a set of classes that implement the data interfaces. This is a very important step and one we need to work together to get right. What do you think I've left out of the current design?

Minnesota VSTS User Group - Code Generation in 2008

Kathleen — Wed, 13 Feb 2008 14:57:00 GMT

I'm stepping out of my comfy sandbox again. Not only am I going to Minnesota in February, but I'm also talking to a VSTS user group - to remember that it's not just about code.

You can get more information here.

I am really looking forward to this talk and sharing perspectives with people that I'm hoping connect on some of the process questions about code generation. While I dropped the ball on updating the abstract, this talk won't just be templating techniques - I'll spend a good bit of time on the research I've been doing moving to an activity metaphor for code generation - raising above the simple process declarations. This is very fun stuff and I'm excited to find a group interested in sharing it. If you're in the area, please come by.

New Jersey SQL Server User Group - SQL Server Stored Procedure Code Generation in 2008

Kathleen — Tue, 12 Feb 2008 15:55:00 GMT

If you can make it to Parsippany, New Jersey on Tuesday Feb. 19, I'll be talking about code generation. This is a SQL group that's been kind enough to host a hard core .NET geek. The common ground is stored procedures and making them easier to write, and especially to maintain. They've said they'll be gentle, and I hope to learn a thing or two. It's always nice to step out of my comfort zone,and I think I can offer a great night.

So, if you live in the area, I'll see you at the meeting!

CurrentTypeInfo and the Context Stack

Kathleen — Tue, 12 Feb 2008 15:46:00 GMT

Creating templates requires a lot of access to the thing you’re currently creating. That’s the current output type, which as I discussed in yesterday I suffix with “Info.” The CurrentTypeInfo is thus what you’re currently outputting.

I neglected to clarify in that post that the Data and the Info classes are in entirely different assemblies. The Data classes could be used with any .NET generation mechanism, including (at least) XML Literal code generation and CodeSmith. The Info objects are relatively specific to my style of code generation.

The CurrentTypeInfo may not be the same throughout the file.

There are a few reasons to combine multiple classes or enums in a file. In some cases, that’s to nest them, and in some cases it’s just to logically combine them in a file. While FxCop has declared them unfashionable, I find nested classes tremendously valuable, especially for organizing Intellisense and keeping well structured classes and naming. If you’re working with nested classes, there is a good chance you’ll need to access not only the current type, but also the encapsulating type. I use a stack for this, and give control of pushing and popping TypeInfo objects from the stack to the templates themselves.

The base item on the stack is the current outer most class in the current file. Once you’ve pulled a class off the stack, such as by popping the base and adding a new base TypeInfo, you can’t access the previous version unless you’ve saved it.

Here’s where you see the flaw I mentioned yesterday – these classes could be in separate namespaces, and I don’t allow for that – yet. I’ll fix it later.

Remember the TypeInfo is the thing you’re outputting. The entity definition it’s built from is ObjectData in my semantics.

The stack is an extremely useful construct for this scenario. You have quick access to information about the class you’re currently outputting. You’ll frequently need this for type information and perhaps calling shared/static methods. You don’t want to recreate its name every time you use it because that would be redundant and hard to maintain. You can also access any of the containing classes, which again is useful in defining types and calling shared/static methods.

While I haven’t done this in CodeSmith, I expect this technique to be viable there. I’m not sure on other platforms, but it’s not specific to XML literal code generation.

Two Parallel Entities - Metadata and Info

Kathleen — Mon, 11 Feb 2008 17:08:00 GMT

One of the confusing things about templating is that you are writing two programs simultaneously and there is no way around it. My brilliant son may write a templating language for a class project, and this is exactly what he wants to address – that and the issue of escaping. You can’t avoid it, he just thinks they should look different – I’m hoping he doesn’t write one in Magyar.

One of the programs you’re writing (or maintaining) is the template program. There is logic in any non-trivial template, regardless of the template style. The other program is the one you’re outputting that will eventually run on your clients computer. In ASP.NET style templates, the output program is the overall template, and the logic is inserted. In XSLT and XML literal generation, the logic of the template program is the overall code and the output template is nested inside.

You are also working with two sets of data – the metadata you’re using as source and what you’re currently outputting. I call the elements in the XML I use as source metadata for .NET templates Object and Property for clarity in the XML. If I called them that in the templates, I’d encounter great confusion between the object definition in metadata and the class I am outputting. That’s particularly painful when it comes to properties.

I solve this by calling suffixing all metadata with the word “Data”. Thus I have ObjectData, PropertyData, CriteriaData, etc. Each of these classes contains the metadata on the corresponding item. I might have an ObjectData for a Customer that had a PropertyData for FirstName. The PropertyData would include things like the maximum length of the field and its caption.

I also need to describe the output – at least the way I’m building templates. I need information about the file I’m creating, the type I’m creating, and in certain locations in the template, the property, function, constructor, etc I’m creating. I identify these by suffixing them with the word “Info”. Thus I have a ClassInfo, PropertyInfo, FunctionInfo, etc. I do not have a CriteriaInfo because I am never outputting a .NET thing called a Criteria. It’s strictly metadata for .NET features.

In the template design I’m describing in this series, the DotNetBase class contains information on the file being output. To be picky, the namespace can actually be associated with part of a file, and I may add this flexibility, but I don’t do that very often in business code, so I have included namespaces at the file level.

Regardless of the template mechanism you use, you need to maintain a clear separation between the template logic and the output logic, and between the input metadata and state about what you’re creating.

Next post: CurrentTypeInfo and the Context Stack

Inheritance and Templates

Kathleen — Mon, 11 Feb 2008 16:25:00 GMT

One of the most valuable things about templates written in .NET code of any style is the ability to use inheritance. This is classic inheritance where local state allows you to push functionality into base class methods that would become cumbersome and have excess dependencies if treated as external utility methods.

Inheritance also allows polymorphism, which is nothing more than a big word for exchangeability – OK, that’s actually a bigger word. Templates are called by something. Life is simpler and harnesses practical if that something is known and predictable, thus it’s logical to have an interface in a common location as a contract to the template interface. If the base class implements this interface and provides functionality via MustInherit/abstract members, there’s further capacity for evolution.

There are three assemblies involved. The template interface is in a common assembly available to all other assemblies. The base classes are in a support assembly which can be reused across many template sets. The templates themselves are in a separate assembly. For now, I have all templates associated with a project in one assembly, but I may change that to isolate the SQL and the .NET templates. In any case, these template assemblies are sets of templates that work together.

Inheritance let’s you push common functionality into the base class. Now that we have extension methods they offer an additional mechanism to remove common functionality from the leaf class. I know Scott Hanselman has said he’ll cut off the pinkies of anyone who uses extension methods on classes you own, and I’ll show you when I get to that in a few days why I disagree (although it’s in the DRN TV show). But that explains why my base classes are focused on state more than functionality. The key state issue is what you’re currently working on. In a template, state is metadata. In a .NET template, the important metadata is the definition for the entity you’re currently creating. In my templates, the ObjectData member of the DotNetBase contains this information.

Stay tuned for the distinction between the input data you’re working on and the thing you’re creating, which explains some of those other members.

.NET Template Organization

Kathleen — Sun, 10 Feb 2008 23:55:00 GMT

So, now that you know where I’m going with this – language neutral templates – I want to step back to the basics. Even if you don’t want to build language neutral templates, there are things to learn along the way about making good XML literal templates, and why this approach might be better than your current code gen mechanism. Note, the examples in this post are NOT language neutral. Not surprisingly, there are specific requirements for language neutral templates and I want to show basic templating with XML literals, then the language neutral templates.

If you look at the XML literal sample I posted a few days ago you’ll notice that the entire contents of the output DataPortalFetch method is created within the template method “MemberDataPortalFetch.” This provide important organization as the most common challenging task in code generation is “I have a problem in my output code right here, how do I find where that is in the template. In a simple template this isn’t too hard, but in more complex templates such as CSLA it can be quite challenging.

Creating templates with an extremely predictable structure is very valuable in creating maintainable templates. To start with each template should have a one to one correspondence with an output class.

Within this class, the mechanism of the predictable structure is one of the primary differences from codes that are code with template segments (XML literal code generation) and templates with code segments (ASP.NET style template). In an ASP.NET template, the template must parallels the output and this organizes the template. In an XML literal template, you provide the organization with Visual Studio then providing navigation. Code is organized my regions, nested classes and member. Each is its own method preceded by Region, NestedClass or Member. This organizes the template. The entry points create a hierarchy working down to the local code (navigation mechanism #1):

   Protected Overrides Function GenerateFile() As String
      Return _
<code>
   <Serializable()> _
Public Class <%= mObjectData.ClassName %>
   Inherits BusinessBase(Of <%= mObjectData.ClassName %>)
   <%= RegionBusinessMethods() %>
   <%= RegionValidationRules() %>
   <%= RegionAuthorizationRules() %>
   <%= RegionFactoryMethods() %>
   <%= RegionDataAccess() %>
   <%= RegionExists() %>
End Class
</code>.Value
   End Function

And

#Region "Business Methods"
Private Function RegionBusinessMethods() As String
Return _
<code>
#Region " Business Methods "

   <%= From prop In mObjectData.Properties Select MemberPropertyFields(prop) %>
   <%= From child In mObjectData.Children Select MemberChildFields(child) %>
   <%= From prop In mObjectData.Properties Select MemberPropertyAccess(prop) %>
   <%= From child In mObjectData.Children Select MemberChildAccess(child) %>
   <%= MemberIsValid() %>
   <%= MemberIsDirty() %>
   <%= MemberGetId() %>

#End Region
</code>.Value

End Function

The carefully named methods also allow you to use alphabetical tools in Visual Studio including the combo box in the upper right of the editor and a class diagram (navigation mechanism #2) which works only because you can predict the name of the member you want from the name of the output member and no other members begin with “Member”, “Region” or “NestedClass”

The third navigation approach is that the template matches the structure of the output as closely as practical. Thus if the output has a region names “Business Methods” the template does as well. The order of items in the template is top to bottom closely paralleling the order of the output. This allows you to cruise down in the file.

Template organization is the first step to great templates.

The Punch Line

Kathleen — Sun, 10 Feb 2008 23:21:00 GMT

I mentioned a few days ago that there was a punch line for the XML Literal Code Generation. I planned to unveil this slowly, but it just sprung out of the box when Carl posted episode #102 (which I thought was due for next Friday.

And I’m afraid that I need to add that I was sick during the taping and my brain running at half capacity. If anything isn’t clear, please let me know.

No matter.

You get to hear sooner.

I’ll be unveiling the details in about the same time frame, so you’ll have the big picture by next week.

So, the drumroll please…

I can create excellent readable templates that output code in Visual Basic or C# - no CodeDOM involved.

I’ll show you how to do that. Watch for more here.

Code for DNR TV # 97 and #98 Sample Code

Kathleen — Sun, 10 Feb 2008 23:14:00 GMT

Sometimes I drop the ball and I did on getting these samples posted. Note that this is for the two episodes I did on 3.5 languages. I think these episodes are a good run through of the features in both languages. The title wound up with the word "Compare" in the title. I'm not sure I really compare them, beyond seeing them side by side, which I think is useful to understand the underlying mechanisms.

You can get the code here.

I will not be posting code for the code generation episode which is episode #102. That code was too transitory for me to want to release. I will be releasing similar code in my blog over the next week or so.

XML Literal Code Genaration - Code again again

Kathleen — Sun, 10 Feb 2008 16:53:00 GMT

Crap.

For now, I'm just removing the coloring. Paste this into VS for coloring. It's much prettier:

   Private Function MemberDataPortalFetch() As String
      ' TODO: Add special handing for timestamp
      Return _
<code>
   Private Overloads Sub DataPortal_Fetch(ByVal criteria As Criteria)
      Using cn As New SqlConnection(<%= mObjectData.ConnectionStringName %>)
         cn.Open()
         Using cm As SqlCommand = cn.CreateCommand
            cm.CommandType = CommandType.StoredProcedure
            cm.CommandText = "get<%= mObjectData.ClassName %>"
            <%= From prop In mObjectData.PrimaryKeys Select _
               <code>
            cm.Parameters.AddWithValue("@<%= prop.Name %>", criteria.<%= prop.Name %>)
               </code> %>

            Using dr As New SafeDataReader(cm.ExecuteReader)
               dr.Read()
               With dr
                  <%= From prop In mObjectData.Properties Select _
                     <code>
                  m<%= prop.Name %> = <%= GetReadMethod(prop) %>("<%= prop.Name %>")
               </code>.Value %>

                  ' load child objects
                  .NextResult()
                  <%= From child In mObjectData.Children Select _
                     <code>
                  m<%= child.Name %> = <%= child.Name %> .Get<%= child.Name %> (dr)
                     </code>.Value %>
               End With
            End Using
         End Using
      End Using
   End Sub
</code>.Value
   End Function

XML Literal Code Generation - Code again

Kathleen — Sun, 10 Feb 2008 16:32:00 GMT

OK, that code listing got friend on load. Let me try again.

Private Function MemberDataPortalFetch() As String Return _<code> Private Overloads Sub DataPortal_Fetch(ByVal criteria As Criteria) Using cn As New SqlConnection(<%= mObjectData.ConnectionStringName %>) cn.Open() Using cm As SqlCommand = cn.CreateCommand cm.CommandType = CommandType.StoredProcedure cm.CommandText = "get<%= mObjectData.ClassName %>" <%= From prop In mObjectData.PrimaryKeys Select _ <code> cm.Parameters.AddWithValue("@<%= prop.Name %>", criteria.<%= prop.Name %>) </code> %> Using dr As New SafeDataReader(cm.ExecuteReader) dr.Read() With dr <%= From prop In mObjectData.Properties Select _ <code> m<%= prop.Name %> = <%= GetReadMethod(prop) %>("<%= prop.Name %>") </code>.Value %> ' load child objects .NextResult() <%= From child In mObjectData.Children Select _ <code> m<%= child.Name %> = <%= child.Name %>.Get<%= child.Name %> (dr) </code>.Value %> End With End Using End Using End Using End Sub</code>.Value End Function

XML Literal Code Generation

Kathleen — Fri, 08 Feb 2008 17:20:00 GMT

You must use VB for XML Literal Code Generation. Sorry, that’s life. I don’t have a crystal ball on this, but for now even if you’re a C# programmer you’ve got to suck it up and use VB. The code you output can be VB or C# however, but the template itself has to be written in VB. If you’re allergic to VB, get shots.

So, what does a microcosmic view of a XML literal template look like? I’ve translated a section of a CSLA template. CSLA is Rocky Lhotka’s architecture and this is a version 2.1 because I think it clarifies the template process:

Private Function MemberDataPortalFetch() As String Return _<code> Private Overloads Sub DataPortal_Fetch(ByVal criteria As Criteria) Using cn As New SqlConnection(<%= mObjectData.ConnectionStringName %>) cn.Open() Using cm As SqlCommand = cn.CreateCommand cm.CommandType = CommandType.StoredProcedure cm.CommandText = "get<%= mObjectData.ClassName %>" <%= From prop In mObjectData.PrimaryKeys Select _ <code> cm.Parameters.AddWithValue("@<%= prop.Name %>", criteria.<%= prop.Name %>) </code> %> Using dr As New SafeDataReader(cm.ExecuteReader) dr.Read() With dr <%= From prop In mObjectData.Properties Select _ <code> m<%= prop.Name %> = <%= GetReadMethod(prop) %>("<%= prop.Name %>") </code>.Value %> ' load child objects .NextResult() <%= From child In mObjectData.Children Select _ <code> m<%= child.Name %> = <%= child.Name %> .Get<%= child.Name %> (dr) </code>.Value %> End With End Using End Using End Using End Sub</code>.Value

End Function

This is .NET code, which you can see because the template output is inside a .NET function named MemberDataPortalFetch which returns a string constructed with XML literals. By offsetting with XML literals, the code of the function is easily seen – it’s in gray. All of the gray code will be in the output, none of the yellow, blue or black code will be in the output, although it will certainly affect the output.

Where replacements are necessary, XML literals have “embedded expressions” which I have the habit of calling “expression holes.” I’ll talk in another post about what mObjectDatais, but an embedded expression can contain whatever expression you need. An expression is something that returns a value within a single line. So, you can call methods and properties, but you can’t call a subroutine.

Embedded expressions can contain LINQ as shown in the third embedded expression. Here we want to add parameters for all the primary keys. Within the LINQ Select clause, I can use any string. Including the nested code block creates a more readable template and consistency, among other things.

A quick glance may leave you thinking that this is just a variation of the ASP.NET style template syntax used by CodeSmith, CodeBreeze, MyGeneration and others. However, it’s quite different. To begin with its organized into members of a template class. Then, the logic is expressed in expressions, not direct template logic. Finally, you can nest back and forth between output text and code to any depth you require. The code blocks within the LINQ statements above contain embedded expressions. These embedded expressions could be further LINQ statements with further code blocks, etc.

I think there are right and wrong ways to build templates with XML literals. I’ll spend the next couple of posts talking about template organization and metadata handling.

Let’s Talk Templates

Kathleen — Thu, 07 Feb 2008 22:07:00 GMT

Code generation templates have several competing pressures – they need to be easy to use and they need to be powerful. They need to be flexible but encourage best practices. If you ever say “my templates got me started then I had to customize them” you’re templating can be better. I’m not aware of any situation where you can’t isolate the handcrafted code from the generated code sufficiently to keep code generation active for the entire life of your application.

When I say full life cycle, I mean from initial prototypes to finally pulling the switch in a decade. Yep a decade. Code generation should aim for evolution and extremely long project life cycles. The metadata of the project can far outlive the technology, even outlive the database. That requires a deep commitment to code generation, and if you don’t have that commitment, the flip side is that code generation can pay back today. At that moment when it feels easier to pull a file out of code generation, or to repeat a similar block of code in two places instead of pushing it into your code gen templates, that’s where the commitment comes in. With or without it, you get payback; it’s just greater the deeper your commitment is to the metadata and templates.

My current client pushes me because he is even more committed to the decade long lifecycle based on code gen than I am. And he’s right. Every time he’s pushed to get something into templates or do code gen better, it’s taken less work and gotten us further in the short term than I expected.

What if you could combine the simplicity of CodeSmith with the power of XSLT?

And what if you could do it primarily in a language and architecture you already know, or should be learning anyway?

Interested?

This is the first post in a blog series to present this templating style. I also covered in a.NET Rocks show I recorded yesterday. I’m very excited about it, but I want to introduce the basics before I tell you the real punch line. This template style is the best way to generate code today and it’s based entirely on tools you’ve got on your desktop right now.

Look for more here and in the .NET Rocks TV episode scheduled to appear Feb. 15^th. The beautiful thing about the techniques I’ll show is that you can smoothly transition from simple to complex templates dialing in the exact complexity appropriate for your organization and application. And, just wait for the punch line.

DNR TV on .NET 3.5 Languages - Part 1 Posted

Kathleen — Fri, 18 Jan 2008 19:55:00 GMT

Check out .NET Rocks TV this week for the first of two parts on .NET 3.5 languages - that's right C# 3.0 and VB 9 together. It makes sense since most features cross over, and where they don't you'll want a handle on the differences.

First Experience with Pipeline Builder

Kathleen — Fri, 11 Jan 2008 00:01:00 GMT

Wow!

System.AddIn the easy way

Build and interface, press a button, build an add in, build a host.

Wow!

So, the pipeline builder tool really works.

You have to do a couple of things not mentioned in the documentation I used. Actually the documentation needs a little work, but its there and not too bad.

Follow the directions for the VS tool

Also

Copy the Blank project to a place to work

Rename the project as desired, or test first then rename later (I am mortified by the idea of millions of "Blank Pipeline Project" assemblies floating aorund)

Add an interface to the Contracts file. It must have the AddInContract attribute and inherit from IContract. If you want to borrow mine:

using System;using System.Collections.Generic;using System.Linq;using System.Text;using System.AddIn.Contract;using PipelineHints;using System.AddIn.Pipeline; namespace KadGen.FirstPipelineBuilderTest.Contracts{ [AddInContract()] public interface GenerateFiles : IContract { string GenerateForObject(string objectName, ref string fileName); string GenerateForStoredProc(string objectName, ref string fileName); string GenerateForDataTable(string objectName, ref string fileName); }}

If you leave off the IContract you’ll get projects but no generated cs files. Ask me how I know!

The only downside is that a lot of you will never experience the pain of creating them by hand and will be therefore so jaded that you don’t realize that the release of this tool is cause for world-wide celebrations!

I'll blog about using this with VB after I've tested that more.