Cluebat-man to the rescue : .NET

Using reflection to ease code maintenance

vanDooren — Tue, 02 Dec 2008 11:35:00 GMT

Currently I am programming an application for performing code analysis on the software that controls the production process. Apart from the parsing of the code, and iterating across a hierarchical representation of the expression tree, all these rules (verifications) have to be executed against all the code.

I could change the code verifier every time I implemented a new rule, but instead I used reflection to do this for me. Every verification that has to be done is represented by a class that implements the right interface. This interface defines the prototype of the method that performs the verification.

interface IActionVerification
{
void Verify(CodeContext context);
}

The context variable contains the complete context of the portion of the code that is being looked at: the current action, step, function block and phase.

When a new code verifier instance is create, it will use reflection to examine the current assembly, and extract all classes that implement this interface.

public CodeVerifier()
{
    actionVerifications = new List<IActionVerification>();
    var currAssembly = Assembly.GetExecutingAssembly();
    foreach(Type t in currAssembly.GetTypes())
    {
      if (null != t.GetInterface(typeof(IActionVerification).Name))
        actionVerifications.Add(
          (IActionVerification)
             currAssembly.CreateInstance(t.FullName));
    }
}

This way, it is very easy to populate the list of all verifications to do. The code verification itself is conceptually as simple as this:

foreach (var verification in actionVerifications)
verification.Verify(context);

Implementing a new verification is the as easy as creating a new class and implementing the interface. After that everything will be done automatically. Ading a new verification is as easy as this:

class ActionExpressionCount : IActionVerification
{
    public void Verify(CodeContext context)
    {
      if (context.Action.ParsedExpression.Count < 1)
        ActionExpressionCountInvalid.LogIssue(context);
    }
}

Hip hip hooray for the .NET framework!

Extending a native C++ project with managed code

vanDooren — Wed, 03 Sep 2008 09:59:00 GMT

One question that comes up from time to time in the newsgroups is ‘I have a native C++ project and I want to extend it with Managed code (e.g. Windows Forms). What do I do?’

The answer is not so complex. It is fairly easy to extend native projects with managed code. In this article I’ll explain how.

What NOT to do

To quote Kate Gregory ‘In the name of all that is good and right: Do not set /CLR for the entire project’.

/CLR tells the compiler that it should compile a source file as C++/CLI code in which it finds both native and managed C++ code. Theoretically you could specify this for the whole project, but this has some serious consequences.

Apart from the fact that it is possible for the compiled output size to explode, there can be significant problems with COM and CRT initialization. I am no interop expert in the matter like Marcus Heege or Kate, but I am willing to take their word for it.

Furthermore, your existing project is possibly validated by QA en unit tested 7 days from Sunday. You don’t want to travel that road again if you can prevent it.

What to do

You want to add managed stuff to your project while impacting the rest of the code as little as humanly possible. To achieve this you have to compartmentalize the managed stuff and give it a native interface that you can use in the native parts of you projects without any hassle.

I think I can best explain this with a practical example.

For the sake of this example I have a native DLL project that exports a function that returns an integer value. Perhaps this function got that integer value previously through an MFC interface, and now that value has to come from a Windows Forms interface.

Add the windows form to your project

This is the first and also the easiest step of the process. Rightclick your project and select ‘Add…->New Item’ and select ‘UI->Windows Form’ in the Visual C++ dialog.

You will then get a message box, telling you that your project will be converted to a managed project. Here you click ‘yes’. Even though you don’t want to compile your project with /clr, Visual Studio itself needs to know that you are doing managed stuff in your project. This is not necessary for compiling your project, but if you want to have the benefit of working with the forms designer and other .NET related things, then it would be a good idea. And you cannot add the form if you choose ‘No’ so it’s not like you have much choice anyway.

Now you have a .NET Windows Form in your code. You can verify the file specific settings if you want, but the cpp file will be compiled with the /clr flag set, and without the use of precompiled headers. Not using precompiled headers is important, because the default precompiled header is compiled without /clr set, and this would lead to conflicts.

Of course, if you need to, you can create a copy of StdAfx.h/cpp cpp files called ‘StdAfxClr.h/cpp’ add them to your project, and configure them to create a managed precompiled header, which you could then use for all files that are compiled with /clr. This is not necessary for small projects, but it could be a useful optimization in large projects.

The interface layer

At this point you can compile and link the entire project, but your native code isn’t yet using the new managed functionality. And because the native code will not be compiled with /clr, it will never do so. In order to make that happen, you will need a thin layer between the native and managed code. This layer will have a native C or C++ interface which can be called by the native code, and a managed implementation that will do all the .NET stuff.

In our example, the interface is 1 simple C style function in Interface.h:

int __stdcall DoManagedStuff(void);

And this function has a simple implementation in Interface.cpp:

int __stdcall DoManagedStuff(void)
{
      DemoForm ^df = gcnew DemoForm();
      df->ShowDialog();
      return df->Value;
}

Interface.h and Interface.cpp are 2 new files that you have to add to your project. You have to manually configure the cpp file to be compiled with /clr, and to not use precompiled headers.

As you can see, the interface layer provides a clean native interface for the managed stuff that you want your code to perform. Of course, you are not limited to C style interfaces. You can also work with classes and make classes with a native interface and a managed implementation. But there are a couple of issues that you need to be aware of. If you want to go there, then it would be a good idea to read the paper written by Herb Sutter which explains the rationale behind C++/CLI, as well as some of the limitations and pitfalls.

Actually, reading that paper is a good idea for anyone working with C++/CLI who also cares about understanding what is actually going on behind the scenes.

Using the interface layer

All the hard work is done. Now you can simply include Interface.h in your native code files, and call ‘DoManagedStuff’ where appropriate.

#include "stdafx.h"
#include "InterfaceLayer.h"
int __stdcall DoFoo(void)
{
return DoManagedStuff();
}

Conclusion

As you can see, extending native code with managed code is not so hard. At least, it isn’t if you maintain a clean break between managed and native code. If you cannot do this, things might become more difficult. For example, using .NET controls on an MFC dialog, or using .NET remoting in a native COM project are things that can be much more tricky.

Another thing you should be aware of is that not all compiler switches can be used in conjunction with the /clr switch. The compiler will inform you if it detects this. The combinations which are not allowed are documented, and can be found in the documentation of the /clr switch itself.

This post was not meant to be an exhaustive how-to to those complex scenarios, but instead an explanation behind the basic ideas.

The demo project for this article can be downloaded under the MIT license as usual.

Excel and OleDb stupidity

vanDooren — Fri, 29 Feb 2008 20:47:00 GMT

This afternoon I was working on a Windows Forms app for the finance guys, to help them allocate costs to systems, proportional to predefined allocation keys.

The app has to import Excel spreadsheet files and perform all the database actions.

The key columns to uniquely identify a cost are text values. Unfortunately, some of those key values do not contain letters, but only numeric characters. And this is where it sucks to have to work with Excel.

This is because Excel knows no datatypes. You can specify formatting for cells, but Excel ignores that when someone asks for cell data. Instead, it scans the first couple of rows to guess the datatype. By default it checks the first 8 rows. If those contains mixed values, it uses a specified default type for those values that are not numeric.

This means that if you use OleDb to retrieve values, the column type can vary per row.

What can you do about it? Well nothing really. You can tweak some values, and hope that you will never encounter the corner cases that you cannot do anything about. There is plenty of information to be found. One good explanation was this.

However, one of the finance guys gave me an incredibly useful tip, which you can use as a workaround. You have to manually make a couple of easy modifications to the excel file, and Excel will always use the text data type.

Assume that the offending data is in column 1, and that the first row is the header row.
You insert 2 empty columns to the right of the offending column.
Put an ' in the first cell of the first empty column (B2).
Then enter =B2&B1in the cell B3.
Select B2 and B3, and double-click the bottom right corner of the selection box. This will copy the data from there to the last row.
B3 will now hold the data as text, because the ' tells Excel that no matter what is in the cell, it is text.
Then you copy column 3, paste it as 'values only' over column 1.
Delete columns 2 and 3 again.

Now you can read the column in OleDb and always get text values for that column.

LINQ To SQL......Server 2005

vanDooren — Wed, 05 Dec 2007 12:06:00 GMT

I am programming a small application for our finance department, for which I wanted to use LINQ for performing the data access, using Visual Studio 2008 Professional.

Unfortunately -and no thanks to MSDN- I found out that currently there is only support for SQL Server 200x, and preferably SQL server 2005, and not every version of SQL Server 2005 at that.

Finance uses MS Access 2003. They already use the database, and it has some exotic connection to MFGPro, involving terminal windows and an FTP connection. So I am not going to move their stuff over to SQL Server 2005 at the moment because the app and the database have to work by the end of the year.

I really think this should have been documented properly. It seems that Microsoft does not want to outright say that they dropped Access from the supported databases list for LINQ. You can't even find a 'supported databases' list on the LINQ home page. According to a forum post by a Microsoft Employee, dropping Access was not a technical problem, but a matter of scheduling and other external factors. Yaaay.

But for now it might as well be named 'LINQ To SQL Server 2005'. I am curious if they will add support for Access later on. I wouldn't hold my breath though.

Back to ADO.NET for now, I guess.

I stand corrected: not all C++ programmers hate VB

vanDooren — Sun, 01 Jul 2007 21:02:02 GMT

As I already mentioned before, it's a small world after all. (Hearing that annoying Disney tune yet in your head J).

I am writing an article about C++/CLI in which I use a quote from my fellow VC++ MVP Kate Gregory. I did a quick search to find her blog in order to link it to her name in the article. I read a couple of blog posts and to my surprise I found one in which she mentioned a blog post I wrote earlier in which I talked about the viability of C++ in this day and age.

In my blog post I mention that I think VB is ugly and that most people in the C++ forums would agree with me.

Apparently there is at least 1 VC++ MVP (Kate) who likes VB. That is at least 1 very skilled and well respected C++ programmer who likes VB.

So I hereby concede that VB is a full fledged member of the .NET programming world, and that there are even C++ programmers who like it. Just as long as I don't have to like it.

Executing LabVIEW VIs through C style function pointers via .NET

vanDooren — Thu, 16 Nov 2006 11:46:00 GMT

Problems exist to be solved.

Some problems are so complex that you cannot solve them when you need to. With other problems it is not a matter of complexity, but a matter of not having all the pieces of the puzzle.

Some problems are so interesting that it pays to keep them in the back of your head, just in case you ever find that missing piece that allows you to solve them.

One of those problems is that with the current LabVIEW external library interface it is impossible to execute LabVIEW VIs in a DLL through a C style function pointer.

This feature does not exist, even though lots of programmers would really benefit from it.

2 weeks ago I was active on the Visual C++ newsgroup as usual, when learned –by pure coincidence- about an advanced function in the .NET framework called Marshal.GetFunctionPointerForDelegate.

I immediately realized that this was the key to solving the old C style function pointer issue

Problem description

Suppose that we have a DLL that exports a function. This function takes a function pointer as an argument, to be executed sometime later to pass some data to the calling application.

The Test DLL code

The idea is to test the ability to run a LabVIEW VI through a function pointer. To test this we need a DLL which will test this for us.

The following very simple test function has been created for this purpose.

This is the expected prototype of the callback function

typedef void (__stdcall *fSimpleCallBack)(int i);

And this is the DLL function that will perform our test:

//Test of the simple callback

void Simplefoo(int i, fSimpleCallBack cb)

{

(cb)(i); //invoke the callback function.

}

As you can see, the LabVIEW VI will be executed with the supplied parameter through the supplied function pointer.

Principal solution

The very first action has to be creating a .NET delegate of which the signature matches the signature of the CallBack function that is required.

Then a proxy class is created with only 2 members: an event that is based on the new delegate function, and a method called ‘GetDelegate’.

LabVIEW has the ability to link a LabVIEW VI with a .NET event. The VI prototype is generated automatically, and matches the event delegate signature.

After registering a VI for the specified event, LabVIEW uses the GetDelegate method of our proxy class to extract the registered delegate from the event. That delegate is then converted to a function pointer via the ‘GetFunctionPointerForDelegate’ method of the Marshal class.

That function pointer can then be supplied to a native function that can execute the LabVIEW VI directly through the function pointer without requiring any modification at all.

A note on GetFunctionPointerForDelegate

The documentation for this method is sparse. It seems that VS2005 and .NET2.0 were rushed out of the door without taking the time to fully document the more advanced functions and providing code samples in the documentation.

Because of this, it is easy to miss that GetFunctionPointerForDelegate returns a pointer that uses the __stdcall calling convention by default.

It is also easy to overlook the fact that you can change this.

If you need a function pointer that uses another calling convention - e.g. __cdecl – then you can achieve this by applying the UnmanagedFunctionPointerAttribute attribute to the delegate definition.

The .NET code

namespace FunctionPointerProxy

{

public class SimpleProxy

{

//Marshal a simple integer parameter as 32 bit integer.

public delegate void SimpleProxyDelegate(

[MarshalAs(UnmanagedType.I4)] int i);

//export an event because that is the only way we are going to get

//LabVIEW to give use a delegate.

public event SimpleProxyDelegate SimpleEvent;

//extract the delegate from the event. If there was no event

//registered, this will trigger an exception which is caught by the

//LabVIEW .NET interface node. No need to worry about it here.

public Delegate GetDelegate()

{

return SimpleEvent.GetInvocationList()[0];

}

This is all the additional code you really need to make it work. The delegate function has to mirror the native callback prototype for 100%. To achieve this you have to use the MarshalAs attribute on each delegate parameter to insure correct behavior.

The event simply has to exist because LabVIEW has only 1 way to produce a delegate for a VI, and that is by registering a VI to an event.

And even then, the delegate is never directly accessible, so we have to create a ‘GetDelegate’ method that extracts the registered delegate, and returns it to LabVIEW.

As far as the .NET framework is concerned, we could simply make the event and the GetDelegate method static to remove the need to instantiate an object that we won’t need for anything else. But there are 2 reasons why we don’t do that:

LabVIEW does not support registering VIs to static events.
Our GetDelegate function would need way to know which delegate you want to get, in case there are multiple instances of callback VIs. Since the event is tied to an instance, it is easy enough to keep track of the events in LabVIEW.

The LabVIEW code

LabVIEW code is graphical, so I have chosen to mark the different steps in the block diagram below, and then add an explanation of each step in a numbered list.

Instantiate a new SimpleProxy Class. We only need it to access the event. Sadly, LabVIEW does not yet support static events.
Register a VI for the Event. The way to do this is to wire the SimpleProxy refnum to the event source and select the appropriate event. Right-click 'VI Ref' and select 'Create Callback VI'. This will automatically generate a VI with the correct input and output parameters for you. This will also automatically wire a static VI reference to the VI Ref input. User parameter can be any LabVIEW type at all that you want to pass to the VI when the .NET event is triggered. We don't need it, but if we did we would have to wire it before we generate the callback VI because it changes the VI prototype.
Now that the event is registered, we can simply get the .NET delegate that represents the VI callback out of it.
Use a static member of the Marshal class to get a funtion pointer to the delegate. This is where the magic happens. A native stub is generated for the delegate at runtime. The unmanaged parameter list is built according to the Marshalling attributes that were attached to the delegate signature.
Cast the Delegate IntPtr to an Int32. Pointers are 32 bit on the x386 platform. We need this 32 bit value to supply it to the DLL function call that expects a 32 bit function pointer. Note that this function pointer uses the standard calling convention by default (see below for more info).
We no longer need the FunctionPointer IntPtr. As long as the delegate itself is still in memory, the native stub will be valid.
Execute the DLL function. This function will use the supplied callback function pointer to execute the callback VI. It will pass the supplied int parameter to the callback vi at the time of the function call.
Unregister the Callback VI from the event, and close the various references that were opened.

Running the test

As is sometimes said: the proof is in the pudding.

To verify that everything works as advertised, I have put a simple dialog box in the callback VI that displays the value of the integer parameter.

Simply run the example and you will see the following dialog box:

Advanced implementation

The principal solution works, but is not yet ideal. Most importantly, there is too much clutter on the LabVIEW diagram for this solution to be aesthetically pleasing.

There are too many manual actions that have to be repeated each time you implement a callback function. I also wanted to show you an example of a callback function that is a bit more complex.

Test DLL

Consider the following callback prototype that is a bit more complex than the previous one:

typedef void (__stdcall *fAdvancedCallBack)(

int i,

char* aString,

wchar_t* wString);

Here we need to supply an integer, an ASCII string and a UNICODE string. The function exported from the DLL has the following implementation:

void Advancedfoo(int i, char* aString, fAdvancedCallBack cb)

{

BSTR uString;

int aLength = lstrlenA(aString);

int uLength = ::MultiByteToWideChar(

CP_ACP, 0, aString, aLength, 0, 0);

if(0 < uLength)

{

uString = SysAllocStringByteLen(0, uLength);

::MultiByteToWideChar(

CP_ACP, 0, aString, aLength, uString, uLength);

}

else

uString = L"";

//invoke the callback function through its function pointer.

(cb)(i, aString, uString);

}

LabVIEW itself does not support UNICODE strings, so we manually transform the supplied ASCII string to UNICODE. Apart from that the function body only contains a function call via the supplied callback pointer.

.NET code

[UnmanagedFunctionPointerAttribute(CallingConvention.Cdecl)]

public class AdvancedProxy : IProxy

{

public delegate int AdvancedProxyDelegate(

//A simple I4 parameter. Nothing special

[MarshalAs(UnmanagedType.I4)] int i,

//An ASCII string. this will be marshalled as UNICODE on the

//.NET side and ASCII on the unmanaged side.

[MarshalAs(UnmanagedType.LPStr)] string aString,

//A UNICODE string. This will be marshalled as UNICODE on the

//.NET side, and UNICODE on the unmanaged side.

[MarshalAs(UnmanagedType.LPWStr)] string wString);

public event AdvancedProxyDelegate AdvancedEvent;

public Delegate GetDelegate()

{

return AdvancedEvent.GetInvocationList()[0];

}

The meaning of the different parts is the same as with the SimpleProxy example. The only real difference – apart from the delegate signature of course – is that all classes that are used as a proxy between a LabVIEW VI and a function pointer should implement the interface IProxy.

This interface has the following definition:

public interface IProxy

{

Delegate GetDelegate();

}

This allows us to make the LabVIEW code for getting a function pointer generic, instead of tied to the type of the callback delegate.

LabVIEW code

As you can see, the LabVIEW code is vastly simplified.

The only thing that we cannot make generic is the event registration, because the event and as a result the VI reference are strongly typed.
The process of retrieving the function pointer is generic because all proxy classes now implement the IProxy class for retrieving the event delegate.
The call to the test function in the DLL now also takes a string parameter that gets expanded to UNICODE inside the test function to supply the callback function with the correct parameters.
Cleanup of the event and references is also generic.

There are 2 new VIs. These are generic, so you can reuse them without limitation. This also makes it easy to support an asynchronous callback mechanism. You can register the event and supply the function pointer to the DLL.

The DLL stores it inside, and uses it any time it needs to. Meanwhile, your LabVIEW program can continue processing. You can keep the references cluster around – for example in a buffer – until such time as you want to disable the callback mechanism again.

Running the demo code

The Visual studio solution is for VS2005, and can be built with all versions of VS2005.

The LabVIEW project uses version 8.2. You need at least version 8.0 to have decent .NET support, and I used 8.2 because that is the latest version at this moment.

To run the demo, simply extract the sources and open and build the Visual studio solution. Then open the LV project and run the sample of your liking.

Conclusion

This article has outlined a relatively simple way to directly execute a LabVIEW VI through a C style function pointer without having to write C or C++ code. The demo code for this article is available under the MIT license as usual.

You still need to provide a .NET class library, but the code involved is so minimal and trivial that it should not stop anyone.

This approach also makes it possible to use the wealth of win32 API functions that require a callback function without having to resort to writing C or C++ code.

In the time it took me to write this article (which took longer than writing the code, as usual) I found 3 other ways to achieve the same result.

One is an ugly kludge that only fakes the callback functionality. The other performs a true callback to a LabVIEW VI, but it is even uglier. The third one is fairly elegant but has some other limiting factors.

I will explain those 3 lesser solutions in another article. Despite the fact that only one of them is useful; the principle behind all 3 of them might be of use in another situation.

Finally, the whole LabVIEW and function pointer issue has existed for a long time among LabVIEW programmers. I have done a bit of research, and as far as I can tell, I am the first to ever solve this problem without requiring C or C++ code.

So here and now I claim to be the first to implement and publicize a working C style function pointer callback mechanism in LabVIEW!

In my next article I will follow up with an example involving structures (Clusters in LabVIEW) and n dimensional arrays.