Cluebat-man to the rescue : Interoperability

Using reflection to ease code maintenance

vanDooren — Tue, 02 Dec 2008 11:35:00 GMT

Currently I am programming an application for performing code analysis on the software that controls the production process. Apart from the parsing of the code, and iterating across a hierarchical representation of the expression tree, all these rules (verifications) have to be executed against all the code.

I could change the code verifier every time I implemented a new rule, but instead I used reflection to do this for me. Every verification that has to be done is represented by a class that implements the right interface. This interface defines the prototype of the method that performs the verification.

interface IActionVerification
{
void Verify(CodeContext context);
}

The context variable contains the complete context of the portion of the code that is being looked at: the current action, step, function block and phase.

When a new code verifier instance is create, it will use reflection to examine the current assembly, and extract all classes that implement this interface.

public CodeVerifier()
{
    actionVerifications = new List<IActionVerification>();
    var currAssembly = Assembly.GetExecutingAssembly();
    foreach(Type t in currAssembly.GetTypes())
    {
      if (null != t.GetInterface(typeof(IActionVerification).Name))
        actionVerifications.Add(
          (IActionVerification)
             currAssembly.CreateInstance(t.FullName));
    }
}

This way, it is very easy to populate the list of all verifications to do. The code verification itself is conceptually as simple as this:

foreach (var verification in actionVerifications)
verification.Verify(context);

Implementing a new verification is the as easy as creating a new class and implementing the interface. After that everything will be done automatically. Ading a new verification is as easy as this:

class ActionExpressionCount : IActionVerification
{
    public void Verify(CodeContext context)
    {
      if (context.Action.ParsedExpression.Count < 1)
        ActionExpressionCountInvalid.LogIssue(context);
    }
}

Hip hip hooray for the .NET framework!

C++ keyword of the day: export

vanDooren — Wed, 24 Sep 2008 10:48:00 GMT

The export keyword is a bit like the Higgs boson of C++. Theoretically it exists, it is described by the standard, and noone has seen it in the wild. :)

Before I get flamed to hell and back: that last part is not entirely true. There is 1 C++ compiler front-end in the world which actually supports it. That is the one made by EDG (Edison Design Group). This is used by Comeau which claim to have the only compiler that is 100% standards compliant and Borland (for which the support of export is a bit vague). I heard that the Intel compiler shipped with it, but I don’t know if this is true or not.

What’s all the fuss about?

It seems a bit strange that a language should contain a keyword that all compiler makers refuse to implement. Comeau has some interesting content here as does Wikipedia.

The reason that I believe 'export' has never really taken off is that is seems to be an significant effort to implement it, combined with the fact that is doesn't achieve all that much. The export keyword allows a C++ programmer to declare a template class in a header, and then provide the implementation separately in a cpp file. This is the normal C++ way of doing things. Unfortunately, there's a snag.

Template class implementations cannot be compiled on their own. Template code itself is meaningless without specification of template arguments. Thus it is that a compiler doesn't know what to do with an implementation until the template class is used somewhere. At that point the template arguments will be known, and the compiler can compile the template class.

Since the template is only compiled then, the easiest thing to do would be to do what most compilers do now: demand that the implementation is in scope when it is used somewhere. I.e. the implementation has to be put in a header file, and that header file has to be included by the source file that is being compiled. the reason for this is that the compiler would simply not know where to look for it if the implementation was in a cpp file somewhere.

The export keyword would solve this by telling the compiler 'Look somewhere else for the implementation'. The compiler would then have to compile the implementation for that class, and save the object code somewhere for that combination of template arguments.

The export keyword sounds like a great thing, but it doesn't solve that many problems.

Separation of interface and implementation

The separation of interface and implementation is one of the cornerstones of C++ philosophy. The traditional way is to have the declaration in a header file, and then put the implementation in a cpp file.

Since a template implementation cannot be compiled anyway, we could just as well put it in a header file.

And then we could also make 1 header file with just the declarations, and underneath that include the header file(s) containing the implementation. The nice thing is that you can then put specializations in different headers to keep the code comprehensible.

This way we keep true to the principle of separation without needing the ‘export’ keyword

Distribution

You can’t distribute template classes in a binary form, no matter whether export is supported or not.

Because templates cannot be compiled on their own, you cannot link them into a static or dynamic library. The only thing you can do is to distribute the source code.

Templates are distributed as source code. The export keyword cannot do anything about that. And if you want to distribute templates, what are you going to do? You can develop your classes using ‘export’, but then they will be usable only by a very limited subset of compilers.

If you develop your classes as if ‘export’ doesn’t exist, then they can be used by all compilers.

Of course you could do both, and use ugly macro magic to construct your code in such a way that would leave the choice to the client programmer. Some people do this. But that is kind of ugly and time consuming. Furthermore, it’s not like it gets you any more clients, because they will be able to use your code in any case.

Compiler performance

It is true that our scheme of using header files is less efficient than using the export keyword. Without ‘export’, a template is compiled whenever it is used. You can mitigate this somewhat by including commonly used templates in a precompiled header file. But the impact will only be somewhat lessened.

And whenever an implementation detail changes, every source file which includes the template has to be recompiled. This can be a real pain in large projects.

Conclusion

‘export’ is facing the chicken or the egg dilemma.

‘export’ is neglected by most compiler vendors because it is not an important language feature -> You cannot use it in code that has to be portable (which is ironic, since it is a standard keyword) -> it is not used in many scenarios -> there is no large codebase depending on support for ‘export’ -> there is little incentive for compiler vendors to start supporting it.

Make no mistake: I would appreciate it if I could use ‘export’, if only because it allows me to organize my template implementations in a standard manner, but I don’t think it’ll happen soon.

Extending a native C++ project with managed code

vanDooren — Wed, 03 Sep 2008 09:59:00 GMT

One question that comes up from time to time in the newsgroups is ‘I have a native C++ project and I want to extend it with Managed code (e.g. Windows Forms). What do I do?’

The answer is not so complex. It is fairly easy to extend native projects with managed code. In this article I’ll explain how.

What NOT to do

To quote Kate Gregory ‘In the name of all that is good and right: Do not set /CLR for the entire project’.

/CLR tells the compiler that it should compile a source file as C++/CLI code in which it finds both native and managed C++ code. Theoretically you could specify this for the whole project, but this has some serious consequences.

Apart from the fact that it is possible for the compiled output size to explode, there can be significant problems with COM and CRT initialization. I am no interop expert in the matter like Marcus Heege or Kate, but I am willing to take their word for it.

Furthermore, your existing project is possibly validated by QA en unit tested 7 days from Sunday. You don’t want to travel that road again if you can prevent it.

What to do

You want to add managed stuff to your project while impacting the rest of the code as little as humanly possible. To achieve this you have to compartmentalize the managed stuff and give it a native interface that you can use in the native parts of you projects without any hassle.

I think I can best explain this with a practical example.

For the sake of this example I have a native DLL project that exports a function that returns an integer value. Perhaps this function got that integer value previously through an MFC interface, and now that value has to come from a Windows Forms interface.

Add the windows form to your project

This is the first and also the easiest step of the process. Rightclick your project and select ‘Add…->New Item’ and select ‘UI->Windows Form’ in the Visual C++ dialog.

You will then get a message box, telling you that your project will be converted to a managed project. Here you click ‘yes’. Even though you don’t want to compile your project with /clr, Visual Studio itself needs to know that you are doing managed stuff in your project. This is not necessary for compiling your project, but if you want to have the benefit of working with the forms designer and other .NET related things, then it would be a good idea. And you cannot add the form if you choose ‘No’ so it’s not like you have much choice anyway.

Now you have a .NET Windows Form in your code. You can verify the file specific settings if you want, but the cpp file will be compiled with the /clr flag set, and without the use of precompiled headers. Not using precompiled headers is important, because the default precompiled header is compiled without /clr set, and this would lead to conflicts.

Of course, if you need to, you can create a copy of StdAfx.h/cpp cpp files called ‘StdAfxClr.h/cpp’ add them to your project, and configure them to create a managed precompiled header, which you could then use for all files that are compiled with /clr. This is not necessary for small projects, but it could be a useful optimization in large projects.

The interface layer

At this point you can compile and link the entire project, but your native code isn’t yet using the new managed functionality. And because the native code will not be compiled with /clr, it will never do so. In order to make that happen, you will need a thin layer between the native and managed code. This layer will have a native C or C++ interface which can be called by the native code, and a managed implementation that will do all the .NET stuff.

In our example, the interface is 1 simple C style function in Interface.h:

int __stdcall DoManagedStuff(void);

And this function has a simple implementation in Interface.cpp:

int __stdcall DoManagedStuff(void)
{
      DemoForm ^df = gcnew DemoForm();
      df->ShowDialog();
      return df->Value;
}

Interface.h and Interface.cpp are 2 new files that you have to add to your project. You have to manually configure the cpp file to be compiled with /clr, and to not use precompiled headers.

As you can see, the interface layer provides a clean native interface for the managed stuff that you want your code to perform. Of course, you are not limited to C style interfaces. You can also work with classes and make classes with a native interface and a managed implementation. But there are a couple of issues that you need to be aware of. If you want to go there, then it would be a good idea to read the paper written by Herb Sutter which explains the rationale behind C++/CLI, as well as some of the limitations and pitfalls.

Actually, reading that paper is a good idea for anyone working with C++/CLI who also cares about understanding what is actually going on behind the scenes.

Using the interface layer

All the hard work is done. Now you can simply include Interface.h in your native code files, and call ‘DoManagedStuff’ where appropriate.

#include "stdafx.h"
#include "InterfaceLayer.h"
int __stdcall DoFoo(void)
{
return DoManagedStuff();
}

Conclusion

As you can see, extending native code with managed code is not so hard. At least, it isn’t if you maintain a clean break between managed and native code. If you cannot do this, things might become more difficult. For example, using .NET controls on an MFC dialog, or using .NET remoting in a native COM project are things that can be much more tricky.

Another thing you should be aware of is that not all compiler switches can be used in conjunction with the /clr switch. The compiler will inform you if it detects this. The combinations which are not allowed are documented, and can be found in the documentation of the /clr switch itself.

This post was not meant to be an exhaustive how-to to those complex scenarios, but instead an explanation behind the basic ideas.

The demo project for this article can be downloaded under the MIT license as usual.

Excel and OleDb stupidity

vanDooren — Fri, 29 Feb 2008 20:47:00 GMT

This afternoon I was working on a Windows Forms app for the finance guys, to help them allocate costs to systems, proportional to predefined allocation keys.

The app has to import Excel spreadsheet files and perform all the database actions.

The key columns to uniquely identify a cost are text values. Unfortunately, some of those key values do not contain letters, but only numeric characters. And this is where it sucks to have to work with Excel.

This is because Excel knows no datatypes. You can specify formatting for cells, but Excel ignores that when someone asks for cell data. Instead, it scans the first couple of rows to guess the datatype. By default it checks the first 8 rows. If those contains mixed values, it uses a specified default type for those values that are not numeric.

This means that if you use OleDb to retrieve values, the column type can vary per row.

What can you do about it? Well nothing really. You can tweak some values, and hope that you will never encounter the corner cases that you cannot do anything about. There is plenty of information to be found. One good explanation was this.

However, one of the finance guys gave me an incredibly useful tip, which you can use as a workaround. You have to manually make a couple of easy modifications to the excel file, and Excel will always use the text data type.

Assume that the offending data is in column 1, and that the first row is the header row.
You insert 2 empty columns to the right of the offending column.
Put an ' in the first cell of the first empty column (B2).
Then enter =B2&B1in the cell B3.
Select B2 and B3, and double-click the bottom right corner of the selection box. This will copy the data from there to the last row.
B3 will now hold the data as text, because the ' tells Excel that no matter what is in the cell, it is text.
Then you copy column 3, paste it as 'values only' over column 1.
Delete columns 2 and 3 again.

Now you can read the column in OleDb and always get text values for that column.

Executing LabVIEW VIs through C style function pointers via .NET

vanDooren — Thu, 16 Nov 2006 11:46:00 GMT

Problems exist to be solved.

Some problems are so complex that you cannot solve them when you need to. With other problems it is not a matter of complexity, but a matter of not having all the pieces of the puzzle.

Some problems are so interesting that it pays to keep them in the back of your head, just in case you ever find that missing piece that allows you to solve them.

One of those problems is that with the current LabVIEW external library interface it is impossible to execute LabVIEW VIs in a DLL through a C style function pointer.

This feature does not exist, even though lots of programmers would really benefit from it.

2 weeks ago I was active on the Visual C++ newsgroup as usual, when learned –by pure coincidence- about an advanced function in the .NET framework called Marshal.GetFunctionPointerForDelegate.

I immediately realized that this was the key to solving the old C style function pointer issue

Problem description

Suppose that we have a DLL that exports a function. This function takes a function pointer as an argument, to be executed sometime later to pass some data to the calling application.

The Test DLL code

The idea is to test the ability to run a LabVIEW VI through a function pointer. To test this we need a DLL which will test this for us.

The following very simple test function has been created for this purpose.

This is the expected prototype of the callback function

typedef void (__stdcall *fSimpleCallBack)(int i);

And this is the DLL function that will perform our test:

//Test of the simple callback

void Simplefoo(int i, fSimpleCallBack cb)

{

(cb)(i); //invoke the callback function.

}

As you can see, the LabVIEW VI will be executed with the supplied parameter through the supplied function pointer.

Principal solution

The very first action has to be creating a .NET delegate of which the signature matches the signature of the CallBack function that is required.

Then a proxy class is created with only 2 members: an event that is based on the new delegate function, and a method called ‘GetDelegate’.

LabVIEW has the ability to link a LabVIEW VI with a .NET event. The VI prototype is generated automatically, and matches the event delegate signature.

After registering a VI for the specified event, LabVIEW uses the GetDelegate method of our proxy class to extract the registered delegate from the event. That delegate is then converted to a function pointer via the ‘GetFunctionPointerForDelegate’ method of the Marshal class.

That function pointer can then be supplied to a native function that can execute the LabVIEW VI directly through the function pointer without requiring any modification at all.

A note on GetFunctionPointerForDelegate

The documentation for this method is sparse. It seems that VS2005 and .NET2.0 were rushed out of the door without taking the time to fully document the more advanced functions and providing code samples in the documentation.

Because of this, it is easy to miss that GetFunctionPointerForDelegate returns a pointer that uses the __stdcall calling convention by default.

It is also easy to overlook the fact that you can change this.

If you need a function pointer that uses another calling convention - e.g. __cdecl – then you can achieve this by applying the UnmanagedFunctionPointerAttribute attribute to the delegate definition.

The .NET code

namespace FunctionPointerProxy

{

public class SimpleProxy

{

//Marshal a simple integer parameter as 32 bit integer.

public delegate void SimpleProxyDelegate(

[MarshalAs(UnmanagedType.I4)] int i);

//export an event because that is the only way we are going to get

//LabVIEW to give use a delegate.

public event SimpleProxyDelegate SimpleEvent;

//extract the delegate from the event. If there was no event

//registered, this will trigger an exception which is caught by the

//LabVIEW .NET interface node. No need to worry about it here.

public Delegate GetDelegate()

{

return SimpleEvent.GetInvocationList()[0];

}

This is all the additional code you really need to make it work. The delegate function has to mirror the native callback prototype for 100%. To achieve this you have to use the MarshalAs attribute on each delegate parameter to insure correct behavior.

The event simply has to exist because LabVIEW has only 1 way to produce a delegate for a VI, and that is by registering a VI to an event.

And even then, the delegate is never directly accessible, so we have to create a ‘GetDelegate’ method that extracts the registered delegate, and returns it to LabVIEW.

As far as the .NET framework is concerned, we could simply make the event and the GetDelegate method static to remove the need to instantiate an object that we won’t need for anything else. But there are 2 reasons why we don’t do that:

LabVIEW does not support registering VIs to static events.
Our GetDelegate function would need way to know which delegate you want to get, in case there are multiple instances of callback VIs. Since the event is tied to an instance, it is easy enough to keep track of the events in LabVIEW.

The LabVIEW code

LabVIEW code is graphical, so I have chosen to mark the different steps in the block diagram below, and then add an explanation of each step in a numbered list.

Instantiate a new SimpleProxy Class. We only need it to access the event. Sadly, LabVIEW does not yet support static events.
Register a VI for the Event. The way to do this is to wire the SimpleProxy refnum to the event source and select the appropriate event. Right-click 'VI Ref' and select 'Create Callback VI'. This will automatically generate a VI with the correct input and output parameters for you. This will also automatically wire a static VI reference to the VI Ref input. User parameter can be any LabVIEW type at all that you want to pass to the VI when the .NET event is triggered. We don't need it, but if we did we would have to wire it before we generate the callback VI because it changes the VI prototype.
Now that the event is registered, we can simply get the .NET delegate that represents the VI callback out of it.
Use a static member of the Marshal class to get a funtion pointer to the delegate. This is where the magic happens. A native stub is generated for the delegate at runtime. The unmanaged parameter list is built according to the Marshalling attributes that were attached to the delegate signature.
Cast the Delegate IntPtr to an Int32. Pointers are 32 bit on the x386 platform. We need this 32 bit value to supply it to the DLL function call that expects a 32 bit function pointer. Note that this function pointer uses the standard calling convention by default (see below for more info).
We no longer need the FunctionPointer IntPtr. As long as the delegate itself is still in memory, the native stub will be valid.
Execute the DLL function. This function will use the supplied callback function pointer to execute the callback VI. It will pass the supplied int parameter to the callback vi at the time of the function call.
Unregister the Callback VI from the event, and close the various references that were opened.

Running the test

As is sometimes said: the proof is in the pudding.

To verify that everything works as advertised, I have put a simple dialog box in the callback VI that displays the value of the integer parameter.

Simply run the example and you will see the following dialog box:

Advanced implementation

The principal solution works, but is not yet ideal. Most importantly, there is too much clutter on the LabVIEW diagram for this solution to be aesthetically pleasing.

There are too many manual actions that have to be repeated each time you implement a callback function. I also wanted to show you an example of a callback function that is a bit more complex.

Test DLL

Consider the following callback prototype that is a bit more complex than the previous one:

typedef void (__stdcall *fAdvancedCallBack)(

int i,

char* aString,

wchar_t* wString);

Here we need to supply an integer, an ASCII string and a UNICODE string. The function exported from the DLL has the following implementation:

void Advancedfoo(int i, char* aString, fAdvancedCallBack cb)

{

BSTR uString;

int aLength = lstrlenA(aString);

int uLength = ::MultiByteToWideChar(

CP_ACP, 0, aString, aLength, 0, 0);

if(0 < uLength)

{

uString = SysAllocStringByteLen(0, uLength);

::MultiByteToWideChar(

CP_ACP, 0, aString, aLength, uString, uLength);

}

else

uString = L"";

//invoke the callback function through its function pointer.

(cb)(i, aString, uString);

}

LabVIEW itself does not support UNICODE strings, so we manually transform the supplied ASCII string to UNICODE. Apart from that the function body only contains a function call via the supplied callback pointer.

.NET code

[UnmanagedFunctionPointerAttribute(CallingConvention.Cdecl)]

public class AdvancedProxy : IProxy

{

public delegate int AdvancedProxyDelegate(

//A simple I4 parameter. Nothing special

[MarshalAs(UnmanagedType.I4)] int i,

//An ASCII string. this will be marshalled as UNICODE on the

//.NET side and ASCII on the unmanaged side.

[MarshalAs(UnmanagedType.LPStr)] string aString,

//A UNICODE string. This will be marshalled as UNICODE on the

//.NET side, and UNICODE on the unmanaged side.

[MarshalAs(UnmanagedType.LPWStr)] string wString);

public event AdvancedProxyDelegate AdvancedEvent;

public Delegate GetDelegate()

{

return AdvancedEvent.GetInvocationList()[0];

}

The meaning of the different parts is the same as with the SimpleProxy example. The only real difference – apart from the delegate signature of course – is that all classes that are used as a proxy between a LabVIEW VI and a function pointer should implement the interface IProxy.

This interface has the following definition:

public interface IProxy

{

Delegate GetDelegate();

}

This allows us to make the LabVIEW code for getting a function pointer generic, instead of tied to the type of the callback delegate.

LabVIEW code

As you can see, the LabVIEW code is vastly simplified.

The only thing that we cannot make generic is the event registration, because the event and as a result the VI reference are strongly typed.
The process of retrieving the function pointer is generic because all proxy classes now implement the IProxy class for retrieving the event delegate.
The call to the test function in the DLL now also takes a string parameter that gets expanded to UNICODE inside the test function to supply the callback function with the correct parameters.
Cleanup of the event and references is also generic.

There are 2 new VIs. These are generic, so you can reuse them without limitation. This also makes it easy to support an asynchronous callback mechanism. You can register the event and supply the function pointer to the DLL.

The DLL stores it inside, and uses it any time it needs to. Meanwhile, your LabVIEW program can continue processing. You can keep the references cluster around – for example in a buffer – until such time as you want to disable the callback mechanism again.

Running the demo code

The Visual studio solution is for VS2005, and can be built with all versions of VS2005.

The LabVIEW project uses version 8.2. You need at least version 8.0 to have decent .NET support, and I used 8.2 because that is the latest version at this moment.

To run the demo, simply extract the sources and open and build the Visual studio solution. Then open the LV project and run the sample of your liking.

Conclusion

This article has outlined a relatively simple way to directly execute a LabVIEW VI through a C style function pointer without having to write C or C++ code. The demo code for this article is available under the MIT license as usual.

You still need to provide a .NET class library, but the code involved is so minimal and trivial that it should not stop anyone.

This approach also makes it possible to use the wealth of win32 API functions that require a callback function without having to resort to writing C or C++ code.

In the time it took me to write this article (which took longer than writing the code, as usual) I found 3 other ways to achieve the same result.

One is an ugly kludge that only fakes the callback functionality. The other performs a true callback to a LabVIEW VI, but it is even uglier. The third one is fairly elegant but has some other limiting factors.

I will explain those 3 lesser solutions in another article. Despite the fact that only one of them is useful; the principle behind all 3 of them might be of use in another situation.

Finally, the whole LabVIEW and function pointer issue has existed for a long time among LabVIEW programmers. I have done a bit of research, and as far as I can tell, I am the first to ever solve this problem without requiring C or C++ code.

So here and now I claim to be the first to implement and publicize a working C style function pointer callback mechanism in LabVIEW!

In my next article I will follow up with an example involving structures (Clusters in LabVIEW) and n dimensional arrays.

Fun with template classes: Manipulating LabVIEW arrays in C++, part 3

vanDooren — Thu, 10 Aug 2006 20:45:00 GMT

This article is a follow up of my second article on interoperability for LabVIEW arrays in C++. That article provided a method for allowing programmers to index into multi dimensional arrays like this:

But that solution was a bit clunky. It involved 4 classes which had 5 [] operators and 2 implementations of the Resize function. There was also no support for memory access via memcpy or something else.

After a bit of redesign, I now have an improved implementation.

The general idea

Functionality is now split in 2 classes. There is a CLvArrayHandlePtr class that represents the array itself, and a CLvArrayIndexer class to provide the [] operator.

You can no longer index in the array object directly. The CLvArrayHandlePtr class is now meant solely for managing the array itself. This makes the array simpler.

The indexing functionality has been put in the CLvArrayIndexer class. Since all that class does is to provide a [] operator and an accessor for the data pointer, it is a very lightweight class that is easy to maintain.

This solution also makes it easier to add extra functionality later on as separate lightweight classes that are tailored to a specific purpose, instead of having to perform all modifications in a class that includes everything and the kitchen sink.

The Revised Array Class

The CLvArrayHandlePtr class is still a template class with a specialization for 1 dimensional arrays. The general principles behind the array has not been changed, so I am not going to reiterate all explanations over here. Instead, I will only explain the changes.

template <class T, int Q=1> class CLvArrayHandlePtr

{

public:

#pragma pack(push)

#pragma pack(1)

typedef struct

{

size dimSize[Q];

T data[1];

} t_LvArray, *p_LvArray, **h_LvArray;

#pragma pack(pop)

h_LvArray* m_HandlePtr;

int Resize(size DimSize[ Q]){ … }

size_t GetSize(void){ … }

void GetDimSizes(size_t DimSizes[ Q]){ … }

T* GetDataPtr(void){ … }

};

template <class T> class CLvArrayHandlePtr<T, 1>

{

public:

#pragma pack(push)

#pragma pack(1)

typedef struct

{

size_t dimSize;

T data[1];

} t_LvArray, *p_LvArray, **h_LvArray;

#pragma pack(pop)

h_LvArray* m_HandlePtr;

int Resize(size_t DimSize){ … }

size_t GetSize(void){ … }

void GetDimSizes(size_t &DimSize){ … }

T* GetDataPtr(void){ … }

};

These classes have roughly the same methods as their previous versions. The most notable differences are:

They now use size_t for dimension sizes.
There is a function for returning the pointer to the first element in the array.
There is a function to retrieve the dimension sizes.

The last 2 points are necessary in order to cleanly work with the 2 new classes.

The array indexer class

The 2 new classes are used solely to allow indexing operations on the LabVIEW array. They have no other added functionality, so they are fairly simple.

template <class T, int Q=1> class CLvArrayIndexer

{

private:

size_t m_TotalSize;

size_t m_DimSizes[ Q];

T *m_Data;

public:

//create a new instance, based on an actual array.

CLvArrayIndexer(CLvArrayHandlePtr<T, Q> LvArray)

{

m_TotalSize = LvArray.GetSize();

LvArray.GetDimSizes(m_DimSizes);

m_Data = LvArray.GetDataPtr();

}

//create a new instance, based on a data pointer and dimension sizes

CLvArrayIndexer(size_t DimSizes[ Q], T* DataPtr)

{

for(int i=0; i<Q; i++)

m_DimSizes[ i] = DimSizes[ i];

m_Data = DataPtr;

}

//return a new indexer with 1 dimension less.

CLvArrayIndexer<T, Q-1> operator[](size_t Index)

{

size_t subDimSize = 1;

for(int i=1; i<Q; i++)

subDimSize *= m_DimSizes[ i];

return CLvArrayIndexer<T, Q-1>(

m_DimSizes + 1,

m_Data + Index * subDimSize);

}

//Get the size of the data segment that represents this subdimension

size_t GetSize(void)

{

return m_TotalSize;

}

//Get the pointer to the first element of this subdimension

T* GetDataPtr(void)

{

return m_Data;

}

};

template <class T> class CLvArrayIndexer<T, 1>

{

private:

size_t m_TotalSize;

T *m_Data;

public:

//create a new instance, based on an actual array.

CLvArrayIndexer(CLvArrayHandlePtr<T, 1> LvArray)

{

m_TotalSize = LvArray.GetSize();

m_Data = LvArray.GetDataPtr();

}

//create a new instance, based on a data pointer and dimension sizes

CLvArrayIndexer(size_t DimSizes[1], T* DataPtr)

{

m_TotalSize = DimSizes[1];

m_Data = DataPtr;

}

//return a reference to the element at the specified index.

T& operator[](size_t Index)

{

return m_Data[Index];

}

//Get the size of the data segment that represents this subdimension

size_t GetSize(void)

{

return m_TotalSize;

}

//Get the pointer to the first element of this subdimension

T* GetDataPtr(void)

{

return m_Data;

}

};

As you can see, there are 2 constructors for creating an indexer. The first takes the original array as an argument. It uses the array methods for getting the data size and data pointers.

The second constructor takes the size and pointer directly. This is used byt CLvArrayIndexer classes that create another CLvArrayIndexer class with a lower dimension.

The [] operator itself is also pretty simple. For multi dimensional indexers, the [] operator returns a new indexer that has 1 less dimension than itself. For 1 dimensional array indexers, it returns a reference to an element inside the original array.

The trick here is that each next [] operator points to a more specific location inside the original array until 1 specific element is referenced.

Using the array and indexer classes

To use the array and indexer classes in not very different from the previous solution.

Given an array with the following declaration:

CLvArrayHandlePtr<double, 3> Array;

Using it is done like this:

size_t size[3] = {

requestCountPages,

requestCountRows,

requestCountColumns};

Array.Resize(size);

CLvArrayIndexer<double, 3> indexer(Array);

double i=0;

for(int page=0; page<requestCountPages; page++)

for(int row=0; row<requestCountRows; row++)

for(int column=0; column<requestCountColumns; column++)

indexer[page][row][column] = i++;

Using the data pointer and size of a sub segment are of course as easy as

Memcpy(Target, indexer.GetDataPtr(), indexer.GetSize());

Afterthoughts

This solution is clean, and easy to use and maintain. I am fairly happy with it.

Of course, what I am not yet happy about is the lack of error handling. For example, the indexer class does not yet check if the array is actually allocated or not. If it isn’t, then the current solution will dereference a NULL pointer.

There is also no size checking to prevent indexers from going out of bounds.

But as before, this code is proof of concept code. I am toying with the idea of implementing those error handlers, but I want to do it in a way that allows you to use either C style return value error handling or C++ style exception handling.

One possibility is to use error handling template classes. However, That will have to wait for next time.

The full code for the classes is available for download, together with the dll code and some LabVIEW vis for testing. Everything is licensed under the MIT license as usual.

Fun with template classes: Manipulating LabVIEW arrays in C++, part 2

vanDooren — Fri, 19 May 2006 08:49:00 GMT

This article is a follow-up on a previous article: 'Manipulating LabVIEW arrays in C++' in which I’ve explained how you could use C++ template classes to work with LabVIEW arrays. There was one annoying issue left to solve: you couldn’t use the standard syntax for indexing in a multidimensional array.

I.e. the following code would not work:

for(int row=0; row<requestCountRows; row++)

for(int column=0; column<requestCountColumns; column++)

Array[row][column] = i++;

After some thinking I was able to redesign the CArrayHandlePtr class to be able to be used like that.

The general idea

The idea behind my solution is to let the [x] operator for an n dimensional array return a new instance of the CArrayHandlePtr class that represents a class with n-1 dimensions of which the first element points to the first element of the subset pointed to by the [x] operator.

In that case, the operation Array[row][column] = 5 is equivalent to ((Array[row]) [column] = 5;

Array[row] returns a temporary 1D array temp. temp is then indexed like this: temp[column]. Since temp is a 1D array, it returns a reference to a specific element.

This approach is scalable to as many dimensions as you want, because a multidimensional index statement is simply broken down into n -1 temporary elements, each of which has a number of dimensional which has 1 dimension les than the previous one.

The implementation problem

The problem with this approach is that LabVIEW arrays do no lend themselves directly to it. The data block is supposed to begin directly after the dimension sizes. To return an array class of a lesser number of dimensions would mean that the selected subset would have to be preceded by the dimension sizes. This is clearly not an option.

To solve this problem we need array helper classes.

The helper classes

The point of a helper class is to be able to target a subset of a multidimensional array. To do this they should have a independent dimension size specifiers and data pointer.

Apart from that, the helper class has 1 simple requirement: the [] operator has to return a new helper of 1 dimension less than itself. If its number of dimensions is 1, it has to return a reference to the specified element.

The helper classes look like this:

template <class T, int Q=1> class CArrayHelper

{

private:

int32* m_Sizes;

T* m_Data;

public:

CArrayHelper(T* Data, int32 *Sizes)

{

m_Data = Data;

m_Sizes = Sizes;

}

CArrayHelper<T, Q-1> operator[](int Index)

{

int subsetSize = 1;

/*calculate the size of a complete subset*/

for(int i=1; i<Q; i++)

subsetSize *= m_Sizes[ i];

/*return an array helper that represents a subset of the original array*/

return CArrayHelper<T, Q-1>(m_Data + subsetSize * Index, m_Sizes +1);

}

};

template <class T> class CArrayHelper<T, 1>

{

private:

int32* m_Sizes;

T* m_Data;

public:

CArrayHelper(T* Data, int32 *Sizes)

{

m_Data = Data;

m_Sizes = Sizes;

}

T& operator[](int Index)

{

return m_Data[Index];

}

};

As you can see, these classes are very simple. Their only purpose is to be able to provide a way to specify a subset of a multidimensional array. That way the [] operators can be concatenated to read 1 specific element.

The actual array classes

There are 2 array classes: one general one and one that is specialized for 1 dimensional arrays. The reason that the specialization is needed is that the [] operator for a multidimensional array should return an array helper class of 1 dimension less, while the [] operator for a 1 dimensional array should return a reference to an element in the array.

template <class T, int Q=1> class CArrayHandlePtr

{

public:

/*type definition for LabVIEW array handle*/

#pragma pack(push)

#pragma pack(1)

typedef struct

{

int dimSize[Q];

T data[1];

} t_LvArray, *p_LvArray, **h_LvArray;

#pragma pack(pop)

h_LvArray* m_HandlePtr;

/*resize the array*/

int Resize(int DimSize[Q])

{

/*implementation omitted*/

}

T& operator[](int Index[Q])

{

/*implementation omitted*/

}

CArrayHelper<T, Q-1> operator[](int Index)

{

int subsetSize = 1;

/*calculate the size of a complete subset*/

for(int i=1; i<Q; i++)

subsetSize *= (**m_HandlePtr)->dimSize[ i];

/*return an array helper that represents a subset of the original array*/

return CArrayHelper<T, Q-1>

((T*)(**m_HandlePtr)->data + subsetSize * Index,

(int32*)(**m_HandlePtr)->dimSize +1);

}

};

template <class T> class CArrayHandlePtr<T, 1>

{

public:

/*type definition for LabVIEW array handle*/

typedef struct

{

int dimSize;

T data[1];

} t_LvArray, *p_LvArray, **h_LvArray;

h_LvArray* m_HandlePtr;

/*resize the array*/

void Resize(int DimSize)

{

/*implementation omitted*/

}

T& operator[](int Index)

{

return (**m_HandlePtr)->data[Index];

}

};

As you can see, the generic array class allow you to use either a complete element specification via the [](int[Q]) operator, or a partial specification via the [](int) operator. The partial specification returns an array helper. One way or another, you will reach a single array element.

Using the arrays

You now have both syntax options for addressing elements inside the array. You can do it like this:

for(int page=0; page<requestCountPages; page++)

for(int row=0; row<requestCountRows; row++)

for(int column=0; column<requestCountColumns; column++)

Array[page][row][column] = i++;

Or like this:

int index[3];

for(int page=0; page<requestCountPages; page++)

{

index[0] = page;

for(int row=0; row<requestCountRows; row++)

{

index[1] = row;

for(int column=0; column<requestCountColumns; column++)

{

index[2] = column;

Array[index] = i++;

}

Both have the same effect.

Afterthoughts

This solution is easier to use than my first LabVIEW array implementation, because you now have both syntax options for indexing in the array.

On top of that, I now know more about partial template specialization than before. It has been a lot of fun already. For my definition of fun at least.

I am still not satisfied with this solution though. We are now using a total of 4 classes, which have between them 5 [] operators and 2 implementations of the Resize function. Any method we want to add to the array class has to be implemented in both the generic and the specific class. There is also no compile time option to enable range checking. Nor has any attention been given to indexing performance optimization.

I think I will write another follow-up article to at least solve the multiple implementations problem.

As always, the complete dll and LabVIEW code can be downloaded used under the MIT license. Enjoy. [;-)]

Fun with template classes: Manipulating LabVIEW arrays in C++, part 1

vanDooren — Tue, 16 May 2006 08:20:00 GMT

One of the best things about LabVIEW is its ability to call external code. This code can be used in the form of a dll, a CIN (custom compiled C code) an ActiveX object or a .NET class.

I have often created dlls (on windows) or shared object libraries (linux) to create an interface between LabVIEW and external native code.

This blog entry is about working with LabVIEW arrays in external code. This is simple as long as you create an interface that takes a pointer to the array data.

For data input this poses no problem. For data output this is also simple as long as you follow the C style: supply a valid data pointer, a buffer size and a pointer to a variable that will receive the number of elements written.

Of course, if you don’t know the number of elements in advance, your code becomes more complex, and will likely result in more external function calls and over-allocation.

To overcome this problem we need to dynamically allocate LabVIEW arrays in our external code that can be passed to LabVIEW. This is much more complex than working with a simple data pointer.

What are LabVIEW arrays?

A LabVIEW array is in fact a structure that contains a length specifier and the actual data elements. The C declaration is like this:

typedef struct

{

int dimSize;

int data[1];

} t_LvArray, *p_LvArray, **h_LvArray;

The array contains a dimension size specifier that indicates the size of the data part, and the data part itself.

The data part declaration is specified as an array with size 1. the reason for this is that the size specifier and the data are part of the same block of data, so data cannot be a pointer.

In order to be able to dereference data, it is declared as an array. The actual size is not 1, but whatever is mentioned in dimSize. It is only declared as an array of 1 byte because 0 sized arrays cannot exist in the C language.

Whenever LabVIEW has to pass a LabVIEW style array to external code, it does this by supplying a relocate able handle.

In case of multidimensional arrays the declaration is somewhat different:

typedef struct

{

int dimSize[NUM_DIMENSIONS];

int data[1];

} t_LvArray, *p_LvArray, **h_LvArray;

Each dimension has an element in the dimSize array to indicate the size of that dimension. For a 3 dimensional array, this array has 3 elements.

The data block is still declared as a 1D array.

It is also worth to note that LabVIEW structures and arrays are packed with 1-byte alignment. I.e. there is no padding between structure fields to align variables on address boundaries. This means that declarations should be done like this to insure proper packing:

#pragma pack(push)

#pragma pack(1)

typedef struct

{

int dimSize[Q];

T data[1];

} t_LvArray, *p_LvArray, **h_LvArray;

#pragma pack(pop)

The syntax will be different for different compilers. This syntax is used with Microsoft Visual C++.

What is the problem with LabVIEW arrays?

On first glance the handles seem easy to use, but there are 2 distinct problems when you want to use them.

The first is simple you need a unique declaration for each data type and each number of dimensions for which you want to create an array.

typedef struct

{

int dimSize

int data[1];

} t_LvArrayInt1D, *p_LvArrayInt1D, **h_LvArrayInt1D;

typedef struct

{

int dimSize;

double data[1];

} t_LvArrayDbl1D, *p_LvArrayDbl1D, **h_LvArrayDbl1D;

typedef struct

{

int dimSize[2];

double data[1];

} t_LvArrayDbl2D, *p_LvArrayDbl2D, **h_LvArrayDbl2D;

This becomes tedious after a while, and it is cumbersome to maintain a whole header file of declarations for all possible types.

The second problem is that you have to manually do all the arithmetic for calculating the position in the data block. You have to have different calculations for each unique combination of data type and number of dimension size. This is very prone to errors, and even more tedious to maintain than the array declarations themselves. The standard indexing notation for multi dimensional array cannot be used because the array size can change at runtime.

The third problem is that it is impossible to represent a zero length array. Zero length C style arrays do not exist. Of course the LabVIEW memory manager programmers could have implemented a hack for this, but they didn’t.

This means that if you want to resize arrays for output, you need to create an interface that takes a pointer to a handle. This way you can allocate a new array if the handle is NULL, or you can resize it if it isn’t. As a result, you are now working with a pointer to a pointer to a pointer to a data block. That is a lot of dereferencing to take care of.

The fourth problem is that the code for resizing and allocating, like the code for indexing in the arrays is unique per data type and per dimension size.

C++ template classes to the rescue

Luckily for us, there is a way around all these issues: C++ template classes.

C++ template classes are the most complex but also most powerful feature of the C++ language. It allows you to define templates for classes and functions that are only specialized for specific types at compile time.

LabVIEW arrays of any type and number of dimensions can be implemented using the following class:

template <class T, int Q=1> class CArrayHandlePtr

{

public:

/*type definition for LabVIEW array handle*/

#pragma pack(push)

#pragma pack(1)

typedef struct

{

int dimSize[Q];

T data[1];

} t_LvArray, *p_LvArray, **h_LvArray;

#pragma pack(pop)

h_LvArray* m_HandlePtr;

/*resize the array*/

void Resize(int DimSize[Q])

{

/*implementation omitted*/

}

T& operator[](int Index[Q])

{

/*implementation omitted*/

}

};

T and Q have to be supplied at compile time. T is the data type of the elements in the array, and Q is the number of dimensions. The compiler will create a correct implementation of t_LvArray that represents the LabVIEW style array.

The class has 2 other functions: a resize functions that allows you to resize the array (or to allocate it if the array handle was NULL), and a [] operator that returns a reference to the element specified in the Index array.

The constructor trick

The template class does not have a constructor. The reason for this is that there is no point in using true LabVIEW arrays except as function arguments to external code.

This means that the array handle pointer will be initialized by LabVIEW. The class has only 1 member variable: the handle pointer itself. If that is supplied by LabVIEW, we change its meaning from a simple address value to a templacized class instance.

This might seem a bit dodgy, but there is no constructor code that needs to be run. If the address member is initialized, the object is considered to be properly constructed. This is entirely legal C++ code.

The only downside is that we have to be sure that the definition of the data type in LabVIEW has to match what we think it to be. There is no way for the external code to know if the array is 2D or 3D for example. It is the responsibility of the programmer to insure this.

Of course, if you decide you want to use true LabVIEW arrays within your C++ code for internal purposes, you can add a constructor that will allocate memory through the LabVIEW memory manager. Just be sure to dispose of it afterwards, and to provide proper copy constructors to prevent heap errors.

Resizing the array

The implementation for the resize function is this:

int Resize(int DimSize[Q])

{

int totalSize = 1;

int errorCode = 0;

/*calculate number of elements*/

for(int i=0; i<Q; i++)

totalSize *= DimSize[ i];

/*convert to number of bytes*/

totalSize *= sizeof(T);

/*add size of the dimension array*/

totalSize += Q * sizeof(int32);

if(0 >= totalSize)

return mgArgErr;

if(NULL != *m_HandlePtr)

{

errorCode = DSSetHSzClr(*m_HandlePtr, totalSize);

if(noErr != errorCode)

return errorCode;

}

else

{

*m_HandlePtr = (h_LvArray)DSNewHClr(totalSize);

if(NULL == *m_HandlePtr)

return mZoneErr;

}

for(int i=0; i<Q; i++)

(**m_HandlePtr)->dimSize[ i] = DimSize[ i];

return noErr;

}

TotalSize is the size in bytes of the complete memory block, including the dimension size specifiers. Some extra checking is done to insure that the total size is a positive number.

Then the handle is resized if it was a valid handle, or a new handle is allocated if it was a NULL handle. Finally, the dimension sizes have to be filled in manually because the memory manager does not know this information.

Indexing in the array

The final functionality that has to be implemented is the [] operator that will allow the programmer to read and write values to and from a position in the array.

T& operator[](int Index[Q])

{

int index = 0;//sizeof(t_LvArray);

for(int i=0; i<Q-1; i++)

{

int subSize = 1;

for(int j=1; j<Q; j++)

subSize *= Index[ i] * (**m_HandlePtr)->dimSize[ j];

index += subSize;

}

index += Index[Q-1];

return (**m_HandlePtr)->data[index];

}

};

Because the real data block is just a flat 1D array, we have to calculate the position in the array that matches with the specified indices. This calculation is not so hard to understand:

With each dimension that is accounted for, the index in that dimension is multiplied with the total number of elements in a sub-dimension. This value (the subSize value) is calculated for each dimension except for the last dimension (the least significant dimension). That index is just added to the intermediate total because it has no number of elements per sub dimension.

The resulting index is the position in the 1D data block that represents the element at the specified multidimensional indices.

Using the array

Using the array is very simple, as can be seen in this example:

int fun2Ddouble(

int requestCountRows,

int requestCountColumns,

CArrayHandlePtr<double, 2> Array)

{

int index[2] = {0,0};

int size[2] = {requestCountRows, requestCountColumns};

Array.Resize(size);

double i=0;

for(int row=0; row<requestCountRows; row++)

for(int column=0; column<requestCountColumns; column++)

{

index[0] = row;

index[1] = column;

Array[index] = i++;

}

return 0;

}

Limitations of this design

Using the magic of C++ templates, we are able to implement 1 class that can represent all LabVIEW arrays of any given size or data type.

That does not mean that we are totally satisfied however. The follow

It is regrettable that we have to use arrays as index to the [] operator. It would be nicer to be able to use true multidimensional array syntax. i.e:

for(int row=0; row<requestCountRows; row++)

for(int column=0; column<requestCountColumns; column++)

Array[row][column] = i++;

See the next chapter for some more info on that.

Another ‘problem’ if you wish, is that the [] operator does not protect against buffer overflow. This is trivial to implement, but I have not done so for performance reasons. And after all, true C style arrays do not protect against it either. It would perhaps be a good idea to implement this using conditional compilation, enabling the programmer to turn this feature on or off at compile time. This is just proof of concept code however, so I am not going to do that for now.

A performance problem is also that the sub dimension sizes are recalculated for all dimensions, each time the [] operator is invoked. It would make more sense to store these values somewhere, but we cannot do that. The requirement that an instance of the template class is 100 % identical with the LabVIEW definition means that we are not allowed to store additional data in the class body.

Afterthoughts

This article explains how you can use C++ template classes to create 1 class that you can use to access LabVIEW arrays of any data type and number of dimension. Bear in mind that the code is just proof of concept code.

The example LabVIEW and C++ code is available for download under the MIT license.

I have already found a solution of the multidimensional syntax problem. With some additional programming it is possible to use the [row][column] notation for 2D arrays, and [page][row][column] notation for 3D arrays.

The solution is not limited by the number of dimensions (you could use 100D arrays for all I care) but it is a bit too complex to include it as part of this article.

I will dedicate my next blog article to the solution for that problem.

Interop between C++ and C or LabVIEW

vanDooren — Thu, 11 May 2006 07:39:00 GMT

One of the questions that recur from time to time is: ‘How do I use C++ classes in a language that has only C bindings, like C itself or LabVIEW?'

The short answer: you can’t.

The long answer: you can’t, but you can wrap the C++ code in C functions, and export those in a wrapper dll, thus allowing you to manipulate C++ objects through a C interface.

The simple solution

The general way to do this is to export a function for creating objects that returns the object pointer to the caller. Another function will take that pointer to do something with it, and a third function will take the pointer to delete it.

You have to cast the pointer to something that can be recognized by the caller. You cannot export a class pointer, because C or LabVIEW have no idea what a class is. To be safe, you have to cast it to an INT_PTR to insure that it is always large enough to hold a pointer.

#define OBJPTR INT_PTR

The 3 essential functions look like this:

int simple_CreateObject(OBJPTR *ObjPtr)
{
*ObjPtr = (OBJPTR) new SomeClass();
if(NULL == *ObjPtr)
return 1;

return NO_ERROR;
}

int simple_UseObject(OBJPTR ObjPtr)
{
return ((SomeClass*)ObjPtr)->DoSomething();
}

int simple_DeleteObject(OBJPTR ObjPtr)
{
delete (SomeClass*)ObjPtr;
return NO_ERROR;
}

This approach is simple and it works. It has at least one serious problem though: the application will crash if it accidentally asks the wrapper to delete the same pointer twice, or to dereference an already deleted pointer.

Especially if you develop this wrapper for use by LabVIEW, I consider it bad taste if you do not shield the LabVIEW program against crashes caused by trivial mistakes.

A better solution

A better solution is to map all object pointers to a unique integer value. The calling program will only receive the integer values that it can supply to other functions to manipulate the C++ object.

The other wrapper functions have to find the pointer that is mapped to that integer value. If there is no mapping, it can simply return an error code instead of crashing the application.

When the object is deleted it is removed from the map so that the application cannot use it anymore.

The integer reference is incremented with each object that is created so that there will never be a duplicate reference.

map<BETTER_OBJ_REF, SomeClass*> g_PtrMap;

int g_Index = 0;

int better_CreateObject(BETTER_OBJ_REF *ObjRef)
{
SomeClass* ptr = NULL;

ptr = new SomeClass();

if(NULL == ptr)
return 1;

if(NO_ERROR == errorCode)
{
g_PtrMap[++g_Index] = ptr;
*ObjRef = g_Index;
}

return NO_ERROR;
}

int better_UseObject(BETTER_OBJ_REF ObjRef)
{
map<BETTER_OBJ_REF, SomeClass*>::iterator iter;
iter = g_PtrMap.find(ObjRef);

if(iter != g_PtrMap.end())
return (iter->second)->DoSomething();

else
return 1;
}

int better_DeleteObject(BETTER_OBJ_REF ObjRef)
{
map<BETTER_OBJ_REF, SomeClass*>::iterator iter;
iter = g_PtrMap.find(ObjRef);

if(iter != g_PtrMap.end())
{
    SomeClass* ptr = iter->second;
    g_PtrMap.erase(iter);
    delete ptr;

return NO_ERROR;

}

else
return 1;

}

Afterthoughts

These 2 examples are proof of concept examples to demonstrate how to use C++ objects in any language that has C bindings.

This is not production quality code though, because of the following reasons:

It does not catch C++ exceptions that could be thrown somewhere in the C++ code.
It does not properly handle the overflow of the g_Index variable after 2^32-1 object allocations.
There is no synchronization of access to the global map. This means the better_xxx functions are not safe for multithreaded usage.
There is no NULL pointer checking in the simple_xxx functions yet.

You can download the demo project with C++ and LabVIEW code. It is licensed under the MIT license so you can use it any way you want.