Cluebat-man to the rescue : ATL

Practical ATL: Solving the race condition in CAtlExeModuleT

vanDooren — Wed, 15 Apr 2009 12:40:00 GMT

When I was spelunking through the ATL header for my previous articles (the class object plumbing) I discovered that there is a serious race condition in the server lifetime management.

I contacted the Microsoft C++ folks, and someone from the libraries group told me that this problem has already been fixed in the next release of Visual C++.

But since the issue still exists in all current ATL projects, I decided to write something about it and publish the fix which will circumvent the problem in the recent versions of ATL (7.0 and upwards if I am correct).

The problem

The module lock count of an ATL COM server is managed in the CAtlExeModuleT<T> class. It is this lock count that keeps the server alive as long as it is nonzero.

The CAtlExeModuleT<T> class manages its reference count via the CComGlobalsThreadModel class increment and decrement methods. CComGlobalsThreadModel is a typedef for CComMultiThreadModel so these methods map to InterlockedIncrement and InterlockedDecrement

The last module lock is released if
- the class object itself has no more external connections (IExternalConnection), and
- there are no more object instances.

As soon as the final module lock is released (module refcount is 0), the shutdown of the server is initiated, which will happen through either a WM_QUIT message or a win32 event object. In both cases there is a time window between the triggering of the shutdown, and the actual call to CoRevokeClassObject. This time window can be several seconds.

This time window causes a race condition between the server (which is already dying at that point) trying to revoke the class object, and a hypothetical client, requesting an interface pointer to the class object. Depending on how the race turns out, the client might end up with an S_OK HRESULT from CoGetClassObject, and an invalid interface pointer. This will cause a critical problem in the client, and may cause an application crash.

The solution

The proper way to manage the server lifetime is to use CoAddRefServerProcess and CoReleaseRefServerProcess to manage the module lockcount. Those functions will (according to MSDN http://msdn.microsoft.com/en-us/library/aa910552.aspx) suspend all class objects when the reference count drops to 0, in an atomic operation.

In that case, the server starts dying AFTER all class objects are suspended. Any incoming client request for the class object will spawn a new server process after the dying server has finished revoking its class objects.

The fix

The code for fixing this is simple. I simply created a new class that inherits from CAtlExeModuleT<T> and then overrides the Lock and Unlock methods.

template <typename T>

class CAtlExeModuleT_NoRace : public CAtlExeModuleT<T>

{

public:

typedef CAtlExeModuleT<T> base;

#if _ATL_VER < 0x0A00

LONG Lock(void)

{

CoAddRefServerProcess();

return base::Lock();

}

LONG Unlock(void)

{

CoReleaseServerProcess();

return base::Unlock();

}

#else

//removed compiler messages

#endif

};

I use CoxxxxServerProcess functions to explicitly manage the class object registration, and then I delegate to the base class which maintains its own reference count.

The reason I do that is that the base::Unlock function is the one which triggers the module shutdown, and I don’t want to mess with that. And Unlock works only when paired with Lock, so the only thing I really need to do is to call the CoxxxxServerProcess routines to prevent possible activation requests. The rest is still done by CAtlExeModuleT<T>.

I put a compiler condition around my implementation of the new Lock and Unlock methods, because they are only needed for current versions of ATL. The next version of ATL won’t have this problem, so it would be dangerous to meddle with the reference counting, especially since I cannot predict what side effects this may have.

The conditional compilation allows you to safely upgrade the project to the next version of ATL. The compiler messages which show up in the compiler output inform you that it is no longer needed to inherit from CAtlExeModuleT_NoRace as shown below:

class CStuff_ServerModule : public CAtlExeModuleT_NoRace< CStuff_ServerModule >

{

public :

DECLARE_LIBID(LIBID_Stuff_ServerLib)

DECLARE_REGISTRY_APPID_RESOURCEID(IDR_STUFF_SERVER, "{597B5DFF-DACE-42B0-9E49-54A09C10B2BB}")

};

The only thing you have to do to use my new module class is to include the header and change CAtlExeModuleT to CAtlExeModuleT_NoRace.

And if you upgrade the project to VC10, then you can simply change it back as indicated by the compiler output messages that are printed out when the compiler parses my class.

Conclusion

I was surprised that this bug was still in the ATL headers after all these years. When I contacted the VC++ team I was half expecting them to tell me that I had missed something. This issue is what caused CoAddRefServerProcess to be created after all, over a decade ago, so a COM programmer implementing lifetime management should be aware of it.

However, the modular and hierarchical nature of ATL made it easy to fix this problem, so I guess that is a plus for ATL. And mistakes happen. It is always a bit painful if it happens in widely used libraries, but I too have had my share of whoopsies over the years. It is sad but it happens.

The header file with the fix is attached to this blog post under the MIT license.

Practial ATL: Implementing a non-standard class object

vanDooren — Fri, 20 Feb 2009 21:03:00 GMT

In my previous article in this series, I explained how class objects are associated with COM objects, and how it all fits together.

Armed with that knowledge, I can now demonstrate how to provide a custom class object that will allow us to support parameterized construction.

If you haven’t already read my previous article on class objects, please do so first because I am going to assume you did J

The non-standard class object interface: IStuffCreator

A class object can have any interface you want, just like the COM object we have created earlier. It is just another COM object after all. My custom interface looks like this:

[
      object,
      uuid(6DD69CDB-3128-432b-B335-773A287E6F06),
      oleautomation,
      helpstring("IStuffCreator Interface"),
      pointer_default(unique)
]
interface IStuffCreator : IUnknown {
[id(1), helpstring("method MakeMeAStuff")]
    HRESULT MakeMeAStuff(
      [in] BSTR name,
      [in] REFIID riid,
      [out, iid_is(riid)] IUnknown** ppStuff);
};

The bits between the square brackets are the attributes that are applied to this interface. ‘Object’ specifies that it is a COM interface. The uuid specifies the GUID that identifies the interface. ‘oleautomation’ specifies that the arguments being used are all automation compatible. The meaning of ‘helpstring’ is pretty obvious, and pointer_default specifies the assumptions that the marshaller can make about pointers.

If you want to know the details about it, then have a look in MSDN for MIDL attributes.

The interface itself is very simple. Instead of deriving from IDispatch like the IStuff interface, this interface derives from IUnknown. Deriving from IDispatch would only complicate matters without any benefit.

The interface has only one method, which is responsible for handing out new stuffs.

The parameter that will be used for constructing the CStuff instance itself is the ‘name’ parameter. If the CStuff would need multiple parameters, they would have to be part of the parameter list so that they can be used at the time of construction. The riid and ppStuff parameters have the same meaning as those of CoCreateInstance.

The declaration of CStuffCreator

If we want to implement this interface, we add a new C++ class with the name ‘CStuffCreator’ to our project, and implement our IStuffCreator interface

class CStuffCreator:
public IStuffCreator,
public CComObjectRootEx<CComGlobalsThreadModel>
{
public:

CStuffCreator(void){}
virtual ~CStuffCreator(void){}

BEGIN_COM_MAP(CStuffCreator)
    COM_INTERFACE_ENTRY(IStuffCreator)
END_COM_MAP()

//IStuffCreator methods
HRESULT STDMETHODCALLTYPE MakeMeAStuff(
    BSTR name,
    REFIID riid,
    IUnknown **ppStuff);

// helper method
void SetVoid(void* pv);
_ATL_CREATORFUNC* m_pfnCreateInstance;
};

As you can see, it is pretty simple. The CComObjectRootEx base class helps with reference counting, and is the same that is used by the other COM objects in our project.

We add IStuffCreator to the interface map using the ATL macros, and then we simply declare the single MakeMeAStuff method that we need to implement.

The SetVoid method and the function pointer are part of the default C++ interface for class objects in ATL, and are explained in my previous article.

The implementation

The implementation of CStuffCreator is even simpler than the declaration:

HRESULT STDMETHODCALLTYPE CStuffCreator::MakeMeAStuff(

BSTR name,

REFIID riid,

IUnknown **ppStuff)

{

CComPtr<IUnknown> newStuff;

HRESULT hr = m_pfnCreateInstance(

NULL, __uuidof(IUnknown), (void**)&newStuff);

if(FAILED(hr))

return hr;

return newStuff->QueryInterface(riid, (void**)ppStuff);

}

void CStuffCreator::SetVoid(void* pv)

{

m_pfnCreateInstance = (_ATL_CREATORFUNC*)pv;

}

Ignore the ‘name’ parameter for now. That will be used when we are going to initialize the new instance. The bare functionality of our class object is to use the function pointer to create a new instance. I could have done this in 1 step instead of 2 (the QueryInterface isn’t necessary at this point) but this way I can slide in some additional functionality later on.

And you can see now that the class object will receive the creator function pointer via the ‘SetVoid’ method. This mechanism was explained in my previous article.

Parameterized construction

The point of using a non-standard class object in this demonstration was to be able to create objects that need some sort of initialization before the client has access to them. Normally, ATL supports only default constructors using, because IClassFactory is a generic interface.

For parameterized constructors, we are going to have to do something special.

There are essentially 2 ways we can do this. The first is to change CStuff to have a parameterized constructor. But ATL can’t handle that, and it would mean we have to abandon the incredible convenience of being able to use the ATL::Creator<T> class, ATL::CComObject class, and other helper classes that take the pain out of COM.

Don Box wrote an article about parameterized construction, many years ago, and this was the approach he took. Of course, ATL was much simpler in those days, and abandoning it was perhaps less of an inconvenience.

The approach I am taking is much simpler: I cheat!

That’s right. I cheat. I am not going to implement parameterized constructors. I am going to fake it. But from the client’s point of view, it’s going to look exactly the same, so he won’t mind. And from the server’s point of view, it will be much simpler to implement, so everybody wins.

The second approach to implementing parameterized construction is to implement an internal interface on CStuff, which will be used for initializing it. The new CStuff instance will not be accessible to anyone until the first interface pointer is handed out by the class object. And we are the ones implementing the class interface, so we can make sure that the new instance is initialized properly before it is released into the wild.

The initialization interface

The interface for initializing a CStuff is pretty simple (I left out the header for clarity):

interface IStuffInit : IUnknown {

[id(1), helpstring("method InitStuffInstance")]

HRESULT InitStuffInstance(

[in] BSTR Name);

};

Our CStuff instances are supposed to have a name, so that’s what needs to be in the init interface. If they would also need an address, a hat, and a pair of pants, we would have to put those in the parameter list of the InitStuffInstance method as well.

Implementing the initialization interface

Implementing the interface in CStuff is easy I highlighted the changes that have to be made to the declaration.

// CStuff

class ATL_NO_VTABLE CStuff :

public CComObjectRootEx<CComMultiThreadModel>,

public CComCoClass<CStuff, &CLSID_Stuff>,

public IDispatchImpl<IStuff, &IID_IStuff, &LIBID_Stuff_ServerLib, /*wMajor =*/ 1, /*wMinor =*/ 0>,

public IStuffInit

{

private:

CComBSTR m_Name;

public:

DECLARE_CLASSFACTORY_EX(CStuffCreator)

// removed irrelevant stuff

BEGIN_COM_MAP(CStuff)

COM_INTERFACE_ENTRY(IStuff)

COM_INTERFACE_ENTRY(IDispatch)

COM_INTERFACE_ENTRY(IStuffInit)

END_COM_MAP()

// removed irrelevant stuff

HRESULT STDMETHODCALLTYPE InitStuffInstance(BSTR Name);

};

First we add IStuffInit to the inheritance list, we add it to the COM map, and then we add the declaration for InitStuffInstance.

Of course, the name has to be part of the objects state, so we add a CComBSTR variable to hold it.

And finally, we use the DECLARE_CLASSFACTORY_EX macro to change the class object for CStuff from CComClassFactory to CStuffCreator.

The actual implementation of the interface consists of the simple code

STDMETHODIMP CStuff::InitStuffInstance(BSTR Name)

{

m_Name = Name;

return S_OK;

}

Note the ease with which CComBSTR allows us to perform simple string operations.

Initializing the new CStuff object

Now that there is a dedicated function to handle the initialization, the changes to CStuffCreator are trivial:

HRESULT STDMETHODCALLTYPE CStuffCreator::MakeMeAStuff(

BSTR name,

REFIID riid,

IUnknown **ppStuff)

{

CComPtr<IStuffInit> newStuff;

HRESULT hr = m_pfnCreateInstance(

NULL,__uuidof(IStuffInit), (void**)&newStuff);

if(FAILED(hr))

return hr;

hr = newStuff->InitStuffInstance(name);

if(FAILED(hr))

return hr;

return newStuff->QueryInterface(riid, (void**)ppStuff);

}

What about double initialization

By now you might be wondering: But what if the client application retrieves this interface and performs another initialization?

However, this will not be possible.

It’s true that I put the interface in our IDL file, and that I implemented it in the CStuff object. But I did not put that interface in the library declaration, nor did I put it in the interface list of the coclass.

So for all practical purposes, this interface is not accessible to the client application. If you are really paranoid, you could argue that an attacker might know the interface GUID, and guess the interface definition, and with some trial and error make it work.

Even then, the fix is quite simple. You can add a boolean to your class that indicates if the object was initialized or not. Set it to false in the constructor, and then the InitStuffInstance can either perform initialization if it is false and set it to true, or return E_FAIL if it is already true.

You don’t even have to care about race conditions, because there aren’t any. Only the class object can perform the real initialization. And since no one else has an interface pointer yet, there can’t be a race.

The client side

Creating a new CStuff has changed a little bit, because now we can’t use CoCreateInstance anymore. So construction is now done in 2 stages:

CComPtr<IStuffCreator> stuffClass;

hr = CoGetClassObject(

__uuidof(Stuff), CLSCTX_ALL, NULL,

__uuidof(IStuffCreator), (void**)&stuffClass);

if(FAILED(hr))

return hr;

//make a new stuff

CComBSTR stuffName = L"Kato";

CComPtr<IStuff> newStuffUnk;

hr = stuffClass->MakeMeAStuff(

stuffName.m_str,

(GUID*)&__uuidof(IStuff),

(IUnknown**)&newStuffUnk);

if(FAILED(hr))

return hr;

It’s still very simple though. First we get the class object, and then we use the IStuffCreator interface to create the new CStuff.

Server lifetime management

Everything works, hip hip hurray… until you destroy the first CStuff instance, and then try to create another one. That is where it all goes pear shaped.

The problem is that we haven’t taken care of managing the lifetime of the server (the exe). The key issue here is the module lock reference count.

Each COM module (exe or dll) has a global reference count to keep track of how many objects are ‘alive’. COM objects exist physically inside a DLL or EXE. So the EXE has to stay alive as long as there are living objects inside. Conversely, as soon as there are no living objects anymore, the EXE should shut down.

What happens in our example is that CStuff raises the module ref count when it is created. And when the CStuff instance destroys itself, it lowers the module ref count. If you remember correctly, the class object itself uses CComObjectNoLock as its COM core. This means that its existence does not influence the module lifetime.

So when the CStuff instance destroys itself and lowers the module ref count, the ref count reaches 0 and the server EXE shuts down.

The client is unaware of this and tries to use the class object again. It still thinks it has a valid interface pointer, but the underlying class object is long gone and there will be a critical error in the client application.

You might say ‘well duh, then don’t use CComObjectNoLock, doofus’ but that has the opposite effect. The COM runtime will always have the interface pointer that was used for registering the class object with CoRegisterClassObject. This means that the class object would not destroy as long as the server lives. And the server would keep living as long as the class object was around. This is clearly not the answer.

IExternalConnection to the rescue

The key to solving this problem is a rarely used standard interface. IExternalConnection is an interface that can be used by an object to track how many external connections there are to itself. This is basically an alternative to the standard object reference count.

There are 2 methods in this interface:

STDMETHODIMP_(DWORD) AddConnection(

DWORD extconn, DWORD dwReserved);

STDMETHODIMP_(DWORD) ReleaseConnection(

DWORD extconn,

DWORD dwReserved,

BOOL bLastUnlockReleases);

If a client requests any class object, the COM runtime will retrieve it. But before the interface pointer is released to the client, it will request some interfaces itself for housekeeping purposes. One of those interfaces is IExternalConnection.

If a client requests an interface pointer to a class object, the marshaller will call AddConnection when it hands out the pointer, and ReleaseConnection when the pointer is destroyed.

So all we have to do is to make sure that a call to AddConnection will increment the module lock count, and a call to ReleaseConnection releases the module lock count.

Implementing IExternalConnection

If you look in MSDN, you won’t find it. It is undocumented, but the ATL headers contain an implementation for IExternalConnection. It’s called IExternalConnectionImpl<T>.

To use it, we only have to change CStuffCreator so that it inherits from it, and add it to the COM interface map.

class CStuffCreator:

public IStuffCreator,

public IExternalConnectionImpl<CStuffCreator>,

public CComObjectRootEx<CComGlobalsThreadModel>

{

public:

//removed irrelevant stuff

BEGIN_COM_MAP(CStuffCreator)

COM_INTERFACE_ENTRY(IStuffCreator)

COM_INTERFACE_ENTRY(IExternalConnection)

END_COM_MAP()

//IExeternalConnectionImpl<T> extension

void STDMETHODCALLTYPE OnReleaseConnection(

bool bThisIsLastUnlock, bool bLastUnlockReleases);

void STDMETHODCALLTYPE OnAddConnection(

bool bThisIsFirstLock);

//removed irrelevant stuff

};

That is really all there is to it. The default implementation does not modify the module lock count. But it has 2 extension points that we can override. So that’s what we do to make sure the module lock count is managed properly.

void STDMETHODCALLTYPE CStuffCreator::OnAddConnection(

bool bThisIsFirstLock)

{

_pAtlModule->Lock();

}

void STDMETHODCALLTYPE CStuffCreator::OnReleaseConnection(

bool bThisIsLastUnlock, bool bLastUnlockReleases)

{

_pAtlModule->Unlock();

IExternalConnectionImpl<CStuffCreator>::OnReleaseConnection(

bThisIsLastUnlock, bLastUnlockReleases);

}

The only special thing that still needs mentioning is that OnReleaseConnection delegates back to the default implementation in IExternalConnectionImpl. It does this because there is something that needs to be done when the last external connection is closed.

void OnReleaseConnection(

bool bThisIsLastUnlock, bool bLastUnlockReleases)

{

if (bThisIsLastUnlock && bLastUnlockReleases)

CoDisconnectObject(static_cast<T*>(this)->GetUnknown(), 0);

}

If the last connection closes, the class object needs to call CoDisConnectObject to destroy its marshalling stub. Now I could copy paste this code into my implementation of OnReleaseConnection, but it is simpler to just delegate back to the base class for stuff that needs doing anyway.

The result

Now that we have provided this implementation of IExternalConnection, the server EXE will stay alive until all CStuff instances are gone, and noone is using the class object anymore. Only then will the EXE terminate.

CComClassFactory revisited

If you look at the interface implemented by CComClassFactory, you’ll notice that IExternalConnection is missing. And yet, server lifetime of CComClassFactory is managed correctly.

The reason for this is that the COM runtime treats IClassFactory class objects differently from other class objects. It will check if the class object supports IClassFactory. And if it does, it calls IClassFactory::LockServer to indicate that someone is using the class object. And it unlocks the class object when the class object reference is released again.

In a way, it uses the IClassFactory in almost the same way as it would use IExternalConnection. And in a way it is too bad that this functionality was included in IClassFactory directly, instead of making IClassFactory inherit from IExternalConnection. The reasons for this are lost in the mists of time. But whatever the reasons, this is why IClassFactory doesn’t need IExternalConnection.

Conclusion

As you can see, implementing a non standard class object is not so hard. As long as you have a basic understanding of the underlying COM principles, ATL takes care of all the dirty work.

And by using these techniques, we can implement COM objects that need initialization before they can safely be used by the client. The source code for this article is available under the MIT license as usual.

And again I’d like to add a thank you for Microsoft’s Tim Springfield and Amit Mohindra

Practical ATL: Understanding the class object

vanDooren — Tue, 10 Feb 2009 07:46:00 GMT

In my previous article in this series, I explained how to create a simple COM server and implement a method that returns an enumerator object.

It was all done very easily, but the ATL wizards hide some of the things that you really ought to know for my next article. For starters: where is the class object?

The class object is the COM object responsible for creating new instances of the CStuff COM object that I demonstrated in a previous article. But we didn’t have to write any code for it, nor is it even visible… so what’s up with that?

Some more on class objects

The class object of a COM server is the object that is actually responsible handing out instances of the COM object. The class object is a COM object itself, but only 1 instance of it normally exists in a module (DLL or EXE) per defined COM object type.

For example, the ‘Stuff’ COM object that I implemented in my previous article uses the standard class factory with IClassFactory as the interface for handing out new CStuff instances.

The most important method in IClassFactory is ‘CreateInstance’ which simply creates a new instance of the COM object and then passes an interface pointer back to the caller.

Normally, this process is invisible to the client application, because ‘CoCreateInstance’ uses IClassFactory behind the scenes so that the client doesn’t have to bother with it.

First it calls ‘CoGetClassObject’ to get the class object for the requested COM object, and then it uses the IClassFactory interface of that class object to create a new instance. That instance is then passed to the caller, as an interface pointer of the type that was requested in the call to ‘CoCreateInstance’.

I think it is safe to say that ‘IClassFactory’ (or one of its relatives) is the interface on the class object of 99.99% of all COM objects that have a class object.

The default class factory

By default, CStuff uses CComClassFactory as its default class object. But looking at the declaration of CStuff, it is not immediately apparent how it does this.

CStuff derives from CCoClass<T>, which has the macro DECLARE_CLASSFACTORY in its implementation.

For EXE servers, this macro expands to DECLARE_CLASSFACTORY_EX(ATL::CComClassFactory), which in turn expands to

typedef ATL::CComCreator< ATL::CComObjectNoLock< ATL::CComClassFactory > > _ClassFactoryCreatorClass;

DLL servers use CComObjectCached instead of CComObjectNoLock, but I don’t cover DLL servers in this article.

As you can remember from the first article that featured CStuff, the C++ COM classes are missing the IUnknown core by default, because the implementation of that core depends on how we want to the object to behave.

This means we have to shove our class into a COM object to make it whole. For class objects, the CComObjectNoLock is most appropriate, because the lifetime of the object should not contribute to the lifetime of the server executable. Otherwise there would be a deadlock.

The server would start, and create the class object. And then the server would continue to live until the class object was destroyed. But since the COM runtime will always have a reference to the class object, the reference count would never reach zero, the class object would never be destroyed, and the server would never be able to shut down.

And finally, this class definition is shoved into a creator object which will help us by creating a properly initialized instance of the class object itself via its static CreateInstance function.

Associating the COM object C++ class with the class factory

So the COM object C++ class has a typedef that identifies the class object type. But how does the executable use this fact?.

This is where things get a little hairy, and possibly start to make ominous squishy sounds J

After the class declaration of CStuff, there is the innocuous line OBJECT_ENTRY_AUTO(__uuidof(Stuff), CStuff)

The OBJECT_ENTRY_AUTO is a macro that expands to this:

#define OBJECT_ENTRY_AUTO(clsid, class) \
      __declspec(selectany) ATL::_ATL_OBJMAP_ENTRY __objMap_##class = {&clsid, class::UpdateRegistry, class::_ClassFactoryCreatorClass::CreateInstance, class::_CreatorClass::CreateInstance, NULL, 0, class::GetObjectDescription, class::GetCategoryMap, class::ObjectMain }; \
      extern "C" __declspec(allocate("ATL$__m")) __declspec(selectany) ATL::_ATL_OBJMAP_ENTRY* const __pobjMap_##class = &__objMap_##class; \
      OBJECT_ENTRY_PRAGMA(class)

Don’t worry if the sight of this code makes you nauseous. That’s normal. I highlighted the important parts

What it does is simple. It’s just how it does it that is ugly. It creates a map entry for the ATL innards of our server executable. In this map entry, it associates the static method for creating the class object with the static method for creating an object instance.

Creating the class object

Now that the class object and the COM object are associated with each other, the next bit of code in the ATL headers is where the magic happens:

_ATL_OBJMAP_ENTRY* pEntry;
if (m_pObjMap != NULL)
{
    pEntry = m_pObjMap;
    while (pEntry->pclsid != NULL && hr == S_OK)
    {
      hr = pEntry->RegisterClassObject(dwClsContext, dwFlags);
      pEntry++;
    }
}

When the ATL module initializes, it will iterate over all elements in the object map and register each one. And this registration process entails the following:

    IUnknown* p = NULL;
    if (pfnGetClassObject == NULL)
      return S_OK;
    HRESULT hRes = pfnGetClassObject(
                         pfnCreateInstance,
                         __uuidof(IUnknown),
                         (LPVOID*) &p);
    if (SUCCEEDED(hRes))
      hRes = CoRegisterClassObject(
               *pclsid, p, dwClsContext, dwFlags, &dwRegister);
    if (p != NULL)
      p->Release();
    return hRes;

If you do a bit of spelunking in the headers, you’ll see that pfnGetClassObject is a function pointer that maps to _ClassFactoryCreatorClass::CreateInstance. So this code simply creates a new class object and registers it.

Again, this is specific to EXE servers. In DLL servers, the class object is created on demand in the DllGetClassObject function, which is called by CoGetClassObject if the COM server is a DLL.

Creating CStuff instances

If you look at the _ClassFactoryCreatorClass::CreateInstance, you’ll see that its first parameter is a void *. And the value that is passed in this parameter by the ATL module is the address of the static method for creating new object instances.

_ClassFactoryCreatorClass::CreateInstance simply creates a new class object ‘p’, and then calls p->SetVoid(pv); to pass the aforementioned function pointer to the class object.

And the SetVoid method is no more complicated then this:

void SetVoid(void* pv)
{
m_pfnCreateInstance = (_ATL_CREATORFUNC*)pv;
}
_ATL_CREATORFUNC* m_pfnCreateInstance;

I guess the original developer hadn’t had his morning coffee yet when he came up with this name. It manages to be 100% factually correct, while being 100% opaque at the same time. But it does the job.

When the class object has to create a new instance, it simply does the following:

STDMETHOD(CreateInstance)(LPUNKNOWN pUnkOuter, REFIID riid, void** ppvObj)
{
    // snip irrelevant code
        hRes = m_pfnCreateInstance(pUnkOuter, riid, ppvObj);
    // snip irrelevant code
    return hRes;
}

Providing a new class factory

Now that you understand how the class object plumbing is hooked up, it is easy to see how you can implement your own class object.

Remember how DECLARE_CLASSFACTORY_EX maps a class object to the _ClassFactoryCreatorClass typedef?

In C++, a typedef in a derived class can override a typedef in a base class. So even though the CCoClass<T> base class says that the _ClassFactoryCreatorClass maps to ATL::CComCreator< ATL::CComObjectNoLock< ATL::CComClassFactory > >,

We can override this by simply putting DECLARE_CLASSFACTORY_EX(CStuffCreator) In the declaration for CStuff. That’s really all there is to it. The macro magic takes care of the rest.

Conclusion

If changing the class object for a COM server is as trivial as adding a single line to your COM server C++ class, then why did I just write 4 pages of stuff?

I wrote all this because if I tell someone to use a macro to do something non-trivial, I want him or her to understand what is going on. There are already far too many programmers on this planet who do things without understanding what it is they do, or how it works. And I am not going to add some more by glossing over the details.

Of course, judging by the number of visitors on this corner of the internet that I call ‘home’, I stand little risk of doing so no matter how crappy I would write J

This article was originally part of my article on implementing a custom class object. But after I spent more than 3 pages on the previous explanation, I thought it would be best to put it in a separate article.

A big thank-you goes to Jim Springfield of the Architects team within the developers division (I think) of Microsoft for taking the time to review this article for correctness, and to Amit Mohindra for helping my questions and articles reach the right people.

And you can find the demo project attached to this article under the MIT license.

Practical ATL: recognizing marshalling problems

vanDooren — Tue, 27 Jan 2009 11:30:00 GMT

Before I publish my next article, I should explain something else about the black magic aspect of ATL / COM: Marshalling.

The Stuff server we made in the previous article is an out-of-proc server. This means that the arguments being passed to / from a stuff instance have to be somehow moved back and forth between different process contexts (or different threading apartments).

In the case of IStuff, we don’t really have to bother, because IStuff derives from IDispatch. And in that case, there is a marshaller (aptly named Universal Marshaller) that can be used to do all this for you.

If IStuff wouldn’t derive from IDispatch, then we would need to provide our own proxy – stub interface that knows how to marshal the interface. With ATL, this is not difficult. You’ll notice that the Stuff solution of my previous article had a ‘Stuff_ServerPS’ project that I didn’t mention before.

That solution will build the proxy stub dll that will perform the marshaling. It is disabled by default, so you have to enable it in the build configuration before it is built. Also remember to rebuild the proxy, every time the IDL files of your project change.

So for the remainder of this article I will show what can happen if there is a marshalling problem. Don’t worry if it all seems ugly, gory, and incomprehensible. Normally you won’t have to deal with issues like this. But if you should ever do, you should at least recognize the signs that you have a marshaling problem.

Confused? It’s the marshaller

If you find yourself in a situation where very weird things happen, suspect a marshalling problem.

If you get the error code E_NOINTERFACE when you know for sure that your object implements it properly, suspect a marshalling problem.

If you get a crash while using an interface pointer that was properly created without getting a HRESULT that indicated a problem,… that’s right: you can bet it’s a marshaling problem.

Example

The article I am currently writing discusses implementing a non standard class object, for the purpose of parameterized construction of the Stuff object. Never mind for the moment about what any of that means.

For all practical purposes: it just means that I defined the following interface via copy / paste / edit:

[

      object,
      local,
      uuid(6DD69CDB-3128-432b-B335-773A287E6F06),
      dual,
      helpstring("IStuffCreator Interface"),
      pointer_default(unique)
]
interface IStuffCreator : IUnknown {
[id(1), helpstring("method MakeMeAStuff")]
    HRESULT MakeMeAStuff(
      [in] BSTR name,
      [in] REFIID riid,
      [out] void ** ppStuff);
};

There were 4 things wrong with this interface (showing just why you shouldn’t define IDL interfaces in a hurry). This has to be some sort of record.

‘local’ shouldn’t be there.
‘dual’ shouldn’t be there.
‘void’ should be ‘IUnknown’
‘ppStuff’ should be attributed with riid_is(riid).

So let’s have a look at the result of these different issues.

Issue 1: local

‘local’ simply shouldn’t be there. ‘local’ is a MIDL attribute that –when use in an interface header- causes MIDL not to generate stubs for that interface.

The effect on our interface is that the IStuff specific methods are not forwarded to the remote server.

You might wonder: why does it exist. The answer is: in case you want to write custom marshalling code, or in case the calls never have to be remoted. In that case you can have a local interface and a remote interface. And usage and marshalling could be optimized.

IUnknown is such an interface. But it has custom stub code to deal with things like remoting

The result of using ‘local’ on my IStuffCreator interface is that trying to call GetEnum on the returned IStuff interface pointer will result in a debug assertion: Run-Time Check Failure #0 - The value of ESP was not properly saved across a function call. This is usually a result of calling a function declared with one calling convention with a function pointer declared with a different calling convention.

And if you have this problem with COM, it is usually indeed an indication that you have a calling convention mismatch. In this case, it is a side effect of the fact that we are trying to execute a method that isn’t remoted.

Issue 2: dual

This one shouldn’t be here either. Dual indicates that the interface implements IDispatch, and this could have an effect on marshalling.

In this case, it worked just fine. Perhaps it is because of the fact that I use my object as a class object, or perhaps because my interface itself uses only arguments that are automation compatible, or perhaps because the stars are aligned in the proper configuration.

I don’t know. And anything you don’t know for certain will hurt you when dealing with DCOM. It has to be correct.

Issue 3: void

This one is interesting.

If I want to pass an interface pointer, it has to be passed as an IUnknown**, instead of a void**. This is because Universal Marshaller has no idea on how to marshal something which has only contextual meaning, nor does the IDL compiler know what you mean with it.

With this I mean that my server knows it will be an interface pointer, and my client will know it is an interface pointer, but the stuff in the middle doesn’t. It sees just a memory address.

I used void** because that is what QueryInterface itself uses. And what’s good enough for QI should be good enough for me. Except QI has custom code for marshalling its arguments across, which I don’t. So QI can use void** because both the proxy and the stub know what is going on.

So what happens when you use void**?

I request the class object like this:

hr = CoGetClassObject(
__uuidof(Stuff),

CLSCTX_ALL,

NULL,

__uuidof(IStuffCreator),

(void**)&stuffClass);

And the resulting hr is 0x80020008: Bad variable type. There is absolutely nothing wrong with this line of code. But because there is something in the interface itself that confuses the marshaller, it tells me to sod off when I want that specific interface.

Credit where it is due: I figured this one out thanks to someone whose name I can’t figure out J. But his blog post is here.

Issue 4: iid_is(riid)

This one is also a good one.

The iid_is informs the marshaller that riid points to the identifier which identifies the interface that is being marshaled.

This way, the marshaller knows how to marshal the interface that was specified with riid. Without that part, it would only marshal the IUnknown part of the interface, and not the IStuff specific part.

As a result, you can use the IUnknown part of the interface without a problem, but if you use the IStuff specific part, you will get an access violation, stack corruption or other interesting phenomenon.

Conclusion

A couple of things can be concluded at this point.

First of all: Marshaling problems are hard to debug, because the problems occur outside of your source code.

If you use any interfaces that don’t derive from IDispatch, then you have to enable the ‘xxxPS’ project in your solution so that the ProxyStub dll will be built and registered. And of course, this dll has to be part of your install procedure. Without this stub, you could get access violations, or you get an E_NOINTERFACE error if the marshaler doesn't know one of those interface exist.

As I have shown, you need to be careful and pay attention when you manually define COM interfaces. If you don’t, then you can have all sorts of hard to diagnose problems.

And if you ever find yourself in a situation where nothing makes sense, then suspect marshalling issues and have a long hard look at your IDL files.

Practical ATL: Implementing an enumerator object

vanDooren — Wed, 21 Jan 2009 19:55:00 GMT

This is my first article about practical ATL examples, which I already mentioned here.

In this example I will create an ATL Server which implements 1 custom interface: IStuff. Hardly an inspiring name, but I couldn’t think of anything better before I had my first coffee.

This interface will have 1 method which will return an enumerator to the caller. Specifically, this enumerator will implement IEnumString.

Enumerators are part of the interface of many real-life COM servers, but creating your own is an arduous task. ATL makes this much easier, but an example makes things so much easier still.

Since client code will have to deal with this enumerator (and to have a demo app) I will also create a client application that creates an instance of the server, consumes the IStuff interface and then enumerate the items contained in the returned enumerator.

Before we get started

If you don’t own a copy of ‘ATL Internals, Second Edition’ yet, I strongly advise you to buy it. It is worth the money, and apart from leading you through all the interesting and gory stuff, it also starts by explaining the actual process of creating an ATL server through the IDE, as well as implementing methods, properties and events.

I am only going to cover that aspect very lightly. It should be detailed enough for anyone who has done C++ development using VS:

Create a new ATL Server project with project name ‘Stuff_Server’ and a solution name ‘Stuff’.
Choose the EXE project type and click ‘finish’
Add a new class to the project, and specify ATL Simple Object. Use ‘Stuff’ as the name of the object.
Switch to classview, and add a new method to the IStuff interface with the prototype ‘HRESULT GetEnum(IUnknown ** ppEnum). ppEnum should be marked as an ‘out’ parameter.

Build the project, and you should be ready for the actual work. The ATL wizards will have implemented all the IDL and DCOM stuff for you, and you only have to implement the functionality.

Finally, I always end by changing the project configuration to use static CRT and ATL runtimes instead of dynamic runtimes. This makes it infinitely more easy to distribute the resulting executables without having to bother with registration and installation of the runtime dlls and their manifests.

Creating the server

Now that the ATL wizards have generated the function skeleton for us, it is time to do something with it:

STDMETHODIMP CStuff::GetEnum(IUnknown** ppEnum)
{

}

The enumerator

Before we can do anything else, we have to think about the actual enumerator for a bit. In our case we implement IEnumString. This is an interface for enumerating string (who’d have thought, eh?).

Enum interfaces all look alike, and ATL helpfully has a stock implementation for generic enumerators that allows you to specify the Interface, GUID, datatype and data type copy traits as template parameters. In order to facilitate coding, we make a typedef for this class:

typedef CComEnum<

IEnumString,

&IID_IEnumString,

LPOLESTR,

_Copy<LPOLESTR> > CComEnumStringAbstract;

This will specialize the enumerator for our purposes. The use of the first 3 parameters is obvious. Don’t worry about the _Copy parameter. I will cover that later.

The enumerator itself is a normal COM object, so it needs lifetime management, interface managements, and all other things you really don’t want to care about for something so simple. At this point, CComEnumStringAbstract will still be an abstract class because it can’t decide how to do these things for you.

In order to make our enumerator a full fledged COM object, we have to shove it into a bare COM object. This is very easy:

typedef CComObject<CComEnumStringAbstract> CComEnumString;

And that is all there is to creating an enumerator! Anyone who has ever implemented one manually will agree that this is infinitely more convenient

Implementing GetEnum

The implementation for GetEnum is also fairly easy (comments removed for clarity)

STDMETHODIMP CStuff::GetEnum(IUnknown** ppEnum)
{
CComEnumString * thEnumObj = NULL;
HRESULT hr = CComEnumString::CreateInstance(&thEnumObj);
if(FAILED(hr))
return hr;

CComPtr<CComEnumString> thEnum(thEnumObj);

LPOLESTR strings[] = {L"One", L"Two", L"Three", NULL};
hr = thEnum->Init(&strings[0], &strings[3], NULL, AtlFlagCopy);
if(FAILED(hr))
return hr;

hr = thEnum->QueryInterface( __uuidof(IUnknown), (void**)ppEnum);
return hr;
}

First we create a new CComEnumString object. It is worth noting that the object returned by CreateInstance has a refcount of 0. Before we do anything with it we should AddRef it, but instead of doing that manually, I do it via assignment to the CComPtr. smart pointer. That way the object also gets released automatically at the end.

Then we have to make sure the enumerator has some data to enumerate. We do this by populating an array, and then telling the enumerator to copy the data out of it into its own internal buffer. This is done via the AtlFlagCopy flag.

It might seem weird if you come from a C background, but the ‘End’ pointer has to point to the first position beyond the last element. This is normal with collections in C++. That is also why I populate the array with a NULL pointer at the end; So that there is something in the array after the last ‘real’ element. This way there is no buffer overflow.

Using AtlFlagCopy has the overhead of having to copy the data, but otoh the enumerator does not have to care about the lifetime of the array. We could also use new[] to create a new array and then use the AtlFlagTakeOwnership flag to indicate that it is up to the enumerator to call delete[] and clean up. But in this example we can’t do that (read below).

When the enumerator is initialized, we return an IUnknown interface pointer to the caller. Since this will have increased the enumerator ref count, the enumerator object will keep on living when the CComPtr goes out of scope and releases its own reference to the enumerator.

There. That is all there is to it on the server side. No vast amounts of copy pasted boilerplate code, but just a couple of lines of powerful ATL incantations J

_Copy<LPOLESTR> revisited

If an enumerator has to copy data to a client, or if it has to copy data into its internal buffers in the ‘Init’ function, it has to know how to do this.

It’s all good and well to do a simple memcpy of an ULONG or a double, but for strings (LPOLESTR) and interface pointers (IUnknown) this won’t work. There’s ref counting and memory ownership to deal with.

Of course, the template class, not knowing about this, can’t anticipate what you want, so instead it needs a policy class to delegate to. That is where the _Copy<> template class comes in. It has 3 methods: Init, Destroy, and Copy. Those methods perform the actual operations needed to copy an instance of T to another instance of T, to initialize a new one, and to destroy one.

There are several stock specializations for _Copy<>, one of which works for LPOLESTR. So rather than copying the LPOLESTR pointer itself, it calls CoTaskMemAlloc to create a buffer for the copy of the string, and then copies over the data.

If you look at my example more closely, you will also see why I cannot simply new up an array of string literals and pass those to the Init function with the AtlFlagTakeOwnership flag.

LPOLESTR is superficially identical to wchar_t*. So using a string literal instead of an LPOLESTR (which is what I do in my example) will work without a hitch. But if I’d new up an array and tell the enumerator to manage it, that is when it would all go pear shaped.

Because the enumerator would not only call delete[] on the array, but it would also call _Copy::Destroy for each of the items. And trying to free up a string literal with CoTaskMemFree will result in a hard to diagnose crash.

Installing the server

On a development station this is done automatically by Visual Studio. But if you need to do this on another PC, you just execute the server from the command line with the /RegServer argument.

Creating the Client

Creating the client application is fairly simple, but I wanted to show how to do it anyway, because then I could demonstrate that the COM server works as intended.

I create a simple win32 console application so that I only had to care about the actual functionality.

Importing the server type library

In the olden days or yore, working with COM servers in a C++ project was a bit tedious. You had to include the IDL files in your project and compile them, then include the resulting header files, add the .c files to your project for the GUID definitions… Not rocket science by any means, but still… a bit messy and ugly.

Now, I have very few kind words for Visual Basic, but to give credit where it is due: Visual Basic is (in large part) responsible for the existence of tlb files.

Visual Basic programmers wanted to use COM too, and the people who made Visual Basic wanted to prevent the application programmers to have anything to do with IDL. They wanted Visual Basic to take care of all that mess behind the scenes.

And thus the type library (tlb) was born.

The type library contains the same stuff as the IDL files, but in a machine readable, programming language independent format.

To use a COM server, you need to do nothing more than add the following line to the StdAfx.h header file of the client program:

#import "Stuff_Server.tlb" no_namespace

You also need to make sure that the compiler knows where to find Stuff_Server.tlb. This is easily done by adding the following line to the ‘Additional include directories’ setting:

..\Stuff_Server\$(ConfigurationName)

If you then compile StdAfx.cpp, the compiler will pull in the type library and generate the appropriate headers which are then automatically included.

The no_namespace attribute tells the compiler that it shouldn’t put the declarations in a specific namespace.

Using the COM server

The client application code itself is fairly trivial:

int _tmain(int argc, _TCHAR* argv[])

{

HRESULT hr = CoInitialize(NULL);

if(SUCCEEDED(hr))

{

CComPtr<IStuff> stuff;

hr = CoCreateInstance(

__uuidof(Stuff), NULL, CLSCTX_ALL,

__uuidof(IStuff), (void**)&stuff );

if(FAILED(hr))

return hr;

CComPtr<IUnknown> thEnumUnk;

hr = stuff->GetEnum((IUnknown **)&thEnumUnk);

if(FAILED(hr))

return hr;

CComPtr<IEnumString> thEnum;

thEnumUnk->QueryInterface(__uuidof(IEnumString), (void**) &thEnum);

{

LPOLESTR str;

hr = thEnum->Next(1, &str, NULL);

if(hr == S_OK )

wprintf(L"%s\n", str);

CoTaskMemFree(str);

}while (hr == S_OK );

}

CoUninitialize();

return 0;

}

First I initialize the COM runtime and then create a new instance of the Stuff object. I request IStuff as the initial interface. I also use the __uuidof keyword to get the GUID for an interface / object instead of using the CLSID_ and IID_ parameters. In my opinion, it is more convenient.

The COM smart pointer takes care of the reference counting for me. This is most convenient, because then I am certain that it will get released properly without me having to worry about it.

Btw, this is also the reason that all the code is placed into a separate scope block after the call to CoInitialize. The smart pointers clean up their interface pointers when they go out of scope. If CoUninitialize would have been called before that, it would lead to weird problems / crashes.

Via the IStuff interface I request an enumerator object. This gets returned as an IUnknow interface, so I have to get the IEnumString interface via the IUnknown::QueryInterface method. So that’s 3 interfaces already, and I still don’t have to care about reference counting J

After that I use the ‘Next’ method of the IEnumString interface to retrieve the items contained in the enumerator. This the only place where I have to perform a manual cleanup action.

The IEnumString interface does not work with BSTRs, it works with LPOLESTRs. While these are code compatible because they are the same type of data, their semantics are not. The _Copy<LPOLESTR> mentioned earlier allocates data for the string using CoTaskMemAlloc. It has to be freed by CoTaskMemFree.

CComBSTR would use SysAllocString and SysFreeString. And while the code would compile and execute perfectly, there would be an crash as soon as the CComBSTR would try to free the string it got from the call to ‘Next’.

Conclusion

As you can see, implementing COM servers and clients can be really easy with the proper use of ATL. And you’ve also seen that implementing an enumerator object is made really easy.

The code demo code is up for download with this article under the MIT license. Have fun.

I will probably write some more ATL / COM stuff soon. Stay tuned for more DCOM archaeology from the mists of time ;-)

Getting started with ATL

vanDooren — Tue, 20 Jan 2009 10:53:00 GMT

I am currently working on some projects where I have to program a DCOM server. There are several reasons why it has to be DCOM and C++, instead of e.g. .NET remoting and C#.

The biggest pain (imo) about DCOM is that the technology stems from an era when the internet was no ubiquitous, and there were no huge forums filled with experts, or blogs by programming gurus and sites like codeproject.com There were newsgroups, but those were generally only used for specific problems.

There is a vast amount of concise information about everything you can do with the .NET framework. For DCOM there is … a big dark void.

Granted, there are several resources out there. But if you look on sites like codeproject, you will find that most of the articles about DCOM and ATL are written before 2003. Most of the information out there will not cover the improvements that have been made to ATL (or DCOM) since the last 5 years.

And when looking into the more obscure corners of DCOM, there are some things that noone really likes to talk about because the documented semantics are so vague, and if you look into things like custom IMoniker implementation… well… that is a story for another day.

Understanding DCOM

The most important thing with DCOM is that you understand the basics really well. I guess this is true for most things, but unlike other things like .NET, DCOM does not cut you any slack, and does not provide you with tons of helpful diagnostic information.

When I started with DCOM a couple of years ago, I first read ‘Essential COM’ by Don Box. This is really the most excellent book on COM ever.

Of course, this information becomes vitally important in the real world when you have to debug a problem, because no tool is going to tell you anything useful with DCOM related crashes.

Understanding ATL

In the real world (unless maintaining legacy code) you use ATL to write COM clients or servers. Personally, I don’t like books that just tell me how to use technology XYZ. I like to understand the ‘why’ and ‘how’ too.

This is where I have to give big thanks to ‘ATL Internals, Second Edition’. It is one of the very few books on ATL that were written for VS2005, and it is still valid for VS2008. It is also an extremely well written book.

If you are starting with ATL (or if you want to know how it works) buy this book. This book is very, very good, and will teach you a lot.

What happens next?

Over the next weeks / months I will probably write a couple of articles describing how to do specific things with ATL. One article that is nearly finished is about implementing and using IEnumString using ATL.

Not rocket science by any measure, but ATL is severely lacking in the area of easy-to-find, up-to-date and to-the-point examples. So I figured that there is still value in writing about the ancient art of circles and lollipops, even if many developers probably think COM and ATL are no longer worth bothering with.

Oh and I released some win32 DCOM demo projects some time ago. They were interesting for me because making them taught me a lot about the raw DCOM stuff that you normally don’t see with ATL. But I’ve always found that it is always useful to know what makes the motor hum.

So while they are not terribly useful for practical purposes, they are well documented and they might be useful if you are working your way through ‘Essential COM’