Cluebat-man to the rescue

VMWare Server 2 Console WTF

vanDooren — Wed, 19 Aug 2009 09:48:00 GMT

I've been using VMWare Server 2 on a Vista 64 host to simulate a network environment for testing our operating procedures. Everything was working fine before I got on holiday. Now I am back for 2 weeks, and suddenly, VMWare thorws a wobbler.

Using the server console, I can start and stop local virtual machines, but whenever I try to connect to the console, I got the following error: 'Error opening the remote virtual machine machinename:8333\16:
An unexplained error occured.'

Brilliant. An error message that is as useless as a bookshelf made from mashed potatoes, that turns up without any changes to the system. And of course, googling did turn up many people with the same problem, but noone with a solution. Eventually, I found someone who suggested a solution for generic connection problems:

On the host computer you need to access the network properties for vmnet1.
find Internet protocol tcp/ip (v4) click properties
in the window use following DNS server address enter the static ip address displayed or enter the ip address 127.0.0.1
hit ok
hit ok again
done ! restart the machine to verify working

I have no idea why this is suddenly necessary, because the machine name is the local name and I can ping it without a problem. And VMWare consoles used to work just fine before my holiday, so I am really a bit annoyed with this. Still, the above fix worked like a charm so I am not going to complain too much.

Long time no see

vanDooren — Mon, 17 Aug 2009 06:53:00 GMT

It's been a while since I blogged here. Lack of inspiration, lack of time, lack of interest... Over the past couple of months, I've been occupied with other stuff.

The guest bathroom needed finishing, and my wife had a number of creative ideas that looked great but also needed a lot of time. I have started doing martial arts again, and I have been practising 2 times per week, so that is another 2 evenings no available for doing tech stuff. And the little time that I had left, I didn;t feel like programming.

But now I am back. I have started participating in the MSDN forums again, and I hope to write some more articles on ATL in the near future, because ATL is just cool.

My registration for tech-ed in Berlin has also been approved last week, so I will be blogging about that in november. Last year I didn't go, so most of the content should be new to me. Too bad they switched from Barcelona to Berlin. Spain can still be very nice in November. Then again, I won't miss the olive oil.

Oh and in case anyone was wondering: I was re-awarded in July, so I'll be an MVP for another year.

Practical ATL: Solving the race condition in CAtlExeModuleT

vanDooren — Wed, 15 Apr 2009 12:40:00 GMT

When I was spelunking through the ATL header for my previous articles (the class object plumbing) I discovered that there is a serious race condition in the server lifetime management.

I contacted the Microsoft C++ folks, and someone from the libraries group told me that this problem has already been fixed in the next release of Visual C++.

But since the issue still exists in all current ATL projects, I decided to write something about it and publish the fix which will circumvent the problem in the recent versions of ATL (7.0 and upwards if I am correct).

The problem

The module lock count of an ATL COM server is managed in the CAtlExeModuleT<T> class. It is this lock count that keeps the server alive as long as it is nonzero.

The CAtlExeModuleT<T> class manages its reference count via the CComGlobalsThreadModel class increment and decrement methods. CComGlobalsThreadModel is a typedef for CComMultiThreadModel so these methods map to InterlockedIncrement and InterlockedDecrement

The last module lock is released if
- the class object itself has no more external connections (IExternalConnection), and
- there are no more object instances.

As soon as the final module lock is released (module refcount is 0), the shutdown of the server is initiated, which will happen through either a WM_QUIT message or a win32 event object. In both cases there is a time window between the triggering of the shutdown, and the actual call to CoRevokeClassObject. This time window can be several seconds.

This time window causes a race condition between the server (which is already dying at that point) trying to revoke the class object, and a hypothetical client, requesting an interface pointer to the class object. Depending on how the race turns out, the client might end up with an S_OK HRESULT from CoGetClassObject, and an invalid interface pointer. This will cause a critical problem in the client, and may cause an application crash.

The solution

The proper way to manage the server lifetime is to use CoAddRefServerProcess and CoReleaseRefServerProcess to manage the module lockcount. Those functions will (according to MSDN http://msdn.microsoft.com/en-us/library/aa910552.aspx) suspend all class objects when the reference count drops to 0, in an atomic operation.

In that case, the server starts dying AFTER all class objects are suspended. Any incoming client request for the class object will spawn a new server process after the dying server has finished revoking its class objects.

The fix

The code for fixing this is simple. I simply created a new class that inherits from CAtlExeModuleT<T> and then overrides the Lock and Unlock methods.

template <typename T>

class CAtlExeModuleT_NoRace : public CAtlExeModuleT<T>

{

public:

typedef CAtlExeModuleT<T> base;

#if _ATL_VER < 0x0A00

LONG Lock(void)

{

CoAddRefServerProcess();

return base::Lock();

}

LONG Unlock(void)

{

CoReleaseServerProcess();

return base::Unlock();

}

#else

//removed compiler messages

#endif

};

I use CoxxxxServerProcess functions to explicitly manage the class object registration, and then I delegate to the base class which maintains its own reference count.

The reason I do that is that the base::Unlock function is the one which triggers the module shutdown, and I don’t want to mess with that. And Unlock works only when paired with Lock, so the only thing I really need to do is to call the CoxxxxServerProcess routines to prevent possible activation requests. The rest is still done by CAtlExeModuleT<T>.

I put a compiler condition around my implementation of the new Lock and Unlock methods, because they are only needed for current versions of ATL. The next version of ATL won’t have this problem, so it would be dangerous to meddle with the reference counting, especially since I cannot predict what side effects this may have.

The conditional compilation allows you to safely upgrade the project to the next version of ATL. The compiler messages which show up in the compiler output inform you that it is no longer needed to inherit from CAtlExeModuleT_NoRace as shown below:

class CStuff_ServerModule : public CAtlExeModuleT_NoRace< CStuff_ServerModule >

{

public :

DECLARE_LIBID(LIBID_Stuff_ServerLib)

DECLARE_REGISTRY_APPID_RESOURCEID(IDR_STUFF_SERVER, "{597B5DFF-DACE-42B0-9E49-54A09C10B2BB}")

};

The only thing you have to do to use my new module class is to include the header and change CAtlExeModuleT to CAtlExeModuleT_NoRace.

And if you upgrade the project to VC10, then you can simply change it back as indicated by the compiler output messages that are printed out when the compiler parses my class.

Conclusion

I was surprised that this bug was still in the ATL headers after all these years. When I contacted the VC++ team I was half expecting them to tell me that I had missed something. This issue is what caused CoAddRefServerProcess to be created after all, over a decade ago, so a COM programmer implementing lifetime management should be aware of it.

However, the modular and hierarchical nature of ATL made it easy to fix this problem, so I guess that is a plus for ATL. And mistakes happen. It is always a bit painful if it happens in widely used libraries, but I too have had my share of whoopsies over the years. It is sad but it happens.

The header file with the fix is attached to this blog post under the MIT license.

Could not create NTDS settings on domain controller...

vanDooren — Tue, 14 Apr 2009 16:24:00 GMT

Could not create NTDS settings yadayadayada on domain controller CN=yadayadayada. The RPC server is unavailable.

This was the error message greeting me (with some meaningful text instead of the yadayadayada of course :)) when I tried to add a backup domain controller to the domain of our test and development network. It was the first time I encountered an error at the DC promotion stage.

I checked the usual things (network connections, privileges, etc) but nothing jumped out at me. Now, if an error dialog mentions RPC, then the usual error is either a DNS error, or a DNS server that has not yet refreshed its zone information.

So I opened the DNS config, and discovered that only 1 NIC of our the new DC was registered in DNS. Our networks are multihomed (3 networks in parallel) and yet only 1 address was registered for the NIC. This was a bit odd. There should be 2 (one network has no DNS on purpose).

A quick ping revealed that one of the NICs had no connectivity. It turned out that the primary DC was a bad network connection, due to some wiggling with the cables. Plugging it all the way and then re-registereing the NICs of the new DC with DNS solved the problem.

I still don't understand why we got that problem, because the disconnected NIC was last in the binding order, so the new DC should have used the available connection anyway. My guess is that it retrieved the address via DNS, and got the address of the NIC that was disconnected. Ah well. Live and learn.

The kerberos client received a KRB_AP_ERR_MODIFIED error

vanDooren — Thu, 02 Apr 2009 10:43:00 GMT

This is what I got in the event logs yesterday afternoon:

Event Type: Error
Event Source: Kerberos
Event Category: None
Event ID: 4
Computer: SE-SMURF01
Description:
The kerberos client received a KRB_AP_ERR_MODIFIED error from the server PC-BLABLA09$. The target name used was . This indicates that the password used to encrypt the kerberos service ticket is different than that on the target server. Commonly, this is due to identically named machine accounts in the target realm (FOO.BAR.STRIPE.LOCAL), and the client realm. Please contact your system administrator.

Event Type: Error
Event Source: Kerberos
Event Category: None
Event ID: 4
Computer: SE-SMURF01
Description:
The kerberos client received a KRB_AP_ERR_MODIFIED error from the server PC-BLA09$. The target name used was RPCSS/PC-BLA10. This indicates that the password used to encrypt the kerberos service ticket is different than that on the target server. Commonly, this is due to identically named machine accounts in the target realm (FOO.BAR.STRIPE.LOCAL), and the client realm. Please contact your system administrator.

I had replaced those machines a week ago, and everything seemed to work fine. So I didn't understand why these errors were suddenly popping up. The applications running on those computers where throwing a wobbler as well. Some googling later I found 2 remarks that were useful.

The first one was that someone fixed it by taking the computer out of the domain, renaming it, changing the SID, and changing the IP address. While this is overkill on the scale of killing a mouse with a thermonuclear weapon, it pointed in the direction of a network level problem.

The second remark was by a Microsoft employee who explained that DNS misconfiguration can be the source of problems like this. If kerberos thinks it is communicating with pcA it encrypts the kerb ticket with the password of pcA. but if the ticket then ends up on pcB because of the DNS mismatch, the above events will be logged.

At that moment I realized that I had changed the IP address of an adapter on PC-BLA10 because it conflicted with PC-BLA09. The reason everything worked fine initially was because that port had been left disconnected until 2 days ago when I configured the correct IP address. The conflict was resolved and the DNS information was updated, but that didn't mean that the DNS caches were up to date. So I cleared the DNS cache of the DNS server, and used ipconfig /flushdns to clear the resolver cache on the domain controller and PC-BLA10, and the problem disappeared.

Scheduled task would not run

vanDooren — Wed, 25 Feb 2009 08:13:00 GMT

The scheduled task for running ntbackup on our fileserver would not run anymore. It would stop with a return code of 0x80, and nothing worthwhile in the log file or event log. It ran fine for months, apart from the occasional hickup such as having no more free tapes in the loader, or a tape not being imported correctly.

I tried running the backup batch file manually, and that worked fine. I ran it under the dedicated backup account, and that worked fine too. But it wouldn't run automatically, and starting the task manually when logged in didn't work either. I couldn't find any useful debugging info so I turned to google. I found several people who has similar problems, but most had to do with a bug in the HID service which somehow interfered with ntbackup. I looked around some more and finally I found something that ended up giving me the final clue.

I opened up the task manager and noticed that there were 21 instances of xcopy still running, 23 instances of psexec, and 3 instances of ntbackup. These were probably left over from some tests I had done with the backup script of one of the new servers (a backup domain controller) which uses the 3 aforementioned programs.

After killing these dead processes I tried to run the backup task again, and everything worked fine this time. In my naivety, I had assumed that ending a task would end the processes it had spawned, but that seems to have been a mistake. Ah well, everything is working again, and now that I know the cause, I can prevent this from happening again.

This is why I try to understand a problem, rather than just rebooting the machine. A reboot would have fixed the problem in less time then I needed to figure this out. But then I wouldn't have known what happened (I hate that) and I wouldn't be able to prevent this from happening again. I would have been stuck in a regular but unknown problem -> reboot cycle.

Practial ATL: Implementing a non-standard class object

vanDooren — Fri, 20 Feb 2009 21:03:00 GMT

In my previous article in this series, I explained how class objects are associated with COM objects, and how it all fits together.

Armed with that knowledge, I can now demonstrate how to provide a custom class object that will allow us to support parameterized construction.

If you haven’t already read my previous article on class objects, please do so first because I am going to assume you did J

The non-standard class object interface: IStuffCreator

A class object can have any interface you want, just like the COM object we have created earlier. It is just another COM object after all. My custom interface looks like this:

[
      object,
      uuid(6DD69CDB-3128-432b-B335-773A287E6F06),
      oleautomation,
      helpstring("IStuffCreator Interface"),
      pointer_default(unique)
]
interface IStuffCreator : IUnknown {
[id(1), helpstring("method MakeMeAStuff")]
    HRESULT MakeMeAStuff(
      [in] BSTR name,
      [in] REFIID riid,
      [out, iid_is(riid)] IUnknown** ppStuff);
};

The bits between the square brackets are the attributes that are applied to this interface. ‘Object’ specifies that it is a COM interface. The uuid specifies the GUID that identifies the interface. ‘oleautomation’ specifies that the arguments being used are all automation compatible. The meaning of ‘helpstring’ is pretty obvious, and pointer_default specifies the assumptions that the marshaller can make about pointers.

If you want to know the details about it, then have a look in MSDN for MIDL attributes.

The interface itself is very simple. Instead of deriving from IDispatch like the IStuff interface, this interface derives from IUnknown. Deriving from IDispatch would only complicate matters without any benefit.

The interface has only one method, which is responsible for handing out new stuffs.

The parameter that will be used for constructing the CStuff instance itself is the ‘name’ parameter. If the CStuff would need multiple parameters, they would have to be part of the parameter list so that they can be used at the time of construction. The riid and ppStuff parameters have the same meaning as those of CoCreateInstance.

The declaration of CStuffCreator

If we want to implement this interface, we add a new C++ class with the name ‘CStuffCreator’ to our project, and implement our IStuffCreator interface

class CStuffCreator:
public IStuffCreator,
public CComObjectRootEx<CComGlobalsThreadModel>
{
public:

CStuffCreator(void){}
virtual ~CStuffCreator(void){}

BEGIN_COM_MAP(CStuffCreator)
    COM_INTERFACE_ENTRY(IStuffCreator)
END_COM_MAP()

//IStuffCreator methods
HRESULT STDMETHODCALLTYPE MakeMeAStuff(
    BSTR name,
    REFIID riid,
    IUnknown **ppStuff);

// helper method
void SetVoid(void* pv);
_ATL_CREATORFUNC* m_pfnCreateInstance;
};

As you can see, it is pretty simple. The CComObjectRootEx base class helps with reference counting, and is the same that is used by the other COM objects in our project.

We add IStuffCreator to the interface map using the ATL macros, and then we simply declare the single MakeMeAStuff method that we need to implement.

The SetVoid method and the function pointer are part of the default C++ interface for class objects in ATL, and are explained in my previous article.

The implementation

The implementation of CStuffCreator is even simpler than the declaration:

HRESULT STDMETHODCALLTYPE CStuffCreator::MakeMeAStuff(

BSTR name,

REFIID riid,

IUnknown **ppStuff)

{

CComPtr<IUnknown> newStuff;

HRESULT hr = m_pfnCreateInstance(

NULL, __uuidof(IUnknown), (void**)&newStuff);

if(FAILED(hr))

return hr;

return newStuff->QueryInterface(riid, (void**)ppStuff);

}

void CStuffCreator::SetVoid(void* pv)

{

m_pfnCreateInstance = (_ATL_CREATORFUNC*)pv;

}

Ignore the ‘name’ parameter for now. That will be used when we are going to initialize the new instance. The bare functionality of our class object is to use the function pointer to create a new instance. I could have done this in 1 step instead of 2 (the QueryInterface isn’t necessary at this point) but this way I can slide in some additional functionality later on.

And you can see now that the class object will receive the creator function pointer via the ‘SetVoid’ method. This mechanism was explained in my previous article.

Parameterized construction

The point of using a non-standard class object in this demonstration was to be able to create objects that need some sort of initialization before the client has access to them. Normally, ATL supports only default constructors using, because IClassFactory is a generic interface.

For parameterized constructors, we are going to have to do something special.

There are essentially 2 ways we can do this. The first is to change CStuff to have a parameterized constructor. But ATL can’t handle that, and it would mean we have to abandon the incredible convenience of being able to use the ATL::Creator<T> class, ATL::CComObject class, and other helper classes that take the pain out of COM.

Don Box wrote an article about parameterized construction, many years ago, and this was the approach he took. Of course, ATL was much simpler in those days, and abandoning it was perhaps less of an inconvenience.

The approach I am taking is much simpler: I cheat!

That’s right. I cheat. I am not going to implement parameterized constructors. I am going to fake it. But from the client’s point of view, it’s going to look exactly the same, so he won’t mind. And from the server’s point of view, it will be much simpler to implement, so everybody wins.

The second approach to implementing parameterized construction is to implement an internal interface on CStuff, which will be used for initializing it. The new CStuff instance will not be accessible to anyone until the first interface pointer is handed out by the class object. And we are the ones implementing the class interface, so we can make sure that the new instance is initialized properly before it is released into the wild.

The initialization interface

The interface for initializing a CStuff is pretty simple (I left out the header for clarity):

interface IStuffInit : IUnknown {

[id(1), helpstring("method InitStuffInstance")]

HRESULT InitStuffInstance(

[in] BSTR Name);

};

Our CStuff instances are supposed to have a name, so that’s what needs to be in the init interface. If they would also need an address, a hat, and a pair of pants, we would have to put those in the parameter list of the InitStuffInstance method as well.

Implementing the initialization interface

Implementing the interface in CStuff is easy I highlighted the changes that have to be made to the declaration.

// CStuff

class ATL_NO_VTABLE CStuff :

public CComObjectRootEx<CComMultiThreadModel>,

public CComCoClass<CStuff, &CLSID_Stuff>,

public IDispatchImpl<IStuff, &IID_IStuff, &LIBID_Stuff_ServerLib, /*wMajor =*/ 1, /*wMinor =*/ 0>,

public IStuffInit

{

private:

CComBSTR m_Name;

public:

DECLARE_CLASSFACTORY_EX(CStuffCreator)

// removed irrelevant stuff

BEGIN_COM_MAP(CStuff)

COM_INTERFACE_ENTRY(IStuff)

COM_INTERFACE_ENTRY(IDispatch)

COM_INTERFACE_ENTRY(IStuffInit)

END_COM_MAP()

// removed irrelevant stuff

HRESULT STDMETHODCALLTYPE InitStuffInstance(BSTR Name);

};

First we add IStuffInit to the inheritance list, we add it to the COM map, and then we add the declaration for InitStuffInstance.

Of course, the name has to be part of the objects state, so we add a CComBSTR variable to hold it.

And finally, we use the DECLARE_CLASSFACTORY_EX macro to change the class object for CStuff from CComClassFactory to CStuffCreator.

The actual implementation of the interface consists of the simple code

STDMETHODIMP CStuff::InitStuffInstance(BSTR Name)

{

m_Name = Name;

return S_OK;

}

Note the ease with which CComBSTR allows us to perform simple string operations.

Initializing the new CStuff object

Now that there is a dedicated function to handle the initialization, the changes to CStuffCreator are trivial:

HRESULT STDMETHODCALLTYPE CStuffCreator::MakeMeAStuff(

BSTR name,

REFIID riid,

IUnknown **ppStuff)

{

CComPtr<IStuffInit> newStuff;

HRESULT hr = m_pfnCreateInstance(

NULL,__uuidof(IStuffInit), (void**)&newStuff);

if(FAILED(hr))

return hr;

hr = newStuff->InitStuffInstance(name);

if(FAILED(hr))

return hr;

return newStuff->QueryInterface(riid, (void**)ppStuff);

}

What about double initialization

By now you might be wondering: But what if the client application retrieves this interface and performs another initialization?

However, this will not be possible.

It’s true that I put the interface in our IDL file, and that I implemented it in the CStuff object. But I did not put that interface in the library declaration, nor did I put it in the interface list of the coclass.

So for all practical purposes, this interface is not accessible to the client application. If you are really paranoid, you could argue that an attacker might know the interface GUID, and guess the interface definition, and with some trial and error make it work.

Even then, the fix is quite simple. You can add a boolean to your class that indicates if the object was initialized or not. Set it to false in the constructor, and then the InitStuffInstance can either perform initialization if it is false and set it to true, or return E_FAIL if it is already true.

You don’t even have to care about race conditions, because there aren’t any. Only the class object can perform the real initialization. And since no one else has an interface pointer yet, there can’t be a race.

The client side

Creating a new CStuff has changed a little bit, because now we can’t use CoCreateInstance anymore. So construction is now done in 2 stages:

CComPtr<IStuffCreator> stuffClass;

hr = CoGetClassObject(

__uuidof(Stuff), CLSCTX_ALL, NULL,

__uuidof(IStuffCreator), (void**)&stuffClass);

if(FAILED(hr))

return hr;

//make a new stuff

CComBSTR stuffName = L"Kato";

CComPtr<IStuff> newStuffUnk;

hr = stuffClass->MakeMeAStuff(

stuffName.m_str,

(GUID*)&__uuidof(IStuff),

(IUnknown**)&newStuffUnk);

if(FAILED(hr))

return hr;

It’s still very simple though. First we get the class object, and then we use the IStuffCreator interface to create the new CStuff.

Server lifetime management

Everything works, hip hip hurray… until you destroy the first CStuff instance, and then try to create another one. That is where it all goes pear shaped.

The problem is that we haven’t taken care of managing the lifetime of the server (the exe). The key issue here is the module lock reference count.

Each COM module (exe or dll) has a global reference count to keep track of how many objects are ‘alive’. COM objects exist physically inside a DLL or EXE. So the EXE has to stay alive as long as there are living objects inside. Conversely, as soon as there are no living objects anymore, the EXE should shut down.

What happens in our example is that CStuff raises the module ref count when it is created. And when the CStuff instance destroys itself, it lowers the module ref count. If you remember correctly, the class object itself uses CComObjectNoLock as its COM core. This means that its existence does not influence the module lifetime.

So when the CStuff instance destroys itself and lowers the module ref count, the ref count reaches 0 and the server EXE shuts down.

The client is unaware of this and tries to use the class object again. It still thinks it has a valid interface pointer, but the underlying class object is long gone and there will be a critical error in the client application.

You might say ‘well duh, then don’t use CComObjectNoLock, doofus’ but that has the opposite effect. The COM runtime will always have the interface pointer that was used for registering the class object with CoRegisterClassObject. This means that the class object would not destroy as long as the server lives. And the server would keep living as long as the class object was around. This is clearly not the answer.

IExternalConnection to the rescue

The key to solving this problem is a rarely used standard interface. IExternalConnection is an interface that can be used by an object to track how many external connections there are to itself. This is basically an alternative to the standard object reference count.

There are 2 methods in this interface:

STDMETHODIMP_(DWORD) AddConnection(

DWORD extconn, DWORD dwReserved);

STDMETHODIMP_(DWORD) ReleaseConnection(

DWORD extconn,

DWORD dwReserved,

BOOL bLastUnlockReleases);

If a client requests any class object, the COM runtime will retrieve it. But before the interface pointer is released to the client, it will request some interfaces itself for housekeeping purposes. One of those interfaces is IExternalConnection.

If a client requests an interface pointer to a class object, the marshaller will call AddConnection when it hands out the pointer, and ReleaseConnection when the pointer is destroyed.

So all we have to do is to make sure that a call to AddConnection will increment the module lock count, and a call to ReleaseConnection releases the module lock count.

Implementing IExternalConnection

If you look in MSDN, you won’t find it. It is undocumented, but the ATL headers contain an implementation for IExternalConnection. It’s called IExternalConnectionImpl<T>.

To use it, we only have to change CStuffCreator so that it inherits from it, and add it to the COM interface map.

class CStuffCreator:

public IStuffCreator,

public IExternalConnectionImpl<CStuffCreator>,

public CComObjectRootEx<CComGlobalsThreadModel>

{

public:

//removed irrelevant stuff

BEGIN_COM_MAP(CStuffCreator)

COM_INTERFACE_ENTRY(IStuffCreator)

COM_INTERFACE_ENTRY(IExternalConnection)

END_COM_MAP()

//IExeternalConnectionImpl<T> extension

void STDMETHODCALLTYPE OnReleaseConnection(

bool bThisIsLastUnlock, bool bLastUnlockReleases);

void STDMETHODCALLTYPE OnAddConnection(

bool bThisIsFirstLock);

//removed irrelevant stuff

};

That is really all there is to it. The default implementation does not modify the module lock count. But it has 2 extension points that we can override. So that’s what we do to make sure the module lock count is managed properly.

void STDMETHODCALLTYPE CStuffCreator::OnAddConnection(

bool bThisIsFirstLock)

{

_pAtlModule->Lock();

}

void STDMETHODCALLTYPE CStuffCreator::OnReleaseConnection(

bool bThisIsLastUnlock, bool bLastUnlockReleases)

{

_pAtlModule->Unlock();

IExternalConnectionImpl<CStuffCreator>::OnReleaseConnection(

bThisIsLastUnlock, bLastUnlockReleases);

}

The only special thing that still needs mentioning is that OnReleaseConnection delegates back to the default implementation in IExternalConnectionImpl. It does this because there is something that needs to be done when the last external connection is closed.

void OnReleaseConnection(

bool bThisIsLastUnlock, bool bLastUnlockReleases)

{

if (bThisIsLastUnlock && bLastUnlockReleases)

CoDisconnectObject(static_cast<T*>(this)->GetUnknown(), 0);

}

If the last connection closes, the class object needs to call CoDisConnectObject to destroy its marshalling stub. Now I could copy paste this code into my implementation of OnReleaseConnection, but it is simpler to just delegate back to the base class for stuff that needs doing anyway.

The result

Now that we have provided this implementation of IExternalConnection, the server EXE will stay alive until all CStuff instances are gone, and noone is using the class object anymore. Only then will the EXE terminate.

CComClassFactory revisited

If you look at the interface implemented by CComClassFactory, you’ll notice that IExternalConnection is missing. And yet, server lifetime of CComClassFactory is managed correctly.

The reason for this is that the COM runtime treats IClassFactory class objects differently from other class objects. It will check if the class object supports IClassFactory. And if it does, it calls IClassFactory::LockServer to indicate that someone is using the class object. And it unlocks the class object when the class object reference is released again.

In a way, it uses the IClassFactory in almost the same way as it would use IExternalConnection. And in a way it is too bad that this functionality was included in IClassFactory directly, instead of making IClassFactory inherit from IExternalConnection. The reasons for this are lost in the mists of time. But whatever the reasons, this is why IClassFactory doesn’t need IExternalConnection.

Conclusion

As you can see, implementing a non standard class object is not so hard. As long as you have a basic understanding of the underlying COM principles, ATL takes care of all the dirty work.

And by using these techniques, we can implement COM objects that need initialization before they can safely be used by the client. The source code for this article is available under the MIT license as usual.

And again I’d like to add a thank you for Microsoft’s Tim Springfield and Amit Mohindra

Practical ATL: Understanding the class object

vanDooren — Tue, 10 Feb 2009 07:46:00 GMT

In my previous article in this series, I explained how to create a simple COM server and implement a method that returns an enumerator object.

It was all done very easily, but the ATL wizards hide some of the things that you really ought to know for my next article. For starters: where is the class object?

The class object is the COM object responsible for creating new instances of the CStuff COM object that I demonstrated in a previous article. But we didn’t have to write any code for it, nor is it even visible… so what’s up with that?

Some more on class objects

The class object of a COM server is the object that is actually responsible handing out instances of the COM object. The class object is a COM object itself, but only 1 instance of it normally exists in a module (DLL or EXE) per defined COM object type.

For example, the ‘Stuff’ COM object that I implemented in my previous article uses the standard class factory with IClassFactory as the interface for handing out new CStuff instances.

The most important method in IClassFactory is ‘CreateInstance’ which simply creates a new instance of the COM object and then passes an interface pointer back to the caller.

Normally, this process is invisible to the client application, because ‘CoCreateInstance’ uses IClassFactory behind the scenes so that the client doesn’t have to bother with it.

First it calls ‘CoGetClassObject’ to get the class object for the requested COM object, and then it uses the IClassFactory interface of that class object to create a new instance. That instance is then passed to the caller, as an interface pointer of the type that was requested in the call to ‘CoCreateInstance’.

I think it is safe to say that ‘IClassFactory’ (or one of its relatives) is the interface on the class object of 99.99% of all COM objects that have a class object.

The default class factory

By default, CStuff uses CComClassFactory as its default class object. But looking at the declaration of CStuff, it is not immediately apparent how it does this.

CStuff derives from CCoClass<T>, which has the macro DECLARE_CLASSFACTORY in its implementation.

For EXE servers, this macro expands to DECLARE_CLASSFACTORY_EX(ATL::CComClassFactory), which in turn expands to

typedef ATL::CComCreator< ATL::CComObjectNoLock< ATL::CComClassFactory > > _ClassFactoryCreatorClass;

DLL servers use CComObjectCached instead of CComObjectNoLock, but I don’t cover DLL servers in this article.

As you can remember from the first article that featured CStuff, the C++ COM classes are missing the IUnknown core by default, because the implementation of that core depends on how we want to the object to behave.

This means we have to shove our class into a COM object to make it whole. For class objects, the CComObjectNoLock is most appropriate, because the lifetime of the object should not contribute to the lifetime of the server executable. Otherwise there would be a deadlock.

The server would start, and create the class object. And then the server would continue to live until the class object was destroyed. But since the COM runtime will always have a reference to the class object, the reference count would never reach zero, the class object would never be destroyed, and the server would never be able to shut down.

And finally, this class definition is shoved into a creator object which will help us by creating a properly initialized instance of the class object itself via its static CreateInstance function.

Associating the COM object C++ class with the class factory

So the COM object C++ class has a typedef that identifies the class object type. But how does the executable use this fact?.

This is where things get a little hairy, and possibly start to make ominous squishy sounds J

After the class declaration of CStuff, there is the innocuous line OBJECT_ENTRY_AUTO(__uuidof(Stuff), CStuff)

The OBJECT_ENTRY_AUTO is a macro that expands to this:

#define OBJECT_ENTRY_AUTO(clsid, class) \
      __declspec(selectany) ATL::_ATL_OBJMAP_ENTRY __objMap_##class = {&clsid, class::UpdateRegistry, class::_ClassFactoryCreatorClass::CreateInstance, class::_CreatorClass::CreateInstance, NULL, 0, class::GetObjectDescription, class::GetCategoryMap, class::ObjectMain }; \
      extern "C" __declspec(allocate("ATL$__m")) __declspec(selectany) ATL::_ATL_OBJMAP_ENTRY* const __pobjMap_##class = &__objMap_##class; \
      OBJECT_ENTRY_PRAGMA(class)

Don’t worry if the sight of this code makes you nauseous. That’s normal. I highlighted the important parts

What it does is simple. It’s just how it does it that is ugly. It creates a map entry for the ATL innards of our server executable. In this map entry, it associates the static method for creating the class object with the static method for creating an object instance.

Creating the class object

Now that the class object and the COM object are associated with each other, the next bit of code in the ATL headers is where the magic happens:

_ATL_OBJMAP_ENTRY* pEntry;
if (m_pObjMap != NULL)
{
    pEntry = m_pObjMap;
    while (pEntry->pclsid != NULL && hr == S_OK)
    {
      hr = pEntry->RegisterClassObject(dwClsContext, dwFlags);
      pEntry++;
    }
}

When the ATL module initializes, it will iterate over all elements in the object map and register each one. And this registration process entails the following:

    IUnknown* p = NULL;
    if (pfnGetClassObject == NULL)
      return S_OK;
    HRESULT hRes = pfnGetClassObject(
                         pfnCreateInstance,
                         __uuidof(IUnknown),
                         (LPVOID*) &p);
    if (SUCCEEDED(hRes))
      hRes = CoRegisterClassObject(
               *pclsid, p, dwClsContext, dwFlags, &dwRegister);
    if (p != NULL)
      p->Release();
    return hRes;

If you do a bit of spelunking in the headers, you’ll see that pfnGetClassObject is a function pointer that maps to _ClassFactoryCreatorClass::CreateInstance. So this code simply creates a new class object and registers it.

Again, this is specific to EXE servers. In DLL servers, the class object is created on demand in the DllGetClassObject function, which is called by CoGetClassObject if the COM server is a DLL.

Creating CStuff instances

If you look at the _ClassFactoryCreatorClass::CreateInstance, you’ll see that its first parameter is a void *. And the value that is passed in this parameter by the ATL module is the address of the static method for creating new object instances.

_ClassFactoryCreatorClass::CreateInstance simply creates a new class object ‘p’, and then calls p->SetVoid(pv); to pass the aforementioned function pointer to the class object.

And the SetVoid method is no more complicated then this:

void SetVoid(void* pv)
{
m_pfnCreateInstance = (_ATL_CREATORFUNC*)pv;
}
_ATL_CREATORFUNC* m_pfnCreateInstance;

I guess the original developer hadn’t had his morning coffee yet when he came up with this name. It manages to be 100% factually correct, while being 100% opaque at the same time. But it does the job.

When the class object has to create a new instance, it simply does the following:

STDMETHOD(CreateInstance)(LPUNKNOWN pUnkOuter, REFIID riid, void** ppvObj)
{
    // snip irrelevant code
        hRes = m_pfnCreateInstance(pUnkOuter, riid, ppvObj);
    // snip irrelevant code
    return hRes;
}

Providing a new class factory

Now that you understand how the class object plumbing is hooked up, it is easy to see how you can implement your own class object.

Remember how DECLARE_CLASSFACTORY_EX maps a class object to the _ClassFactoryCreatorClass typedef?

In C++, a typedef in a derived class can override a typedef in a base class. So even though the CCoClass<T> base class says that the _ClassFactoryCreatorClass maps to ATL::CComCreator< ATL::CComObjectNoLock< ATL::CComClassFactory > >,

We can override this by simply putting DECLARE_CLASSFACTORY_EX(CStuffCreator) In the declaration for CStuff. That’s really all there is to it. The macro magic takes care of the rest.

Conclusion

If changing the class object for a COM server is as trivial as adding a single line to your COM server C++ class, then why did I just write 4 pages of stuff?

I wrote all this because if I tell someone to use a macro to do something non-trivial, I want him or her to understand what is going on. There are already far too many programmers on this planet who do things without understanding what it is they do, or how it works. And I am not going to add some more by glossing over the details.

Of course, judging by the number of visitors on this corner of the internet that I call ‘home’, I stand little risk of doing so no matter how crappy I would write J

This article was originally part of my article on implementing a custom class object. But after I spent more than 3 pages on the previous explanation, I thought it would be best to put it in a separate article.

A big thank-you goes to Jim Springfield of the Architects team within the developers division (I think) of Microsoft for taking the time to review this article for correctness, and to Amit Mohindra for helping my questions and articles reach the right people.

And you can find the demo project attached to this article under the MIT license.

Practical ATL: recognizing marshalling problems

vanDooren — Tue, 27 Jan 2009 11:30:00 GMT

Before I publish my next article, I should explain something else about the black magic aspect of ATL / COM: Marshalling.

The Stuff server we made in the previous article is an out-of-proc server. This means that the arguments being passed to / from a stuff instance have to be somehow moved back and forth between different process contexts (or different threading apartments).

In the case of IStuff, we don’t really have to bother, because IStuff derives from IDispatch. And in that case, there is a marshaller (aptly named Universal Marshaller) that can be used to do all this for you.

If IStuff wouldn’t derive from IDispatch, then we would need to provide our own proxy – stub interface that knows how to marshal the interface. With ATL, this is not difficult. You’ll notice that the Stuff solution of my previous article had a ‘Stuff_ServerPS’ project that I didn’t mention before.

That solution will build the proxy stub dll that will perform the marshaling. It is disabled by default, so you have to enable it in the build configuration before it is built. Also remember to rebuild the proxy, every time the IDL files of your project change.

So for the remainder of this article I will show what can happen if there is a marshalling problem. Don’t worry if it all seems ugly, gory, and incomprehensible. Normally you won’t have to deal with issues like this. But if you should ever do, you should at least recognize the signs that you have a marshaling problem.

Confused? It’s the marshaller

If you find yourself in a situation where very weird things happen, suspect a marshalling problem.

If you get the error code E_NOINTERFACE when you know for sure that your object implements it properly, suspect a marshalling problem.

If you get a crash while using an interface pointer that was properly created without getting a HRESULT that indicated a problem,… that’s right: you can bet it’s a marshaling problem.

Example

The article I am currently writing discusses implementing a non standard class object, for the purpose of parameterized construction of the Stuff object. Never mind for the moment about what any of that means.

For all practical purposes: it just means that I defined the following interface via copy / paste / edit:

[

      object,
      local,
      uuid(6DD69CDB-3128-432b-B335-773A287E6F06),
      dual,
      helpstring("IStuffCreator Interface"),
      pointer_default(unique)
]
interface IStuffCreator : IUnknown {
[id(1), helpstring("method MakeMeAStuff")]
    HRESULT MakeMeAStuff(
      [in] BSTR name,
      [in] REFIID riid,
      [out] void ** ppStuff);
};

There were 4 things wrong with this interface (showing just why you shouldn’t define IDL interfaces in a hurry). This has to be some sort of record.

‘local’ shouldn’t be there.
‘dual’ shouldn’t be there.
‘void’ should be ‘IUnknown’
‘ppStuff’ should be attributed with riid_is(riid).

So let’s have a look at the result of these different issues.

Issue 1: local

‘local’ simply shouldn’t be there. ‘local’ is a MIDL attribute that –when use in an interface header- causes MIDL not to generate stubs for that interface.

The effect on our interface is that the IStuff specific methods are not forwarded to the remote server.

You might wonder: why does it exist. The answer is: in case you want to write custom marshalling code, or in case the calls never have to be remoted. In that case you can have a local interface and a remote interface. And usage and marshalling could be optimized.

IUnknown is such an interface. But it has custom stub code to deal with things like remoting

The result of using ‘local’ on my IStuffCreator interface is that trying to call GetEnum on the returned IStuff interface pointer will result in a debug assertion: Run-Time Check Failure #0 - The value of ESP was not properly saved across a function call. This is usually a result of calling a function declared with one calling convention with a function pointer declared with a different calling convention.

And if you have this problem with COM, it is usually indeed an indication that you have a calling convention mismatch. In this case, it is a side effect of the fact that we are trying to execute a method that isn’t remoted.

Issue 2: dual

This one shouldn’t be here either. Dual indicates that the interface implements IDispatch, and this could have an effect on marshalling.

In this case, it worked just fine. Perhaps it is because of the fact that I use my object as a class object, or perhaps because my interface itself uses only arguments that are automation compatible, or perhaps because the stars are aligned in the proper configuration.

I don’t know. And anything you don’t know for certain will hurt you when dealing with DCOM. It has to be correct.

Issue 3: void

This one is interesting.

If I want to pass an interface pointer, it has to be passed as an IUnknown**, instead of a void**. This is because Universal Marshaller has no idea on how to marshal something which has only contextual meaning, nor does the IDL compiler know what you mean with it.

With this I mean that my server knows it will be an interface pointer, and my client will know it is an interface pointer, but the stuff in the middle doesn’t. It sees just a memory address.

I used void** because that is what QueryInterface itself uses. And what’s good enough for QI should be good enough for me. Except QI has custom code for marshalling its arguments across, which I don’t. So QI can use void** because both the proxy and the stub know what is going on.

So what happens when you use void**?

I request the class object like this:

hr = CoGetClassObject(
__uuidof(Stuff),

CLSCTX_ALL,

NULL,

__uuidof(IStuffCreator),

(void**)&stuffClass);

And the resulting hr is 0x80020008: Bad variable type. There is absolutely nothing wrong with this line of code. But because there is something in the interface itself that confuses the marshaller, it tells me to sod off when I want that specific interface.

Credit where it is due: I figured this one out thanks to someone whose name I can’t figure out J. But his blog post is here.

Issue 4: iid_is(riid)

This one is also a good one.

The iid_is informs the marshaller that riid points to the identifier which identifies the interface that is being marshaled.

This way, the marshaller knows how to marshal the interface that was specified with riid. Without that part, it would only marshal the IUnknown part of the interface, and not the IStuff specific part.

As a result, you can use the IUnknown part of the interface without a problem, but if you use the IStuff specific part, you will get an access violation, stack corruption or other interesting phenomenon.

Conclusion

A couple of things can be concluded at this point.

First of all: Marshaling problems are hard to debug, because the problems occur outside of your source code.

If you use any interfaces that don’t derive from IDispatch, then you have to enable the ‘xxxPS’ project in your solution so that the ProxyStub dll will be built and registered. And of course, this dll has to be part of your install procedure. Without this stub, you could get access violations, or you get an E_NOINTERFACE error if the marshaler doesn't know one of those interface exist.

As I have shown, you need to be careful and pay attention when you manually define COM interfaces. If you don’t, then you can have all sorts of hard to diagnose problems.

And if you ever find yourself in a situation where nothing makes sense, then suspect marshalling issues and have a long hard look at your IDL files.

File a bug report and make Visual Studio a better product

vanDooren — Thu, 22 Jan 2009 06:50:00 GMT

If you find a bug in Visual Studio, report it on http://connect.microsoft.com If the bug affects the version of VS that is in development at that time, there is a significant chance it will get fixed before the next release. Just this week I got confirmation that all 3 bugs that I filed in the last months have been resolved.

The first one is a nasty one. If you add an ATL project to an existing, non-empty solution, the ATL wizards will either hang the IDE or fail to work when adding an ATL object. I found a workaround for it, but it involved hand editing the solution file.

The second one is a small bug in the TR1 regex parser. No workaround, other than to restructure the regex.

And the last one was an error in the MSDN documentation about open modes with the fopen family, which counts as a bug because it triggered runtime errors.

All 3 have been fixed. I especially appreciate the first one. I am working on ATL projects, and I develop my COM server in the same solution as a couple of related non-ATL projects. So every time I work with ATL projects, I can have a warm and fuzzy feeling that because of me, VS is now a better product. :)

Practical ATL: Implementing an enumerator object

vanDooren — Wed, 21 Jan 2009 19:55:00 GMT

This is my first article about practical ATL examples, which I already mentioned here.

In this example I will create an ATL Server which implements 1 custom interface: IStuff. Hardly an inspiring name, but I couldn’t think of anything better before I had my first coffee.

This interface will have 1 method which will return an enumerator to the caller. Specifically, this enumerator will implement IEnumString.

Enumerators are part of the interface of many real-life COM servers, but creating your own is an arduous task. ATL makes this much easier, but an example makes things so much easier still.

Since client code will have to deal with this enumerator (and to have a demo app) I will also create a client application that creates an instance of the server, consumes the IStuff interface and then enumerate the items contained in the returned enumerator.

Before we get started

If you don’t own a copy of ‘ATL Internals, Second Edition’ yet, I strongly advise you to buy it. It is worth the money, and apart from leading you through all the interesting and gory stuff, it also starts by explaining the actual process of creating an ATL server through the IDE, as well as implementing methods, properties and events.

I am only going to cover that aspect very lightly. It should be detailed enough for anyone who has done C++ development using VS:

Create a new ATL Server project with project name ‘Stuff_Server’ and a solution name ‘Stuff’.
Choose the EXE project type and click ‘finish’
Add a new class to the project, and specify ATL Simple Object. Use ‘Stuff’ as the name of the object.
Switch to classview, and add a new method to the IStuff interface with the prototype ‘HRESULT GetEnum(IUnknown ** ppEnum). ppEnum should be marked as an ‘out’ parameter.

Build the project, and you should be ready for the actual work. The ATL wizards will have implemented all the IDL and DCOM stuff for you, and you only have to implement the functionality.

Finally, I always end by changing the project configuration to use static CRT and ATL runtimes instead of dynamic runtimes. This makes it infinitely more easy to distribute the resulting executables without having to bother with registration and installation of the runtime dlls and their manifests.

Creating the server

Now that the ATL wizards have generated the function skeleton for us, it is time to do something with it:

STDMETHODIMP CStuff::GetEnum(IUnknown** ppEnum)
{

}

The enumerator

Before we can do anything else, we have to think about the actual enumerator for a bit. In our case we implement IEnumString. This is an interface for enumerating string (who’d have thought, eh?).

Enum interfaces all look alike, and ATL helpfully has a stock implementation for generic enumerators that allows you to specify the Interface, GUID, datatype and data type copy traits as template parameters. In order to facilitate coding, we make a typedef for this class:

typedef CComEnum<

IEnumString,

&IID_IEnumString,

LPOLESTR,

_Copy<LPOLESTR> > CComEnumStringAbstract;

This will specialize the enumerator for our purposes. The use of the first 3 parameters is obvious. Don’t worry about the _Copy parameter. I will cover that later.

The enumerator itself is a normal COM object, so it needs lifetime management, interface managements, and all other things you really don’t want to care about for something so simple. At this point, CComEnumStringAbstract will still be an abstract class because it can’t decide how to do these things for you.

In order to make our enumerator a full fledged COM object, we have to shove it into a bare COM object. This is very easy:

typedef CComObject<CComEnumStringAbstract> CComEnumString;

And that is all there is to creating an enumerator! Anyone who has ever implemented one manually will agree that this is infinitely more convenient

Implementing GetEnum

The implementation for GetEnum is also fairly easy (comments removed for clarity)

STDMETHODIMP CStuff::GetEnum(IUnknown** ppEnum)
{
CComEnumString * thEnumObj = NULL;
HRESULT hr = CComEnumString::CreateInstance(&thEnumObj);
if(FAILED(hr))
return hr;

CComPtr<CComEnumString> thEnum(thEnumObj);

LPOLESTR strings[] = {L"One", L"Two", L"Three", NULL};
hr = thEnum->Init(&strings[0], &strings[3], NULL, AtlFlagCopy);
if(FAILED(hr))
return hr;

hr = thEnum->QueryInterface( __uuidof(IUnknown), (void**)ppEnum);
return hr;
}

First we create a new CComEnumString object. It is worth noting that the object returned by CreateInstance has a refcount of 0. Before we do anything with it we should AddRef it, but instead of doing that manually, I do it via assignment to the CComPtr. smart pointer. That way the object also gets released automatically at the end.

Then we have to make sure the enumerator has some data to enumerate. We do this by populating an array, and then telling the enumerator to copy the data out of it into its own internal buffer. This is done via the AtlFlagCopy flag.

It might seem weird if you come from a C background, but the ‘End’ pointer has to point to the first position beyond the last element. This is normal with collections in C++. That is also why I populate the array with a NULL pointer at the end; So that there is something in the array after the last ‘real’ element. This way there is no buffer overflow.

Using AtlFlagCopy has the overhead of having to copy the data, but otoh the enumerator does not have to care about the lifetime of the array. We could also use new[] to create a new array and then use the AtlFlagTakeOwnership flag to indicate that it is up to the enumerator to call delete[] and clean up. But in this example we can’t do that (read below).

When the enumerator is initialized, we return an IUnknown interface pointer to the caller. Since this will have increased the enumerator ref count, the enumerator object will keep on living when the CComPtr goes out of scope and releases its own reference to the enumerator.

There. That is all there is to it on the server side. No vast amounts of copy pasted boilerplate code, but just a couple of lines of powerful ATL incantations J

_Copy<LPOLESTR> revisited

If an enumerator has to copy data to a client, or if it has to copy data into its internal buffers in the ‘Init’ function, it has to know how to do this.

It’s all good and well to do a simple memcpy of an ULONG or a double, but for strings (LPOLESTR) and interface pointers (IUnknown) this won’t work. There’s ref counting and memory ownership to deal with.

Of course, the template class, not knowing about this, can’t anticipate what you want, so instead it needs a policy class to delegate to. That is where the _Copy<> template class comes in. It has 3 methods: Init, Destroy, and Copy. Those methods perform the actual operations needed to copy an instance of T to another instance of T, to initialize a new one, and to destroy one.

There are several stock specializations for _Copy<>, one of which works for LPOLESTR. So rather than copying the LPOLESTR pointer itself, it calls CoTaskMemAlloc to create a buffer for the copy of the string, and then copies over the data.

If you look at my example more closely, you will also see why I cannot simply new up an array of string literals and pass those to the Init function with the AtlFlagTakeOwnership flag.

LPOLESTR is superficially identical to wchar_t*. So using a string literal instead of an LPOLESTR (which is what I do in my example) will work without a hitch. But if I’d new up an array and tell the enumerator to manage it, that is when it would all go pear shaped.

Because the enumerator would not only call delete[] on the array, but it would also call _Copy::Destroy for each of the items. And trying to free up a string literal with CoTaskMemFree will result in a hard to diagnose crash.

Installing the server

On a development station this is done automatically by Visual Studio. But if you need to do this on another PC, you just execute the server from the command line with the /RegServer argument.

Creating the Client

Creating the client application is fairly simple, but I wanted to show how to do it anyway, because then I could demonstrate that the COM server works as intended.

I create a simple win32 console application so that I only had to care about the actual functionality.

Importing the server type library

In the olden days or yore, working with COM servers in a C++ project was a bit tedious. You had to include the IDL files in your project and compile them, then include the resulting header files, add the .c files to your project for the GUID definitions… Not rocket science by any means, but still… a bit messy and ugly.

Now, I have very few kind words for Visual Basic, but to give credit where it is due: Visual Basic is (in large part) responsible for the existence of tlb files.

Visual Basic programmers wanted to use COM too, and the people who made Visual Basic wanted to prevent the application programmers to have anything to do with IDL. They wanted Visual Basic to take care of all that mess behind the scenes.

And thus the type library (tlb) was born.

The type library contains the same stuff as the IDL files, but in a machine readable, programming language independent format.

To use a COM server, you need to do nothing more than add the following line to the StdAfx.h header file of the client program:

#import "Stuff_Server.tlb" no_namespace

You also need to make sure that the compiler knows where to find Stuff_Server.tlb. This is easily done by adding the following line to the ‘Additional include directories’ setting:

..\Stuff_Server\$(ConfigurationName)

If you then compile StdAfx.cpp, the compiler will pull in the type library and generate the appropriate headers which are then automatically included.

The no_namespace attribute tells the compiler that it shouldn’t put the declarations in a specific namespace.

Using the COM server

The client application code itself is fairly trivial:

int _tmain(int argc, _TCHAR* argv[])

{

HRESULT hr = CoInitialize(NULL);

if(SUCCEEDED(hr))

{

CComPtr<IStuff> stuff;

hr = CoCreateInstance(

__uuidof(Stuff), NULL, CLSCTX_ALL,

__uuidof(IStuff), (void**)&stuff );

if(FAILED(hr))

return hr;

CComPtr<IUnknown> thEnumUnk;

hr = stuff->GetEnum((IUnknown **)&thEnumUnk);

if(FAILED(hr))

return hr;

CComPtr<IEnumString> thEnum;

thEnumUnk->QueryInterface(__uuidof(IEnumString), (void**) &thEnum);

{

LPOLESTR str;

hr = thEnum->Next(1, &str, NULL);

if(hr == S_OK )

wprintf(L"%s\n", str);

CoTaskMemFree(str);

}while (hr == S_OK );

}

CoUninitialize();

return 0;

}

First I initialize the COM runtime and then create a new instance of the Stuff object. I request IStuff as the initial interface. I also use the __uuidof keyword to get the GUID for an interface / object instead of using the CLSID_ and IID_ parameters. In my opinion, it is more convenient.

The COM smart pointer takes care of the reference counting for me. This is most convenient, because then I am certain that it will get released properly without me having to worry about it.

Btw, this is also the reason that all the code is placed into a separate scope block after the call to CoInitialize. The smart pointers clean up their interface pointers when they go out of scope. If CoUninitialize would have been called before that, it would lead to weird problems / crashes.

Via the IStuff interface I request an enumerator object. This gets returned as an IUnknow interface, so I have to get the IEnumString interface via the IUnknown::QueryInterface method. So that’s 3 interfaces already, and I still don’t have to care about reference counting J

After that I use the ‘Next’ method of the IEnumString interface to retrieve the items contained in the enumerator. This the only place where I have to perform a manual cleanup action.

The IEnumString interface does not work with BSTRs, it works with LPOLESTRs. While these are code compatible because they are the same type of data, their semantics are not. The _Copy<LPOLESTR> mentioned earlier allocates data for the string using CoTaskMemAlloc. It has to be freed by CoTaskMemFree.

CComBSTR would use SysAllocString and SysFreeString. And while the code would compile and execute perfectly, there would be an crash as soon as the CComBSTR would try to free the string it got from the call to ‘Next’.

Conclusion

As you can see, implementing COM servers and clients can be really easy with the proper use of ATL. And you’ve also seen that implementing an enumerator object is made really easy.

The code demo code is up for download with this article under the MIT license. Have fun.

I will probably write some more ATL / COM stuff soon. Stay tuned for more DCOM archaeology from the mists of time ;-)

Getting started with ATL

vanDooren — Tue, 20 Jan 2009 10:53:00 GMT

I am currently working on some projects where I have to program a DCOM server. There are several reasons why it has to be DCOM and C++, instead of e.g. .NET remoting and C#.

The biggest pain (imo) about DCOM is that the technology stems from an era when the internet was no ubiquitous, and there were no huge forums filled with experts, or blogs by programming gurus and sites like codeproject.com There were newsgroups, but those were generally only used for specific problems.

There is a vast amount of concise information about everything you can do with the .NET framework. For DCOM there is … a big dark void.

Granted, there are several resources out there. But if you look on sites like codeproject, you will find that most of the articles about DCOM and ATL are written before 2003. Most of the information out there will not cover the improvements that have been made to ATL (or DCOM) since the last 5 years.

And when looking into the more obscure corners of DCOM, there are some things that noone really likes to talk about because the documented semantics are so vague, and if you look into things like custom IMoniker implementation… well… that is a story for another day.

Understanding DCOM

The most important thing with DCOM is that you understand the basics really well. I guess this is true for most things, but unlike other things like .NET, DCOM does not cut you any slack, and does not provide you with tons of helpful diagnostic information.

When I started with DCOM a couple of years ago, I first read ‘Essential COM’ by Don Box. This is really the most excellent book on COM ever.

Of course, this information becomes vitally important in the real world when you have to debug a problem, because no tool is going to tell you anything useful with DCOM related crashes.

Understanding ATL

In the real world (unless maintaining legacy code) you use ATL to write COM clients or servers. Personally, I don’t like books that just tell me how to use technology XYZ. I like to understand the ‘why’ and ‘how’ too.

This is where I have to give big thanks to ‘ATL Internals, Second Edition’. It is one of the very few books on ATL that were written for VS2005, and it is still valid for VS2008. It is also an extremely well written book.

If you are starting with ATL (or if you want to know how it works) buy this book. This book is very, very good, and will teach you a lot.

What happens next?

Over the next weeks / months I will probably write a couple of articles describing how to do specific things with ATL. One article that is nearly finished is about implementing and using IEnumString using ATL.

Not rocket science by any measure, but ATL is severely lacking in the area of easy-to-find, up-to-date and to-the-point examples. So I figured that there is still value in writing about the ancient art of circles and lollipops, even if many developers probably think COM and ATL are no longer worth bothering with.

Oh and I released some win32 DCOM demo projects some time ago. They were interesting for me because making them taught me a lot about the raw DCOM stuff that you normally don’t see with ATL. But I’ve always found that it is always useful to know what makes the motor hum.

So while they are not terribly useful for practical purposes, they are well documented and they might be useful if you are working your way through ‘Essential COM’

Cold, Coffee, and Developers

vanDooren — Fri, 09 Jan 2009 07:04:00 GMT

We are going through an unusual spell of cold weather at the moment. It was -15C when I left for work this morning. The intense cold also caused some of the outside water pipes on our site to freeze. In particular, the ones running to the temporary trailers where I am located atm. No water -> no coffee....

Except for me, that is. I have been making my own coffee for a long time now (pouring hot water over a filter with hand ground arabica beans). This means not only am I the only one here drinking good coffee, the last 2 days I was the only one drinking coffee :D.

And I know several peopel whose day did not begin well because of it. But I didn't gloat. Poking fun at people going through caffeine withdrawal is just too dangerous ;)

And for something compeltely different; I read this on Raymond's blog:

Generally speaking, programmers don't do the visual design. I mean, these are people who are lucky if they are wearing matching socks when they come to work, if they even remember to wear socks at all. And you want them to design a color scheme?

I usually manage to wear matching socks, but when it comes to choosing interior decoration, paint colors, curtains and other stuff, my wife is firmly in charge. She has a veto on my clothes too :)

Maybe there are people which are both great developers and great graphics designers. But I'm not one of them, for sure. I prefer command line apps, services, COM servers, drivers, and other low level stuff. I am good with code, not with graphical stuff. The rare few times I need to have a GUI, the best I can come up with is the default grey dialog based application, and a menu if I am feeling creative.

Found an interesting bug in the Visual Studio IDE

vanDooren — Thu, 08 Jan 2009 07:18:00 GMT

I know, I know, ... .NET is all the rage, and DCOM is legacy technology, best not touched by up and coming programmers with good hair and sharp look, lest they appear 'uncool' or even worse: 'obsolete'.

But some of us still use DCOM for a number of good reasons. And ATL takes much of the pain out of DCOM development. Unfortunately, the Visual Studio IDE doesn't always cooperate. I made a simple MFC dialog client application, and then tried to add the ATL Server project that the client would connect to. VS was clearly concerned about my attempts to use DCOM instead of .NET, and decided to sabotage my attempts to do anything useful with this new ATL project.

https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=391506

But luckily, I was able to figure it out and outsmart VS :)

More Terminal Server License Server weirdness

vanDooren — Tue, 06 Jan 2009 06:40:00 GMT

Last week I had a very peculiar problem with the Terminal Server licensing.

I couldn't connect to the licensing server anymore, even though it was running. The Terminal Server Licensing app couldn't detect it anymore. The only thing that still worked was a network ping, but as far as Windows was concerned, we were running unlicensed.

The weird thing was that Terminal Server itself disagreed, and kept working without complaints. One of these servers had been running for a year already, so it was not the 120 day grace period that was hiding an underlying problem.

At the time I didn't really understand why it still worked, but my colleague pointed out the the system time was wrong. The licensing server is a virtual machine, and for some reason, the time of the virtual machine was not linked to the NTP controlled time of the host. And as we all know, if the system times are off by n minutes, the mutual authentication fails and no secure connection can be established.

As it turns out, Terminal Server Licensing is ever more stupid than I already knew. Not only doesn't it count licenses, but the Terminal Servers don't even connect to the license server to ask for licensing. Because if they did, then they would have complained about it since connecting was no longer possible.

Backing up event logs in their normal binary format

vanDooren — Fri, 19 Dec 2008 13:15:00 GMT

Some time ago I was looking for a way to make backups of the eventlogs of our server, preferably without me ever having to do anything anymore J

This was not success. There are a couple of tools to make text exports from eventlogs. These have the disadvantage that they make the logs very large and unsuitable for looking at with the even viewer. As an added bonus, these are slow too (since every item has to be read individually).

EventQuery.vbs is one of these tools and it is so dog slow that it is almost a crime that it has been included with Windows. Ever wanted to bring a domain controller slow to a crawl? Simple: Use EventQuery.vbs to export the security event log. That’ll do it.

Anyway, I checked with my fellow MVPs, and there is no built-in (or free) tool to save event logs as binary event log files, so I gave up.

But then I accidentally discovered that there is a win32 API that exposes the event log functionality. And as luck would have it, I am a programmer and thus perfectly suited to make such a thing myself. It would have been trivial to make it at work, but I decided to do this at home so that I could share the code.

It is of course perfectly possible that such a free tool exists and I simply didn't find it.

The code

The code is simple, though the line count was increased by the desire to implement error handling.

int _tmain(int argc, _TCHAR* argv[])
{
_TCHAR* computerName = NULL;
_TCHAR* logName = NULL;
_TCHAR* file = NULL;

//populate the variables, based on the input arguments
if(argc == 3)
{
    computerName = NULL;
    logName = argv[1];
    file = argv[2];
}
else if(argc == 4)
{
    computerName = argv[1];
    logName = argv[2];
    file = argv[3];
}
else
{
    Usage();
    return -1;
}

//Open the eventlog
HANDLE hEventLog = OpenEventLog(computerName, logName);
if(NULL == hEventLog)
{
    _tprintf_s(TEXT("Could not open eventlog %s on computer %s\n"),
      logName, computerName);
    PrintError();
    return -1;
}

//backup the event log to file
if(FALSE == BackupEventLog(hEventLog, file))
{
    _tprintf_s(TEXT("Could not save eventlog to file %s\n"), file);
    PrintError();
    CloseEventLog(hEventLog);
    return -1;
}

CloseEventLog(hEventLog);
return 0;
}

There is little to tell about the code. It can be invoked with either 2 or 3 command line arguments:

The name of the computer of which the logs have to be backed up. (optional)
The name of the event lot to back up
The path of the file to which the event log needs to be saved.

The application opens the event log, and then makes the backup and closes the eventlog. If something goes wrong, the error code and error message are printed to the command line.

I thought about making the error handing a little more elegant (so that PrintError and CloseEventlog were not used multiple times in the code) but with only 2 function calls, the amount of error handling code would not diminish or be any better.

I left the PrintError function out of this blog because it doesn’t do anything exciting. It retrieves the error code and prints out the corresponding error message.

What you need to know before using this tool

I’ve been using this tool for a week now, and it is working splendidly, but there are a couple of things that aren’t mentioned in the MSDN documentation of the API.

The first is that the supplied path is absolute for the computer of which you are making an even log backup. So if the path is D:\app.evt, it will make the backup to D:\app.evt on the remote computer.

The second thing you need to know is that providing a UNC name for the target file doesn’t really improve things all that much. Because what is happening under the hood is that it is not you who is making the backup.

The event log service is in charge of the event log files, and you are asking it to make a backup. And that is where it all goes pear shaped. The event log service runs with LOCAL_SYSTEM credentials. This means that by default, it has no credentials that are valid on a remote computer. So you cannot make backup files in network locations, because the event log service of the remote computer has no privileges to access the location.

There are 2 exceptions to this rule. 1) the remote location is on the remote computer itself. 2) the remote computer is a backup domain controller.

It is a little known fact (I didn’t know it either) that LOCAL_SYSTEM on a domain controller is a domain administrative account that can pretty much do anything and everything on any domain computer, including the things that even a domain admin cannot.

Conclusion

I needed a convenient tool for making backups of event log files, and now I have one. Both the source and the binary are included as attachments. The source is available under the MIT license, and the exes are available for free to do with them what you want, as long as it is understood that whatever you do with them is your responsibility, not mine.

The eventtool_clr exe is something I made to create and delete event logs. I don’t think I blogged about it yet, but what it does is so simple it’s hardly worth mentioning. Have a look at the sources for more info.

The binary is for those who just want to use it and don’t care. The source is for those who’d rather not run an untrusted binary from some guy on the internet on their internal servers J The sources are simple enough to audit and compile.

On a side note, The API to backup event logs is not exposed on the .NET api for event logs. I don’t know exactly why, but I suspect that it is because of the problems I mentioned. The API is confusing enough that it is really not fit for generic use. After all, you can’t backup to remote systems, and the path is absolute for the remote computer.

I also didn’t want a .NET program, because for this type of thing, nothing beats the convenience and performance of a compiled binary that needs no runtime DLLs or external frameworks.

Don't people read anymore before commenting?

vanDooren — Wed, 03 Dec 2008 10:37:00 GMT

Blogs are funny things. We all have different reasons to write blog posts. Apart form the general stuff that almost everybody sometimes writes about (opinion pieces, general comments about something or other) I like writing tech articles.

Programming is my hobby, and sometimes I like to set myself a challenge, or want to see what sort of interesting things I can do with a given API / language / platform. Afterwards, I write an article about what I did, partially because I actually like writing things like that, and partially because you don't know if you understand something fully until you can sucessfully explain it to someone else, leaving out no details.

You'd be amazed about how many things you take for granted until you are forced to explain them step by step. And you'll quickly find out that some things you've never thought about twice were wrong.

Anyway, It takes a lot of time to write a detailed article about something technical (much longer than you'd expect). And I always hope that if you care enough about it to comment on it, you've actually read it. But sadly, I am often disappointed.

Take this article for example. I wrote it as an example of how it is possible to create a fifo queue without locks. In the article I mention that there is 1 writer, and 1 reader, and that my implementation is thread safe, specifically because of that. And after all, this is a simplified proof of concept of something, so lack of advanced features (like signalling) is to be expected.

And yet a lot of people comment the same thing every time 'Your code is wrong, this bit has a race condition. this is a basic mistake' and variations thereof. And then I think to myself 'Yes, of course there is a bloody race condition. That is why I mention that there should be only 1 reader and 1 writer!' What makes it even worse is that a number of people already commented this, and that I've already answered that comment several times.

I am definitely not a prima donna programmer. I never mind explaining something about my code or design, even if I have to do it a couple of times. And I really welcome comments, bugreports and other feedback about my code. If I made a mistake, I'd like to know. But I hate it when people start spewing comments without even bothering to read the article and / or the comments that have already been posted. Is it really that hard not to make an idiot of yourself?

Using reflection to ease code maintenance

vanDooren — Tue, 02 Dec 2008 11:35:00 GMT

Currently I am programming an application for performing code analysis on the software that controls the production process. Apart from the parsing of the code, and iterating across a hierarchical representation of the expression tree, all these rules (verifications) have to be executed against all the code.

I could change the code verifier every time I implemented a new rule, but instead I used reflection to do this for me. Every verification that has to be done is represented by a class that implements the right interface. This interface defines the prototype of the method that performs the verification.

interface IActionVerification
{
void Verify(CodeContext context);
}

The context variable contains the complete context of the portion of the code that is being looked at: the current action, step, function block and phase.

When a new code verifier instance is create, it will use reflection to examine the current assembly, and extract all classes that implement this interface.

public CodeVerifier()
{
    actionVerifications = new List<IActionVerification>();
    var currAssembly = Assembly.GetExecutingAssembly();
    foreach(Type t in currAssembly.GetTypes())
    {
      if (null != t.GetInterface(typeof(IActionVerification).Name))
        actionVerifications.Add(
          (IActionVerification)
             currAssembly.CreateInstance(t.FullName));
    }
}

This way, it is very easy to populate the list of all verifications to do. The code verification itself is conceptually as simple as this:

foreach (var verification in actionVerifications)
verification.Verify(context);

Implementing a new verification is the as easy as creating a new class and implementing the interface. After that everything will be done automatically. Ading a new verification is as easy as this:

class ActionExpressionCount : IActionVerification
{
    public void Verify(CodeContext context)
    {
      if (context.Action.ParsedExpression.Count < 1)
        ActionExpressionCountInvalid.LogIssue(context);
    }
}

Hip hip hooray for the .NET framework!

Visual Studio 2008 on Windows 2003

vanDooren — Fri, 21 Nov 2008 20:39:00 GMT

I've had a weird problem on my development station for some time now. Every time I ended a debugging session of a C# project, visual studio froze for about 10 seconds. It also happened if I changed project settings, but I never had any problems with C++ projects.

My development station at work is a DELL Precision 670 running windows 2003 server. It has 2 dual core Xeon cpus, 4 GB of memory, an nvidia 8500 graphics board and a WD sata hard disk. It is fast, so any lags or slowness would not be hardware related.

Now, if I was Mark Russinovich, I would use my extensive debugging experience to track down the underlying problem. But I am not, and since the majority of my work is C++, and I don't start and stop debugging sessions frequently, I could not be bothered to spend a lot of time to hunt down the problem. Installing VS2008 SP1 didn't fix the problem so I suspected it had something to do with my local configuration.

Today I finally figured it out: This was caused by the windows 2003 'Enhanced Internet Security configuration'. Removing that from my system solved the problem and VS2008 was finally as snappy as I'd expect.

I don't know exactly what the problem was, but C# projects do much more with XML and html than C++ projects. I think it is probable that C# projects do something that triggers the security configuration, which then doesn't respond because there is no user to acknowledge a prompt. 10 seconds looks like a reasonable connection timeout, so it is at least plausible.

I still would like to figure out the real cause of the problem, but at least my problem is solved. If you experience anything similar on Windows 2003, it might be worth trying this.

In my case I simply removed the entire Enhanced Internet Security configuration from my system. It is not connected to the internet, so the risk is pretty much nonexistent.

Passed 70-290 today

vanDooren — Fri, 14 Nov 2008 21:48:00 GMT

I have been silent on my blog for the last couple of weeks. This is because I spent all my time preparing for my first Microsoft exam in a couple of years. But it has been worth it. Today I passed the Microsoft certification exam 70-290: Managing and Maintaining a Windows Server 2003 Environment.

As I am really a software developer, cunningly disguished as a sysadmin, I doubted I would make it at the first attempt. But I took a lot of time to prepare, and tested all the topics in a test environment.

Long story short: I aced the exam with a perfect 100% score :)
The lady in the test centre told me that I was only the second to ever ace a Microsoft exam like that in that test centre, and the first outsider. The other guy
worked there as a trainer.

What makes this particular exam score even sweeter is that
a) I only used the MSPress book (no exam cram crap)
b) I pulverized the record high scores of the guys in the ICT department. :D

Definitely a nice way to start the weekend.