Carlos Quintero (Microsoft MVP) blog

This bug eluded me during several days because there was another bug that already caused VS 2010 Beta 2 to hang, but finally a couple of days ago I realized that was seeing a different issue (no commandbarbutton involved in this new bug) and I was able to isolate and report it today:

VS 2010 Beta 2 Bug: adding CommandBarPopup to CommandbarPopup causes VS to hang if CommandBarPopup was added to Toolbar before

https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=507650

As I expressed some days ago, my feeling with VS 2010 so far (after four builds) is that once you patch your add-in to (somehow) bypass a bug, the add-in finds a new one.

The (VB.NET) code to reproduce it is:

Imports System
Imports Microsoft.VisualStudio.CommandBars
Imports Extensibility
Imports EnvDTE
Imports EnvDTE80

Public Class Connect
   Implements IDTExtensibility2

   Private _applicationObject As DTE2
   Private _addInInstance As AddIn
   Private _commandBarToolbar As CommandBar
   Private _commandBarPopupOnToolbar As CommandBarPopup
   Private _commandBarPopupMenu As CommandBarPopup
   Private _commandBarPopupOnMenu As CommandBarPopup

   Public Sub OnConnection(ByVal application As Object, ByVal connectMode As ext_ConnectMode, ByVal addInInst As Object, ByRef custom As Array) Implements IDTExtensibility2.OnConnection

      _applicationObject = CType(application, DTE2)
      _addInInstance = CType(addInInst, AddIn)

      Select Case connectMode

         Case ext_ConnectMode.ext_cm_AfterStartup
            InitializeAddIn()

         Case ext_ConnectMode.ext_cm_Startup
            ' OnStartupComplete will be called

      End Select

   End Sub

   Public Sub OnStartupComplete(ByRef custom As Array) Implements IDTExtensibility2.OnStartupComplete
      InitializeAddIn()
   End Sub

   Private Sub InitializeAddIn()

      Dim commandBars As CommandBars

      Try

         commandBars = CType(_applicationObject.CommandBars(), CommandBars)

         If MessageBox.Show("Create toolbar before menu?", "", MessageBoxButtons.YesNo) = DialogResult.Yes Then

            ' This hangs VS 2010 Beta 2
            CreateToolbar(commandBars)
            CreateMenu(commandBars)

         Else

            ' This causes nested menus but doesn't hang VS 2010 Beta 2
            CreateMenu(commandBars)
            CreateToolbar(commandBars)

         End If

      Catch ex As Exception
         System.Windows.Forms.MessageBox.Show(ex.ToString)
      End Try

   End Sub

   Private Sub CreateToolbar(ByVal commandBars As CommandBars)

      _commandBarToolbar = commandBars.Add(Name:="My toolbar", Position:=MsoBarPosition.msoBarTop, Temporary:=True)
      _commandBarPopupOnToolbar = AddCommandBarPopupToCommandBar(_commandBarToolbar, "MyPopupOnToolbar", "My Popup On Toolbar")
      _commandBarToolbar.Visible = True

   End Sub

   Private Sub CreateMenu(ByVal commandBars As CommandBars)

      Dim menuCommandBar As CommandBar

      menuCommandBar = commandBars.Item("MenuBar")

      _commandBarPopupMenu = AddCommandBarPopupToCommandBar(menuCommandBar, "MyPopupMenu", "My Menu")
      _commandBarPopupOnMenu = AddCommandBarPopupToCommandBar(_commandBarPopupMenu.CommandBar, "MyPopupOnMenu", "My Popup On Menu")
      _commandBarPopupMenu.Visible = True

   End Sub

   Public Sub OnDisconnection(ByVal disconnectMode As ext_DisconnectMode, ByRef custom As Array) Implements IDTExtensibility2.OnDisconnection

      Try

         If Not _commandBarPopupOnMenu Is Nothing Then
            _commandBarPopupOnMenu.Delete()
         End If

         If Not _commandBarPopupOnToolbar Is Nothing Then
            _commandBarPopupOnToolbar.Delete()
         End If

         If Not _commandBarPopupMenu Is Nothing Then
            _commandBarPopupMenu.Delete()
         End If

         If Not _commandBarToolbar Is Nothing Then
            _commandBarToolbar.Delete()
         End If

      Catch ex As Exception
         System.Windows.Forms.MessageBox.Show(ex.ToString)
      End Try

   End Sub

   Public Sub OnAddInsUpdate(ByRef custom As Array) Implements IDTExtensibility2.OnAddInsUpdate
   End Sub

   Public Sub OnBeginShutdown(ByRef custom As Array) Implements IDTExtensibility2.OnBeginShutdown
   End Sub

   Friend Shared Function AddCommandBarPopupToCommandBar(ByVal objParentCommandBar As CommandBar, _
 ByVal sCommandBarName As String, ByVal sCaption As String) As CommandBarPopup

      Dim objCommandBarPopup As CommandBarPopup = Nothing
      Dim objCommandBar As CommandBar
      Dim objCommandBarControl As CommandBarControl
      Dim iPosition As Integer

      If Not (objParentCommandBar Is Nothing) Then

         iPosition = objParentCommandBar.Controls.Count + 1

         objCommandBarControl = objParentCommandBar.Controls.Add(Type:=MsoControlType.msoControlPopup, Before:=iPosition)

         objCommandBarPopup = CType(objCommandBarControl, CommandBarPopup)

         objCommandBar = objCommandBarPopup.CommandBar

         If Not (objCommandBar Is Nothing) Then

            objCommandBar.Name = sCommandBarName

            objCommandBarPopup.Caption = sCaption

         End If

      End If

      Return objCommandBarPopup

   End Function

End Class

In the post CommandBarButton.Mask property deprecated in VS 2010 I mentioned that VS 2010 will deprecate the Mask property of CommandBarButton to specify bitmap transparency, accepting 32-bit bitmaps with transparency in the alpha channel in the CommandBarButton.Picture property instead.

The VS team told me some weeks ago that the CommandBarButton.Picture would accept also in next VS 2010 builds 24-bit bitmaps with RGB=0,254,0 (almost pure green) as transparent color (as VS has always accepted for commands). I have verified this week that VS 2010 Beta 2 accepts that possibility correctly.

Bottom line: VS 2010 will accept both in Commands (AddNamedCommand) and CommandBarButton.Picture:

• 32-bit bitmaps with transparency in the alpha channel
• 24-bit bitmaps with RGB=0,254,0 (almost pure green) as transparent color

Back in VS.NET 2002/2003 the only way to change the style (Icon only, or Icon + Caption) of a CommandBarButton created from a Command was to cast the CommandBarControl returned by Command.AddControl to CommandBarButton and use the CommandBarButton.Style property.

If you use that approach in VS 2010 Beta 2 you will find this bug that I already mentioned:

VS 2010 Beta 2 Bug: CommandBarButton.Style not honored, always Icon + Caption
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=500403

That means that if the toolbar of your add-in has 30 buttons which should be shown as "only icon", due to this bug they will shown as "icon + caption" and you hardly will see 7 of them...

Fortunately there is a workaround thanks to the EnvDTE80.Commands2.AddNamedCommand2 method that was introduced in VS 2005. So you can:

• Cast the EnvDTE.DTE.Commands (or EnvDTE80.DTE2.Commands) property to EnvDTE80.Commands2
• Call the Commands2.AddNamedCommand2 method passing in the CommandStyleFlags parameter the EnvDTE80.vsCommandStyle.vsCommandStylePict value

I hope this helps.

This one happens when an add-in:

- Creates a temporary toolbar using CommandBars.Add

- Creates a temporary CommandBarPopup on that toolbar

- Creates a command and tries to add a CommandBarButton from that command to that CommandBarPopup using Command.Add(CommandBarPopup.CommandBar)

The bug report is here:

VS 2010 Beta 2 Bug: adding CommandBarButton from Command to CommandbarPopup on Toolbar causes VS to hang

https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=503871

The (VB.NET) code to reproduce the problem is:

Imports System
Imports Microsoft.VisualStudio.CommandBars
Imports Extensibility
Imports EnvDTE
Imports EnvDTE80

Public Class Connect
   Implements IDTExtensibility2
   
   Private _applicationObject As DTE2
   Private _addInInstance As AddIn
   Private _myCommandBarToolbar As CommandBar
   Private _myCommandBarPopup As CommandBarPopup

   Public Sub OnConnection(ByVal application As Object, ByVal connectMode As ext_ConnectMode, ByVal addInInst As Object, ByRef custom As Array) Implements IDTExtensibility2.OnConnection

      _applicationObject = CType(application, DTE2)
      _addInInstance = CType(addInInst, AddIn)

      Select Case connectMode

         Case ext_ConnectMode.ext_cm_AfterStartup
            InitializeAddIn()

         Case ext_ConnectMode.ext_cm_Startup
            ' OnStartupComplete will be called

      End Select

   End Sub

   Public Sub OnStartupComplete(ByRef custom As Array) Implements IDTExtensibility2.OnStartupComplete
      InitializeAddIn()
   End Sub

   Private Sub InitializeAddIn()

      Const MY_COMMAND_NAME As String = "MyCommand"

      Dim command As Command = Nothing
      Dim commandBars As CommandBars
      Dim commandBarControlObject As Object

      Try

         commandBars = CType(_applicationObject.CommandBars(), CommandBars)

         _myCommandBarToolbar = commandBars.Add(Name:="My toolbar", Position:=MsoBarPosition.msoBarTop)

         ' Create a commandbar popup
         _myCommandBarPopup = CType(_myCommandBarToolbar.Controls.Add(Type:=MsoControlType.msoControlPopup), CommandBarPopup)
         _myCommandBarPopup.Caption = "My CommandBarPopup"

         ' Try to retrieve existing command
         Try
            command = _applicationObject.Commands.Item(_addInInstance.ProgID & "." & MY_COMMAND_NAME)
         Catch ex As Exception
         End Try

         ' Create command
         If command Is Nothing Then
            command = _applicationObject.Commands.AddNamedCommand(_addInInstance, MY_COMMAND_NAME, "My command", "My tooltip", True, 59)
         End If

         ' This line hangs VS 2010 Beta 2
         commandBarControlObject = command.AddControl(_myCommandBarPopup.CommandBar)

      Catch ex As Exception
         System.Windows.Forms.MessageBox.Show(ex.ToString)
      End Try

   End Sub

   Public Sub OnDisconnection(ByVal disconnectMode As ext_DisconnectMode, ByRef custom As Array) Implements IDTExtensibility2.OnDisconnection

      Try

         If Not _myCommandBarPopup Is Nothing Then
            _myCommandBarPopup.Delete()
         End If

         If Not _myCommandBarToolbar Is Nothing Then
            _myCommandBarToolbar.Delete()
         End If

      Catch ex As Exception
         System.Windows.Forms.MessageBox.Show(ex.ToString)
      End Try

   End Sub

   Public Sub OnAddInsUpdate(ByRef custom As Array) Implements IDTExtensibility2.OnAddInsUpdate
   End Sub

   Public Sub OnBeginShutdown(ByRef custom As Array) Implements IDTExtensibility2.OnBeginShutdown
   End Sub
   
End Class

After 3 hours isolating this to the minimal expression (which is tedious if your add-in is has lots of user interface items and debugging a VS 2010 add-in on a virtual machine is painfully slow), here it is:

VS 2010 Beta 2 Bug: deleting a CommandbarPopup unloading add-in causes VS to disappear
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=503093

(the problem happens if the add-in creates two (not just one) CommandBarPopups on a parent CommandBarPopup and tries to delete them when unloaded)

I can't express how disappointed, frustrated (and more things!) I am with VS 2010 Beta 2 regarding correct behavior of commandbars for add-ins. It seems that once a bug is fixed in a VS 2010 build and an add-in can pass that point, new bugs surface that prevent its correct loading (and I have tested several builds since Beta 1).

Suffice to make you aware of these bugs:

VS 2010 Beta 2 Bug: CommandBarButton of add-in duplicated in new commandbar when already created in another commandbar
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=502289

(For my MZ-Tools add-in with tons of buttons, menus and popups, only that bug of VS 2010 Beta 2 has completely ruined it, not to mention the time it took me to isolate it until I realized what was happening because initially the add-in seemed to hang, due to the number of duplications that were occurring at the glacial speed of VS 2010 debugging an add-in on another VS 2010 instance on a virtual machine)

VS 2010 Beta 2 Bug: CommandBarButton.Style not honored, always Icon + Caption
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=500403

VS 2010 Beta 2 Bug: System.AccessViolationException attempting to create a temporary popup commandbar
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=500408

VS 2010 Beta2: InvalidCastException casting DirectCast(CommandBarControl, CommandBarPopup).CommandBar.Parent back to CommandBarControl
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=499483

I realize that changing the COM-based commandbars of VS 2008 by new WPF-based commandbars in VS 2010 is a daunting and difficult task, but if the next milestone after Beta 2 is Release Candidate (where only critical bugs are fixed) then Beta 2 should be already free of such nasty bugs. Or there should be a known roadmap of betas / CTPs until RTM so that developers of add-ins can know how many builds they will have to test and rest assured that they will have enough weeks/months to test their add-ins (considering that in Beta 2 they can't get them even loaded in Visual Studio).

I think that since Microsoft builds and extends Visual Studio with packages (SDK) and not add-ins, commandbars/commands/buttons of packages get better behavior earlier in the development cycle of Visual Studio. But for add-ins (automation) every time that commandbars are changed in some VS release, you have to wait until the RTM to check is some bug that could ruin your add-in is fixed. I experienced the same with VS 2005, which introduced 24-bit bitmaps (rather than the 16-color bitmaps of VS.NET 2003) and a different assembly for Microsoft.VisualStudio.Commandbars.

Microsoft has reported that this problem will be fixed:

 " 'Object must be the same type as the enum' exception calling CommandBars.Add from an add-in"
http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=462766

Notice that the "Embed Interop Types" property seems to have been renamed to "Isolated" in VS 2010 Beta 2, in one of those renaming changes that make things more obscure...

FWIW, I have verified that VS 2010 Beta 2 certainly fixes the issue of supporting RGB=0,254,0 for transparent color in 24-bit command pictures that I mentioned here.

So, if your add-in targets VS 2005, 2008 and 2010 you can use that color for the transparent background color of command pictures and they won't appear with lime green in Beta 2 as it happened in Beta 1 or previous CTPs builds.

If your add-in only targets VS 2010 it is preferable to use the new support for 32-bit command bitmaps. I guess that in a few years Microsoft will only support 32-bit bitmaps in Visual Studio.

This is going to be messy, so step by step:

Imagine that you are developing an add-in that uses a temporary user interface (not permanent one), and creates a couple of commandbars of kind "toolbar", and many commandbars of kind "popup" (with submenus).

Since it uses a temporary user interface, it must delete those commandbars when unloaded.

For that purpose, it must keep a reference to each created commandbar at class level (all this is explained in the article of the previous link).

Since it creates lots of commandbars, rather than keeping each one individually, they are stored in a class-level collection. Of which type?

CommandBars of kind "Toolbar" have the "Microsoft.VisualStudio.CommandBars.CommandBar" type but commandbars of kind "popup" have the Microsoft.VisualStudio.CommandBars.CommandBarPopup" type, so both can't be stored in the same typed collection, but we would like to use a single collection, not two, or not a collection of System.Object.

Fortunately the Microsoft.VisualStudio.CommandBars.CommandBarPopup type has a CommandBar property, and since the Microsoft.VisualStudio.CommandBars.CommandBar type has a Parent property (typed as System.Object, that should return a Microsoft.VisualStudio.CommandBars.CommandBarPopup actually), you can use a generic typed collection List<Microsoft.VisualStudio.CommandBars.CommandBar>, which stores the CommandBar of toolbars, and the CommandBarPopup.CommandBar of popup commandbars.

When the add-in is unloaded, it iterates the commandbars of the collection to delete them:

• If the CommandBar.Type is MsoBarType.msoBarTypeNormal, the commandbar is a toolbar and can be deleted calling the Delete method.
• If the CommandBar.Type is MsoBarType.msoBarTypePopup, the commandbar is a popup so we get its Parent property (which is a System.Object) and we cast it to Microsoft.VisualStudio.CommandBars.CommandBarPopup. Then, we call the Delete method of CommandBarPopup.

(As I mentioned, my MZ-Tools add-ins seems to use every conceivable technique provided by the Microsoft.VisualStudio.CommandBars API)

However, this approach that worked perfectly in VS 2005/2008, doesn't work in VS 2010. Why?

It happens that VS 2010 uses WPF-based commandbars, but add-ins for VS 2010 still use the Microsoft.VisualStudio.CommandBars Interop (ActiveX) assembly for backwards compatibility.

That interop assembly provides the Microsoft.VisualStudio.CommandBars.CommandBarPopup type, which is actually an interface (although it doesn't follow the convention for interface names), not a class. The same happens with the CommandBar and CommandBarControl types, which are interfaces, not classes. The CommandBarButton type is a class, though. To complicate things, you have the CommandBars class and the _CommandBars interface... anyway:

In VS 2005/2008, which use native (COM) commandbars, the classes that actually implement those interfaces are native, and from a managed add-in you see them through a System.__ComObject which is the Runtime Callable Wrapper (RCW) that you get when you don't have an interop assembly. But since they implement the interfaces provided by the Microsoft.VisualStudio.CommandBars assembly, the add-in works fine.

In VS 2010, which uses managed (.NET) WPF commandbars, the classes that actually implement those interfaces are managed and are provided by the Microsoft.VisualStudio.PlatformUI.Automation namespace in the managed (.NET) Microsoft.VisualStudio.Shell.UI.Internal.dll assembly (in one of the VS 2010 folders).

When you create a commandbar popup calling:

Microsoft.VisualStudio.CommandBars.CommandBarControl commandBarControl;

commandBarControl = parentCommandBar.Controls.Add(MsoControlType.msoControlPopup);

you get some class instance that implements the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface. So you can do this cast:

Microsoft.VisualStudio.CommandBars.CommandBarPopup commandBarPopup;

commandBarPopup = (Microsoft.VisualStudio.CommandBars.CommandBarPopup) commandBarControl;

In VS 2010, the actual class that implements that Microsoft.VisualStudio.CommandBars.CommandBarPopup interface is the Microsoft.VisualStudio.PlatformUI.Automation.CommandBarPopup._Marshaler class.

However, when you do this:

commandBarPopup.CommandBar.Parent

in VS 2005/2008 you get an object that implements the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface (as expected), so the approach described in the beginning of this post worked.

In VS 2010 you get an instance of the Microsoft.VisualStudio.PlatformUI.Automation.CommandBarPopup class, not of the Microsoft.VisualStudio.PlatformUI.Automation.CommandBarPopup._Marshaler class that you got when creating it.

It happens that the Microsoft.VisualStudio.PlatformUI.Automation.CommandBarPopup class doesn't implement the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface. It implements IMarshaledObject<CommandBarPopup> instead. It also has a Marshaller property that would return the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface implementation.

So, the approach that worked with VS 2005/2008 doesn't work with VS 2010.

Since you can't cast or get the Microsoft.VisualStudio.CommandBars.CommandBarPopup interface and an add-in targeting VS 2005/2008/2010 with the same dll can't include a reference to the Microsoft.VisualStudio.Shell.UI.Internal.dll, keeping this approach the only way would be to call its Delete method would be through Reflection.

If you are testing your add-in with the recently released VS 2010 Beta 2 chances are that you have noticed that the new WPF-based commandbars, buttons and icons are not yet ready for prime time. It seems that my MZ-Tools add-in uses every possible method, property and technique offered by the Microsoft.VisualStudio.CommandBars API so I am discovering lots of bugs (I have reported 15 so far) since I started testing with Beta 1. I reported these two ones before Beta 2 but the fixed will come in builds after Beta 2:

VS 2010 Beta 2 Bug: System.AccessViolationException attempting to create a temporary popup commandbar
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=500408

VS 2010 Beta 2 Bug: CommandBarButton.Style not honored, always Icon + Caption
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=500403

If you are developing add-ins for in-house use, chances are that you can drive, encourage or even force the whole organization to use the latest version of Visual Studio and therefore your add-in needs to target only that version. But if you are developing add-ins for commercial or open-source purposes, to be used by lots of people, your add-in needs to target several IDE versions (nowadays, at the very least VS 2005, 2008 and 2010). To simplify your life as a developer, you try to keep a single code base and a single binary dll. For example:

- You create the add-in with VS 2005 (and therefore .NET Framework 2.0 / CLR 2.0) with a single project, referencing EnvDTE and EnvDTE80, VSLangProj, VSLangProj2 and VSLangProj80, generate a single DLL, but you create the proper .AddIn file and put it in the proper folder to target VS 2008 too.

or:

- You create the add-in with VS 2008 (targeting .NET Framework 2.0 / CLR 2.0) with a single project, with the same references than above, taking care of not referencing EnvDTE90 or VSLangProj90 (which are specific to VS 2008), generate a single DLL, but you create the proper .AddIn file and put in in the proper folder to target VS 2005 too.

Both approaches work. Some people use the first one to ensure 100% that the add-in will work with VS 2005 (since it was created with that IDE and .NET Framework 2.0) although I personally use the second one (I like VS 2008 much more than VS 2005) with no problems so far.

If for some reason the add-in needs to know the IDE version where is hosted or behave differently dynamically depending on the IDE version, it can guess it using DTE.Version or DTE.RegistryRoot, but there is no need to have separate DLLs if both VS 2005 and VS 2008 can use the same CLR 2.0, .NET Framework 2.0 and the same set of references.

Now, VS 2010 comes. So, your natural approach is to keep your existing code, binary dll and make it target VS 2010 adjusting the .AddIn file. Although VS 2010 uses a new CLR (4.0) instead of CLR 2.0, a CLR can load assemblies compiled against the previous version, in the same way that VS.NET 2003 (using CLR 1.1) could use add-ins created with VS.NET 2002 (using CLR 1.0) if the add-in was properly registered.

When you create your add-in with VS 2005 and you want to debug it on VS 2008, you just simply go to the Debug tab of the project properties and set the Start Action to start the devenv.exe of VS 2008, and this works perfectly.

Similarly, when you create your add-in with VS 2008 and you want to debug it on VS 2010, this worked perfectly until VS 2010 Beta 2. With the Beta 2 released two days ago you get the error:

The debugger's protocol is incompatible with the debuggee

I have confirmed with Microsoft that this is not a bug, it is by design: VS 2008 (using CLR 2.0) or VS 2005 won't be able to debug CLR 2.0 add-ins hosted in VS 2010 (which uses CLR 4.0). Which is a pity, but this means that your workarounds in the scenarios above are:

• Use VS 2010 to create a separate binary add-in DLL (CLR 4.0) for VS 2010 only. For me, this is the worst workaround, only considered as a last resort.
• Use VS 2005/2008 to create a single binary add-in DLL (CLR 2.0) to target VS 2005, 2008 and 2010, giving up debugging on VS 2010, resorting to MessageBoxes in your code to guess what's going on if you need to debug a problem that can't be reproduced when the add-in is hosted on VS 2005 or VS 2008.
• Use VS 2010 to create a single binary add-in DLL (targeting CLR 2.0 /  .NET Framework 2.0, not CLR 4.0 / .NET Framework 4.0) to target VS 2005 / 2008 / 2010. I haven't test this yet but I hope it works. Even if VS 2010 can't debug a CLR 2.0 add-in hosted on another VS 2010 instance, you can temporarily switch it to target CLR 4.0 and this should work. Once the problem is debugged and solved, you can switch it back to CLR 2.0. I hope that VS 2010 can also debug a CLR 2.0 add-in hosted on VS 2005 or VS 2008.

Let me know if you have more insights about all this.

As you know if you develop add-ins, Visual Studio 2005 introduced XML-based add-ins that use an .AddIn file that you must place in the proper folder for Visual Studio to show the add-in in the Add-In Manager. Which XML parser Visual Studio uses to parse .AddIn files wouldn't be of interest if it wasn't because for some people their Add-In Manager didn't show XML-based add-ins... and the problem was solved installing MSXML. While I didn't experience that problem personally, I have experienced a related problem with VS 2010 Beta2 and older CTPs.

The problem was that when I tried to create an add-in with the add-in wizard I got this misleading error message (misleading because I do have the proper language installed).

"An error occurred, and the wizard could not generate the project. Verify that the programming language is properly installed"

And more interestingly, if a place an .AddIn file (even an empty .AddIn file!) in one of the folders that VS scans for .AddIn files, VS 2010 fails to load with the following error message:

"The application cannot start"

and then it closes!

I remembered the problem of the empty Add-In Manager, I thought it could be related, so I verified with the Process Monitor tool that VS was certainly failing to get the MSXML2.Document.6.0 ProgId from the registry just before scanning folders for .AddIn files. I installed MSXML 6.0 and lo and behold, the problem was solved. 

So, I discussed this with several people inside the VS team and they confirmed that VS 2010 will use MSXML 6.0 to parse .AddIn files, etc. (I am not sure if VS 2005/2008 uses that version or older ones). It happens that I was using a virtual machine with Windows XP SP2, and it doesn't include MSXML 6.0 (it includes MSXML 3.0 SP5), but VS 2010 Beta 2 / CTPs didn't install it either. When I asked why VS 2010 didn't install it as a prerequisite, the answer was that VS 2010 will require SP3 of Windows XP (which includes MSXML6), and the final release of VS 2010 will refuse to install on Windows XP SP2, something that the current betas / CTPs don't enforce yet.

Now I hope that the Visual Studio Team modifies VS 2010 to show better behavior and diagnostics if, for whatever reason, Visual Studio doesn't find the MSXML parser that it requires. We as developers tend to focus on code paths where things go as expected, and pay less attention to edge cases, but when things go bad, it causes a lot of pain to other people trying to diagnose and solve the problem. Something better than silent errors (empty Add-In Manager), crashes without further information or misleading error messages can be done...

As you already know from my posts in the last months, VS 2010 will use WPF-based commandbars and will introduce support for 32-bit bitmaps with built-in transparency in the alpha channel for pictures of commands of add-ins, apart from the old way of using 24-bit bitmaps with RGB=0,254,0 for background color. While VS 2010 Beta 1 only supports the new way, I've been told that Beta 2 will support the old way for backwards compatibility.

Toolwindows in VS 2005 and higher have always supported either 32-bit bitmaps with built-in transparency in the alpha channel, or 24-bit bitmaps with RGB=255,0,255 as background color, and VS 2010 will support both too.

The third and last area problematic with custom pictures is when you want to set the picture of a CommandBarButton directly (not through a command). While some people use this approach to avoid satellite dlls, the only scenario where I personally use and would recommend this approach is to provide a context menu in some listview or treeview with menu entries that don't have a command behind (because they are operations for the listview or treeview such as "add", "edit", "remove" items, not global operations). To make such context menu with a look & feel 100% consistent with Visual Studio I create a CommandBarPopup. I haven't written yet an article to show this but the basic idea is to create a commandbar popup calling:

myCommandBarPopup = dte.CommandBars.Add(Name:="MyCommandBar", Position:=MsoBarPosition.msoBarPopup, Temporary:=True)

and then add CommandbarButtons calling:

myCommandBarButton = DirectCast(myCommandBarPopup.Controls.Add(...), CommandBarButton)

and then set the Caption, Picture and Mask properties:

myCommandBarButton.Caption = "..."

myCommandBarButton.Picture = ...

myCommandBarButton.Mask = ...

and then set-up an event handler for the Click event.

Notice that you have to provide a "picture" bitmap, which in VS 2005 and 2008 should be a 24-bit bitmap with any background color, and a "mask" bitmap which should use White=255,255,255 for the pixels that should be transparent color and Black=0,0,0 for the pixels that are actually colored in the "picture" bitmap. Since this has the inconvenience of being forced to provide two bitmaps per button, most people use a custom method to generate the mask image from the picture image.

This is not going to work in VS 2010, though. Since VS 2010 supports 32-bit bitmaps with built-in transparency in the alpha channel, Microsoft is making the CommandBarButton.Mask property deprecated (you get an exception if used). So, if your add-in for VS 2005 / 2008 uses the CommandBarButton.Mask property in one of the two scenarios that I have mentioned, you will need to revisit it and adjust it accordingly to make it work with VS 2010. Assuming that you want the same binary dll to work with VS 2005, 2008 and 2010, one approach is to design 32-bit bitmaps images with built-in transparency (for the VS 2010 version) and generate programmatically the 24-bit "picture" bitmap and the "mask" bitmap for the 2005 and 2008 versions. If your add-in only targets VS 2010, you no longer have to use the CommandBarButton.Picture approach to avoid the satellite dll since VS 2010 will get rid of satellite dlls for add-ins.

The problem that always existed in Visual Studio with the generated grayscale picture for a disabled command of an add-in (or CommandBarButton without a command) is going to be fixed in Visual Studio 2010, according to the tests that I have done. That is, an add-in with a command that provides a 24-bit bitmap in a satellite dll with RGB=0,254,0 as background color (the old way compatible with VS 2005 / 2008) now shows a crisp and defined disabled picture if the command is disabled. Notice that providing a 32-bit bitmap with built-in transparency (the new way introduced in VS 2010) is not required to get this "fix", it is something that Microsoft has fixed internally even for 24-bit bitmaps in the new WPF-based commandbars of VS 2010.

I have been unable to test if the same applies to CommandBarButtons whose picture is provided through the Picture property (without a command) but I think it will be fixed too.

It always fascinates me how "creative" we the developers can be solving problems that shouldn't exist in the first place. For example, Microsoft has been too much creative in the last decade providing multiple tricky ways to solve the lack of transparency support in the original bitmap format. And now I am seeing in several places an approach that developers of add-ins are using to avoid the satellite DLLs to provide custom pictures. The first place where I saw it was in the very MSDN documentation about displaying a custom icon on the add-in button, in the Community Content section by Miguel Ferreira.

The basic idea is to give up the picture of the command, and to modify the picture of the CommandBarButton created from the command through its Picture and Mask properties. The implementation gets more complicated because those properties have the COM IPictureDisp type and therefore conversions from managed System.Drawing.Bitmap are required, and another creative technique is used to get the mask from a picture pixel by pixel rather than providing it directly (which would force to create two bitmaps per button).

I flagged that community content as "bug" because it doesn't solve the fundamental problem, that is, providing a picture for the command, not just for its buttons. The picture of a command is something totally visible to the user when she clicks the "Tools", "Customize" menu, "Commands" tab, "Add-Ins" section. While it may seem not very important a lacking command picture there, it happens that with that window open VS enters in a special mode that allows the user to drag the command on a toolbar to create a button, and the button would get the picture of the command.

I saw this approach too yesterday in this post of Roy Osherove.

Fortunately these hacks won't be necessary in VS 2010, which will get rid of satellite DLLs for add-ins "officially".

I am now investigating why the grayscale image that Visual Studio 2005 / 2008 generates for a disabled button is so horrible/blurry for pictures of add-in commands and so crisp and precise for pictures of VS commands. VS 2005 introduced True Color bitmaps (VS.NET 2003 used 16-color bitmaps) so it should be able to generate good-looking grayscale images as it does with its own pictures (if I am correct that the disabled image is autogenerated and not provided separately through some way not available to add-ins). There are two scenarios where disabled buttons look bad:

- CommandBarButtons created from a command (via Command.AddControl). In this case the image of the button is actually the image of the command which in turn is supplied in a satellite dll in True Color (24 bit) bitmap format, so there is no much room to play...

- CommandBarButtons created on a CommandBar created by the add-in to be used as context menu somewhere, for example in the context menu of a list (New, Remove, etc.). The add-in creates a VS commandbar rather than a .NET context menu to provide the same look and feel. In this scenario, there is no command, the CommandBarButton is created via CommandBar.Controls.Add. When you cast the returned CommandBarControl to CommandBarButton you can use the Picture and Mask properties (COM IPictureDisp type) to set the image and the transparency mask, so there is much more room to play with the bitmap format, the color depth (32-bit, 24-bit, 8-bit), and how you get a COM IPictureDisp from a managed System.Drawing.Bitmap.

So far I have been unable to solve this, I will post again if I find a resolution for this issue. If someone know the solution, let me know.

I have found today three links on the web about issues with transparency in custom pictures of buttons that Office add-ins have suffered too:

Alpha channels, masks, color keys, oh my! (By Eric Faller about the Office Ribbon)
http://blogs.msdn.com/jensenh/archive/2006/11/27/ribbonx-image-faq.aspx

How To Create a Transparent Picture For Office CommandBar Buttons (MSDN KB Article)
http://support.microsoft.com/kb/288771

How To Set the Mask and Picture Properties for Office XP CommandBars (MSDN KB Article)
http://support.microsoft.com/kb/286460

Given that Visual Studio borrowed its commandbars from Office, it's no wonder it has suffered the same problems.

I think I forgot to blog about the resolution of this bug:

devenv.exe /resetaddin doesn't fully reset the add-in
http://msmvps.com/blogs/carlosq/archive/2009/06/23/devenv-exe-resetaddin-doesn-t-fully-reset-the-add-in.aspx

whose Microsoft Connect bug report is:

https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=469322

The problem is, you see, that devenv.exe /resetaddin doesn't remove permanent commandbars, only commands and buttons on commandbars. That means that if you are using permanent commandbars and not temporary ones, at the very least your uninstaller needs to take care of removing permanent commandbars explicitly because devenv.exe /resetaddin won't do it. This can be done creating an instance of EnvDTE.DTE, locating the commandbar and calling EnvDTE.DTE.Commands.RemoveCommandBar (see HOWTO: Removing commands and UI elements during Visual Studio .NET add-in uninstallation). And not only the uninstaller: when debugging you may need to remove the commandbar too because VS 2005 and higher fire a new UISetup phase each time that you debug, so you get duplicated commandbars.

The temporary vs permanent approaches for commandbars is a nightmare for two reasons:

• First, their own existence. A single approach would be much better. And the temporary one would be preferable. The reason for the permanent one is that for add-ins with a large number of commandbars and buttons, it takes less time to persist the UI on disk and retrieve it from there when launching VS than recreating the UI. I think that there aren't many add-ins with lots of commandbars and buttons (certainly my MZ-Tools add-in is one with such large number of buttons, but I keep using the temporary approach). But if performance is a problem, then it is better to fix that rather than inventing a new approach. Some ideas for Microsoft to increase the performance of add-ins using the temporary approach are:
  
  • Provide a EnvDTE.Commands.Exist(commandName) method to test if a command exists or not (to create it only if it doesn't exist). Currently you have to call the Item method which causes an (expensive) exception if the command doesn't exist.
  • Provide a EnvDTE.AddInCommands collection that returns only the commands of add-ins, not the hundreds of commands of Visual Studio.
  • Make EnvDTE.Command.AddControl to behave as follows: if you are adding a button to a menu, set the CommandBarButton.Style to msoButtonIconAndCaption (VS already does this), but if you are adding it to a toolbar, set the CommandBarButton.Style to msoButtonIcon since it is more likely that style (without caption) on a toolbar. Currently VS defaults to msoButtonIconAndCaption also in this case, which means that after getting the CommandBarControl, you have to cast it to CommandBarButton and then change the Style property, which means that the button needs to be redrawn (a performance hit). I already reported this two years ago to no avail: Wrong default vsCommandStyle for CommandBarButtons created from add-ins. Even more, Visual Studio doesn't work as supposed to do: Problems with ContextUIGUIDs and vsCommandDisabledFlagsValue in EnvDTE.Commands.AddNamedCommand (a bug yet to be acknowledged and fixed).
  • Simplify the way of getting transparent pictures for commands. The transparent color has been RGB=0,254,0 for long time which means that a color remapping must be performed. Hopefully this has been addressed in VS 2010 with support for 32-bit bitmaps with transparency in the alpha channel.
    
• Second, the APIs are not consistent, which creates a lot of confusion. If both approaches need to exist, at least they would have to use the same methods with just one boolean parameter indicating if a commandbar is permanent or temporary. Currently you have:
  
  • CommandBars.Add to add temporary commandbars
  • CommandBar.Delete to remove temporary commandbars
  • Commands.AddCommandBar to add permanent commandbars (even if the CommandBars.Add method above has a Temporary parameter!)
  • Commands.RemoveCommandBar to remove permanent commandbars

Anyone can see that that mix of methods to add commandbars is bound to cause problems (why is there a AddCommandBar method in the Commands collection?) and certainly it is: the bug of devenv.exe /resetaddin can't be fixed because the Commands.AddCommandBar method lacks a fundamental piece of information: the AddIn instance that is adding the commandbar. Without that information the IDE doesn't know which add-in is the owner of a permanent commandbar and therefore can't remove it.

In my last post Visual Studio 2010 getting rid of satellite DLLs for add-ins I mentioned that VS 2010 will provide two approaches to avoid satellite DLLs for command pictures of add-ins:

The first one was clear: command pictures will be able to be embedded in the own DLL of the add-in.

The second one was not so clear:

"Commands.AddNamedCommand2 will now support an IPicture".

After that post I exchanged e-mails with Suzanne Hansen (Program Manager of Visual Studio Platform Shell Team) and Jeff Robison (the developer implementing the changes). While I thought that a new EnvDTE100.Commands3.AddNamedCommand3 would be added supporting managed System.Drawing.Bitmap or System.Drawing.Icon types, they explained me that the Bitmap parameter of the current EnvDTE80.Commands2.AddNamedCommand2 method is actually a Variant/Object, not an integer (as it happens with the old EnvDTE.Commands.AddNamedCommand method) and therefore it allows you to pass other types apart from an integer (denoting a numeric bitmap id). Jeff took advantage of this to accept an IPicture type without introducing a new AddNamedCommand method with a different signature.

The bad news was that, alas, IPicture is a native COM type, not a managed type, so developers of managed add-ins would have to use hacks such as OleCreatePictureIndirect or System.Windows.Forms.AxHost.GetIPictureDispFromPicture to convert a managed System.Drawing.Bitmap to a native COM IPicture or IPictureDisp. I was familiar with those techniques (see my article HOWTO: Creating custom pictures for Visual Studio .NET add-ins commands, buttons and toolwindows) and they were ugly to me, and prone to problems with transparency as happened in the past.

So, I suggested to make EnvDTE80.Commands2.AddNamedCommand2 to accept an IntPtr type that would denote the handle of a managed bitmap (System.Drawing.Bitmap.GetHBitmap). The EnvDTE.Window.SetTabPicture already accepts that happily to set the picture of a toolwindow.

They devoted some days to think about all that and a couple of days ago Suzanne informed me that Jeff was able to make EnvDTE80.Commands2.AddNamedCommand2 accept managed System.Drawing.Bitmap or System.Drawing.Icon apart from the native IPicture type added previously. This couldn't be greater news. Not only you will be able to get rid of embedded bitmaps, so can also use icons if you are not comfortable with 32-bit bitmaps with alpha channel for transparency.

FWIW if you are developing a package and not an add-in:

"Jeff has also made a similar change for package authors – a new API, AddNamedCommands3, will be available that will also accept Bitmaps, Icons and IPictures."

While these changes won't be available in the next VS 2010 Beta 2, they will be in subsequent builds. It is also great that they did these changes so late in the VS 2010 development cycle. Usually when Beta 2 is reached it is "feature completed" and no changes such as those are allowed, just bug and performance fixes.

I am looking forward to test these new changes. At last Visual Studio will allow you to decide:

• The way to provide custom pictures for add-ins:
  • Embedded bitmaps in satellite Dll
  • Embedded bitmaps in add-in Dll
  • Directly in the EnvDTE80.Commands2.AddNamedCommand2 method

• The type of image to use:
  • 32-bit System.Drawing.Bitmap with alpha channel for transparency
  • System.Drawing.Icon (built-in transparency)

That plethora of choices together with the absence of COM stuff is what makes a good EnvDTE API for add-in developers.

Jeff and Suzanne, very good job!!

Two years after I reported it, it seems that MS finally understood that this was a bug and I have been notified today that it is no longer closed as "By design" but as "Fixed":

Standard user unable to load/unload COM-based add-ins registered for all users with the VS 2008 Add-in Manager on Windows Vista
http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=365846

This bug has plagued all of us developing add-ins with COM-registration (rather than .AddIn XML registration) for all users (not for current user) for backwards compatibility with VS.NET 2002 and/or 2003. Too bad that the fix comes so late that I doubt there will be a fix for VS 2008, only for VS 2010...but anyway, it will make VS 2010 a better product.