You may have noticed that Microsoft have had another burst of renaming. The tester’s tool in VS2010 started with the codename of Camaro during the CTP phase, this became Microsoft Test & Lab Manager (MTLM) in the Beta 1 and 2 and now in the RC it is call Microsoft Test Manager (MTM).
Other than me constantly referring to things by the wrong name, the main effect of this is to make searching on the Internet a bit awkward, you have to try all three names to get good coverage. In my small corner of the Internet, I will try to help by updating my existing MTLM tag to MTM and update the description appropriately.
If you look on Google Groups you will find the start of a thread trying to organise some AltNet Beers session in Leeds, only the same lines as the London, Bristol etc. events. There appears to be no date set as yet, but keep an eye open, they are a great way to meet like minded people.
I have at last worked all the way through setting up my portable end to end demo of testing using Windows Test and Lab Manager. The last error I had to resolve was the tests not running in the lab environment (though working locally on the development PC). My the Lab Workflow build was recorded as a partial success i.e. it built, it deployed but all the tests failed.
I have not found a way to see the detail of why the tests failed in VS2010 Build Explorer. However, if you:
- Go into MTLM,
- Pick Testing Center
- Select the Test Tab
- Pick the Analyze Test Results link
- Pick the test run you want view
- The last item in the summary is the error message , as you can see in my case it was that the whole run failed not any of the individual tests themselves
So my error was “Build directory of the test run is not specified or does not exist”. This was caused because the Test Controller (for me running as Network Service) could not see the contents of the drop directory. The drop directory is where the test automation assemblies are published as part of the build. Once I gave Network Service read rights to access the \\TFS2010\Drops share my tests, and hence my build, ran to completion.
It has been a interesting journey to get this system up and running. MTLM when you initially look at it is very daunting, you have to get a lot of ducks in a row and there are many pitfalls on the way. If any part fails then nothing works, it feels like a bit of a house of cards. However if you work though it step by step I think you will come to see that the underlying architecture of how it hangs together is not as hard to understand as it initially seems. It is complex and has to be done right, but you can at least see why things need to be done. Much of this perceived complexity for me a developer is that I had to setup a number of ITPro products I am just not that familiar with such as SCOM and Hyper-V Manager. Maybe the answer is to make your evaluation of this product a joint Dev/ITPro project so you both learn.
I would say that getting the first build going (and hence the underlying infrastructure) seems to be the worst part. I feel that now I have a platform I understand reasonably, that producing different builds will not be too bad. I suspect the next raft of complexity will appear when I need a radically different test VM (or worse still a networks of VMs) to deploy and test against.
So my recommendation to anyone who is interest in this product is to get your hands dirty, you are not going to understand it by reading or watching videos, you need to build one. So find some hardware, lots of hardware!
I recently been working with a client who has been seeing strange problems when they try to create new workitems via a SharePoint portal site on a TFS2010 Beta2 installation. They appeared to have a fully working TFS2010 installation, but when they came in on a morning they found that even though they could login to the TFS created SharePoint team site they could not create a new workitems, they got an “Error 403 Access Forbidden”.
If they logged into SharePoint as a user with system administration rights it all worked fine. Now here is the strange bit, if they then logged in the user who got the 403 error it all worked fine, but when they came in the next morning it all happened again.
Turned out the issue was due to underling file access rights, once these were fixed all was OK. basically only the admin user had enough rights to populate a cache. Why this had occurred was still a bit of a mystery, but it is something you might see on any SharePoint installation. If you see a issue similar to this, the best option is to use Process Monitor to see if there are any file IO problems. This should point you in the right direction