Overflow from the VSTS demo last night

Acouple of other things that I neglected to blog about last night:

- A while back, someone from the Visual Studio team posted a request forpostcards from Visual Studio customers (the link escapes me now). The deal wasthat if customers sent in postcards or notes about how they used Visual Studio,they’d be put up in a hallway to help remind the team who they’re buildingsoftware for. As we were walking through 41 last night, I noticed the displaytaking up a good portion of a hallway on the 5th floor. I thoughtthat was rather cool.

- Jason and Tom definitely went above and beyond the call of duty in giving usthat demo last night. We didn’t get back to MS until about quarter tonine, and they went strong until 11pm. Tom had just flown back from a vacation inEurope and was completely jet lagged, but didn’t balk for a second whenthe opportunity came up to demo his work. To me, that shows a tremendous amountof dedication to customers and pride in one’s work. I have a hugeamount of respect for that.

Thoughts about Burton itself:

- I really like what they’ve done around unit testing private members. Unittesting non-public methods necessitates a tradeoff. One hand, you can includeyour unit tests in the project and declare any members you need to test as “internal”.However, this requires you to conditionally compile out your unit tests. If youunit tests depend on configuration files, you need to take additional action toexclude those files from any installation packages you build. Depending on thetools you’re using to create your .msi’s, this can get complicated.On the other hand, you can externalize your unit tests into a separate project,but then you need to write a bunch of messy reflection code to get at theprivate methods you’re trying to test. I’ve done both, and neitheris particularly attractive.

VSTS 2005 solves this problem very elegantly. James Newkirk sort of hinted atthe approach in a recentblog post. The approach that Burton actually takes is a bit cooler thanthat, because they leverage code generation to create a strongly-typed “accessor”class to access private members as if they were public. You can then invokethese members in your unit tests without writing any reflection code yourself. Thus,you can get the best of both worlds – external unit tests plus easyprogramming model.

I didn’t get to see if the code generation system was smart enough tokeep pace with a class definition as it evolves. It would be very nice if newpublic members were automatically added to the generated class when I add theirprivate equivalent to the real class definition. I should never have to alterthe generated code – if a class is generated, the code generator shouldown the maintenance of that class from that point forward.

 

#1 Ryan Milligan on 6.23.2004 at 6:59 PM

I'm thrilled to hear that you're so excited about Burton and test generation in general, and the private accessor framework in particular. Despite the overwhelmingly positive tone of your post, I know that, in the software industry, even the most glowing reviews usually come with an "if only..." attached to them, so please, if you have any "if only..." requests for this feature, pass them along to me or Jason as soon as you can so we can give them the consideration they deserve.As to maintaining the accessors as you author the code, I agree that this would be great. The solution that we have at present is that every time one generates a test for a private member of a class (or for a private class, or for a public class with only a private constructor -- basically, any test that will need to call into a private accessor), the system will re-generate the private accessor for that class from scratch, over-writing what we did last time. So, you don't have to worry about needing to update it manually.Having considered a bunch of ideas, refreshing the accessor is likely to remain an explicit gesture, rather than something that happens under the covers while you edit the code under test.Of course, we would be interested to hear any opinions on the matter, so feel free to e-mail or post any questions or feedback that you may have.-- Ryan MilliganVisual Studio Team System, Test Generation