Paulo Morgado

# re: Test Driven Development - Is it good or bad?@ Tuesday, April 03, 2007 1:14 AM

I agree. I've worked in several strong agile-process groups that have created customer-accepted software that CUSTOMERS raved about. It's really customer-design-driven software, all tested with quality unit tests written immediately after development. The design thinking was natural, logical, optimized, contract-based, clearly understood, and documented. And the maintainence costs were minimal. And it was great personal fun to rapidly solve customer-related design problems every day.

Now recently I have experienced test-driven-development - the real-evangelist-pushed-thing - for about a month. I discovered that the amount of test development time and test code greatly exceeds that needed for the application code itself. And the amount of features becoming periodically available to the customer in a timely fashion needed for good adaptive feedback? Very few, just a trickle. This very slow completion of features obviously results in very high development costs. And with TDD there is also a terrific  explosion of the number of classes used, creating technical confusion for developers and a big maintenance headache for new support staff. And - as is typical for TDD projects - the code documentation for important non-obvious explanations is essentially non-existant.

Someday soon, companies will wake up and stop throwing their money - and their developers - away on TDD. And remember: XP came from just one guy writing in Smalltalk about 12 years ago whose first major XP project FAILED at Chrysler Credit corporation; the highly successful Agile process arose as a response to XP from hundreds of professionals 6 or 7 years later who wanted something much more effective and practical than XP.

Dwight Spencer

# re: Test Driven Development - Is it good or bad?@ Tuesday, April 03, 2007 6:27 AM

this is getting into a nasty habit :) again, i disagree. Sure, you can "auto-generate" your tests with visual studio tools (i recall having seen it somewhere, though i haven't tried it) but the idea of using tdd is that you're only writing the needed code for a test to pass. if you do that with that new methodology you've just invented, then you should be ok. Now, the problem is that when you write your code first, normally your tests won't be able to test everything you've put in the method. to do that you'll often (if not always) need to perform some refactoring. if you do that (I really doubt that anyone does it!), cool. if you don't, you end up with the same low quality code you're saying most guys end up with.

so, to me, TDD, when used correctly, is great! the problem is that it's too hard to find someone that is willing to spend the necessary time on it. Unfortunately, I must leave now (vacations, remember? :)), but I'll be coming back to add more rants on this :)

Luis Abreu

# re: Test Driven Development - Is it good or bad?@ Thursday, April 05, 2007 10:37 AM

I said i'd be back :)

I didn't said I was using TDD. I said I've tried to use it(once or twice) in the past and I really liked the results. I do like the wikipedia definition of TDD:

http://en.wikipedia.org/wiki/Test-driven_development

Since I don't want to get stuck in another discussion about the meaning of the word driven, I'm going to leave this thread while I can :)

Luis Abreu

# re: Test Driven Development - Is it good or bad?@ Friday, May 04, 2007 6:25 PM

Paulo,

I agree about people who use buzzwords without knowing what they mean. But I disagree about TDD.

I've been using TDD with some success, on and off, for the past few years (pretty much since I learned about it). I've used it on everything from web services to web applications and console applications.

Of course you need to do up-front design to one degree or another, and requirements must be clear enough; this can be done iteratively. But once the requirements have shown you what needs to be done,  and the design has shown you the shape of your code, I find it very worthwhile to create unit tests. These tests define what behaviors are important to me. These are the things that my code will have to do before it can be considered "done".

The unit tests are constructed so that they fail first. This is done so that you know that the unit test _can_ fail - it's no good writing a unit test which cannot fail. The code is then written to cause the unit test (and all of your other related unit tests) to succeed. The test has now tested the code. You can then safely refactor both the code and the tests, because they test each other. If you screw up the unit test, it will fail, as it will if you screw up the code.

The result is an executable statement of the correctness of the code; a statement you can test on every check-in, or every nightly build, or repeatedly during the day. The feedback is good for your self-confidence ("it works, it still works, it's _still_ working even now! Damn but I'm good!)

Now, I'll admit that I don't have any numbers as to how this affects code quality. I'm keeping an eye on this on my current project. I want to see if there's any correlation between the numebr of defects reported vs. whether or not the code was developed via TDD. I also want to see if it's the case that the defects are due to me not creating the unit test which should have caught the defects!

John

John Saunders

# re: Test Driven Development - Is it good or bad?@ Tuesday, May 15, 2007 10:17 PM

When speaking to an executive, do you really tell him that the studies and case studies he's read from reputable research firms are all lies? The studies that associate test-associated development with higher quality? There is so much evidence for the benefit of TDD (disgarding objections the exact meaning of the terms used) that this post can't be considered well-informed.

Jeffrey Palermo

# re: Test Driven Development - Is it good or bad?@ Wednesday, May 16, 2007 2:05 AM

Indeed, you should copyright it

Scott Bellware

# re: Test Driven Development - Is it good or bad?@ Wednesday, May 16, 2007 6:28 AM

This post to me sounds more like the rant of a misguided developer that has failed to grasp onto TDD (or any variant there of).  I wonder if the auther has really given testing a solid try?

Derik Whittaker

# re: Test Driven Development - Is it good or bad?@ Wednesday, May 16, 2007 9:26 AM

I've been doing TDD for about 2 years now, and at first, I did have the sort of class explosion you mentioned.  It was only after I put it together with the GoF, Fowler's PoEAA, Evans' DDD and eventually behavior-driven design did good designs emerge from my software and tests.  I had to completely relearn how to design, now always doing the simplest thing that works and doing a sort of JIT design.  Every test I write now centers around behavior and functionality.  TDD is a paradigm shift in your development practices, and takes time and effort to master.

As for using TypeMock with unit tests, I had the same experience with it as I did with typed DataSets.  Easy to use the first time, but an absolute PITA to maintain.  I always felt my tests should be concerned with the public, observable behavior (or state) of a module.  Any time I started tinkering with testing the internals, my tests started breaking much more often.

I can't speak for all TDD practitioners, but all I need to do is look at code I wrote test-after (or test-never) and code I wrote test-first to know I could never go back.  I think your mileage may vary on TDD, but at least we can all agree it's better to have code with tests than without.  Now back to Michael Feathers' book...

Jimmy Bogard

# re: Test Driven Development - Is it good or bad?@ Wednesday, May 16, 2007 9:42 AM

I've been doing TDD for about 2 years now, and at first, I did have the sort of class explosion you mentioned.  It was only after I put it together with the GoF, Fowler's PoEAA, Evans' DDD and eventually behavior-driven design did good designs emerge from my software and tests.  I had to completely relearn how to design, now always doing the simplest thing that works and doing a sort of JIT design.  Every test I write now centers around behavior and functionality.  TDD is a paradigm shift in your development practices, and takes time and effort to master.

As for using TypeMock with unit tests, I had the same experience with it as I did with typed DataSets.  Easy to use the first time, but an absolute PITA to maintain.  I always felt my tests should be concerned with the public, observable behavior (or state) of a module.  Any time I started tinkering with testing the internals, my tests started breaking much more often.

I can't speak for all TDD practitioners, but all I need to do is look at code I wrote test-after (or test-never) and code I wrote test-first to know I could never go back.  I think your mileage may vary on TDD, but at least we can all agree it's better to have code with tests than without.  Now back to Michael Feathers' book...

Jimmy Bogard

# re: Test Driven Development - Is it good or bad?@ Wednesday, May 16, 2007 3:01 PM

I'm still out on true TDD (writing the tests first) but I do have some numbers from one of my previous projects. The project was an ASP .NET web application with about 20k lines of code or so. I was the main developer and there were two testers executing the test plan. I utilized NMock2 for mocking and csUnit for my unit tests. Part of my Nant build script was to run unit tests and create a code coverage report in NCover. There were a total of about 300 bugs logged against the project (this is an estimate, the project had 360 bugs total but I don't know how many of those were duplicates or pre-existing before I started working with the code, but I think 50-60 of those is a fair estimate). I had unit tests on both the database and the business layer, for a total of 107 unit tests with about 90% code coverage on those layers (according to NCover). For the complete life of this project, every single one of those 300 or so bugs was logged against bad requirements or UI issues. There were no bugs logged at any time against any code that was covered by a unit test. I did NO modifications to ANY code that was covered by unit tests as a result of a defect (there were two requirements changes in the middle of the project that required code changes but these caused me to change the unit tests also). My point is that proper use of unit tests to ensure that objects behave the way they are supposed to will have a huge impact on the quality of the software that you are creating.

Jeff Tucker

# re: Test Driven Development - Is it good or bad?@ Wednesday, May 23, 2007 5:49 PM

You should design to test! Not being able to test in isolation is a sign of a poor design due to high coupling and low cohesion. (IMHO) Following the principles of good design will lead to testable code. Apparently you have not had an "Ah, ha!" moment when a unit test catches a bug introduced by another bug fix that is so obscure you would swear no test plan was going to catch. Typically I find these when I write cross-check unit tests. I've had a few that would have made it to production and cost my company way more than the time it took me to design for testing and to write the tests. One other note, only test what is important to test! Automatically generating a unit test is worthless. I understand how everyone's opinion is based on their own experiences though.

Mark Lindell

# re: Test Driven Development - Is it good or bad?@ Monday, May 28, 2007 3:13 AM

@Dwight

Same can be said of procedural vs. OO. It depends on the people, not TDD.

Adam