J. Timothy King has a nice piece on the twelve benefits of writing unit tests first. Unfortunately, he seriously undermines his message by ending with this:
I too pity fools who donāt write unit tests. But writing the tests first? Not for me, too limiting!
During development, I like to keep the code as loose and maleable as possible. So often I find that I really need to play with code and experiment with strucuture and idioms, to make the code clear and concise. Sometimes its not clear how to do that until you are well into the development cycle. I often massively rip apart and rearrange lots of code because things I didnāt think of when first designing it became apparent much later.
I hate too much upfront design in the code, I think its unrealistic to rely on getting the design right with nothing concrete to validate assumptions. TDD creates all this inertia around the initial interfaces in the code, which start to affect the design. Or at least I think it does, Iāve never actually tried it. But thatās my fear.
And dammit, I just donāt want toā¦because Iām special and different. And youāre not the boss of me.
I do see tremendous value in unit tests, but I think TDD is going overboard. I could be wrong, who has time to try out every amazing new methodology?.
Hi, Jeff. Iām glad you wrote this. You are correct that I didnāt do much to sell test-first to developers who donāt do unit tests at all. Actually, I was writing primarily to developers who already do test-first. For the other group, youāre right that any step in the right direction is better than nothing. I myself did a lot of test-after-and-only-when-I-feel-like-it before I did test-first. And before that, I avoided automated tests completely.
So, yes, Iāve been there. I used to feel the way Damien does. And I tried a little testing, and a little more, and a little more, and now Iām a religious zealot. 
My one problem with test religion is that it can totally blind someone to design mistakes, or entirely unconsidered effects. For instance, if you donāt properly study the API of a library youāre using, or worse if itās badly (or un-)documented, functions can have side effects that the caller has never even considered, and thus code that passes all the tests still fails on other input. (Fuzzers can help where unit tests arenāt sufficient.) Or the tester will simply never consider the possibility that a closed file handle (as a simple example) could be passed to the function. The permutations of failure cases can easily overrun the ability of tests to find them; what they can do is to keep a coder from forgetting a case they found six months ago, and document it for others who wouldnāt see it.
(As for design mistakes, I must not be the only one to catch myself thinking āit passed all the tests, so it must be a correct solutionā¦ā at some point.)
Iām with Damien. My coding process is a lot more unstructured and involves a lot of changes, code refactoring, deleting, reorganizing ā¦
When Iāve tried test-first I find myself frustrated that I need to go back and āportā the tests to the new design. You could argue that I should be thinking about the design more before I start coding, but I find writing code, trying out little snippets and getting some real output helps me think through the problem in a way that I just canāt on paper or a whiteboard.
The times when test-first works is when Iām implementing something thatās already designed (porting a class from another language, implementing an algorithm, etc.) But that tends to be the exception.
The value is in having the tests. How you get them written is less important (IMO). Test-first/during/after? Its all good.
My focus in software development is always on economic value. Do you get more out of something than you put into it? For me (and each individual is, well, individual), itās not paying off. I wrote the unit tests for a 400 line class using NUnit. I spent about equal time writing the class and writing/maintaining the unit tests. I could have done as good a job with less effort without NUnit.
I believe this is true for me because of my coding style. Iām very careful when I code, making sure each line of code is exactly what I want. Never guess, never assume, always look things up if not sure, write lots of little test programs. This makes me a slow coder compared to others, but I think it pays back in the end.
Hopefully, the unit testing technology will improve over time so that I can find economic value in unit testing.
Ironically, the first half of Damienās post sounds like a TDDers description of why they TDD. Part of the beauty of unit tests is that you can refactor your code and still have confidence that it works. Otherwise, how do you know? Manual retesting? Stepping through the debugger? Also, using the tests to drive the design reduces the need for upfront design, not increases it.
One of the advantages of writing the tests first is that it guarantees that your code is testable. Itās far, far too easy to write a bunch of code, go back to write unit tests, and realized that everything is so tightly coupled that itās extremely difficult to unit test. Iāve seen it time and time again.
My current project, we did a good amount of upfront requirements development and design ā and have a good idea of what weāre doing. Usually weāre taking a day or so to code up what we wrote down, and then the unit tests, and thatās working pretty well.
Kevin,
"Part of the beauty of unit tests is that you can refactor your code and still have confidence that it works."
I think he was referring to low-level functions that may be radicaly changed or eliminated entirely and new ones created. Obviously a unit test is less than useless on each individual function in that case, but at a higher level youāre entirely correct, assuming you havenāt completely tossed out your API for a new one. If you go through a few iterations of that, TDD is just extra overhead guaranteed to waste your companyās time. (Itās like copy editing a first draft - you might not think of it that way because most professionals do their first few drafts in their head or on paper/meetings/email.)
In this case you would test with as much confidence as you have that code will last. Once you feel youāve started to settle on a stable design, work testing through it.
It all comes down to how testing can fit into your - and your teamās - code-writing style (assuming your style is useful and produces timely, working code), not making your process more inefficient by forcing tests into every step. Happily marrying the two is always the best solution.
Iām a big fan of TDD, and I donāt feel you need to be an experienced unit test writer before you can start writing test-first. Certainly there are advantages to having unit tests, regardless of when they are written, but I feel the extra advantages of test-first are significantly greater. Case in point: just yesterday I was writing, well, a bunch of things - but one of them was a hashset of a new, small class. I wrote a test to put an item into the set twice, then wrote the obvious code to make it pass - and it failed. I had forgotten (this is Java, fwiw) to write a hashCode() function for my new class. A moment or two and it was working right. Hereās the thing. Writing test-after, I wouldnāt have bothered. Iāve got a set, I can put things into it, what could possibly go wrong? And when the bug surfaced, Iād have been blindsided - had a very hard time finding this stupid mistake. With TDD, I make (roughly) the same number of mistakes, but I catch them a lot sooner and theyāre much easier to fix. Itās not a panacea, and not every programmer likes it, but itās mighty good for some of us.
Iāve just recently started trying TDD on a new subproject in an existing large project. I like the idea and Iām seeing some benefit from it. I even found a bug in pre-existing code almost right away, though itās something that would have turned up very quickly even without the unit tests, it was just code that hadnāt been used very much. But it was still a nice plus that I found it without ever having run the actual program, just the tests, and thus felt like a compile-time error.
The bigger problem Iām facing is that the program has a great deal of database and network access going on, and that stuff is hard to test standalone. To really do a lot of it effectively, there would have to be a whole test-only version of the database layer so that everything thatās calling into or out of it could be tested without an actual database connection. Writing that would be a large undertaking, and I am very skeptical that it would be worthwhile.
Nevertheless, Iām writing tests where I can, and often before code, but not always. Just about any metholodogy can be harmful when taken to extremes, and TDD seems no different. But thereās still a lot of good that can come from it, and a lot of good times to use it.
Actually I thought it was an atrocious article, full of sloppy thinking and riddled with condescension. These twelve supposed benefits of TDD range the gamut from ridiculously untrue (āunit tests prove that your code actually worksā) to personal preferences (āitās more fun to code with tests than withoutā), to unscientific generalizations from limited personal anecdotes or perhaps selected case studies (āitās faster than writing code without testsā), to solutions to problems few people encounter (āit virtually eliminates coderās blockā). He lost me LONG before the last sentence.
No, Iām with James Bach. There are No Best Practices; a āBest Practiceā is a rule devoid of context[1]. As Andy Hunt points out in his awesome presentation, āHerding Racehorses, Herding Sheepā[2], context-free rules are only the first stage towards holistic comprehension. Theyāre great for novices, but experienced practicioners violate these rules all time. In chess, for example, a beginner might learn ānever lose a piece without winning a piece of equal or greater valueā - and that piece of advice, if taken to heart, will improve their game a great deal. But itās not a silver bullet. In fact, their chess ability will reach an early plateau if they donāt start learning the situations in which they NEED to sacrifice material in exchange for position.
When I hear someone say, āwe should use TDD all the timeā, itās very clear to me that theyāre a software engineering novice. They havenāt quite understood in what circumstances TDD works best, which circumstances it has no effect either way, and yes, in which circumstances itās just a plumb waste of time.
Where does TDD shine? TDD works best in new āfrom scratchā development, in code which lends itself to well-defined, testable outputs (eg, text or API calls, not-eg UI or AI algorithms), in code which is algorithmically simple enough to run in sub-millisecond times, and in code which is largely self-contained (rather than dependent on a network, database, or large object model environment or outside framework).
Letās face it. These are very limiting criteria, and thereās a lot of real-world code which canāt be shoehorned to fit this model. Donāt think that because TDD worked so well for you in your college project or cool little app that you wrote, that itās the One True Way that All Development Must Be Done.
I have been written to write a developer test from time to time. Who hasnāt? Sometimes itās a unit test, sometimes itās an system integration test (testing the interfaction of several units). But I donāt write tests for everything. Like I said, some code I simply canāt test. Other code is so simple, Iād be wasting my time trying to test it. But beyond that, Iām in the practice of looking at the bugs that I myself, or my coworkers have written, and asking myself, āhow could this have been avoidedā? And the answer is almost never āa unit test would have caught thatā. I just donāt see where the ROI is in banging out all these tests that donāt catch anything.
And quite frankly, the few merits TDD has (āunit tests make better designsā) is a level of hand-holding I outgrew long ago. Iāve learned (mostly from painful experience maintaining other peopleās code) what code smells I should avoid. The question āhow should this function be used?ā comes far more naturally to me than āhow should this function be implemented?ā. I donāt need TDD to force me to think that way. I know to keep my objects small, to write referentially transparent code, to not write Rube Goldberg machines, to minimize my usage of if statements, to tell rather than ask, to avoid casting and pointer arithmetic, to avoid code which transfers ownership of objects, to use std::string and std::vector rather than fixed-sized C-style buffers, to avoid premature optimization and premature abstraction, to prefer composition and aggregation over inheritence, to be careful to distinguish between a count of bytes or a count of characters, and the zillion other things which practicioners of TDD eventually stumble upon.
For me, TDD zealots are just one of a long line of religious loonies knocking on my door, trying to prove that my way of doing things is irrepairably broken and the only path to salvation is Jesus, Mohammed, Krishna, Dianetics, or Unit Testing. And maybe thereās a good idea or two to be gleaned from all the mountains of bullshit. But you know what? Keep your crazy religion, 'cause I donāt need it.
[1] - http://www.rtpspin.org/Information/030327prs.ppt
[2] - http://www.satisfice.com/blog/archives/27
Yes, thereās a good reason if I ever heard one. When the customer comes into the meeting to ask for some silly feature he dreamt up, the proper answer is:
a. "Thatās the dumbest thing I ever heard."
b. "While we could look into it, it would mean a lot of lost time and extra costs, and my gut tells me the benefit is not worth the cost."
c. āWe have tests! Tests! One must not desecrate the holy tests!ā
And if it really is a great idea you wish youād thought of, youāll still want to work it in or shuffle things around so it can be worked in cleanly in the future, anyway.
Iām with Alyosha. 100%. Unit tests are for beginners. Beyond that theyāre a monumental waste of time and effort.
PS This is just an opinion. Itās mine and thereās no reason at all for it to be yours.
Alyosha said:
āAnd those people can do less with the wet-behind-the-ears TDD zealots with their condescension, their pity, their starry-eyed idealism and their āif youāve never tasted the kool-aid youāll never know what itās likeā.ā
But it is still true, if you never try kool-aid, you will never know what it is like. Every now and then you do have to taste the kool-aid to experience new things.
I am a Coke drinker. Back in the day when they had those taste tests between Coke and Pepsi, I always knew which one was which. So, I would always go to the Pepsi ones, and choose Coke. Why? Because that was what I liked. I didnāt want to give those Pepsi drinkers the satisfaction of saying I liked Pepsi more.
To this day I only drink Coke. Does it limit my options? Yes, half the restraunts in the US serve Pepsi (might even be more than half). So, I am disappointed half the time. Maybe if I wasnāt so set in my ways, I might be able to appreciate Pepsi every now and then.
I think you are picking a very hard line stance like I do with my drink choice. I feel that if you opened up a little, you just might be able to appreciate the āother sideā more.
You are definately allowed your own opinion of TDD, but from some of the stuff you said, you may not have found the correct techniques that make it successful for a great many developers. The reason I feel this way is because of these comments:
"When I hear someone say, āwe should use TDD all the timeā, itās very clear to me that theyāre a software engineering novice.ā
and
"Letās face it. These are very limiting criteria, and thereās a lot of real-world code which canāt be shoehorned to fit this model. Donāt think that because TDD worked so well for you in your college project or cool little app that you wrote, that itās the One True Way that All Development Must Be Done.ā
So, it doesnāt matter if you do it or not. You may write some great code that doesnāt need testing. I can appreciate that. I donāt test everything, but I am moving to more testing in my development. This testing is in real-world line of business applications (inventory, accounting, etc). So to say it only works in college projects is completely false!
Hi Jeff
I read a lot of your articles in CodeProject like:
http://www.codeproject.com/aspnet/CaptchaControl.asp
http://www.codeproject.com/aspnet/WebFileManager.asp
http://www.codeproject.com/vb/net/MhtBuilder.asp
Reading your topic about Unit Testing, I ask: Do you unit test asp.net applications? Do you use NUnitAsp? Or what?
Regards
Fabio Alves
Sadly, itās the āseniorā developers that are often most resistent to trying new techniques like TDD - years of doing things a certain has them set in their ways.
The fact is, thereās very little code that canāt be test-driven in some way. Iāve test driven complex web UIs, database code, OLAP code, Web Services. They may be trickier to get a handle on at first, but once you learn some tricks of the trade, itās totally doable. Maybe your coverage wonāt be quite as good, but as Jeff says, some tests are better than no tests. If not for you, then for the next guy who has to take over your work.
Itās amazing how often your here āTDD doesnāt workā, followed shortly thereafter by āoh, but Iāve never tried itā. Unfortunately, TDD is one of those things that you often canāt convince someone the value of until they do try it - people are very attached to the way they are used to doing things, and the advantages of TDD can be subtle and difficult to appreciate without trying it out.
Foxy - while I generally donāt think of unit tests as a ācustomerā specification (thatās what customer acceptance tests are for), unit tests are great specifications about what the expected behavior of the code is.
Have you ever inherited a bit of code, gone to change it, and realized you donāt actually know what the repercussions of your change would be? Having a battery of unit tests to validate your change is immensely valuable.
Fabio,
Not to answer for Jeff, but if you use the Model View Presenter pattern for testing, your UI should have almost no code in it that needs testing. Just a little info for you if you donāt know about it.
Jeremy Miller is a pretty good resource for TDD. Here is a post on the MVP pattern for ASP.Net:
http://codebetter.com/blogs/jeremy.miller/archive/2006/02/01/137457.aspx
Sorry, but Jeremy Millder is totally, totally out to lunch. He either works on extremely simple web apps, or he doesnāt know what the hell heās talking about. Either way, it is in fact possible to have a view that is a little bit more than trivial and needs to be tested. The totally contrived example he provides is near useless.