The risks with TDD

Lately there has been a lot of debate about Test Driven Development. For me I started consider the usage of TDD when Rich Hickey formulated it as "you wouldn't drive a car by crashing into the guard-rails". I would change the metaphor slightly to mean "you don't design cars not crash into the guard-rails".

Whether you hit the guard-rails or not is not a decent measure whether a car is good or not. The higher idea of a car is not something that don't crashes into the guard rails. The higher idea of a car is more of something like a tool to transport yourself and some more people and stuff quickly, safe, cheap, and in a joyful manner from where you are now to some place farther away than you could comfortably walk. No guard rails involved.

"But the test-cases is a form of strict requirement specifications!" you say. Of course they are. It's necessary to specify requirements in some form, to detect logic inconsistencies and communicate the design (to make others able to criticize it). Write test cases is a way of doing this, but usually not the best way.

Another problem which is perpendicular to get a logically consistent view of the problem is the problem of avoiding/catch trivial implementation mistakes early. By trivial I mean something like a comparison with > instead of >=. These mistakes usually don't really touch the higher idea of the program, but are of course crucial to get correct. Handmade test cases, asserts, contract programming and even generative testing are great tools for this. Don't mix solving these bugs with detecting higher level logic inconsistencies of your code. Trivial bugs are relieving to correct, but never of any significant value for the program. It's not the hard problem.

My baby message-queue example
The risk with test-driven development is that you start to code before you know what to code. It is of course a good way to get yourself out of analysis paralysis, but in my experience, there are to little 'analysis paralysis' compared to 'cowboy coding that gets impossible to manage and extend later on'.

Let's say you are in need of some kind of messaging system. Your system is spread over several computers, and you have already utilized ad-hoc socket stream formats and various REST-APIs in an unsustainable way, just to get the whole thing going. You shrug at the idea of expanding the system. Wouldn't it be nice to solve this problem once and for all?

Let's say we try to formulate some test cases about how this messaging system could work. Obviously you'll need some way to connect the program to the messaging system bus, and some way to send and receive messages.

It could be a test-case like this:

(let [connection (connect "some-message-bus")]
   (is (up? connection)))

and

(let [connection (connect "some-message-bus")]
   (send! connection "hello")
   (is (== (receive! connection) "hello"))))

This is great! It is simple! I can see that there's a function connect that need to be implemented. This needs to return an object that, given to a function up? returns true.

I also want to be able to send and receive things on each bus.

This is actually highly enlightening. It's a minimalistic interface but it captures a small API. I know what I want and we can even discuss parts of the design given these small lines of code.

Of course there's a ton of various things I need to add - some event loop facility, networking support to be able to connect remote computers, reliance of the message queue, potentially other behaviours, but I could very well code up something that would work quite OK in process just given this simple test. Almost like magic!

Never underestimate the real problem in distributed computing
It could easily be the case that we actually would be in need of quite a different solution than the one we thought we needed - maybe we didn't parallelize the problem enough and did all the socket shuffling in vain (I've seen that more than once). And how do we handle that connections are lost? Can we unsubscribe a bus? What happens if we subscribe to many queues. What if servers are down? New servers join the cluster? Authentication? Am I the first person in the Universe facing this particular problem and is it unique of its kind? (that last one was a rhetoric question).

Depending on the specific problem the computing cluster should solve, there are many different ways to approach it. My small TDD approach has the insidious side-effect that it actually makes me narrow down my view of the problem way too early. It's very hard to kill code you wrote start over, almost from scratch. Good test takes effort to specify (and rightly so). To refactor tests probably takes even more effort than to refactor code.

The TDD way to navigate through the large space of solutions, can easily get stuck on a local maxima.

I can do nothing but approve Rich Hickeys idea of "hammock time". You really have to understand and be able to keep the whole problem you try to solve, and as many of its parameters and quirks. One good way to do this is really to try to solve the problem (preferably on paper) and after that see how others did solve similar problems, code up a small prototype, see how others code would solve the same problem, make sure you know how to the whole stack would work. When this work is done: TDD would work just fine.

TDD is a great scaffold, but a most often a really shitty sketch. Actually test cases have too high fidelity and is to slow to write and change, and you can mistake them for "real test cases that should be used to test the final solution on".

Test cases is not a good blue print either - test cases does not say how the solution should work, only how it should behave. If you really want to code this way (which is really powerful), use a declarative programming language, like datalog instead of abusing your precious brain time with being a manual  compiler for your own, made up, non-standard, most likely hard-to-understand declarative programming language. (Those are more common than you might think).

In summary
The major risk with TDD is that one get carried away on the wrong track, starts to solve some problem, get some kick from passing tests, continues in that direction for more passing tests kicks, and never really takes the opportunity to really think and reason systematically about the whole problem one have at hand.

Don't be mean to yourself and your friends, always think hard on your problem and possible solutions AFK before coding. If that is not possible, something is wrong, and your code will likely be as messy as the problem it tries to solve.

When you know exactly what the outlines of the problem are and how you want to solve this problem - then TDD is one of several tools to get your code super duper great. Use it accordingly. Thanks.