The Desire for 100% Coverage

wrapped kitty
Credit: flickr / sumptinelse

I recently handed in an assignment on testing. It was fairly simple, and I expected it to take just a couple of hours and not teach me anything. That was not  the case. I really think that every situation is an opportunity to learn something (even if it’s not what you think you will).

Lesson 1: JUnit 4 is awesome. Parametrized testing is a huge improvement.

Lesson 2: Twitter is an incredible source of knowledge. OK, I knew this, but a guy who I have books by responding to a random question is incredibly cool (or, I’m a giant nerd. Whatever).

Lesson 3: It’s important to design your test suite well. We started with 20 test cases, the purpose of which was to test one function, but 5 of them were really testing the constructor. It was clear to me that by well designing a test suite (from the spec, not the code) I could get better coverage with half as many test cases (of course, with parametrized testing this is not such a terrible thing).

Lesson 4: Coverage is seductive. I got a little caught up trying to get 100% coverage, adding tests to exercise paths in the code, rather than aspects of the specification.

Lesson 5: No amount of testing will fix bad design. After I’d written all my tests, I noted that it wasn’t really necessary to test simple get and set methods and lack of coverage there wasn’t really an issue. Then, I looked more closely and realized that those methods (that I hadn’t tested), however trivial, shouldn’t be there at all. The class was modeling a date object, and the tests were to make sure that invalid dates couldn’t be generated. But, using these simple set methods I could easily make invalid dates. Really, the class should have been immutable – but no amount of testing will point that out.