Effective Tests

Another blog post based on my Testing Anti-Patterns Talk

Effective Tests

One of the biggest things that helped me understand TDD was to write out what benefit I gained from having tests. Here’s my basic “testing mission statement”:

Tests reassure me that I didn’t break anything and when I do break things they help me find the problem.

If tests don’t reassure me that everything is okay or they don’t help me find a problem when something breaks they aren’t effective. I believe effective tests have two things in common; they are trustworthy and they are simple.

Trustworthy

A test is trustworthy when it fails when code is broken, passes when code is working, and tests what it claims to test. I’ve seen a lot of tests that aren’t trustworthy. The common symptom of an untrustworthy test is that you’re new on the project, you run the tests, and Test ABC fails. You ask someone else and they say, “Oh yeah, that one. It fails sometimes just run it again to see if it passes.”. This is the test that cried wolf. It complains and complains when nothing is wrong and when something is actually wrong you ignore it.

There are other kinds of untrustworthy tests. There’s the test that only passes when you are on the corporate network. Or the test that only passes when it should really fail. Or the test that says it verifies that 144 widgets fit in a case but actually verifies that 12 whazzits are $19.99 with tax.

Unreliable tests can’t be trusted. They don’t reassure me that everything is really okay and they don’t help me track down my bugs. Many people have been put off testing all together because they’ve been burned one too many times by an unreliable test.

Simple

Tests are only able to help me write better software if I run them consistently. When its simple to run tests I’m more likely to do it. Ideally all you need to do to run a test suite is type a single rake command.

Effective tests also help me debug problems when they fail. A simple test that’s clearly written with no ‘clever’ tricks makes it easy to figure out what failed and follow the problem back to the source. A complicated test with multiple layers of indirection or nesting can make it nearly impossible figure out what went wrong.

Also effective tests are simple to write. Testing shouldn’t feel like a wresting with Medusa. It should feel as easy as writing the implementation. If the testing is painful there’s a good chance the implementation is painful too. In fact, ‘hard to get under test’, is a code smell.