A kernel unit-testing framework

(oh, how times are changing. I am forced to rebuke myself for use of 'his' in the parent comment!)

I worked in many projects that had strict 80% coverage requirement for new code. There was a fair amount of bullshit tests, but:
1) Even bullshit tests periodically helped.
2) The advantages of non-bullshit tests far outweighed the occasional discomfort of having to write BS tests to tick the right box.

Nothing is perfect, but with tests it's pretty much impossible to have too many of them.

> If they're forced to write a test, the chance is very low that the test will do much more than validate what the programmer already knew about his code

The point of requiring tests with a patch is not to catch bugs in the patch at the time of submission. Those tests are to catch people breaking that code in the future. Thus, even "does this feature work at all" tests are useful in the long run.

For a very long time Firefox has had a policy of requiring tests with patches, or else an explanation of why a test is impractical. I was involved before and after the policy was introduced and that policy has been extremely valuable.

Your comment reads like you think that one bad experience invalidates the whole practice. Unit-testing, as an approach to rigour in software engineering, is not easy but has enormous payoffs. As roc points out, preventing regressions is a great reason to have a suite of unit tests that a developer can use, on her own machine, before committing, to make sure that there aren't any unintended consequences to the code changes she's making. This has a second benefit in having a quick cycle time -- being able to fix bugs and regressions while the intent of your code is in your head, and you find another way to solve the problem at hand. (We get for free a third benefit to an extensive unit test suite: other developers have said what they intend their functional code to deal with, recorded as examples of positive and negative behaviour in their test suite.)

Read the articles [1], [2], [3] and join to community of practitioners.

Let's not bike-shed different flavours of automated testing. Let's not re-invent something well covered by history. Consider Chicago-style "red-green-refactor" to write failing test cases, fill in merely enough code to pass the tests and then to reshape your code structure to something you and other people can maintain or London-style 'design the interactions between components in your system': test their interfaces with contracts, and use your own inteface contracts with 'test doubles' of external systems (i.e. a mock evolved into a wrapper around external libraries, with a standardised interface[4]) to avoid your unit tests escaping their unitary boundaries.

1: https://github.com/testdouble/contributing-tests/wiki/Lon...
2: https://github.com/testdouble/contributing-tests/wiki/Det...
3: https://github.com/testdouble/contributing-tests/wiki/Don...
3: http://codemanship.co.uk/parlezuml/blog/?postid=987

>Tests that are written half-heartedly for the most part contribute to brittleness and inflexibility more than anything else. If someone is motivated to write a good test, they will do it by default. If they're forced to write a test, the chance is very low that the test will do much more than validate what the programmer already knew about his code[.]

Let's reconsider your first sentence: were we to add some training on test case design to our edict mandating test cases, what might happen? And those brittle and inflexible tests will need reliability designed into them or they don't warrant the trust we grant our automated test suites. A significant part of test case lifecycle is to cull the redundant ones -- something which happens naturally when unit-level test cases are the 'record of intent' from the author of the functional or production code. That's to say that irrelevant functionality will have failing unit tests, and nobody would advocate writing code so that these irrelevant tests pass.

Second sentence: People don't know what good code, good design or good tests look like without training and connection to a community of experts. We can work together to grow a pool of knowledge about this.

Third sentence: We can use this record of assumptions about the code as examples of intent. On top of that, we can train people to consider the equivalence classes of test inputs which cause the software to do the same work and arrive at the same output -- and have tests for these. We can also train people to consider and test what happens to bad input data -- which usually results in systems built defensively to reject bad input and to recover from unintended behaviour.

It sounds like you had a bad experience, which sucks, I'm sure. There are ways to pick yourself up and to practise better unit testing.

K3n.

If your project has robust code review process then bullshit tests will have a much harder way in. I require tests to be present for new code while I'm doing the code review mostly because it shows that the developer thought about the API, about what inputs are valid and what inputs are invalid and how the code should react to those (also: failure modes and error handling). I review the tests just as hard as the actual code (though it's often tempting to skip this part or make it less thorough).

And as others said, even bullshit tests have value sometimes.