Konubinix' opinionated web of thoughts

Consider Tests as Being an Instrumental Objective

Fleeting

When we write tests, it is often the case that we have indicators telling how well our test suite is behaving.

Also, those indicators often come with thresholds discriminating “good” (colored in green) from “bad” (colored in red).

People often feel those thresholds as coercive stuff that needs to be satisfied.

I think this is a case of a measure that became a goal. I would suggest then to consider:

  1. what was the final objective to fulfill when setting up the indicator in the first place?
  2. does satisfying the intermediate goal of reaching the threshold keep people in the direction of the final objective.

If the threshold stops being appropriate, then just drop it. If otherwise:

  1. all things being equal, satisfying the threshold seems more appropriate that not satisfying it,
  2. and you cannot setup another mean of fulfilling the final goal,

Then, even though you might consider this a burden, it might well be a good objective. But the fact you consider it a burden in the first place might say that you don’t agree with one of the above statement.

In my mind, the final objective behind writing tests is to ensure the code is as robust as you wish it is and detect regressions in a deterministic way.

A pretty common example of such objectives is the code coverage. People tend to try to blindly increase the code coverage without consideration for how appropriate the test are, with regard to this final objective. We sometimes end up with pretty useless mock object with the sole purpose of entering obscure corner case of obscure functions in a very artificial way. Chances are that when this code is reached in real life, it would be in other circumstances and then those tests would have given a false idea of robustness.

If you realize that you just try to make the indicator « happy » but don’t think about the robustness of your code will writing tests, I warmly suggest you just stop writing tests.

Notes linking here