Beware fake code
Dry-run scripting options are useful but dangerous. This is an instance of something more general: fake code is dangerous.
By "fake code" I mean, approximately, code paths that only exist for testing. The two commonest examples are --dry-run code paths and is_for_test arguments to functions that modify their behavior. I'm not talking about beta environments, echo endpoints, or similar infrastructure. I think of those as authentic code; it's just that testing is what that code authentically does. Whatever your preferred metaphor, I hope the distinction is clear.
The problems with fake code are simple. First, it leaves real code untested. If your function does one thing in testing mode and something else otherwise, that "something else" isn't being tested.1 Second, the extra testing code can cause problems:
- It can require an import that causes any number of build errors.
- It can break the linter.
- It can break the build in other ways.
- It can hurt performance, sometimes non-obviously.2
- It can bloat your code base.
- It makes your code less readable.
- It can make you and your teammates think your code is better-tested than it is.
I once worked on a team that had implemented what I'll call "ghost mode" during a migration: callers to the new service specified at the API level whether they were making a real or ghost-mode call. Every API endpoint in the new service had different code paths for real and ghost mode. Rather than testing the new service, we were constructing an elaborate theater while also trying to do a migration, and leaving the first-order code largely untested. We wound up frustrated.3
Fake code is not, always and everywhere, wrong. But it is a code smell, to be avoided when possible. The basic technique for doing so is: there is usually a boundary separating the rest of the code from the disjunctive, fake-or-real part of the code. Instead of making the boundary a fake-or-real-code boundary, make it a boundary between function calls or modules, so that the "core" can be tested properly and authentically.
If you'd like further reading:
- I describe the technique a bit here...
- ...and in the original dry-run post.
- Martin Fowler's Refactoring is full of techniques for avoiding this and other problems.
- So is Michael Feathers' Working Effectively with Legacy Code.
Or, at least, it's not being tested with the testing mode. It's rare for such functions to be properly tested by other means, because one doesn't think to add testing arguments to functions that are otherwise being tested properly.↩
Logging, for example, is a common behavior of fake code. It seems harmless until it unexpectedly gets called many times and degrades performance, causes an out-of-memory error, or hits a syntax error on unexpected inputs.↩
These teammates, by the way, were strong engineers who cared deeply about shipping safe code for their customers. What I'm calling "ghost mode" had, for mysterious reasons of collective behavior, taken on a life of its own. It doesn't look like a good idea when you describe it clearly out loud, but once an idea has taken hold, it's rarely described clearly out loud. Describing things clearly out loud is a superpower.↩