Nate Meyvis

The encapsulation-performance tradeoff

Maintaining proper encapsulation is great, but can make it harder (or impossible) to optimize performance. Here are some examples:

  1. A function or service has the ostensible job of returning all items that satisfy criterion X, but all its callers (or all its current callers) only need some subset of those items.1 Because it's faster to retrieve that subset, and because sending back the smaller payload also saves some resources, it's tempting to just make the function a subset-retrieving function, even if its name or documentation says otherwise. You must therefore choose between two implementations, one of which saves resources and the other of which is more intelligible, extensible, and nameable. The latter preserves encapsulation much better.2
  2. There is a genre of function that works toward some goal, but in a short-circuit-friendly way. Talk to your preferred LLM about "anytime algorithms" or why engines have separate functions for any-hit and closest-hit algorithms in ray tracing. It's great fun, and builds intuition usefully even if you aren't in game development or a similar field.3 These functions tend to require callers to know a lot about what a return value means (and those values often involve many different concepts), to know about several functions that do similar things, or both. Either way, encapsulation, or at least complexity management, suffers for the sake of performance.
  3. Poor performance often comes from requests that need to access and somehow merge results from several data sources. Some of the resulting performance degradation is just statistical (e.g., for operations that are blocked by the slowest of calls to N databases). Much of it, though, often comes from things we do to preserve encapsulation. Many of these queries involve special cases where we could call into the data sources in clever orders or for subsets of the data, but in ways that are hard to generalize and describe. The more optimized data-management patterns tend to sacrifice encapsulation.

We talk a lot about trading off some system performance characteristics against others: the space-time tradeoff is famous. There's a lot less discussion about about the tradeoff between system performance and code quality.4 This is largely healthy: most code-quality improvements to most codebases would have either no effect or good effects on performance.5 Sometimes, though, there really is a code-quality / system-performance tradeoff, and these tradeoffs tend to involve encapsulation. We need to understand these tradeoffs, or at least admit that they exist, if we want to make good decisions about them.

  1. I briefly discuss this example in a previous encapsulation post.↩

  2. One common approach here is to have the function accept a parameter that indicates which mode it is to operate in (return-all-results or fast). This isn't always a bad idea, but it doesn't solve the problem. You can make it a boolean fast_mode parameter, but then you have a parameter with a secretive name, and you've broken encapsulation again. You can make it a params parameter (and if you think I'm joking about this, you probably haven't worked on a lot of huge legacy codebases), but then you still have to choose between (i) weird secret names or (ii) literally describing what optimizations will be made (and people almost never succeed at getting those literal descriptions right). Then you need to actually implement the function, and you'll usually face a tradeoff between (i) just having two branches with very different implementations (which preserves performance benefits, but now why are you making your callers pass a "which function I'm calling" parameter instead of just having them call different functions?) or (ii) having more shared code in the implementations, which generally sacrifices performance benefits that were the whole reason to have the hard-to-describe version of the function. I hope, one of these days, to write a post about the whole business of params or *_mode arguments to functions. (Or maybe this footnote is that post?)↩

  3. I'd be embarrassed to admit how much of my programming knowledge is at least indirectly caused by my feeling insecure about how I stack up against game devs.↩

  4. The biggest exception here is the case of incurring tech debt to speed up development: people talk about this all the time. As I've discussed before, however, I think this situation is generally misunderstood and mishandled.↩

  5. This is an instance of the general principle that, until code quality is very good, improving a codebase along one dimension tends to improve it (or at least not degrade it) along every other relevant dimension. But that's another post, or perhaps another way to put a central point of this post.↩

#encapsulation #sociology of software #software