On dependencies in small projects

Dan Luu Tweeted:

The places where I most regret taking a dependency are often very low effort projects (like the SSG for my blog), where the complexity of the dependency wasn't worth it.

I've chewed on this for three days. It feels like a viewquake. Some reflections:

  1. The survivor bias in the sample of "here's a thing I hacked together on a weekend" is obvious but easy to overlook.
  2. Also obvious but easy to underestimate: there are strong social norms encouraging people to exaggerate (i) how fast they've shipped small projects and (ii) how well those projects work.
  3. A lot of small projects, certainly including those that rely on a dependency the way a personal site relies on a static site generator, do not survive. I have lots of first- and third-personal evidence here.
  4. Most dependencies that are maintained well enough to avoid crippling bugs also add features regularly. Over time, and often despite maintainers' best efforts, this can make basic "plug and play" usage harder.
  5. Small projects, not just in software, are harder to pull off than they seem. So here's another way to frame the point I take Dan to be making: Let's say a project seems small. You do it without taking on a dependency, and it's basically a success, but it took more effort than it seemed it should. So we have extra effort to account for, and at least two possible causes: (i) the actual intrinsic difficulty of the project and (ii) the fact that you didn't get a boost from the dependency. Because we underestimate (i), it's natural to overestimate (ii).

Home page