The bitter lesson resists memeification
Here's the bitter lesson:
"In the long run, general approaches that scale with available computational power tend to outperform ones based on domain-specific understanding."
It seems to me that this lesson is:
- Important;
- Counterintuitive;
- Little-known, even in software, relative to its importance;
- Hard to describe beyond the definition, including by metaphor.
My evidence for (4) is that I can't think of a good metaphor for the bitter lesson, and I dislike all the metaphors AI tools generate when I ask them. Metaphorically or not, don't think I've ever successfully explained the bitter lesson to someone who didn't already know about it, though I've rarely tried. (And even here, I'm just relying on that Wikipedia link.)
I suspect that it's under-appreciated precisely because it has, so far, resisted all attempts at snappy metaphor or transmission-apt description. (If you know of one, please let me know!) If you are looking to expand your conceptual arsenal, you're probably more likely ignorant of the bitter lesson than comparatively important lessons that are more transmissible.
Some might say that the bitter lesson is important only in a certain niche, but I don't think that's right. Many of us have to choose how much and explicitly to encode domain knowledge in systems, or at least to bet (implicitly) at how well different third-party tools will encode and extend knowledge. Moreover, AI is big enough that there's general interest in what works, and how well.
So: if you don't know what the bitter lesson is, it's worth knowing about. But you might find that making it intelligible, beyond reciting a definition, is a tough and slippery thing to do.