Lambda
Here is Patrick McKenzie:
I have decided since no one popularized word yet: lambda is the general factor of performance above SOTA LLM baseline.
Notes:
- I am primarily posting this because a term for this is so useful inter- and interpersonally.
- Wins over replacement is a pretty good analogy for this, but the replacement-level players are becoming arbitrarily numerous and inexpensive.
- None of this is to say that lambda is the only way to evaluate one's skills. Organizations will use sub-LLM-level outputs for a long time, and of course there are all sorts of reasons to value one's abilities that have little to do with an LLM baseline. (I enjoy studying for trivia competitions very much, and I will forever be much worse than computers at them.)
- The wins-over-replacement metaphor suggests that doubling one's lambda might be much more than twice as economically valuable; a 2*N-win player's open-market salary is generally more than twice that of an N-win player.1
- For any given (positive) amount of lambda you might have, a 1% increase in absolute ability will translate to a lot more than a 1% increase in your lambda.
Finally: Dan Luu's recent piece on ambition made a big splash (and quite justly). As is so often the case, actually reading it has large rewards, in part because perhaps the most interesting bit comes at the very end. Here's the end of footnote 7:
Perhaps almost all or perhaps even all of the software skills I have will become irrelevant in the near future. But we're not there yet. At the moment, programming skills seem higher leverage than ever before. There's something a bit funny about the current moment, where it seems like having the skills that models are replacing makes one much more effective than people who are trying to use models without having those skills. I like Max Bittker's framing that models are very good at counterfeiting and the level of expertise it takes to distinguish a counterfeit from the real thing has been increasing as models improve.
As is so often the case these days, this is a very optimistic conjecture with respect to certain skills and a very pessimistic conjecture with respect to others.
Or so some combination of LLMs and FanGraphs tells me.↩