Notes on AI "cathedrals"
Here is Zac Hill on software "cathedrals" (via this very flattering Matt Glassman post). Summary: this think tank Fellow is an absolute crusher with generative AI.
It's a good read. Some thoughts:
- I suspect that the shape of work matters here. Emmet, it seems, makes a lot of research dashboards and reports. A knack for making skills (or any Skill-like AI components) is particularly useful in this kind of project, because there is a kind of broad domain expertise that automates particularly well this way. Knowing where to get the data, how to clean it, what to look out for, what kinds of analyses to do, how to display it... it sounds dismissive to call anything a "bag of tricks," but there's a kind of practical expertise that involves a lot of (i) medium-tricky mini-expertises and (ii) knowing when to apply them. I mean this the opposite of dismissively. (I'm optimistic that, for example, we'll see great AI-assisted work in philology and classics for analogous reasons--but that's another post.)
- Simon Willison is a good pure-software example of someone who's disciplined about formalizing chunks of knowledge and making them AI-legible (see, e.g., this project and his early discussion of the Skill structure).
- Emmet-style AI-mechanization of expertise scales better, so far, than the AI mechanization of building and maintaining large software systems. Perhaps this will change some day, but for now, at least, I wouldn't conclude that all expressions of human expertise are highly amenable to scaling with AI. Note that the most impressive specifics in Hill's piece have a lot to do with Emmet's relentlessness in codifying and AI-encapsulating his research methodology. The higher-level "AI is also planning and organizing this!" glue is probably useful, but I'd guess that most of the value is coming from the lower-level building blocks.1
- But I could be wrong! I have an interest in the enduring value of my own expertise in software systems. But I'm well aware that there's a big gap between "anyone can do this" and "only I and other AI-forward trad-programmers can do this." As I've written before, I constantly wonder about the "gardeners in Topeka" out there who know things, are good at organizing, and are just plain smart. It seems at least possible that hundreds or thousands of them will be able and motivated to surpass me in software-system construction and maintenance pretty quickly.
These profiles are not just fun but extremely valuable. If you are or know an Emmet type, scaling work with AI, I'd be grateful if you wrote about it.
Note also that "Emmet built a powerlifting app in half an hour" is impressive, but probably doable by any competent AI engineer. I suspect that just telling AI "make me a Juggernaut Method powerlifting app" is likely to work, and the hardest part would be installing it on your phone and ironing out whatever bugs and clunky UX there are. (Again, I don't mean this dismissively, and I don't doubt that Emmet did a lot either to prevent or to accelerate a lot of those ironing-out issues.)↩