Decision model application of the day
I've been thinking about API design in the AI era and about dividing responsibilities between a service and an AI agent calling into that service.
I have no firm conclusions here yet, but am still here to report that LLM-friendly APIs enable all sorts of useful things that would be impractical to build into the services themselves.
So, I am a trivia fan who:
- Runs Zippyflash, a spaced-repetition site, which exposes an API;
- Is a bit behind on his flashcards these days;
- Wanted to spice up a study session today.
So, I had Claude sort my overdue questions into categories and spin up a quiz for me in the style of a beloved quiz show. It has me type in the answers and calls into a decision model to tell me if I'm right or wrong. I can override the model's judgments, but there's rarely a reason to. Even without tuning the prompt, the decision model has been nearly perfect in sorting my responses into correct, incorrect, and other cases.1 Here, for example, is Jev recognizing an unlisted, not-completely-obvious alternate answer:
The point isn't that this is possible; it's that it's easy. I did it on a whim. Claude2 one-shotted a usable version in about 5 minutes, and the few modifications I wanted were similarly easy. Just making this for myself took less effort and time than I would have spent writing a hypothetical feature request to a human.3
Consider keeping a decision model API key available in an .env file your LLM knows about, if you're into this sort of thing; managing secrets can be the majority of the human time of a small project these days.
I expect to modify and permute this in any number of ways; I love drilling flashcards more than almost anyone, but even I find myself wanting a bit more variety. As LLMs get better, plugging into an API and asking for something new gets safer, faster, and easier. I expect more of my software usage to follow this pattern.
The other cases are "prompt me" (e.g., if I write "goose" when asked for the state bird of Hawai'i, "throw this question out" (if the model detects that the question is somehow ill-formed), and "did not attempt" (which still counts as wrong on the Zippyflash back-end but does not deduct points in the game).↩
Opus 5.5.↩
...if I were not also the human running Zippyflash.↩