Unreasonably effective LLM pattern of the day
A certain kind of document management is embarrassingly important to me these days. as more and more of my projects are fundamentally the maintenance, interrogation, and development of some corpus. So, for example:
- I curate financial documents, and I use LLMs to ask questions about it (e.g., to help with taxes).
- I curate academic-year documents, and I use LLMs to ask questions about it ("wait, when is X happening?").
- If I'm planning a major purchase, I curate documents for that, too.
As LLMs get more powerful, I've felt projects pushed into this structure from both directions. In the academic-year or purchase-planning case, LLMs make it so much easier to ask questions of a corpus of diverse documents that it's worth thinking of those as explicit projects. In the financial-management case, LLMs can now do some of the work of bespoke software, turning those from more structured, siloed projects to looser corpus-maintenance projects. Meanwhile:
- AI services are eager to offer me project, document-storage, and/or namespacing services for these use cases.
- But I want to be asking different AIs about the same documents, at least potentially.
- And even if I didn't want to be using different LLMs over the same corpus, I still find many of those "project" interfaces clunky and confusing.
I therefore find myself in the counterintuitive and (again) somewhat embarrassing state where my ability to use revolutionary AI tools is depending on a skill (just folder management, in some sense) that one might expect any competent computer user, and certainly any software professional, to have mastered.
Embarrassed or not, however, I'm getting a ton of use and convenience out of a little document-context-management tool I've built. It maintains "contexts," which aren't much more than sets of documents. It exposes a very fast API (in this spirit) for doing everything you'd expect: listing contexts; adding, removing, and renaming files; and so on. It has a nice web GUI, although most of my interaction with it is asking LLMs to use it via API. It is the rare impulsive side project that is more useful than I'd first imagined it would be.
Further notes:
- Yes, I could just use S3 (or any similar service), but I'm getting extra value from (i) building in some small extra conveniences, (ii) designing the API the way I want it, and (iii) having the GUI I want. That said, I can't shake the embarrassment at feeling this much pleasure in having made a small wrapper around namespaced blob storage.
- I've made suitable modifications to my
AGENTS.mdand similar files, so that it's easy to open an LLM, ask it questions about my projects, and get intelligible answers. I never took the OpenClaw plunge, don't use Muse, and have generally avoided the "govern my life with AI" genre of project, so this is new to me.1 - I think I see a lot of projects aiming to make LLM chats portable; a tool like this gives me almost all of that utility (plus a lot of other utility), because if a given LLM chat is useful, that utility can often be expressed either in a document or in modifications to a corpus of documents.2
Anyway! A tool that makes it very easy for LLMs to interact programmatically with namespaced sets of documents is very useful these days.
It most reminds me of Gmail search from some months ago, though now I have to click on the Gemini button in Gmail to ask questions of AI in Gmail. I miss the experience from May, though: I want the big text box to trigger AI search, and I want both the AI results (if there are any) and the normal list-of-emails search results. But that's another post.↩
I'm a bit surprised there isn't more "avoid ecosystem lock-in!" discourse around AI and LLMs. One conjecture is that anti-lock-in arguments tended to be ethically inflected, and there's heavy competition for attention to ethical dimensions of AI. (Put more simply: a lot of people who would be worried about lock-in as an ethical issue are way more worried about other ethical issues these days.)↩