Notes on long-running LLM tasks

27 Sep, 2026

I often read about getting LLMs to work for a long time toward a goal. So, for example, Claude has a /goal command, and many of us (directly or indirectly) care a lot about how models do on TerminalBench, which is made up of tasks in which agents get one shot at doing a big chunk of work. More generally, long-running tasks are the sort of thing you hear about on podcasts, read about on Twitter, and so on: how large a bite can an agent chew?

There are some obvious, good reasons to care about this:

1. Lots of work fits this paradigm well: certain kinds of optimization, research, and experimentation provide clear examples.
2. It's reasonable to expect that more and more work will be amenable to this kind of structure, so caring about long-running tasks is a way of keeping perspective about the future of the craft.
3. Even if work isn't ideally suited for this structure, it might be better than alternatives (e.g., when no project-relevant human will be available for a while).
4. It's really cool.

But there remain many cases in which much smaller units of LLM-work are best, or at least best for me, for several reasons:

1. In many kinds of software development, the goal state is very hard to define. This is particularly true for traditional user-facing services, which tend to have tons and tons of subtle requirements that you only learn about from experience. Yes, you can (and sometimes should) work with AI to try to anticipate these, but for now, I much prefer to work more incrementally on this kind of problem.
2. For now, at least, LLMs often still overfit a system to the specification and still really struggle to encapsulate subsystems properly. Perhaps this is my own fault, though I've tried various techniques to mitigate this (with, e.g., better prompting, better AGENTS.md files, and periodic sweeps over the codebase to encapsulate things that need to be encapsulated). I suspect that AI will get a lot better at this over time, but for now I get much more maintainable results the more incrementally I build with AI.¹
3. Why, exactly, would it be good for me to accomplish something with an AI in a 100-minute chunk instead of N 5-minute chunks? (As a first approximation, N = 20, but in practice it's quite a bit smaller, because my intermediate corrections tend to be beneficial.) Yes, this is O(N) extra interrupts to handle, but I want those interrupts: for now, at least, they're efficient. I can make sure smaller subsystems work the way I think they ought to work, I can correct misunderstandings, and I can often find new possibilities in the project.
4. One of my favorite work patterns these days is to be working in small chunks on a central, important project, and to also have some exploratory longer-running tasks running on the side.
5. I wonder whether a lot of the interest in longer-running tasks comes from work environments where engineers don't have the means or incentive to maximize their productivity, or where long stretches of non-programming time are necessary. In some contexts, the relevant payoff is probably not supposed to be "get this project done faster or better" but rather "get something done during those afternoon meetings" or "extend my Ping-Pong session."

Again, I suspect this is all quite specific to the kind of work you're doing. Many people certainly are getting large rewards from setting up long-running AI sessions. But if you find that you're doing your best work in much shorter units, please know that you're not alone.

#generative AI #productivity #psychology of software #sociology of software #software