Is coding getting much more empirical?
It's now common to note that programming is becoming more empirical, in this sense: you learn whether it works by seeing whether it works, not from close acquaintance with the code.
This specific aspect of programming with generative is less new than it might seem. Complex systems are notorious for having lots of emergent behaviors that no human could deduce just from reading the code--not least because there's often too much code for a human to read.
Some evidence for this:
- It's a hallmark of a senior engineer to be much more disciplined and thorough about observability and monitoring. This is largely because these are the only ways to learn about at least some aspects of a complex software system.
- Even where in principle we should be able to reason directly about the code, we often don't.
- It's a pretty rare codebase were one can trusts it will function as specified entirely on the basis of tests' passing. Things can often go wrong, and the tests often exist precisely because unanticipated things have gone wrong.
- Remember coding raw CSS?
None of this is to say that programming with generative AI is not hugely different from programming without it. And even in this respect, programming is likely to feel more and more different, as we find new, generative-AI-optimized ways to observe our systems. For now, though, I don't think this aspect of programming has changed as much as so many others have. I have some new monitoring tools, and the whole process takes place at warp speed, but the process itself is recognizable.