Continuity in long-running AI systems
under-developmentWhy systems that handle every individual task well still drift away from the project they serve.
- Problem
- A long-running system can be correct at every step and wrong overall. Each task is handled competently; the direction of the work still degrades.
- Why it matters
- Most evaluation measures single-turn or single-task quality. The failure that costs real projects the most time is not a wrong answer, it is a slow loss of the thread across many correct answers.
- Observation
- Observed repeatedly while building and operating long-running systems on top of language models. The models were not the weak point; the absence of anything carrying intention forward was.
- Hypothesis
- Continuity is a property of the surrounding system, not of the model. It requires durable state that outlives any single conversation.
- Public material
- Described here at the level of the problem. Essays in preparation.