Agent improvement techniques, compared
Filed July 27, 2026
No coding agent improves on its own. The model is frozen and the context window empties every session. Every “self-improving agent” is really one of six techniques bolted on around the model. They are not interchangeable: each captures a different kind of knowledge, is written by a different party, and fails a different way. Here is the honest comparison. (One disclosure up front: we build Agent PIP, which lives in the postmortem-loop row.)
The six techniques
| Technique | Captures | Written by | Fires | Fails when |
|---|---|---|---|---|
| Rules file | Conventions | You | Session start | Nobody writes the rule; the file bloats |
| Accumulated memory | Facts and preferences | The agent | When judged relevant | The agent forgets what it would rather forget |
| Postmortem loop | Incidents and commitments | The agent, under review | Every session, on the record | Skipping the ritual after quiet weeks |
| Machine checks | Known failure patterns | You, once per pattern | Every run, mechanically | Only catches what you encoded |
| Pre-action gates | Forbidden actions | You, per offense | Before the action | Blunt; blocks the action, not the judgment |
| AI review | Anything a reviewer would catch | A second model | After the work | Catches, doesn't teach; same mistake next week |
1. Rules files (CLAUDE.md, AGENTS.md, .cursor/rules)
Standing instructions loaded every session. Cheap, portable, universally supported: the baseline everyone should have. Their weakness is the author: rules get written in the heat of a re-fix, come out vague, and accumulate without an editor. A rules file is only as good as its maintenance.
2. Accumulated memory (auto memory, Mem0, Letta, agentmemory)
The agent stores observations as it works and retrieves the relevant ones later. Excellent for volume: preferences, project quirks, hundreds of small facts nobody would hand-write. The structural weakness: the agent curates its own record, and retrieval is relevance-based. A serious mistake is precisely the memory least likely to be filed prominently and re-surfaced at the right moment.
3. The postmortem loop (lessons-learned, PIPs)
After an incident, the agent accounts for what happened, what it assumed, and what it should have done. The outcome is a written commitment reviewed every session, with follow-ups when it recurs. This is the only technique aimed at judgment mistakes: the ones no lint rule can encode. Hand-rolled versions are a lessons-learned file plus discipline; Agent PIP runs it as a formal HR meeting with the paper trail included. The weakness of both: it’s a ritual, and rituals get skipped.
4. Machine checks (tests, lint, CI, regression evals)
Turn a past failure into a check that fails mechanically, forever. The strongest technique per unit of effort: a regression test doesn’t rely on anyone’s memory, including the agent’s. But it only covers what you encoded: checks are a ratchet against known failures, not a teacher of judgment.
5. Pre-action gates (ThumbGate and kin)
Block the known-bad tool call before it runs. Ideal for the non-negotiables: force-push, prod migrations, rm on the wrong directory. Deliberately blunt: a gate stops an action, it doesn’t improve the reasoning that proposed it.
6. AI review (CodeRabbit, second-model review)
A second model critiques the first’s output before humans see it. Catches real defects and raises the floor, but it’s a filter, not a feedback loop. Unless review findings flow into one of the other techniques, the same mistake arrives again next week, freshly caught.
What actually compounds
The techniques stack, and the good setups converge on the same shape: a rules file for conventions, machine checks for encodable failures, and a postmortem loop that decides, incident by incident, what deserves a rule, a check, a gate, or a written commitment. That routing step is the part most setups skip, and it’s where improvement either compounds or quietly stops.