17. History compaction
When context approaches the model limit, AVL Code automatically compacts older history into a summary, so conversations can go on indefinitely.
17.1 Triggers
- Manual: session tab right-click → Compact, or
/compact. - Automatic: the assistant triggers from its own side when its context approaches the limit; doesn't interrupt the current turn.
17.2 Visibility after compaction
Compaction is append-only: original messages are kept on disk, only flagged "hidden from current turn". You can expand the compaction summary to see the originals.
17.3 Structured info preserved
A compaction summary keeps at least:
- The latest plan and todo state
- Conclusive results of key tool calls
- Key user-confirmed decisions
- Required facts (paths, version numbers, etc.)
17.4 Multiple compactions
Long conversations can go through multiple compactions; each is additive (not a replacement). The earliest content can always be reached by drilling down.
17.4.1 Auto-compaction abort shows the real reason
When auto-compaction is aborted, the chat area shows the specific reason instead of a generic message — so you can tell whether it was a config issue, the context, or something else. Graceful abort / session preservation behaviour is unchanged.
17.4.2 Context-overflow root-fix
Conversations could occasionally crash with an upstream 400 and auto-compaction couldn't rescue them. The real cause is usually not under-counted tokens but a single oversized tool result — e.g. a sec.* strings extraction returning close to 300 KB and nearly filling the whole context window; two back-to-back results push the next request over the window and the upstream rejects it (these rejections don't return usage info, so reactive compaction can't see them in time).
Two-layer fix:
- Tool-result clamping: every tool result is first clamped to a safe size (about 6% of the context window, safely UTF-8-character-boundary truncated with a "continue reading" marker) — eliminates "one step blowing up the window" at the source.
- Estimation guard at send time: anchored on the previous turn's real usage + a local estimate of this turn's delta; if it predicts overflow, it proactively goes through "compact then continue" instead of firing a doomed-to-be-rejected request.
17.4.3 Reliable continuation after auto-compaction
After history compaction, the AI sometimes stopped to send only a greeting / report the current time / ask "what would you like me to do?" instead of finishing the work. This release toughens it up:
- The continuation instruction is now explicitly directive: it tells the AI to autonomously continue from the next step on incomplete / in-progress / blocked tasks in the compaction summary, not to send a greeting or report the time.
- The clock prompt isn't injected at the tail of the continuation turn, so weak models don't mistake the current time for the latest focus and ignore todos.
- If continuation still stops on a question, an auto "continue" nudge is sent (mimicking your manual "continue") to push the work forward.
- The progress bar reuses the manual
/compactone (percentage / ETA / live word count) — no more vague "processing".
17.4.4 Compaction no longer false-times-out on thinking models
Auto-compaction previously used the same fixed overall deadline for every model. Reasoning / extended-thinking models can spend a while in the thinking phase before emitting visible tokens, so they got killed mid-thought and the summary failed — which broke continuation. This release switches the wait to an idle watchdog: as long as the server keeps producing (including thinking blocks), the wait extends; only true silence triggers timeout. Thinking models can now finish their compaction summary; the conversation no longer gets cut off mid-stream.
17.4.5 Whitespace-only replies no longer falsely end the turn
Reasoning models sometimes emit chunks containing only newlines / spaces in their thinking interludes. Before, the termination check read "non-empty output = still working", so those whitespace chunks were perversely interpreted as "done" and the whole turn was cut short. The check is now "empty after trimming whitespace" — mid-thought whitespace no longer false-stops; the full reasoning stays intact.
17.5 KV cache prefix-cache management (perf)
Upstream model services work on a "prefix cache": system prompt → tool list → conversation history is one long string; as long as bytes at the front don't change, the cache hits; one byte at the front and the whole tail re-runs.
AVL Code addresses three "per-turn churn" sources to keep the cache warm:
| Source | Mitigation |
|---|---|
| Tool list order | Sorted by tool name to be deterministic (Go map iteration was random; the biggest unconditional win every turn) |
| Goal block + memory recall | Stripped from the prefix; reinjected at the end of the turn (invisible, not persisted) — costs only the current-turn delta, doesn't invalidate the prefix |
| Project instructions (AGENTS.md) | Deliberately kept in the prefix — stable within a session, re-reads hit the cache; only invalidates on edit (the desired behaviour) |
TUI mode is treated the same way — the CLI also moves the goal block to end-of-turn ephemeral injection, so desktop and terminal cache behaviour match.
The effect: in long conversations and multi-turn tool calls, cache-hit ratio rises noticeably — faster responses and lower token cost. This optimisation is invisible to the user — no configuration needed.
17.6 Memory Palace (long-term memory)
AVL Code's built-in Memory Palace is the cross-session long-term memory store. At the end of a conversation, HookStop triggers a memory-candidate extraction that lands in the current palace's draft inbox (_pending/ subdirectory) for your review; once approved, the entry is filed into the relevant Room. In the slash command palette: /memorize <text> writes one directly; /memory opens the palace panel.
17.6.1 Smarter over time: recall weighting and negative feedback
Memories are no longer sorted purely by time or match score:
- Recall + outcome dual scoring: each memory carries
recall_count(times surfaced) andsuccess_count(times the run completed well after it was used). Weighted together, memories that get hit a lot and actually help rise to the top; ones that keep getting pulled up but don't pan out slowly sink, so they stop crowding judgement. - Rare-term weighting: at retrieval, rarer / more distinctive terms count for more — the truly relevant entry is easier to surface instead of being drowned out by generic words.
- Skip if irrelevant: memories with low relevance to the current question are no longer injected wholesale; only your pinned entries are kept — saves tokens and reduces noise.
- Lessons from the judge checker auto-distill: issues found by the self-check gate's
judgechecker (see §10.11.1) are automatically distilled into memory drafts in the review inbox — next time a similar problem comes up, the lesson is right there.
17.6.2 Conflicts go to "pending review", not hard overwrite
When a new memory contradicts an existing one (same Subject + Predicate, different Object — typical of preference / decision flips), it doesn't overwrite directly — it lands in the draft inbox tagged with ConflictHint = old_memory_id:
- The frontend
MemoryReviewPanelhighlights "conflicts with existing" on that entry. - You can supersede with one click (old entry marked expired) / keep the old / reject the new.
- Complementary to
MergeHint(triples overlap heavily with an existing entry → likely duplicate, suggest merge): merge = should combine, conflict = should supersede.
17.6.3 Memory health board
Settings → Memory Palace → Health Board lays out each palace's state on one screen:
| Metric | Meaning |
|---|---|
| Total / expired / never recalled | Overview |
| Room count / per-room counts | Capacity distribution |
| Hot list (Top 10) | Most-recalled — memories that pull their weight |
| Cold list (≤ 20) | Never recalled — consider pruning |
| Low-value list (≤ 20) | Recalled enough (≥ 5) but success rate ≤ 20% — negative feedback is already demoting them; consider manual review |
| Duplicate clusters (≤ 20) | Same-room memories whose triples overlap heavily (≥ 2/3 hit) — merge suggested |
| Draft inbox pending count | pending_duplicates + pending_conflicts |
Each entry links directly to the underlying memory to pin / edit / delete; clusters can merge with one click.
17.6.4 Auto-reflect (retrospective learning, GRAI+KISS)
Turn on *Settings → Agent → Reflection → "Auto-reflect (retrospective learning)"* (off by default, since it costs an LLM call) and a retrospective runs automatically when a multi-step task wraps up: it reviews the run by the GRAI frame — Goal, Result (from the run's trace digest + self-heal stats), Analysis (5-WHY, controllable causes first), Insight — and sediments actionable lessons by the KISS taxonomy (Keep / Improve / Stop / Start) so next time a similar situation can reuse them.
- Only fires at the wrap-up of a task that did multi-step work: single-turn Q&A never triggers it; it runs async and doesn't block you.
- Lessons land in the Memory Palace's "pending draft inbox" (
_pending, roomscratch, kinddecision, never decays) and only enter the live memory bank for retrieval after you approve them (up to 8 per run, deduped by title). - Entirely fail-safe: memory off / no LLM / parse failure are silently skipped and never affect the task itself.
The "self-heal stats" the retrospective reads are exactly the heal history from §8.5.1 — together they form the experience-learning loop: self-heal learns from history to tune its attempts, and the retrospective sediments lessons for your review.