Analysis of Claude Code’s Long-Context Management Strategy

Based on the source-code reconstruction project for @anthropic-ai/claude-code@2.1.88
Core files: services/compact/, utils/toolResultStorage.ts, query.ts, services/contextCollapse/

Overview

Claude Code doesn’t simply rely on the LLM’s 256K/512K/1M context window to brute-force its way through. Instead, it’s designed as a multi-layered, progressive context management system. There are 6 layers of defense, triggered in order from lightest to heaviest, ensuring that AI response quality and speed are maintained at any interaction length.

Core Architecture: Execution Order in the Query Loop

每次 AI API 调用前，按以下顺序执行：

1. applyToolResultBudget()     — 裁剪超大工具结果（零 API 开销）
2. Snip Compact                — 移除旧的 API round groups
3. Microcompact                — 清理旧工具结果（缓存编辑 or 时间触发）
4. Context Collapse              — 实验性细粒度管理（可选，替代 autocompact）
5. Auto Compact                — 如果以上都不够，AI 摘要化整个对话
                                ↓
                           发送 API 请求

Comparison of key characteristics across each layer’s strategy:

Layer 1: Microcompact

The lightest-weight operation — no additional API calls required, handled purely via local or server-side edits.

1a. Cached Microcompact

Leverages the Anthropic API’s cache_edits feature to mark certain tokens as “deleted” on the server side, without breaking the prompt cache prefix. This is the most efficient layer.

Only cleans up specific tools:

const COMPACTABLE_TOOLS = new Set([
  'Read', 'Bash', 'Grep', 'Glob', 'WebSearch', 'WebFetch', 'Edit', 'Write'
])

Trigger logic: Counter-based triggering — retains the most recent N tool results; older results beyond the threshold are deleted via the cache_edits API.

Key characteristics:

• Does not modify local message content
• Uses remote configuration from GrowthBook (trigger threshold, keep recent)
• Only runs on the main thread, to prevent forked agents from polluting global state
• Returned message content is unchanged; cache_reference and cache_edits are added at the API layer

Source path: services/compact/microCompact.ts

1b. Time-based Microcompact

When the user has been inactive for a long time (exceeding the configured gapThresholdMinutes), it indicates that previous tool results are now stale. The system clears all tool result content except the last N, replacing them with [Old tool result content cleared].

Trigger conditions:

function evaluateTimeBasedTrigger(messages, querySource) {
  // 需要：enabled + 主线程来源 + 最后一个 assistant 消息的时间间隔 > 阈值
  const gapMinutes = (Date.now() - lastAssistant.timestamp) / 60_000
  return gapMinutes >= config.gapThresholdMinutes
}

Clearing logic:

// 清除旧工具结果，只保留最近的 keepRecent 个
const keepRecent = Math.max(1, config.keepRecent)
const keepSet = new Set(compactableIds.slice(-keepRecent))
const clearSet = new Set(compactableIds.filter(id => !keepSet.has(id)))

// 将清除的内容替换为固定文本
return { ...block, content: '[Old tool result content cleared]' }

Key design decisions:

• Retains at least 1 result (Math.max(1, keepRecent)), preventing a zero-context situation from clearing everything
• Resets cached-MC state after clearing (since the server-side cache is now invalid)
• Notifies prompt cache break detection of the expected token drop, to avoid false positives

Source path: services/compact/microCompact.ts

Layer 2: Tool Result Persistence and Aggregate Budget

Large tool results are not placed directly into the message context — they’re persisted to disk.

Single Tool Result Persistence

Each tool declares its own maxResultSizeChars; anything exceeding the threshold is written to a disk file rather than injected into context:

// 解析每个工具的持久化阈值
function getPersistenceThreshold(toolName, declaredMax) {
  // Read 工具 = Infinity（不自持久化，避免 Read→file→Read 循环）
  if (!Number.isFinite(declaredMax)) return Infinity

  // GrowthBook 远程配置可按工具名覆盖阈值
  const overrides = growthbook.getFeature('tengu_satin_quoll', {})
  if (overrides?.[toolName]) return overrides[toolName]

  // 默认：min(工具声明值, 50_000 chars)
  return Math.min(declaredMax, DEFAULT_MAX_RESULT_SIZE_CHARS)
}

Persisted result format:

<persisted-output>
Output saved to: /path/to/.claude/sessions/{sessionId}/tool-results/{toolUseId}.txt
Preview (first 2000 chars): ...
</persisted-output>

Aggregate Tool Result Budget

Beyond individual tool limits, there’s also cross-tool, cross-session total budget control:

// 状态必须稳定以保证 prompt cache 前缀不变
type ContentReplacementState = {
  seenIds: Set<string>        // 已经见过的 tool_use_id，命运已锁定
  replacements: Map<string, string>  // 被替换成预览的映射
}

applyToolResultBudget() is called before each query loop begins:

// 计算所有 tool_result 的总 token 数
const totalTokens = calculateTotalToolResultTokens(messages)

// 超出预算时，按先入先出策略替换最旧的结果为预览
if (totalTokens > effectiveMaxTokens) {
  // 找到可替换的最旧工具结果
  // 将其内容替换为 <persisted-output> 标签和预览
  // 记录到 replacements map 中（保证下次调用时一致）
}

Key design decisions:

• seenIds ensures results that have already had replacement decisions made are not processed a second time
• Shares caching decisions across sub-agents via ContentReplacementState
• Supports subagents rebuilding a parent agent’s replacement state from sidechain records

Source path: utils/toolResultStorage.ts

Layer 3: Snip Compact (History Trimming)

A trimming mechanism controlled by feature('HISTORY_SNIP'). Snip is not summarization — it directly removes old API-round groups (assistant message + corresponding tool_result pairs) to reduce token usage.

// snipTokensFreed 传递给 autocompact，使阈值检查反映 Snip 移除的 token
// tokenCountWithEstimation 本身看不到节省（因为它读的是 API usage）

Snip runs before autocompact — if Snip has already brought the token count below the threshold, autocompact won’t trigger, avoiding unnecessary API calls.

Source path: services/compact/snipCompact.ts

Layer 4: Auto Compact

This is the core context compression mechanism, and also the heaviest layer.

Trigger Threshold

// 各层级缓冲值
const AUTOCOMPACT_BUFFER_TOKENS      = 13_000  // 保留 13K 余量 → 触发自动压缩
const WARNING_THRESHOLD_BUFFER_TOKENS  = 20_000  // 警告线（UI 提示）
const ERROR_THRESHOLD_BUFFER_TOKENS    = 20_000  // 错误线（阻断新请求）
const MANUAL_COMPACT_BUFFER_TOKENS     = 3_000   // 手动压缩极限

// 有效上下文窗口 = 总窗口 - 摘要预留
function getEffectiveContextWindowSize(model) {
  const reservedTokensForSummary = min(getMaxOutputTokens(model), 20_000)
  return contextWindow - reservedTokensForSummary
}

// 触发阈值 = 有效窗口 - 13K 缓冲
function getAutoCompactThreshold(model) {
  return getEffectiveContextWindowSize(model) - AUTOCOMPACT_BUFFER_TOKENS
}

Taking a 200K model as an example:

• Effective window ≈ 200K - 20K (summarization reserve) = 180K
• Auto-compact threshold = 180K - 13K = 167K tokens

Compression Flow

compactConversation():
  1. 执行 pre_compact hooks（用户可注入清理逻辑）
  2. 先尝试 Session Memory Compact
     └── 用 AI 生成记忆摘要代替完整压缩（更经济）
  3. 调用 compactConversation 核心逻辑：
     a. 从消息中剥离图片/文档块（它们不是摘要所需）
     b. 启动 forked agent，发送压缩 prompt
     c. AI 生成对话摘要
     d. 如果摘要请求本身 prompt_too_long → 截断最旧的 round 重试（最多 2 次）
  4. 压缩成功后：
     a. 清除 fileStateCache
     b. 并行生成 post-compact attachments:
        - 最近修改的文件（最多 5 个，每个 5K token）
        - 异步 Agent 状态
        - Plan 文件
        - 已调用的技能内容（25K token 预算，每个技能 5K）
        - MCP 工具增量
     c. 写入 compact boundary marker
     d. 写入摘要消息
  5. 执行 post_compact hooks 恢复关键上下文
  6. 重置 cache read baseline

Resulting size after compression: Approximately 20-40K tokens (summary + system prompt + tool schema + re-injected key context), far below the original 128-256K.

Circuit Breaker

const MAX_CONSECUTIVE_AUTOCOMPACT_FAILURES = 3

// 连续 3 次压缩失败后放弃，避免浪费 API 调用
// 注释记录：BQ 数据显示有 1,279 个 session 连续失败 50-3272 次
// 每天浪费约 25 万次 API 调用

Session Memory Compact (Prioritized over Full Compression)

Before triggering a full compression, it first attempts to generate a Session Memory summary. If the session memory system can generate a sufficient summary to relieve token pressure, a full AI summarization isn’t needed.

// 先试 session memory compact，成功了直接返回
const sessionMemoryResult = await trySessionMemoryCompaction(
  messages, agentId, threshold
)
if (sessionMemoryResult) return { wasCompacted: true, ... }

// 不行再走完整压缩

Partial Compact

In addition to fully compressing the entire conversation, it also supports partial compression:

enum PartialCompactDirection {
  'from',   // 保留 pivot 之前的，摘要之后 → 保留 prompt cache 前缀
  'up_to',  // 保留 pivot 之后的，摘要之前 → 需清理旧 cache
}

Users can choose to compress a portion of the conversation without affecting the rest.

Re-injecting Key Context After Compression

Compact isn’t a blanket wipe. After successful compression, key information is re-injected via attachments:

Source path: services/compact/compact.ts, services/compact/autoCompact.ts

Layer 5: API-Native Context Management

The Anthropic API provides a context_management feature; the client configures the strategy via getAPIContextManagement():

// 工具结果清除策略
{
  type: 'clear_tool_uses_20250919',
  trigger: { type: 'input_tokens', value: 180_000 },      // 超过 180K 触发
  clear_at_least: { type: 'input_tokens', value: 140_000 },  // 至少清到剩 40K
  exclude_tools: ['Edit', 'Write', 'NotebookEdit'],        // 编辑器结果保留
}

// Thinking 块清除策略
{
  type: 'clear_thinking_20251015',
  keep: 'all'  // 正常保留所有 thinking
  // 或：{ type: 'thinking_turns', value: 1 }  // 长时间闲置后只保留最后一个
}

Key design decisions:

• This is a server-side strategy, working in conjunction with the client
• clear_tool_uses clears tool calls and results, but retains editor operations in exclude_tools
• clear_thinking retains thinking context (chain-of-thought is important for reasoning)
• Default thresholds: trigger at 180K, target 40K (retain the last 40K tokens)

Source path: services/compact/apiMicrocompact.ts

Layer 6: Context Collapse

An internal experimental feature controlled by feature('CONTEXT_COLLAPSE'). This is a more fine-grained context management system:

• At 90% context utilization, begins commit-saving granular context
• At 95%, blocking spawn save
• When enabled, autocompact is suppressed (to avoid a race condition — autocompact triggers at ~93%, right in the middle of collapse’s 90-95% range)
• Collapse IS the context management system when enabled (it’s the primary system, not a supplementary layer)

Source path: services/contextCollapse/

Key Design Principles

1. Prompt Cache Hit Rate Is the Core Metric

All compression strategies take prompt cache into account:

• Cached Microcompact is optimal precisely because it deletes content via cache_edits without breaking the cache prefix
• partialCompact('from') preserves the prompt cache prefix
• Resets the cache read baseline after compression, to avoid misreporting legitimate cache drops as breaks
• Compression itself uses a forked agent that shares the parent’s prompt cache (promptCacheSharingEnabled = true)

// 实验确认：不共享 cache 的路径是 98% cache miss
// 浪费约 0.76% 的 fleet cache_creation (~38B tokens/day)
const promptCacheSharingEnabled = growthbook.getFeature(
  'tengu_compact_cache_prefix', true
)

2. Tool Results Are the Main Culprit Behind Context Bloat

A single grep -r or cat large_file can gobble up 5-10K tokens. Claude Code’s strategy:

• Individual tool results have a cap (maxResultSizeChars), default 50K characters
• The Read tool has maxResultSizeChars = Infinity (self-throttled via maxTokens, avoiding a Read→persist→Read loop)
• Cross-tool results have an aggregate budget (applyToolResultBudget)
• Oversized results are written to a file, with only a preview given
• Expired results are batch-cleared via microcompact

3. Auto-Compression Can Fail, So a Circuit Breaker Is Needed

Looking at real-world data, some sessions had 3000+ consecutive compression failures, wasting 250,000 API calls per day. The circuit breaker design:

const MAX_CONSECUTIVE_AUTOCOMPACT_FAILURES = 3

// 每次压缩失败计数+1，成功后重置为 0
// 达到 3 次后直接返回 false，不再尝试
if (tracking.consecutiveFailures >= 3) {
  logWarn('autocompact: circuit breaker tripped — skipping future attempts')
  return { wasCompacted: false }
}

4. Compression Isn’t Losing Information, It’s Reorganizing It

After successful compaction, key context is restored via multi-layer re-injection:

压缩前 (128K+)           压缩后 (~30K)
─────────────────       ─────────────────
系统 prompt (~15K)       系统 prompt (~15K)
工具 schema (~10K)       工具 schema (~10K)
对话历史 (~80K)    →     对话摘要 (~3K)
工具结果 (~20K)          关键文件快照 (~10K)
附件 (~3K)               MCP 指令 (~2K)
                         技能内容 (~5K)
                         Boundary marker (~1K)
                         Session Start hooks (~1K)

5. Multi-Level Disable Controls

Users can turn off different layers as needed:

DISABLE_COMPACT       // 关闭所有压缩
DISABLE_AUTO_COMPACT  // 只关自动压缩，保留手动 /compact
USE_API_CLEAR_TOOL_RESULTS  // 启用 API 侧工具结果清除
USE_API_CLEAR_TOOL_USES     // 启用 API 侧工具调用清除

Takeaways for Low-Code Development Agent Teams

Takeaway 1: Don’t Bet on a Single Strategy — Design Multiple Lines of Defense

Claude Code doesn’t “compress when full” — it uses 6 layers of strategy ordered from lightest to heaviest. The lightest (microcompact) is a purely local, zero-cost operation; only the heaviest (auto compact) requests AI-driven full-conversation summarization.

Application: Long conversations/workflows in a low-code platform should also have a tiered strategy of light-weight cleanup (removing expired intermediate results) + medium-weight (time-triggered clearing) + heavy-weight (AI summarization compression).

Takeaway 2: Tool Results Are the Main Culprit Behind Context Bloat

The long output of 5 shell commands can gobble up 5-10K tokens. Claude Code controls this through three layers:

1. Per-tool cap: each tool declares its own maxResultSizeChars
2. Aggregate budget: the total of all tool results cannot exceed a threshold
3. Expiration cleanup: microcompact batch-clears old results

Application: In a low-code platform, each node’s output should have a cap mechanism, and the total output across nodes should also have budget control. Different node types (read/write/query) should have different caps.

Takeaway 3: Prompt Cache Is a Cost Variable You Can’t Ignore

All compression strategies take prompt cache hit rate into account. Cached Microcompact became the optimal path precisely because it deletes content without breaking the cache prefix.

Application: If your system has a caching mechanism, the design of your compression strategy must treat “cache hit rate” as the core metric, not just “token count.” Breaking the cache once can be more expensive than keeping some redundant tokens.

Takeaway 4: Any Automated AI Operation Needs a Circuit Breaker

Looking at production data, automated compression can waste massive numbers of API calls in an unrecoverable state. A 3-failure circuit breaker is a reasonable starting point.

Application: Any automated AI operation in a low-code platform (compression, summarization, retry, re-routing) needs a circuit breaker — set a maximum number of consecutive failures after which attempts stop.

Takeaway 5: Key Context Must Be Re-injected After Compression

After compression, Claude Code doesn’t let the AI start completely from scratch. It carefully designs a post-compact re-injection strategy:

• Recently modified file states (lets the AI know the current code)
• Plan file (lets the AI know what to do)
• Invoked skills (lets the AI know which instructions already exist)
• MCP tool delta (lets the AI know which tools are available)

Application: After AI summarization of a conversation or compression of a workflow, it must not be left “amnesiac.” You need to design a post-compact re-injection strategy that re-injects key context in the form of structured attachments.

Takeaway 6: Time as a Compression Trigger

time-based microcompact is a simple yet elegant design: a long period of user inactivity = previous intermediate results are no longer needed. This is more natural than token-count-based triggering.

Application: In low-code workflows, if a task has been paused for too long, its intermediate results (caches, temporary data) are very likely no longer needed and can be safely cleaned up.

Takeaway 7: Fault Tolerance for Compression Itself

compactConversation itself can also encounter prompt_too_long (i.e., the conversation that needs compression has grown so long that even the compression request can’t be sent). It solves this by truncating the oldest API round and retrying:

// 压缩请求本身也 prompt_too_long？截断最旧的内容重试
for (ptlAttempts = 0; ; ptlAttempts++) {
  summary = await callCompactAPI(messages)
  if (!summary.startsWith('Prompt is too long')) break
  if (ptlAttempts >= MAX_PTL_RETRIES) throw new Error('不可恢复')
  messages = truncateOldestRounds(messages)
}

Application: Compression/summarization operations themselves also need a fallback path — you can’t assume they’ll always succeed.