mirror of
https://github.com/anomalyco/opencode.git
synced 2026-02-01 22:48:16 +00:00
142 lines
4.7 KiB
Markdown
142 lines
4.7 KiB
Markdown
## Cache eviction
|
||
|
||
Add explicit bounds for long-lived in-memory state
|
||
|
||
---
|
||
|
||
### Summary
|
||
|
||
Several in-memory caches grow without limits during long sessions. We’ll introduce explicit eviction (LRU + TTL + size caps) for sessions/messages/file contents and global per-directory sync stores.
|
||
|
||
---
|
||
|
||
### Goals
|
||
|
||
- Prevent unbounded memory growth from caches that survive navigation
|
||
- Add consistent eviction primitives shared across contexts
|
||
- Keep UI responsive under heavy usage (many sessions, large files)
|
||
|
||
---
|
||
|
||
### Non-goals
|
||
|
||
- Perfect cache hit rates or prefetch strategies
|
||
- Changing server APIs or adding background jobs
|
||
- Persisting caches for offline use
|
||
|
||
---
|
||
|
||
### Current state
|
||
|
||
- Global sync uses per-directory child stores without eviction in `packages/app/src/context/global-sync.tsx`.
|
||
- File contents cached in `packages/app/src/context/file.tsx` with no cap.
|
||
- Session-heavy pages include `packages/app/src/pages/session.tsx` and `packages/app/src/pages/layout.tsx`.
|
||
|
||
---
|
||
|
||
### Proposed approach
|
||
|
||
- Introduce a shared cache utility that supports:
|
||
- `maxEntries`, `maxBytes` (approx), and `ttlMs`
|
||
- LRU ordering with explicit `touch(key)` on access
|
||
- deterministic `evict()` and `clear()` APIs
|
||
- Apply the utility to:
|
||
- global-sync per-directory child stores (cap number of directories kept “hot”)
|
||
- file contents cache (cap by entries + bytes, with TTL)
|
||
- session/message caches (cap by session count, and optionally message count)
|
||
- Add feature flags per cache domain to allow partial rollout (e.g. `cache.eviction.files`).
|
||
|
||
---
|
||
|
||
### Phased implementation steps
|
||
|
||
1. Add a generic cache helper
|
||
|
||
- Create `packages/app/src/utils/cache.ts` with a small, dependency-free LRU+TTL.
|
||
- Keep it framework-agnostic and usable from Solid contexts.
|
||
|
||
Sketch:
|
||
|
||
```ts
|
||
type CacheOpts = {
|
||
maxEntries: number
|
||
ttlMs?: number
|
||
maxBytes?: number
|
||
sizeOf?: (value: unknown) => number
|
||
}
|
||
|
||
function createLruCache<T>(opts: CacheOpts) {
|
||
// get, set, delete, clear, evictExpired, stats
|
||
}
|
||
```
|
||
|
||
2. Apply eviction to file contents
|
||
|
||
- In `packages/app/src/context/file.tsx`:
|
||
- wrap the existing file-content map in the LRU helper
|
||
- approximate size via `TextEncoder` length of content strings
|
||
- evict on `set` and periodically via `requestIdleCallback` when available
|
||
- Add a small TTL (e.g. 10–30 minutes) to discard stale contents.
|
||
|
||
3. Apply eviction to global-sync child stores
|
||
|
||
- In `packages/app/src/context/global-sync.tsx`:
|
||
- track child stores by directory key in an LRU with `maxEntries`
|
||
- call a `dispose()` hook on eviction to release subscriptions and listeners
|
||
- Ensure “currently active directory” is always `touch()`’d to avoid surprise evictions.
|
||
|
||
4. Apply eviction to session/message caches
|
||
|
||
- Identify the session/message caching touchpoints used by `packages/app/src/pages/session.tsx`.
|
||
- Add caps that reflect UI needs (e.g. last 10–20 sessions kept, last N messages per session if cached).
|
||
|
||
5. Add developer tooling
|
||
|
||
- Add a debug-only stats readout (console or dev panel) for cache sizes and eviction counts.
|
||
- Add a one-click “clear caches” action for troubleshooting.
|
||
|
||
---
|
||
|
||
### Data migration / backward compatibility
|
||
|
||
- No persisted schema changes are required since this targets in-memory caches.
|
||
- If any cache is currently mirrored into persistence, keep keys stable and only change in-memory retention.
|
||
|
||
---
|
||
|
||
### Risk + mitigations
|
||
|
||
- Risk: evicting content still needed causes extra refetches and flicker.
|
||
- Mitigation: always pin “active” entities and evict least-recently-used first.
|
||
- Risk: disposing global-sync child stores could leak listeners if not cleaned up correctly.
|
||
- Mitigation: require an explicit `dispose()` contract and add dev assertions for listener counts.
|
||
- Risk: approximate byte sizing is imprecise.
|
||
- Mitigation: combine entry caps with byte caps and keep thresholds conservative.
|
||
|
||
---
|
||
|
||
### Validation plan
|
||
|
||
- Add tests for `createLruCache` covering TTL expiry, LRU ordering, and eviction triggers.
|
||
- Manual scenarios:
|
||
- open many files and confirm memory stabilizes and UI remains responsive
|
||
- switch across many directories and confirm global-sync does not continuously grow
|
||
- long session navigation loop and confirm caches plateau
|
||
|
||
---
|
||
|
||
### Rollout plan
|
||
|
||
- Land cache utility first with flags default off.
|
||
- Enable file cache eviction first (lowest behavioral risk).
|
||
- Enable global-sync eviction next with conservative caps and strong logging in dev.
|
||
- Enable session/message eviction last after observing real usage patterns.
|
||
|
||
---
|
||
|
||
### Open questions
|
||
|
||
- What are the current session/message cache structures and their ownership boundaries?
|
||
- Which child stores in `global-sync.tsx` have resources that must be disposed explicitly?
|
||
- What caps are acceptable for typical workflows (files open, directories visited, sessions viewed)?
|