opencode/specs/03-request-throttling.md

Request throttling

Debounce and cancel high-frequency server calls

---

Summary

Some user interactions trigger bursts of server requests that can overlap and return out of order. We’ll debounce frequent triggers and cancel in-flight requests (or ignore stale results) for file search and LSP refresh.

---

Goals

• Reduce redundant calls from file search and LSP refresh
• Prevent stale responses from overwriting newer UI state
• Preserve responsive typing and scrolling during high activity

---

Non-goals

• Changing server-side behavior or adding new endpoints
• Implementing global request queues for all SDK calls
• Persisting search results across reloads

---

Current state

• File search calls sdk.client.find.files via files.searchFilesAndDirectories.
• LSP refresh is triggered frequently (exact call sites vary, but the refresh behavior is high-frequency).
• Large UI modules involved include packages/app/src/pages/layout.tsx and packages/app/src/components/prompt-input.tsx.

---

Proposed approach

• Add a small request coordinator utility:
  • debounced triggering (leading/trailing configurable)
  • cancellation via AbortController when supported
  • stale-result protection via monotonic request ids when abort is not supported
• Integrate coordinator into:
  • files.searchFilesAndDirectories (wrap sdk.client.find.files)
  • LSP refresh call path (wrap refresh invocation and ensure only latest applies)

---

Phased implementation steps

1. Add a debounced + cancellable helper

• Create packages/app/src/utils/requests.ts with:
  • createDebouncedAsync(fn, delayMs)
  • createLatestOnlyAsync(fn) that drops stale responses
• Prefer explicit, readable primitives over a single complex abstraction.

Sketch:

function createLatestOnlyAsync<TArgs extends unknown[], TResult>(
  fn: (args: { input: TArgs; signal?: AbortSignal }) => Promise<TResult>,
) {
  let id = 0
  let controller: AbortController | undefined

  return async (...input: TArgs) => {
    id += 1
    const current = id
    controller?.abort()
    controller = new AbortController()

    const result = await fn({ input, signal: controller.signal })
    if (current !== id) return
    return result
  }
}

2. Apply to file search

• Update files.searchFilesAndDirectories to:
  • debounce input changes (e.g. 150–300 ms)
  • abort prior request when a new query begins
  • ignore results if they are stale
• Ensure “empty query” is handled locally without calling the server.

3. Apply to LSP refresh

• Identify the refresh trigger points used during typing and file switching.
• Add:
  • debounce for rapid triggers (e.g. 250–500 ms)
  • cancellation for in-flight refresh if supported
  • last-write-wins behavior for applying diagnostics/results

4. Add feature flags and metrics

• Add flags:
  • requests.debounce.fileSearch
  • requests.latestOnly.lspRefresh
• Add simple dev-only counters for “requests started / aborted / applied”.

---

Data migration / backward compatibility

• No persisted data changes.
• Behavior is compatible as long as UI state updates only when the “latest” request resolves.

---

Risk + mitigations

• Risk: aggressive debounce makes UI feel laggy.
  • Mitigation: keep delays small and tune separately for search vs refresh.
• Risk: aborting requests may surface as errors in logs.
  • Mitigation: treat AbortError as expected and do not log it as a failure.
• Risk: SDK method may not accept AbortSignal.
  • Mitigation: use request-id stale protection even without true cancellation.

---

Validation plan

• Manual scenarios:
  • type quickly in file search and confirm requests collapse and results stay correct
  • trigger LSP refresh repeatedly and confirm diagnostics do not flicker backward
• Add a small unit test for latest-only behavior (stale results are ignored).

---

Rollout plan

• Ship helpers behind flags default off.
• Enable file search debounce first (high impact, easy to validate).
• Enable LSP latest-only next, then add cancellation if SDK supports signals.
• Keep a quick rollback by disabling the flags.

---

Open questions

• Does sdk.client.find.files accept an abort signal today, or do we need stale-result protection only?
• Where is LSP refresh initiated, and does it have a single chokepoint we can wrap?
• What debounce values feel best for common repos and slower machines?