opencode/specs/08-app-e2e-smoke-suite.md

App E2E Smoke Suite (CI)

Implement a small set of high-signal, low-flake Playwright tests to run in CI.

These tests are intended to catch regressions in the “core shell” of the app (navigation, dialogs, prompt UX, file viewer, terminal), without relying on model output.

---

Summary

Add 6 smoke tests to packages/app/e2e/:

• Settings dialog: open, switch tabs, close
• Prompt slash command: /open opens the file picker dialog
• Prompt @mention: @<file> inserts a file pill token
• Model picker: open model selection and choose a model
• File viewer: open a known file and assert contents render
• Terminal: open terminal, verify Ghostty mounts, create a second terminal

---

Progress

• 1. Settings dialog open / switch / close (packages/app/e2e/settings.spec.ts)
• 2. Prompt slash command path: /open opens file picker (packages/app/e2e/prompt-slash-open.spec.ts)
• 3. Prompt @mention inserts a file pill token (packages/app/e2e/prompt-mention.spec.ts)
• 4. Model selection UI works end-to-end (packages/app/e2e/model-picker.spec.ts)
• 5. File viewer renders real file content (packages/app/e2e/file-viewer.spec.ts)
• 8. Terminal init + create new terminal (packages/app/e2e/terminal-init.spec.ts)

---

Goals

• Tests run reliably in CI using the existing local runner (packages/app/script/e2e-local.ts).
• Cover “wiring” regressions across UI + backend APIs:
  • dialogs + command routing
  • prompt contenteditable parsing
  • file search + file read + code viewer render
  • terminal open + pty creation + Ghostty mount
• Avoid assertions that depend on LLM output.
• Keep runtime low (these should be “smoke”, not full workflows).

---

Non-goals

• Verifying complex model behavior, streaming correctness, or tool call semantics.
• Testing provider auth flows (CI has no secrets).
• Testing share, MCP, or LSP download flows (disabled in the e2e runner).

---

Current State

Existing tests in packages/app/e2e/ already cover:

• Home renders + server picker opens
• Directory route redirects to /session
• Sidebar collapse/expand
• Command palette opens/closes
• Basic session open + prompt input + (optional) prompt/reply flow
• File open via palette (but shallow assertion: tab exists)
• Terminal panel toggles (but doesn’t assert Ghostty mounted)
• Context panel open

We want to add a focused smoke layer that increases coverage of the most regression-prone UI paths.

---

Proposed Tests

All tests should use the shared fixtures in:

• packages/app/e2e/fixtures.ts (for sdk, directory, gotoSession)
• packages/app/e2e/utils.ts (for modKey, promptSelector, terminalToggleKey)

Prefer creating new spec files rather than overloading existing ones, so it’s easy to run these tests as a group via grep.

Suggested file layout:

• packages/app/e2e/settings.spec.ts
• packages/app/e2e/prompt-slash-open.spec.ts
• packages/app/e2e/prompt-mention.spec.ts
• packages/app/e2e/model-picker.spec.ts
• packages/app/e2e/file-viewer.spec.ts
• packages/app/e2e/terminal-init.spec.ts

Name each test with a “smoke” prefix so CI can run only this suite if needed.

1) Settings dialog open / switch / close

Purpose: catch regressions in dialog infra, settings rendering, tabs.

Steps:

1. await gotoSession().
2. Open settings via keybind (preferred for stability): await page.keyboard.press(${modKey}+Comma).
3. Assert dialog visible (page.getByRole('dialog')).
4. Click the "Shortcuts" tab (role tab, name "Shortcuts").
5. Assert shortcuts view renders (e.g. the search field placeholder or reset button exists).
6. Close with Escape and assert dialog removed.

Notes:

• If Meta+Comma / Control+Comma key name is flaky, fall back to clicking the sidebar settings icon.
• Favor role-based selectors over brittle class selectors.
• If Escape doesn’t dismiss reliably (tooltips can intercept), fall back to clicking the dialog overlay.

Implementation: packages/app/e2e/settings.spec.ts

Acceptance criteria:

• Settings dialog opens reliably.
• Switching to Shortcuts tab works.
• Escape closes the dialog.

2) Prompt slash command path: /open opens file picker

Purpose: validate contenteditable parsing + slash popover + builtin command dispatch (distinct from mod+p).

Steps:

1. await gotoSession().
2. Click prompt (promptSelector).
3. Type /open.
4. Press Enter (while slash popover is active).
5. Assert a dialog appears and contains a textbox (the file picker search input).
6. Close dialog with Escape.

Acceptance criteria:

• /open triggers file.open and opens DialogSelectFile.

3) Prompt @mention inserts a file pill token

Purpose: validate the most fragile prompt behavior: structured tokens inside contenteditable.

Steps:

1. await gotoSession().
2. Focus the prompt.
3. Type @packages/app/package.json.
4. Press Tab to accept the active @mention suggestion.
5. Assert a pill element is inserted:
   • page.locator('[data-component="prompt-input"] [data-type="file"][data-path="packages/app/package.json"]') exists.

Acceptance criteria:

• A file pill is inserted and has the expected data-* attributes.
• Prompt editor remains interactable (e.g. typing a trailing space works).

4) Model selection UI works end-to-end

Purpose: validate model list rendering, selection wiring, and prompt footer updating.

Implementation approach:

• Use /model to open the model selection dialog (builtin command).

Steps:

1. await gotoSession().
2. Focus prompt, type /model, press Enter.
3. In the model dialog, pick a visible model that is not the current selection (if available).
4. Use the search field to filter to that model (use its id from the list item's data-key to avoid time-based model visibility drift).
5. Select the filtered model.
6. Assert dialog closed.
7. Assert the prompt footer now shows the chosen model name.

Acceptance criteria:

• A model can be selected without requiring provider auth.
• The prompt footer reflects the new selection.

5) File viewer renders real file content

Purpose: ensure file search + open + file.read + code viewer render all work.

Steps:

1. await gotoSession().
2. Open file picker (either mod+p or /open).
3. Search for packages/app/package.json.
4. Click the matching file result.
5. Ensure the new file tab is active (click the package.json tab if needed so the viewer mounts).
6. Assert the code viewer contains a known substring:
   • "name": "@opencode-ai/app".
7. Optionally assert the file tab is active and visible.

Acceptance criteria:

• Code view shows expected content (not just “tab exists”).

8) Terminal init + create new terminal

Purpose: ensure terminal isn’t only “visible”, but actually mounted and functional.

Steps:

1. await gotoSession().
2. Open terminal with terminalToggleKey (currently Control+Backquote).
3. Assert terminal container exists and is visible: [data-component="terminal"].
4. Assert Ghostty textarea exists: [data-component="terminal"] textarea.
5. Create a new terminal via keybind (terminal.new is ctrl+alt+t).
6. Assert terminal tab count increases to 2.

Acceptance criteria:

• Ghostty mounts (textarea present).
• Creating a new terminal results in a second tab.

---

CI Stability + Flake Avoidance

These tests run with fullyParallel: true in packages/app/playwright.config.ts. Keep them isolated and deterministic.

• Avoid ordering-based assertions: never assume a “first” session/project/file is stable unless you filtered by unique text.
• Prefer deterministic targets:
  • use packages/app/package.json rather than bare package.json (multiple hits possible)
  • for models, avoid hardcoding a single model id; pick from the visible list and filter by its data-key instead
• Prefer robust selectors:
  • role selectors: getByRole('dialog'), getByRole('textbox'), getByRole('tab')
  • stable data attributes already present: promptSelector, [data-component="terminal"]
• Keep tests local and fast:
  • do not submit prompts that require real model replies
  • avoid page.waitForTimeout; use expect(...).toBeVisible() and expect.poll when needed
• Watch for silent UI failures:
  • capture page.on('pageerror') and fail test if any are emitted
  • optionally capture console errors (page.on('console', ...)) and fail on type==='error'
• Cleanup:
  • these tests should not need to create sessions
  • if a test ever creates sessions or PTYs directly, clean up with SDK calls in finally

---

Validation Plan

Run locally:

• cd packages/app
• bun run test:e2e:local -- --grep smoke

Verify:

• all new tests pass consistently across multiple runs
• overall e2e suite time does not increase significantly

---

Open Questions

• Should we add a small helper in packages/app/e2e/utils.ts for “type into prompt contenteditable” to reduce duplication?
• Do we want to gate these smoke tests with a dedicated @smoke naming convention (or test.describe('smoke', ...)) so CI can target them explicitly?