logseq/docs/agent-guide/064-logseq-cli-integration-test-shell-refactor.md

# Logseq CLI Integration Test Shell Refactor Implementation Plan

Goal: Build a new `cli-e2e` test suite that refactors `logseq.cli.integration-test` into babashka-driven shell e2e tests for compiled `logseq-cli` and `db-worker-node`, excluding `sync`, `login`, and `logout`, with full in-scope subcommand coverage and key-option coverage.

Architecture: Use a declarative command case manifest in `cli-e2e/` and execute every case via explicit shell command strings so test execution is transparent and copy-pasteable.
Architecture: Run a build preflight that compiles `logseq-cli` and `db-worker-node` artifacts before every e2e run, then run cases against those compiled artifacts only.
Architecture: Enforce completeness with a coverage checker that fails when any in-scope subcommand or any declared key option is not covered by at least one case.

Tech Stack: Babashka, babashka.cli, babashka.process, EDN manifests, Node.js, shadow-cljs build targets `logseq-cli` and `db-worker-node`, existing Logseq CLI and db-worker-node runtime.

Related: Relates to `/Users/rcmerci/gh-repos/logseq/docs/agent-guide/053-logseq-cli-async-test-isolation.md`.
Related: Relates to `/Users/rcmerci/gh-repos/logseq/docs/agent-guide/058-db-worker-node-revision-and-cli-server-list.md`.
Related: Relates to `/Users/rcmerci/gh-repos/logseq/docs/agent-guide/060-cli-graph-list-legacy-graph-dir-rename-command.md`.

## Problem statement

The current `src/test/logseq/cli/integration_test.cljs` suite mostly calls `logseq.cli.main/run!` directly instead of executing `logseq-cli` as a shell command.
This makes command-line behavior, quoting, piping, process lifecycle, and compiled artifact boundaries under-tested.
The current suite also relies heavily on `with-redefs` for `sync` and auth flows, which hides process-level integration issues.
The suite location and runtime are tied to `shadow-cljs :test`, while the new requirement is an independent `cli-e2e/` babashka flow that tests compiled binaries.
The new requirement also asks for in-scope subcommand completeness with key option coverage enforcement, which is not explicitly guaranteed today.

## Current implementation snapshot

| Area | Current file | Current behavior | Gap against requirement |
| --- | --- | --- | --- |
| CLI integration tests | `/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/integration_test.cljs` | Runs in cljs test bundle and invokes `run!` directly for most tests. | Not fully shell-first and not isolated under `cli-e2e/`. |
| CLI compiled artifact | `/Users/rcmerci/gh-repos/logseq/static/logseq-cli.js` | Generated by `shadow-cljs` target `logseq-cli`. | Not currently enforced as precondition in integration test flow. |
| db-worker runtime used by CLI server lifecycle | `/Users/rcmerci/gh-repos/logseq/dist/db-worker-node.js` | Spawned by `logseq.cli.server/db-worker-script-path`. | Must be rebuilt before e2e to ensure latest runtime. |
| Existing integration runner entrypoint | `/Users/rcmerci/gh-repos/logseq/package.json` (`cljs:run-integration-test`) | Runs `node static/tests.js -r '^(logseq.cli.integration-test).*'`. | Keeps tests inside cljs test harness instead of babashka shell runner. |

## In-scope command and key option inventory

The e2e suite target excludes every `sync *` command and excludes `login` and `logout`.
The suite treats global options and command options as separate coverage buckets.
The option inventory below is the key-option scope for this phase.

| Scope | Commands | Options to cover |
| --- | --- | --- |
| Global | Any in-scope command invocation | `--help`, `--version`, `--config`, `--graph`, `--data-dir`, `--output`, `--verbose`. |
| graph | `graph list`, `graph create`, `graph switch`, `graph remove`, `graph validate`, `graph info`, `graph export`, `graph import` | `--fix`, `--type`, `--file`, `--input`. |
| list | `list page`, `list tag`, `list property` | `--expand`, `--fields`, `--limit`, `--offset`, `--sort`, `--order`, `--journal-only`, `--with-properties`, `--with-extends`, `--with-classes`, `--with-type`. |
| upsert | `upsert block`, `upsert page`, `upsert tag`, `upsert property` | `--id`, `--uuid`, `--target-id`, `--target-uuid`, `--target-page`, `--pos`, `--content`, `--blocks-file`, `--status`, `--update-tags`, `--update-properties`, `--remove-tags`, `--remove-properties`, `--page`, `--name`, `--type`, `--cardinality`, `--hide`, `--public`. |
| remove | `remove block`, `remove page`, `remove tag`, `remove property` | `--id`, `--uuid`, `--name`. |
| query | `query`, `query list` | `--query`, `--name`, `--inputs`. |
| show | `show` | `--id`, `--uuid`, `--page`, `--linked-references`, `--level`, stdin `--id` input path. |
| server | `server list`, `server status`, `server start`, `server stop`, `server restart` | Command-specific graph use through `--graph`. |
| doctor | `doctor` | `--dev-script`. |
| completion | `completion` | Positional `zsh` or `bash`, and `--shell`. |

## Testing Plan

I will follow `@test-driven-development` for this refactor and write failing tests first in the new `cli-e2e` harness before implementation logic.
I will keep the tests behavior-first by asserting process exit code, stdout or stderr payload, file-system side effects, and db state changes after shell commands.

I will add a failing manifest coverage test that enumerates all in-scope subcommands and key options and fails when any item is missing from executed cases.
I will add failing runner tests that assert commands are executed as explicit shell strings and that each case logs the exact command line.
I will add failing preflight tests that assert build preflight runs before cases and that required compiled artifacts exist.
I will add failing e2e cases for each in-scope command and each key option from the inventory table, using positive and validation-error scenarios.
I will add regression tests for piping and stdin (`query | show --id`) to preserve shell-native workflows.
I will add tests that verify db-worker-node lifecycle commands (`server start/status/restart/stop`) operate against compiled runtime.

NOTE: I will write *all* tests before I add any implementation behavior.

## Target layout and integration points

The new harness will live under `/Users/rcmerci/gh-repos/logseq/cli-e2e/`.
The existing `src/test/logseq/cli/integration_test.cljs` namespace will be reduced or split so in-scope command responsibilities move to the new harness.

```text
+----------------------+        shell exec        +------------------------------+
| cli-e2e bb runner    | -----------------------> | node static/logseq-cli.js    |
| (bb + EDN manifests) |                          +--------------+---------------+
+----------+-----------+                                         |
           |                                                     | spawns
           | preflight build                                    v
           |                                         +------------------------------+
           +---------------------------------------> | dist/db-worker-node.js       |
                                                     | via logseq.cli.server         |
                                                     +------------------------------+
```

## Detailed implementation plan

1. Create `/Users/rcmerci/gh-repos/logseq/cli-e2e/` with `bb.edn`, `README.md`, and minimal source directories.
2. Add `/Users/rcmerci/gh-repos/logseq/cli-e2e/bb.edn` tasks `build`, `test`, and `list-cases`.
3. Use `@clojure-babashka-cli` patterns in `bb.edn` to parse `--case`, `--include`, `--skip-build`, and `--verbose`.
4. Add `/Users/rcmerci/gh-repos/logseq/cli-e2e/src/logseq/cli/e2e/paths.clj` to resolve absolute repo-root aware paths.
5. Add `/Users/rcmerci/gh-repos/logseq/cli-e2e/src/logseq/cli/e2e/shell.clj` with a single shell execution utility that records exact command strings.
6. Add `/Users/rcmerci/gh-repos/logseq/cli-e2e/src/logseq/cli/e2e/preflight.clj` to run build preflight commands.
7. Implement preflight commands as `clojure -M:cljs compile logseq-cli db-worker-node` followed by `yarn db-worker-node:compile:bundle`.
8. Add preflight artifact existence checks for `/Users/rcmerci/gh-repos/logseq/static/logseq-cli.js`, `/Users/rcmerci/gh-repos/logseq/static/db-worker-node.js`, `/Users/rcmerci/gh-repos/logseq/dist/db-worker-node.js`, and `/Users/rcmerci/gh-repos/logseq/dist/db-worker-node-assets.json`.
9. Add `/Users/rcmerci/gh-repos/logseq/cli-e2e/spec/non_sync_inventory.edn` to declare required command and key-option coverage excluding sync, login, and logout.
10. Add `/Users/rcmerci/gh-repos/logseq/cli-e2e/spec/non_sync_cases.edn` to declare all shell cases with `:cmd`, `:expect`, and `:covers`.
11. Add `/Users/rcmerci/gh-repos/logseq/cli-e2e/src/logseq/cli/e2e/coverage.clj` that computes missing command and key-option coverage from manifests.
12. Write a failing test that coverage checker reports missing entries before cases are added.
13. Add `/Users/rcmerci/gh-repos/logseq/cli-e2e/src/logseq/cli/e2e/fixtures.clj` for temp data-dir, temp config, and graph seed helpers.
14. Implement graph seed helper using explicit shell commands only, not direct namespace calls.
15. Add base cases for global key-option coverage in `/Users/rcmerci/gh-repos/logseq/cli-e2e/spec/non_sync_cases.edn`.
16. Add base cases for every `graph` subcommand and its options, including negative validation paths.
17. Add base cases for every `list` subcommand and key list options with deterministic assertions.
18. Add base cases for every `upsert` subcommand and key options, including selector conflict errors and file-based block insertion.
19. Add base cases for every `remove` subcommand and key options, including uuid and name resolution behavior.
20. Add base cases for `query` and `query list`, including `--query`, `--name`, and `--inputs` paths.
21. Add base cases for `show` including `--id`, `--uuid`, `--page`, `--linked-references`, `--level`, and stdin id input.
22. Add base cases for `server` commands and assert lifecycle transitions through `server status`.
23. Add base cases for `doctor --dev-script` and assert readable script check behavior.
24. Add base cases for `completion` positional and `--shell` usage and assert shell snippet markers.
25. Add case-runner assertions for stdout parsing in `json`, `edn`, and `human` output modes.
26. Add `/Users/rcmerci/gh-repos/logseq/cli-e2e/src/logseq/cli/e2e/report.clj` to print pass or fail summary with missing coverage details.
27. Add `/Users/rcmerci/gh-repos/logseq/cli-e2e/README.md` with setup, run commands, and expected artifacts.
28. Add root task wiring in `/Users/rcmerci/gh-repos/logseq/bb.edn` for `dev:cli-e2e` delegating to `cli-e2e/bb.edn`.
29. Add package script wiring in `/Users/rcmerci/gh-repos/logseq/package.json` for `cli:e2e` and `cli:e2e:skip-build`.
30. Move or slim in-scope tests out of `/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/integration_test.cljs` and keep only compatibility coverage that still belongs in cljs unit scope.
31. Add a deprecation note in `/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/integration_test.cljs` pointing engineers to `cli-e2e/`.
32. Run the full new `cli-e2e` suite with build preflight enabled and confirm zero missing coverage.
33. Run a focused subset using `--case` to validate developer ergonomics and rerun speed.
34. Run `bb dev:lint-and-test` to confirm the migration does not break existing unit test workflows.

## Verification commands and expected outcomes

| Command | Expected outcome |
| --- | --- |
| `bb -f /Users/rcmerci/gh-repos/logseq/cli-e2e/bb.edn build` | Compiles CLI and db-worker artifacts and passes artifact existence checks. |
| `bb -f /Users/rcmerci/gh-repos/logseq/cli-e2e/bb.edn test` | Runs all in-scope cases and prints zero missing command or key-option coverage. |
| `bb -f /Users/rcmerci/gh-repos/logseq/cli-e2e/bb.edn test --case show-stdin-id` | Runs one targeted case with exact shell command output shown. |
| `bb dev:cli-e2e` | Delegates to `cli-e2e` full run from repository root. |
| `bb dev:lint-and-test` | Existing lint and unit suites remain green after migration. |

## Edge cases to include in case manifests

1. Graph names and data-dir paths that include spaces and legacy-encoded names.
2. `show --id` stdin behavior with empty stdin and invalid EDN input.
3. Boolean option coercion for `--with-type`, `--linked-references`, `--hide`, and `--public`.
4. Mutually exclusive selectors such as `upsert page --id` with `--page` and `remove block --id` with `--uuid`.
5. Output format assertions for `--output human`, `--output json`, and `--output edn`.
6. `query` argument validation for `--query` and `--name` conflict and invalid `--inputs` EDN.
7. `completion` unsupported shell error path.
8. `server` command behavior when the target graph server is missing or already running.
9. `doctor --dev-script` when static script exists and when path is unreadable in a controlled failure case.
10. Ensure excluded commands (`sync`, `login`, `logout`) are rejected by coverage checker if accidentally added to in-scope inventory.

## Clarifications and assumptions captured in implementation

1. Key option coverage means each in-scope key option must appear in at least one behaviorally asserted case and not merely in help text.
2. Combinatorial explosion is controlled by one-option-at-least-once coverage plus targeted interaction cases for known conflicts.
3. Sync commands plus `login` and `logout` are explicitly excluded from this plan and remain covered by other test scopes.
4. The canonical runtime under test for CLI-managed server lifecycle is `dist/db-worker-node.js`, so bundle build is part of preflight.
5. The new suite is the source of truth for shell e2e behavior, while cljs namespace tests remain unit and component level.

## Testing Details

The new tests validate real behavior at process boundaries by invoking compiled binaries through shell commands and asserting outputs, files, and db effects.
Coverage checks verify every declared in-scope subcommand and key option has at least one asserted case.
Failure output will include missing coverage entries and the exact failing shell command for fast diagnosis.

## Implementation Details

- Use `cli-e2e/spec/non_sync_inventory.edn` as the single source of required in-scope command and key-option coverage.
- Use `cli-e2e/spec/non_sync_cases.edn` as declarative executable test cases with explicit `:covers`.
- Use one shared shell executor to keep command logging and error reporting consistent.
- Build preflight must run by default and may be skipped only with explicit `--skip-build`.
- Keep all test setup through shell commands to satisfy shell-first requirement.
- Use temp dirs per case to avoid cross-test contamination.
- Keep `login` and `logout` out of this suite by explicit inventory exclusions and checker guards.
- Preserve at least one pipeline test that uses stdout to stdin command chaining.
- Keep root task wiring minimal and avoid coupling `cli-e2e` to existing cljs test runner.
- Document command examples and troubleshooting in `cli-e2e/README.md`.

## Question

Do you want `login` and `logout` to stay permanently out of `cli-e2e`, or should we plan a separate optional suite for them under a dedicated `:manual` or `:network` tag?


---