Logseq CLI Integration Test Shell Refactor Implementation Plan

Goal: Build a new cli-e2e test suite that refactors logseq.cli.integration-test into babashka-driven shell e2e tests for compiled logseq-cli and db-worker-node, excluding sync, login, and logout, with full in-scope subcommand coverage and key-option coverage.

Architecture: Use a declarative command case manifest in cli-e2e/ and execute every case via explicit shell command strings so test execution is transparent and copy-pasteable. Architecture: Run a build preflight that compiles logseq-cli and db-worker-node artifacts before every e2e run, then run cases against those compiled artifacts only. Architecture: Enforce completeness with a coverage checker that fails when any in-scope subcommand or any declared key option is not covered by at least one case.

Tech Stack: Babashka, babashka.cli, babashka.process, EDN manifests, Node.js, shadow-cljs build targets logseq-cli and db-worker-node, existing Logseq CLI and db-worker-node runtime.

Related: Relates to /Users/rcmerci/gh-repos/logseq/docs/agent-guide/053-logseq-cli-async-test-isolation.md. Related: Relates to /Users/rcmerci/gh-repos/logseq/docs/agent-guide/058-db-worker-node-revision-and-cli-server-list.md. Related: Relates to /Users/rcmerci/gh-repos/logseq/docs/agent-guide/060-cli-graph-list-legacy-graph-dir-rename-command.md.

Problem statement

The current src/test/logseq/cli/integration_test.cljs suite mostly calls logseq.cli.main/run! directly instead of executing logseq-cli as a shell command. This makes command-line behavior, quoting, piping, process lifecycle, and compiled artifact boundaries under-tested. The current suite also relies heavily on with-redefs for sync and auth flows, which hides process-level integration issues. The suite location and runtime are tied to shadow-cljs :test, while the new requirement is an independent cli-e2e/ babashka flow that tests compiled binaries. The new requirement also asks for in-scope subcommand completeness with key option coverage enforcement, which is not explicitly guaranteed today.

Current implementation snapshot

Area	Current file	Current behavior	Gap against requirement
CLI integration tests	`/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/integration_test.cljs`	Runs in cljs test bundle and invokes `run!` directly for most tests.	Not fully shell-first and not isolated under `cli-e2e/`.
CLI compiled artifact	`/Users/rcmerci/gh-repos/logseq/static/logseq-cli.js`	Generated by `shadow-cljs` target `logseq-cli`.	Not currently enforced as precondition in integration test flow.
db-worker runtime used by CLI server lifecycle	`/Users/rcmerci/gh-repos/logseq/dist/db-worker-node.js`	Spawned by `logseq.cli.server/db-worker-script-path`.	Must be rebuilt before e2e to ensure latest runtime.
Existing integration runner entrypoint	`/Users/rcmerci/gh-repos/logseq/package.json` (`cljs:run-integration-test`)	Runs `node static/tests.js -r '^(logseq.cli.integration-test).*'`.	Keeps tests inside cljs test harness instead of babashka shell runner.

In-scope command and key option inventory

The e2e suite target excludes every sync * command and excludes login and logout. The suite treats global options and command options as separate coverage buckets. The option inventory below is the key-option scope for this phase.

Scope	Commands	Options to cover
Global	Any in-scope command invocation	`--help`, `--version`, `--config`, `--graph`, `--data-dir`, `--output`, `--verbose`.
graph	`graph list`, `graph create`, `graph switch`, `graph remove`, `graph validate`, `graph info`, `graph export`, `graph import`	`--fix`, `--type`, `--file`, `--input`.
list	`list page`, `list tag`, `list property`	`--expand`, `--fields`, `--limit`, `--offset`, `--sort`, `--order`, `--journal-only`, `--with-properties`, `--with-extends`, `--with-classes`, `--with-type`.
upsert	`upsert block`, `upsert page`, `upsert tag`, `upsert property`	`--id`, `--uuid`, `--target-id`, `--target-uuid`, `--target-page`, `--pos`, `--content`, `--blocks-file`, `--status`, `--update-tags`, `--update-properties`, `--remove-tags`, `--remove-properties`, `--page`, `--name`, `--type`, `--cardinality`, `--hide`, `--public`.
remove	`remove block`, `remove page`, `remove tag`, `remove property`	`--id`, `--uuid`, `--name`.
query	`query`, `query list`	`--query`, `--name`, `--inputs`.
show	`show`	`--id`, `--uuid`, `--page`, `--linked-references`, `--level`, stdin `--id` input path.
server	`server list`, `server status`, `server start`, `server stop`, `server restart`	Command-specific graph use through `--graph`.
doctor	`doctor`	`--dev-script`.
completion	`completion`	Positional `zsh` or `bash`, and `--shell`.

Testing Plan

I will follow @test-driven-development for this refactor and write failing tests first in the new cli-e2e harness before implementation logic. I will keep the tests behavior-first by asserting process exit code, stdout or stderr payload, file-system side effects, and db state changes after shell commands.

I will add a failing manifest coverage test that enumerates all in-scope subcommands and key options and fails when any item is missing from executed cases. I will add failing runner tests that assert commands are executed as explicit shell strings and that each case logs the exact command line. I will add failing preflight tests that assert build preflight runs before cases and that required compiled artifacts exist. I will add failing e2e cases for each in-scope command and each key option from the inventory table, using positive and validation-error scenarios. I will add regression tests for piping and stdin (query | show --id) to preserve shell-native workflows. I will add tests that verify db-worker-node lifecycle commands (server start/status/restart/stop) operate against compiled runtime.

NOTE: I will write all tests before I add any implementation behavior.

Target layout and integration points

The new harness will live under /Users/rcmerci/gh-repos/logseq/cli-e2e/. The existing src/test/logseq/cli/integration_test.cljs namespace will be reduced or split so in-scope command responsibilities move to the new harness.

+----------------------+        shell exec        +------------------------------+
| cli-e2e bb runner    | -----------------------> | node static/logseq-cli.js    |
| (bb + EDN manifests) |                          +--------------+---------------+
+----------+-----------+                                         |
           |                                                     | spawns
           | preflight build                                    v
           |                                         +------------------------------+
           +---------------------------------------> | dist/db-worker-node.js       |
                                                     | via logseq.cli.server         |
                                                     +------------------------------+

Detailed implementation plan

Create /Users/rcmerci/gh-repos/logseq/cli-e2e/ with bb.edn, README.md, and minimal source directories.
Add /Users/rcmerci/gh-repos/logseq/cli-e2e/bb.edn tasks build, test, and list-cases.
Use @clojure-babashka-cli patterns in bb.edn to parse --case, --include, --skip-build, and --verbose.
Add /Users/rcmerci/gh-repos/logseq/cli-e2e/src/logseq/cli/e2e/paths.clj to resolve absolute repo-root aware paths.
Add /Users/rcmerci/gh-repos/logseq/cli-e2e/src/logseq/cli/e2e/shell.clj with a single shell execution utility that records exact command strings.
Add /Users/rcmerci/gh-repos/logseq/cli-e2e/src/logseq/cli/e2e/preflight.clj to run build preflight commands.
Implement preflight commands as clojure -M:cljs compile logseq-cli db-worker-node followed by pnpm db-worker-node:compile:bundle.
Add preflight artifact existence checks for /Users/rcmerci/gh-repos/logseq/static/logseq-cli.js, /Users/rcmerci/gh-repos/logseq/static/db-worker-node.js, /Users/rcmerci/gh-repos/logseq/dist/db-worker-node.js, and /Users/rcmerci/gh-repos/logseq/dist/db-worker-node-assets.json.
Add /Users/rcmerci/gh-repos/logseq/cli-e2e/spec/non_sync_inventory.edn to declare required command and key-option coverage excluding sync, login, and logout.
Add /Users/rcmerci/gh-repos/logseq/cli-e2e/spec/non_sync_cases.edn to declare all shell cases with :cmd, :expect, and :covers.
Add /Users/rcmerci/gh-repos/logseq/cli-e2e/src/logseq/cli/e2e/coverage.clj that computes missing command and key-option coverage from manifests.
Write a failing test that coverage checker reports missing entries before cases are added.
Add /Users/rcmerci/gh-repos/logseq/cli-e2e/src/logseq/cli/e2e/fixtures.clj for temp data-dir, temp config, and graph seed helpers.
Implement graph seed helper using explicit shell commands only, not direct namespace calls.
Add base cases for global key-option coverage in /Users/rcmerci/gh-repos/logseq/cli-e2e/spec/non_sync_cases.edn.
Add base cases for every graph subcommand and its options, including negative validation paths.
Add base cases for every list subcommand and key list options with deterministic assertions.
Add base cases for every upsert subcommand and key options, including selector conflict errors and file-based block insertion.
Add base cases for every remove subcommand and key options, including uuid and name resolution behavior.
Add base cases for query and query list, including --query, --name, and --inputs paths.
Add base cases for show including --id, --uuid, --page, --linked-references, --level, and stdin id input.
Add base cases for server commands and assert lifecycle transitions through server status.
Add base cases for doctor --dev-script and assert readable script check behavior.
Add base cases for completion positional and --shell usage and assert shell snippet markers.
Add case-runner assertions for stdout parsing in json, edn, and human output modes.
Add /Users/rcmerci/gh-repos/logseq/cli-e2e/src/logseq/cli/e2e/report.clj to print pass or fail summary with missing coverage details.
Add /Users/rcmerci/gh-repos/logseq/cli-e2e/README.md with setup, run commands, and expected artifacts.
Add root task wiring in /Users/rcmerci/gh-repos/logseq/bb.edn for dev:cli-e2e delegating to cli-e2e/bb.edn.
Add package script wiring in /Users/rcmerci/gh-repos/logseq/package.json for cli:e2e and cli:e2e:skip-build.
Move or slim in-scope tests out of /Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/integration_test.cljs and keep only compatibility coverage that still belongs in cljs unit scope.
Add a deprecation note in /Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/integration_test.cljs pointing engineers to cli-e2e/.
Run the full new cli-e2e suite with build preflight enabled and confirm zero missing coverage.
Run a focused subset using --case to validate developer ergonomics and rerun speed.
Run bb dev:lint-and-test to confirm the migration does not break existing unit test workflows.

Verification commands and expected outcomes

Command	Expected outcome
`bb -f /Users/rcmerci/gh-repos/logseq/cli-e2e/bb.edn build`	Compiles CLI and db-worker artifacts and passes artifact existence checks.
`bb -f /Users/rcmerci/gh-repos/logseq/cli-e2e/bb.edn test`	Runs all in-scope cases and prints zero missing command or key-option coverage.
`bb -f /Users/rcmerci/gh-repos/logseq/cli-e2e/bb.edn test --case show-stdin-id`	Runs one targeted case with exact shell command output shown.
`bb dev:cli-e2e`	Delegates to `cli-e2e` full run from repository root.
`bb dev:lint-and-test`	Existing lint and unit suites remain green after migration.

Edge cases to include in case manifests

Graph names and data-dir paths that include spaces and legacy-encoded names.
show --id stdin behavior with empty stdin and invalid EDN input.
Boolean option coercion for --with-type, --linked-references, --hide, and --public.
Mutually exclusive selectors such as upsert page --id with --page and remove block --id with --uuid.
Output format assertions for --output human, --output json, and --output edn.
query argument validation for --query and --name conflict and invalid --inputs EDN.
completion unsupported shell error path.
server command behavior when the target graph server is missing or already running.
doctor --dev-script when static script exists and when path is unreadable in a controlled failure case.
Ensure excluded commands (sync, login, logout) are rejected by coverage checker if accidentally added to in-scope inventory.

Clarifications and assumptions captured in implementation

Key option coverage means each in-scope key option must appear in at least one behaviorally asserted case and not merely in help text.
Combinatorial explosion is controlled by one-option-at-least-once coverage plus targeted interaction cases for known conflicts.
Sync commands plus login and logout are explicitly excluded from this plan and remain covered by other test scopes.
The canonical runtime under test for CLI-managed server lifecycle is dist/db-worker-node.js, so bundle build is part of preflight.
The new suite is the source of truth for shell e2e behavior, while cljs namespace tests remain unit and component level.

Testing Details

The new tests validate real behavior at process boundaries by invoking compiled binaries through shell commands and asserting outputs, files, and db effects. Coverage checks verify every declared in-scope subcommand and key option has at least one asserted case. Failure output will include missing coverage entries and the exact failing shell command for fast diagnosis.

Implementation Details

Use cli-e2e/spec/non_sync_inventory.edn as the single source of required in-scope command and key-option coverage.
Use cli-e2e/spec/non_sync_cases.edn as declarative executable test cases with explicit :covers.
Use one shared shell executor to keep command logging and error reporting consistent.
Build preflight must run by default and may be skipped only with explicit --skip-build.
Keep all test setup through shell commands to satisfy shell-first requirement.
Use temp dirs per case to avoid cross-test contamination.
Keep login and logout out of this suite by explicit inventory exclusions and checker guards.
Preserve at least one pipeline test that uses stdout to stdin command chaining.
Keep root task wiring minimal and avoid coupling cli-e2e to existing cljs test runner.
Document command examples and troubleshooting in cli-e2e/README.md.

Question

Do you want login and logout to stay permanently out of cli-e2e, or should we plan a separate optional suite for them under a dedicated :manual or :network tag?

15 KiB Raw Permalink Blame History