clone/logseq
1
0
Fork 0
mirror of https://github.com/logseq/logseq.git synced 2026-05-24 20:54:09 +00:00
Code Issues Packages Projects Releases Wiki Activity

enhance(cli): add options in export cmd

Browse Source
This commit is contained in:
rcmerci
2026-05-03 13:37:16 +08:00
parent faf6317bb7
commit 7b68c57056
9 changed files with 554 additions and 95 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
docs/agent-guide/logseq-cli/001-logseq-cli.md
View File

@@ -298,6 +298,10 @@ Graph management commands use `--graph` or the configured current graph, dependi
- `graph validate --graph <name> [--fix]` delegates validation to the worker.
- `graph info --graph <name>` reads graph metadata and `logseq.kv/*` values; human output redacts sensitive KV keys matching token/secret/password.
- `graph export --graph <name> --type edn|sqlite --file <path>` exports EDN graph data or a SQLite snapshot.
- For `--type edn`, the CLI also accepts `--include-timestamps`, `--exclude-built-in-pages`, and `--exclude-namespaces <csv>`.
- `--exclude-namespaces` is normalized as trimmed CSV values with empty segments removed and duplicates collapsed; the worker receives the final value as a keyword set.
- `--exclude-namespaces` intentionally reduces some backend export validation strictness because excluded ontology namespaces cannot be validated the same way.
- For `--type sqlite`, the CLI rejects those EDN-only flags and invokes `:thread-api/backup-db-sqlite` so the worker writes the snapshot directly to the requested file path.
- `graph import --graph <name> --type edn|sqlite --input <path>` imports EDN or SQLite data. SQLite import requires the target graph to be missing; EDN import can target a graph action that the worker can open.
Graph backups are SQLite snapshot helpers under the graph's backup directory:

293
docs/agent-guide/logseq-cli/004-graph-export.md Normal file
View File

@@ -0,0 +1,293 @@
# CLI Graph Export Optimization Implementation Plan
Goal: Improve `logseq graph export` so it exposes the useful EDN graph export options already supported by the worker stack and removes avoidable SQLite export overhead without adding a new thread API unless existing APIs prove insufficient.
Architecture: Keep `graph export` as the single public command surface and reuse the current `db-worker-node` transport contract instead of introducing a parallel export path.
Architecture: Route EDN exports through the existing `:thread-api/export-edn` graph-options payload, route SQLite exports through the existing `:thread-api/backup-db-sqlite` direct-to-path capability, and reject option combinations that do not make sense for the chosen export type.
Tech Stack: ClojureScript, Node `fs`, Logseq CLI command parsing, `db-worker-node` HTTP transport, `logseq.db.sqlite.export`, and existing CLI/db-worker tests.
Related: Builds on `/Users/rcmerci/gh-repos/logseq/docs/agent-guide/logseq-cli/001-logseq-cli.md`.
## Problem statement
The current `logseq graph export` implementation is functionally correct but unnecessarily narrow.
The public CLI surface in `/Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/graph.cljs` exposes only `--type` and `--file`, even though the worker-side EDN export path already supports several graph export options through `logseq.db.sqlite.export/build-export`.
The current SQLite export path is also more expensive than it needs to be.
`/Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/graph.cljs` currently calls `:thread-api/export-db-base64`, receives a base64 string over HTTP, decodes it back into a `Buffer`, and only then writes it to disk.
That means the command performs extra encoding, extra decoding, and extra in-memory copying for the largest export payload shape.
At the same time, the worker already exposes `/Users/rcmerci/gh-repos/logseq/src/main/frontend/worker/db_core.cljs` `:thread-api/backup-db-sqlite`, which checkpoints the DB and writes the SQLite file directly to a destination path.
The existing implementation therefore has a mismatch between current capability and current command design.
There is also a discoverability and migration gap.
The older CLI exposed several EDN graph export options in `/Users/rcmerci/gh-repos/logseq/deps/cli/src/logseq/cli/spec.cljs`, and several of those options still map cleanly onto the current worker export implementation.
The implementation plan should recover the useful options that still fit the new CLI architecture while explicitly rejecting old options that belonged to the legacy API-server workflow or to dev-only validation flows.
A key constraint for this work is to avoid adding another worker thread API unless profiling or a hard behavior gap proves that the existing APIs cannot support the desired CLI behavior.
## Testing Plan
Implementation should follow @Test-Driven Development.
I will add command parsing tests in `/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/commands_test.cljs` before changing any implementation so the new CLI surface is locked down first.
I will add execution tests in `/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/commands_test.cljs` that prove EDN exports forward graph-options correctly and SQLite exports switch from `:thread-api/export-db-base64` to `:thread-api/backup-db-sqlite`.
I will add completion coverage in `/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/completion_generator_test.cljs` so the new export options appear in generated shell completions and `--type`-specific completions remain coherent.
I will update output formatting coverage in `/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/format_test.cljs` only if the human success line or error wording changes.
I will add or extend worker integration coverage in `/Users/rcmerci/gh-repos/logseq/src/test/frontend/worker/db_worker_node_test.cljs` only where the CLI plan depends on an already-existing worker contract that is currently under-specified by tests.
I will run the new parsing tests first and confirm they fail for the expected missing-option behavior before implementation begins.
I will run the new execute-path tests next and confirm they fail because the command still calls `:thread-api/export-db-base64` and still omits the new EDN graph-options payload.
I will run targeted completion tests after adding the option metadata and confirm the completion failures are caused by the new command surface not yet being wired in.
I will finish by running the focused CLI and worker test namespaces plus the relevant doc-facing smoke checks.
NOTE: I will write *all* tests before I add any implementation behavior.
## Current implementation snapshot
The current export flows are:
```text
CLI graph export --type edn
-> /Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/graph.cljs
-> transport/invoke :thread-api/export-edn
-> worker db_core.cljs calls sqlite-export/build-export
-> CLI writes EDN to --file
CLI graph export --type sqlite
-> /Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/graph.cljs
-> transport/invoke :thread-api/export-db-base64
-> worker db_core.cljs exports bytes then encodes base64
-> CLI decodes base64 to Buffer
-> CLI writes SQLite file to --file
```
The worker capabilities that already exist are:
```text
:thread-api/export-edn
Supports {:export-type :graph :graph-options {...}} through sqlite-export/build-export.
:thread-api/backup-db-sqlite
Checkpoints the DB and writes a SQLite backup directly to a destination path.
:thread-api/export-db-base64
Exports SQLite as base64 for transport, but adds avoidable payload inflation for file export.
```
The underlying EDN graph export options currently supported by `/Users/rcmerci/gh-repos/logseq/deps/db/src/logseq/db/sqlite/export.cljs` are:
| Option | Current backend support | Notes |
| --- | --- | --- |
| `:include-timestamps?` | Yes | Graph option already documented in `build-graph-export`. |
| `:exclude-namespaces` | Yes | Useful for ontology-heavy graphs, but validation becomes less strict. |
| `:exclude-built-in-pages?` | Yes | Supported today, with existing backend caveats. |
| `:exclude-files?` | Yes | Direct graph option already wired in backend export, but intentionally not exposed by this plan. |
## Goals
Expose the useful EDN graph export options that already map cleanly onto the current worker implementation.
Make SQLite export stop paying the base64 round-trip cost when the command’s only goal is to write a file to disk.
Keep one clear `graph export` command path instead of creating a second export command family.
Keep the worker API surface unchanged unless an implementation step proves that reuse is impossible.
Preserve current import compatibility by continuing to emit the same EDN graph export structure and the same SQLite snapshot format.
## Non-goals
Do not reintroduce the legacy API-server-token export flow from `/Users/rcmerci/gh-repos/logseq/deps/cli/src/logseq/cli/commands/export_edn.cljs`.
Do not revive the legacy Markdown zip export command from `/Users/rcmerci/gh-repos/logseq/deps/cli/src/logseq/cli/commands/export.cljs` as part of this work.
Do not expose non-graph export types such as `:page`, `:block`, `:view-nodes`, or `:graph-ontology` under `graph export` in this phase.
Do not expose old dev-only export switches such as `--validate`, `--roundtrip`, or `--catch-validation-errors` unless a separate product requirement appears.
Do not add a new streaming or file-writing thread API unless the existing `:thread-api/backup-db-sqlite` and `:thread-api/export-edn` paths prove insufficient.
## Recommended option scope
The old option inventory in `/Users/rcmerci/gh-repos/logseq/deps/cli/src/logseq/cli/spec.cljs` should be split into adopt, reject, and defer groups.
| Old option | Recommendation | Reason |
| --- | --- | --- |
| `--include-timestamps` | Adopt for `--type edn` | Directly supported by current graph export backend. |
| `--exclude-namespaces` | Adopt for `--type edn` | Directly supported and useful for seeded ontology graphs. |
| `--exclude-built-in-pages` | Adopt for `--type edn` | Directly supported by current backend export. |
| `--exclude-files` | Defer | Backend supports it, but this plan intentionally keeps the initial option set smaller. |
| `--validate` | Reject for now | Old command did extra local validation work that current CLI does not expose through worker APIs. |
| `--roundtrip` | Reject for now | Dev-only workflow that does more than export and would need separate orchestration. |
| `--catch-validation-errors` | Reject for now | Dev-only and intentionally permits invalid output. |
| Legacy `--api-server-token` | Reject | Belongs to the old API-server architecture, not current db-worker-node CLI. |
| Legacy EDN `--export-type` variants | Reject | `graph export` should remain a graph export command in this phase. |
| Legacy Markdown export command | Reject | Different artifact type and command semantics. |
## Proposed CLI behavior
The public command should remain `logseq graph export --type edn|sqlite --file <path> [--graph <name>]`.
When `--type edn` is selected, the command should additionally allow `--include-timestamps`, `--exclude-built-in-pages`, and `--exclude-namespaces <csv>`.
When `--type sqlite` is selected, those EDN-only options should be rejected with a clear invalid-options error instead of being silently ignored.
The EDN branch should keep calling `:thread-api/export-edn`, but it should pass `{:export-type :graph :graph-options {...}}` instead of the current hard-coded `{:export-type :graph}` payload.
The SQLite branch should stop calling `:thread-api/export-db-base64` and should instead call `:thread-api/backup-db-sqlite` with the resolved destination path.
The command should preserve the current success shape of writing to `--file` and then returning a normal `:graph-export` success result.
The human formatter can stay minimal unless we decide to include a short option summary in structured output.
## Execution steps
1. Write the new failing parser tests in `/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/commands_test.cljs` for each adopted EDN-only export option.
2. Write the failing parser tests that prove those EDN-only options are rejected for `--type sqlite`.
3. Write the failing execution test in `/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/commands_test.cljs` that expects EDN exports to forward `:graph-options` to `:thread-api/export-edn`.
4. Write the failing execution test in `/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/commands_test.cljs` that expects SQLite exports to call `:thread-api/backup-db-sqlite` instead of `:thread-api/export-db-base64`.
5. Write the failing completion tests in `/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/completion_generator_test.cljs` for the new export options.
6. Run the focused parser and execution tests and confirm every new case fails for the intended reason.
7. Extend `graph-export-spec` in `/Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/graph.cljs` with the adopted EDN-only options and keep the option descriptions implementation-aligned.
8. Add a small export-options normalization helper in `/Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/graph.cljs` that builds the `:graph-options` map only for `--type edn`.
9. Reuse existing CLI parsing conventions for comma-delimited multi-value options when implementing `--exclude-namespaces`, rather than inventing a one-off parser shape.
10. Update action-building in `/Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/graph.cljs` and `/Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/commands.cljs` so the selected export options are available at execution time and invalid combinations are rejected early.
11. Change the EDN branch of `execute-graph-export` in `/Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/graph.cljs` to call `:thread-api/export-edn` with `{:export-type :graph :graph-options ...}`.
12. Change the SQLite branch of `execute-graph-export` in `/Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/graph.cljs` to call `:thread-api/backup-db-sqlite` with the destination file path and remove the base64 decode path.
13. Keep `:thread-api/export-db-base64` unchanged in `/Users/rcmerci/gh-repos/logseq/src/main/frontend/worker/db_core.cljs` unless another caller analysis proves it is now dead code and safe to remove in a separate cleanup.
14. Update command help examples in `/Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/graph.cljs` so at least one EDN example demonstrates the new options and one SQLite example still shows the simple snapshot case.
15. Update `/Users/rcmerci/gh-repos/logseq/docs/cli/logseq-cli.md` to document the adopted EDN-only options and the fact that SQLite export is a direct snapshot path.
16. Update `/Users/rcmerci/gh-repos/logseq/docs/agent-guide/logseq-cli/001-logseq-cli.md` so the current-state CLI guide stays aligned with the new command surface.
17. Run the focused CLI command, completion, and format tests again and confirm they all pass after implementation.
18. Run the focused worker tests that cover `export-edn`, `backup-db-sqlite`, and SQLite import/export compatibility to confirm the reused worker contract is still green.
19. Refactor any duplicated export option assembly or validation code only after the green phase is complete.
20. Re-run all touched test namespaces after refactoring and confirm the suite stays green.
## File-by-file plan
`/Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/graph.cljs` will hold the public option spec changes, export option normalization, command examples, and the `execute-graph-export` switch from base64 SQLite export to direct backup export.
`/Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/commands.cljs` will need any command-validation updates required to reject EDN-only options for `--type sqlite` and to preserve current missing-type or missing-file behavior.
`/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/commands_test.cljs` will be the primary place for parse and execute regressions.
`/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/completion_generator_test.cljs` should be updated so shell completion coverage matches the new command surface.
`/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/format_test.cljs` only needs updates if the human success line or invalid-option wording changes.
`/Users/rcmerci/gh-repos/logseq/src/test/frontend/worker/db_worker_node_test.cljs` should be reviewed to ensure the direct backup-based SQLite export path is covered by the existing worker behavior tests.
`/Users/rcmerci/gh-repos/logseq/docs/cli/logseq-cli.md` must be updated because it is the operator-facing CLI reference.
`/Users/rcmerci/gh-repos/logseq/docs/agent-guide/logseq-cli/001-logseq-cli.md` must be updated because it is the implementation-aligned guide for future agents.
## Edge cases and failure handling
The implementation should reject EDN-only flags when `--type sqlite` is chosen.
The implementation should continue to fail fast when `--type` or `--file` is missing.
The implementation should preserve current graph selection semantics so `--graph` remains optional only when config already selects a graph.
The implementation should verify how `--exclude-namespaces` behaves when the user passes empty segments, duplicate values, or surrounding whitespace, and tests should define the normalization rule explicitly.
The implementation should preserve the current backend caveat that `exclude-namespaces` reduces some validation strictness, and the CLI docs should describe this briefly rather than pretending the option is free.
The implementation should confirm whether the destination parent directory error message from direct backup export is acceptable or whether CLI should wrap it with a clearer file-path-specific message.
The implementation should ensure the SQLite direct backup path still works when the server process owns the file write and the target path is outside the graph root.
The implementation should ensure that overwriting an existing export file follows the current Node filesystem behavior intentionally and is documented or validated consistently.
## Why this plan avoids a new thread API
The current worker stack already exposes the two capabilities this feature needs.
EDN graph export options already flow through `:thread-api/export-edn` into `logseq.db.sqlite.export/build-export`.
SQLite file export already has a direct-to-path worker contract in `:thread-api/backup-db-sqlite`.
Adding a new export thread API in this phase would duplicate behavior, expand the compatibility surface, and work against the repository rule to prefer one clear path when an existing path is already sufficient.
A new thread API should only be reconsidered if real profiling shows that the reused APIs still miss a required behavior or introduce an unacceptable correctness issue.
## Verification commands
Run `bb dev:test -v logseq.cli.commands-test/test-verb-subcommand-parse-graph-import-export` after writing the parser tests and expect new export-option assertions to fail before implementation.
Run `bb dev:test -v logseq.cli.commands-test/test-execute-graph-export` after writing the execution tests and expect the SQLite branch assertion to fail until the command stops using `:thread-api/export-db-base64`.
Run `bb dev:test -v logseq.cli.completion-generator-test` after adding completion coverage and expect new cases to fail before the command spec is updated.
Run `bb dev:test -v logseq.cli.format-test` if formatter strings change.
Run `bb dev:test -v frontend.worker.db-worker-node-test/db-worker-node-backup-db-sqlite` and the existing export/import worker tests after the CLI path changes to confirm the reused worker contract remains valid.
Run `bb dev:lint-and-test` after the focused suites are green if the change footprint stays small enough for a full verification pass.
## Testing Details
The test additions should prove behavior at the command boundary, not just helper internals.
The key behaviors are that the CLI accepts only the intended export options, forwards the exact EDN graph-options payload to the worker, routes SQLite file exports through direct backup instead of base64 transport, and keeps user-facing help and completion aligned.
Worker coverage should remain focused on actual export and import behavior rather than on transport implementation details.
## Implementation Details
- Reuse `:thread-api/export-edn` for all EDN export work in this phase.
- Reuse `:thread-api/backup-db-sqlite` for SQLite file export in this phase.
- Do not add a new thread API unless an implementation blocker is proven.
- Keep `graph export` scoped to full-graph exports only.
- Adopt only the old EDN options that map directly onto current `build-graph-export` support.
- Reject EDN-only options for SQLite instead of silently ignoring them.
- Keep help examples and shell completion synchronized with the spec changes.
- Keep `/Users/rcmerci/gh-repos/logseq/docs/cli/logseq-cli.md` and `/Users/rcmerci/gh-repos/logseq/docs/agent-guide/logseq-cli/001-logseq-cli.md` aligned with the implementation.
- Treat `exclude-namespaces` normalization as a tested contract.
- Defer dev-only validation and roundtrip workflows to a separate plan if they are still needed.
## Question
No further product questions are currently blocking implementation.
Locked decisions for this plan are:
- `--exclude-namespaces` should follow the existing CLI convention of a single comma-delimited value.
- `graph export --type sqlite` should keep the current overwrite behavior.
- `--exclude-files` should remain out of scope for this phase even though the backend supports it.
---

4
docs/cli/logseq-cli.md
View File

@@ -115,6 +115,10 @@ Graph commands:
- `graph validate --graph <name>` - validate graph data
- `graph info [--graph <name>]` - show graph metadata (defaults to current graph)
- `graph export --type edn|sqlite --file <path> [--graph <name>]` - export a graph to EDN or SQLite
- EDN export also accepts `--include-timestamps`, `--exclude-built-in-pages`, and `--exclude-namespaces <csv>`
- `--exclude-namespaces` trims CSV tokens, ignores empty tokens, removes duplicates, and can reduce backend export validation strictness
- SQLite export writes the snapshot directly to the destination path through `db-worker-node` instead of round-tripping a base64 payload through the CLI
- EDN-only export flags are rejected when `--type sqlite` is selected
- `graph import --type edn|sqlite --input <path> --graph <name>` - import a graph from EDN or SQLite (new graph only)
- `graph backup list` - list backup snapshots under `<root-dir>/graphs/<graph>/backup`
- `graph backup create [--graph <name>] [--name <label>]` - create a backup snapshot for the selected graph

13
src/main/logseq/cli/command/example.cljs
View File

@@ -6,7 +6,7 @@
[promesa.core :as p]))
(def ^:private phase1-groups
["list" "upsert" "remove" "query" "search" "show"])
["graph" "list" "upsert" "remove" "query" "search" "show"])
(defn- command-path->label
[cmds]
@@ -23,11 +23,12 @@
(defn phase1-target-entries
[base-table]
(->> base-table
(filter (fn [entry]
(contains? (set phase1-groups)
(first (:cmds entry)))))
vec))
(let [phase1-group-set (set phase1-groups)]
(->> base-table
(filter (fn [entry]
(contains? phase1-group-set
(first (:cmds entry)))))
vec)))
(defn- selector-definitions
[base-table]

114
src/main/logseq/cli/command/graph.cljs
View File

@@ -20,7 +20,12 @@
:validate #{"edn" "sqlite"}}
:file {:desc "Export file path"
:coerce common-graph/expand-home
:complete :file}})
:complete :file}
:include-timestamps {:desc "Include timestamps in export"
:coerce :boolean}
:exclude-built-in-pages {:desc "Exclude built-in pages"
:coerce :boolean}
:exclude-namespaces {:desc "Namespaces to exclude from properties and classes"}})
(def ^:private graph-import-spec
{:type {:desc "Import type"
@@ -50,7 +55,8 @@
(def ^:private backup-db-file-name "db.sqlite")
(def entries
[(core/command-entry ["graph" "list"] :graph-list "List graphs" {})
[(core/command-entry ["graph" "list"] :graph-list "List graphs" {}
{:examples ["logseq graph list"]})
(core/command-entry ["graph" "create"] :graph-create "Create graph" {}
{:examples ["logseq graph create --graph my-graph"]})
(core/command-entry ["graph" "switch"] :graph-switch "Switch current graph" {}
@@ -63,10 +69,12 @@
(core/command-entry ["graph" "info"] :graph-info "Graph metadata" {}
{:examples ["logseq graph info --graph my-graph"]})
(core/command-entry ["graph" "export"] :graph-export "Export graph" graph-export-spec
{:examples ["logseq graph export --graph my-graph --type edn --file /tmp/my-graph.edn"]})
{:examples ["logseq graph export --graph my-graph --type edn --file /tmp/my-graph.edn --include-timestamps --exclude-built-in-pages --exclude-namespaces user,project"
"logseq graph export --graph my-graph --type sqlite --file /tmp/my-graph.sqlite"]})
(core/command-entry ["graph" "import"] :graph-import "Import graph" graph-import-spec
{:examples ["logseq graph import --graph my-graph --type edn --input /tmp/my-graph.edn"]})
(core/command-entry ["graph" "backup" "list"] :graph-backup-list "List graph backups" {})
(core/command-entry ["graph" "backup" "list"] :graph-backup-list "List graph backups" {}
{:examples ["logseq graph backup list --graph my-graph"]})
(core/command-entry ["graph" "backup" "create"] :graph-backup-create "Create graph backup" graph-backup-create-spec
{:examples ["logseq graph backup create --graph my-graph"
"logseq graph backup create --graph my-graph --name nightly"]})
@@ -87,6 +95,60 @@
[value]
(some-> value string/lower-case string/trim))
(def ^:private graph-export-edn-only-option-keys
#{:include-timestamps :exclude-built-in-pages :exclude-namespaces})
(defn- parse-csv-option
[value]
(when (some? value)
(->> (string/split (str value) #",")
(map string/trim)
(remove string/blank?)
vec)))
(defn- normalize-csv-option
[value]
(when-let [values (seq (parse-csv-option value))]
(string/join "," values)))
(defn normalize-options
[command opts]
(if (= command :graph-export)
(cond-> opts
(contains? opts :exclude-namespaces) (update :exclude-namespaces normalize-csv-option))
opts))
(defn invalid-options?
[command opts]
(let [export-type (normalize-import-export-type (:type opts))
edn-only-options-specified? (some #(contains? opts %) graph-export-edn-only-option-keys)]
(cond
(and (= command :graph-export)
(= export-type "sqlite")
edn-only-options-specified?)
"graph export --type sqlite does not accept --include-timestamps, --exclude-built-in-pages, or --exclude-namespaces"
(and (= command :graph-export)
(= export-type "edn")
(contains? opts :exclude-namespaces)
(not (seq (:exclude-namespaces opts))))
"graph export --exclude-namespaces must include at least one non-empty value"
:else
nil)))
(defn- graph-export-options
[{:keys [include-timestamps exclude-built-in-pages exclude-namespaces]}]
(let [exclude-namespaces' (some->> exclude-namespaces
parse-csv-option
(map keyword)
set
not-empty)]
(cond-> {}
include-timestamps (assoc :include-timestamps? true)
exclude-built-in-pages (assoc :exclude-built-in-pages? true)
exclude-namespaces' (assoc :exclude-namespaces exclude-namespaces'))))
(defn- missing-graph-error
[]
{:ok? false
@@ -249,17 +311,19 @@
:graph (core/repo->graph repo)}})))
(defn build-export-action
[repo export-type file]
[repo export-type file options]
(if-not (seq repo)
{:ok? false
:error {:code :missing-repo
:message "repo is required for export"}}
{:ok? true
:action {:type :graph-export
:repo repo
:graph (core/repo->graph repo)
:export-type export-type
:file file}}))
:action (cond-> {:type :graph-export
:repo repo
:graph (core/repo->graph repo)
:export-type export-type
:file file}
(= export-type "edn")
(assoc :graph-options (graph-export-options options)))}))
(defn build-import-action
[repo import-type input]
@@ -514,23 +578,25 @@
[action config]
(-> (p/let [cfg (cli-server/ensure-server! config (:repo action))
export-type (:export-type action)
export-result (case export-type
"edn"
payload (cond-> {:export-type :graph}
(seq (:graph-options action))
(assoc :graph-options (:graph-options action)))
export-result (when (= export-type "edn")
(transport/invoke cfg
:thread-api/export-edn
false
[(:repo action) {:export-type :graph}])
"sqlite"
(transport/invoke cfg
:thread-api/export-db-base64
true
[(:repo action)])
(throw (ex-info "unsupported export type" {:export-type export-type})))
data (if (= export-type "sqlite")
(js/Buffer.from export-result "base64")
export-result)
format (if (= export-type "sqlite") :sqlite :edn)]
(transport/write-output {:format format :path (:file action) :data data})
[(:repo action) payload]))
_ (case export-type
"edn"
(transport/write-output {:format :edn
:path (:file action)
:data export-result})
"sqlite"
(transport/invoke cfg
:thread-api/backup-db-sqlite
true
[(:repo action) (:file action)])
(throw (ex-info "unsupported export type" {:export-type export-type}))) ]
{:status :ok
:data {:message (str "wrote " (:file action))}})))

13
src/main/logseq/cli/commands.cljs
View File

@@ -303,7 +303,8 @@
(defn- validate-option-contracts
[summary {:keys [command list-invalid-options-message remove-invalid-options-message
show-invalid-options-message debug-invalid-options-message]}]
show-invalid-options-message debug-invalid-options-message
graph-invalid-options-message]}]
(cond
(and (list-validation-commands command)
list-invalid-options-message)
@@ -313,6 +314,9 @@
remove-invalid-options-message)
(command-core/invalid-options-result summary remove-invalid-options-message)
(and (= command :graph-export) graph-invalid-options-message)
(command-core/invalid-options-result summary graph-invalid-options-message)
(and (= command :show) show-invalid-options-message)
(command-core/invalid-options-result summary show-invalid-options-message)
@@ -397,6 +401,8 @@
(list-command/invalid-options? command opts))
:remove-invalid-options-message (when (remove-validation-commands command)
(remove-command/invalid-options? command opts))
:graph-invalid-options-message (when (= command :graph-export)
(graph-command/invalid-options? command opts))
:show-invalid-options-message (when (= command :show)
(show-command/invalid-options? opts))
:debug-invalid-options-message (when (= command :debug-pull)
@@ -414,7 +420,8 @@
[summary {:keys [command opts args cmds spec long-desc examples]}]
(let [opts (-> opts
command-core/normalize-opts
(#(list-command/normalize-options command %)))
(#(list-command/normalize-options command %))
(#(graph-command/normalize-options command %)))
args (vec args)
cmd-summary (command-core/command-summary {:cmds cmds
:spec spec
@@ -590,7 +597,7 @@
:graph-export
(let [export-type (graph-command/normalize-import-export-type (:type options))]
(graph-command/build-export-action repo export-type (:file options)))
(graph-command/build-export-action repo export-type (:file options) options))
:graph-import
(let [import-repo (command-core/resolve-repo (:graph options))

37
src/test/logseq/cli/command/example_test.cljs
View File

@@ -20,7 +20,8 @@
(deftest test-phase1-target-filter
(let [targets (example-command/phase1-target-entries phase1-base-table)
groups (set (map (comp first :cmds) targets))]
groups (set (map (comp first :cmds) targets))
graph-export-present? (some #(= ["graph" "export"] (:cmds %)) targets)]
(testing "phase1 includes inspect/edit groups"
(is (contains? groups "list"))
(is (contains? groups "upsert"))
@@ -29,8 +30,9 @@
(is (contains? groups "search"))
(is (contains? groups "show")))
(testing "phase1 excludes graph management commands"
(is (not (contains? groups "graph"))))))
(testing "phase1 includes graph export examples and graph prefix support"
(is graph-export-present?)
(is (contains? groups "graph")))))
(deftest test-build-example-entries
(let [entries (example-command/build-example-entries phase1-base-table)
@@ -38,15 +40,14 @@
(testing "builds prefix selectors"
(is (contains? cmds-set ["example" "upsert"]))
(is (contains? cmds-set ["example" "query"]))
(is (contains? cmds-set ["example" "show"])))
(is (contains? cmds-set ["example" "show"]))
(is (contains? cmds-set ["example" "graph"])))
(testing "builds exact selectors"
(is (contains? cmds-set ["example" "upsert" "page"]))
(is (contains? cmds-set ["example" "search" "block"]))
(is (contains? cmds-set ["example" "query" "list"])))
(testing "does not build uncovered selectors"
(is (not (contains? cmds-set ["example" "graph"])))))
(is (contains? cmds-set ["example" "query" "list"]))
(is (contains? cmds-set ["example" "graph" "export"]))))
(testing "all generated entries use :example command keyword"
(is (every? #(= :example (:command %))
@@ -61,6 +62,24 @@
(is (= ["upsert page"] (get-in result [:action :matched-commands])))
(is (seq (get-in result [:action :examples])))))
(testing "builds graph export exact selector action"
(let [result (example-command/build-action phase1-base-table ["example" "graph" "export"])
examples (get-in result [:action :examples])]
(is (true? (:ok? result)))
(is (= "graph export" (get-in result [:action :selector])))
(is (= ["graph export"] (get-in result [:action :matched-commands])))
(is (= ["logseq graph export --graph my-graph --type edn --file /tmp/my-graph.edn --include-timestamps --exclude-built-in-pages --exclude-namespaces user,project"
"logseq graph export --graph my-graph --type sqlite --file /tmp/my-graph.sqlite"]
examples))))
(testing "builds graph prefix selector action"
(let [result (example-command/build-action phase1-base-table ["example" "graph"])
matched-commands (set (get-in result [:action :matched-commands]))]
(is (true? (:ok? result)))
(is (= "graph" (get-in result [:action :selector])))
(is (contains? matched-commands "graph export"))
(is (seq (get-in result [:action :examples])))))
(testing "builds prefix selector action"
(let [result (example-command/build-action phase1-base-table ["example" "upsert"])]
(is (true? (:ok? result)))
@@ -68,7 +87,7 @@
(is (<= 2 (count (get-in result [:action :matched-commands]))))))
(testing "rejects unknown selector"
(let [result (example-command/build-action phase1-base-table ["example" "graph"])]
(let [result (example-command/build-action phase1-base-table ["example" "sync"])]
(is (false? (:ok? result)))
(is (= :unknown-command (get-in result [:error :code])))))

155
src/test/logseq/cli/commands_test.cljs
View File

@@ -1,13 +1,12 @@
(ns logseq.cli.commands-test
(:require [cljs.test :refer [async deftest is testing]]
[babashka.cli :as cli]
(:require [babashka.cli :as cli]
[cljs.test :refer [async deftest is testing]]
[clojure.string :as string]
[logseq.cli.command.add :as add-command]
[logseq.cli.command.graph :as graph-command]
[logseq.cli.command.list :as list-command]
[logseq.cli.command.query :as query-command]
[logseq.cli.command.server :as server-command]
[logseq.db.frontend.rules :as rules]
[logseq.cli.command.show :as show-command]
[logseq.cli.command.sync :as sync-command]
[logseq.cli.command.upsert :as upsert-command]
@@ -15,6 +14,7 @@
[logseq.cli.server :as cli-server]
[logseq.cli.style :as style]
[logseq.cli.transport :as transport]
[logseq.db.frontend.rules :as rules]
[promesa.core :as p]))
(defn- strip-ansi
@@ -197,8 +197,8 @@
["search block" "search page" "search property" "search tag"]
["search block" "search page" "search property" "search tag"]]
["example"
["example upsert" "example upsert page" "example show"]
["example upsert" "example show"]]]]
["example graph" "example graph export" "example upsert" "example upsert page" "example show"]
["example graph" "example upsert" "example show"]]]]
(let [result (binding [style/*color-enabled?* true]
(commands/parse-args [group]))
summary (:summary result)
@@ -351,15 +351,14 @@
(testing "example query executes (not group help)"
(let [result (commands/parse-args ["example" "query"])]
(is (= :example (:command result)))
(is (not (:help? result)))))
)
(is (not (:help? result))))))
(deftest test-parse-args-example-selectors
(testing "example supports exact selectors"
(doseq [args [["example" "upsert" "page"]
["example" "show"]
["example" "search" "block"]]]
["example" "search" "block"]
["example" "graph" "export"]]]
(let [result (commands/parse-args args)]
(is (true? (:ok? result)))
(is (= :example (:command result))))))
@@ -367,13 +366,14 @@
(testing "example supports prefix selectors"
(doseq [args [["example" "upsert"]
["example" "list"]
["example" "query"]]]
["example" "query"]
["example" "graph"]]]
(let [result (commands/parse-args args)]
(is (true? (:ok? result)))
(is (= :example (:command result))))))
(testing "example rejects uncovered selectors"
(let [result (commands/parse-args ["example" "graph"])]
(let [result (commands/parse-args ["example" "sync"])]
(is (false? (:ok? result)))
(is (= :unknown-command (get-in result [:error :code]))))))
@@ -1604,9 +1604,7 @@
(testing "remove block rejects invalid id vector"
(let [result (commands/parse-args ["remove" "block" "--id" "[1 \"no\"]"])]
(is (false? (:ok? result)))
(is (= :invalid-options (get-in result [:error :code])))))
)
(is (= :invalid-options (get-in result [:error :code]))))))
(deftest test-verb-subcommand-parse-upsert-entity
(testing "upsert tag parses with name"
@@ -2184,6 +2182,19 @@
(is (= "edn" (get-in result [:options :type])))
(is (= "export.edn" (get-in result [:options :file])))))
(testing "graph export parses EDN-only options"
(let [result (commands/parse-args ["graph" "export"
"--type" "edn"
"--file" "export.edn"
"--include-timestamps"
"--exclude-built-in-pages"
"--exclude-namespaces" "user,project"])]
(is (true? (:ok? result)))
(is (= :graph-export (:command result)))
(is (= true (get-in result [:options :include-timestamps])))
(is (= true (get-in result [:options :exclude-built-in-pages])))
(is (= "user,project" (get-in result [:options :exclude-namespaces])))))
(testing "graph import parses with type, input, and repo"
(let [result (commands/parse-args ["graph" "import"
"--type" "sqlite"
@@ -2195,6 +2206,16 @@
(is (= "import.sqlite" (get-in result [:options :input])))
(is (= "demo" (get-in result [:options :graph])))))
(testing "graph export rejects EDN-only options for sqlite"
(let [result (commands/parse-args ["graph" "export"
"--type" "sqlite"
"--file" "export.sqlite"
"--include-timestamps"
"--exclude-built-in-pages"
"--exclude-namespaces" "user,project"])]
(is (false? (:ok? result)))
(is (= :invalid-options (get-in result [:error :code])))))
(testing "graph export requires type"
(let [result (commands/parse-args ["graph" "export" "--file" "export.edn"])]
(is (false? (:ok? result)))
@@ -2217,6 +2238,15 @@
(is (false? (:ok? result)))
(is (= :missing-graph (get-in result [:error :code])))))
(testing "graph import rejects unknown type"
(let [result (commands/parse-args ["graph" "import"
"--type" "zip"
"--input" "import.zip"
"--graph" "demo"])]
(is (false? (:ok? result)))
(is (= :invalid-options (get-in result [:error :code]))))))
(deftest test-verb-subcommand-parse-sync-graph-requirements
(testing "sync download requires graph"
(let [result (commands/parse-args ["sync" "download"])]
(is (false? (:ok? result)))
@@ -2240,16 +2270,9 @@
(let [result (commands/parse-args ["sync" "ensure-keys" "--upload-keys"])]
(is (true? (:ok? result)))
(is (= :sync-ensure-keys (:command result)))
(is (= true (get-in result [:options :upload-keys])))))
(testing "graph import rejects unknown type"
(let [result (commands/parse-args ["graph" "import"
"--type" "zip"
"--input" "import.zip"
"--graph" "demo"])]
(is (false? (:ok? result)))
(is (= :invalid-options (get-in result [:error :code])))))
(is (= true (get-in result [:options :upload-keys]))))))
(deftest test-verb-subcommand-parse-server-cleanup
(testing "server status is removed and parses as unknown command"
(let [result (commands/parse-args ["server" "status" "--graph" "demo"])]
(is (false? (:ok? result)))
@@ -2273,7 +2296,7 @@
(is (= :graph-backup-list (:command result)))))
(testing "graph backup create parses with optional name"
(let [result (commands/parse-args ["graph" "backup" "create" "--name" "nightly"]) ]
(let [result (commands/parse-args ["graph" "backup" "create" "--name" "nightly"])]
(is (true? (:ok? result)))
(is (= :graph-backup-create (:command result)))
(is (= "nightly" (get-in result [:options :name])))))
@@ -2281,7 +2304,7 @@
(testing "graph backup create with name carries source and naming components"
(let [result (commands/parse-args ["graph" "backup" "create"
"--graph" "demo"
"--name" "nightly"]) ]
"--name" "nightly"])]
(is (true? (:ok? result)))
(is (= "demo" (get-in result [:options :graph])))
(is (= "nightly" (get-in result [:options :name])))))
@@ -2289,7 +2312,7 @@
(testing "graph backup restore parses with --src and --dst"
(let [result (commands/parse-args ["graph" "backup" "restore"
"--src" "demo-nightly"
"--dst" "demo-restored"]) ]
"--dst" "demo-restored"])]
(is (true? (:ok? result)))
(is (= :graph-backup-restore (:command result)))
(is (= "demo-nightly" (get-in result [:options :src])))
@@ -2367,15 +2390,28 @@
(is (true? (:ok? result)))
(is (= :graph-export (get-in result [:action :type])))))
(testing "graph export builds normalized EDN graph-options"
(let [parsed {:ok? true
:command :graph-export
:options {:type "edn"
:file "export.edn"
:include-timestamps true
:exclude-built-in-pages true
:exclude-namespaces " user, project ,,user "}}
result (commands/build-action parsed {:graph "demo"})]
(is (true? (:ok? result)))
(is (= {:include-timestamps? true
:exclude-built-in-pages? true
:exclude-namespaces #{:user :project}}
(get-in result [:action :graph-options])))))
(testing "graph import requires repo"
(let [parsed {:ok? true
:command :graph-import
:options {:type "edn" :input "import.edn"}}
result (commands/build-action parsed {})]
(is (false? (:ok? result)))
(is (= :missing-repo (get-in result [:error :code])))))
)
(is (= :missing-repo (get-in result [:error :code]))))))
(deftest test-build-action-graph-backup
(testing "graph backup list resolves source repo from --graph"
@@ -3806,7 +3842,7 @@
(p/let [result (commands/execute action {})]
(is (= :ok (:status result)))
(is (= [[:save-block [{:block/uuid (uuid "00000000-0000-0000-0000-000000000001")
:block/title "Updated heading"} {}]]
:block/title "Updated heading"} {}]]
[:move-blocks [[(uuid "00000000-0000-0000-0000-000000000001")]
(uuid "00000000-0000-0000-0000-000000000002")
{:sibling? false :bottom? true}]]
@@ -3929,14 +3965,14 @@
(deftest test-execute-graph-import-rejects-existing-graph
(async done
(let [{:keys [action]} (graph-command/build-import-action "logseq_db_demo" "sqlite" "/tmp/test-db.sqlite")]
(-> (p/with-redefs [cli-server/list-graphs (fn [_] ["demo"])
cli-server/ensure-server! (fn [_ _]
(throw (ex-info "should not start server" {})))]
(p/let [result (commands/execute action {})]
(is (= :error (:status result)))
(is (= :graph-exists (get-in result [:error :code])))))
(p/catch (fn [e] (is false (str "unexpected error: " e))))
(p/finally done)))))
(-> (p/with-redefs [cli-server/list-graphs (fn [_] ["demo"])]
cli-server/ensure-server! (fn [_ _]
(throw (ex-info "should not start server" {})))
(p/let [result (commands/execute action {})]
(is (= :error (:status result)))
(is (= :graph-exists (get-in result [:error :code])))))
(p/catch (fn [e] (is false (str "unexpected error: " e))))
(p/finally done)))))
(deftest test-execute-graph-import-edn-allows-existing-graph
(async done
@@ -4003,23 +4039,40 @@
{:exported true}))
transport/write-output (fn [opts]
(swap! write-calls conj opts))]
(p/let [edn-result (commands/execute {:type :graph-export :repo "logseq_db_demo" :graph "demo" :export-type "edn" :file "/tmp/export.edn" :allow-missing-graph true} {})
sqlite-result (commands/execute {:type :graph-export :repo "logseq_db_demo" :graph "demo" :export-type "sqlite" :file "/tmp/export.sqlite" :allow-missing-graph true} {})]
(p/let [edn-result (commands/execute {:type :graph-export
:repo "logseq_db_demo"
:graph "demo"
:export-type "edn"
:graph-options {:include-timestamps? true
:exclude-built-in-pages? true
:exclude-namespaces #{:user :project}}
:file "/tmp/export.edn"
:allow-missing-graph true}
{})
sqlite-result (commands/execute {:type :graph-export
:repo "logseq_db_demo"
:graph "demo"
:export-type "sqlite"
:file "/tmp/export.sqlite"
:allow-missing-graph true}
{})]
(is (= :ok (:status edn-result)))
(is (= :ok (:status sqlite-result)))
(is (= "edn" (get-in edn-result [:context :export-type])))
(is (= "/tmp/export.edn" (get-in edn-result [:context :file])))
(is (= "sqlite" (get-in sqlite-result [:context :export-type])))
(is (= "/tmp/export.sqlite" (get-in sqlite-result [:context :file])))
(is (= [[:thread-api/export-edn false ["logseq_db_demo" {:export-type :graph}]]
[:thread-api/export-db-base64 true ["logseq_db_demo"]]]
(is (= [[:thread-api/export-edn false ["logseq_db_demo" {:export-type :graph
:graph-options {:include-timestamps? true
:exclude-built-in-pages? true
:exclude-namespaces #{:user :project}}}]]
[:thread-api/backup-db-sqlite true ["logseq_db_demo" "/tmp/export.sqlite"]]]
@invoke-calls))
(is (= 2 (count @write-calls)))
(let [[edn-write sqlite-write] @write-calls]
(is (= {:format :edn :path "/tmp/export.edn" :data {:exported true}} edn-write))
(is (= :sqlite (:format sqlite-write)))
(is (= "/tmp/export.sqlite" (:path sqlite-write)))
(is (= "sqlite" (.toString (:data sqlite-write) "utf8"))))))
(is (= 1 (count @write-calls)))
(is (= {:format :edn
:path "/tmp/export.edn"
:data {:exported true}}
(first @write-calls)))))
(p/catch (fn [e] (is false (str "unexpected error: " e))))
(p/finally done)))))
@@ -4158,9 +4211,9 @@
"test-repo" config)]
(is (true? (:ok? action-result)))
(-> (p/with-redefs [cli-server/ensure-server! (fn [config _] config)
transport/invoke (fn [_ _method _ args]
(reset! captured-args (second args))
(p/resolved []))]
transport/invoke (fn [_ _method _ args]
(reset! captured-args (second args))
(p/resolved []))]
(query-command/execute-query (:action action-result) config))
(p/then (fn [_]
(let [args @captured-args

16
src/test/logseq/cli/completion_generator_test.cljs
View File

@@ -131,6 +131,11 @@
(is (= #{"edn" "sqlite"} (get-in export-entry [:spec :type :validate]))))
(testing "export-spec :file has :complete :file"
(is (= :file (get-in export-entry [:spec :file :complete]))))
(testing "export-spec includes EDN-only options"
(is (= :boolean (get-in export-entry [:spec :include-timestamps :coerce])))
(is (= :boolean (get-in export-entry [:spec :exclude-built-in-pages :coerce])))
(is (= "Namespaces to exclude from properties and classes"
(get-in export-entry [:spec :exclude-namespaces :desc]))))
(testing "import-spec :type has :validate set"
(is (= #{"edn" "sqlite"} (get-in import-entry [:spec :type :validate]))))
(testing "import-spec :input has :complete :file"
@@ -283,6 +288,10 @@
(is (string/includes? output "_logseq()")))
(testing "output ends with compdef _logseq logseq"
(is (string/includes? output "compdef _logseq logseq")))
(testing "graph export completion includes EDN-only options"
(is (re-find #"(?s)_logseq_graph_export\(\).*--include-timestamps" output))
(is (re-find #"(?s)_logseq_graph_export\(\).*--exclude-built-in-pages" output))
(is (re-find #"(?s)_logseq_graph_export\(\).*--exclude-namespaces" output)))
(testing "boolean flags emit alias grouping"
(is (string/includes? output "'{-v,--verbose}'[")))
(testing "global profile flag is present in zsh completion"
@@ -435,9 +444,12 @@
(is (string/includes? output "_logseq_is_value_opt()")))
(testing "output ends with complete -F _logseq logseq"
(is (string/includes? output "complete -F _logseq logseq")))
(testing "graph export case includes --type and --file"
(testing "graph export case includes --type, --file, and EDN-only options"
(is (string/includes? output "--type"))
(is (string/includes? output "--file")))
(is (string/includes? output "--file"))
(is (string/includes? output "--include-timestamps"))
(is (string/includes? output "--exclude-built-in-pages"))
(is (string/includes? output "--exclude-namespaces")))
(testing "graph backup options include --name, --src, and --dst"
(is (string/includes? output "--name"))
(is (string/includes? output "--src"))