019-logseq-cli-data-dir-permissions.md

This commit is contained in:
rcmerci
2026-01-28 14:31:28 +08:00
parent 7aba504052
commit 723c5d5a9e
9 changed files with 379 additions and 103 deletions

View File

@@ -0,0 +1,82 @@
# Logseq CLI Data-dir Permission Checks Plan
Goal: Make logseq-cli validate read/write access for `data-dir` before it tries to start or communicate with db-worker-node, and surface a clear error when permissions are missing.
Architecture: The CLI resolves `data-dir` in `logseq.cli.config` and uses it via `logseq.cli.server` to spawn and manage `db-worker-node`. `db-worker-node` also depends on `data-dir` for logs, locks, and SQLite storage via `frontend.worker.platform.node`.
Tech Stack: ClojureScript, Node.js fs APIs, promesa, logseq-cli, db-worker-node.
## Problem statement
`data-dir` is used for locks, logs, and the local SQLite DB. Today, permission issues show up as late runtime exceptions (e.g., during lock creation or log file writes) with unclear error output. The CLI should proactively check that `data-dir` is readable and writable and return a clear error before attempting db-worker-node actions.
## Current behavior summary
- `logseq.cli.config/resolve-config` defaults `:data-dir` to `~/.logseq/cli-graphs`.
- `logseq.cli.server` resolves and uses `data-dir` for locks and server discovery.
- `frontend.worker.db-worker-node` writes logs and lock files under `data-dir` and delegates storage to `frontend.worker.platform.node`, which creates directories as needed.
- No explicit read/write permission checks exist; errors bubble up from fs operations.
## Requirements
- CLI validates that `data-dir` is a directory with read and write permission.
- If `data-dir` does not exist, CLI attempts to create it (recursive). If creation or access fails, CLI returns an error.
- CLI surfaces a clear error code and message that includes the failing path.
- The check must run before any db-worker-node lifecycle or graph access that relies on `data-dir`.
## Non-goals
- Do not change db-worker-node storage layout or lock format.
- Do not add new CLI options for data-dir.
- Do not change API server behavior.
## Design decisions
- Treat `data-dir` as required to be read/write for all local-graph CLI commands.
- Convert permission failures into a consistent CLI error code (e.g., `:data-dir-permission`) and message.
- Reuse the same permission check in db-worker-node entrypoint to guard direct invocation.
## Implementation plan
### 1) Add a data-dir permission helper
- Create a helper namespace (e.g., `src/main/logseq/cli/data_dir.cljs`) that:
- Expands `~` and normalizes the path.
- If missing, attempts `fs.mkdirSync` with `{:recursive true}`.
- Verifies the path is a directory (`fs.statSync`).
- Verifies read/write access with `fs.accessSync` (R_OK | W_OK).
- Throws `ex-info` with `{:code :data-dir-permission :path <path> :cause <node-code>}` on failure.
### 2) Wire the check into CLI flow
- In `src/main/logseq/cli/main.cljs`, after `config/resolve-config`, call the permission helper before `commands/build-action`/`commands/execute`.
- If the CLI supports API-token-only commands that do not touch local graphs, gate the check to only run for actions that require local graph access or server management.
- Map thrown permission errors into CLI error output with a clear message (e.g., "data-dir is not readable/writable: <path>").
### 3) Add a safety check in db-worker-node
- In `src/main/frontend/worker/db_worker_node.cljs`, run the same permission helper (or a small local equivalent) before `install-file-logger!` and before `platform-node/node-platform`.
- When this check fails, print a concise error to stderr and exit with code 1 to avoid partial startup.
### 4) Update CLI error formatting
- In `src/main/logseq/cli/format.cljs`, add an error hint for `:data-dir-permission` (e.g., "Check filesystem permissions or set LOGSEQ_CLI_DATA_DIR").
- Ensure error output contains the path and permission type (read/write).
### 5) Tests
- Add unit tests in `src/test/logseq/cli` for the permission helper:
- Non-existent path that can be created succeeds.
- Path that is a file (not directory) fails.
- Read-only directory fails (use chmod to remove write permission in tmp dir).
- Add an integration test that runs CLI with `--data-dir` pointing to a non-writable directory and asserts the CLI returns error code `:data-dir-permission`.
- Add a graph-create case where `graph-dir` cannot be created (no mkdir permission) and assert a clear error is returned.
- Add a db-worker-node test (if there is a suitable harness) or extend existing CLI integration tests to assert db-worker-node start fails fast with the new error.
## Open questions
Resolved:
- Always check `data-dir` permissions, even when an API-server token is provided.
- Only create `data-dir` when a command needs it (local graph or server operations), not eagerly for all commands.
---

View File

@@ -8,6 +8,7 @@
[frontend.worker.platform.node :as platform-node]
[frontend.worker.state :as worker-state]
[lambdaisland.glogi :as log]
[logseq.cli.data-dir :as data-dir]
[logseq.db :as ldb]
[promesa.core :as p]))
@@ -300,65 +301,68 @@
port 0]
(if-not (seq repo)
(p/rejected (ex-info "repo is required" {:code :missing-repo}))
(do
(install-file-logger! {:data-dir data-dir
:repo repo
:log-level (keyword (or log-level "info"))})
(reset! *ready? false)
(set-main-thread-stub!)
(-> (p/let [platform (platform-node/node-platform {:data-dir data-dir
:event-fn handle-event!})
proxy (db-core/init-core! platform)
_ (<init-worker! proxy (or rtc-ws-url ""))
{:keys [path lock]} (db-lock/ensure-lock! {:data-dir data-dir
:repo repo
:host host
:port port})
_ (reset! *lock-info {:path path :lock lock})
_ (let [method-kw :thread-api/create-or-open-db
method-str (normalize-method-str method-kw)]
(<invoke! proxy method-str method-kw false [repo {}]))]
(let [stop!* (atom nil)
server (make-server proxy {:bound-repo repo
:stop-fn (fn []
(when-let [stop! @stop!*]
(stop!)))})]
(p/create
(fn [resolve reject]
(.listen server port host
(fn []
(let [address (.address server)
actual-port (if (number? address)
address
(.-port address))
stop! (fn []
(p/create
(fn [resolve _]
(reset! *ready? false)
(doseq [^js res @*sse-clients]
(try
(.end res)
(catch :default _)))
(reset! *sse-clients #{})
(when-let [lock-path (:path @*lock-info)]
(db-lock/remove-lock! lock-path))
(.close server (fn [] (resolve true))))))]
(reset! *ready? true)
(reset! stop!* stop!)
(p/let [lock' (assoc (:lock @*lock-info) :port actual-port)
_ (db-lock/update-lock! (:path @*lock-info) lock')]
(resolve {:host host
:port actual-port
:server server
:stop! stop!})))))
(.on server "error" (fn [error]
(when-let [lock-path (:path @*lock-info)]
(db-lock/remove-lock! lock-path))
(reject error)))))))
(p/catch (fn [e]
(when-let [lock-path (:path @*lock-info)]
(db-lock/remove-lock! lock-path))
(throw e))))))))
(try
(let [data-dir (data-dir/ensure-data-dir! data-dir)]
(install-file-logger! {:data-dir data-dir
:repo repo
:log-level (keyword (or log-level "info"))})
(reset! *ready? false)
(set-main-thread-stub!)
(-> (p/let [platform (platform-node/node-platform {:data-dir data-dir
:event-fn handle-event!})
proxy (db-core/init-core! platform)
_ (<init-worker! proxy (or rtc-ws-url ""))
{:keys [path lock]} (db-lock/ensure-lock! {:data-dir data-dir
:repo repo
:host host
:port port})
_ (reset! *lock-info {:path path :lock lock})
_ (let [method-kw :thread-api/create-or-open-db
method-str (normalize-method-str method-kw)]
(<invoke! proxy method-str method-kw false [repo {}]))]
(let [stop!* (atom nil)
server (make-server proxy {:bound-repo repo
:stop-fn (fn []
(when-let [stop! @stop!*]
(stop!)))})]
(p/create
(fn [resolve reject]
(.listen server port host
(fn []
(let [address (.address server)
actual-port (if (number? address)
address
(.-port address))
stop! (fn []
(p/create
(fn [resolve _]
(reset! *ready? false)
(doseq [^js res @*sse-clients]
(try
(.end res)
(catch :default _)))
(reset! *sse-clients #{})
(when-let [lock-path (:path @*lock-info)]
(db-lock/remove-lock! lock-path))
(.close server (fn [] (resolve true))))))]
(reset! *ready? true)
(reset! stop!* stop!)
(p/let [lock' (assoc (:lock @*lock-info) :port actual-port)
_ (db-lock/update-lock! (:path @*lock-info) lock')]
(resolve {:host host
:port actual-port
:server server
:stop! stop!})))))
(.on server "error" (fn [error]
(when-let [lock-path (:path @*lock-info)]
(db-lock/remove-lock! lock-path))
(reject error)))))))
(p/catch (fn [e]
(when-let [lock-path (:path @*lock-info)]
(db-lock/remove-lock! lock-path))
(throw e)))))
(catch :default e
(p/rejected e))))))
(defn main
[]
@@ -370,16 +374,23 @@
(when-not (seq repo)
(show-help!)
(.exit js/process 1))
(p/let [{:keys [stop!] :as daemon}
(start-daemon! {:data-dir data-dir
:repo repo
:rtc-ws-url rtc-ws-url
:log-level (:log-level opts)})]
(log/info :db-worker-node-ready {:host (:host daemon) :port (:port daemon)})
(let [shutdown (fn []
(-> (stop!)
(p/finally (fn []
(log/info :db-worker-node-stopped nil)
(.exit js/process 0)))))]
(.on js/process "SIGINT" shutdown)
(.on js/process "SIGTERM" shutdown)))))
(-> (p/let [{:keys [stop!] :as daemon}
(start-daemon! {:data-dir data-dir
:repo repo
:rtc-ws-url rtc-ws-url
:log-level (:log-level opts)})]
(log/info :db-worker-node-ready {:host (:host daemon) :port (:port daemon)})
(let [shutdown (fn []
(-> (stop!)
(p/finally (fn []
(log/info :db-worker-node-stopped nil)
(.exit js/process 0)))))]
(.on js/process "SIGINT" shutdown)
(.on js/process "SIGTERM" shutdown)))
(p/catch (fn [error]
(let [data (ex-data error)
message (or (.-message error) (str error))]
(when (= :data-dir-permission (:code data))
(.error js/console message)
(.exit js/process 1))
(throw error)))))))

View File

@@ -0,0 +1,42 @@
(ns logseq.cli.data-dir
"Data-dir validation and normalization for the CLI and db-worker-node."
(:require ["fs" :as fs]
["os" :as os]
["path" :as node-path]
[clojure.string :as string]))
(def ^:private default-data-dir "~/.logseq/cli-graphs")
(defn- expand-home
[path]
(if (and (seq path) (string/starts-with? path "~"))
(node-path/join (.homedir os) (subs path 1))
path))
(defn normalize-data-dir
[path]
(node-path/resolve (expand-home (or path default-data-dir))))
(defn ensure-data-dir!
[path]
(let [path (normalize-data-dir path)]
(try
(when-not (fs/existsSync path)
(fs/mkdirSync path #js {:recursive true}))
(let [stat (fs/statSync path)]
(when-not (.isDirectory stat)
(throw (ex-info (str "data-dir is not a directory: " path)
{:code :data-dir-permission
:path path
:cause "ENOTDIR"}))))
(let [constants (.-constants fs)
mode (bit-or (.-R_OK constants) (.-W_OK constants))]
(fs/accessSync path mode))
path
(catch :default e
(if (= :data-dir-permission (:code (ex-data e)))
(throw e)
(throw (ex-info (str "data-dir is not readable/writable: " path)
{:code :data-dir-permission
:path path
:cause (.-code e)})))))))

View File

@@ -88,6 +88,7 @@
:missing-content "Use --content or pass content as args"
:missing-query "Use --query <edn>"
:unknown-query "Use `logseq query list` to see available queries"
:data-dir-permission "Check filesystem permissions or set LOGSEQ_CLI_DATA_DIR"
nil))
(defn- format-error

View File

@@ -4,6 +4,7 @@
(:require [clojure.string :as string]
[logseq.cli.commands :as commands]
[logseq.cli.config :as config]
[logseq.cli.data-dir :as data-dir]
[logseq.cli.format :as format]
[logseq.cli.version :as version]
[promesa.core :as p]))
@@ -39,39 +40,59 @@
:output (version/format-version)})
:else
(let [cfg (config/resolve-config (:options parsed))
action-result (commands/build-action parsed cfg)]
(if-not (:ok? action-result)
(p/resolved {:exit-code 1
:output (format/format-result {:status :error
:error (:error action-result)
:command (:command parsed)
:context (select-keys (:options parsed)
[:repo :graph :page :block])}
cfg)})
(-> (commands/execute (:action action-result) cfg)
(p/then (fn [result]
(let [opts (cond-> cfg
(:output-format result)
(assoc :output-format (:output-format result)))]
{:exit-code 0
:output (format/format-result result opts)})))
(p/catch (fn [error]
(let [data (ex-data error)
message (cond
(and (= :http-error (:code data)) (seq (:body data)))
(str "http request failed (" (:status data) "): " (:body data))
(let [cfg (config/resolve-config (:options parsed))]
(try
(let [cfg (assoc cfg :data-dir (data-dir/ensure-data-dir! (:data-dir cfg)))
action-result (commands/build-action parsed cfg)]
(if-not (:ok? action-result)
(p/resolved {:exit-code 1
:output (format/format-result {:status :error
:error (:error action-result)
:command (:command parsed)
:context (select-keys (:options parsed)
[:repo :graph :page :block])}
cfg)})
(-> (commands/execute (:action action-result) cfg)
(p/then (fn [result]
(let [opts (cond-> cfg
(:output-format result)
(assoc :output-format (:output-format result)))]
{:exit-code 0
:output (format/format-result result opts)})))
(p/catch (fn [error]
(let [data (ex-data error)
message (cond
(and (= :http-error (:code data)) (seq (:body data)))
(str "http request failed (" (:status data) "): " (:body data))
(some? (:message data))
(:message data)
(some? (:message data))
(:message data)
:else
(or (.-message error) (str error)))]
{:exit-code 1
:output (format/format-result {:status :error
:error {:code :exception
:message message}}
cfg)}))))))))))
:else
(or (.-message error) (str error)))]
(if (= :data-dir-permission (:code data))
{:exit-code 1
:output (format/format-result {:status :error
:error {:code :data-dir-permission
:message message
:path (:path data)}}
cfg)}
{:exit-code 1
:output (format/format-result {:status :error
:error {:code :exception
:message message}}
cfg)})))))))
(catch :default error
(let [data (ex-data error)
message (or (.-message error) (str error))]
(if (= :data-dir-permission (:code data))
(p/resolved {:exit-code 1
:output (format/format-result {:status :error
:error {:code :data-dir-permission
:message message
:path (:path data)}}
cfg)})
(throw error))))))))))
(defn main
[& args]

View File

@@ -25,6 +25,28 @@
[data-dir repo]
(node-path/join data-dir (worker-util/encode-graph-dir-name repo)))
(defn- ensure-repo-dir!
[data-dir repo]
(let [path (repo-dir data-dir repo)]
(try
(when-not (fs/existsSync path)
(fs/mkdirSync path #js {:recursive true}))
(let [stat (fs/statSync path)]
(when-not (.isDirectory stat)
(throw (ex-info (str "graph-dir is not a directory: " path)
{:code :data-dir-permission
:path path
:cause "ENOTDIR"}))))
(let [constants (.-constants fs)
mode (bit-or (.-R_OK constants) (.-W_OK constants))]
(fs/accessSync path mode))
path
(catch :default e
(throw (ex-info (str "graph-dir is not readable/writable: " path)
{:code :data-dir-permission
:path path
:cause (.-code e)}))))))
(defn lock-path
[data-dir repo]
(node-path/join (repo-dir data-dir repo) "db-worker.lock"))
@@ -180,6 +202,7 @@
[config repo]
(let [data-dir (resolve-data-dir config)
path (lock-path data-dir repo)]
(ensure-repo-dir! data-dir repo)
(p/let [existing (read-lock path)
_ (cleanup-stale-lock! path existing)
_ (when (not (fs/existsSync path))

View File

@@ -95,6 +95,21 @@
date-str (yyyymmdd (js/Date.))]
(node-path/join repo-dir (str "db-worker-node-" date-str ".log"))))
(deftest db-worker-node-data-dir-permission-error
(async done
(let [data-dir (node-helper/create-tmp-dir "db-worker-readonly")
repo (str "logseq_db_perm_" (subs (str (random-uuid)) 0 8))]
(fs/chmodSync data-dir 365)
(-> (db-worker-node/start-daemon! {:data-dir data-dir
:repo repo})
(p/then (fn [_]
(is false "expected data-dir permission error")))
(p/catch (fn [e]
(let [data (ex-data e)]
(is (= :data-dir-permission (:code data)))
(is (= (node-path/resolve data-dir) (:path data))))))
(p/finally (fn [] (done)))))))
(deftest db-worker-node-creates-log-file
(async done
(let [daemon (atom nil)

View File

@@ -0,0 +1,41 @@
(ns logseq.cli.data-dir-test
(:require ["fs" :as fs]
["path" :as node-path]
[cljs.test :refer [deftest is testing]]
[frontend.test.node-helper :as node-helper]
[logseq.cli.data-dir :as data-dir]))
(deftest ensure-data-dir-creates-missing-dir
(testing "creates missing directories and returns normalized path"
(let [base (node-helper/create-tmp-dir "data-dir")
target (node-path/join base "nested" "dir")]
(is (not (fs/existsSync target)))
(let [resolved (data-dir/ensure-data-dir! target)]
(is (fs/existsSync target))
(is (.isDirectory (fs/statSync target)))
(is (= (node-path/resolve target) resolved))))))
(deftest ensure-data-dir-rejects-file-path
(testing "rejects paths that are files"
(let [base (node-helper/create-tmp-dir "data-dir-file")
target (node-path/join base "file.txt")]
(fs/writeFileSync target "x")
(try
(data-dir/ensure-data-dir! target)
(is false "expected data-dir permission error")
(catch :default e
(let [data (ex-data e)]
(is (= :data-dir-permission (:code data)))
(is (= (node-path/resolve target) (:path data)))))))))
(deftest ensure-data-dir-rejects-read-only-dir
(testing "rejects directories without write permission"
(let [target (node-helper/create-tmp-dir "data-dir-readonly")]
(fs/chmodSync target 365)
(try
(data-dir/ensure-data-dir! target)
(is false "expected data-dir permission error")
(catch :default e
(let [data (ex-data e)]
(is (= :data-dir-permission (:code data)))
(is (= (node-path/resolve target) (:path data)))))))))

View File

@@ -4,7 +4,9 @@
[cljs.reader :as reader]
[cljs.test :refer [deftest is async]]
[clojure.string :as string]
[frontend.worker-common.util :as worker-util]
[frontend.test.node-helper :as node-helper]
[logseq.cli.command.core :as command-core]
[logseq.cli.main :as cli-main]
[logseq.db.frontend.property :as db-property]
[promesa.core :as p]))
@@ -121,6 +123,44 @@
(is false (str "unexpected error: " e))
(done)))))))
(deftest test-cli-data-dir-permission-error
(async done
(let [data-dir (node-helper/create-tmp-dir "db-worker-readonly")]
(fs/chmodSync data-dir 365)
(-> (p/let [cfg-path (node-path/join (node-helper/create-tmp-dir "cli") "cli.edn")
_ (fs/writeFileSync cfg-path "{:output-format :json}")
result (run-cli ["graph" "list"] data-dir cfg-path)
payload (parse-json-output result)]
(is (= 1 (:exit-code result)))
(is (= "error" (:status payload)))
(is (= "data-dir-permission" (get-in payload [:error :code])))
(is (string/includes? (get-in payload [:error :message]) data-dir))
(done))
(p/catch (fn [e]
(is false (str "unexpected error: " e))
(done)))))))
(deftest test-cli-graph-create-readonly-graph-dir
(async done
(let [data-dir (node-helper/create-tmp-dir "db-worker-graph-readonly")
repo "readonly-graph"
repo-id (command-core/resolve-repo repo)
repo-dir (node-path/join data-dir (worker-util/encode-graph-dir-name repo-id))]
(fs/mkdirSync repo-dir #js {:recursive true})
(fs/chmodSync repo-dir 365)
(-> (p/let [cfg-path (node-path/join (node-helper/create-tmp-dir "cli") "cli.edn")
_ (fs/writeFileSync cfg-path "{:output-format :json}")
result (run-cli ["graph" "create" "--repo" repo] data-dir cfg-path)
payload (parse-json-output result)]
(is (= 1 (:exit-code result)))
(is (= "error" (:status payload)))
(is (= "data-dir-permission" (get-in payload [:error :code])))
(is (string/includes? (get-in payload [:error :message]) repo-dir))
(done))
(p/catch (fn [e]
(is false (str "unexpected error: " e))
(done)))))))
(deftest test-cli-graph-create-and-info
(async done
(let [data-dir (node-helper/create-tmp-dir "db-worker")]