026-logseq-cli-query-output.md (2)

docs/agent-guide/026-logseq-cli-query-output.md

@@ -1,9 +1,9 @@
 # Logseq CLI Query Output Piping Implementation Plan
 
-Goal: Remove the space-to-comma transformation in CLI query human output while keeping query results usable in shell pipelines like xargs and direct stdin piping with logseq show.
+Goal: Remove the space-to-comma transformation in CLI query human output while preserving EDN output and ensuring `logseq show --id` accepts ids both via argument and stdin pipelines.
 
-Architecture: Adjust human formatting for query results to preserve EDN output for general results and emit line-oriented output only for scalar id collections.
-Architecture: Extend logseq show -id to read ids from stdin when no id argument is provided and update integration tests to validate both xargs and direct stdin pipelines.
+Architecture: Keep human formatting for query results as EDN output (no line-oriented transformation) and update tests to validate pipeline usage with intact EDN.
+Architecture: Extend logseq show --id to read ids from stdin when present, whether or not an id argument is provided, and update integration tests to validate both xargs and direct stdin pipelines.
 
 Tech Stack: ClojureScript, Logseq CLI, db-worker-node, Node-based integration tests.
 
@@ -11,64 +11,60 @@ Related: Relates to docs/agent-guide/025-logseq-cli-builtin-status-priority-quer
 
 ## Testing Plan
 
-I will add an integration test that ensures a human-output query can be piped into xargs and then into logseq show for multiple ids.
-I will add an integration test that ensures a human-output query can be piped directly into logseq show -id via stdin when no id value is passed.
-I will update the existing integration test that currently pipes human query output directly into logseq show so it asserts the new line-oriented behavior and stdin ingestion.
-I will add a focused unit test for format-query-results that covers scalar collections, non-scalar collections, and nil results.
-I will add a unit test for show id parsing that covers missing id arg with stdin provided and missing id arg without stdin.
+I will add an integration test that ensures a human-output query can be piped into xargs and then into logseq show --id for multiple ids, with the query output preserved as EDN.
+I will add an integration test that ensures a human-output query can be piped directly into logseq show --id via stdin, with or without an explicit id argument.
+I will update the existing integration test that currently pipes human query output directly into logseq show so it asserts EDN preservation and stdin ingestion.
+I will add a focused unit test for format-query-results that covers EDN output for scalar collections, non-scalar collections, and nil results.
+I will add a unit test for show id parsing that covers stdin provided, stdin blank, and explicit id values.
 
 NOTE: I will write all tests before I add any implementation behavior.
 
 ## Problem statement
 
 The CLI currently replaces spaces with commas in format-query-results, which is a lossy transformation that makes output less readable.
-Removing that transformation will reintroduce spaces in EDN vectors and lists, which breaks shell pipelines that rely on whitespace splitting.
-We need to remove the space-to-comma logic while still ensuring the pipeline commands in the request work reliably.
-The show command needs to accept ids from stdin when -id is present but no explicit id argument is provided.
+We need to remove the space-to-comma logic while preserving EDN output (including spaces) for query results.
+Pipelines should remain usable by allowing logseq show --id to accept ids from stdin and from explicit arguments interchangeably.
 
 ## Plan
 
 1. Read the existing formatting logic in /Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/format.cljs and document current behavior in a quick note.
 2. Read the show command option parsing in /Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/show.cljs and /Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/id.cljs to understand current id validation.
 3. Locate the current integration test that verifies human query output piping in /Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/integration_test.cljs and map the required changes.
-4. Define the new output rule for format-query-results as a comment in the plan: if the result is a sequential collection of scalar ids, output one id per line, otherwise output the EDN string unchanged.
-5. Define the new show -id stdin rule as a comment in the plan: when -id is present but no id value is provided, read stdin, trim it, and treat it as the id or id vector string for parsing.
-6. Write the failing unit tests for format-query-results in a new test namespace under /Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/format_test.cljs that covers nil, scalar-id sequences, and nested maps.
-7. Write failing unit tests for show id parsing in /Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/command/show_test.cljs that cover stdin provided, stdin blank, and explicit id values.
+4. Define the new output rule for format-query-results as a comment in the plan: always return EDN output unchanged (including spaces), and remove space-to-comma logic.
+5. Define the new show --id stdin rule as a comment in the plan: when stdin is non-empty, parse it as the id or id vector string and use it regardless of whether an id argument is provided.
+6. Write the failing unit tests for format-query-results in a new test namespace under /Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/format_test.cljs that covers nil, scalar-id sequences, and nested maps, all preserved as EDN strings.
+7. Write failing unit tests for show id parsing in /Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/command/show_test.cljs that cover stdin provided (overriding or complementing args), stdin blank, and explicit id values.
 8. Run the unit tests to confirm the failure before implementation using bb dev:test with the new namespaces.
 9. Write a failing integration test that uses the exact pipeline command from the request by invoking a shell command from the test harness in /Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/integration_test.cljs for xargs usage.
-10. Write a failing integration test that uses the exact pipeline command from the request for direct stdin piping into logseq show -id with no argument.
+10. Write a failing integration test that uses the exact pipeline command from the request for direct stdin piping into logseq show --id with or without an argument.
 11. Run the integration tests to confirm the failure before implementation using bb dev:test with the specific test names.
-12. Implement the new format-query-results behavior in /Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/format.cljs using a helper predicate for scalar ids and line-join logic.
-13. Remove the string/replace space-to-comma logic from format-query-results in /Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/format.cljs.
-14. Implement stdin ingestion for show -id in /Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/show.cljs, using a helper to read from stdin only when -id is present and no id argument is provided.
-15. Update the existing human-output pipeline integration test to align with the new output behavior and assert both xargs and direct stdin usage.
+12. Implement the new format-query-results behavior in /Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/format.cljs by removing the space-to-comma logic and preserving EDN output.
+13. Implement stdin ingestion for show --id in /Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/show.cljs, using a helper to read from stdin when non-empty regardless of whether an id argument is provided.
+14. Update the existing human-output pipeline integration test to align with the EDN-preserving output and assert both xargs and direct stdin usage.
 16. Re-run the unit tests and the integration tests to validate the new behavior.
 17. Update deps/cli/README.md examples if they reference comma-transformed output, and add a short note describing the new xargs-friendly and stdin-friendly behavior for id lists.
 
 ## Testing Details
 
-The unit tests will exercise behavior by calling format-query-results with representative results and asserting the exact emitted string for id lists and non-id data.
-The integration tests will execute logseq query with a task-search or custom datalog query, pipe the output through xargs into logseq show, and pipe directly into logseq show -id via stdin with no id argument.
-The show stdin behavior test will assert that missing -id input without stdin returns a clear error and that stdin is parsed the same as an id argument.
+The unit tests will exercise behavior by calling format-query-results with representative results and asserting the exact emitted EDN string for id lists and non-id data.
+The integration tests will execute logseq query with a task-search or custom datalog query, pipe the EDN output through xargs into logseq show --id, and pipe directly into logseq show --id via stdin.
+The show stdin behavior test will assert that missing --id input without stdin returns a clear error and that stdin is parsed the same as an explicit id argument.
 
 ## Implementation Details
 
-- Add a helper predicate for scalar ids that accepts integers and rejects maps, vectors, and strings.
-- Detect sequential results that are entirely scalar ids and return a newline-joined string of ids.
 - Preserve the existing safe-read-string validation to avoid changing behavior for invalid EDN strings.
 - Keep non-scalar results as their original EDN string output.
 - Maintain nil handling so that "nil" still prints as "nil".
-- Ensure the output is stable for xargs by avoiding embedded spaces in the line-oriented mode.
-- Add stdin reading to show -id when the option is present but its value is missing.
-- Parse stdin through the same id parsing function used for explicit -id values.
-- Keep existing errors when -id is missing and stdin is empty or whitespace.
+- Remove the space-to-comma transformation while keeping EDN output intact, including spaces.
+- Add stdin reading to show --id when stdin is non-empty, even if an id argument is provided.
+- Parse stdin through the same id parsing function used for explicit --id values.
+- Keep existing errors when --id is missing and stdin is empty or whitespace.
 - Update integration tests to use the exact pipeline command from the requirement.
 - Use @test-driven-development for all implementation steps.
 
 ## Question
 
-The line-oriented output applies only to vectors of numeric ids.
-Logseq show -id reads stdin only when -id is present with no value.
+Query results remain in EDN form (e.g., `[1 2 3]`) with no line-oriented transformation.
+Logseq show --id accepts ids via explicit argument and via stdin pipelines (stdin takes precedence when provided).
 
 ---

src/main/logseq/cli/command/show.cljs

@@ -29,6 +29,13 @@
   []
   (.toString (fs/readFileSync 0) "utf8"))
 
+(defn- stdin-available?
+  []
+  (try
+    (let [stat (fs/fstatSync 0)]
+      (or (.isFIFO stat) (.isFile stat)))
+    (catch :default _ false)))
+
 (defn- normalize-stdin-id
   [value]
   (let [text (string/trim (or value ""))]
@@ -45,12 +52,16 @@
 
 (defn- resolve-stdin-id
   [options]
-  (if (:id-from-stdin? options)
-    (let [stdin (if (contains? options :stdin)
-                  (:stdin options)
-                  (read-stdin))]
-      (assoc options :id (normalize-stdin-id stdin)))
-    options))
+  (let [id-present? (or (contains? options :id) (some? (:id options)))
+        stdin (cond
+                (contains? options :stdin) (:stdin options)
+                (:id-from-stdin? options) (read-stdin)
+                (and id-present? (stdin-available?)) (read-stdin)
+                :else nil)
+        normalized (normalize-stdin-id stdin)]
+    (if (string/blank? normalized)
+      options
+      (assoc options :id normalized))))
 
 (defn invalid-options?
   [opts]

src/main/logseq/cli/format.cljs

@@ -182,25 +182,13 @@
             (:pid server)])
          (or servers []))))
 
-(defn- scalar-id?
-  [value]
-  (and (number? value) (integer? value)))
-
-(defn- scalar-id-seq?
-  [value]
-  (and (sequential? value)
-       (seq value)
-       (every? scalar-id? value)))
-
 (defn- format-query-results
   [result]
   (let [edn-str (pr-str result)
         parsed (common-util/safe-read-string {:log-error? false} edn-str)
         valid? (or (some? parsed) (= "nil" (string/trim edn-str)))]
     (if valid?
-      (if (scalar-id-seq? result)
-        (string/join "\n" (map str result))
-        edn-str)
+      edn-str
       edn-str)))
 
 (defn- format-query-list

src/test/logseq/cli/command/show_test.cljs

@@ -23,7 +23,23 @@
       (is (= [1 2 3] (get-in result [:action :ids])))
       (is (true? (get-in result [:action :multi-id?])))))
 
-  (testing "blank stdin returns invalid options"
+  (testing "stdin overrides explicit id when present"
+    (let [result (show-command/build-action {:id "99"
+                                             :stdin "[1 2]"}
+                                            "logseq_db_demo")]
+      (is (true? (:ok? result)))
+      (is (= [1 2] (get-in result [:action :ids])))
+      (is (true? (get-in result [:action :multi-id?])))))
+
+  (testing "blank stdin falls back to explicit id"
+    (let [result (show-command/build-action {:id "99"
+                                             :stdin "   "}
+                                            "logseq_db_demo")]
+      (is (true? (:ok? result)))
+      (is (= 99 (get-in result [:action :id])))
+      (is (= [99] (get-in result [:action :ids])))))
+
+  (testing "blank stdin returns invalid options when id is missing"
     (let [result (show-command/build-action {:id ""
                                              :id-from-stdin? true
                                              :stdin "   "}

src/test/logseq/cli/format_test.cljs

@@ -171,12 +171,12 @@
       (is (= "Line 1\nLine 2" result)))))
 
 (deftest test-human-output-query-results
-  (testing "scalar id collections render one id per line"
+  (testing "scalar id collections preserve EDN formatting"
     (let [result (format/format-result {:status :ok
                                         :command :query
                                         :data {:result [1 2 3]}}
                                        {:output-format nil})]
-      (is (= "1\n2\n3" result))))
+      (is (= "[1 2 3]" result))))
 
   (testing "non-scalar collections preserve EDN formatting"
     (let [value [{:db/id 1 :block/title "Alpha"}

src/test/logseq/cli/integration_test.cljs

@@ -1180,6 +1180,20 @@
                                        " :where"
                                        " [?e :block/title ?title]"
                                        " [(clojure.string/includes? ?title ?q)]]")
+                       query-json-result (run-cli ["--repo" "query-pipe-graph"
+                                                   "query"
+                                                   "--query" query-text
+                                                   "--inputs" (pr-str ["Pipe"])]
+                                                  data-dir cfg-path)
+                       query-json-payload (parse-json-output query-json-result)
+                       query-ids (get-in query-json-payload [:data :result])
+                       query-human-result (run-cli ["--repo" "query-pipe-graph"
+                                                    "--output" "human"
+                                                    "query"
+                                                    "--query" query-text
+                                                    "--inputs" (pr-str ["Pipe"])]
+                                                   data-dir cfg-path)
+                       query-human-output (string/trim (:output query-human-result))
                        node-bin (shell-escape (.-execPath js/process))
                        cli-bin (shell-escape (node-path/resolve "static/logseq-cli.js"))
                        data-arg (shell-escape data-dir)
@@ -1209,6 +1223,7 @@
                        stop-result (run-cli ["server" "stop" "--repo" "query-pipe-graph"]
                                             data-dir cfg-path)
                        stop-payload (parse-json-output stop-result)]
+                 (is (= (pr-str query-ids) query-human-output))
                  (is (string/includes? output "Pipe One"))
                  (is (string/includes? output "Pipe Two"))
                  (is (= "ok" (:status stop-payload)))
@@ -1239,6 +1254,13 @@
                                        " :where"
                                        " [?e :block/title ?title]"
                                        " [(clojure.string/includes? ?title ?q)]]")
+                       query-json-result (run-cli ["--repo" "query-stdin-graph"
+                                                   "query"
+                                                   "--query" query-text
+                                                   "--inputs" (pr-str ["Pipe"])]
+                                                  data-dir cfg-path)
+                       query-json-payload (parse-json-output query-json-result)
+                       query-ids (get-in query-json-payload [:data :result])
                        query-result (run-cli ["--repo" "query-stdin-graph"
                                               "--output" "human"
                                               "query"
@@ -1261,6 +1283,7 @@
                        stop-result (run-cli ["server" "stop" "--repo" "query-stdin-graph"]
                                             data-dir cfg-path)
                        stop-payload (parse-json-output stop-result)]
+                 (is (= (pr-str query-ids) ids-text))
                  (is (= "ok" (:status show-payload)))
                  (is (contains? root-titles "PipePage"))
                  (is (some? pipe-one))