* fix test
* 058-db-worker-node-revision-and-cli-server-list.md
* 059-cli-doctor-server-revision-mismatch-warning.md
* chore: remove unused LOGIN-URL
* fix: auth-token reset to nil when 'sync status'
* fix: update sync-start-poll-interval-ms
* enhance: 'upsert block' support update --content and --status
* add logseq-cli skill
* fix(cli): treat id-only show targets as missing entities
Make `show` treat id-only pull results as non-existent entities so deleted ids no longer render as empty blocks, and add regressions for deleted block/page lookups.
🤖 Generated with [eca](https://eca.dev)
Co-Authored-By: eca <noreply@eca.dev>
* fix: integration test leaves hanging processes
Hanging db-worker-node.js processes left after running
tests because servers weren't shut down
* fix: multiple issues with --fields option for list* commands
- 4 visible fields couldn't be used with --fields
- Only print selected fields/columns. Previous behavior didn't make
sense as unspecified fields were still listed and took up horizontal space
- Allow --fields to work when --expand isn't used. Previous behavior
wasn't explained to user and needlessly limiting
* enhance: add user-only option to list commands
* enhance: list commands can sort by all visibile fields
Remove outdated name field which had no effect. Also remove duplication
of these values in option declaration, completion declaration and
completion tests
* fix: --expand option drops fields for list commands
* fix(cli): subcmd --help
* enhance(cli): add examples in --help
* 060-cli-graph-list-legacy-graph-dir-rename-command.md
* fix test
* fix lint
* fix: sync download
* enhance(cli): sync subcommand checks corresponding config vars
* fix: multiple fields not visible for list commands
* uuid, classes, properties, extends and description fields not visible
for human output even though data is available and users are asking
to display it with --with-* or --fields
* Also fix recent regression for property types no longer displaying by
default
* Also fix --with-extends, --with-properties and --with-classes
depending on undocumented --extra option. Options should be declarative and
not have hidden dependencies
* 061-graph-dir-space-preserve-canonical.md
* 062-cli-list-default-sort-updated-at.md
* enhance: cli completion for negating command boolean options
with '--no-'. Don't do global options as most global options don't
apply to all commands and are already turned off by default
* fix: overlapping cli option user-only
I introduced this not realizing --include-built-in is the opposite.
Now that --no-include-built-in autocompletion is available, there's
no reason to have confusing, opposite options of each other
* enhance: add cli completion for comma separated options like --fields
Simply add a :multiple-values key to an option spec
* fix: cli completion of aliases
babashka.cli doesn't support -o=val completion for alias options e.g.
'-f=created-at'. Instead they now correctly complete '-f created-at'
* enhance: allow all list fields to be sorted
Removes needing to maintain a separate config
just for sorting and allows for more sorting.
Also remove needless home-rolled validation
when babashka.cli already provides it
* fix: existing mcp updates intermittently failing
The new cli uses :db/id as its primary id while the existing mcp uses
:block/uuid. These changes on list tools cause slow and sometimes breaking interactions
for updates. It also causes buggy/inconsistent mcp behavior b/n mcp cli and server as there
are different implentations being called. The new cli should use its own
thread-api/* fns and replace MCP when it can meet existing functionality - working
updates and provide a CLI interface.
Also fix mcp.tools dissoc bug which had been fixed in the newer list fns
* enhance: all cli options with allowed values validate
those values. We didn't validate some of them as we were relying on
home-rolled validations when we could be using babashka.cli's built-in
validation. Switch to using :validate SET by default and allow for
optional :values
* fix: cli completion of graphs, pages and queries
Wasn't reading json output correctly, didn't have a correct
_logseq_current_graph and caching was causing verification issues
* 063-logseq-cli-upsert-block-custom-many-property.md
* fix: invalid target-uuid option gives wrong error
about unknown command. Source of bug was that uuid options
are being validated downstream and can throw codes that aren't
recognized later. This is needless complexity so fix by validating
the option at cmdline parse time. Also change this for all
other uuid options to simplify them. Also tweaked cli.main which
swallowed daemon error codes instead of reporting them
* enhance: sort cli options alphabetically
* fix: search not starting or working
Per Claude this may have been triggered by adbaf10abc
* fix: undo + redo and possibly more in desktop
Electron app wasn't setting up transact fn like browser does
* fix: upsert --update-tags|remove-tags NON-TAG silently passes
even though no tag is added. Should error instead. Also improve
nonexistent tag error to include name and improve wording
on two upsert options
* enhance(cli): migrate non-sync integration coverage to cli-e2e shell suite
* fix test
* enhance(dev): Add --verbose to cli E2E
Also add tests to nearest workflow
* enhance(cli-e2e): add progress reporting and -h/--help to test runner
* update cli-e2e/agents.md
* test(cli-e2e): block-upsert-update-id-custom-many-property-json
* fix(cli): found by case block-upsert-update-id-custom-many-property-json
* enhance(cli-e2e): use 'bash -c' instead of 'bash -lc'
* enhance(cli-e2e): add bb cleanup
* fix: upsert with invalid EDN for tags and properties optiosn gives incorrect
error - "target or update/remove options"
* fix: invalid :default property value incorrectly passes
but displays warning in the app. For example --update-properties='{"p1"
4}, pretends to pass but is invalid for a :default p1 property. Also
fixes other :default property values incorrectly being passed e.g.
boolean. Also improve reporting of http-request failures as not
including response body gave user no idea why certain property values like
keywords failed
* enhance: improve error handling for create/update only upsert options
so they aren't used incorrectly in the wrong mode. Also update help
so all create/update only options are clearly labeled and remove
unused code for remove options
* fix: nonexistent string :date property value silently passes
and doesn't set anything. Now it errors clearly.
Also improve property validation for --{update/remove}-properties to
include property name
* enhance(cli-e2e): update spec :cmd -> :cmds
* fix(cli): show cmd 500s with :datetime properties
Crashed with datascript lookup errors like "Highest supported entity id is
2147483647, got 1774299180000". Also add display support for :datetime
properties
* enhance(cli): add --ref-id-footer in show cmd
* fix(cli): silently ignores several validation failures
that only the desktop app sees. Ignoring validation errors than causes
further errors as the cli doesn't see updates and doesn't understand the
validation constraints it faces. For example, updating a block with a
private tag e.g. `logseq upsert block --id=219
--update-tags='["Journal"]'` should fail with an explicit message
instead of pretending to succeed
* fix: new property and class idents don't have suffix
when created from app or cli. Moving db-worker to node changed the
js/process exist check in db-ident ns. Brought back env var for
straightforward fix. I checked other process checks and this was the
only one affected this way. Also remove double testing for graph-parser as
it hasn't provided any benefit and yet another place where we kept
juggling this env var
* enhance(dev): unlock-graph option for nbb cli
to allow querying of graph that has been used with new cli
* enhance(cli): import edn can import into existing graph
This functionality is available in the old cli so making sure it's
available in the new one. Also update messaging so it's clear when a new
graph is created
* 066-logseq-cli-list-property-cardinality-column.md
* 067-logseq-cli-json-namespaced-keys.md
* enhance(cli): add --profile global option
* enhance(cli): add search subcmd
* fix: read file cli options don't recognize '~'
These paths autocomplete fine and should be usable
* fix: cleanly print unexpected errors for option handling
from commands/build-action. For example, if --blocks-file has a
nonexistent file or if --blocks or --blocks-file are invalid EDN we
would print full stacktraces which is confusing for user-facing errors.
Update error handling so that it matches
how other unexpected option failures are handled e.g. `logseq graph
import -t edn -g cli-test2 --input=blah`
* fix: upsert page property options can't handle user properties
Fails with 'unknown built-in' error instead of just handling it.
Also fix an incorrect test teardown from previous commit
* fix(cli): propagate server error codes and handle invalid query as 400
* enhance(cli): add example subcmd
* fix(cli): create page fails with 'page-not-found'
Some page names are modified during creation for internal reasons e.g.
'upsert page --page=foo#bar' results in the page 'fooHashTag-bar'. Use
uuid from create-page operation for more reliable
identification of created pages
Also finish fix for 29a7a77a6b
* fix(cli): shouldn't include '#' in page-name
* fix(cli): shouldn't include '#' in page-name(2)
* enhance(cli): search cmd use --content option
* enhance(cli): update search cmd hint
* enhance(cli): add more examples in example cmd
* enhance(cli): update skill md
* fix(cli): create property doesn't validate property title
App doesn't have this issue as property creation ends up going through
update branch. Also improve error message for incorrect tagged nodes.
"does not reference" is too ambiguous and doesn't mention that there's
something wrong with the tags relationship
* fix(cli): upsert property many to one fails
with cryptic "Invalid data" instead of helpful
"This property can't change from multiple values to one value because it
has existing data."
* fix(cli): upsert property --type fails for json and string
These property types were failing with 'Invalid data' since they weren't
exposed for user properties
* enhance(cli): completion for properties and tags
* fix(cli): `upsert property --type` on a property
that has property values makes a graph invalid as property values then
have the wrong types for their property pair. This isn't possible in the
UI because the property type change checks for existing data but the
db-worker-node level does not. This :logseq.property/type update bug was
also possible from plugin/API calls
* feat(cli): add 'graph backup' subcmd
* fix lint
* enhance(cli): update --profile output
* fix: lints and typos
* fix: can't build db-worker-node
@vercel/ncc dep accidentally removed with latest merge to master
* fix(cli): upsert tag/property can't find tag/property
when there is a property and tag with the same name e.g. `logseq upsert
tag --name=Asset` gave incorrect tag-name-conflict error. The query fns
for property and tag incorrectly assumed that :block/name is unique and
didn't account for the app which does allow for properties, tags and
pages to all have the same name. This fix also allows properties and
tags to be created if there is a page with the same name
* fix: cli e2e
* fix: (wip) sync upload
* optional for prod api
* enhance(cli): unify option --page/--name
* fix(cli): --update-{tags|properties} can't distinguish
between tags/properties with same name. Same bug as
0809a53e79
* fix(cli): unable to view help for query command
There was no way to see available query options. Now `query -h`
shows command options while `query` shows subcommands
* feat(cli): add 'debug pull' cmd
* fix lint
* fix: download graph should prefer ':thread-api/db-sync-download-graph-by-id'
* fix(cli): query --inputs fails unexpectedly for incorrect
input. `--inputs=b1` was throwing 'No protocol method ICounted.-count defined'
instead of intended error message
* fix(cli): built-in queries should return more than
just integers. Also remove block-search since `logseq search` does this
better
* fix(cli): custom-query fails if name is a keyword
Bug caused by name binding masking core name fn.
Wouldn't have this bug if there weren't so many
:shadowed-var exclusions in clj-kondo/config.edn.
With --verbose, also show stack for exceptions. Leads
to faster fixing for us (and LLMs)
* enhance(cli): custom queries can optionally use
built-in rules by specifying '%' at end of :in
* dev: add db-worker-node-repl skill
* fix(cli): upsert property can alter private attributes
of built-in properties e.g. `logseq upsert property --name=Alias --no-public`
and also --hide. These attributes aren't modifiable by users in the app
as they can result in visibility, editing and and searching bugs.
The app safeguards this by disabling built-ins for components
that call :upsert-property but the cli and api did not
Fixes
https://test.logseq.com/#/page/69cd9329-4602-42c1-a813-8491d5e7d7a0
* refactor(db-worker,wip): remove <invoke-main-thread (1)
* refactor(db-worker,wip): remove asset related <invoke-main-thread (2)
* refactor(db-worker,wip): remove :thread-api/input-idle <invoke-main-thread (3)
* refactor(db-worker): remove <invoke-main-thread (4)
* refactor(sync): consolidate worker auth state and e2ee password flow
* feat(cli-e2e): add db-sync part
* enhance(cli): show command displays public built-in props
Adds support for show to display several public built-in props
that weren't visible for. Confirmed this works for scheduled,
deadline, priority, description and user* properties
* fix(cli): show command doesn't display :map properties
Confirmed it does show all other user creatable property types
including :json
* fix(cli): show status style and lookup
LLM was handling nonexistent file statuses instead of the db statuses.
Had also queried for :block/name on status ents which never exist
* fix(cli): remove show handling for nonexistent attribute
:block/content hasn't existed for awhile in the db version. It only
exists in some parts of publish for backwards compatible handling of old
db graphs
* enhance(cli): upsert cmds can be piped directly in show cmd
Also fix lint
* test(cli-e2e): add happy-path sync cases
* test(cli-e2e): add 2 sync cases
* feat(cli): add task related cmds
* enhance(cli): display width in all list-* cmds
* enhance(cli): options cleanup in list cmds
* feat(cli): add 'list node'
* enhance(cli): options cleanup in task cmds
* fix(cli): list task --fields option doesn't filter
* fix(cli): node fields columns
uuid field didn't exist and ident is nonsensical as `list node` explicitly
filters out all tags and properties which are the nodes with idents
* enhance(cli): completion for 2 new options to `list node`
Also bring back :status completion for task commands
* enhance(cli): :values completion handles whitespace
Any value in :values or :validate cli specs didn't autocomplete
correctly. For example, for the --status option "in review"
autocompleted as two separate entries. Given that other completion like
:pages, :tags and :properties handled whitespaced completion, it seems
reasonable for this to work for configured :values completion
* fix(cli): fix 4 subcommands that have bugs
caused by new deletion/recyle behavior from
174cdfd865. Fixes:
* remove page didn't recycle page and didn't consider existing recycled
pages
* search page returned recycled pages
* search block returned blocks from recycled pages
* show --page returned recycled pages
* enhance(cli): available values hints for --status/--priority
* enhance(cli): add id option to remove page
Just like tag and property subcommands. Also
allows user to workaround name conflicts which tag
and property subcommands don't have to deal with
* fix(cli): `page remove` incorrectly assumes page name
is unique and randomly deletes first name found. The app
allows for a tag, property or page to share the same name.
Fixed this by throwing an error, alerting them to delete by id
* update agents.md
* fix(cli): upsert page on deleted/recycled page
restores it. Also disable editing of deleted page to keep consistent
with app
* feat(cli): add skill show/install
* enhance(cli): add logseq-cli-maintenance skill
* refactor(cli): centralize output mode handling
* enhance(cli): remove 'server status'; add 'server cleanup'
* enhance(cli-e2e): add sync suite fixtures, timings, and port cleanup
* feat(cli): add list/upsert asset
* fix (cli): upsert commands fail with partial updates
Running `upsert block -c b1 --update-tags='["nonexistent-tag"]'`
errors for the nonexistent tag but surprisingly still
creates a block with no indication that it was created. Fixed this
and a similar case for `upsert page`. Also cleaned up `upsert task`
which had unnecessary error throwing which could also result in the
same error type. Longer term we need a better solution than moving
all error/validation checks to the beginning of each upsert
* enhance(cli): update list asset output columns
* fix(cli): upsert task --priority not working
* enhance(cli-e2e): add step timings and manifest lint/template system
* enhance(cli-e2e): update wait_sync_status in cli-e2e-sync
* enhance(cli): humanize human-mode output formatting
* enhance(cli): humanize human-mode output formatting (2)
* fix(cli): `remove block --id` can remove built-in nodes
files, empty-placeholder and import-tx idents could all be deleted.
Ents with :kv/value would error unintentionally with
'No protocol method INode.-del defined for type null:'.
Now these all gracefully error
* fix(cli): `logseq upsert block --id` modifies built-in nodes
it shouldn't be able to. Somes of these bugs are caused by same incorrect
definition of built-in? as found in
22c7736751. --content and --pos could modify all
built-in nodes. *properties and *tags options could modify private
built-in nodes which the app is not able to do and the CLI should be
disallowed from doing as well
* fix(cli): `remove block --id` gives false positive
for pages and doesn't delete them. Should instead error explicitly for pages,
like with other subcommands when the given target isn't the right type
* fix(cli): multiple upsert commands randomly choose first page found
when an ambiguous page is given instead of erroring to ask for the
specific page. Like dd6b8975c7, there are still several
paths where we incorrectly assume :block/name is unique (and don't
take into account recyling). This commit fixed the following options:
* upsert block --target-page
* upsert task --target-page
* upsert asset --target-page
* upsert page --page
There are still two bugs where ambiguous page names leads to random
page selection but which can't be addressed easily because there is no
way for a user to select by page id in those contexts. Put a TODO next
to the ensure-first-page! workaround
* fix: drag n drop asset doesn't work in electron
Last console error was
"Error: No protocol method ICounted.-count defined for type object: [object ArrayBuffer]"
* fix(cli): some list commands showing unused options
asset, task and node were showing 2 options that don't apply
to them
* enhance(cli): breadcrumb in show
* enhance(cli): breadcrumb in show (2)
* enhance(cli): atomic behavior for upsert commands
* fix(cli): random property value selection with --update-properties
and same names. It is ok to error with this option as
users can choose an integer to look up e.g. --update-properties='{"node" 191}'
Fixes
https://test.logseq.com/#/page/69de70a0-d455-4d40-8247-878a5cf89e57
* enhance(cli): upserted :block/title supports integer/id block refs
Supporting integers allows us to fix
https://test.logseq.com/#/page/69de6e3e-6c60-476c-9347-9f377e7b2fb9 and
rely on standard error reporting when multiple pages with the same name
are found
* fix(cli): graph backup create fails to create graph
* enhance(cli): completion for 3-level commands
Handles `logseq graph backup[TAB]` and `logseq graph backup remove[TAB]`
* fix(cli): update cli skill
Update outdated commands and remove references section which has a
misleading command and a nonexistent file
* fix(cli): other updates after merge master
* fix lint
* enhance(cli-e2e): update e2ee-password option for cli-e2e-sync
* update cli agents.md
* fix sync test
* update agents.md
* revert changes from 1bdba92 related to sync
* fix(sync): handle snapshot reset and tx epoch rollback
Fix snapshot-stream reset coercion so first non-empty chunk actually applies reset during full uploads.
Broadcast changed(t-now) after finished snapshot upload so connected peers learn the new snapshot epoch immediately.
Handle remote tx rollback on clients (hello/changed with remote-tx < local-tx) by rebasing local tx/checksum cursor and requesting pull from the new baseline.
Harden ws broadcast in non-WS test contexts and add regression tests for reset-on-first-chunk and finished-upload changed broadcast.
Verified with: cli-e2e sync-multi-batch-operations (passes).
* fix(cli): fail fast on invalid inputs and remove legacy fallbacks
* fix(cli): sync error doesn't show hint
Running `logseq sync remote-graphs` didn't show :missing-auth hint
* fix: web app can't sync after merge master
* enhance(cli): Add more docs for cli.edn
Especially custom queries and env vars
* use dev script when not release
* fix(e2ee): use native secret storage and init remote sync config
* fix: cli created graphs are not encrypted
* fix: checkpoint sqlite wal with truncate
* fix: no need to export graph when switching graph anymore
* enhance(backup): share sqlite backup flow and trigger via save shortcut
* fix: can't download graph on electron
* fix(db-worker): stop runtimes cleanly on shutdown and graph delete
* fix(sync): clean temp download pools on success and failure
* fix(db-worker): stop remote runtime when closing current graph
* fix: lint and tests
* remove warning on missing-home-dir since electron ipc is not called
* enhance(cli-e2e): add jobs-based(default=4) parallel case execution for non-sync cases
* fix: download doesn't cleanup temp db
* fix: db-worker.lock shouldn't be deleted when downloading graph
* fix(cli-e2e): disable keychain only in e2e runs
* fix(db-worker): keep electron-owned daemon attached
* chore: remove cleanup-orphan-processes
* enhance: no need to block waiting previous graph server down
* add NO_COLOR environment variable to cli-e2e configuration
* update shell command to use 'bash' instead of '/bin/bash'
* normalize JSON-escaped Windows paths in output assertions
* fix(cli): honor HOME when resolving global skill install path
* enhance(cli-e2e): update cli-e2e-sync, fix cli-e2e --jobs
* dev: remove unnecessary logs and repeated compilation of db-worker-node
* feat(cli-e2e): 'bb dev:cli-e2e-sync' support for parallel execution
* fix(cli): wait for db-worker exit on stop
* feat(cli): add browser open command and spawn options for different platforms
* fix(db-worker): keep fresh locks during sync startup
* fix test
* feat(i18n): internationalize password modal titles and submit button
* chore: add skills registry for managing agent skills and their triggers
* enhance(cli): centralize db-worker server discovery
* fix lint
* update bb.edn
* fix(sync,cli): dont upload the same graph multi times
* fix(cli-e2e): dont delete skill.md in e2e cleanup
* enhance(cli): default list order to desc
* refactor(cli): remove --data-dir, add --root-dir
--data-dir: ~/logseq/graphs
--root-dir: ~/logseq
All other configuration files and data directories are derived from the root-dir
* use vite instead of ncc
* add minify
* update .agent/skills dir
* feat(skill): add logseq-repl skill
* feat(skill): update logseq-debug-workflow
* enhance(skill): update logseq-debug-workflow, logseq-repl
* enhance(skill): update logseq-repl
* fix(cli): config.edn reading and writing incorrectly
from a literal '~' directory under current directory
because it wasn't expanded
* fix(cli): hint not showing up for sync subcommands
e.g. `sync status` and e2ee-password-not-found error
* fix(cli): sync status fails with unactionable e2ee-password-not-found error
`sync status` can fail with error to use nonexistent
--e2ee-password option. `status` is a read only operation and isn't in
authenticated-sync-actions so just remove e2ee check for it
* enhance(cli): update db-worker-node options
* fix(test): fails on CI about some permission issue on directories
* chore: remove ENABLE-FILE-SYNC-PRODUCTION
* chore: remove ENABLE_RTC_SYNC_PRODUCTION
* fix(test): invalid HOME env
* fix(skill): update logseq-repl
* fix: local-tx not initialized when creating a new graph
* fix(cli,db-worker): able to upload the graph again after delete related remote graph
* refactor: move create-remote-graph to db worker
also, don't use :logseq.kv/graph-uuid to detect whether it's still
a remote graph, since it's not retracted when deleting a remote graph.
* fix: no need to load db from disk since data is written directly
* fix(cli,db-worker): not keep empty new graph when sync download failed
* fix(db): validate sqlite format on import
* chore: remove unused `node` persist-db impl
* fix: addresses ipc errors in general
* fix: don't throw error for mkdir/stat
* fix: download remote graph should set sync metadata too
* fix: client/server checksum mismatch
---------
Co-authored-by: eca <noreply@eca.dev>
Co-authored-by: Gabriel Horner <gabriel@logseq.com>
Co-authored-by: Tienson Qin <tiensonqin@gmail.com>
Co-authored-by: Mega Yu <yuhg2310@gmail.com>
`sync status` can fail with error to use nonexistent
--e2ee-password option. `status` is a read only operation and isn't in
authenticated-sync-actions so just remove e2ee check for it
