logseq/docs/agent-guide/051-logseq-cli-sync-upload-fix.md

# Logseq CLI Sync Upload Fix Plan

Goal: Fix `logseq sync upload` so a CLI-managed local graph can be uploaded successfully with the current `logseq-cli` + `db-worker-node` architecture, including the first upload of a graph that does not yet have a local `graph-id`.

Architecture: Keep the CLI surface small and move the real fix into the worker-side sync layer so `:thread-api/db-sync-upload-graph` becomes self-sufficient: it should resolve or create the remote graph when needed, persist the resulting graph metadata locally, and then perform snapshot upload.

Architecture: Remove the current error-swallowing behavior in worker upload so failures propagate back through `db-worker-node` and the CLI can report an actual error instead of a false success.

Tech Stack: ClojureScript, `logseq-cli`, `db-worker-node`, `frontend.worker.sync`, db-sync HTTP endpoints, promesa, babashka test runner.

Related: Builds on `docs/agent-guide/047-logseq-cli-sync-command.md`.

Related: Relates to `docs/cli/logseq-cli.md` and `docs/developers/desktop-db-worker-node.md`.

## Problem statement

With the current implementation, `logseq sync upload --graph <name>` is wired only as:

- CLI `sync upload`
- `logseq.cli.command.sync/execute`
- `:thread-api/db-sync-upload-graph`
- `frontend.worker.sync/upload-graph!`

That path assumes the local graph already has a usable remote `graph-id`.

However, `frontend.worker.sync/upload-graph!` currently fails when either `http-base` or `graph-id` is missing:

- `http-base` comes from CLI sync config and is present in the provided `cli.edn`.
- `graph-id` is read from local graph metadata via `get-graph-id`.
- For a fresh local graph that has never been uploaded or downloaded before, `graph-id` is typically missing.

The desktop/UI flow does not have this problem because `frontend.handler.db_based.sync/<rtc-upload-graph!` first calls `<rtc-create-graph!`, persists the returned graph id locally, and only then invokes `:thread-api/db-sync-upload-graph`.

The CLI flow does not perform that preflight step, so first-time upload cannot work reliably.

There is a second bug that makes the failure harder to see: `frontend.worker.sync/upload-graph!` currently catches errors and only logs them with `js/console.error`, which can turn a failed upload into a misleading success result from the CLI perspective.

In practice, with a `cli.edn` that already provides `ws-url`, `http-base`, `auth-token`, and `e2ee-password`, upload can still fail because the missing piece is remote graph bootstrap and proper error propagation, not just config.

## Current implementation findings

### 1. CLI upload path is too thin for first-time upload

`/Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/sync.cljs`

`execute` for `:sync-upload` currently does only one repo-scoped invoke:

- `:thread-api/set-db-sync-config`
- `:thread-api/db-sync-upload-graph`

There is no CLI-side or worker-side step that ensures the remote graph exists before upload starts.

### 2. Worker upload requires an existing local graph id

`/Users/rcmerci/gh-repos/logseq/src/main/frontend/worker/sync.cljs`

`upload-graph!` currently derives:

- `base` from `http-base-url`
- `graph-id` from `get-graph-id`

If either value is missing, it rejects with `ex-info "db-sync missing upload info"`.

This is correct as a low-level invariant, but it means the higher-level upload orchestration is incomplete.

### 3. Desktop flow already has the missing bootstrap logic

`/Users/rcmerci/gh-repos/logseq/src/main/frontend/handler/db_based/sync.cljs`

`<rtc-upload-graph!` does:

1. `<rtc-create-graph!`
2. persist returned graph metadata locally
3. call `:thread-api/db-sync-upload-graph`
4. refresh remote graphs / start sync

So the product already has a working conceptual flow, but the CLI path bypasses the graph-creation step.

### 4. Upload errors are swallowed

`/Users/rcmerci/gh-repos/logseq/src/main/frontend/worker/sync.cljs`

`upload-graph!` ends with a broad `p/catch` that only does `js/console.error`.

That means transport and CLI layers cannot reliably distinguish success from failure.

### 5. Test coverage misses the real failing case

Current CLI tests mostly verify transport wiring, for example that `sync upload` invokes `:thread-api/db-sync-upload-graph`.

What is missing is an end-to-end behavior test for:

- local graph without graph id
- configured `http-base` + `auth-token`
- worker creating or resolving a remote graph
- successful upload
- failure surfacing when remote bootstrap or snapshot upload fails

## Root cause summary

The broken behavior is not primarily a config parsing problem.

The actual root causes are:

1. `sync upload` assumes local sync metadata already exists, but first-time CLI upload has no step to create or resolve a remote graph.
2. Worker upload swallows errors, so CLI output can be misleading.
3. Desktop and CLI implement different upload orchestration paths, which caused the CLI path to miss required bootstrap logic.
4. There is no regression test covering first-time upload from a local-only graph.

## Desired behavior

Given a CLI config containing valid sync credentials and endpoints, `logseq sync upload --graph <name>` should:

1. start or reuse the graph's `db-worker-node`
2. apply sync config to the worker
3. detect whether the local graph already has a remote `graph-id`
4. if missing, resolve or create the remote graph
5. persist the resolved/created graph id and sync metadata locally
6. upload the graph snapshot
7. return a real success result only when upload actually succeeds
8. return a real error with context when any step fails

This should work both for:

- an already-linked graph that has a local `graph-id`
- a fresh local graph being uploaded for the first time

## Recommended design

### Option A: Fix inside worker upload orchestration

Preferred approach: make `frontend.worker.sync/upload-graph!` capable of ensuring remote graph identity before snapshot upload.

This keeps the CLI simple and makes `:thread-api/db-sync-upload-graph` a complete unit of behavior.

Suggested worker-side flow:

1. Read local `graph-id`.
2. If present, continue with existing upload logic.
3. If absent, list remote graphs visible to the current auth context.
4. Match by canonical graph name.
5. If a matching remote graph exists, bind local graph to that `graph-id`.
6. If no matching remote graph exists, create one with the current graph name and schema version.
7. Persist returned graph metadata locally.
8. Continue snapshot upload using the resolved `graph-id`.

This mirrors the desktop behavior conceptually while avoiding duplicated orchestration in CLI code.

### Option B: Add CLI-specific preflight before invoking upload

Alternative approach: keep worker upload low-level and add a new CLI-side preflight that creates or resolves the remote graph before invoking `:thread-api/db-sync-upload-graph`.

I do not recommend this because it duplicates sync orchestration across CLI and desktop-adjacent code and makes the worker thread API less useful as a stable contract.

## Scope

### In scope

- Fix first-time `sync upload` for CLI-managed graphs.
- Make worker upload propagate failures.
- Reuse existing remote graph APIs (`GET /graphs`, `POST /graphs`) instead of inventing a new cloud protocol.
- Add tests for first-time upload, already-linked upload, and failure propagation.
- Update CLI docs so `sync upload` behavior is explicit.

### Out of scope

- Changing the cloud snapshot upload protocol.
- Adding a new top-level CLI command for graph creation.
- Reworking websocket sync start/stop behavior.
- Large refactors unrelated to upload bootstrap.

## Implementation plan

### Phase 1. Add failing tests for the broken behavior

1. Add a worker sync test in `/Users/rcmerci/gh-repos/logseq/src/test/frontend/worker/db_sync_test.cljs` for first-time upload when local graph has no graph id.
2. Stub remote graph listing to return no match, stub graph creation to return a new id, and assert upload continues with that id.
3. Add a companion test where remote graph listing already contains the same graph name and assert upload reuses the existing graph id instead of creating a duplicate graph.
4. Add a failure test that simulates graph creation failure and asserts the promise rejects rather than resolving silently.
5. Add a failure test that simulates snapshot upload failure and asserts the CLI-visible result is an error, not success.
6. Add a CLI command execution/integration test in `/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/command/sync_test.cljs` or CLI integration tests that covers the real first-upload path, not just method wiring.

### Phase 2. Move remote graph bootstrap into worker sync code

7. Add a worker helper in `/Users/rcmerci/gh-repos/logseq/src/main/frontend/worker/sync.cljs` to ensure a usable remote graph id before snapshot upload.
8. That helper should read the local graph name from repo name using the same canonicalization rules already used by CLI and desktop sync.
9. If local graph id exists, return it immediately.
10. If local graph id is missing, call existing remote graph list API.
11. If a graph with the same canonical name exists, reuse its `graph-id` and persist it locally.
12. If no match exists, call the existing graph creation API and persist the new `graph-id` locally.
13. Keep local metadata writes inside worker code so later sync commands (`sync start`, asset upload, etc.) see a consistent graph identity.

### Phase 3. Reuse or extract desktop graph-creation logic

14. Avoid leaving two divergent graph-creation implementations.
15. Extract the parts of `frontend.handler.db_based.sync/<rtc-create-graph!` that are transport/persistence logic into worker-shared code, or reimplement them once in `frontend.worker.sync` and have desktop code reuse that worker-oriented path where practical.
16. Keep UI-only behaviors such as notifications and state refresh in the frontend handler layer.
17. Ensure the shared implementation supports `graph-e2ee?` and schema version consistently.

### Phase 4. Fix error propagation

18. Remove the broad catch-and-only-log behavior at the end of `frontend.worker.sync/upload-graph!`.
19. If local logging is still desired, log and rethrow, or convert to a structured `ex-info` with context.
20. Ensure `db-worker-node` HTTP invoke returns a failure status when upload bootstrap or upload body transfer fails.
21. Verify CLI output paths show the actual error code/message.

### Phase 5. Clarify E2EE behavior for first-time upload

22. Decide what `graph-e2ee?` should be when a local graph has no sync metadata yet.
23. Prefer to match the current desktop default for fresh cloud graph creation, which treats missing value as encrypted unless explicitly false.
24. Make that default explicit in code and tests instead of relying on incidental `nil` handling.
25. Ensure the provided `e2ee-password` config is actually used when a new encrypted remote graph is created.
26. Add at least one test covering fresh upload with encrypted-graph setup.

### Phase 6. Update docs and command expectations

27. Update `/Users/rcmerci/gh-repos/logseq/docs/cli/logseq-cli.md` so `sync upload` explicitly documents first-time upload behavior.
28. Document that upload will resolve an existing same-name remote graph or create a new remote graph when no graph id is present locally.
29. Document the failure modes clearly: auth failure, graph creation failure, snapshot upload failure, and local DB missing.
30. Do not include real tokens or passwords in docs or tests.

## Detailed design notes

### Remote graph resolution rules

For first-time upload, use the local graph name after stripping the internal db prefix in the same way current sync code already does for remote graph matching.

Recommended resolution order:

1. local `graph-id`
2. exact remote graph name match
3. create remote graph

If graph name matching can return more than one candidate, fail with a deterministic conflict error instead of guessing.

### Metadata persistence

After graph resolution/creation succeeds, persist at least:

- `:logseq.kv/graph-uuid`
- `:logseq.kv/graph-remote?`
- `:logseq.kv/graph-rtc-e2ee?`

This ensures later commands such as `sync start`, asset operations, and subsequent uploads operate on the same graph identity.

### Error model

Do not return success when worker upload has failed.

Prefer structured `ex-info` errors with enough context for CLI formatting, such as:

- `:repo`
- `:graph-id`
- `:graph-name`
- `:status`
- `:url`
- remote response body when safe

### Why the fix belongs in worker sync

`db-sync-upload-graph` is already the contract used by CLI.

If the worker API stays incomplete and CLI grows its own upload bootstrap logic, we will keep drifting away from the desktop path and make sync semantics harder to maintain.

Putting graph bootstrap in worker sync makes the behavior available to every caller and keeps the CLI transport layer thin.

## Verification plan

### Targeted automated verification

- `bb dev:test -v frontend.worker.db-sync-test`
- `bb dev:test -v logseq.cli.command.sync-test`
- `bb dev:test -v logseq.cli.integration-test`

### Manual verification with CLI config

Use a `cli.edn` with these keys configured:

- `:ws-url`
- `:http-base`
- `:auth-token`
- `:e2ee-password`

Do not commit real token values.

Manual checks:

1. Create a brand-new local graph with no stored `graph-id`.
2. Run `logseq sync upload --graph <name>`.
3. Confirm the command returns success only after remote graph bootstrap + snapshot upload finish.
4. Run `logseq sync status --graph <name>` and confirm the graph now has stable remote metadata.
5. Re-run `logseq sync upload --graph <name>` and confirm it reuses the existing graph id.
6. Force a failing auth token and confirm the CLI returns an error instead of false success.
7. Force a server-side snapshot upload failure and confirm the CLI returns an error with useful context.

## Acceptance criteria

The fix is complete when all of the following are true:

- `logseq sync upload --graph <name>` works for a fresh local graph with no prior `graph-id`.
- The command reuses an existing same-name remote graph when appropriate.
- Worker upload failures propagate to CLI callers and no longer appear as success.
- Desktop and CLI upload bootstrap logic no longer diverge in a risky way.
- Test coverage includes first-time upload and failure propagation.
- CLI docs describe first-time upload behavior and required config without exposing secrets.

## Risks and mitigations

### Risk: duplicate remote graphs

If upload always creates a graph when local `graph-id` is missing, we may create duplicates.

Mitigation: list remote graphs first and only create when there is no exact name match.

### Risk: inconsistent E2EE defaults

Fresh graph creation from CLI may accidentally create an unencrypted graph while desktop defaults to encrypted.

Mitigation: make fresh-upload `graph-e2ee?` behavior explicit and test it.

### Risk: hidden regressions in desktop flow

If desktop and worker share more code, UI expectations could shift.

Mitigation: keep only transport/bootstrap logic shared; keep UI state refresh and notifications in frontend handler code.

## Suggested file touch list

Primary implementation files:

- `/Users/rcmerci/gh-repos/logseq/src/main/frontend/worker/sync.cljs`
- `/Users/rcmerci/gh-repos/logseq/src/main/frontend/handler/db_based/sync.cljs`
- `/Users/rcmerci/gh-repos/logseq/src/main/logseq/cli/command/sync.cljs`

Primary test files:

- `/Users/rcmerci/gh-repos/logseq/src/test/frontend/worker/db_sync_test.cljs`
- `/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/command/sync_test.cljs`
- `/Users/rcmerci/gh-repos/logseq/src/test/logseq/cli/integration_test.cljs`

Primary docs:

- `/Users/rcmerci/gh-repos/logseq/docs/cli/logseq-cli.md`

## Question

No open question. The current implementation gap is clear enough to proceed with a worker-centered fix.