Commit Graph

41 Commits

Author SHA1 Message Date
Tienson Qin
5d602c9d58 fix(db-sync): drop stale lookup-ref tx datoms 2026-03-12 08:18:52 +08:00
Tienson Qin
0055e8128c add d1 sql table indexes 2026-03-04 11:18:58 +08:00
Tienson Qin
8f923a911d Revert "client hello now sends since"
This reverts commit 2799a57074.
2026-03-03 22:37:30 +08:00
Tienson Qin
2799a57074 client hello now sends since 2026-03-03 20:52:44 +08:00
Tienson Qin
1df918728b fix: ws latency 2026-02-26 17:42:25 +08:00
Tienson Qin
ae9a37c889 fix: sync missing :block/title 2026-02-26 10:32:25 +08:00
Tienson Qin
084b5193c3 fix: sync doesn't work on node adapter 2026-02-12 19:31:24 +08:00
Tienson Qin
adff65c6bc support unencrypted graph 2026-02-12 02:15:14 +08:00
Tienson Qin
5d515a8e28 fix: don't throw error for exp/iss not found 2026-02-11 21:11:04 +08:00
Tienson Qin
10a8e3c491 fix: integrate large graph fixes from ulcaefna
Discord link: https://discord.com/channels/725182569297215569/1400122537744007198/1470853605753094288

1. Added snapshot streaming endpoint on worker:

  - deps/db-sync/src/logseq/db_sync/worker/routes/sync.cljs
  - deps/db-sync/src/logseq/db_sync/worker/handler/sync.cljs
      - Added :sync/snapshot-stream
      - Increased snapshot-download-batch-size to 5000
      - Added schema-init fallback probing (kvs, tx_log, sync_meta)
      - Improved HTTP error responses with debug fields

2. Switched frontend graph download from R2 URL flow to direct stream:

  - src/main/frontend/handler/db_based/sync.cljs
      - Removed /snapshot/download + cleanup DELETE flow
      - Uses direct GET /snapshot/stream

3. Applied large-graph upload stability fixes:

  - src/main/frontend/worker/sync.cljs
      - upload-kvs-batch-size changed 2000 -> 500
      - Reworked offload-large-titles-in-datoms to avoid deep promise chains
      - Fixed upload progress message to use updated loaded count
  - src/main/frontend/worker/sync/crypt.cljs
      - Batched <encrypt-datoms in chunks of 5000
  - src/main/frontend/worker/sync/client_op.cljs
      - update-graph-uuid now retracts old :graph-uuid datoms before add

4. Applied worker-side error visibility and catch hardening:

  - deps/db-sync/src/logseq/db_sync/worker.cljs
  - deps/db-sync/src/logseq/db_sync/worker/dispatch.cljs
  - deps/db-sync/src/logseq/db_sync/worker/coerce.cljs
  - deps/db-sync/src/logseq/db_sync/worker/http.cljs
  - deps/db-sync/src/logseq/db_sync/worker/handler/index.cljs

5. Added/updated tests for these behaviors:
2026-02-11 14:41:26 +08:00
Gabriel Horner
cd7c1ef50e enhance(dev): add linters for db-sync
Fix minor kondo errors. Also removed obvious dead code found by carve linter
2026-02-09 12:25:50 -05:00
Tienson Qin
54844ff95f no need to run db-sync tests in build CI
because it's already handled by its own dep ci test
2026-02-06 19:08:52 +08:00
Tienson Qin
e428d52ee3 fix: can't upsert user in node adapter 2026-02-06 18:13:25 +08:00
Tienson Qin
7a915a31b1 remove static auth because signin/up doesn't support custom login 2026-02-06 17:35:30 +08:00
Tienson Qin
8bbc0fede8 enhance: gzip download graph 2026-02-06 16:30:01 +08:00
Tienson Qin
2ea618b76f fix: don't notify source client with presence update 2026-02-06 15:55:55 +08:00
Gabriel Horner
c1ca626b36 enhance(dev): add workflow to run db-sync's tests
Also disable test ns that has hanging tests. Was able to pinpoint
where tests are hanging and left logging there to fix it in the future
2026-02-04 16:36:16 -05:00
Tienson Qin
b8445a6395 node.js adapter 2026-01-31 00:51:35 +08:00
Tienson Qin
e00a037624 fix: don't run sync for local graphs that's detached from server 2026-01-30 03:00:28 +08:00
Tienson Qin
96e2be398f fix: asset op 2026-01-30 02:37:40 +08:00
Tienson Qin
8e815b85b8 remove buggy code 2026-01-27 23:28:44 +08:00
Tienson Qin
f46c245788 refactor: keep worker as thin as possible 2026-01-26 22:26:53 +08:00
Tienson Qin
cd0a2568b1 refactor: use reitit for worker routes 2026-01-26 21:33:15 +08:00
Tienson Qin
ebf768e340 fix: extends cycle 2026-01-26 18:38:47 +08:00
Tienson Qin
0cf01ba780 fix: parent cycle 2026-01-26 17:36:34 +08:00
Tienson Qin
59e1cd9af9 enhance(ux): able to remove graph access or leave a graph 2026-01-26 15:54:25 +08:00
Tienson Qin
1f71f9e78e e2ee 2026-01-22 18:50:05 +08:00
Tienson Qin
bebddd677e batch insert 2026-01-22 00:30:49 +08:00
Tienson Qin
3f4685f862 fix: graph role 2026-01-21 23:05:28 +08:00
Tienson Qin
fe313b50dc fix: ensure user doesn't have graphs with same name 2026-01-21 22:56:51 +08:00
Tienson Qin
83a3a7cdf9 use R2 for both graph upload/download 2026-01-21 22:43:09 +08:00
Tienson Qin
8c74eb2736 fix: tests 2026-01-21 18:32:58 +08:00
Tienson Qin
cc95768288 Switched API-facing map keys to hyphenated keywords 2026-01-21 18:21:00 +08:00
Tienson Qin
4fb4e87066 add users and graph_members tables 2026-01-21 07:08:50 +08:00
Tienson Qin
556af5581b fix: put fix after rebase 2026-01-16 19:35:05 +08:00
Tienson Qin
f6114afe47 add cycle tests 2026-01-15 10:41:50 +08:00
Tienson Qin
2afc7fe48d another try of client fix 2026-01-13 18:54:20 +08:00
Tienson Qin
c03a5183df refactor: fix data in server instead of client 2026-01-12 23:56:50 +08:00
rcmerci
4cbb01f78c fix(db-sync): fix test 2026-01-11 15:44:06 +08:00
Tienson Qin
b0309307f7 wip: missing parent && order conflicts 2026-01-11 11:18:57 +08:00
rcmerci
0864b485f4 refactor(db-sync): rename worker-sync to db-sync 2026-01-10 17:52:33 +08:00