Commit Graph

107 Commits

Author SHA1 Message Date
Tienson Qin
81d1ded924 fix(sync): gate rtc until snapshot upload finishes 2026-03-12 20:00:29 +08:00
Tienson Qin
f538491096 chore: reduce test noise and infer warnings 2026-03-12 17:51:23 +08:00
Tienson Qin
80b48b7b86 enhance(rtc): improve snapshot import progress 2026-03-12 17:51:23 +08:00
Tienson Qin
7386eec6db fix(db-sync): server shouldn't delete addrs 2026-03-12 08:28:38 +08:00
Tienson Qin
5d602c9d58 fix(db-sync): drop stale lookup-ref tx datoms 2026-03-12 08:18:52 +08:00
Tienson Qin
83a716f907 chore: bump datascript 2026-03-11 14:49:12 +08:00
Tienson Qin
0055e8128c add d1 sql table indexes 2026-03-04 11:18:58 +08:00
Tienson Qin
8f923a911d Revert "client hello now sends since"
This reverts commit 2799a57074.
2026-03-03 22:37:30 +08:00
Tienson Qin
2799a57074 client hello now sends since 2026-03-03 20:52:44 +08:00
Tienson Qin
7d268fbd42 fix: lint 2026-02-26 22:57:25 +08:00
Tienson Qin
225cdac8ae disable invocation logs 2026-02-26 22:54:14 +08:00
Tienson Qin
1df918728b fix: ws latency 2026-02-26 17:42:25 +08:00
Tienson Qin
5f2ec2ff71 fix: no log for sync server 2026-02-26 14:54:42 +08:00
Tienson Qin
ae9a37c889 fix: sync missing :block/title 2026-02-26 10:32:25 +08:00
Tienson Qin
daf7fc1475 enhance: p r to add emoji reaction 2026-02-12 23:03:28 +08:00
Tienson Qin
4a104ec0aa fix: error log 2026-02-12 20:33:00 +08:00
Tienson Qin
64b218896a code cleanup suggested by gabriel 2026-02-12 20:08:20 +08:00
Tienson Qin
084b5193c3 fix: sync doesn't work on node adapter 2026-02-12 19:31:24 +08:00
Tienson Qin
adff65c6bc support unencrypted graph 2026-02-12 02:15:14 +08:00
Tienson Qin
5d515a8e28 fix: don't throw error for exp/iss not found 2026-02-11 21:11:04 +08:00
Tienson Qin
dbef84fbb4 fix: auto-reconnect after long sleep by adding ws health check 2026-02-11 15:15:40 +08:00
Tienson Qin
10a8e3c491 fix: integrate large graph fixes from ulcaefna
Discord link: https://discord.com/channels/725182569297215569/1400122537744007198/1470853605753094288

1. Added snapshot streaming endpoint on worker:

  - deps/db-sync/src/logseq/db_sync/worker/routes/sync.cljs
  - deps/db-sync/src/logseq/db_sync/worker/handler/sync.cljs
      - Added :sync/snapshot-stream
      - Increased snapshot-download-batch-size to 5000
      - Added schema-init fallback probing (kvs, tx_log, sync_meta)
      - Improved HTTP error responses with debug fields

2. Switched frontend graph download from R2 URL flow to direct stream:

  - src/main/frontend/handler/db_based/sync.cljs
      - Removed /snapshot/download + cleanup DELETE flow
      - Uses direct GET /snapshot/stream

3. Applied large-graph upload stability fixes:

  - src/main/frontend/worker/sync.cljs
      - upload-kvs-batch-size changed 2000 -> 500
      - Reworked offload-large-titles-in-datoms to avoid deep promise chains
      - Fixed upload progress message to use updated loaded count
  - src/main/frontend/worker/sync/crypt.cljs
      - Batched <encrypt-datoms in chunks of 5000
  - src/main/frontend/worker/sync/client_op.cljs
      - update-graph-uuid now retracts old :graph-uuid datoms before add

4. Applied worker-side error visibility and catch hardening:

  - deps/db-sync/src/logseq/db_sync/worker.cljs
  - deps/db-sync/src/logseq/db_sync/worker/dispatch.cljs
  - deps/db-sync/src/logseq/db_sync/worker/coerce.cljs
  - deps/db-sync/src/logseq/db_sync/worker/http.cljs
  - deps/db-sync/src/logseq/db_sync/worker/handler/index.cljs

5. Added/updated tests for these behaviors:
2026-02-11 14:41:26 +08:00
Gabriel Horner
cd7c1ef50e enhance(dev): add linters for db-sync
Fix minor kondo errors. Also removed obvious dead code found by carve linter
2026-02-09 12:25:50 -05:00
Gabriel Horner
8bf63e9715 chore: docs on running wrangler locally 2026-02-09 11:31:41 -05:00
Tienson Qin
54844ff95f no need to run db-sync tests in build CI
because it's already handled by its own dep ci test
2026-02-06 19:08:52 +08:00
Tienson Qin
e428d52ee3 fix: can't upsert user in node adapter 2026-02-06 18:13:25 +08:00
Tienson Qin
7a915a31b1 remove static auth because signin/up doesn't support custom login 2026-02-06 17:35:30 +08:00
Tienson Qin
8bbc0fede8 enhance: gzip download graph 2026-02-06 16:30:01 +08:00
Tienson Qin
2ea618b76f fix: don't notify source client with presence update 2026-02-06 15:55:55 +08:00
Gabriel Horner
c1ca626b36 enhance(dev): add workflow to run db-sync's tests
Also disable test ns that has hanging tests. Was able to pinpoint
where tests are hanging and left logging there to fix it in the future
2026-02-04 16:36:16 -05:00
Tienson Qin
ef17ce59c3 add sentry to collect errors 2026-02-01 23:39:25 +08:00
Tienson Qin
f26f984b07 fix: latency 2026-01-31 07:32:13 +08:00
Tienson Qin
8f76e4dbb0 don't initialize schema at request time 2026-01-31 07:01:30 +08:00
Tienson Qin
0fa673978f debug latency 2026-01-31 06:44:01 +08:00
Tienson Qin
a9c7a35d77 disable smart placement 2026-01-31 05:38:47 +08:00
Tienson Qin
ddb4db21c6 add session api to use read replica 2026-01-31 05:14:48 +08:00
Tienson Qin
328d8e2bf3 fix: use prod d1 db 2026-01-31 04:23:17 +08:00
Tienson Qin
b8445a6395 node.js adapter 2026-01-31 00:51:35 +08:00
Tienson Qin
3adff65c90 add prod env for sync 2026-01-30 20:55:55 +08:00
Tienson Qin
408739950a fix: malli schema 2026-01-30 17:29:59 +08:00
Tienson Qin
63c9c65c95 fix: no need to broadcast online users when updating presence 2026-01-30 17:16:59 +08:00
Tienson Qin
e00a037624 fix: don't run sync for local graphs that's detached from server 2026-01-30 03:00:28 +08:00
Tienson Qin
96e2be398f fix: asset op 2026-01-30 02:37:40 +08:00
Tienson Qin
61c877de4f server returns tx/reject when transaction failed 2026-01-29 16:29:16 +08:00
Tienson Qin
8e815b85b8 remove buggy code 2026-01-27 23:28:44 +08:00
Tienson Qin
a8b7199a5e server no longer check cycle 2026-01-26 22:34:00 +08:00
Tienson Qin
f46c245788 refactor: keep worker as thin as possible 2026-01-26 22:26:53 +08:00
Tienson Qin
cd0a2568b1 refactor: use reitit for worker routes 2026-01-26 21:33:15 +08:00
Tienson Qin
143a548f04 fix: more lint error 2026-01-26 20:00:31 +08:00
Tienson Qin
ebf768e340 fix: extends cycle 2026-01-26 18:38:47 +08:00