mirror of
https://github.com/logseq/logseq.git
synced 2026-05-02 18:06:32 +00:00
133 lines
3.9 KiB
Markdown
133 lines
3.9 KiB
Markdown
# Logseq DB Sync (deps/db-sync)
|
|
|
|
This package contains the DB sync server code and tests used by Logseq.
|
|
It includes the Cloudflare Worker implementation and a Node.js adapter for self-hosting.
|
|
|
|
## Requirements
|
|
- Node.js (see repo root for required version)
|
|
- Clojure (for shadow-cljs builds)
|
|
|
|
## Build and Test
|
|
|
|
### Cloudflare Worker
|
|
|
|
```bash
|
|
cd deps/db-sync
|
|
yarn watch
|
|
|
|
# open another terminal
|
|
cd deps/db-sync/worker
|
|
wrangler dev
|
|
```
|
|
|
|
### D1 Schema (Worker)
|
|
|
|
The worker no longer initializes schema at request time. Apply the D1 schema
|
|
via migrations during deployment/CI.
|
|
|
|
```bash
|
|
cd deps/db-sync/worker
|
|
wrangler d1 migrations apply logseq-sync-graph-meta-staging --env staging
|
|
wrangler d1 migrations apply logseq-sync-graphs-prod --env prod
|
|
```
|
|
|
|
For local development, run `wrangler d1 migrations apply DB --local`.
|
|
|
|
### Production Graph Lookup
|
|
|
|
Show the graphs available to a production user by `username` or `user id`:
|
|
|
|
```bash
|
|
cd deps/db-sync
|
|
yarn show-graphs-for-user --username alice
|
|
yarn show-graphs-for-user --user-id us-east-1:example-user-id
|
|
```
|
|
|
|
The script uses `worker/wrangler.toml`, runs against the remote D1 binding `DB`,
|
|
defaults to `--env prod`, and prints JSON when `--json` is added.
|
|
|
|
Download a graph snapshot into a local sqlite debug file matching local graph DB schema (`kvs` table only):
|
|
|
|
```bash
|
|
cd deps/db-sync
|
|
yarn download-graph-db --graph-id 6f2d7f6f-xxxx-xxxx-xxxx-xxxxxxxxxxxx --admin-token <admin-token>
|
|
```
|
|
|
|
You can also pass `--admin-token <token>` or set `DB_SYNC_ADMIN_TOKEN`. The output defaults to
|
|
`tmp/graph-<graph-id>.snapshot.sqlite` and can be changed with `--output`.
|
|
|
|
Delete the graphs owned by a production user after an explicit confirmation:
|
|
|
|
```bash
|
|
cd deps/db-sync
|
|
yarn delete-graphs-for-user --username alice
|
|
yarn delete-graphs-for-user --user-id us-east-1:example-user-id
|
|
```
|
|
|
|
The delete script shows the owned graphs first and requires typing `DELETE`
|
|
before it calls the worker delete endpoint for each graph. Set
|
|
`DB_SYNC_BASE_URL` and `DB_SYNC_ADMIN_TOKEN` or pass `--base-url` and
|
|
`--admin-token` when running it.
|
|
|
|
Delete a user completely (owned graphs, memberships, keys, and user row):
|
|
|
|
```bash
|
|
cd deps/db-sync
|
|
yarn delete-user-totally --username alice
|
|
yarn delete-user-totally --user-id us-east-1:example-user-id
|
|
```
|
|
|
|
The script prints all linked graphs first, deletes owned graphs through the
|
|
admin graph delete endpoint, then removes the user's remaining D1 references.
|
|
It requires typing `DELETE` as confirmation.
|
|
|
|
### Node.js Adapter (self-hosted)
|
|
|
|
Build the adapter:
|
|
|
|
```bash
|
|
cd deps/db-sync
|
|
npm run build:node-adapter
|
|
```
|
|
|
|
Run the adapter with Cognito auth:
|
|
|
|
```bash
|
|
DB_SYNC_PORT=8787 \
|
|
COGNITO_ISSUER=https://cognito-idp.us-east-1.amazonaws.com/us-east-1_dtagLnju8 \
|
|
COGNITO_CLIENT_ID=69cs1lgme7p8kbgld8n5kseii6 \
|
|
COGNITO_JWKS_URL=https://cognito-idp.us-east-1.amazonaws.com/us-east-1_dtagLnju8/.well-known/jwks.json \
|
|
node worker/dist/node-adapter.js
|
|
```
|
|
|
|
### Tests
|
|
|
|
Run db-sync tests (includes Node adapter tests):
|
|
|
|
```bash
|
|
cd deps/db-sync
|
|
npm run test:node-adapter
|
|
```
|
|
|
|
## Environment Variables
|
|
|
|
| Variable | Purpose |
|
|
| --- | --- |
|
|
| DB_SYNC_PORT | HTTP server port |
|
|
| DB_SYNC_BASE_URL | External base URL for asset links |
|
|
| DB_SYNC_ADMIN_TOKEN | Admin-only token for operator graph deletion endpoints |
|
|
| DB_SYNC_DATA_DIR | Data directory for sqlite + assets |
|
|
| DB_SYNC_STORAGE_DRIVER | Storage backend selection (sqlite) |
|
|
| DB_SYNC_ASSETS_DRIVER | Assets backend selection (filesystem) |
|
|
| SENTRY_DSN | Sentry DSN |
|
|
| SENTRY_RELEASE | Release identifier for Sentry events and sourcemaps |
|
|
| SENTRY_ENVIRONMENT | Sentry environment name (prod, staging, etc.) |
|
|
| SENTRY_TRACES_SAMPLE_RATE | Traces sample rate (0.0 - 1.0) |
|
|
| COGNITO_ISSUER | Cognito issuer URL |
|
|
| COGNITO_CLIENT_ID | Cognito client id |
|
|
| COGNITO_JWKS_URL | Cognito JWKS URL |
|
|
|
|
## Notes
|
|
- Protocol definitions live in `docs/agent-guide/db-sync/protocol.md`.
|
|
- DB sync implementation guide is in `docs/agent-guide/db-sync/db-sync-guide.md`.
|