Commit Graph

86 Commits

Author SHA1 Message Date
Gabriel Horner
ed805f039f chore: cleanup bb tasks after file graph removal
Remove needless 'db-' prefix for several dev tasks. Remove linter for
file and db graph separation. It served us well :)
2026-01-22 13:19:58 -05:00
Gabriel Horner
37af8f690e fix: remove remaining uses of file graph specific attributes
Removed all file graph uses of :block/namespace, :block/file, :block/type and
:block/properties except for graph-parser which still uses them for db
importer.
Removed all uses of :block/format except for graph-parser and
src/main/frontend
2026-01-22 13:19:58 -05:00
Gabriel Horner
ef96a8d031 chore: remove db scripts made redundant by CLI 2025-12-12 10:17:12 -05:00
Gabriel Horner
8fc56bfaa3 refactor: Update importer to use validate-local-db
Also update validation in DB scripts:
* replace alternative validate-db impl with standard CLI one
* Add validation to import script
* Choose to decouple other DB scripts from validate-db as it may go away
  later
2025-12-04 10:30:56 -05:00
Gabriel Horner
7bb8655d71 chore: remove unused errors-by-type 2025-11-19 23:31:04 -05:00
Gabriel Horner
79e75060d6 enhance(cli): Add validate command for local graph(s)
Improves :errors of invalid errors to be more user friendly.
Also enable closed-value validation optionally as it is useful
to have higher data quality for non RTC graphs
2025-11-19 17:51:45 -05:00
Tienson Qin
7ed7fb4f31 enhance: prefer ldb/transact! than d/transact! 2025-10-13 19:03:11 +08:00
Gabriel Horner
32a29f5f3d enhance(dev): CLI can diff other export types 2025-06-26 17:02:14 -04:00
Gabriel Horner
0cb5c99502 enhance(dev): CLI can export other types e.g. graph-ontology 2025-06-26 12:12:50 -04:00
Tienson Qin
92ec65d61d Merge branch 'master' into refactor/page-parent 2025-06-15 08:27:33 +08:00
Gabriel Horner
3fe790d4d7 enhance(dev): scripts can read or write db graphs as full path files
This allows for easier use of debugging graphs as they are exported by
users as a file. Also DRYed up duplicated helper
2025-05-23 13:59:02 -04:00
Gabriel Horner
691b8866ef refactor: mv another common ns out of db dir
Allows for more improved separate linting
2025-04-28 11:27:49 -04:00
Gabriel Horner
6ce701ccc3 fix: remove clj-kondo/ignore
that was moved to incorrect location.  Also fix clj-kondo config
so those ignores aren't needed anymore
2025-04-08 13:36:17 -04:00
Tienson Qin
1aca680ae4 Merge branch 'feat/db' into perf/app-start 2025-03-31 21:53:37 +08:00
Gabriel Horner
dd4b68a140 fix: export for overlapping property+class
overlapping entity with blocks was exporting duplicate blocks
2025-03-28 10:39:20 -04:00
Gabriel Horner
9dfb6713ca enhance: export+import supports properties
that double/overlap as classes
2025-03-27 17:02:32 -04:00
Gabriel Horner
e5061f1504 enhance: add export option to exclude-files
Useful for demo graphs as it's noisy and not useful to include since
imported graphs already have it. Also added descriptions for properties
graph to share it
2025-03-27 09:13:14 -04:00
Gabriel Horner
17acbbc8d4 enhance(dev): option to catch export validation errors
to write invalid edn and debug issues quicker
2025-03-26 10:25:54 -04:00
Gabriel Horner
f99a89f9b3 enhance: add :logseq.kv data to graph export
Addresses
https://github.com/logseq/logseq/pull/11784#pullrequestreview-2697690952.
Also improves :set option and removes outdated code in a test
2025-03-19 15:41:12 -04:00
Gabriel Horner
210f583cae enhance(dev): add set option for diff script 2025-03-18 13:00:43 -04:00
Gabriel Horner
332b7a0825 fix: lints and new scripts having same path bugs as feat/db 2025-03-17 12:08:44 -04:00
Gabriel Horner
9b9c22933f fix: built-in pages option
also renamed option and fixed a sorting bug
2025-03-17 11:59:16 -04:00
Gabriel Horner
c215617a0f fix extra exported property for graph
and add import option to diff script
2025-03-17 11:59:16 -04:00
Gabriel Horner
5c07ff3fe7 enhance: option to ignore built-ins for graph export
useful for lambda which has buggy built-in pages
2025-03-17 11:59:16 -04:00
Gabriel Horner
d7721981af enhance: add :exclude-namespaces export option
Useful for graphs with existing ontologies to avoid noisy
and needless exporting. Also update related scripts and noted
that create script has always been able to also update
2025-03-17 11:59:16 -04:00
Gabriel Horner
c344f4a577 CLI can create graphs from exported sqlite.build EDN 2025-03-17 11:59:13 -04:00
Gabriel Horner
ab9796ae60 Add scripts for exporting and diffing graphs
Allows for quick export<->import QA of more complex graphs
and any examples made in app
2025-03-17 11:58:30 -04:00
Gabriel Horner
fecb42f49d fix(dev): deps scripts not handling relative and absolute paths for graphs
transact and create didn't handle relative paths. All scripts except
import didn't handle absolute paths
2025-03-17 11:09:44 -04:00
Gabriel Horner
0cc9205e99 enhance(dev): option to query multiple graphs 2025-03-13 16:00:01 -04:00
Tienson Qin
5d2ab65d7c Merge branch 'feat/db' into feat/blocks-action-bar 2025-03-08 13:55:33 +08:00
Gabriel Horner
7917afaa45 chore: update nbb scripts to be repl friendly
With https://github.com/babashka/nbb/issues/95 done, this allows
all nbb script namespaces to required or used from a repl
2025-03-07 11:49:28 -05:00
Gabriel Horner
1ccdcab8a9 enhance(dev): optionally validate after creating a db
Doing this enough that an option makes this faster to invoke
and faster (no re-open) on large graphs
2025-03-07 11:45:17 -05:00
Gabriel Horner
c6a2ea6959 fix(dev): script fails on -h
Also minor db scripts cleanup
2025-03-06 16:21:21 -05:00
Gabriel Horner
8b8055dd72 refactor: use namespaced keyword for block export
instead of confusing :build/block. Also improved related tests,
simplified import steps, added some missing sqlite.build docs,
and fixed :build/uuid not working for some existing journals.
2025-02-10 15:18:45 -05:00
Gabriel Horner
455ccec296 chore: rename sqlite.build edn to remove mention
of schema. schema concept no longer exists and these old
names would be confusing
2025-01-21 14:22:16 -05:00
Gabriel Horner
68896c257f Merge branch 'feat/db' into refactor/block-schema 2025-01-16 17:11:53 -05:00
Gabriel Horner
769e68a3c6 enhance(dev): add :dispatch-key to validate-db! errors
so it's available to tests and to app
2025-01-15 18:04:47 -05:00
Tienson Qin
a7e9d073cd chore: rename :property/* to :logseq.property/* 2025-01-15 14:59:08 +08:00
Tienson Qin
93a7034249 Remove more cases of :block/schema 2025-01-13 10:00:00 +08:00
Gabriel Horner
b113967c9c enhance(dev): cli validation includes dispatch key
use dispatch-key to immediately find invalid schema
2025-01-10 10:06:43 -05:00
Tienson Qin
68f758cdc9 fix: invalid data
/Validate current graph will try to fix those errors.

fixes https://github.com/logseq/db-test/issues/205.
2025-01-09 21:15:12 +08:00
Tienson Qin
8a0a05183e refactor: remove :block/format for db graphs (#11667) 2025-01-05 12:10:47 +08:00
Gabriel Horner
1bba10c266 fix: graph counts after validation are wrong
after type->tags migration. Also update recommended validate cli options
to default to true. Also clean up graph counts spread across two
validators and importer. The property-pairs count in the UI were way off
because they were only counting pairs that are validated by malli-schema
2025-01-04 02:14:39 -05:00
rcmerci
b0379811c0 fix: fix validate db test (2) 2024-12-02 13:59:06 +08:00
rcmerci
d2c877b484 fix: fix validate db test 2024-12-02 13:51:27 +08:00
Gabriel Horner
e231192cda enhance: add import metadata
Useful for debugging different types of imports and knowing
when import was done, especially since it can differ from when
graph was created
2024-10-17 13:23:30 -04:00
Gabriel Horner
fbcaeb5e20 fix: example edn uses outdated property type 2024-08-20 17:10:28 -04:00
Tienson Qin
f46372b346 chore: keep using class instead of tag for internal usage 2024-08-06 07:15:41 +08:00
Tienson Qin
411b05e7ba Prefer to use ldb/{type}? check when possible 2024-08-05 17:51:24 +08:00
Tienson Qin
4083a2d0a0 enhance: prefer tag instead of class on UI
Some users are confused with Classes/Objects, so on UI it'll be
Tags/Nodes.
2024-08-05 15:06:54 +08:00