Commit Graph

349 Commits

Author SHA1 Message Date
Gabriel Horner
045cd5f537 enhance: import block tags that are
specified via tags property. Fixes
https://github.com/logseq/db-test/issues/434
2025-12-18 13:35:42 -05:00
Gabriel Horner
b4f3e364e6 fix: duplication of exporter config leads to bugs
in script and test. Also fix <export-file not being waited on and new docs
after #12260
2025-12-15 12:15:48 -05:00
Tienson Qin
33db791ac0 fix: import Zotero pdf blocks (#12260)
fix: support zotero files when importing pdf annotations
2025-12-15 23:28:58 +08:00
Gabriel Horner
8fc56bfaa3 refactor: Update importer to use validate-local-db
Also update validation in DB scripts:
* replace alternative validate-db impl with standard CLI one
* Add validation to import script
* Choose to decouple other DB scripts from validate-db as it may go away
  later
2025-12-04 10:30:56 -05:00
Tienson Qin
e311e1b85c fix: don't create child tags when saving inline 2025-11-21 00:33:34 +08:00
Gabriel Horner
c91661f022 fix: import code block inside quote block
Fixes https://github.com/logseq/db-test/issues/523
2025-11-04 10:48:09 -05:00
Tienson Qin
248fdeb422 fix: tests 2025-09-29 17:37:04 -04:00
Tienson Qin
971cec54bc fix(import): remove asset byte-array usage to prevent OOM
This commit also combine read-asset and copy-asset to read-and-copy-asset
2025-09-29 17:37:04 -04:00
Tienson Qin
248d8c33f2 refactor: remove block/path-refs (#12081)
1. refactor: use :block/refs and has-ref rule instead of path-refs
2. remove block/path-refs
3. removes :block-parent rule since there's already :parent
4. enhance: skip pipeline calculation for rtc initial download tx
5. refactor(rtc): remove memoize

---------

Co-authored-by: rcmerci <rcmerci@gmail.com>
2025-09-03 20:33:33 +08:00
Gabriel Horner
a81274cd0d fix: import quoted math blocks
and lists in quote blocks. Addresses main issue in https://github.com/logseq/db-test/issues/434
2025-09-02 13:52:32 -04:00
Gabriel Horner
f4a8b9438e fix: block references to annotations
Don't rewrite annotation uuids as we don't do that for other existing blocks
2025-07-22 08:57:21 -04:00
Gabriel Horner
ad5c776f46 fix: doc files in assets shouldn't be extracted
Asset files that are edn or md were being treated like assets and
extracted as doc files. This was noticed because pdf annotation edn was being
needlessly extracted
2025-07-22 08:57:21 -04:00
Gabriel Horner
3f21c297b2 enhance: import markdown annotations
for pdfs and any children blocks under highlight. Only
imports text from md annotations
2025-07-22 08:57:21 -04:00
Gabriel Horner
4c9d063323 enhance: import pdf area annotations 2025-07-22 08:57:21 -04:00
Gabriel Horner
536f39cbbd enhance: import basic pdf text annotation
Only read edn for now. Works in UI and CLI.
Also correctly keeps annotation edn from being copied since
it's no longer used
2025-07-22 08:57:21 -04:00
Tienson Qin
9bede53a34 fix: tests 2025-06-29 15:21:09 +08:00
Gabriel Horner
42f0519797 fix: import property that changes from :node to :default
on same page :as :node property is defined. Also affects :date ->
:default. Fixes https://github.com/logseq/db-test/issues/340
2025-06-23 14:02:28 -04:00
Gabriel Horner
58d3abbafb fix: use ast blocks for simple queries conversions
Fixes cases where {{query nested in BEGIN_* blocks were incorrectly
being converted to queries. Also made ast block handling faster
by only doing one prewalk and cleaned up handling of queries
2025-06-23 19:04:03 +08:00
Gabriel Horner
cc05518ea5 fix: use ast blocks for advanced queries conversion
Fixes cases where BEGIN_QUERY nested in BEGIN_SRC were incorrectly
being converted to queries e.g. most queries in
https://docs.logseq.com/#/page/advanced%20queries
2025-06-23 19:04:03 +08:00
Gabriel Horner
507ff3ce29 enhance: import page and block embeds
One embed per block as that's what DB version supports.
Fixes https://github.com/logseq/db-test/issues/198
2025-06-23 19:04:03 +08:00
Gabriel Horner
fdff75cdf8 fix: import not handling scheduled and deadline property
on same block. Fixes https://github.com/logseq/db-test/issues/318
2025-06-20 17:06:52 -04:00
Gabriel Horner
8c6146a00d enhance: import markdown quotes
Went ahead and implemented ast block to string
as regex version of this would've been to buggy
2025-06-18 13:54:48 -04:00
Gabriel Horner
5b4f43d6f4 enhance: import quotes as #Quote
Can only support one quote per block so text around quotes
and multiple quotes aren't possible
Fixes https://github.com/logseq/db-test/issues/195
2025-06-18 13:54:48 -04:00
Tienson Qin
7b46076170 Merge branch 'master' into refactor/page-parent 2025-06-17 19:44:34 +08:00
Gabriel Horner
ace71b2b25 enhance: asset import supports multiple assets
per block and preserves text around assets.
These were both enabled by moving asset blocks to Asset page
2025-06-17 19:36:05 +08:00
Gabriel Horner
501063e625 fix: importing assets for more complex blocks like paragraphs with links
Used extract-blocks to group ast blocks for each block instead of
previous buggy version. Also renamed bindings that were all confusingly
`block`
2025-06-17 19:36:05 +08:00
Gabriel Horner
f318d68530 enhance: import assets handles
unused assets by still copying them by name and handles
existing assets
2025-06-17 19:36:05 +08:00
Gabriel Horner
bb31c4e6f1 fix: importing assets in subdirectories
Also fix lint
2025-06-17 19:36:05 +08:00
Gabriel Horner
4524fc8370 enhance: import asset metadata
Also handle multiple assets per block and displaying that to user
2025-06-17 19:36:05 +08:00
Gabriel Horner
90c7fc5775 enhance: UI imports assets as #Asset
Also made importer and db asset creation use same helper fns
2025-06-17 19:36:05 +08:00
Gabriel Horner
38734bde40 enhance: CLI imports assets as #Asset
Had to split assets into two steps now that asset naming is dependent
on uuids. Also add explicit asserts in exporter to catch silent
failure in tests
2025-06-17 19:36:05 +08:00
Tienson Qin
92ec65d61d Merge branch 'master' into refactor/page-parent 2025-06-15 08:27:33 +08:00
Tienson Qin
88216d2992 fix: able to convert an existing page to a property (#11940)
* enhance: use page as property

* fix: ensures block/uuid doesn't change after converting to property
2025-06-14 13:17:33 +08:00
Gabriel Horner
7cc9ddc26b chore: bump docs graph being tested
Hadn't been updated in over a year
2025-06-13 10:43:42 -04:00
Tienson Qin
d7f0c248ff fix: bring back :ref/default-open-blocks-level 2025-06-02 11:19:10 +08:00
Tienson Qin
33b6433dc9 wip: move file parse to worker 2025-05-29 00:39:51 +08:00
Gabriel Horner
471889729e fix: graphs with existing Library page are imported
incorrectly and merged with new built-in page when :property-classes
option is used
2025-05-21 14:31:12 -04:00
Tienson Qin
6c12676a08 fix: check pages only 2025-05-17 05:46:23 +08:00
Tienson Qin
42163df4b3 Merge branch 'feat/db' into refactor/page-parent 2025-05-17 05:36:58 +08:00
Tienson Qin
7dbb3eb3bd wip: fix graph-parser tests 2025-05-17 02:49:42 +08:00
Gabriel Horner
6c07d8838e fix: invalid keywords can be created by user
In https://github.com/logseq/db-test/issues/278, user has graph that
contains invalid edn keyword caused by name part of keyword starting
with a number e.g. :user.property/2ndsomething. Added thorough tests
based on reference doc for both nbb and cljs versions of buggy fn. For
cljs version, allowed a few more special characters in as they've
all been confirmed to be valid edn
2025-05-16 14:02:17 -04:00
Gabriel Horner
a16eb2eb10 chore: bump exporter test time
exporter taking little longer on docs graph with recent
change where advanced query rewrite is included in test
2025-05-16 13:46:26 -04:00
Tienson Qin
42778d36ae Merge branch 'feat/db' into refactor/page-parent 2025-05-16 23:27:20 +08:00
Tienson Qin
dca6ef95ee add migration 2025-05-16 21:31:06 +08:00
Gabriel Horner
dda13d61b9 enhance: improved config.edn for new db graphs
New db config.edn don't include any of the file graph keys or comments.
Previously all comments were confusingly left lying around.
Also moved conversion of imported config.edn so that CLI can use it.
Also temporarily moved one dep to
https://github.com/nextjournal/nbb-test-runner/pull/2 until the PR is
merged
2025-05-15 12:14:46 -04:00
Gabriel Horner
f34a8d57a0 fix: whitespace on advanced query string import
Fixes https://github.com/logseq/db-test/issues/193
2025-05-14 16:06:23 -04:00
Tienson Qin
61d5aa3605 Rename :logseq.property/parent to :logseq.property.class/extends 2025-05-14 17:56:35 +08:00
Tienson Qin
5e1a968530 Merge branch 'master' into feat/db 2025-05-13 18:22:20 +08:00
Gabriel Horner
4f65a00c68 refactor: rename :logseq.task properties to :logseq.property
These are the only built-in properties that don't start with
:logseq.property
2025-05-06 12:29:09 +08:00
Gabriel Horner
6a635ecb49 refactor: mv common ns out of db ns
Also move common and file fns out of db ns. Cleanup allows
for better separation linting
2025-04-28 10:37:13 -04:00