Commit Graph

359 Commits

Author SHA1 Message Date
Gabriel Horner
e7bfd53a7c fix: import of namespace tags
w/ 2+ levels of namespaces. This should fix
https://github.com/logseq/db-test/issues/679
2026-01-27 16:01:25 -05:00
Gabriel Horner
23d6f1538a fix: import of unlinked pdf annotations
This addresses the importer failing for
https://github.com/logseq/db-test/issues/732
2026-01-26 12:05:44 -05:00
Gabriel Horner
538d10a20a fix: rm whiteboard and tldraw from graph-parser
Also disable test block which is causing an intermittent ref issue
to investigate later
2026-01-22 13:19:58 -05:00
Gabriel Horner
05ffa3eeb6 chore: remove fns made unused by OG removal
Found unused ones in deps/common, from unused frontend.db aliases
and from fns that only existed for their corresponding tests
2026-01-22 13:19:58 -05:00
Gabriel Horner
4bb7430682 fix: rm the last file-based namespaces in db dep
Also remove the following related file schema
attributes and their dependent code:
:block/marker, :block/priority, :block/scheduled,
:block/deadline,:block/repeated?, :block/pre-block?,
:block/properties-order, :block/properties-text-values, :block/macros,
:block/invalid-properties
2026-01-22 13:19:58 -05:00
Gabriel Horner
ab137a962a fix: remove file graph imports
Remove mention of removed JSON import in descriptions. Also mv frontend.util.fs
to where they are used instead of keeping an outdated file ns
2026-01-22 13:19:58 -05:00
Gabriel Horner
37d9ebda1f chore: remove file-specific graph-parser code
Also removed file graph code for markdown export as a prereq
2026-01-22 13:19:57 -05:00
Gabriel Horner
2bcc331916 fix: remove gp-db ns, especially since it's unused
or incorrect in case of graph-view. Removed file graph
specific tests that depended on gp-db
2026-01-22 13:19:57 -05:00
Gabriel Horner
b91b892181 chore: remove unused file ns graph-parser.cli 2026-01-22 13:19:57 -05:00
Gabriel Horner
98ab051328 fix: importing journals with tags
Fixes https://github.com/logseq/db-test/issues/588
2026-01-20 10:24:19 -05:00
Gabriel Horner
045cd5f537 enhance: import block tags that are
specified via tags property. Fixes
https://github.com/logseq/db-test/issues/434
2025-12-18 13:35:42 -05:00
Gabriel Horner
b4f3e364e6 fix: duplication of exporter config leads to bugs
in script and test. Also fix <export-file not being waited on and new docs
after #12260
2025-12-15 12:15:48 -05:00
Tienson Qin
33db791ac0 fix: import Zotero pdf blocks (#12260)
fix: support zotero files when importing pdf annotations
2025-12-15 23:28:58 +08:00
Gabriel Horner
8fc56bfaa3 refactor: Update importer to use validate-local-db
Also update validation in DB scripts:
* replace alternative validate-db impl with standard CLI one
* Add validation to import script
* Choose to decouple other DB scripts from validate-db as it may go away
  later
2025-12-04 10:30:56 -05:00
Tienson Qin
e311e1b85c fix: don't create child tags when saving inline 2025-11-21 00:33:34 +08:00
Gabriel Horner
c91661f022 fix: import code block inside quote block
Fixes https://github.com/logseq/db-test/issues/523
2025-11-04 10:48:09 -05:00
Tienson Qin
248fdeb422 fix: tests 2025-09-29 17:37:04 -04:00
Tienson Qin
971cec54bc fix(import): remove asset byte-array usage to prevent OOM
This commit also combine read-asset and copy-asset to read-and-copy-asset
2025-09-29 17:37:04 -04:00
Tienson Qin
248d8c33f2 refactor: remove block/path-refs (#12081)
1. refactor: use :block/refs and has-ref rule instead of path-refs
2. remove block/path-refs
3. removes :block-parent rule since there's already :parent
4. enhance: skip pipeline calculation for rtc initial download tx
5. refactor(rtc): remove memoize

---------

Co-authored-by: rcmerci <rcmerci@gmail.com>
2025-09-03 20:33:33 +08:00
Gabriel Horner
a81274cd0d fix: import quoted math blocks
and lists in quote blocks. Addresses main issue in https://github.com/logseq/db-test/issues/434
2025-09-02 13:52:32 -04:00
Gabriel Horner
f4a8b9438e fix: block references to annotations
Don't rewrite annotation uuids as we don't do that for other existing blocks
2025-07-22 08:57:21 -04:00
Gabriel Horner
ad5c776f46 fix: doc files in assets shouldn't be extracted
Asset files that are edn or md were being treated like assets and
extracted as doc files. This was noticed because pdf annotation edn was being
needlessly extracted
2025-07-22 08:57:21 -04:00
Gabriel Horner
3f21c297b2 enhance: import markdown annotations
for pdfs and any children blocks under highlight. Only
imports text from md annotations
2025-07-22 08:57:21 -04:00
Gabriel Horner
4c9d063323 enhance: import pdf area annotations 2025-07-22 08:57:21 -04:00
Gabriel Horner
536f39cbbd enhance: import basic pdf text annotation
Only read edn for now. Works in UI and CLI.
Also correctly keeps annotation edn from being copied since
it's no longer used
2025-07-22 08:57:21 -04:00
Tienson Qin
9bede53a34 fix: tests 2025-06-29 15:21:09 +08:00
Gabriel Horner
42f0519797 fix: import property that changes from :node to :default
on same page :as :node property is defined. Also affects :date ->
:default. Fixes https://github.com/logseq/db-test/issues/340
2025-06-23 14:02:28 -04:00
Gabriel Horner
58d3abbafb fix: use ast blocks for simple queries conversions
Fixes cases where {{query nested in BEGIN_* blocks were incorrectly
being converted to queries. Also made ast block handling faster
by only doing one prewalk and cleaned up handling of queries
2025-06-23 19:04:03 +08:00
Gabriel Horner
cc05518ea5 fix: use ast blocks for advanced queries conversion
Fixes cases where BEGIN_QUERY nested in BEGIN_SRC were incorrectly
being converted to queries e.g. most queries in
https://docs.logseq.com/#/page/advanced%20queries
2025-06-23 19:04:03 +08:00
Gabriel Horner
507ff3ce29 enhance: import page and block embeds
One embed per block as that's what DB version supports.
Fixes https://github.com/logseq/db-test/issues/198
2025-06-23 19:04:03 +08:00
Gabriel Horner
fdff75cdf8 fix: import not handling scheduled and deadline property
on same block. Fixes https://github.com/logseq/db-test/issues/318
2025-06-20 17:06:52 -04:00
Gabriel Horner
8c6146a00d enhance: import markdown quotes
Went ahead and implemented ast block to string
as regex version of this would've been to buggy
2025-06-18 13:54:48 -04:00
Gabriel Horner
5b4f43d6f4 enhance: import quotes as #Quote
Can only support one quote per block so text around quotes
and multiple quotes aren't possible
Fixes https://github.com/logseq/db-test/issues/195
2025-06-18 13:54:48 -04:00
Tienson Qin
7b46076170 Merge branch 'master' into refactor/page-parent 2025-06-17 19:44:34 +08:00
Gabriel Horner
ace71b2b25 enhance: asset import supports multiple assets
per block and preserves text around assets.
These were both enabled by moving asset blocks to Asset page
2025-06-17 19:36:05 +08:00
Gabriel Horner
501063e625 fix: importing assets for more complex blocks like paragraphs with links
Used extract-blocks to group ast blocks for each block instead of
previous buggy version. Also renamed bindings that were all confusingly
`block`
2025-06-17 19:36:05 +08:00
Gabriel Horner
f318d68530 enhance: import assets handles
unused assets by still copying them by name and handles
existing assets
2025-06-17 19:36:05 +08:00
Gabriel Horner
bb31c4e6f1 fix: importing assets in subdirectories
Also fix lint
2025-06-17 19:36:05 +08:00
Gabriel Horner
4524fc8370 enhance: import asset metadata
Also handle multiple assets per block and displaying that to user
2025-06-17 19:36:05 +08:00
Gabriel Horner
90c7fc5775 enhance: UI imports assets as #Asset
Also made importer and db asset creation use same helper fns
2025-06-17 19:36:05 +08:00
Gabriel Horner
38734bde40 enhance: CLI imports assets as #Asset
Had to split assets into two steps now that asset naming is dependent
on uuids. Also add explicit asserts in exporter to catch silent
failure in tests
2025-06-17 19:36:05 +08:00
Tienson Qin
92ec65d61d Merge branch 'master' into refactor/page-parent 2025-06-15 08:27:33 +08:00
Tienson Qin
88216d2992 fix: able to convert an existing page to a property (#11940)
* enhance: use page as property

* fix: ensures block/uuid doesn't change after converting to property
2025-06-14 13:17:33 +08:00
Gabriel Horner
7cc9ddc26b chore: bump docs graph being tested
Hadn't been updated in over a year
2025-06-13 10:43:42 -04:00
Tienson Qin
d7f0c248ff fix: bring back :ref/default-open-blocks-level 2025-06-02 11:19:10 +08:00
Tienson Qin
33b6433dc9 wip: move file parse to worker 2025-05-29 00:39:51 +08:00
Gabriel Horner
471889729e fix: graphs with existing Library page are imported
incorrectly and merged with new built-in page when :property-classes
option is used
2025-05-21 14:31:12 -04:00
Tienson Qin
6c12676a08 fix: check pages only 2025-05-17 05:46:23 +08:00
Tienson Qin
42163df4b3 Merge branch 'feat/db' into refactor/page-parent 2025-05-17 05:36:58 +08:00
Tienson Qin
7dbb3eb3bd wip: fix graph-parser tests 2025-05-17 02:49:42 +08:00