Commit Graph

922 Commits

Author SHA1 Message Date
Tienson Qin
b2497c0cc5 fix: pages starting with "@" fail to import from markdown
related to https://github.com/logseq/db-test/issues/680
2025-12-24 22:30:10 +08:00
Tienson Qin
1ae9e2c496 fix: extends tags query and support eid|title|db-ident
Previously `tags` doesn't return blocks that have children tag(s).

Also, `tags` support both db/ident and block/title to make it easy
to use:
(tags ?b #{144})
(tags ?b #{"Task"})
(tags ?b #{:logseq.class/Task})
2025-12-24 14:37:57 +08:00
Gabriel Horner
045cd5f537 enhance: import block tags that are
specified via tags property. Fixes
https://github.com/logseq/db-test/issues/434
2025-12-18 13:35:42 -05:00
Gabriel Horner
b4f3e364e6 fix: duplication of exporter config leads to bugs
in script and test. Also fix <export-file not being waited on and new docs
after #12260
2025-12-15 12:15:48 -05:00
Tienson Qin
33db791ac0 fix: import Zotero pdf blocks (#12260)
fix: support zotero files when importing pdf annotations
2025-12-15 23:28:58 +08:00
Tienson Qin
270f13f11a fix: support external pdf files when importing from file graphs 2025-12-15 23:02:33 +08:00
Tienson Qin
81cbc1e701 fix: pages don't have #Page tag when importing from markdown files
related to https://github.com/logseq/db-test/issues/641
2025-12-15 14:49:17 +08:00
Tienson Qin
f594e2034f fix(rtc): batch store and validate db (#12249)
* fix(rtc): batch store and validate db

* fix: logseq.db/transact! shouldn't distinct tx-data

since move-op include 2 steps:
1. insert-block
2. update-attrs

This results in db invalid after step 1.

* refactor: add transact-with-temp-conn!

* bump nbb-logseq and add tests for ldb/transact* fns

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: rcmerci <rcmerci@gmail.com>
2025-12-09 20:47:37 +08:00
Gabriel Horner
8fc56bfaa3 refactor: Update importer to use validate-local-db
Also update validation in DB scripts:
* replace alternative validate-db impl with standard CLI one
* Add validation to import script
* Choose to decouple other DB scripts from validate-db as it may go away
  later
2025-12-04 10:30:56 -05:00
Gabriel Horner
f8064253a2 fix: remove misplaced ignore 2025-12-02 10:11:11 -05:00
Tienson Qin
38baa56286 fix: lint 2025-11-26 18:36:16 +08:00
Tienson Qin
73d4ee7caa Merge branch 'master' into enhance/ios-native-navigation 2025-11-24 22:36:50 +08:00
charlie
49407d561a refactor(assets): rename local-asset? to local-relative-asset? and update references 2025-11-21 18:02:06 +08:00
Tienson Qin
a18626283b fix: graph parser tests 2025-11-21 03:09:25 +08:00
Tienson Qin
055a5cf6ca fix: parse inline tag as page 2025-11-21 01:46:35 +08:00
Gabriel Horner
43b09095ac fix: graph-parser lint 2025-11-20 12:13:04 -05:00
Tienson Qin
c488335f9d fix: typo 2025-11-21 01:09:13 +08:00
Tienson Qin
20aecdb27c fix: inline tag resolved as page when saving block 2025-11-21 00:52:43 +08:00
Tienson Qin
e311e1b85c fix: don't create child tags when saving inline 2025-11-21 00:33:34 +08:00
Gabriel Horner
c91661f022 fix: import code block inside quote block
Fixes https://github.com/logseq/db-test/issues/523
2025-11-04 10:48:09 -05:00
rcmerci
6ffa6ab5d6 enhance: upgrade com.lambdaisland/glogi, add timestamp to log 2025-10-31 18:04:37 +08:00
Tienson Qin
ace82d7151 fix: don't create new card page when importing file graph
fix https://github.com/logseq/db-test/issues/521
2025-10-30 01:34:43 +08:00
Tienson Qin
7dc155b28b fix: file graph import 2025-10-29 21:28:27 +08:00
Gabriel Horner
2500fe888a fix: dev import can't import
All CLI imports were failing with "Maximum call stack size exceeded".
2025-10-22 10:37:25 -04:00
Tienson Qin
999618a406 refactor: compute pipeline tx-data before d/transact! 2025-10-15 15:31:42 +08:00
Gabriel Horner
27d5a369fd fix: bump to latest nbb-logseq to fix most logseq.db dependent ns 2025-10-13 21:34:00 -04:00
Tienson Qin
7ed7fb4f31 enhance: prefer ldb/transact! than d/transact! 2025-10-13 19:03:11 +08:00
Gabriel Horner
846f9307c3 enhance: notify users when assets exceed max size
so they are aware why asset was ignored. Also if asset ids are not
present we should fail loudly like we everywhere else in the importer.
Users should know when data is missing instead of hiding it in the
console. Also refactor confusing usage of :block/uuid to :asset-created?
as it was only used to track the first time an asset is built to be
transacted
2025-09-29 17:37:04 -04:00
Tienson Qin
91657bfcce fix: rpath may not be provided 2025-09-29 17:37:04 -04:00
Tienson Qin
248fdeb422 fix: tests 2025-09-29 17:37:04 -04:00
Tienson Qin
971cec54bc fix(import): remove asset byte-array usage to prevent OOM
This commit also combine read-asset and copy-asset to read-and-copy-asset
2025-09-29 17:37:04 -04:00
Gabriel Horner
f93f889f8a chore: bump to latest nbb w/ datascript update 2025-09-03 10:00:45 -04:00
Tienson Qin
248d8c33f2 refactor: remove block/path-refs (#12081)
1. refactor: use :block/refs and has-ref rule instead of path-refs
2. remove block/path-refs
3. removes :block-parent rule since there's already :parent
4. enhance: skip pipeline calculation for rtc initial download tx
5. refactor(rtc): remove memoize

---------

Co-authored-by: rcmerci <rcmerci@gmail.com>
2025-09-03 20:33:33 +08:00
Gabriel Horner
a81274cd0d fix: import quoted math blocks
and lists in quote blocks. Addresses main issue in https://github.com/logseq/db-test/issues/434
2025-09-02 13:52:32 -04:00
Gabriel Horner
6a84e56984 enhance: working spike for export md command
export namespaces aren't reused yet. Fixed bug w/ get-block-and-children
not invoking correctly in nbb. Bumped all deps to use latest nbb-logseq
2025-08-26 11:27:34 -04:00
Gabriel Horner
707f49bc4f fix: nbb-logseq bundling and quirky datascript requires (#12032)
* fix: nbb-logseq bundling and quirky datascript requires

* fix: bump nbb-logseq for node version incompatibility
2025-08-06 13:16:50 -04:00
Gabriel Horner
6ef1c62e95 chore: mv sanitize-filename into exporter
should fix outliner CI as it never loads exporter
2025-07-22 12:24:14 -04:00
Gabriel Horner
5de565dcce fix: some pdf annotations weren't imported
. Creation of edn+md paths from pdf path were lacking sanitization which
resulted in edn and md files not being associated with pdf

Fixes https://github.com/logseq/db-test/issues/196#issuecomment-3059363947.
Also fix incorrect dep
2025-07-22 12:24:14 -04:00
Gabriel Horner
6b3240f4b5 fix: imported pdf annotation image not showing up
in pdf viewer under Outline > Highlights
2025-07-22 08:57:21 -04:00
Gabriel Horner
f4a8b9438e fix: block references to annotations
Don't rewrite annotation uuids as we don't do that for other existing blocks
2025-07-22 08:57:21 -04:00
Gabriel Horner
ad5c776f46 fix: doc files in assets shouldn't be extracted
Asset files that are edn or md were being treated like assets and
extracted as doc files. This was noticed because pdf annotation edn was being
needlessly extracted
2025-07-22 08:57:21 -04:00
Gabriel Horner
3f21c297b2 enhance: import markdown annotations
for pdfs and any children blocks under highlight. Only
imports text from md annotations
2025-07-22 08:57:21 -04:00
Gabriel Horner
4c9d063323 enhance: import pdf area annotations 2025-07-22 08:57:21 -04:00
Gabriel Horner
536f39cbbd enhance: import basic pdf text annotation
Only read edn for now. Works in UI and CLI.
Also correctly keeps annotation edn from being copied since
it's no longer used
2025-07-22 08:57:21 -04:00
Gabriel Horner
65640ed885 fix: macro file graph properties are undeclared
Looks like we forgot to do this when we brought them back in
ac31842cec
2025-07-14 17:58:42 -04:00
Tienson Qin
9bede53a34 fix: tests 2025-06-29 15:21:09 +08:00
Gabriel Horner
7968058d4b fix: asset links with regex characters not getting updated
Fixes https://github.com/logseq/db-test/issues/358
2025-06-27 10:51:18 -04:00
Gabriel Horner
42f0519797 fix: import property that changes from :node to :default
on same page :as :node property is defined. Also affects :date ->
:default. Fixes https://github.com/logseq/db-test/issues/340
2025-06-23 14:02:28 -04:00
Gabriel Horner
957eab73d8 fix: import fails on query sort with built-in properties
Fix for https://github.com/logseq/db-test/issues/341
2025-06-23 09:17:26 -04:00
Gabriel Horner
58d3abbafb fix: use ast blocks for simple queries conversions
Fixes cases where {{query nested in BEGIN_* blocks were incorrectly
being converted to queries. Also made ast block handling faster
by only doing one prewalk and cleaned up handling of queries
2025-06-23 19:04:03 +08:00