Gabriel Horner
b4f3e364e6
fix: duplication of exporter config leads to bugs
...
in script and test. Also fix <export-file not being waited on and new docs
after #12260
2025-12-15 12:15:48 -05:00
Gabriel Horner
8fc56bfaa3
refactor: Update importer to use validate-local-db
...
Also update validation in DB scripts:
* replace alternative validate-db impl with standard CLI one
* Add validation to import script
* Choose to decouple other DB scripts from validate-db as it may go away
later
2025-12-04 10:30:56 -05:00
Gabriel Horner
2500fe888a
fix: dev import can't import
...
All CLI imports were failing with "Maximum call stack size exceeded".
2025-10-22 10:37:25 -04:00
Tienson Qin
999618a406
refactor: compute pipeline tx-data before d/transact!
2025-10-15 15:31:42 +08:00
Tienson Qin
7ed7fb4f31
enhance: prefer ldb/transact! than d/transact!
2025-10-13 19:03:11 +08:00
Gabriel Horner
846f9307c3
enhance: notify users when assets exceed max size
...
so they are aware why asset was ignored. Also if asset ids are not
present we should fail loudly like we everywhere else in the importer.
Users should know when data is missing instead of hiding it in the
console. Also refactor confusing usage of :block/uuid to :asset-created?
as it was only used to track the first time an asset is built to be
transacted
2025-09-29 17:37:04 -04:00
Tienson Qin
971cec54bc
fix(import): remove asset byte-array usage to prevent OOM
...
This commit also combine read-asset and copy-asset to read-and-copy-asset
2025-09-29 17:37:04 -04:00
Gabriel Horner
536f39cbbd
enhance: import basic pdf text annotation
...
Only read edn for now. Works in UI and CLI.
Also correctly keeps annotation edn from being copied since
it's no longer used
2025-07-22 08:57:21 -04:00
Tienson Qin
7b46076170
Merge branch 'master' into refactor/page-parent
2025-06-17 19:44:34 +08:00
Gabriel Horner
f318d68530
enhance: import assets handles
...
unused assets by still copying them by name and handles
existing assets
2025-06-17 19:36:05 +08:00
Gabriel Horner
bb31c4e6f1
fix: importing assets in subdirectories
...
Also fix lint
2025-06-17 19:36:05 +08:00
Gabriel Horner
4524fc8370
enhance: import asset metadata
...
Also handle multiple assets per block and displaying that to user
2025-06-17 19:36:05 +08:00
Gabriel Horner
90c7fc5775
enhance: UI imports assets as #Asset
...
Also made importer and db asset creation use same helper fns
2025-06-17 19:36:05 +08:00
Gabriel Horner
38734bde40
enhance: CLI imports assets as #Asset
...
Had to split assets into two steps now that asset naming is dependent
on uuids. Also add explicit asserts in exporter to catch silent
failure in tests
2025-06-17 19:36:05 +08:00
Tienson Qin
92ec65d61d
Merge branch 'master' into refactor/page-parent
2025-06-15 08:27:33 +08:00
Gabriel Horner
3fa32ae990
fix(dev): CLI import not copying assets to correct location
2025-06-09 12:38:23 -04:00
Gabriel Horner
38e05aa6b8
enhance: new graph from script saves git sha in :kv/value
...
Also for
https://test.logseq.com/#/page/6835e455-69f6-4bbd-9865-b76fa57ce457
2025-05-27 12:16:15 -04:00
Gabriel Horner
3fe790d4d7
enhance(dev): scripts can read or write db graphs as full path files
...
This allows for easier use of debugging graphs as they are exported by
users as a file. Also DRYed up duplicated helper
2025-05-23 13:59:02 -04:00
Gabriel Horner
fecb42f49d
fix(dev): deps scripts not handling relative and absolute paths for graphs
...
transact and create didn't handle relative paths. All scripts except
import didn't handle absolute paths
2025-03-17 11:09:44 -04:00
Gabriel Horner
7917afaa45
chore: update nbb scripts to be repl friendly
...
With https://github.com/babashka/nbb/issues/95 done, this allows
all nbb script namespaces to required or used from a repl
2025-03-07 11:49:28 -05:00
Tienson Qin
926d05c185
feat: table columns pinning ( #11693 )
...
* Add property :logseq.property.table/pinned-columns
* feat: table column pinning
* enhance: hide :id column by default
---------
Co-authored-by: charlie <xyhp915@qq.com >
2025-01-24 11:17:09 +08:00
Gabriel Horner
6c58af2605
enhance(dev): importing cli debug option prints all tx
...
related to a failing path. Helpfully provides more context
and also doesn't misleadingly print tx when it's not related to an
import failure with another path
2025-01-23 16:15:53 -05:00
Tienson Qin
2e12918d5c
Merge branch 'feat/db' into refactor/block-schema
2025-01-14 14:17:24 +08:00
Gabriel Horner
63eaacdbe1
enhance(dev): import cli and tests print original error's ex-data
2025-01-13 13:27:39 -05:00
Gabriel Horner
1ab621069d
enhance: importer displays one notification about ignored files
...
rather than one notification per file. Also ignore pdf highlight pages
as user graphs shouldn't fail hard on features that aren't imported yet.
This allowed the user graph in
https://github.com/logseq/db-test/issues/191 to import without errors
2025-01-13 11:44:02 -05:00
Tienson Qin
e318e0d6c1
Merge branch 'feat/db' into refactor/block-schema
2025-01-12 16:48:19 +08:00
Gabriel Horner
2782eea315
enhance(dev): CLI import prints last tx
...
since most import failures are due to a failing tx
2025-01-10 15:22:37 -05:00
Gabriel Horner
8e81d7d855
enhance(dev): task to import multiple file graphs easily
...
Also add a --continue option to see all errors on a CLI import
2024-11-21 10:30:58 -05:00
Gabriel Horner
5541205674
enhance: default to importing all tags
...
Fixes LOG-3235. Still default to not importing all tags in tests and
scripts to simplify their workflows
2024-11-20 13:31:42 -05:00
Gabriel Horner
3154b849dd
enhance(dev): add latest import options to script
2024-11-12 10:56:05 -05:00
Gabriel Horner
0fcbe9fe89
fix(dev): import script's user options
2024-11-06 15:22:45 -05:00
Gabriel Horner
8b9d2a601e
enhance(dev): print ignored properties for CLI import
2024-09-12 17:25:42 -04:00
Gabriel Horner
4b365da70c
fix: display more helpful notifications for invalid
...
properties and classes that can't build a db-ident. Part of LOG-3176
2024-07-24 16:35:05 -04:00
Gabriel Horner
33785dca66
fix: db-import of :query-sort-by property
...
Also fix warning notifications printing empty error logs.
Part of LOG-3176
2024-07-11 18:00:21 -04:00
Gabriel Horner
f0ecf03c4a
fix: db import of tagged page with pageTags
...
Part of LOG-3176
2024-07-02 16:50:00 -04:00
Gabriel Horner
9ee09372aa
fix: cleanup inconsistent and incorrect import error handling
...
Some notified errors didn't include error object. Import cli didn't fail
on failure and stack was reversed. Error notifications sometimes
included full error which makes errors unreadable in CLI and possibly
confusing for users. Also fixed missing classpath. Part of
LOG-3176
2024-06-28 11:16:22 -04:00
Gabriel Horner
e619874dc4
enhance: add acceptance test for file to db graph import
...
Part of LOG-3092
2024-06-26 15:59:09 -04:00
Gabriel Horner
5520bc1d33
fix: import single file from CLI didn't print error
...
Also improved CLI error reporting so sci stack is readable and
not filled with stack fn docstrings
2024-06-26 15:59:09 -04:00
Gabriel Horner
95db811f55
fix: frontend tests shouldn't depend on sqlite3
...
frontend.test.helper was accidentally coupled to sqlite3 via
logseq.outliner.db-pipeline. Split out sqlite3 dependent fns to
logseq.outliner.cli so it's clear this shouldn't be coupled to frontend
2024-06-14 14:41:24 -04:00
Gabriel Horner
0e1ada2ef6
enhance: bb task for creating graphs from EDN file
...
Converted inferred graph to an EDN file now that this task exists. Also
merge last of tasks.create-graph to relevant ns so that external users
can also create such tasks
2024-06-11 10:09:37 -04:00
Gabriel Horner
e08e52acd3
enhance: db graph importer supports optional parent class property
2024-03-29 10:37:09 -04:00
Gabriel Horner
345fd96e2a
enhance: full file graph import works from CLI
...
Also cleaned up naming, added docstrings and fixed a bug that
failed import hard when user didn't input any tag-classes or
property-classes
2024-03-13 16:23:49 -04:00
Gabriel Horner
2f599919fb
fix: favorites not importing
...
Also made underlying fns reusable from UI and nbb compatible.
Fixes LOG-3031
2024-03-12 17:57:47 -04:00
Gabriel Horner
09816b6427
enhance: assets can be imported on CLI
...
Also add individual asset error handling, like is done for doc files
2024-03-05 17:05:18 -05:00
Gabriel Horner
9669755cab
enhance: detect and add properties to imported classes
...
fixes LOG-3069
2024-03-04 23:05:06 -05:00
Gabriel Horner
0b0299b28f
fix: more import fixes after testing more graphs
...
- collapsed:: false was failing an import
- ignore more attributes for existing pages
- also fixed import script not handling macros and absolute paths
correctly
2024-02-29 15:06:49 -05:00
Gabriel Horner
49cc17dea5
fix: update property values for any type to :default change
...
that occur within same file. Also made importing behavior reproducible
across runs and made import script script less verbose by default.
Part of LOG-2985
2024-02-29 13:01:09 -05:00
Gabriel Horner
b32eb7e9be
enhance: automatically handle properties that change to :default
...
docs graph :platforms property now imports correctly rather than
ignored 30 times. Also allow import script to import multiple files
for testing purposes. Part of LOG-2985
2024-02-28 17:16:09 -05:00
Gabriel Horner
44d60808c1
fix: correctly detect property type
...
for property values containing text and refs.
Also allow import script to only import a file for testing.
Part of LOG-2985
2024-02-27 16:27:26 -05:00
Gabriel Horner
13439ed699
enhance: port two more import steps to be script compatible
...
also fix save-file! not waiting on async transact before post-save
2024-02-27 12:07:25 -05:00