Commit Graph

616 Commits

Author SHA1 Message Date
Gabriel Horner
b34325be77 fix: two bugs with updating tags during import
fixes LOG-3073
2024-03-01 15:51:19 -05:00
Gabriel Horner
487c86ac83 enhance: import float values with type :number
They were imported as :default b/c graph-parser didn't handle them.
Part of LOG-2985
2024-03-01 12:07:45 -05:00
Gabriel Horner
0b0299b28f fix: more import fixes after testing more graphs
- collapsed:: false was failing an import
- ignore more attributes for existing pages
- also fixed import script not handling macros and absolute paths
  correctly
2024-02-29 15:06:49 -05:00
Gabriel Horner
49cc17dea5 fix: update property values for any type to :default change
that occur within same file. Also made importing behavior reproducible
across runs and made import script script less verbose by default.
Part of LOG-2985
2024-02-29 13:01:09 -05:00
Gabriel Horner
31b344d3c1 enhance: ignore blank property values from templates
to improves property type detection. Also fixed a bug in
upstream-properties detection and added more docs + better names related
to it. Part of LOG-2985
2024-02-29 11:18:18 -05:00
Gabriel Horner
b32eb7e9be enhance: automatically handle properties that change to :default
docs graph :platforms property now imports correctly rather than
ignored 30 times. Also allow import script to import multiple files
for testing purposes. Part of LOG-2985
2024-02-28 17:16:09 -05:00
Gabriel Horner
9569bed720 fix: property value with mixed refs not imported
correctly. Also fixed lints
2024-02-28 11:08:53 -05:00
Gabriel Horner
44d60808c1 fix: correctly detect property type
for property values containing text and refs.
Also allow import script to only import a file for testing.
Part of LOG-2985
2024-02-27 16:27:26 -05:00
Gabriel Horner
13439ed699 enhance: port two more import steps to be script compatible
also fix save-file! not waiting on async transact before post-save
2024-02-27 12:07:25 -05:00
Gabriel Horner
3e3e9a303b fix graph-parser lints 2024-02-26 16:30:12 -05:00
Gabriel Horner
e3cc4bad36 enhance: speed up import of doc files and make them scriptable
Refactored the largest part of the import to used promesa which made it
nbb compatible and made large imports like docs 10-25% faster.  Also
updated db-import script to use this work
2024-02-26 16:20:03 -05:00
Gabriel Horner
d8f41f22c2 fix: class page schemas have a new optional attribute 2024-02-26 11:11:01 -05:00
Tienson Qin
b1aae957e0 fix: esc doesn't save tags 2024-02-24 16:21:03 +08:00
Gabriel Horner
02ba0b3c87 enhance: configurable logging and user notification during import
Don't need to sprinkle prn's anymore or hide errors that shouldn't
be happening while developing
2024-02-24 00:55:14 -05:00
Gabriel Horner
a61ee21e25 enhance: add basic script to test db graph import
also tweaked query script to work with db graphs by relative path
2024-02-24 00:42:46 -05:00
Gabriel Horner
356657d36b fix: adding uuids on imported classes causes multiple bugs
Caused invalid blocks as properties that were on the same block
as new class had invalid uuids. Also caused content to have
incorrect mangled uuids. Part of LOG-2985
2024-02-23 15:23:47 -05:00
Gabriel Horner
eb994662a6 enhance: user can choose properties' values to import as classes
The user specified properties are then no longer needed since the
concept is built-in. Part of LOG-2985
2024-02-23 12:42:16 -05:00
Gabriel Horner
dad848fa73 enhance: improve property type detection
with macros. This improves :url property detection in docs
graph and other similar graphs. part of LOG-2985
2024-02-22 16:00:03 -05:00
Gabriel Horner
531a8ef533 enhance: import :filters and helpfully ignore :icon
Better to ignore :icon and explain to user than import them as broken with no explanation.
part of LOG-2985
2024-02-22 10:12:09 -05:00
Gabriel Horner
37135fd242 enhance: import tags for existing pages - LOG-2985
also fixed namespaces and aliases missing from existing pages
2024-02-21 17:16:21 -05:00
Gabriel Horner
a010298fc6 enhance: handle invalid property value changes
Ignore invalid property values and notify user, rather than
import them as broken. Part of LOG-2985
2024-02-21 15:30:27 -05:00
Gabriel Horner
95a43e24a3 clean up property importing - part of LOG-2985
- infer block property values by properties, not refs
- remove unnecessary property-changes state
- centralize property-change logging and handling
2024-02-21 11:45:40 -05:00
Gabriel Horner
9591c51997 enhance: import properties of existing pages
including property pages. part of LOG-2985
2024-02-20 23:02:44 -05:00
Gabriel Horner
ddefb677f2 enhance: import most page properties - part of LOG-2985
Also fix validation and bugs with built-in props. Add additional info
on parent-left conflicts as I encountered a couple while testing
pre-block removal
2024-02-20 14:50:49 -05:00
Gabriel Horner
4f3c4c9e03 fix: handle a property :default to :page change
Part of LOG-2985. :page property values would just be references so
use properties-text-values over `str`. Also added import timing
2024-02-19 11:34:42 -05:00
Gabriel Horner
7734bc8791 enhance: import can detect property type changes
and handle auto-converting a value changing from :default to any prop type
2024-02-17 00:24:50 -05:00
Gabriel Horner
04e5c79ee0 fix: invalid property values by unsupported template blocks
Correctly ignore template properties until it's supported
2024-02-16 12:36:27 -05:00
Gabriel Horner
3bf7ee5645 enhance: import support for :page and infer :date properties 2024-02-16 11:36:06 -05:00
Gabriel Horner
0bba0a2ca9 enhance: import built-in logseq.* props - part of LOG-2977 2024-02-15 15:01:01 -05:00
Gabriel Horner
18d3c87016 enhance: support built-in property imports. part of LOG-2985
:query-properties and :query-sort-by need translating. Don't
override block/schema of built-in properties. Also display
property closed values in block data cmd
2024-02-15 12:27:36 -05:00
Gabriel Horner
3b745ac035 enhance: first pass at importing block properties
part of LOG-2985. Still somethings failing for docs graph
2024-02-14 17:16:36 -05:00
Gabriel Horner
2ff0cc1702 enhance: import whiteboards
Fixes LOG-2981. Also start setting up block properties for LOG-2985
2024-02-13 15:41:56 -05:00
Gabriel Horner
48b974d64e enhance: all page refs in imported content are special ids
close LOG-3055. Also fix graph-parser errors being ignored
2024-02-12 22:02:54 -05:00
Gabriel Horner
6dc1272bf2 fix: lint and remove unused ast in file graph exporter
Also remove unused options on d/transact!
2024-02-12 15:52:52 -05:00
Gabriel Horner
06ad57efff fix: page tags causing invalid properties on larger graphs
Tags that had their own pages were resulting in invalid properties. This
is because we were re-transacting existing pages as new pages
with new uuids which resulted in invalid properties. Fixes LOG-2984.
Also moved graph-parser export (import from db perspective) to its own
ns
2024-02-12 12:33:13 -05:00
Gabriel Horner
26bc6f0d1d chore: mv logseq.graph-parser.util.db to db dep
This ns is more appropriate in db and was only in graph-parser because
the date-time-util lib wasn't accessible to db until #10933 landed.
graph-parser dep is specific to file graphs and as much as possible
anything unrelated to this should not be in this dep
2024-02-09 16:16:33 -05:00
Tienson Qin
62b90b7ae7 Merge branch 'feat/db' into perf/lazy-load-data 2024-02-09 09:28:36 +08:00
Gabriel Horner
dab6b5aff4 enhance: add support for importing page and block tags
Allows users to specify which tags are tag classes and
the rest are just normal pages. Fixes LOG-2984
2024-02-08 16:49:27 -05:00
Tienson Qin
6f90958e4b Merge branch 'feat/db' into perf/lazy-load-data 2024-02-03 12:52:35 +08:00
Gabriel Horner
ca752834c1 enhance: basic block tags for import - part of LOG-2984 2024-02-02 17:39:08 -05:00
Gabriel Horner
b6a952fd7d fix: assets not copied for import
Caused by a commit yesterday. Also removed unused
import-file-to-db-graph options
2024-02-02 16:16:18 -05:00
Gabriel Horner
5b4bfd9089 enhance: file graph import ignores hidden files 2024-01-31 15:01:39 -05:00
Tienson Qin
c1a15c0708 Merge branch 'feat/db' into perf/lazy-load-data 2024-02-01 03:58:04 +08:00
Gabriel Horner
bf56cd3252 chore: move legacy fns to legacy ns
title property is for file graphs and should be kept with its ns
to avoid confusion
2024-01-31 11:11:14 -05:00
Tienson Qin
99b6d543f0 fix: more async queries and export to roam 2024-01-30 20:03:06 +08:00
Tienson Qin
3addb5905b perf: load partial data when loading a graph
Instead of all the datoms
2024-01-29 13:17:37 +08:00
Gabriel Horner
77bc1d5d6b chore: update better-sqlite3 and mldoc in deps to match app 2024-01-23 11:36:32 -05:00
Gabriel Horner
689cf34cc3 fix: lints and a test
nbb test was failing b/c outliner had graph-parser as a dependency
but we hadn't manually added the npm dependency
2024-01-23 09:38:16 -05:00
Gabriel Horner
f573b8e821 fix: file graph import not persisting to disk 2024-01-17 12:57:16 -05:00
Gabriel Horner
0168ef0082 fix: 2 bugs with file import
These were silently failing and causing a number of pages to
not import
2024-01-16 17:01:23 -05:00