Commit Graph

402 Commits

Author SHA1 Message Date
Gabriel Horner
a61ee21e25 enhance: add basic script to test db graph import
also tweaked query script to work with db graphs by relative path
2024-02-24 00:42:46 -05:00
Gabriel Horner
356657d36b fix: adding uuids on imported classes causes multiple bugs
Caused invalid blocks as properties that were on the same block
as new class had invalid uuids. Also caused content to have
incorrect mangled uuids. Part of LOG-2985
2024-02-23 15:23:47 -05:00
Gabriel Horner
eb994662a6 enhance: user can choose properties' values to import as classes
The user specified properties are then no longer needed since the
concept is built-in. Part of LOG-2985
2024-02-23 12:42:16 -05:00
Gabriel Horner
dad848fa73 enhance: improve property type detection
with macros. This improves :url property detection in docs
graph and other similar graphs. part of LOG-2985
2024-02-22 16:00:03 -05:00
Gabriel Horner
531a8ef533 enhance: import :filters and helpfully ignore :icon
Better to ignore :icon and explain to user than import them as broken with no explanation.
part of LOG-2985
2024-02-22 10:12:09 -05:00
Gabriel Horner
37135fd242 enhance: import tags for existing pages - LOG-2985
also fixed namespaces and aliases missing from existing pages
2024-02-21 17:16:21 -05:00
Gabriel Horner
a010298fc6 enhance: handle invalid property value changes
Ignore invalid property values and notify user, rather than
import them as broken. Part of LOG-2985
2024-02-21 15:30:27 -05:00
Gabriel Horner
95a43e24a3 clean up property importing - part of LOG-2985
- infer block property values by properties, not refs
- remove unnecessary property-changes state
- centralize property-change logging and handling
2024-02-21 11:45:40 -05:00
Gabriel Horner
9591c51997 enhance: import properties of existing pages
including property pages. part of LOG-2985
2024-02-20 23:02:44 -05:00
Gabriel Horner
ddefb677f2 enhance: import most page properties - part of LOG-2985
Also fix validation and bugs with built-in props. Add additional info
on parent-left conflicts as I encountered a couple while testing
pre-block removal
2024-02-20 14:50:49 -05:00
Gabriel Horner
4f3c4c9e03 fix: handle a property :default to :page change
Part of LOG-2985. :page property values would just be references so
use properties-text-values over `str`. Also added import timing
2024-02-19 11:34:42 -05:00
Gabriel Horner
7734bc8791 enhance: import can detect property type changes
and handle auto-converting a value changing from :default to any prop type
2024-02-17 00:24:50 -05:00
Gabriel Horner
04e5c79ee0 fix: invalid property values by unsupported template blocks
Correctly ignore template properties until it's supported
2024-02-16 12:36:27 -05:00
Gabriel Horner
3bf7ee5645 enhance: import support for :page and infer :date properties 2024-02-16 11:36:06 -05:00
Gabriel Horner
0bba0a2ca9 enhance: import built-in logseq.* props - part of LOG-2977 2024-02-15 15:01:01 -05:00
Gabriel Horner
18d3c87016 enhance: support built-in property imports. part of LOG-2985
:query-properties and :query-sort-by need translating. Don't
override block/schema of built-in properties. Also display
property closed values in block data cmd
2024-02-15 12:27:36 -05:00
Gabriel Horner
3b745ac035 enhance: first pass at importing block properties
part of LOG-2985. Still somethings failing for docs graph
2024-02-14 17:16:36 -05:00
Gabriel Horner
2ff0cc1702 enhance: import whiteboards
Fixes LOG-2981. Also start setting up block properties for LOG-2985
2024-02-13 15:41:56 -05:00
Gabriel Horner
48b974d64e enhance: all page refs in imported content are special ids
close LOG-3055. Also fix graph-parser errors being ignored
2024-02-12 22:02:54 -05:00
Gabriel Horner
6dc1272bf2 fix: lint and remove unused ast in file graph exporter
Also remove unused options on d/transact!
2024-02-12 15:52:52 -05:00
Gabriel Horner
06ad57efff fix: page tags causing invalid properties on larger graphs
Tags that had their own pages were resulting in invalid properties. This
is because we were re-transacting existing pages as new pages
with new uuids which resulted in invalid properties. Fixes LOG-2984.
Also moved graph-parser export (import from db perspective) to its own
ns
2024-02-12 12:33:13 -05:00
Gabriel Horner
26bc6f0d1d chore: mv logseq.graph-parser.util.db to db dep
This ns is more appropriate in db and was only in graph-parser because
the date-time-util lib wasn't accessible to db until #10933 landed.
graph-parser dep is specific to file graphs and as much as possible
anything unrelated to this should not be in this dep
2024-02-09 16:16:33 -05:00
Tienson Qin
62b90b7ae7 Merge branch 'feat/db' into perf/lazy-load-data 2024-02-09 09:28:36 +08:00
Gabriel Horner
dab6b5aff4 enhance: add support for importing page and block tags
Allows users to specify which tags are tag classes and
the rest are just normal pages. Fixes LOG-2984
2024-02-08 16:49:27 -05:00
Tienson Qin
6f90958e4b Merge branch 'feat/db' into perf/lazy-load-data 2024-02-03 12:52:35 +08:00
Gabriel Horner
ca752834c1 enhance: basic block tags for import - part of LOG-2984 2024-02-02 17:39:08 -05:00
Gabriel Horner
b6a952fd7d fix: assets not copied for import
Caused by a commit yesterday. Also removed unused
import-file-to-db-graph options
2024-02-02 16:16:18 -05:00
Gabriel Horner
5b4bfd9089 enhance: file graph import ignores hidden files 2024-01-31 15:01:39 -05:00
Tienson Qin
c1a15c0708 Merge branch 'feat/db' into perf/lazy-load-data 2024-02-01 03:58:04 +08:00
Gabriel Horner
bf56cd3252 chore: move legacy fns to legacy ns
title property is for file graphs and should be kept with its ns
to avoid confusion
2024-01-31 11:11:14 -05:00
Tienson Qin
99b6d543f0 fix: more async queries and export to roam 2024-01-30 20:03:06 +08:00
Tienson Qin
3addb5905b perf: load partial data when loading a graph
Instead of all the datoms
2024-01-29 13:17:37 +08:00
Gabriel Horner
77bc1d5d6b chore: update better-sqlite3 and mldoc in deps to match app 2024-01-23 11:36:32 -05:00
Gabriel Horner
689cf34cc3 fix: lints and a test
nbb test was failing b/c outliner had graph-parser as a dependency
but we hadn't manually added the npm dependency
2024-01-23 09:38:16 -05:00
Gabriel Horner
f573b8e821 fix: file graph import not persisting to disk 2024-01-17 12:57:16 -05:00
Gabriel Horner
0168ef0082 fix: 2 bugs with file import
These were silently failing and causing a number of pages to
not import
2024-01-16 17:01:23 -05:00
Tienson Qin
cea105cc34 fix: wrong ns import 2024-01-16 16:01:18 +08:00
Tienson Qin
a619850998 Merge branch 'feat/db' into refactor/pipeline-worker 2024-01-16 15:42:18 +08:00
Gabriel Horner
34222b3436 chore: bump nbb-logseq that points to upstream nbb 2024-01-15 18:07:49 -05:00
Gabriel Horner
2305d09cc8 chore: DRY up duplicated vars and needless/confusing aliases 2024-01-15 16:51:27 -05:00
Gabriel Horner
ca020fcdb3 fix: loading all new deps namespaces in nbb
- bb test:load-all-namespaces-with-nbb now passes for all deps
- Updated nbb-logseq to support cljs.core/Atom
- Adjusted datascript.impl.entity with previous datascript.db require
  so that correct loading occurs in nbb
- fixed missing deps in logseq.outliner
- Converted a minor cljs.spec to malli since malli is nbb compatible
- Changed exists? to some? since they are functionally equivalent for
  the given arg and some? is nbb compatible
2024-01-15 12:43:16 -05:00
Tienson Qin
b55e4e7380 fix: db requests could be dropped if some requests not finished yet 2024-01-14 12:55:55 +08:00
Gabriel Horner
951e7cc841 fix: ns-docstring and large-var deps lints
Also fixed db-and-file-graphs-separate lint and moved gp-property tests
that should be moved since their source code has moved from frontend
to gp-property
2024-01-12 17:03:09 -05:00
Gabriel Horner
690764aa52 fix: unused time dep and duplicated time-ms fns
cljs-time shouldn't be added to nbb.edn since it's already included in
nbb-logseq (since parts of it aren't nbb compatible). duplicated
time-ms fns can now be removed since they are available to all deps
from common dep
2024-01-12 10:41:22 -05:00
Gabriel Horner
cdddce0120 fix deps db test and most lints
Also deleted duplicated gp-util.{page-ref,block-ref} files that were moved to common
and moved a related test to common dep
2024-01-11 14:35:04 -05:00
Gabriel Horner
a81e18e9a6 fix: update nbb-logseq and make entity-plus nbb compatible
deps can now load in nbb and all nbb tests pass except for db
2024-01-11 13:10:06 -05:00
Gabriel Horner
a2bb7e4479 fix: adding timestamps to imports shouldn't change updated-at 2024-01-10 16:06:20 -05:00
Tienson Qin
9ee04feeb5 Merge branch 'feat/db' into refactor/pipeline-worker 2024-01-10 16:13:56 +08:00
Tienson Qin
271f34838a fix: page property test 2024-01-10 05:29:52 +08:00
Gabriel Horner
3cb980bf91 fix: invalid pages and whiteboard pages from import
- some pages are missing block/journal? and block/original-name
- removed another file graph only attribute
- fixed whiteboard block schema missing :block/path-refs
- pulled out importing block to its own fn

Part of LOG-2990
2024-01-09 16:03:49 -05:00