Commit Graph

276 Commits

Author SHA1 Message Date
Gabriel Horner
df10beb325 enhance: add schema and validation to sqlite.build
Also renamed block-level keys that are specific to the ns to have
a :build namespace. A lot easier to read, maintain and extend later.
Also cleaned up page naming to consistently require :block/original-name
2024-06-03 12:46:53 -04:00
Gabriel Horner
2456deb097 enhance: create sqlite.build ns as a better
load-test-files helper for db graphs and deps.
Most of this ns is split off from create-graph.
Migrated pipeline-test to correct outliner dep and
moved previous pipeline-test to correct db dep
2024-06-03 12:46:53 -04:00
Gabriel Horner
03167ae19c enhance: extract block content refs for generated graphs
Also update content with ids and create pages as needed
2024-05-31 17:19:25 -04:00
Gabriel Horner
790d0460a1 fix: remove + disable :block/macros for db graphs
part of LOG-3137
2024-05-30 11:01:37 -04:00
Gabriel Horner
effde81161 enhance: all user classes have :db/ident
This is important as a user's ontology (properties + classes) is now describable
with unique identifiers. This will allow for easy sharing and
importing of ontologies between users.
Also <create-class! now exists to create classes consistently. Will
be helpful when we want to provide some validation for classes
2024-05-28 16:52:12 -04:00
Gabriel Horner
74274bfd82 fix: create-graph graphs with properties on properties
unable to load. Subtle bug with transacting. Now that user properties
modify  schema, schema tx needs to be separated out from use of new
schema. Also add :block/refs to schema debug file and other minor cleanup
2024-05-24 12:09:22 -04:00
Gabriel Horner
26cf70f6cf fix: update schema and remove unused :name in :icon key 2024-05-23 09:45:38 -04:00
Gabriel Horner
ceb3681cfe fix: nbb scripts updates :block/refs the same as app
nbb was only handling path-refs previously. Can remove hacky
managing of refs in create-graph. Still a TODO to fix schema graph
lazy loading bug. Related to LOG-2819
2024-05-22 16:51:50 -04:00
Gabriel Horner
67ff14038a enhance: add :config option to schema graph
also fix outliner lints
2024-05-22 13:37:40 -04:00
Gabriel Horner
04e71a9a23 Bump datascript for nbb-logseq 2024-05-22 09:37:16 -04:00
Gabriel Horner
2a37687198 chore: create graph scripts use keyword names for :classes
Just like :properties for config to be consistent. Also tweaked
generated classes to inherit from Root, even for schema graph
2024-05-20 15:43:05 -04:00
Gabriel Horner
c5911a571b fix: loading of classes for script-generated graphs
Lazy loading of refs for classes isn't supported in the UI yet so
no need to self-inflect this. Also can move schema graph from :string
to :default usage now
2024-05-20 10:45:27 -04:00
Gabriel Horner
0284816f22 chore: replace uses of :string in tests and properties graph
Also removed test which tested an unused fn
2024-05-16 16:28:43 -04:00
Gabriel Horner
d37df950fa fix: property rules now work with property ref values
Also remove use of :string properties in tests
2024-05-16 15:05:37 -04:00
Gabriel Horner
43dfdbe0ae fix: convert remaining built-in :string props to :default 2024-05-16 11:22:09 -04:00
Gabriel Horner
0238aff2ba fix: replace uses of :string closed with :default closed
and remove :string from closed possibilities
2024-05-14 10:34:54 -04:00
Gabriel Horner
330571effe fix: properties of :classes and :properties didn't generate
from scripts. Fixes schema graph generation
2024-05-13 15:33:47 -04:00
Gabriel Horner
e55927086f fix: creating :number and :url closed values from config
Also add schema hack for fully-loaded being persisted. Also
add :default property examples to example properties graph now that it's
possible
2024-05-13 12:33:31 -04:00
rcmerci
5bb8e03550 refactor(rtc,wip): re-implement update-op part in rtc,
use avt-coll data in update-op to simplify the code.
Handle different attributes in a unified way, instead of
dealing with each different attribute separately
2024-05-11 20:54:14 +08:00
Gabriel Horner
fcf6ebd0e4 fix: page properties for :default, :number and :url
Also fix page property example in properties graph and removed
unused branches in validation
2024-05-10 16:40:42 -04:00
Gabriel Horner
64786bb258 fix: create :number and :url properties from scripts 2024-05-10 15:19:10 -04:00
Gabriel Horner
fcbfde5a57 fix: generating :classes for schema graph
and any other create graph scripts. Also removed last
of uuid/db-id management in external scripts
2024-05-09 23:32:16 -04:00
Tienson Qin
45f14d4b76 refactor: property/schema.properties
Replace [:block/schema :properties] with ref type property/schema.properties.

This commit also introduces :logseq.class/base so that all the other
classes will be its children.
2024-05-09 08:29:11 +08:00
Gabriel Horner
8b15b64360 chore: disable unsupported :page and :date generation in properties graph 2024-05-08 12:08:38 -04:00
Gabriel Horner
a8081ef73a chore: update linter to validate more file-graph attrs aren't in db 2024-05-06 09:40:33 -04:00
Gabriel Horner
333490fde6 enhance: create-graph scripts can generate classes via :classes
Any script can pass a :classes config key. schema script no longer needs
to manage db ids or db idents
2024-05-03 16:24:46 -04:00
Gabriel Horner
d4bc5f22ff fix: large graph generation for some cases
Fixes some generation cases like 1 or 2 pages with 1000 blocks.
Larger generation with 10k+ blocks still fail b/c of block/order
generation. Also remove deleted :block-id-fn
2024-05-01 17:21:54 -04:00
Tienson Qin
1ec4796eff refactor: replace :block/left with :block/order
Finally no need to worry about parent-left conflicts and broken chain.
With :block/order, we only need to re-compute new orders for siblings
with same order (it can happens if there're bugs in our code, or
updates from rtc), but it doesn't break UI.

Another huge potential benefit after discussing with Zhiyuan is:
Ee might be able to simplify both RTC and undo/redo, currently, we
need to handle each new op for both of them, with recently
refactorings like properties being db attributes, :block/order
is a string instead of a ref, we can handle most property value
conflicts using last-write-wins, and others (e.g. :block/parent,
property with :default type) specifically.

I haven't fixed the issues of using :block/left in RTC and undo/redo,
because we might change both soon.
2024-05-02 01:12:22 +08:00
Gabriel Horner
27ed8c7f29 enhance: allow user classes to have db idents
This allows imported and eventually user classes to have db idents.
Updated schema example graph which now imports classes as
:schema.class/X. Also fix schema properties which were removed from
the debug file and remove unused property uuids
2024-05-01 12:20:26 -04:00
Gabriel Horner
fec63839bd fix: update properties graph to generate string-closed
instead of default-closed. Also add string-many examples
2024-04-30 14:23:49 -04:00
Gabriel Horner
a35bc9b0d6 Bump nbb-logseq version for datascript and ordered-map changes 2024-04-30 10:10:11 -04:00
Tienson Qin
723f0a5bab fix: class ordered properties 2024-04-29 12:27:18 +08:00
Tienson Qin
5d4ae66d86 revert: remove :block/properties 2024-04-28 19:44:53 +08:00
Gabriel Horner
4276a3f109 fix: replace file graph helper in db graph namespaces
with db graph equivalent
2024-04-25 12:19:58 -04:00
Gabriel Horner
3681ac354b fix: move db-pipeline and datascript-report back to outliner dep
graph-parser is primarily a file graph dep that is maintained separately
from db graphs. We want to move as many namespaces out of it as
possible. Also moved differing count assertions to
their respective tests
2024-04-25 11:22:14 -04:00
Tienson Qin
b55c064305 fix: move :block/path-res calculation to dep graph-parser 2024-04-25 16:50:04 +08:00
Tienson Qin
d416687b7b fix: :block/original-name shouldn't be unique
There're at least two use cases for same page original names:
1. multiple clients can create pages with the same title
2. multiple tagged blocks can have the same :block/original-name
2024-04-24 22:49:57 +08:00
Tienson Qin
8d9e6da41e refactor: use :block/type for journal pages instead of :block/journal?
breaking change:
removes :block/journal attribute
2024-04-24 20:12:15 +08:00
Tienson Qin
fef4e896e1 fix: use existing property pair if exists 2024-04-24 19:13:48 +08:00
Gabriel Horner
2f0fc3c18c fix: large graph creation having blank and incorrect block counts
large graph chunking assumed a certain tx order which isn't guaranteed.
There are also some edge cases where blocks could get separated from
each other which are now fixed. Also fixed a bug with validating user
properties and made small enhancement for query script
2024-04-23 09:25:07 -04:00
Tienson Qin
1cd1914d1d Enhance: show generated db's directory in the create graph script 2024-04-22 09:24:03 +08:00
Gabriel Horner
8f411fad42 enhance: allow user properties with other namespaces
to allow for imported properties to have a different namespace than the
one the editor generates. Update schema example graph which now imports
properties as :schema.property/X
2024-04-19 16:54:06 -04:00
Gabriel Horner
449d4d3600 fix: generate schema graph with new properties
Also remove unused bindings and basic validation to graph creation
2024-04-19 10:45:30 -04:00
Gabriel Horner
390892e524 fix: build all closed value property types correctly
Fixes building of :url and :number closed value properties which
affected properties graph and would effect future built-in properties.
Also simplify translation of property values by keeping it only
in create-graph namespaces
2024-04-17 15:52:55 -04:00
Gabriel Horner
0aa4bda66c fix: inlined db-based-graph? calls in db-property
Addressed fixmes for db-based-graph fns that were inlined to prevent
circular dependencies b/n db-property and sqlite-util.  Moved multi
graph fns to db-property-util and existing db-property-util to
db-property-build to address this
2024-04-16 14:43:28 -04:00
Gabriel Horner
90a7fe62c6 fix: generating graph with properties 2024-04-12 17:07:58 -04:00
Gabriel Horner
cbcff4bae8 fix: schema graph creation for previous properties schema
Also add db/id to query script and fix outdated case insensitive
constraint on property db idents
2024-04-11 17:46:49 -04:00
Tienson Qin
419eac1b16 Merge branch 'feat/db' into refactor/db-properties-schema 2024-04-11 19:56:34 +08:00
rcmerci
5d28129f81 fix(lang): remove non-exist key 2024-04-11 14:47:50 +08:00
Gabriel Horner
4f8d76c79f Merge branch 'feat/db' into refactor/db-properties-schema 2024-04-09 16:29:46 -04:00