Commit Graph

79 Commits

Author SHA1 Message Date
Gabriel Horner
7e6f5878d6 fix: not filter query bug
for db graphs. not filter was returning all blocks
including all built-in properties. Also fixed a couple
tests to be db-graph compatible
2024-06-06 10:57:34 -04:00
Gabriel Horner
8f6f52d630 fix: sort-by filter for db graphs
Also fixed a dsl-query sorting bug where nil property values where
incorrectly being treated as greater than non-nil values. Also
fixed a bug where blocks with an explicit :block/created-at weren't
being set for sqlite-build
2024-06-05 16:57:51 -04:00
Gabriel Horner
29faedc4d0 fix: db graph tests with children, created-at properties
and journal pages
2024-06-05 14:47:38 -04:00
Gabriel Horner
e7f795103d fix: simple task queries and start fixing db dsl-query tests
Tests caught that task query wasn't working. Also disabled
namaspace test and rule for db graphs since they aren't supported
there
2024-06-04 15:53:36 -04:00
Tienson Qin
6eb6e60b55 Bring back namespaces/page merge/nested pages 2024-04-07 13:48:55 +08:00
Tienson Qin
60d4fca0ba wip: remove :block/name uniqueness for db based graphs
This PR also remove supports for:
1. merge pages when renaming a page to existing page
2. namespaces such as a/b/c
3. nested page such as [[a [[nested page]]]]

Pages merge might be added back depends on RTC, but it should be
decoupled from renaming, otherwise it's too complex.

Namespaces and nested pages have been contributed some critical bugs
that lead data-loss, they're so complex together with page alias,
it's just impossible to have a good test coverage and ensure the app
is stable, especially when page rename and RTC.
2024-04-04 15:57:48 +08:00
Tienson Qin
9194a671fd Move outliner core to its own dep 2024-01-03 18:02:48 +08:00
Tienson Qin
018e1ae8ee Merge branch 'master' into feat/db 2023-12-23 14:18:25 +08:00
Tienson Qin
ea3f5973b7 Merge branch 'feat/db' into feat/datascript-storage 2023-12-06 01:10:23 +08:00
ksqsf
ef60c92e30 fix: and-or query 2023-12-04 13:46:04 -05:00
Gabriel Horner
e85557bacd Fix graph-parser lint
Also bump property perf for db-graph as it's a little slower but
still within reasonable range
2023-08-01 16:48:35 -04:00
Gabriel Horner
991c0994e5 All query-dsl tests pass for db graphs
Add to CI. Also fix graph-parser lint and typo
2023-08-01 16:34:48 -04:00
Gabriel Horner
81ae67cdbe Add timestamp support to sort-by for db graphs 2023-07-31 16:34:33 -04:00
Gabriel Horner
8e360405d6 Fix sort-by for user properties with db graphs
sort-by-queries test passes except for timestamps.
Had to introduce another query to resolve property uuids.
Made it as efficient as possible with datoms and pull.
Refactored load-test-files to support journal files and
allow page and block properties from one file
2023-07-31 16:07:23 -04:00
Gabriel Horner
4a5dd5bd83 Allow dsl-query tests to run from existing :block/content
Generate :file/blocks from content thus allowing more tests to run
under db graph. Also loosened a couple query-dsl tests to pass
with db graphs without sacrificing test accuracy
2023-07-26 16:07:41 -04:00
Gabriel Horner
3f52860cf1 Fix perf issue with property rules for db graphs
Use multiple clauses instead as confirmed by perf test.
Using multiple clauses made rule preparation for d/q more complex
so introduced extract-rules
2023-07-25 17:20:17 -04:00
Gabriel Horner
0ca7d1a4de Fix multi-valued properties and enable first property graph test
Run db prop test with: DB_GRAPH=1 node static/tests.js -i focus
2023-07-24 16:42:36 -04:00
Gabriel Horner
8697d59b72 Add test to prevent regression on #9119 2023-04-24 11:08:35 +08:00
Tienson Qin
db6fc6b7ee feat: simple query builder (#8774)
Simple query builder

---------

Co-authored-by: Gabriel Horner <gabriel@logseq.com>
Co-authored-by: charlie <xyhp915@qq.com>
2023-03-21 14:27:00 +08:00
Gabriel Horner
9d9e976b8c Merge branch 'master' into refactor/refs-from-property-values 2022-09-21 15:34:47 -04:00
Gabriel Horner
91d6d14720 Fix sort-by not working for blocks
- Also add sort-by support for pages which fixes #5643
- Add sort-by tests
- Fix sample producing weird queries e.g. mixed block and page clauses
- Clean up last, hard to read test setup in query-dsl
2022-09-20 12:20:17 -04:00
Tienson Qin
9e2d6e7cfb fix: page refs 2022-09-09 12:37:57 +08:00
Tienson Qin
336fa578d5 refactor: replace directives(front matter or org) with properties 2022-09-09 12:37:55 +08:00
Gabriel Horner
55ccc2d17c Rename config to :rich-property-values?
Name should allow for additional functionality later
2022-08-23 09:55:15 -04:00
Gabriel Horner
39b43b363c Fix page-properties bug and dsl-query bug
parser test caught page properties bug and dsl-query tests once written
correctly exposed another bug. Also remove unit test as they are
replaced by integration style tests in graph-parser-test
2022-08-11 16:32:00 -04:00
Gabriel Horner
0719163d30 Split out parse-property for use in dsl-query and tests
- Add tests for all *property dsl queries with and without new
  config option
- Add tests for property persistence
- Add tests for property relationships
2022-08-11 16:32:00 -04:00
Gabriel Horner
405183db09 Add config option to allow for more free text in property values
Still need to test that *property operators work correctly when this
option is on
2022-08-11 16:32:00 -04:00
Tienson Qin
a1ca6820df Fix: broken linked references (#6105)
* enhance: clicking the refresh button if a query is slow

* fix: skip running slow queries if there's no need to refresh

* fix: linked reference filtering does not work on TASK items

close #1669

* fix: each block should have its own page as a reference when query page references

* fix: references

* fix: non consecutive blocks in query result and filtered linked references

* simplify filters logic

* fix: ref numbers

* Save both :block/refs and :block/path-refs for different usage

* fix: block refs

* enhance: move scheduled and deadlines to its own ns

* linked references performance tweaks

* mouse hover filters icon to expand the collapsed linked refs

* perf: react/refresh! once instead twice

* enhance: compute flashcards every hour instead of every 5s

* feat: macros as blocks

* feat: flashcards list

* fix: refed pages count

* fix: can't select in linked references

* fix: block editing on linked refs

* perf: editing in linked references

* enhance: update srs cards number when clicking flashcards

* Add a test for the case fixed in #6100

* Address feedbacks from Gabriel

* fix: Block Ref Indicator is missing from the references-blocks class

close #5375

* fix: referenced pages

* fix: page refs

* fix: Using filters pushed the title property to the second block

close #5845

Co-authored-by: Gabriel Horner <gabriel@logseq.com>
2022-08-04 13:03:12 +08:00
Tienson Qin
49acb54178 feat: add properties as linked references 2022-07-07 01:42:16 +08:00
Tienson Qin
904eff6d9d Core outliner operations refactoring (#4880)
* Add outliner nested transact!

Copied the code mostly from https://github.com/logseq/logseq/pull/4671
by zhiyuan

* refactor: insert-blocks

* fix: insert-blocks

* fix: move cursor to the last block when inserting

* fix: replace the current block when inserting and its content is empty

* keep only :insert-blocks

* expose only :delete-blocks

* Use existing implementations for move-nodes-up-down and

indent/outdent.

* fix editing state not updated immediately

* fix editing status

* fix: avoid recursive copy

* fix: inserting blocks after an empty block

* Implement move-blocks with insert-blocks

* fix: block left

* Implement move-blocks-up-down with move-blocks

* fix: paste text

* Implement indent-outdent-blocks with move-blocks

* fix: indent/outdent

* feat: multiple blocks drag && drop

* fix: indent/outdent blocks

* fix: drag drop

* Port unit tests for outliner.core

* enhance: open collapsed parent when indenting blocks

* refactor: block selection

* fix: indent/outdent blocks with different levels

* Add instrument on invalid outliner structure

* fix: can't write a block if the page has any outdated blocks

* fix: editing status for empty page

* fix: multiple drag & drop

* fix: drag & drop disallows moving from parents to its child

* fix: public property

* fix: can't delete first empty block

* Remove unused code

* fix: e2e tests

A workaround is to not select/highlight the block when pressing esc if it has
fenced code.

* remove unused code

* Add batch transaction test

* fix: update :block/page when dragging targets' children to another page

* Add more tests

* Simplify extract

* Replace db/get-conn with db/get-db

* Simplify extracting blocks from ast

* Code cleanup

* Code cleanup

* Add outliner core fuzzy tests

* Remove unused code

* fix: cursor not jump to the upper block when pressing Enter in the beginning

* fix: Enter in the beginning of a non-empty block

* Fix lint warnings

* Add editor random e2e tests

* Fix typo

* enhance: move some fns and add some comments

* enhance(outliner): add page-block? util

* fix: increase td width to prevent content overflow

Signed-off-by: Yue Yang <g1enyy0ung@gmail.com>

* First pass at file tests for file-sync

Each action usually passes by 5th try

* Fix two incorrect calls caught by tests

* More test improvements

- Easier auth setup
- subdirectory is configurable
- list graphs api also exercised

* Address cleanup from #3839

- Remove unused translation key
- Delete or TODO commented code
- Capitalize notifications to users

* fix quick capture template not working

* enhance(sync): add logout

* enhance: add logout i18n

* fix(plugin): sometimes plugin settings of gui not work when entry from app settings

* enable show-brackets? toggle for orgmode [[file:./pages/demo.org][demo]]

* fix(sync): fix unfinishable sync loop

* feature: logseq protocol; refactor persistGraph

* fix: deeplink support

* fix: broadcast persist graph on opening new graph with logseq protocol

* feat: logseq protocol open action for page-name and uuid

* fix: logseq protocol graph param validation

* ux: copy logseq URL of block

* enhance: remove the redundant 'open' from logseq protocol (v0.1)

* ux: page dropdown button for copy page URL

* chore: logseq protocol comments

* don't create new contents file when changing format

Logseq now creates a new contents file when users try to toggle the
preferred format, which causes file duplications error.

* fix pasting in src block not working on iOS

close https://github.com/logseq/logseq/issues/4914

* fix playing video goes into editing mode on iOS

* fix copy to clipboard failure on iOS

* add Podfile item

* fix mobile toolbar order not persisting after restart

* test(e2e): add test for backspace and cursor pos (#4896)

* test(e2e): add test for backspace and cursor pos
* fix(test): refine, fix wrong helper

* fix(ui): warn about illegal git commit interval

* enhance(editor): allow global git cmd shortcut

* style(settings): line-space of general/journals

* enhance(editor): accept enter in dummy block

Fix #4931

* fix editing state not updated immediately

* fix: can't write a block if the page has any outdated blocks

TODO: clean outdated blocks

* fix: editing status for empty page

* Random tree for outliner core tests

* Add pre assertions and fn docs based on Zhiyuan's suggestions

* Made some changes based on Gabriel's suggestions

* fix: tests

* fix: save current block before moving

* Updated the timeout to 100ms based on llcc's suggestion

https://github.com/logseq/logseq/pull/4880#discussion_r851966301

* api-insert-new-block! supports replace-empty-target?

* fix: replace all :reuse-last-block? usage

Co-authored-by: rcmerci <rcmerci@gmail.com>
Co-authored-by: Yue Yang <g1enyy0ung@gmail.com>
Co-authored-by: Gabriel Horner <gabriel@logseq.com>
Co-authored-by: llcc <lzhes43@gmail.com>
Co-authored-by: charlie <xyhp915@qq.com>
Co-authored-by: Junyi Du <junyidu.cn@gmail.com>
Co-authored-by: Andelf <andelf@gmail.com>
2022-04-19 11:14:38 +08:00
Gabriel Horner
6b935a8d58 Fix page-ref rule which was not bindable and failed fast
- page-ref had a :block/name embedded lookup ref which failed fast.
  datomic advises against this -
  https://docs.datomic.com/on-prem/schema/identity.html#:~:text=You%20cannot%20use%20them%20in%20the%20body%20of%20a%20query%2C%20use%20datalog%20clauses%20instead
- page-ref can handle binding args like last commit for page
- page-ref not failing fast on non-existing pages means that user can get meaningful results
  in more complex queries as seen in test
- Also remove try/catch as there should be no more queries that
  unintentionally cause the 'Nothing found for entity' error
- Also fixed lookup ref in srs cards which can be seen in stacktrace of
  #4576
2022-03-17 16:15:04 +08:00
Gabriel Horner
96da98e5bd Fix Tienson's nested not case 2022-03-16 16:04:59 +08:00
Gabriel Horner
a41dd0abfc Remove parse catch for consistent error handling
For advanced queries we error when invalid queries are written. We
should do the same for simple query.
Also add a test for tag query format
2022-03-16 16:04:59 +08:00
Gabriel Horner
8468602881 query-dsl + tests cleanup
Remove old string checks that no longer apply. Also added another blank
string case that causes an issue
2022-03-16 16:04:59 +08:00
Gabriel Horner
5f0efcf2d8 Fix another nested boolean case
Also removed or nested complexity from
903a757ce2 that no longer seems necessary
2022-03-16 16:04:59 +08:00
Gabriel Horner
3a19687988 Fix nested boolean case - close #4510 2022-03-16 16:04:59 +08:00
Gabriel Horner
79bbf21f7d Allow advanced queries to pull in dsl rules as needed
- Remove unused query' and when checks from custom-query
- Update bb command to use correct watch
- Provide $EXAMPLE mode in query dsl test to document rule usage
2022-03-10 15:43:07 +08:00
Gabriel Horner
a7b230d71d Organize testing helpers and namespaces
Start a helper ns and put test specific namespaces under frontend.test.
2022-03-10 15:43:07 +08:00
Gabriel Horner
d955918525 Display explicit error for one remaining react-query (advanced query)
This is a follow up to #4434. I've now confirmed that all uses of
react/react-query benefit from an explicit exception as they will now
see an explicit error instead of 0 results which is confusing for the
user and harder for us to debug
2022-03-09 14:10:38 +08:00
Gabriel Horner
f9cb132482 Fix simple queries not working in advanced query - close #4498
Also removed unused repo fn in query-dsl ns and configured custom-query
to throw exception
2022-03-09 14:10:38 +08:00
Gabriel Horner
4c777fc502 Convert remaining query operators to use rules
- Removed counter and uniq-symbol as those are no longer needed
- build-query cleanly maps conditions to their fns
- Query dsl tests have been fully decoupled
2022-03-08 16:27:30 +08:00
Gabriel Horner
198c3363f7 More query dsl improvements
- Add tests for 5 operators that weren't tested
- Converted all operators to rules except two
2022-03-08 16:27:30 +08:00
Gabriel Horner
bcfb9d99ee Move *page-tags operators to rules
Also converted their tests to the new format and finally made test-parse
small enough to be considered more readable
2022-03-08 16:27:30 +08:00
Gabriel Horner
80ef31f4c0 Fixes #4101 by moving todo/task and priority to rules
Task and priority weren't composable in most boolean settings. Rules
fixes that
2022-03-08 16:27:30 +08:00
Gabriel Horner
12395c61fb Introduce rules to query-dsl
Fixes #4217 as each use of page-property have unique bindings that don't
clobber each other
2022-03-01 16:46:27 -05:00
Gabriel Horner
ce618443bf Improve query dsl tests
- Each dsl test can load its own test data
- Refactored property and page-property tests to test accuracy
  of each
- Also added humane-test-output to make diff easier to read when tests
  fail
- Also fixed weird behavior of test runner that runs all tests when no
  tests selected
2022-03-01 16:42:19 -05:00
Gabriel Horner
c6d03b67ab Merge branch 'master' into dev/var-sizes-linter 2022-02-08 12:02:39 -05:00
Tienson Qin
6151ca9c09 fix: CI tests 2022-02-08 22:54:18 +08:00
Gabriel Horner
1df386ecca Add regression tests for querying properties with links
These tests were fixed by f1728c48cb

Also cleaned up more of query and tests i.e. split up large fns
2022-02-08 14:47:08 +08:00
Gabriel Horner
5a126027eb Fix #4007 to allow linked integer page links to match property query 2022-02-08 14:47:08 +08:00