Feat: new file name escaping rules (#6134)

* feat: new file name escaping for namespace

feat: new file name decoding back to page name

* test: file name sanitization

feat: use _0x to encode %

feat: don't create title property

test: extra URL encoding for escaped file names

fix: fit pdf prefix into new file name rules

fix: encoding rules on some characters

fix: handle the buggy file names imported by users

fix: pdf block ref failed to jump

fix: #6167

* fix: enhance backward compatibility

chore: title validation

test: fix namespace queries test

chore: use index version stored in config.edn instead of search.versions

* feat: convert old version graph mechanism

ui: file conversion UI

feat: rename files for conversion

feat: don't trigger conversion when title property is manually edited

fix: file conflict notification while renaming files on some OS

feat: re-index on update version

feat: clicking NO in the re-index dialog would update the index-ver flag to suppress the dialog

feat: use html entities for reserved char escaping

dev: remove unresolved vars & minor refactor

chore: move file name sanity from gp-util to fs-util, as it's for encoding only but not parsing

test: update file name tests to html entities rule

test: convert files from dir ver 3 for repo_tests

feat: convert Windows reserved file names

fix: save index version into idb instead of file

fix: decode uri of path while parsing files on mobile

fix: couple dir version and index version to ensure only re-index on converted dirs

feat: go back to url-encode for special chars

* chore: fix lint

chore: improve codebase to address Gabriel's comments

fix: remove file remnants on add conflict

fix: remove file remnants on rename conflict

chore: add test ns to nbb runner

Also fix typoed fn and remove unused code

* fix: issues of rebase PR6134 to master after file-sync merged

* feat: switchable filename format

* fix: use  go block to replace promesa for rename all with blocking

* feat: re-index after apply rename all

* ui: file conversion enhancement

* fix: merging filename format PR with master

* fix: filename format lint & CI

* ui: filename format flow

* fix: error handling on the rare internal file path confliction case

* chore: shorten component code for files-breaking-changed

* chore: fix CI

* Minor fixes per latest code review

- Remove unused page-name-order
- Update catch usage to be consistent with what's on master
- Place state fn in right place
- Wording fixes:
  - select and apply -> manual. There are no checkboxes for the user
  - Update -> Edit. We use edit for all other settings button
  - Alternatives to starting sentences with May. Not a common way to
    start a sentence
  - update outdated template comment

* ux: rename instruction update

* ux: rename instruction update (2)

* Tweak wording of conversion modal

Simplifed first paragraph and explained the page to the user in first
sentence, may isn't a common way to start sentences and updated outdated
wording

* Fix large-var warning by splitting out a piece of component

* fix: right slash on Windows; legacy format file sanitization

* fix: triple lowbar renaming fns

Co-authored-by: Gabriel Horner <gabriel@logseq.com>
This commit is contained in:
Junyi Du
2022-10-08 15:47:45 +08:00
committed by GitHub
parent 6007d6061f
commit 0e4037ab61
49 changed files with 1182 additions and 331 deletions

View File

@@ -0,0 +1,165 @@
(ns frontend.components.conversion
(:require [clojure.core.async :as async]
[cljs.core.async.interop :refer [p->c]]
[promesa.core :as p]
[electron.ipc :as ipc]
[logseq.graph-parser.util :as gp-util]
[frontend.util :as util]
[frontend.state :as state]
[frontend.ui :as ui]
[frontend.handler.page :as page-handler]
[frontend.handler.conversion :refer [supported-filename-formats write-filename-format! calc-rename-target]]
[frontend.db :as db]
[frontend.context.i18n :refer [t]]
[rum.core :as rum]))
(defn- ask-for-re-index
"Multiple-windows? (optional) - if multiple exist on the current graph
Dont receive param `repo` as `graph/ask-for-re-index` event doesn't accept repo param"
([]
(p/let [repo (state/get-current-repo)
multiple-windows? (ipc/ipc "graphHasMultipleWindows" repo)]
(ask-for-re-index multiple-windows?)))
([multiple-windows?]
(state/pub-event! [:graph/ask-for-re-index (atom multiple-windows?)
(ui/admonition :tip [:p (t :file-rn/re-index)])])))
(defn- <close-modal-on-done
"Ask users to re-index when the modal is exited"
[]
(async/go (state/close-settings!)
(async/<! (async/timeout 100)) ;; modal race condition requires investigation
(ask-for-re-index)))
(rum/defc legacy-warning
[repo *target-format *dir-format *solid-format]
[:div ;; Normal UX stage 1: show the admonition & button for users using legacy format
(ui/admonition :warning [:p (t :file-rn/format-deprecated)])
[:p (t :file-rn/instruct-1)]
[:p (t :file-rn/instruct-2)
(ui/button (t :file-rn/confirm-proceed) ;; the button is for triple-lowbar only
:class "text-md p-2 mr-1"
:on-click #(do (reset! *target-format :triple-lowbar)
(reset! *dir-format (state/get-filename-format repo)) ;; assure it's uptodate
(write-filename-format! repo :triple-lowbar)
(reset! *solid-format :triple-lowbar)))]
[:p (t :file-rn/instruct-3)]])
(rum/defc filename-format-select
"A dropdown menu for selecting the target filename format"
[*target-format disabled?]
[:div.it.sm:grid.sm:grid-cols-3.sm:gap-4.sm:items-start
[:label.block.text-sm.font-medium.leading-5
(t :file-rn/select-format)
[:select.form-select.is-small {:disabled disabled?
:value (name @*target-format)
:on-change (fn [e]
(let [format-str (util/evalue e)]
(reset! *target-format (keyword format-str))))}
(for [format supported-filename-formats]
(let [format-str (name format)]
[:option {:key format-str :value format-str} format-str]))]]])
;; UI for files that have been breaking changed. Conversion required to revert the change.
;; UI logic:
;; When dropdown box item switched, activate the `Proceed` button;
;; When `Proceed` button clicked, write the filename format to config and allow the renaming actions
(rum/defcs files-breaking-changed < rum/reactive
(rum/local nil ::pages) ;; pages require renaming, a map of {path -> [page-entity file-entity]}
(rum/local nil ::dir-format) ;; format previously (on `proceed `button clicked)
(rum/local nil ::target-format) ;; format to be converted to (on `proceed` button clicked)
(rum/local nil ::solid-format) ;; format persisted to config
(rum/local false ::switch-disabled?) ;; disable the dropdown box when proceeded
[state]
(let [repo (state/sub :git/current-repo)
*dir-format (::dir-format state)
*target-format (::target-format state)
*solid-format (::solid-format state)
*pages (::pages state)
need-persist? (not= @*solid-format @*target-format)
*switch-disabled? (::switch-disabled? state)]
(when (nil? @*pages) ;; would triggered on initialization
(let [pages-with-file (db/get-pages-with-file repo)
the-keys (map (fn [[_page file]] (:file/path file)) pages-with-file)]
(reset! *pages (zipmap the-keys pages-with-file))))
(when (and (nil? @*dir-format) ;; would triggered on initialization
(nil? @*solid-format)
(nil? @*target-format))
(let [config-format (state/get-filename-format repo)]
(reset! *dir-format config-format)
(reset! *solid-format config-format)
(reset! *target-format :triple-lowbar)))
[:div
(when (state/developer-mode?)
[:div
(filename-format-select *target-format @*switch-disabled?)
(ui/button (t :file-rn/select-confirm-proceed) ;; the button is for persisting selected format
:disabled (not need-persist?)
:class "text-sm p-1 mr-1"
:on-click #(do (reset! *dir-format (state/get-filename-format repo)) ;; assure it's uptodate
(write-filename-format! repo @*target-format)
(reset! *solid-format @*target-format)
(reset! *switch-disabled? true)))
[:hr]])
[:h1.title (t :settings-page/filename-format)]
[:div.rounded-md.opacity-70
[:p (t :file-rn/filename-desc-1)]
[:p (t :file-rn/filename-desc-2)]
[:p (t :file-rn/filename-desc-3)]
[:p (t :file-rn/filename-desc-4)]]
(when (= @*solid-format :legacy)
(legacy-warning repo *target-format *dir-format *solid-format))
[:div.cp__settings-files-breaking-changed {:disabled need-persist?} [:hr]
(let [rename-items (->> (vals @*pages)
(map (fn [[page file]]
(when-let [ret (calc-rename-target page (:file/path file) @*dir-format @*target-format)]
(merge ret {:page page :file file}))))
(remove nil?))
<rename-all #(async/go (doseq [{:keys [file target status]} rename-items]
(when (not= status :unreachable)
(async/<! (p->c (page-handler/rename-file! file target (constantly nil) true)))))
(<close-modal-on-done))]
(if (not-empty rename-items)
[:div ;; Normal UX stage 2: close stage 1 UI, show the action description as admolition
(if (and (= @*solid-format :triple-lowbar)
(= @*dir-format :legacy))
(ui/admonition :tip [:p (t :file-rn/need-action)])
[:p (t :file-rn/need-action)])
[:p
(ui/button
(str (t :file-rn/all-action) " (" (count rename-items) ")")
:on-click <rename-all
:class "text-md p-2 mr-1")
(t :file-rn/or-select-actions)
[:a {:on-click <close-modal-on-done}
(t :file-rn/close-panel)]
(t :file-rn/or-select-actions-2)]
[:p (t :file-rn/legend)]
[:table.table-auto
[:tbody
(for [{:keys [page file status target old-title changed-title]} rename-items]
(let [path (:file/path file)
src-file-name (gp-util/path->file-name path)
tgt-file-name (str target "." (gp-util/path->file-ext path))
rm-item-fn #(swap! *pages dissoc path)
rename-fn #(page-handler/rename-file! file target rm-item-fn)
rename-but [:a {:on-click rename-fn
:title (t :file-rn/apply-rename)}
[:span (t :file-rn/rename src-file-name tgt-file-name)]]
rename-but-sm (ui/button
(t :file-rn/rename-sm)
:on-click rename-fn
:class "text-sm p-1 mr-1"
:style {:word-break "normal"})]
[:tr {:key (:block/name page)}
[:td [:div [:p "📄 " old-title]]
(case status
:breaking ;; if properety title override the title, it't not breaking change
[:div [:p "🟡 " (t :file-rn/suggest-rename) rename-but]
[:p (t :file-rn/otherwise-breaking) " \"" changed-title \"]]
:unreachable
[:div [:p "🔴 " (t :file-rn/unreachable-title changed-title)]]
[:div [:p "🟢 " (t :file-rn/optional-rename) rename-but]])]
[:td rename-but-sm]]))]]]
[:div "🎉 " (t :file-rn/no-action)]))]]))