mirror of
https://github.com/logseq/logseq.git
synced 2026-05-24 20:54:09 +00:00
push
This commit is contained in:
26
deps/db-sync/README.md
vendored
26
deps/db-sync/README.md
vendored
@@ -153,11 +153,37 @@ Cloudflare runtime flow:
|
||||
| CLOUDFLARE_REPO_CLONE_COMMAND | Optional repo clone command template for Cloudflare sandbox |
|
||||
| CLOUDFLARE_HEALTH_RETRIES | Cloudflare sandbox health check retry count |
|
||||
| CLOUDFLARE_HEALTH_INTERVAL_MS | Cloudflare sandbox health check retry interval (ms) |
|
||||
| GITHUB_TOKEN | Fallback token used for both git push and PR API calls |
|
||||
| GITHUB_PUSH_TOKEN | Optional token used only for git push (preferred over `GITHUB_TOKEN`) |
|
||||
| GITHUB_PR_TOKEN | Optional token used only for PR creation (preferred over `GITHUB_TOKEN`) |
|
||||
| GITHUB_API_BASE | Optional GitHub API base URL override (default `https://api.github.com`) |
|
||||
| GITHUB_DEFAULT_BASE_BRANCH | Default PR base branch fallback (default `main`) |
|
||||
| OPENAI_API_KEY | Passed into Cloudflare sandbox runtime env (if set) |
|
||||
| ANTHROPIC_API_KEY | Passed into Cloudflare sandbox runtime env (if set) |
|
||||
| OPENAI_BASE_URL | Passed into Cloudflare sandbox runtime env (if set) |
|
||||
| ANTHROPIC_BASE_URL | Passed into Cloudflare sandbox runtime env (if set) |
|
||||
|
||||
## M14 Publish Endpoint
|
||||
|
||||
Agent sessions now expose:
|
||||
|
||||
`POST /sessions/:session-id/pr`
|
||||
|
||||
This endpoint is available to any authenticated collaborator and supports:
|
||||
- push only (`{"create-pr": false}`)
|
||||
- push + PR (`{"create-pr": true}` or omitted)
|
||||
|
||||
Response `status` values:
|
||||
- `pushed`
|
||||
- `pr-created`
|
||||
- `manual-pr-required`
|
||||
|
||||
If PR credentials are missing or PR API creation fails after a successful push, response includes `manual-pr-url`.
|
||||
|
||||
For Cloudflare deploys, store tokens as Worker secrets:
|
||||
- `wrangler secret put GITHUB_PUSH_TOKEN --env <staging|prod>`
|
||||
- `wrangler secret put GITHUB_PR_TOKEN --env <staging|prod>`
|
||||
|
||||
## Notes
|
||||
- Protocol definitions live in `docs/agent-guide/db-sync/protocol.md`.
|
||||
- DB sync implementation guide is in `docs/agent-guide/db-sync/db-sync-guide.md`.
|
||||
|
||||
@@ -1,23 +1,24 @@
|
||||
# M14: Git Push + Optional PR Submission
|
||||
# M14: Git Push + PR for All Users
|
||||
|
||||
Status: Planned
|
||||
Target: Enable agent sessions to push committed changes to a git repo and optionally create a pull request.
|
||||
Status: Implemented
|
||||
Target: Enable all authenticated collaborators to use agent sessions to push committed changes and create pull requests.
|
||||
|
||||
## Goal
|
||||
Allow task agents to complete a full delivery loop:
|
||||
1) edit + commit locally
|
||||
2) push branch to remote
|
||||
3) optionally submit a PR when requested by user/task context
|
||||
3) submit a PR when requested by user/task context
|
||||
|
||||
## Why M14
|
||||
- Current milestones cover repo clone and runtime provisioning, but stop short of remote delivery.
|
||||
- Users still need manual handoff for pushing and PR creation.
|
||||
- Push/PR should not be restricted to manager-only workflows.
|
||||
- End-to-end agent outcomes should include publish-ready outputs.
|
||||
|
||||
## Scope
|
||||
1) Add an explicit agent capability profile for git delivery in task/session payload handling.
|
||||
1) Add an explicit agent capability profile for git delivery in task/session payload handling with no role-based gating.
|
||||
2) Enable authenticated `git push` from sandbox runtime for supported providers.
|
||||
3) Add optional PR submission path (non-mandatory per task).
|
||||
3) Add PR submission path available to all authenticated collaborators.
|
||||
4) Persist and stream audit events for push/PR steps and failures.
|
||||
5) Keep existing agent/session flows backward compatible.
|
||||
|
||||
@@ -30,8 +31,8 @@ Allow task agents to complete a full delivery loop:
|
||||
|
||||
### WS1: Agent Capability + Session Payload
|
||||
- Define M14 capability contract for:
|
||||
- `push-enabled`
|
||||
- `pr-optional`
|
||||
- `push-enabled` (default true)
|
||||
- `pr-enabled` (default true)
|
||||
- Thread capability through session creation and runtime payload assembly.
|
||||
|
||||
### WS2: Runtime Git Push
|
||||
@@ -41,15 +42,15 @@ Allow task agents to complete a full delivery loop:
|
||||
- commit verification
|
||||
- push with error classification and retries where safe
|
||||
|
||||
### WS3: Optional PR Submission
|
||||
- Add configurable PR toggle/intent in task or message flow.
|
||||
- Create PR only when requested and branch push succeeds.
|
||||
### WS3: PR Submission (All Users)
|
||||
- Add PR intent in task or message flow for every authenticated collaborator.
|
||||
- Create PR when requested and branch push succeeds.
|
||||
- Return PR URL and metadata in session events.
|
||||
|
||||
### WS4: Observability + Safety
|
||||
- Emit structured events for:
|
||||
- push started/succeeded/failed
|
||||
- pr started/succeeded/failed/skipped
|
||||
- pr started/succeeded/failed
|
||||
- Keep failures non-destructive and user-visible with actionable messages.
|
||||
|
||||
### WS5: Tests + Docs
|
||||
@@ -57,8 +58,8 @@ Allow task agents to complete a full delivery loop:
|
||||
- Document env/config requirements for auth and provider behavior.
|
||||
|
||||
## Exit Criteria
|
||||
1) Agent can push committed changes to configured repo from an active session.
|
||||
2) PR submission works when enabled and is skipped cleanly when disabled.
|
||||
1) Any authenticated collaborator can push committed changes to configured repo from an active session.
|
||||
2) Any authenticated collaborator can submit a PR from an active session.
|
||||
3) Session event stream includes push/PR lifecycle and final links/errors.
|
||||
4) Existing non-M14 sessions continue to work unchanged.
|
||||
|
||||
4) No manager/member role gate exists for push/PR actions.
|
||||
5) Existing non-M14 sessions continue to work unchanged.
|
||||
|
||||
@@ -234,13 +234,26 @@
|
||||
[:mode {:optional true} :string]
|
||||
[:permission-mode {:optional true} :string]
|
||||
[:api-token {:optional true} :string]
|
||||
[:auth-json {:optional true} :string]]]]])
|
||||
[:auth-json {:optional true} :string]]]]
|
||||
[:capabilities {:optional true}
|
||||
[:map
|
||||
[:push-enabled {:optional true} :boolean]
|
||||
[:pr-enabled {:optional true} :boolean]]]])
|
||||
|
||||
(def sessions-message-request-schema
|
||||
[:map
|
||||
[:message :string]
|
||||
[:kind {:optional true} :string]])
|
||||
|
||||
(def sessions-pr-request-schema
|
||||
[:map
|
||||
[:title {:optional true} :string]
|
||||
[:body {:optional true} :string]
|
||||
[:head-branch {:optional true} :string]
|
||||
[:base-branch {:optional true} :string]
|
||||
[:create-pr {:optional true} :boolean]
|
||||
[:force {:optional true} :boolean]])
|
||||
|
||||
(def sessions-create-response-schema
|
||||
[:map
|
||||
[:session-id :string]
|
||||
@@ -265,6 +278,16 @@
|
||||
[:ok :boolean]
|
||||
[:flushed {:optional true} :int]])
|
||||
|
||||
(def sessions-pr-response-schema
|
||||
[:map
|
||||
[:status :string]
|
||||
[:head-branch :string]
|
||||
[:base-branch {:optional true} :string]
|
||||
[:pr-url {:optional true} :string]
|
||||
[:manual-pr-url {:optional true} :string]
|
||||
[:force {:optional true} :boolean]
|
||||
[:message {:optional true} :string]])
|
||||
|
||||
(def http-request-schemas
|
||||
{:graphs/create graph-create-request-schema
|
||||
:graph-members/create graph-member-create-request-schema
|
||||
@@ -274,7 +297,8 @@
|
||||
:e2ee/graph-aes-key e2ee-graph-aes-key-request-schema
|
||||
:e2ee/grant-access e2ee-grant-access-request-schema
|
||||
:sessions/create sessions-create-request-schema
|
||||
:sessions/message sessions-message-request-schema})
|
||||
:sessions/message sessions-message-request-schema
|
||||
:sessions/pr sessions-pr-request-schema})
|
||||
|
||||
(def http-response-schemas
|
||||
{:graphs/list graphs-list-response-schema
|
||||
@@ -306,6 +330,7 @@
|
||||
:sessions/resume sessions-resume-response-schema
|
||||
:sessions/interrupt http-ok-response-schema
|
||||
:sessions/cancel http-ok-response-schema
|
||||
:sessions/pr sessions-pr-response-schema
|
||||
:sessions/events sessions-events-response-schema
|
||||
:error http-error-response-schema})
|
||||
|
||||
|
||||
@@ -44,6 +44,11 @@
|
||||
:cloudflare-repo-clone-command (env-value env "CLOUDFLARE_REPO_CLONE_COMMAND")
|
||||
:cloudflare-health-retries (env-value env "CLOUDFLARE_HEALTH_RETRIES")
|
||||
:cloudflare-health-interval-ms (env-value env "CLOUDFLARE_HEALTH_INTERVAL_MS")
|
||||
:github-token (env-value env "GITHUB_TOKEN")
|
||||
:github-push-token (env-value env "GITHUB_PUSH_TOKEN")
|
||||
:github-pr-token (env-value env "GITHUB_PR_TOKEN")
|
||||
:github-api-base (env-value env "GITHUB_API_BASE")
|
||||
:github-default-base-branch (env-value env "GITHUB_DEFAULT_BASE_BRANCH")
|
||||
:openai-api-key (env-value env "OPENAI_API_KEY")
|
||||
:anthropic-api-key (env-value env "ANTHROPIC_API_KEY")
|
||||
:openai-base-url (env-value env "OPENAI_BASE_URL")
|
||||
|
||||
@@ -77,6 +77,11 @@
|
||||
(aset "CLOUDFLARE_REPO_CLONE_COMMAND" (:cloudflare-repo-clone-command cfg))
|
||||
(aset "CLOUDFLARE_HEALTH_RETRIES" (:cloudflare-health-retries cfg))
|
||||
(aset "CLOUDFLARE_HEALTH_INTERVAL_MS" (:cloudflare-health-interval-ms cfg))
|
||||
(aset "GITHUB_TOKEN" (:github-token cfg))
|
||||
(aset "GITHUB_PUSH_TOKEN" (:github-push-token cfg))
|
||||
(aset "GITHUB_PR_TOKEN" (:github-pr-token cfg))
|
||||
(aset "GITHUB_API_BASE" (:github-api-base cfg))
|
||||
(aset "GITHUB_DEFAULT_BASE_BRANCH" (:github-default-base-branch cfg))
|
||||
(aset "OPENAI_API_KEY" (:openai-api-key cfg))
|
||||
(aset "ANTHROPIC_API_KEY" (:anthropic-api-key cfg))
|
||||
(aset "OPENAI_BASE_URL" (:openai-base-url cfg))
|
||||
|
||||
244
deps/db-sync/src/logseq/db_sync/worker/agent/do.cljs
vendored
244
deps/db-sync/src/logseq/db_sync/worker/agent/do.cljs
vendored
@@ -5,6 +5,7 @@
|
||||
[logseq.db-sync.platform.core :as platform]
|
||||
[logseq.db-sync.worker.agent.runtime-provider :as runtime-provider]
|
||||
[logseq.db-sync.worker.agent.session :as session]
|
||||
[logseq.db-sync.worker.agent.source-control :as source-control]
|
||||
[logseq.db-sync.worker.http :as http]
|
||||
[promesa.core :as p]))
|
||||
|
||||
@@ -114,12 +115,61 @@
|
||||
(broadcast-event! self event)
|
||||
{:session session :event event})))))
|
||||
|
||||
(defn- <append-publish-event!
|
||||
[^js self event-type data]
|
||||
(<append-event! self {:type event-type
|
||||
:data data
|
||||
:ts (common/now-ms)}))
|
||||
|
||||
(defn- session-conflict [message]
|
||||
(http/error-response message 409))
|
||||
|
||||
(defn- terminal-status? [status]
|
||||
(contains? #{"completed" "failed" "canceled"} status))
|
||||
|
||||
(defn- session-capabilities
|
||||
[session]
|
||||
(merge {:push-enabled true
|
||||
:pr-enabled true}
|
||||
(when (map? (:capabilities (:task session)))
|
||||
(:capabilities (:task session)))))
|
||||
|
||||
(defn- push-enabled?
|
||||
[session]
|
||||
(true? (:push-enabled (session-capabilities session))))
|
||||
|
||||
(defn- pr-enabled?
|
||||
[session]
|
||||
(true? (:pr-enabled (session-capabilities session))))
|
||||
|
||||
(defn- repo-url-from-session
|
||||
[session]
|
||||
(some-> (get-in session [:task :project :repo-url]) str string/trim not-empty))
|
||||
|
||||
(defn- generated-head-branch
|
||||
[session-id]
|
||||
(let [suffix (some-> session-id
|
||||
str
|
||||
string/lower-case
|
||||
(string/replace #"[^a-z0-9-]+" "-")
|
||||
(string/replace #"-+" "-")
|
||||
(string/replace #"^-+" "")
|
||||
(string/replace #"-+$" ""))]
|
||||
(str "logseq-agent/" (if (string/blank? suffix) "session" suffix))))
|
||||
|
||||
(defn- default-base-branch
|
||||
[^js env]
|
||||
(or (some-> (aget env "GITHUB_DEFAULT_BASE_BRANCH") str string/trim not-empty)
|
||||
"main"))
|
||||
|
||||
(defn- error-reason
|
||||
[error]
|
||||
(let [reason (some-> error ex-data :reason)]
|
||||
(cond
|
||||
(keyword? reason) (name reason)
|
||||
(string? reason) reason
|
||||
:else nil)))
|
||||
|
||||
(defn- <terminate-runtime! [^js self runtime]
|
||||
(set! (.-runtime-events-stream self) nil)
|
||||
(if-not (map? runtime)
|
||||
@@ -382,6 +432,197 @@
|
||||
(:kind body))
|
||||
(http/json-response :sessions/message {:ok true}))))))))))
|
||||
|
||||
(defn- <manual-pr-required-response!
|
||||
[^js self {:keys [user-id head-branch base-branch manual-url reason force? message]}]
|
||||
(p/let [_ (<append-publish-event! self "git.pr.manual"
|
||||
{:by user-id
|
||||
:head-branch head-branch
|
||||
:base-branch base-branch
|
||||
:manual-pr-url manual-url
|
||||
:reason reason})]
|
||||
(http/json-response :sessions/pr
|
||||
{:status "manual-pr-required"
|
||||
:head-branch head-branch
|
||||
:base-branch base-branch
|
||||
:manual-pr-url manual-url
|
||||
:force force?
|
||||
:message message})))
|
||||
|
||||
(defn- <create-pr-response!
|
||||
[^js self {:keys [user-id repo-url head-branch base-branch title description pr-token force? manual-url]}]
|
||||
(-> (p/let [_ (<append-publish-event! self "git.pr.started"
|
||||
{:by user-id
|
||||
:head-branch head-branch
|
||||
:base-branch base-branch})
|
||||
pr-result (source-control/<create-pull-request! (.-env self)
|
||||
pr-token
|
||||
repo-url
|
||||
{:title title
|
||||
:body description
|
||||
:head-branch head-branch
|
||||
:base-branch base-branch})
|
||||
_ (<append-publish-event! self "git.pr.succeeded"
|
||||
{:by user-id
|
||||
:head-branch head-branch
|
||||
:base-branch base-branch
|
||||
:pr-url (:url pr-result)
|
||||
:pr-id (:id pr-result)})]
|
||||
(http/json-response :sessions/pr
|
||||
{:status "pr-created"
|
||||
:head-branch head-branch
|
||||
:base-branch base-branch
|
||||
:pr-url (:url pr-result)
|
||||
:force force?
|
||||
:message "branch pushed and pull request created"}))
|
||||
(p/catch (fn [error]
|
||||
(p/let [_ (<append-publish-event! self "git.pr.failed"
|
||||
{:by user-id
|
||||
:head-branch head-branch
|
||||
:base-branch base-branch
|
||||
:reason (error-reason error)
|
||||
:error (str error)})]
|
||||
(<manual-pr-required-response! self
|
||||
{:user-id user-id
|
||||
:head-branch head-branch
|
||||
:base-branch base-branch
|
||||
:manual-url manual-url
|
||||
:reason "api-failed"
|
||||
:force? force?
|
||||
:message "branch pushed; pull request API failed, use manual URL"}))))))
|
||||
|
||||
(defn- <handle-pr-after-push!
|
||||
[^js self current-session body user-id repo-url head-branch force? create-pr?]
|
||||
(p/let [pr-token (source-control/pr-token (.-env self))
|
||||
base-branch (or (source-control/sanitize-branch-name (:base-branch body))
|
||||
(when (string? pr-token)
|
||||
(source-control/<default-branch! (.-env self)
|
||||
pr-token
|
||||
repo-url))
|
||||
(default-base-branch (.-env self)))]
|
||||
(cond
|
||||
(false? create-pr?)
|
||||
(http/json-response :sessions/pr
|
||||
{:status "pushed"
|
||||
:head-branch head-branch
|
||||
:base-branch base-branch
|
||||
:force force?
|
||||
:message "branch pushed"})
|
||||
|
||||
(not (pr-enabled? current-session))
|
||||
(http/forbidden)
|
||||
|
||||
:else
|
||||
(let [title (or (some-> (:title body) str string/trim not-empty)
|
||||
(str "Agent updates for session " (:id current-session)))
|
||||
description (or (some-> (:body body) str string/trim not-empty)
|
||||
"Automated changes from agent session.")
|
||||
manual-url (source-control/manual-pr-url repo-url head-branch base-branch)]
|
||||
(if-not (string? pr-token)
|
||||
(<manual-pr-required-response! self
|
||||
{:user-id user-id
|
||||
:head-branch head-branch
|
||||
:base-branch base-branch
|
||||
:manual-url manual-url
|
||||
:reason "missing-token"
|
||||
:force? force?
|
||||
:message "branch pushed; create pull request manually"})
|
||||
(<create-pr-response! self
|
||||
{:user-id user-id
|
||||
:repo-url repo-url
|
||||
:head-branch head-branch
|
||||
:base-branch base-branch
|
||||
:title title
|
||||
:description description
|
||||
:pr-token pr-token
|
||||
:force? force?
|
||||
:manual-url manual-url}))))))
|
||||
|
||||
(defn- <perform-pr-push!
|
||||
[^js self current-session body user-id repo-url runtime force? create-pr? push-branch-fn]
|
||||
(let [provider (runtime-provider/resolve-provider (.-env self) runtime)
|
||||
push-token (source-control/push-token (.-env self))
|
||||
head-branch (source-control/resolve-head-branch (:head-branch body)
|
||||
(generated-head-branch (:id current-session)))]
|
||||
(if-not (string? head-branch)
|
||||
(http/bad-request "invalid head branch")
|
||||
(-> (p/let [_ (<append-publish-event! self "git.push.started"
|
||||
{:by user-id
|
||||
:head-branch head-branch
|
||||
:force force?})
|
||||
push-result (push-branch-fn provider
|
||||
runtime
|
||||
{:session-id (:id current-session)
|
||||
:repo-url repo-url
|
||||
:head-branch head-branch
|
||||
:force force?
|
||||
:push-token push-token})
|
||||
_ (<append-publish-event! self "git.push.succeeded"
|
||||
{:by user-id
|
||||
:head-branch head-branch
|
||||
:force force?
|
||||
:remote (:remote push-result)})]
|
||||
(<handle-pr-after-push! self
|
||||
current-session
|
||||
body
|
||||
user-id
|
||||
repo-url
|
||||
head-branch
|
||||
force?
|
||||
create-pr?))
|
||||
(p/catch (fn [error]
|
||||
(p/let [_ (<append-publish-event! self "git.push.failed"
|
||||
{:by user-id
|
||||
:head-branch head-branch
|
||||
:reason (error-reason error)
|
||||
:error (str error)})]
|
||||
(http/error-response (str error) 500))))))))
|
||||
|
||||
(defn- handle-pr [^js self request]
|
||||
(let [push-branch-fn runtime-provider/<push-branch!]
|
||||
(.then (common/read-json request)
|
||||
(fn [result]
|
||||
(let [body (if (nil? result)
|
||||
{}
|
||||
(js->clj result :keywordize-keys true))
|
||||
user-id (user-id-from-request request)
|
||||
force? (true? (:force body))
|
||||
create-pr? (if (contains? body :create-pr)
|
||||
(true? (:create-pr body))
|
||||
true)]
|
||||
(if-not (string? user-id)
|
||||
(http/unauthorized)
|
||||
(p/let [current-session (<get-session self)]
|
||||
(cond
|
||||
(nil? current-session)
|
||||
(http/not-found)
|
||||
|
||||
(terminal-status? (:status current-session))
|
||||
(session-conflict "session is not writable")
|
||||
|
||||
(not (push-enabled? current-session))
|
||||
(http/forbidden)
|
||||
|
||||
:else
|
||||
(let [repo-url (repo-url-from-session current-session)
|
||||
runtime (:runtime current-session)]
|
||||
(cond
|
||||
(not (string? repo-url))
|
||||
(http/bad-request "missing repo url")
|
||||
|
||||
(not (map? runtime))
|
||||
(session-conflict "session runtime unavailable")
|
||||
|
||||
:else
|
||||
(<perform-pr-push! self
|
||||
current-session
|
||||
body
|
||||
user-id
|
||||
repo-url
|
||||
runtime
|
||||
force?
|
||||
create-pr?
|
||||
push-branch-fn)))))))))))
|
||||
|
||||
(defn- handle-cancel [^js self request]
|
||||
(let [user-id (user-id-from-request request)]
|
||||
(if-not (string? user-id)
|
||||
@@ -530,6 +771,9 @@
|
||||
(= path "/__session__/cancel")
|
||||
(handle-cancel self request)
|
||||
|
||||
(= path "/__session__/pr")
|
||||
(handle-pr self request)
|
||||
|
||||
(= path "/__session__/stream")
|
||||
(handle-stream self request)
|
||||
|
||||
|
||||
@@ -4,11 +4,18 @@
|
||||
[body]
|
||||
(when (map? body)
|
||||
(let [attachments (:attachments body)
|
||||
attachments (when (sequential? attachments) (vec attachments))]
|
||||
attachments (when (sequential? attachments) (vec attachments))
|
||||
capabilities (if (map? (:capabilities body))
|
||||
(:capabilities body)
|
||||
{})
|
||||
capabilities (merge {:push-enabled true
|
||||
:pr-enabled true}
|
||||
(select-keys capabilities [:push-enabled :pr-enabled]))]
|
||||
(cond-> {:id (:session-id body)
|
||||
:source {:node-id (:node-id body)
|
||||
:node-title (:node-title body)}
|
||||
:intent {:content (:content body)}
|
||||
:project (:project body)
|
||||
:agent (:agent body)}
|
||||
:agent (:agent body)
|
||||
:capabilities capabilities}
|
||||
(some? attachments) (assoc-in [:intent :attachments] attachments)))))
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
(:require [clojure.string :as string]
|
||||
[lambdaisland.glogi :as log]
|
||||
[logseq.db-sync.worker.agent.sandbox :as sandbox]
|
||||
[logseq.db-sync.worker.agent.source-control :as source-control]
|
||||
[promesa.core :as p]))
|
||||
|
||||
;; -----------------------
|
||||
@@ -386,6 +387,44 @@
|
||||
(escape-shell-single repo-url) "' '" (escape-shell-single repo-dir) "'"
|
||||
" && chmod -R u+rw '" (escape-shell-single repo-dir) "'"))))))
|
||||
|
||||
(defn- classify-push-error
|
||||
[error]
|
||||
(let [data (ex-data error)
|
||||
{:keys [stderr stdout]} (when (map? data) data)
|
||||
message (str (or (ex-message error) "")
|
||||
"\n"
|
||||
(or stderr "")
|
||||
"\n"
|
||||
(or stdout ""))
|
||||
message (string/lower-case message)]
|
||||
(cond
|
||||
(or (string/includes? message "authentication failed")
|
||||
(string/includes? message "could not read username")
|
||||
(string/includes? message "permission denied")
|
||||
(string/includes? message "repository not found"))
|
||||
:auth
|
||||
|
||||
(or (string/includes? message "src refspec")
|
||||
(string/includes? message "does not match any"))
|
||||
:no-commits
|
||||
|
||||
(or (string/includes? message "non-fast-forward")
|
||||
(string/includes? message "[rejected]")
|
||||
(string/includes? message "fetch first"))
|
||||
:remote-rejected
|
||||
|
||||
:else
|
||||
:unknown)))
|
||||
|
||||
(defn- push-command
|
||||
[{:keys [repo-dir remote-url head-branch force]}]
|
||||
(str "set -e; "
|
||||
"cd '" (escape-shell-single repo-dir) "'; "
|
||||
"git rev-parse --is-inside-work-tree >/dev/null; "
|
||||
"git push '" (escape-shell-single remote-url) "' HEAD:refs/heads/"
|
||||
(escape-shell-single head-branch)
|
||||
(when force " --force")))
|
||||
|
||||
(defn session-payload [task]
|
||||
(let [agent (or (get-in task [:agent :provider])
|
||||
(get-in task [:agent :id])
|
||||
@@ -566,6 +605,15 @@
|
||||
(->promise (.exec sandbox cmd))
|
||||
(throw (ex-info "cloudflare sandbox missing exec method" {}))))
|
||||
|
||||
(defn- cloudflare-exec-output
|
||||
[result]
|
||||
{:stdout (or (aget result "stdout") "")
|
||||
:stderr (or (aget result "stderr") "")
|
||||
:exit-code (or (aget result "exitCode")
|
||||
(aget result "exit_code"))
|
||||
:success (let [success (aget result "success")]
|
||||
(if (boolean? success) success true))})
|
||||
|
||||
(defn- <cloudflare-health-once! [sandbox port agent-token]
|
||||
(-> (js/Promise.resolve (<cloudflare-exec! sandbox (cloudflare-health-command port agent-token)))
|
||||
(.then (fn [result]
|
||||
@@ -789,6 +837,7 @@
|
||||
(<provision-runtime! [this session-id task])
|
||||
(<open-events-stream! [this runtime])
|
||||
(<send-message! [this runtime message])
|
||||
(<push-branch! [this runtime opts])
|
||||
(<terminate-runtime! [this runtime]))
|
||||
|
||||
(defrecord SpritesProvider [env]
|
||||
@@ -862,6 +911,57 @@
|
||||
(p/let [_ (sprites-exec-post! env name ["bash" "-lc" script])]
|
||||
true))))
|
||||
|
||||
(<push-branch! [_ runtime opts]
|
||||
(let [name (:sprite-name runtime)
|
||||
session-id (:session-id opts)
|
||||
repo-url (:repo-url opts)
|
||||
head-branch (:head-branch opts)
|
||||
force? (true? (:force opts))
|
||||
repo-dir (get-repo-dir session-id)
|
||||
remote-url (or (source-control/push-remote-url repo-url (:push-token opts))
|
||||
repo-url)]
|
||||
(when-not (string? name)
|
||||
(throw (ex-info "missing sprite-name on runtime" {:runtime runtime})))
|
||||
(when-not (string? repo-dir)
|
||||
(throw (ex-info "missing repo dir for push"
|
||||
{:reason :missing-repo-dir
|
||||
:session-id session-id})))
|
||||
(when-not (string? remote-url)
|
||||
(throw (ex-info "missing remote url for push"
|
||||
{:reason :missing-remote-url
|
||||
:repo-url repo-url})))
|
||||
(when-not (string? head-branch)
|
||||
(throw (ex-info "missing head branch for push"
|
||||
{:reason :missing-branch})))
|
||||
(let [script (push-command {:repo-dir repo-dir
|
||||
:remote-url remote-url
|
||||
:head-branch head-branch
|
||||
:force force?})]
|
||||
(-> (p/let [result (sprites-exec-post! env name ["bash" "-lc" script])
|
||||
{:keys [stdout stderr exit-code]} (sprites-exec-output result)]
|
||||
(when (and (number? exit-code) (not (zero? exit-code)))
|
||||
(throw (ex-info "git push failed"
|
||||
{:reason :git-exit
|
||||
:stdout stdout
|
||||
:stderr stderr
|
||||
:exit-code exit-code})))
|
||||
{:head-branch head-branch
|
||||
:repo-url repo-url
|
||||
:force force?
|
||||
:remote "origin"})
|
||||
(p/catch (fn [error]
|
||||
(p/rejected
|
||||
(if-let [data (ex-data error)]
|
||||
(ex-info (ex-message error)
|
||||
(assoc data
|
||||
:provider "sprites"
|
||||
:reason (or (:reason data)
|
||||
(classify-push-error error))))
|
||||
(ex-info "git push failed"
|
||||
{:provider "sprites"
|
||||
:reason (classify-push-error error)
|
||||
:error (str error)})))))))))
|
||||
|
||||
(<terminate-runtime! [_ runtime]
|
||||
(if-not (string? (:sprite-name runtime))
|
||||
(p/resolved nil)
|
||||
@@ -890,6 +990,12 @@
|
||||
agent-token (local-dev-token env runtime)]
|
||||
(sandbox/<send-message base-url agent-token (:session-id runtime) message)))
|
||||
|
||||
(<push-branch! [_ _runtime _opts]
|
||||
(p/rejected
|
||||
(ex-info "local-dev runtime provider does not support managed git push"
|
||||
{:reason :unsupported
|
||||
:provider "local-dev"})))
|
||||
|
||||
(<terminate-runtime! [_ runtime]
|
||||
(let [base-url (local-dev-base-url env runtime)
|
||||
agent-token (local-dev-token env runtime)
|
||||
@@ -965,6 +1071,61 @@
|
||||
(let [sandbox (cloudflare-sandbox env sandbox-id)]
|
||||
(<cloudflare-send-message! sandbox port agent-token session-id message))))
|
||||
|
||||
(<push-branch! [_ runtime opts]
|
||||
(let [sandbox-id (:sandbox-id runtime)
|
||||
session-id (:session-id opts)
|
||||
repo-url (:repo-url opts)
|
||||
head-branch (:head-branch opts)
|
||||
force? (true? (:force opts))
|
||||
port (or (:sandbox-port runtime) (cloudflare-agent-port env))
|
||||
repo-dir (get-repo-dir session-id)
|
||||
remote-url (or (source-control/push-remote-url repo-url (:push-token opts))
|
||||
repo-url)]
|
||||
(when-not (string? sandbox-id)
|
||||
(throw (ex-info "missing sandbox-id on runtime" {:runtime runtime})))
|
||||
(when-not (string? repo-dir)
|
||||
(throw (ex-info "missing repo dir for push"
|
||||
{:reason :missing-repo-dir
|
||||
:session-id session-id})))
|
||||
(when-not (string? remote-url)
|
||||
(throw (ex-info "missing remote url for push"
|
||||
{:reason :missing-remote-url
|
||||
:repo-url repo-url})))
|
||||
(when-not (string? head-branch)
|
||||
(throw (ex-info "missing head branch for push"
|
||||
{:reason :missing-branch})))
|
||||
(let [sandbox (cloudflare-sandbox env sandbox-id)
|
||||
script (push-command {:repo-dir repo-dir
|
||||
:remote-url remote-url
|
||||
:head-branch head-branch
|
||||
:force force?})]
|
||||
(-> (p/let [result (<cloudflare-exec! sandbox script)
|
||||
{:keys [stdout stderr exit-code success]} (cloudflare-exec-output result)]
|
||||
(when (or (false? success)
|
||||
(and (number? exit-code) (not (zero? exit-code))))
|
||||
(throw (ex-info "git push failed"
|
||||
{:reason :git-exit
|
||||
:stdout stdout
|
||||
:stderr stderr
|
||||
:exit-code exit-code})))
|
||||
{:head-branch head-branch
|
||||
:repo-url repo-url
|
||||
:force force?
|
||||
:remote "origin"
|
||||
:sandbox-port port})
|
||||
(p/catch (fn [error]
|
||||
(p/rejected
|
||||
(if-let [data (ex-data error)]
|
||||
(ex-info (ex-message error)
|
||||
(assoc data
|
||||
:provider "cloudflare"
|
||||
:reason (or (:reason data)
|
||||
(classify-push-error error))))
|
||||
(ex-info "git push failed"
|
||||
{:provider "cloudflare"
|
||||
:reason (classify-push-error error)
|
||||
:error (str error)})))))))))
|
||||
|
||||
(<terminate-runtime! [_ runtime]
|
||||
(let [sandbox-id (:sandbox-id runtime)
|
||||
session-id (:session-id runtime)]
|
||||
|
||||
192
deps/db-sync/src/logseq/db_sync/worker/agent/source_control.cljs
vendored
Normal file
192
deps/db-sync/src/logseq/db_sync/worker/agent/source_control.cljs
vendored
Normal file
@@ -0,0 +1,192 @@
|
||||
(ns logseq.db-sync.worker.agent.source-control
|
||||
(:require [clojure.string :as string]
|
||||
[promesa.core :as p]))
|
||||
|
||||
(defn- non-empty-str
|
||||
[value]
|
||||
(when (string? value)
|
||||
(let [trimmed (string/trim value)]
|
||||
(when-not (string/blank? trimmed)
|
||||
trimmed))))
|
||||
|
||||
(defn- strip-git-suffix
|
||||
[repo-name]
|
||||
(when-let [repo-name (non-empty-str repo-name)]
|
||||
(string/replace repo-name #"\.git$" "")))
|
||||
|
||||
(defn- github-https-ref
|
||||
[repo-url]
|
||||
(when-let [[_ owner repo]
|
||||
(some->> repo-url
|
||||
non-empty-str
|
||||
(re-matches #"^https?://github\.com/([^/]+)/([^/]+?)(?:\.git)?/?$"))]
|
||||
{:provider "github"
|
||||
:owner owner
|
||||
:name (strip-git-suffix repo)}))
|
||||
|
||||
(defn- github-ssh-ref
|
||||
[repo-url]
|
||||
(when-let [[_ owner repo]
|
||||
(some->> repo-url
|
||||
non-empty-str
|
||||
(re-matches #"^git@github\.com:([^/]+)/([^/]+?)(?:\.git)?$"))]
|
||||
{:provider "github"
|
||||
:owner owner
|
||||
:name (strip-git-suffix repo)}))
|
||||
|
||||
(defn- github-ssh-url-ref
|
||||
[repo-url]
|
||||
(when-let [[_ owner repo]
|
||||
(some->> repo-url
|
||||
non-empty-str
|
||||
(re-matches #"^ssh://git@github\.com/([^/]+)/([^/]+?)(?:\.git)?/?$"))]
|
||||
{:provider "github"
|
||||
:owner owner
|
||||
:name (strip-git-suffix repo)}))
|
||||
|
||||
(defn repo-ref
|
||||
[repo-url]
|
||||
(let [repo-url (non-empty-str repo-url)]
|
||||
(some->> [(github-https-ref repo-url)
|
||||
(github-ssh-ref repo-url)
|
||||
(github-ssh-url-ref repo-url)]
|
||||
(remove nil?)
|
||||
first)))
|
||||
|
||||
(defn sanitize-branch-name
|
||||
[branch]
|
||||
(let [branch (non-empty-str branch)]
|
||||
(when (and branch
|
||||
(not= branch "HEAD")
|
||||
(not (string/starts-with? branch "/"))
|
||||
(not (string/ends-with? branch "/"))
|
||||
(not (string/starts-with? branch "."))
|
||||
(not (string/ends-with? branch "."))
|
||||
(not (string/includes? branch " "))
|
||||
(not (string/includes? branch ".."))
|
||||
(not (string/includes? branch "@{"))
|
||||
(not (re-find #"[~^:?*\[\]\\]" branch)))
|
||||
branch)))
|
||||
|
||||
(defn resolve-head-branch
|
||||
[requested fallback]
|
||||
(or (sanitize-branch-name requested)
|
||||
(sanitize-branch-name fallback)))
|
||||
|
||||
(defn manual-pr-url
|
||||
[repo-url head-branch base-branch]
|
||||
(when-let [{:keys [provider owner name]} (repo-ref repo-url)]
|
||||
(case provider
|
||||
"github"
|
||||
(str "https://github.com/"
|
||||
owner
|
||||
"/"
|
||||
name
|
||||
"/pull/new/"
|
||||
(js/encodeURIComponent (or base-branch "main"))
|
||||
"..."
|
||||
(js/encodeURIComponent (or head-branch "")))
|
||||
nil)))
|
||||
|
||||
(defn- api-base-url
|
||||
[^js env]
|
||||
(or (some-> (aget env "GITHUB_API_BASE") non-empty-str)
|
||||
"https://api.github.com"))
|
||||
|
||||
(defn pr-token
|
||||
[^js env]
|
||||
(or (some-> (aget env "GITHUB_PR_TOKEN") non-empty-str)
|
||||
(some-> (aget env "GITHUB_TOKEN") non-empty-str)))
|
||||
|
||||
(defn push-token
|
||||
[^js env]
|
||||
(or (some-> (aget env "GITHUB_PUSH_TOKEN") non-empty-str)
|
||||
(some-> (aget env "GITHUB_TOKEN") non-empty-str)))
|
||||
|
||||
(defn push-remote-url
|
||||
[repo-url token]
|
||||
(when-let [{:keys [provider owner name]} (repo-ref repo-url)]
|
||||
(when (= "github" provider)
|
||||
(if (string? token)
|
||||
(str "https://x-access-token:" (js/encodeURIComponent token) "@github.com/" owner "/" name ".git")
|
||||
(str "https://github.com/" owner "/" name ".git")))))
|
||||
|
||||
(defn- parse-json-safe
|
||||
[text]
|
||||
(if-not (string? text)
|
||||
{}
|
||||
(try
|
||||
(js->clj (js/JSON.parse text) :keywordize-keys true)
|
||||
(catch :default _
|
||||
{}))))
|
||||
|
||||
(defn <default-branch!
|
||||
[^js env token repo-url]
|
||||
(if-not (string? token)
|
||||
(p/resolved nil)
|
||||
(if-let [{:keys [provider owner name]} (repo-ref repo-url)]
|
||||
(if-not (= "github" provider)
|
||||
(p/resolved nil)
|
||||
(let [url (str (api-base-url env) "/repos/" owner "/" name)
|
||||
headers (doto (js/Headers.)
|
||||
(.set "accept" "application/vnd.github+json")
|
||||
(.set "authorization" (str "Bearer " token))
|
||||
(.set "x-github-api-version" "2022-11-28"))]
|
||||
(p/let [resp (js/fetch url #js {:method "GET" :headers headers})
|
||||
status (.-status resp)
|
||||
text (.text resp)
|
||||
payload (parse-json-safe text)]
|
||||
(if (<= 200 status 299)
|
||||
(some-> (:default_branch payload) non-empty-str)
|
||||
nil))))
|
||||
(p/resolved nil))))
|
||||
|
||||
(defn <create-pull-request!
|
||||
[^js env token repo-url {:keys [title body head-branch base-branch draft]}]
|
||||
(cond
|
||||
(not (string? token))
|
||||
(p/rejected (ex-info "missing github token" {:reason :missing-token}))
|
||||
|
||||
(not (string? title))
|
||||
(p/rejected (ex-info "missing pull request title" {:reason :invalid-request}))
|
||||
|
||||
(not (string? body))
|
||||
(p/rejected (ex-info "missing pull request body" {:reason :invalid-request}))
|
||||
|
||||
:else
|
||||
(if-let [{:keys [provider owner name]} (repo-ref repo-url)]
|
||||
(if-not (= "github" provider)
|
||||
(p/rejected (ex-info "unsupported scm provider" {:reason :unsupported-provider
|
||||
:provider provider}))
|
||||
(let [url (str (api-base-url env) "/repos/" owner "/" name "/pulls")
|
||||
payload (cond-> {:title title
|
||||
:body body
|
||||
:head (or head-branch "")
|
||||
:base (or base-branch "main")}
|
||||
(true? draft) (assoc :draft true))
|
||||
headers (doto (js/Headers.)
|
||||
(.set "accept" "application/vnd.github+json")
|
||||
(.set "content-type" "application/json")
|
||||
(.set "authorization" (str "Bearer " token))
|
||||
(.set "x-github-api-version" "2022-11-28"))]
|
||||
(p/let [resp (js/fetch url #js {:method "POST"
|
||||
:headers headers
|
||||
:body (js/JSON.stringify (clj->js payload))})
|
||||
status (.-status resp)
|
||||
text (.text resp)
|
||||
parsed (parse-json-safe text)]
|
||||
(if (<= 200 status 299)
|
||||
{:id (:number parsed)
|
||||
:url (:html_url parsed)
|
||||
:state (cond
|
||||
(true? (:draft parsed)) "draft"
|
||||
(= "closed" (:state parsed)) "closed"
|
||||
:else "open")
|
||||
:head-branch (or (get-in parsed [:head :ref]) head-branch)
|
||||
:base-branch (or (get-in parsed [:base :ref]) base-branch)}
|
||||
(p/rejected (ex-info "create pull request failed"
|
||||
{:reason :api-error
|
||||
:status status
|
||||
:body parsed}))))))
|
||||
(p/rejected (ex-info "invalid repo url" {:reason :invalid-repo-url
|
||||
:repo-url repo-url})))))
|
||||
@@ -139,6 +139,25 @@
|
||||
(forward-request stub do-url "GET" headers nil))
|
||||
(http/error-response "server error" 500)))))
|
||||
|
||||
(defn- handle-pr [{:keys [env request url claims route]}]
|
||||
(let [session-id (get-in route [:path-params :session-id])]
|
||||
(if-not (string? session-id)
|
||||
(http/bad-request "invalid session id")
|
||||
(.then (common/read-json request)
|
||||
(fn [result]
|
||||
(let [raw-body (if (nil? result)
|
||||
{}
|
||||
(js->clj result :keywordize-keys true))
|
||||
body (http/coerce-http-request :sessions/pr raw-body)]
|
||||
(if (nil? body)
|
||||
(http/bad-request "invalid body")
|
||||
(if-let [^js stub (session-stub env session-id)]
|
||||
(let [headers (base-headers request claims)
|
||||
body-json (js/JSON.stringify (clj->js body))
|
||||
do-url (str (.-origin url) "/__session__/pr")]
|
||||
(forward-request stub do-url "POST" headers body-json))
|
||||
(http/error-response "server error" 500)))))))))
|
||||
|
||||
(defn handle [{:keys [route] :as ctx}]
|
||||
(case (:handler route)
|
||||
:sessions/create (handle-create ctx)
|
||||
@@ -148,6 +167,7 @@
|
||||
:sessions/resume (handle-control ctx "/__session__/resume")
|
||||
:sessions/interrupt (handle-control ctx "/__session__/interrupt")
|
||||
:sessions/cancel (handle-cancel ctx)
|
||||
:sessions/pr (handle-pr ctx)
|
||||
:sessions/events (handle-events ctx)
|
||||
:sessions/stream (handle-stream ctx)
|
||||
(http/not-found)))
|
||||
|
||||
@@ -31,6 +31,7 @@
|
||||
["/resume" {:methods {"POST" :sessions/resume}}]
|
||||
["/interrupt" {:methods {"POST" :sessions/interrupt}}]
|
||||
["/cancel" {:methods {"POST" :sessions/cancel}}]
|
||||
["/pr" {:methods {"POST" :sessions/pr}}]
|
||||
["/events" {:methods {"GET" :sessions/events}}]
|
||||
["/stream" {:methods {"GET" :sessions/stream}}]]]])
|
||||
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
(ns logseq.db-sync.agent-do-test
|
||||
(:require [cljs.test :refer [async deftest is testing]]
|
||||
[clojure.string :as string]
|
||||
[logseq.db-sync.worker.agent.do :as agent-do]))
|
||||
[logseq.db-sync.worker.agent.do :as agent-do]
|
||||
[logseq.db-sync.worker.agent.runtime-provider :as runtime-provider]))
|
||||
|
||||
(defn- make-agent-storage []
|
||||
(let [data (js/Map.)]
|
||||
@@ -584,3 +585,76 @@
|
||||
(.catch (fn [error]
|
||||
(is false (str "unexpected error: " error))
|
||||
(done))))))))
|
||||
|
||||
(deftest pr-endpoint-requires-authenticated-user-test
|
||||
(testing "session publish endpoint requires x-user-id header"
|
||||
(async done
|
||||
(let [self (make-self #js {})
|
||||
headers {"content-type" "application/json"}]
|
||||
(-> (.put (.-storage self)
|
||||
"session"
|
||||
(clj->js {:id "sess-pr-auth"
|
||||
:status "running"
|
||||
:task {:project {:repo-url "https://github.com/example/repo"}}
|
||||
:runtime {:provider "local-dev"
|
||||
:session-id "sess-pr-auth"}
|
||||
:audit {}
|
||||
:created-at 0
|
||||
:updated-at 0}))
|
||||
(.then (fn [_]
|
||||
(agent-do/handle-fetch self
|
||||
(json-request "http://db-sync.local/__session__/pr"
|
||||
"POST"
|
||||
{:create-pr false
|
||||
:head-branch "m14/pr-auth"}
|
||||
headers))))
|
||||
(.then (fn [resp]
|
||||
(is (= 401 (.-status resp)))
|
||||
(done)))
|
||||
(.catch (fn [error]
|
||||
(is false (str "unexpected error: " error))
|
||||
(done))))))))
|
||||
|
||||
(deftest pr-endpoint-push-only-success-test
|
||||
(testing "session publish endpoint supports push-only flow"
|
||||
(async done
|
||||
(let [env #js {"AGENT_RUNTIME_PROVIDER" "local-dev"}
|
||||
self (make-self env)
|
||||
headers {"content-type" "application/json"
|
||||
"x-user-id" "user-1"}]
|
||||
(-> (.put (.-storage self)
|
||||
"session"
|
||||
(clj->js {:id "sess-pr-push-only"
|
||||
:status "running"
|
||||
:task {:project {:repo-url "https://github.com/example/repo"}}
|
||||
:runtime {:provider "local-dev"
|
||||
:session-id "sess-pr-push-only"}
|
||||
:audit {}
|
||||
:created-at 0
|
||||
:updated-at 0}))
|
||||
(.then (fn [_]
|
||||
(with-redefs [runtime-provider/<push-branch!
|
||||
(fn [_provider _runtime opts]
|
||||
(js/Promise.resolve
|
||||
{:head-branch (:head-branch opts)
|
||||
:repo-url (:repo-url opts)
|
||||
:force (:force opts)
|
||||
:remote "origin"}))]
|
||||
(agent-do/handle-fetch self
|
||||
(json-request "http://db-sync.local/__session__/pr"
|
||||
"POST"
|
||||
{:create-pr false
|
||||
:head-branch "m14/push-only"
|
||||
:force true}
|
||||
headers)))))
|
||||
(.then (fn [resp]
|
||||
(is (= 200 (.-status resp)))
|
||||
(.then (<json resp)
|
||||
(fn [body]
|
||||
(is (= "pushed" (:status body)))
|
||||
(is (= "m14/push-only" (:head-branch body)))
|
||||
(is (= true (:force body)))
|
||||
(done)))))
|
||||
(.catch (fn [error]
|
||||
(is false (str "unexpected error: " error))
|
||||
(done))))))))
|
||||
|
||||
@@ -44,5 +44,24 @@
|
||||
:repo-url "https://github.com/example/repo"}
|
||||
:agent {:provider "codex"
|
||||
:api-token "token-123"
|
||||
:auth-json "{\"tokens\":{\"access_token\":\"abc\"}}"}}
|
||||
:auth-json "{\"tokens\":{\"access_token\":\"abc\"}}"}
|
||||
:capabilities {:push-enabled true
|
||||
:pr-enabled true}}
|
||||
normalized)))))
|
||||
|
||||
(deftest sessions-pr-coerce-test
|
||||
(testing "accepts sessions/pr request payload"
|
||||
(let [body {:title "feat: add m14 publish"
|
||||
:body "This change enables push + optional PR."
|
||||
:head-branch "m14/publish"
|
||||
:base-branch "main"
|
||||
:create-pr true
|
||||
:force false}
|
||||
coerced (http/coerce-http-request :sessions/pr body)]
|
||||
(is (= body coerced))))
|
||||
(testing "accepts empty sessions/pr payload (defaults resolved in handler)"
|
||||
(is (= {} (http/coerce-http-request :sessions/pr {}))))
|
||||
(testing "rejects invalid sessions/pr payload"
|
||||
(is (nil? (http/coerce-http-request :sessions/pr {:create-pr "yes"})))
|
||||
(is (nil? (http/coerce-http-request :sessions/pr {:force "no"})))
|
||||
(is (nil? (http/coerce-http-request :sessions/pr {:head-branch 42})))))
|
||||
|
||||
@@ -184,6 +184,82 @@
|
||||
(is false (str "unexpected error: " error))
|
||||
(done)))))))
|
||||
|
||||
(deftest local-dev-provider-push-branch-unsupported-test
|
||||
(async done
|
||||
(let [env #js {"SANDBOX_AGENT_URL" "http://127.0.0.1:2468"}
|
||||
provider (runtime-provider/create-provider env "local-dev")
|
||||
runtime {:provider "local-dev"
|
||||
:base-url "http://127.0.0.1:2468"
|
||||
:session-id "sess-push-unsupported"}]
|
||||
(-> (runtime-provider/<push-branch! provider
|
||||
runtime
|
||||
{:session-id "sess-push-unsupported"
|
||||
:repo-url "https://github.com/example/repo"
|
||||
:head-branch "feature/m14"
|
||||
:force false
|
||||
:push-token "token-1"})
|
||||
(.then (fn [_]
|
||||
(is false "expected local-dev push to reject as unsupported")
|
||||
(done)))
|
||||
(.catch (fn [error]
|
||||
(let [data (ex-data error)]
|
||||
(is (= :unsupported (:reason data)))
|
||||
(is (= "local-dev" (:provider data))))
|
||||
(done)))))))
|
||||
|
||||
(deftest sprites-provider-push-branch-command-test
|
||||
(async done
|
||||
(let [captured (atom nil)
|
||||
env #js {"SPRITE_TOKEN" "sprite-token"}
|
||||
provider (runtime-provider/create-provider env "sprites")
|
||||
runtime {:provider "sprites"
|
||||
:sprite-name "sprite-1"
|
||||
:sandbox-port 2468
|
||||
:session-id "sess-push"}
|
||||
original-fetch js/fetch]
|
||||
(set! js/fetch
|
||||
(fn [request init]
|
||||
(let [url (fetch-url request)
|
||||
method (fetch-method request init)]
|
||||
(if (and (= "POST" method)
|
||||
(string/includes? url "/v1/sprites/sprite-1/exec"))
|
||||
(let [parsed (js/URL. url)
|
||||
cmds (vec (.getAll (.-searchParams parsed) "cmd"))
|
||||
script (nth cmds 2 nil)]
|
||||
(reset! captured {:url url
|
||||
:method method
|
||||
:script script})
|
||||
(js/Promise.resolve
|
||||
(js/Response.
|
||||
(js/JSON.stringify #js {:ok true})
|
||||
#js {:status 200
|
||||
:headers #js {"content-type" "application/json"}})))
|
||||
(js/Promise.resolve
|
||||
(js/Response.
|
||||
(js/JSON.stringify #js {:error "unexpected request"})
|
||||
#js {:status 500
|
||||
:headers #js {"content-type" "application/json"}}))))))
|
||||
(-> (runtime-provider/<push-branch! provider
|
||||
runtime
|
||||
{:session-id "sess-push"
|
||||
:repo-url "https://github.com/example/repo"
|
||||
:head-branch "feature/m14"
|
||||
:force true
|
||||
:push-token "token-1"})
|
||||
(.then (fn [result]
|
||||
(set! js/fetch original-fetch)
|
||||
(is (= "feature/m14" (:head-branch result)))
|
||||
(is (= "https://github.com/example/repo" (:repo-url result)))
|
||||
(is (= true (:force result)))
|
||||
(is (string/includes? (:script @captured) "git push"))
|
||||
(is (string/includes? (:script @captured) "feature/m14"))
|
||||
(is (string/includes? (:script @captured) "x-access-token:token-1"))
|
||||
(done)))
|
||||
(.catch (fn [error]
|
||||
(set! js/fetch original-fetch)
|
||||
(is false (str "unexpected error: " error))
|
||||
(done)))))))
|
||||
|
||||
(deftest sprites-provider-send-message-test
|
||||
(async done
|
||||
(let [captured (atom nil)
|
||||
|
||||
35
deps/db-sync/test/logseq/db_sync/agent_source_control_test.cljs
vendored
Normal file
35
deps/db-sync/test/logseq/db_sync/agent_source_control_test.cljs
vendored
Normal file
@@ -0,0 +1,35 @@
|
||||
(ns logseq.db-sync.agent-source-control-test
|
||||
(:require [cljs.test :refer [async deftest is testing]]
|
||||
[logseq.db-sync.worker.agent.source-control :as source-control]))
|
||||
|
||||
(deftest parse-github-repo-test
|
||||
(testing "parses github https and ssh repo urls"
|
||||
(is (= {:provider "github" :owner "logseq" :name "web"}
|
||||
(source-control/repo-ref "https://github.com/logseq/web.git")))
|
||||
(is (= {:provider "github" :owner "logseq" :name "web"}
|
||||
(source-control/repo-ref "git@github.com:logseq/web.git")))))
|
||||
|
||||
(deftest manual-pr-url-test
|
||||
(testing "builds github manual pull request url"
|
||||
(is (= "https://github.com/logseq/web/pull/new/main...m14%2Fbranch"
|
||||
(source-control/manual-pr-url
|
||||
"https://github.com/logseq/web.git"
|
||||
"m14/branch"
|
||||
"main")))))
|
||||
|
||||
(deftest create-pr-rejects-missing-token-test
|
||||
(async done
|
||||
(-> (source-control/<create-pull-request! #js {}
|
||||
nil
|
||||
"https://github.com/logseq/web.git"
|
||||
{:title "M14 publish"
|
||||
:body "M14 publish body"
|
||||
:head-branch "m14/publish"
|
||||
:base-branch "main"})
|
||||
(.then (fn [_]
|
||||
(is false "expected create-pr to reject without token")
|
||||
(done)))
|
||||
(.catch (fn [error]
|
||||
(let [data (ex-data error)]
|
||||
(is (= :missing-token (:reason data))))
|
||||
(done))))))
|
||||
@@ -3,8 +3,10 @@
|
||||
[logseq.db-sync.agent-do-test]
|
||||
[logseq.db-sync.agent-request-test]
|
||||
[logseq.db-sync.agent-runtime-provider-test]
|
||||
[logseq.db-sync.agent-source-control-test]
|
||||
[logseq.db-sync.node-config-test]
|
||||
[logseq.db-sync.platform-test]
|
||||
[logseq.db-sync.worker-routes-test]
|
||||
[shadow.test :as st]
|
||||
[shadow.test.env :as env]))
|
||||
|
||||
|
||||
@@ -80,4 +80,7 @@
|
||||
(is (= "session-11" (get-in match [:path-params :session-id]))))
|
||||
(let [match (routes/match-route "POST" "/sessions/session-12/interrupt")]
|
||||
(is (= :sessions/interrupt (:handler match)))
|
||||
(is (= "session-12" (get-in match [:path-params :session-id]))))))
|
||||
(is (= "session-12" (get-in match [:path-params :session-id]))))
|
||||
(let [match (routes/match-route "POST" "/sessions/session-13/pr")]
|
||||
(is (= :sessions/pr (:handler match)))
|
||||
(is (= "session-13" (get-in match [:path-params :session-id]))))))
|
||||
|
||||
3
deps/db-sync/worker/wrangler.toml
vendored
3
deps/db-sync/worker/wrangler.toml
vendored
@@ -52,6 +52,7 @@ new_sqlite_classes = [ "Sandbox" ]
|
||||
COGNITO_JWKS_URL = "https://cognito-idp.us-east-1.amazonaws.com/us-east-1_dtagLnju8/.well-known/jwks.json"
|
||||
COGNITO_ISSUER = "https://cognito-idp.us-east-1.amazonaws.com/us-east-1_dtagLnju8"
|
||||
COGNITO_CLIENT_ID = "69cs1lgme7p8kbgld8n5kseii6"
|
||||
GITHUB_DEFAULT_BASE_BRANCH = "main"
|
||||
|
||||
[env.staging]
|
||||
name = "logseq-sync-staging"
|
||||
@@ -60,6 +61,7 @@ name = "logseq-sync-staging"
|
||||
COGNITO_JWKS_URL = "https://cognito-idp.us-east-1.amazonaws.com/us-east-1_dtagLnju8/.well-known/jwks.json"
|
||||
COGNITO_ISSUER = "https://cognito-idp.us-east-1.amazonaws.com/us-east-1_dtagLnju8"
|
||||
COGNITO_CLIENT_ID = "69cs1lgme7p8kbgld8n5kseii6"
|
||||
GITHUB_DEFAULT_BASE_BRANCH = "main"
|
||||
|
||||
[[env.staging.durable_objects.bindings]]
|
||||
name = "LOGSEQ_SYNC_DO"
|
||||
@@ -105,6 +107,7 @@ name = "logseq-sync-prod"
|
||||
COGNITO_JWKS_URL = "https://cognito-idp.us-east-1.amazonaws.com/us-east-1_dtagLnju8/.well-known/jwks.json"
|
||||
COGNITO_ISSUER = "https://cognito-idp.us-east-1.amazonaws.com/us-east-1_dtagLnju8"
|
||||
COGNITO_CLIENT_ID = "69cs1lgme7p8kbgld8n5kseii6"
|
||||
GITHUB_DEFAULT_BASE_BRANCH = "main"
|
||||
|
||||
[[env.prod.durable_objects.bindings]]
|
||||
name = "LOGSEQ_SYNC_DO"
|
||||
|
||||
359
docs/agent-guide/004-m14-git-push-pr.md
Normal file
359
docs/agent-guide/004-m14-git-push-pr.md
Normal file
@@ -0,0 +1,359 @@
|
||||
# M14 Git Push and Optional PR Implementation Plan
|
||||
|
||||
Goal: Enable every authenticated collaborator to trigger agent-session git push and optional pull request creation with no role-based gating.
|
||||
|
||||
Architecture: Add a session-level publish endpoint that orchestrates push and PR from the existing Agent Session Durable Object. Reuse runtime-provider abstractions for sandbox execution and add a small source-control provider layer for GitHub PR creation and manual fallback links. Keep session/event APIs backward compatible and expose publish status through the existing event stream.
|
||||
|
||||
Tech Stack: ClojureScript Worker + Durable Objects, existing runtime providers (`sprites`, `cloudflare`, `local-dev`), Git CLI in sandbox, GitHub REST API, Malli request and response coercion.
|
||||
|
||||
Related: Builds on `deps/db-sync/docs/milestones/agents/14-m14-git-push-and-optional-pr.md`, `deps/db-sync/src/logseq/db_sync/worker/agent/do.cljs`, and `deps/db-sync/src/logseq/db_sync/worker/agent/runtime_provider.cljs`.
|
||||
|
||||
## Problem statement
|
||||
Current agent milestones stop at local commit behavior inside session sandboxes and do not provide a first-class publish flow.
|
||||
|
||||
Users cannot reliably complete the delivery loop from task execution to remote branch and PR without manual out-of-band steps.
|
||||
|
||||
M14 must enable push and PR for all authenticated collaborators, not only special roles, while preserving existing session behavior.
|
||||
|
||||
The existing architecture already has what we need for orchestration, including session-scoped Durable Object state, runtime-provider dispatch, and event streaming.
|
||||
|
||||
The missing pieces are publish-specific API routes, source-control provider logic, runtime push execution hooks, and user-facing invocation paths.
|
||||
|
||||
## Testing Plan
|
||||
I will follow @test-driven-development for every implementation slice, running RED, GREEN, and REFACTOR in small batches.
|
||||
|
||||
I will add route-level tests to verify `/sessions/:session-id/pr` matching and handler wiring in `deps/db-sync/test/logseq/db_sync/worker_routes_test.cljs`.
|
||||
|
||||
I will add schema coercion tests for publish request and response payloads in `deps/db-sync/test/logseq/db_sync/agent_request_test.cljs` and new targeted schema tests if needed.
|
||||
|
||||
I will add runtime-provider tests for push command assembly and provider-specific execution behavior in `deps/db-sync/test/logseq/db_sync/agent_runtime_provider_test.cljs`.
|
||||
|
||||
I will add Agent Session DO behavior tests for push success, push failure, PR success, manual PR fallback, and no-role-gate behavior in `deps/db-sync/test/logseq/db_sync/agent_do_test.cljs`.
|
||||
|
||||
I will add source-control unit tests for repo URL parsing, branch sanitization, PR URL fallback, and GitHub API error mapping in a new test namespace under `deps/db-sync/test/logseq/db_sync/`.
|
||||
|
||||
I will add a lightweight frontend behavior test or deterministic handler-level test for publish action dispatch if the current frontend test harness supports it, otherwise I will document manual verification steps.
|
||||
|
||||
I will run focused tests after each slice and run the full db-sync test command before completion.
|
||||
|
||||
NOTE: I will write *all* tests before I add any implementation behavior.
|
||||
|
||||
## Background-agents inspiration
|
||||
The `background-agents` implementation demonstrates a clean separation between push transport and PR API calls.
|
||||
|
||||
The same codebase also demonstrates resilient branch resolution and manual PR fallback when user OAuth is not available.
|
||||
|
||||
We will reuse the architectural pattern but adapt it to db-sync’s simpler session model and current runtime-provider contracts.
|
||||
|
||||
| Pattern from background-agents | Where it appears | M14 decision in db-sync |
|
||||
| --- | --- | --- |
|
||||
| Provider abstraction for source control operations. | `packages/control-plane/src/source-control/types.ts`. | Adopt with a smaller GitHub-first interface in `deps/db-sync/src/logseq/db_sync/worker/agent/source_control.cljs`. |
|
||||
| Session endpoint that performs push then PR. | `POST /sessions/:id/pr` in `packages/control-plane/src/router.ts`. | Adopt with `POST /sessions/:session-id/pr` in db-sync routes and DO handler. |
|
||||
| Push auth separated from PR auth. | `generatePushAuth()` and user OAuth in `durable-object.ts`. | Adapt by using server-side configured credentials for both push and PR in M14, with manual PR fallback when PR creds are missing. |
|
||||
| Branch sanitization and precedence rules. | `branch-resolution.ts`. | Adopt equivalent sanitization and head-branch resolution helpers in db-sync. |
|
||||
| Manual PR fallback artifact and URL return. | `buildManualPrFallbackResponse(...)`. | Adopt API-level fallback response and stream events, without artifact table because db-sync session state is KV-like. |
|
||||
| Push completion via sandbox event resolver map. | `pendingPushResolvers` in `durable-object.ts`. | Skip for now because db-sync can execute push synchronously through runtime-provider command execution. |
|
||||
|
||||
## Scope
|
||||
M14 will add a publish endpoint that can push a session branch and optionally create a pull request.
|
||||
|
||||
M14 will enforce collaborator-level access by requiring authentication only, with no manager or member role gate.
|
||||
|
||||
M14 will support GitHub first for PR API integration and use manual PR URL fallback when PR API credentials are absent.
|
||||
|
||||
M14 will keep existing session create, stream, pause, resume, interrupt, and cancel behaviors unchanged for callers that do not use publish.
|
||||
|
||||
M14 will emit explicit publish lifecycle events so UI and audit tooling can observe progress and failures.
|
||||
|
||||
## Non-goals
|
||||
M14 will not implement auto-merge, reviewer assignment workflows, or repository hosting providers beyond GitHub.
|
||||
|
||||
M14 will not introduce a new participant table or user OAuth token storage model in db-sync.
|
||||
|
||||
M14 will not rework session runtime lifecycle semantics beyond what is needed to execute push and PR safely.
|
||||
|
||||
## Target API contract
|
||||
The concrete publish contract for M14 is shown below.
|
||||
|
||||
| Method | Path | Purpose |
|
||||
| --- | --- | --- |
|
||||
| `POST` | `/sessions/:session-id/pr` | Push branch and optionally create PR. |
|
||||
|
||||
| Request field | Type | Required | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| `title` | string | no | Required when `create-pr` is true, default generated from task title otherwise. |
|
||||
| `body` | string | no | Required when `create-pr` is true, default generated template otherwise. |
|
||||
| `base-branch` | string | no | Defaults to provider default branch or repo default branch. |
|
||||
| `head-branch` | string | no | Defaults to current branch if safe, otherwise generated branch name. |
|
||||
| `create-pr` | boolean | no | Defaults to true, false means push-only. |
|
||||
| `force` | boolean | no | Defaults to false for safe push behavior. |
|
||||
|
||||
| Response field | Type | Notes |
|
||||
| --- | --- | --- |
|
||||
| `status` | string | `pushed`, `pr-created`, `manual-pr-required`, or `error`. |
|
||||
| `head-branch` | string | Final pushed branch. |
|
||||
| `base-branch` | string | PR target branch when applicable. |
|
||||
| `pr-url` | string | Present when PR is created. |
|
||||
| `manual-pr-url` | string | Present when fallback is required. |
|
||||
| `message` | string | Human-readable summary for UI and logs. |
|
||||
|
||||
## Event model additions
|
||||
M14 will append events using the existing session event envelope.
|
||||
|
||||
| Event type | Data keys | When emitted |
|
||||
| --- | --- | --- |
|
||||
| `git.push.started` | `by`, `head-branch`, `force` | Before push execution begins. |
|
||||
| `git.push.succeeded` | `by`, `head-branch`, `remote` | After push succeeds. |
|
||||
| `git.push.failed` | `by`, `head-branch`, `error`, `reason` | After push fails. |
|
||||
| `git.pr.started` | `by`, `head-branch`, `base-branch` | Before PR creation call. |
|
||||
| `git.pr.succeeded` | `by`, `pr-url`, `head-branch`, `base-branch` | After PR is created. |
|
||||
| `git.pr.manual` | `by`, `manual-pr-url`, `head-branch`, `base-branch`, `reason` | When PR API is skipped or unavailable. |
|
||||
| `git.pr.failed` | `by`, `head-branch`, `base-branch`, `error`, `reason` | When PR API call fails. |
|
||||
|
||||
## Architecture sketch
|
||||
The publish flow keeps control in the session DO and uses runtime/provider adapters.
|
||||
|
||||
```text
|
||||
Logseq UI or agent-triggered call
|
||||
|
|
||||
v
|
||||
POST /sessions/:id/pr (worker.handler.agent)
|
||||
|
|
||||
v
|
||||
AgentSessionDO /__session__/pr
|
||||
1) validate + audit
|
||||
2) resolve branch
|
||||
3) runtime-provider push
|
||||
4) source-control PR (optional)
|
||||
5) append + broadcast events
|
||||
|
|
||||
v
|
||||
SSE /sessions/:id/stream
|
||||
```
|
||||
|
||||
## Detailed implementation plan
|
||||
This section is intentionally bite-sized so each step is a 2-5 minute action.
|
||||
|
||||
### Phase 1: Request and route contracts
|
||||
1. Add `sessions-pr-request-schema` and `sessions-pr-response-schema` to `deps/db-sync/src/logseq/db_sync/malli_schema.cljs`.
|
||||
|
||||
2. Register `:sessions/pr` in `http-request-schemas` and `http-response-schemas` in `deps/db-sync/src/logseq/db_sync/malli_schema.cljs`.
|
||||
|
||||
3. Add optional capability contract fields to `sessions-create-request-schema` in `deps/db-sync/src/logseq/db_sync/malli_schema.cljs` using kebab-case keys only.
|
||||
|
||||
4. Extend `normalize-session-create` in `deps/db-sync/src/logseq/db_sync/worker/agent/request.cljs` to persist capability profile into task payload with defaults.
|
||||
|
||||
5. Add `POST /sessions/:session-id/pr` route entry in `deps/db-sync/src/logseq/db_sync/worker/routes/index.cljs`.
|
||||
|
||||
6. Add route coverage tests for the new path in `deps/db-sync/test/logseq/db_sync/worker_routes_test.cljs`.
|
||||
|
||||
7. Run `bb dev:test -v logseq.db-sync.worker-routes-test/match-route-sessions-test` and confirm the new route test fails before implementation and passes after wiring.
|
||||
|
||||
### Phase 2: Agent handler forwarding
|
||||
1. Add `handle-pr` request forwarding function to `deps/db-sync/src/logseq/db_sync/worker/handler/agent.cljs`.
|
||||
|
||||
2. Reuse `base-headers` and `forward-request` in `deps/db-sync/src/logseq/db_sync/worker/handler/agent.cljs` so publish requests keep user identity headers.
|
||||
|
||||
3. Add `:sessions/pr` branch in `handle` dispatch in `deps/db-sync/src/logseq/db_sync/worker/handler/agent.cljs`.
|
||||
|
||||
4. Validate and coerce publish request body with `:sessions/pr` schema before forwarding in `deps/db-sync/src/logseq/db_sync/worker/handler/agent.cljs`.
|
||||
|
||||
5. Add a focused handler test namespace if missing to verify `/sessions/:id/pr` forwards to `/__session__/pr`.
|
||||
|
||||
6. Run the focused handler test and confirm red then green behavior.
|
||||
|
||||
### Phase 3: Source control abstraction
|
||||
1. Create `deps/db-sync/src/logseq/db_sync/worker/agent/source_control.cljs` with a minimal provider protocol for repo parsing, manual PR URL building, and PR creation.
|
||||
|
||||
2. Add GitHub URL parsing helpers in `deps/db-sync/src/logseq/db_sync/worker/agent/source_control.cljs` for HTTPS and SSH remote forms.
|
||||
|
||||
3. Add branch-name normalization and sanitization helpers in `deps/db-sync/src/logseq/db_sync/worker/agent/source_control.cljs`.
|
||||
|
||||
4. Implement GitHub PR creation in `deps/db-sync/src/logseq/db_sync/worker/agent/source_control.cljs` using `js/fetch` and strict HTTP status handling.
|
||||
|
||||
5. Add manual PR URL fallback builder in `deps/db-sync/src/logseq/db_sync/worker/agent/source_control.cljs`.
|
||||
|
||||
6. Add source-control error classification with stable `:reason` values in `deps/db-sync/src/logseq/db_sync/worker/agent/source_control.cljs`.
|
||||
|
||||
7. Add a new test namespace `deps/db-sync/test/logseq/db_sync/agent_source_control_test.cljs` for URL parsing, branch sanitization, fallback URL generation, and API error mapping.
|
||||
|
||||
8. Run `bb dev:test -v logseq.db-sync.agent-source-control-test` and keep this test set green before moving on.
|
||||
|
||||
### Phase 4: Runtime-provider push execution
|
||||
1. Extend `RuntimeProvider` protocol in `deps/db-sync/src/logseq/db_sync/worker/agent/runtime_provider.cljs` with a push execution entrypoint.
|
||||
|
||||
2. Implement push execution for `SpritesProvider` in `deps/db-sync/src/logseq/db_sync/worker/agent/runtime_provider.cljs` by executing git commands inside the repo directory.
|
||||
|
||||
3. Implement push execution for `CloudflareProvider` in `deps/db-sync/src/logseq/db_sync/worker/agent/runtime_provider.cljs` via existing sandbox exec helpers.
|
||||
|
||||
4. Implement explicit unsupported behavior for `LocalDevProvider` in `deps/db-sync/src/logseq/db_sync/worker/agent/runtime_provider.cljs` with actionable error data.
|
||||
|
||||
5. Add command builders that avoid shell injection by strict branch validation in `deps/db-sync/src/logseq/db_sync/worker/agent/runtime_provider.cljs`.
|
||||
|
||||
6. Classify push failures into deterministic reason codes such as `auth`, `no-branch`, `no-commits`, `remote-rejected`, and `unknown`.
|
||||
|
||||
7. Add push-path tests in `deps/db-sync/test/logseq/db_sync/agent_runtime_provider_test.cljs` for sprites and cloudflare command composition.
|
||||
|
||||
8. Add tests that local-dev push returns a structured unsupported error in `deps/db-sync/test/logseq/db_sync/agent_runtime_provider_test.cljs`.
|
||||
|
||||
9. Run `bb dev:test -v logseq.db-sync.agent-runtime-provider-test` and verify all publish-related cases.
|
||||
|
||||
### Phase 5: Session DO publish orchestration
|
||||
1. Add publish request parsing and validation helpers to `deps/db-sync/src/logseq/db_sync/worker/agent/do.cljs`.
|
||||
|
||||
2. Add capability guard helpers in `deps/db-sync/src/logseq/db_sync/worker/agent/do.cljs` with defaults `push-enabled=true` and `pr-enabled=true`.
|
||||
|
||||
3. Add branch resolution helper in `deps/db-sync/src/logseq/db_sync/worker/agent/do.cljs` using request head branch, runtime branch, and generated fallback.
|
||||
|
||||
4. Add `handle-pr` in `deps/db-sync/src/logseq/db_sync/worker/agent/do.cljs` that requires authenticated user header and active runtime.
|
||||
|
||||
5. Emit `git.push.started` and `git.push.succeeded` or `git.push.failed` events in `deps/db-sync/src/logseq/db_sync/worker/agent/do.cljs`.
|
||||
|
||||
6. Invoke runtime-provider push method from `deps/db-sync/src/logseq/db_sync/worker/agent/do.cljs` and return actionable errors.
|
||||
|
||||
7. Add optional PR path in `deps/db-sync/src/logseq/db_sync/worker/agent/do.cljs` controlled by `create-pr` request flag.
|
||||
|
||||
8. Emit `git.pr.started`, `git.pr.succeeded`, `git.pr.manual`, or `git.pr.failed` in `deps/db-sync/src/logseq/db_sync/worker/agent/do.cljs`.
|
||||
|
||||
9. Persist latest publish metadata under session state in `deps/db-sync/src/logseq/db_sync/worker/agent/do.cljs` for replay and UI hydration.
|
||||
|
||||
10. Add `/__session__/pr` path dispatch in `handle-fetch` in `deps/db-sync/src/logseq/db_sync/worker/agent/do.cljs`.
|
||||
|
||||
11. Add idempotency guard using request header key if present in `deps/db-sync/src/logseq/db_sync/worker/agent/do.cljs` so retries do not duplicate PRs.
|
||||
|
||||
12. Add publish behavior tests in `deps/db-sync/test/logseq/db_sync/agent_do_test.cljs` covering success, failure, manual fallback, and unauthenticated rejection.
|
||||
|
||||
13. Add tests confirming no role-based gate exists for publish in `deps/db-sync/test/logseq/db_sync/agent_do_test.cljs`.
|
||||
|
||||
14. Run `bb dev:test -v logseq.db-sync.agent-do-test` and keep publish scenarios green.
|
||||
|
||||
### Phase 6: Configuration and environment wiring
|
||||
1. Add publish-related env keys to `deps/db-sync/src/logseq/db_sync/node/config.cljs` for SCM provider and credentials.
|
||||
|
||||
2. Pass these env keys into runtime Worker env in `deps/db-sync/src/logseq/db_sync/node/server.cljs`.
|
||||
|
||||
3. Add non-secret provider vars to `deps/db-sync/worker/wrangler.toml` as placeholders for deploy-time secret config.
|
||||
|
||||
4. Add doc updates for required secrets and optional PR mode in `deps/db-sync/README.md`.
|
||||
|
||||
5. Confirm node adapter startup still succeeds with missing optional PR vars and returns manual fallback.
|
||||
|
||||
### Phase 7: Frontend invocation path for collaborators
|
||||
1. Add a publish request helper in `src/main/frontend/handler/agent.cljs` that posts to `/sessions/:id/pr` using current auth token.
|
||||
|
||||
2. Add default PR title and body generators in `src/main/frontend/handler/agent.cljs` derived from task title and session context.
|
||||
|
||||
3. Add UI action controls in `src/main/frontend/components/agent_chat.cljs` for `Push only` and `Push + PR`.
|
||||
|
||||
4. Disable publish buttons while chat is streaming or session is missing in `src/main/frontend/components/agent_chat.cljs`.
|
||||
|
||||
5. Show success and failure notifications from publish responses in `src/main/frontend/handler/agent.cljs`.
|
||||
|
||||
6. Merge publish events into session state as normal events in `src/main/frontend/handler/agent.cljs` and rely on existing stream logic.
|
||||
|
||||
7. Run manual UI verification with two different authenticated users to confirm both users can execute publish actions.
|
||||
|
||||
### Phase 8: Protocol and milestone docs
|
||||
1. Add sessions publish API docs to `docs/agent-guide/db-sync/protocol.md`.
|
||||
|
||||
2. Update `deps/db-sync/docs/milestones/agents/14-m14-git-push-and-optional-pr.md` with implementation status and final contract.
|
||||
|
||||
3. Add an operator runbook section in `deps/db-sync/README.md` for credential setup and common publish failures.
|
||||
|
||||
4. Document provider support matrix and fallback behavior in `deps/db-sync/README.md`.
|
||||
|
||||
## Verification commands
|
||||
Run each command from repo root unless noted.
|
||||
|
||||
```bash
|
||||
bb dev:test -v logseq.db-sync.worker-routes-test/match-route-sessions-test
|
||||
```
|
||||
|
||||
Expected output is a passing test that includes the `/sessions/:session-id/pr` route.
|
||||
|
||||
```bash
|
||||
bb dev:test -v logseq.db-sync.agent-runtime-provider-test
|
||||
```
|
||||
|
||||
Expected output is passing push command and provider behavior cases.
|
||||
|
||||
```bash
|
||||
bb dev:test -v logseq.db-sync.agent-do-test
|
||||
```
|
||||
|
||||
Expected output is passing publish orchestration and event emission cases.
|
||||
|
||||
```bash
|
||||
cd deps/db-sync && npm run test:node-adapter
|
||||
```
|
||||
|
||||
Expected output is zero failing tests and no regression in node adapter behavior.
|
||||
|
||||
```bash
|
||||
bb dev:lint-and-test
|
||||
```
|
||||
|
||||
Expected output is a clean lint and test run for the full workspace.
|
||||
|
||||
## Edge cases to handle explicitly
|
||||
Push must fail cleanly when the runtime provider does not support shell git execution.
|
||||
|
||||
Push must fail cleanly when branch name is invalid or resolves to a protected branch without force permission.
|
||||
|
||||
Push must return actionable auth errors when token configuration is missing or invalid.
|
||||
|
||||
PR creation must return manual fallback URL when PR API credentials are unavailable but push succeeded.
|
||||
|
||||
PR creation must not create duplicates when the same idempotency key is retried.
|
||||
|
||||
Session publish must reject unauthenticated requests and must not check manager or member role.
|
||||
|
||||
Publish must fail with a clear message when session runtime was already terminated and no repo workspace is available.
|
||||
|
||||
Repo URL parsing must reject unsupported host formats and return a deterministic reason.
|
||||
|
||||
Event stream consumers must tolerate new `git.*` event types without UI crashes.
|
||||
|
||||
## Rollout strategy
|
||||
Ship backend publish API and events first behind a configuration flag so production can validate credentials and provider behavior.
|
||||
|
||||
Enable frontend buttons after backend validation in staging confirms stable publish outcomes.
|
||||
|
||||
Turn on default frontend publish actions in production after two-user collaborator verification succeeds.
|
||||
|
||||
## Testing Details
|
||||
Tests will verify behavior boundaries, including route access, request coercion, runtime push execution, PR fallback logic, and event-stream observability under success and failure.
|
||||
|
||||
The tests will validate real response payloads and session state transitions instead of implementation details, mocks-only assertions, or data-structure snapshots.
|
||||
|
||||
## Implementation Details
|
||||
- Add `POST /sessions/:session-id/pr` route and handler forwarding through existing session stub flow.
|
||||
- Add publish schemas and capability fields in `malli_schema.cljs` with backward-compatible defaults.
|
||||
- Add GitHub-first source-control helper namespace for PR creation and manual fallback URL generation.
|
||||
- Extend runtime-provider protocol for provider-specific git push execution.
|
||||
- Implement deterministic branch sanitization and resolution before any push.
|
||||
- Add publish orchestration in Agent Session DO with `git.*` lifecycle events.
|
||||
- Keep collaborator access model authentication-only with no manager or role gate.
|
||||
- Add node and worker configuration for SCM credentials and publish behavior toggles.
|
||||
- Add frontend publish actions for `Push only` and `Push + PR` in agent chat UI.
|
||||
- Update protocol and milestone docs with final API, env requirements, and fallback semantics.
|
||||
|
||||
## Question
|
||||
Should M14 support only GitHub in the first implementation, or do we need Bitbucket or GitLab stubs in the same milestone.
|
||||
Just GitHub.
|
||||
|
||||
Should push use force mode by default for generated branches, or should force be opt-in only via request flag.
|
||||
Force mode.
|
||||
|
||||
Should publish be callable only by authenticated users from UI, or must we also support in-sandbox tool invocation with a dedicated session-scoped token in M14.
|
||||
From UI.
|
||||
|
||||
Should PR title and body defaults come from task metadata only, or should we extract a summary from recent assistant events for better defaults.
|
||||
Extract a summary for better defaults.
|
||||
|
||||
Should runtime auto-termination on `session.completed` be delayed briefly to improve publish reliability when users click publish after completion.
|
||||
Yes, delayed.
|
||||
|
||||
---
|
||||
|
||||
No need to care about backward-compatible.
|
||||
@@ -110,3 +110,35 @@
|
||||
- `DELETE /assets/:graph-id/:asset-uuid.:ext`
|
||||
- Delete asset. Response: `{"ok":true}`.
|
||||
- Asset error responses: `{"error":"invalid asset path"}` (400), `{"error":"not found"}` (404), `{"error":"asset too large"}` (413), `{"error":"method not allowed"}` (405), `{"error":"missing assets bucket"}` (500).
|
||||
|
||||
### Agent Sessions
|
||||
- `POST /sessions`
|
||||
- Create or resume an agent session for a task. Response: `{"session-id":"...","status":"...","stream-url":"..."}`.
|
||||
- `GET /sessions/:session-id`
|
||||
- Session metadata. Response: `{"session-id":"...","status":"...","task":{...},"audit":{...},"created-at":<ms>,"updated-at":<ms>}`.
|
||||
- `POST /sessions/:session-id/messages`
|
||||
- Send a user message to the running session. Body: `{"message":"...","kind":"user"}`.
|
||||
- Response: `{"ok":true}`.
|
||||
- `GET /sessions/:session-id/events?since=<ms>&limit=<n>`
|
||||
- Poll session events. Response: `{"events":[{event-id, session-id, type, ts, data}...]}`.
|
||||
- `GET /sessions/:session-id/stream`
|
||||
- SSE stream of session events.
|
||||
- `POST /sessions/:session-id/pause`
|
||||
- Pause the session. Response: `{"ok":true}`.
|
||||
- `POST /sessions/:session-id/resume`
|
||||
- Resume the session and flush queued messages. Response: `{"ok":true,"flushed":<n>}`.
|
||||
- `POST /sessions/:session-id/interrupt`
|
||||
- Interrupt active execution (transitions to paused). Response: `{"ok":true}`.
|
||||
- `POST /sessions/:session-id/cancel`
|
||||
- Cancel the session and terminate runtime. Response: `{"ok":true}`.
|
||||
- `POST /sessions/:session-id/pr`
|
||||
- Push the session branch and optionally create a PR. Body fields:
|
||||
- `create-pr` (optional, default `true`)
|
||||
- `force` (optional, default `false`)
|
||||
- `head-branch` (optional)
|
||||
- `base-branch` (optional)
|
||||
- `title` / `body` (optional PR metadata)
|
||||
- Response statuses:
|
||||
- `{"status":"pushed", ...}` for push-only
|
||||
- `{"status":"pr-created","pr-url":"...", ...}` for push + PR success
|
||||
- `{"status":"manual-pr-required","manual-pr-url":"...", ...}` when branch push succeeded but PR must be created manually
|
||||
|
||||
@@ -9,6 +9,7 @@
|
||||
[frontend.state :as state]
|
||||
[logseq.shui.hooks :as hooks]
|
||||
[logseq.shui.ui :as shui]
|
||||
[promesa.core :as p]
|
||||
[rum.core :as rum]))
|
||||
|
||||
(defn- normalized-text
|
||||
@@ -401,9 +402,12 @@
|
||||
(map message->chat-message)
|
||||
(remove nil?))
|
||||
[draft set-draft!] (rum/use-state "")
|
||||
[publish-mode set-publish-mode!] (rum/use-state nil)
|
||||
trimmed-draft (string/trim (or draft ""))
|
||||
busy? (contains? #{"submitted" "streaming"} chat-status)
|
||||
input-disabled? (or (not session-started?) (not (agent-handler/task-ready? block)))
|
||||
publish-busy? (some? publish-mode)
|
||||
publish-disabled? (or input-disabled? busy? publish-busy?)
|
||||
can-send? (and (not input-disabled?)
|
||||
(not (string/blank? trimmed-draft))
|
||||
(not busy?))
|
||||
@@ -411,7 +415,13 @@
|
||||
(when (and can-send? base session-id)
|
||||
(set-draft! "")
|
||||
(-> (.sendMessage chat #js {:text trimmed-draft})
|
||||
(.catch (fn [_] nil)))))]
|
||||
(.catch (fn [_] nil)))))
|
||||
publish! (fn [create-pr?]
|
||||
(when (and base session-id (not publish-disabled?))
|
||||
(set-publish-mode! (if create-pr? :pr :push))
|
||||
(-> (agent-handler/<publish-session! block {:create-pr? create-pr?})
|
||||
(p/catch (fn [_] nil))
|
||||
(p/finally (fn [] (set-publish-mode! nil))))))]
|
||||
(hooks/use-effect!
|
||||
(fn []
|
||||
(when (agent-handler/task-ready? block)
|
||||
@@ -450,6 +460,31 @@
|
||||
:agent-label agent-label
|
||||
:class "h-full"
|
||||
:content-class-name "gap-3 px-2 py-2 sm:px-3"})]
|
||||
[:div.mt-3.flex.items-center.justify-between.gap-2
|
||||
[:div.text-xs.opacity-60
|
||||
(if publish-busy?
|
||||
"Publishing changes..."
|
||||
"Publish session changes")]
|
||||
[:div.flex.items-center.gap-2
|
||||
(shui/button
|
||||
{:size :sm
|
||||
:variant :outline
|
||||
:class "h-7 px-2 text-xs"
|
||||
:disabled publish-disabled?
|
||||
:on-click (fn [_]
|
||||
(publish! false))}
|
||||
(if (= publish-mode :push)
|
||||
"Pushing..."
|
||||
"Push"))
|
||||
(shui/button
|
||||
{:size :sm
|
||||
:class "h-7 px-2 text-xs"
|
||||
:disabled publish-disabled?
|
||||
:on-click (fn [_]
|
||||
(publish! true))}
|
||||
(if (= publish-mode :pr)
|
||||
"Creating PR..."
|
||||
"Push + PR"))]]
|
||||
(shui/agent-chat-prompt-input
|
||||
{:value draft
|
||||
:on-value-change set-draft!
|
||||
|
||||
@@ -96,7 +96,9 @@
|
||||
:content content
|
||||
:attachments attachments
|
||||
:project project
|
||||
:agent agent})))
|
||||
:agent agent
|
||||
:capabilities {:push-enabled true
|
||||
:pr-enabled true}})))
|
||||
|
||||
(def ^:private stream-reconnect-delay-ms 1500)
|
||||
|
||||
@@ -399,3 +401,78 @@
|
||||
:started-at (util/time-ms)})
|
||||
(<connect-session-stream! block-uuid stream-url)
|
||||
resp))))))
|
||||
|
||||
(defn- publish-request-body
|
||||
[{:keys [title body head-branch base-branch create-pr? force?]}]
|
||||
(cond-> {:create-pr (if (nil? create-pr?) true (true? create-pr?))
|
||||
:force (true? force?)}
|
||||
(string? (blank->nil title)) (assoc :title (blank->nil title))
|
||||
(string? (blank->nil body)) (assoc :body (blank->nil body))
|
||||
(string? (blank->nil head-branch)) (assoc :head-branch (blank->nil head-branch))
|
||||
(string? (blank->nil base-branch)) (assoc :base-branch (blank->nil base-branch))))
|
||||
|
||||
(defn- publish-status-message
|
||||
[resp]
|
||||
(or (:message resp)
|
||||
(case (:status resp)
|
||||
"pushed" "Branch pushed."
|
||||
"pr-created" "Pull request created."
|
||||
"manual-pr-required" "Branch pushed. Create pull request manually."
|
||||
"Publish finished.")))
|
||||
|
||||
(defn- publish-error-message
|
||||
[error]
|
||||
(or (some-> (ex-data error) :body :message)
|
||||
(some-> (ex-data error) :body :error)
|
||||
(some-> error ex-message)
|
||||
"Publish failed."))
|
||||
|
||||
(defn <publish-session!
|
||||
[block opts]
|
||||
(let [base (db-sync/http-base)
|
||||
block-uuid (:block/uuid block)
|
||||
session (session-state block-uuid)
|
||||
session-id (or (:session-id session)
|
||||
(some-> block-uuid str))]
|
||||
(cond
|
||||
(not base)
|
||||
(do
|
||||
(notification/show! "DB sync is not configured." :error false)
|
||||
(p/resolved nil))
|
||||
|
||||
(not (task-ready? block))
|
||||
(do
|
||||
(notification/show! "Task needs Project (with Git Repo) and Agent." :warning)
|
||||
(p/resolved nil))
|
||||
|
||||
(not (:session-id session))
|
||||
(do
|
||||
(notification/show! "Start the agent session before publishing." :warning)
|
||||
(p/resolved nil))
|
||||
|
||||
:else
|
||||
(p/let [_ (js/Promise. user-handler/task--ensure-id&access-token)
|
||||
raw-body (publish-request-body opts)
|
||||
body (coerce-http-request :sessions/pr raw-body)]
|
||||
(if (nil? body)
|
||||
(do
|
||||
(notification/show! "Invalid publish payload." :error false)
|
||||
nil)
|
||||
(-> (db-sync/fetch-json (str base "/sessions/" session-id "/pr")
|
||||
{:method "POST"
|
||||
:headers {"content-type" "application/json"}
|
||||
:body (js/JSON.stringify (clj->js body))}
|
||||
{:response-schema :sessions/pr})
|
||||
(p/then (fn [resp]
|
||||
(update-session-state! block-uuid {:last-publish resp
|
||||
:last-publish-at (util/time-ms)})
|
||||
(notification/show! (publish-status-message resp)
|
||||
(if (= "manual-pr-required" (:status resp))
|
||||
:warning
|
||||
:success)
|
||||
false)
|
||||
(<fetch-events! block)
|
||||
resp))
|
||||
(p/catch (fn [error]
|
||||
(notification/show! (publish-error-message error) :error false)
|
||||
(p/rejected error)))))))))
|
||||
|
||||
Reference in New Issue
Block a user