Commit Graph

15237 Commits

Author SHA1 Message Date
Michael Bolin
7f3c36a32c tests: characterize macOS sandbox link writes 2026-05-08 18:39:35 -07:00
Michael Bolin
3dc8d984bb tests: cover sandbox link write behavior 2026-05-08 18:39:35 -07:00
pakrym-oai
c579da41b1 Move file watcher out of core (#21290)
## Why

The app-server watcher relocation leaves the generic filesystem watcher
as the last watcher-specific implementation still living inside
`codex-core`. Moving that code to a small crate keeps `codex-core`
focused on thread execution and lets app-server depend on the watcher
without reaching back into core for filesystem watching primitives.

This PR is stacked on #21287.

## What changed

- Added a new `codex-file-watcher` crate containing the existing watcher
implementation and its unit tests.
- Updated app-server `fs_watch`, `skills_watcher`, and listener state to
import watcher types from `codex-file-watcher`.
- Removed the `file_watcher` module and `notify` dependency from
`codex-core`.
- Updated Cargo workspace metadata and `Cargo.lock` for the new internal
crate.

## Validation

- `cargo check -p codex-file-watcher -p codex-core -p codex-app-server`
- `cargo test -p codex-file-watcher`
- `cargo test -p codex-app-server
skills_changed_notification_is_emitted_after_skill_change`
- `just bazel-lock-update`
- `just bazel-lock-check`
- `just fix -p codex-file-watcher`
- `just fix -p codex-core`
- `just fix -p codex-app-server`
2026-05-08 18:19:23 -07:00
pakrym-oai
408e6218ab Reapply "Move skills watcher to app-server" (#21652)
## Why

PR #21460 reverted the earlier move of skills change watching from
`codex-core` into app-server. This reapplies that boundary change so
app-server owns client-facing `skills/changed` notifications and core no
longer carries the watcher.

## What

- Restore the app-server `SkillsWatcher` and register it from thread
listener setup.
- Remove the core-owned skills watcher and its core live-reload
integration surface.
- Restore app-server coverage for `skills/changed` notifications after a
watched skill file changes.

## Validation

- `cargo test -p codex-app-server --test all
suite::v2::skills_list::skills_changed_notification_is_emitted_after_skill_change
-- --exact --nocapture`
- `cargo test -p codex-core --lib --no-run`
2026-05-08 17:41:15 -07:00
Michael Bolin
9a952be62c Merge 9b0ffba286 into sapling-pr-archive-bolinfest 2026-05-08 17:36:30 -07:00
Michael Bolin
2afc43a17d Merge ebddba6119 into sapling-pr-archive-bolinfest 2026-05-08 17:36:16 -07:00
Michael Bolin
9b0ffba286 tests: characterize macOS sandbox link writes 2026-05-08 17:36:05 -07:00
Michael Bolin
ebddba6119 tests: cover sandbox link write behavior 2026-05-08 17:36:05 -07:00
Owen Lin
95ca276373 sqlite: no more destructive version bumps (#21847)
## Why

We'd like SQLite state to become required and load-bearing. As a first
step, let's remove the mechanism that allows us to blow away the SQLite
DB on a version bump, and instead rely on graceful migrations.

The original motivation
([PR](https://github.com/openai/codex/pull/10623)) behind this mechanism
was to care less about backwards compatibility while SQLite was being
landed, but I'd say it's quite important now to keep the data in it.

## What changed

- Make `STATE_DB_FILENAME` and `LOGS_DB_FILENAME` the full canonical
filenames: `state_5.sqlite` and `logs_2.sqlite`.
- Remove `STATE_DB_VERSION` / `LOGS_DB_VERSION` and the helper that
constructed filenames from versions.
- Stop `StateRuntime::init` from scanning for or deleting older SQLite
DB filenames at startup.
- Delete the tests that encoded legacy state/logs DB deletion behavior.

## Verification

- `cargo test -p codex-state`
2026-05-08 17:29:44 -07:00
Michael Bolin
23fdcd9e3f Merge 2f237af389 into sapling-pr-archive-bolinfest 2026-05-08 17:19:35 -07:00
Michael Bolin
d171cae8bd merge commit for archive created by Sapling 2026-05-08 17:19:16 -07:00
Michael Bolin
2f237af389 tests: characterize macOS sandbox link writes 2026-05-08 17:19:00 -07:00
Michael Bolin
a38c8474f3 tests: cover sandbox link write behavior 2026-05-08 17:19:00 -07:00
Michael Bolin
ef87897c5c Merge 5ee77cb024 into sapling-pr-archive-bolinfest 2026-05-08 17:13:47 -07:00
Michael Bolin
762d077472 merge commit for archive created by Sapling 2026-05-08 17:13:29 -07:00
Michael Bolin
5ee77cb024 tests: characterize macOS sandbox link writes 2026-05-08 17:13:22 -07:00
Michael Bolin
b50e4f8225 tests: cover sandbox link write behavior 2026-05-08 17:13:22 -07:00
Michael Bolin
2cc24cb6c2 merge commit for archive created by Sapling 2026-05-08 17:07:18 -07:00
Michael Bolin
e798cf3e40 tests: characterize macOS sandbox link writes 2026-05-08 17:06:58 -07:00
Michael Bolin
953e2ae5a6 Merge 5b07ee6141 into sapling-pr-archive-bolinfest 2026-05-08 17:04:06 -07:00
Michael Bolin
5b07ee6141 tests: cover sandbox link write behavior 2026-05-08 17:03:33 -07:00
Celia Chen
bd42660cb4 feat: add Bedrock Mantle client agent header (#21840)
## Why

Amazon Bedrock Mantle needs a stable client-agent header so requests
from the built-in Bedrock provider can be identified as coming from
Codex for safety stack.

## What changed

- Added `x-amzn-mantle-client-agent: codex` to the built-in Amazon
Bedrock provider default HTTP headers.
2026-05-08 23:58:41 +00:00
Ruslan Nigmatullin
0c8d42525e [daemon] Add app-server daemon lifecycle management (#20718)
## Why

Desktop and mobile Codex clients need a machine-readable way to
bootstrap and manage `codex app-server` on remote machines reached over
SSH. The same flow is also useful for bringing up app-server with
`remote_control` enabled on a fresh developer machine and keeping that
managed install current without requiring a human session.

## What changed

- add the new experimental `codex-app-server-daemon` crate and wire it
into `codex app-server daemon` lifecycle commands: `start`, `restart`,
`stop`, `version`, and `bootstrap`
- add explicit `enable-remote-control` and `disable-remote-control`
commands that persist the launch setting and restart a running managed
daemon so the change takes effect immediately
- emit JSON success responses for daemon commands so remote callers can
consume them directly
- support a Unix-only pidfile-backed detached backend for lifecycle
management
- assume the standalone `install.sh` layout for daemon-managed binaries
and always launch `CODEX_HOME/packages/standalone/current/codex`
- add bootstrap support for the standalone managed install plus a
detached hourly updater loop
- harden lifecycle management around concurrent operations, pidfile
ownership, stale state cleanup, updater ownership, managed-binary
preflight, Unix-only rejection, forced shutdown after the graceful
window, and updater process-group tracking/cleanup
- document the experimental Unix-only support boundary plus the
standalone bootstrap/update flow in
`codex-rs/app-server-daemon/README.md`

## Verification

- `cargo test -p codex-app-server-daemon -p codex-cli`
- live pid validation on `cb4`: `bootstrap --remote-control`, `restart`,
`version`, `stop`

## Follow-up

- Add updater self-refresh so the long-lived `pid-update-loop` can
replace its own executable image after installing a newer managed Codex
binary.
2026-05-08 16:51:16 -07:00
starr-openai
faa5d4a5e2 Increase exec-server environment transport timeouts (#21825)
## Why

The environment-backed exec-server transport currently hardcodes 5
second connect and initialize timeouts in `client_transport.rs`. That is
short for SSH-backed stdio environments and remote websocket
environments, and there is currently no way to raise those values from
`CODEX_HOME/environments.toml`.

This stacked follow-up raises the default environment transport timeouts
and lets each configured environment override them in
`environments.toml`.

## What Changed

- raise the default environment transport connect and initialize
timeouts from 5s to 10s
- store concrete timeout values on `ExecServerTransportParams` instead
of hardcoding them in `connect_for_transport(...)`
- add `connect_timeout_sec` and `initialize_timeout_sec` to
`[[environments]]` entries in `environments.toml`
- apply parse-time defaults so runtime transport code receives fully
resolved timeout values
- reject `connect_timeout_sec` on stdio environments because it only
applies to websocket transports
- extend parser tests to cover the new fields and defaults

## Stack

- base: https://github.com/openai/codex/pull/21794
- this PR: configurable environment transport timeouts

## Validation

- `cd
/Users/starr/code/codex-worktrees/exec-env-timeouts-config-20260508/codex-rs
&& just fmt`
- not run: tests

---------

Co-authored-by: Codex <noreply@openai.com>
2026-05-08 16:33:29 -07:00
Michael Zeng
8f4020846e [codex] support executor registry remote environments (#21323)
## Summary

Support registry-backed remote executors end to end so downstream
services can resolve an executor id into an exec-server URL and make
that environment available to Codex without relying on the legacy cloud
environments flow.

## What changed

- switch remote executor registration to the executor registry bootstrap
contract
- allow named remote environments to be inserted into
`EnvironmentManager` at runtime
- add the experimental app-server RPC `environment/add` so initialized
experimental clients can register those remote environments for later
`thread/start` and `turn/start` selection

## Validation

Ran focused validation locally:
- `cargo test -p codex-exec-server environment_manager_`
- `cargo test -p codex-exec-server
register_executor_posts_with_bearer_token_header`
- `cargo test -p codex-app-server-protocol`
2026-05-08 16:30:07 -07:00
Michael Bolin
626a07b5a0 Merge 8874edd5b3 into sapling-pr-archive-bolinfest 2026-05-08 16:15:29 -07:00
Michael Bolin
8874edd5b3 tests: demonstrate macos hard-link sandbox escape 2026-05-08 16:15:11 -07:00
lt-oai
80a408e201 Support openai library tool (#20293)
Support chatgpt library tool
2026-05-08 22:56:13 +00:00
Ruslan Nigmatullin
1b86906fa1 app-server: support daemon-safe restart handling (#21831)
## Why

The app-server daemon work needs two app-server behaviors to be safe
when lifecycle management is driven by a helper process:

- a readiness probe must not become the process-wide client identity
just because it connects first
- a graceful reload signal needs to keep draining active turns even if
it is delivered more than once

## What changed

- Treat `codex_app_server_daemon` initialization as a probe-only client
for process-global originator and user-agent suffix state.
- Distinguish forceable shutdown signals from graceful-only ones, and
treat Unix `SIGHUP` as graceful-only while leaving `SIGTERM` and Ctrl-C
forceable.
- Add regression coverage for daemon probe initialization and repeated
`SIGHUP` delivery while a turn is still running.

## Testing

- `cargo test -p codex-app-server`
  - The new daemon-probe and repeated-`SIGHUP` coverage passed.
- The run still failed in the existing
`suite::conversation_summary::get_conversation_summary_by_relative_rollout_path_resolves_from_codex_home`
and
`suite::conversation_summary::get_conversation_summary_by_thread_id_reads_rollout`
tests because their initialize handshake timed out.
- `cargo test -p codex-app-server --test all
suite::conversation_summary::`
- Reproduced the same two existing initialize-timeout failures in
isolation.
2026-05-08 15:47:51 -07:00
starr-openai
dac108f2f1 Make environment provider snapshots path-free (#21794)
## Summary
- make EnvironmentProvider::snapshot path-free and keep providers
focused on provider-owned remote environments
- let provider snapshots request local inclusion via include_local, with
environments.toml including local and CODEX_EXEC_SERVER_URL excluding
local
- move reserved local environment construction into EnvironmentManager
using ExecServerRuntimePaths

Follow-up to https://github.com/openai/codex/pull/20667

## Testing
- just fmt
- git diff --check
- devbox: bazel build --bes_backend= --bes_results_url=
//codex-rs/exec-server:exec-server
- devbox: bazel test --bes_backend= --bes_results_url=
//codex-rs/exec-server:exec-server-unit-tests

Co-authored-by: Codex <noreply@openai.com>
2026-05-08 15:30:00 -07:00
Michael Bolin
24111790f0 ci: check out PR head commits in workflows (#21835)
## Why

PR CI should test the exact commit that was pushed to the PR branch. By
default, GitHub's `pull_request` event checks out a synthetic merge
commit from `refs/pull/<number>/merge`, so the tested tree can include
an implicit merge with the current base branch instead of matching the
pushed head SHA.

Using the PR head SHA makes each check result correspond to a concrete
commit the author submitted. This also behaves better for stacked PR
workflows, including Sapling stacks and other Git stack tooling: a
middle PR's head commit already contains the lower stack changes in its
tree, without pulling in commits above it or GitHub's temporary merge
ref.

## What Changed

- Set every `actions/checkout` in `pull_request` workflows under
`.github/workflows` to use `github.event.pull_request.head.sha` on PR
events and `github.sha` otherwise.
- Updated `blob-size-policy` to compare
`github.event.pull_request.base.sha` and
`github.event.pull_request.head.sha`, since it no longer checks out
GitHub's merge commit where `HEAD^1`/`HEAD^2` represented the PR range.

## Verification

- Parsed the edited workflow YAML files with Ruby.
- Checked that every checkout block in the `pull_request` workflows has
the PR-head `ref`.
2026-05-08 15:14:33 -07:00
Michael Bolin
024a36c118 merge commit for archive created by Sapling 2026-05-08 14:51:59 -07:00
Michael Bolin
da5505c670 Move workspace roots onto thread/session state and stop using active permission profile modifications as an overlay for writable roots. Existing app-server threads now preserve their persisted PermissionProfile value across resume, fork, and turn updates; permissions requests on existing threads only update the active named profile after validating it exists. Workspace roots can be updated independently, and SandboxPolicy::WorkspaceWrite no longer stores its own writable_roots. 2026-05-08 14:51:41 -07:00
Michael Bolin
fdc31c7e15 Merge 998a82daf0 into sapling-pr-archive-bolinfest 2026-05-08 14:51:29 -07:00
Michael Bolin
998a82daf0 ci: check out PR head commits in workflows 2026-05-08 14:51:15 -07:00
Michael Bolin
da5889957c Merge b6dc57e035 into sapling-pr-archive-bolinfest 2026-05-08 14:45:28 -07:00
Michael Bolin
b6dc57e035 apply-patch: avoid sandbox link write-through 2026-05-08 14:41:27 -07:00
Michael Bolin
c56590d0a9 merge commit for archive created by Sapling 2026-05-08 14:40:43 -07:00
Michael Bolin
7402d29902 apply-patch: avoid sandbox link write-through 2026-05-08 14:40:24 -07:00
Michael Bolin
c8910c8233 merge commit for archive created by Sapling 2026-05-08 14:28:40 -07:00
Michael Bolin
3ebbcbca31 apply-patch: avoid sandbox link write-through 2026-05-08 14:28:19 -07:00
Michael Bolin
7a7756ce40 Merge 63d0cc87d3 into sapling-pr-archive-bolinfest 2026-05-08 14:27:56 -07:00
Michael Bolin
63d0cc87d3 Move workspace roots onto thread/session state and stop using active permission profile modifications as an overlay for writable roots. Existing app-server threads now preserve their persisted PermissionProfile value across resume, fork, and turn updates; permissions requests on existing threads only update the active named profile after validating it exists. Workspace roots can be updated independently, and SandboxPolicy::WorkspaceWrite no longer stores its own writable_roots. 2026-05-08 14:27:42 -07:00
Michael Bolin
67f7f4bad9 Merge 841a91d42d into sapling-pr-archive-bolinfest 2026-05-08 14:21:49 -07:00
Michael Bolin
841a91d42d apply-patch: avoid sandbox link write-through 2026-05-08 14:21:31 -07:00
Matthew Zeng
2f3a2d7a86 Using cached connector directory for discoverable tools list (#21497)
## Summary

Startup tool construction currently depends on connector directory
metadata for `tool_suggest` discoverables. On a cold directory cache,
that can put slow connector-directory requests on the blocking path even
though the tools array only needs directory data for install
suggestions, not for the live connector MCP tools themselves.

This PR keeps the discoverables path off that cold network fetch:
- read connector directory metadata from cache only when building
discoverable tools
- persist connector directory metadata to
`~/.codex/cache/codex_app_directory/<hash>.json` and use it to hydrate
the in-memory cache on later runs before the normal refresh path updates
it
- use connector-directory-specific cache naming to distinguish this
metadata cache from the separate Codex Apps tools-spec cache

This reduces first-turn startup work without changing how live connector
MCP tools are sourced. Longer term, directory-backed install suggestions
should move to a search-based flow so they no longer need to be inlined
into the tools prompt at all.

## Testing

- `cargo test -p codex-connectors`
- `cargo test -p codex-chatgpt`
- `cargo test -p codex-core
request_plugin_install_is_available_without_search_tool_after_discovery_attempts`
- `cargo test -p codex-core
tool_suggest_uses_connector_id_fallback_when_directory_cache_is_empty`
2026-05-08 14:14:11 -07:00
Michael Bolin
36b7954abe Merge ecb02fdc53 into sapling-pr-archive-bolinfest 2026-05-08 14:05:18 -07:00
Michael Bolin
ecb02fdc53 Move workspace roots onto thread/session state and stop using active permission profile modifications as an overlay for writable roots. Existing app-server threads now preserve their persisted PermissionProfile value across resume, fork, and turn updates; permissions requests on existing threads only update the active named profile after validating it exists. Workspace roots can be updated independently, and SandboxPolicy::WorkspaceWrite no longer stores its own writable_roots. 2026-05-08 14:05:11 -07:00
Michael Bolin
b3aba87592 Merge 950d4add9f into sapling-pr-archive-bolinfest 2026-05-08 13:51:28 -07:00
Michael Bolin
950d4add9f Move workspace roots onto thread/session state and stop using active permission profile modifications as an overlay for writable roots. Existing app-server threads now preserve their persisted PermissionProfile value across resume, fork, and turn updates; permissions requests on existing threads only update the active named profile after validating it exists. Workspace roots can be updated independently, and SandboxPolicy::WorkspaceWrite no longer stores its own writable_roots. 2026-05-08 13:51:19 -07:00