Commit Graph

15342 Commits

Author SHA1 Message Date
Ahmed Ibrahim
ebe75bb683 Route Python SDK turn notifications by ID (#21778)
## Why

The Python SDK previously protected the stdio transport with a single
active turn-consumer guard. That avoided competing reads from stdout,
but it also meant one `Codex`/`AsyncCodex` client could not stream
multiple active turns at the same time. Notifications could also arrive
before the caller received a `TurnHandle` and registered for streaming,
so the SDK needed an explicit routing layer instead of letting
individual API calls read directly from the shared transport.

## What Changed

- Added a private `MessageRouter` that owns per-request response queues,
per-turn notification queues, pending turn-notification replay, and
global notification delivery behind a single stdout reader thread.
- Generated typed notification routing metadata so turn IDs come from
known payload shapes instead of router-side attribute guessing, with
explicit fallback handling for unknown notification payloads.
- Updated sync and async turn streaming so `TurnHandle.stream()`/`run()`
and `stream_text()` consume only notifications for their own turn ID,
while `AsyncAppServerClient` no longer serializes all transport calls
behind one async lock.
- Cleared pending turn-notification buffers when unregistered turns
complete so never-consumed turn handles do not leave stale queues
behind.
- Removed the internal stream-until helper now that turn completion
waiting can register directly with routed turn notifications.
- Updated Python SDK docs and focused tests for concurrent transport
calls, interleaved turn routing, buffered early notifications, unknown
notification routing, async delegation, and routed turn completion
behavior.

## Validation

- `uv run --extra dev ruff format scripts/update_sdk_artifacts.py
src/codex_app_server/_message_router.py src/codex_app_server/client.py
src/codex_app_server/generated/notification_registry.py
tests/test_client_rpc_methods.py
tests/test_public_api_runtime_behavior.py
tests/test_async_client_behavior.py`
- `uv run --extra dev ruff check scripts/update_sdk_artifacts.py
src/codex_app_server/_message_router.py src/codex_app_server/client.py
src/codex_app_server/generated/notification_registry.py
tests/test_client_rpc_methods.py
tests/test_public_api_runtime_behavior.py
tests/test_async_client_behavior.py`
- `uv run --extra dev pytest tests/test_client_rpc_methods.py
tests/test_public_api_runtime_behavior.py
tests/test_async_client_behavior.py`
- `git diff --check`

---------

Co-authored-by: Codex <noreply@openai.com>
2026-05-09 04:16:23 +00:00
sayan-oai
77d9223e9f [codex] compact network context rendering (#21875)
## Why

The model-visible `<network>` context currently repeats indentation and
a pair of XML tags for every allowed or denied domain. Large domain sets
spend a surprising amount of prompt budget on that scaffolding instead
of the actual policy values.

## What changed

- Render allowed domains as one comma-separated `<allowed>` value
instead of one element per domain.
- Render denied domains the same way.
- Keep the full allow/deny domain sets model-visible while updating the
serialization and settings-update coverage for the denser shape.

## Example

Before:
```xml
<network enabled="true">
  <allowed>api.example.test</allowed>
  <allowed>cdn.example.test</allowed>
  <denied>blocked.example.test</denied>
</network>
```

After:
```xml
<network enabled="true"><allowed>api.example.test,cdn.example.test</allowed><denied>blocked.example.test</denied></network>
```

## Validation

- `cargo test -p codex-core environment_context`
- `cargo test -p codex-core
build_settings_update_items_emits_environment_item_for_network_changes`
- Ran a local `codex` session with a real network context containing 121
allowed domains and 42 denied domains, then inspected the raw prompt
with `raw_token_viewer_cli.py`. With the same domain set, the rendered
`<network>` section shrank from 7,175 characters across 161 lines to
3,666 characters on one line, and the containing environment-context
block fell from 6,428 tokens to 5,379 tokens.
2026-05-09 03:52:48 +00:00
xl-openai
479491ed89 feat: Add role-aware plugin share context APIs (#21867)
Expose discoverability and full share principals in share context, carry
roles through save/updateTargets, hydrate local shared plugin reads, and
keep share URLs only under plugin.shareContext.
2026-05-08 20:46:39 -07:00
Michael Bolin
b40cdf80b7 Merge 7f3c36a32c into sapling-pr-archive-bolinfest 2026-05-08 18:40:12 -07:00
Michael Bolin
f7022d55bd merge commit for archive created by Sapling 2026-05-08 18:39:55 -07:00
Michael Bolin
7f3c36a32c tests: characterize macOS sandbox link writes 2026-05-08 18:39:35 -07:00
Michael Bolin
3dc8d984bb tests: cover sandbox link write behavior 2026-05-08 18:39:35 -07:00
pakrym-oai
c579da41b1 Move file watcher out of core (#21290)
## Why

The app-server watcher relocation leaves the generic filesystem watcher
as the last watcher-specific implementation still living inside
`codex-core`. Moving that code to a small crate keeps `codex-core`
focused on thread execution and lets app-server depend on the watcher
without reaching back into core for filesystem watching primitives.

This PR is stacked on #21287.

## What changed

- Added a new `codex-file-watcher` crate containing the existing watcher
implementation and its unit tests.
- Updated app-server `fs_watch`, `skills_watcher`, and listener state to
import watcher types from `codex-file-watcher`.
- Removed the `file_watcher` module and `notify` dependency from
`codex-core`.
- Updated Cargo workspace metadata and `Cargo.lock` for the new internal
crate.

## Validation

- `cargo check -p codex-file-watcher -p codex-core -p codex-app-server`
- `cargo test -p codex-file-watcher`
- `cargo test -p codex-app-server
skills_changed_notification_is_emitted_after_skill_change`
- `just bazel-lock-update`
- `just bazel-lock-check`
- `just fix -p codex-file-watcher`
- `just fix -p codex-core`
- `just fix -p codex-app-server`
2026-05-08 18:19:23 -07:00
pakrym-oai
408e6218ab Reapply "Move skills watcher to app-server" (#21652)
## Why

PR #21460 reverted the earlier move of skills change watching from
`codex-core` into app-server. This reapplies that boundary change so
app-server owns client-facing `skills/changed` notifications and core no
longer carries the watcher.

## What

- Restore the app-server `SkillsWatcher` and register it from thread
listener setup.
- Remove the core-owned skills watcher and its core live-reload
integration surface.
- Restore app-server coverage for `skills/changed` notifications after a
watched skill file changes.

## Validation

- `cargo test -p codex-app-server --test all
suite::v2::skills_list::skills_changed_notification_is_emitted_after_skill_change
-- --exact --nocapture`
- `cargo test -p codex-core --lib --no-run`
2026-05-08 17:41:15 -07:00
Michael Bolin
9a952be62c Merge 9b0ffba286 into sapling-pr-archive-bolinfest 2026-05-08 17:36:30 -07:00
Michael Bolin
2afc43a17d Merge ebddba6119 into sapling-pr-archive-bolinfest 2026-05-08 17:36:16 -07:00
Michael Bolin
9b0ffba286 tests: characterize macOS sandbox link writes 2026-05-08 17:36:05 -07:00
Michael Bolin
ebddba6119 tests: cover sandbox link write behavior 2026-05-08 17:36:05 -07:00
Owen Lin
95ca276373 sqlite: no more destructive version bumps (#21847)
## Why

We'd like SQLite state to become required and load-bearing. As a first
step, let's remove the mechanism that allows us to blow away the SQLite
DB on a version bump, and instead rely on graceful migrations.

The original motivation
([PR](https://github.com/openai/codex/pull/10623)) behind this mechanism
was to care less about backwards compatibility while SQLite was being
landed, but I'd say it's quite important now to keep the data in it.

## What changed

- Make `STATE_DB_FILENAME` and `LOGS_DB_FILENAME` the full canonical
filenames: `state_5.sqlite` and `logs_2.sqlite`.
- Remove `STATE_DB_VERSION` / `LOGS_DB_VERSION` and the helper that
constructed filenames from versions.
- Stop `StateRuntime::init` from scanning for or deleting older SQLite
DB filenames at startup.
- Delete the tests that encoded legacy state/logs DB deletion behavior.

## Verification

- `cargo test -p codex-state`
2026-05-08 17:29:44 -07:00
Michael Bolin
23fdcd9e3f Merge 2f237af389 into sapling-pr-archive-bolinfest 2026-05-08 17:19:35 -07:00
Michael Bolin
d171cae8bd merge commit for archive created by Sapling 2026-05-08 17:19:16 -07:00
Michael Bolin
2f237af389 tests: characterize macOS sandbox link writes 2026-05-08 17:19:00 -07:00
Michael Bolin
a38c8474f3 tests: cover sandbox link write behavior 2026-05-08 17:19:00 -07:00
Michael Bolin
ef87897c5c Merge 5ee77cb024 into sapling-pr-archive-bolinfest 2026-05-08 17:13:47 -07:00
Michael Bolin
762d077472 merge commit for archive created by Sapling 2026-05-08 17:13:29 -07:00
Michael Bolin
5ee77cb024 tests: characterize macOS sandbox link writes 2026-05-08 17:13:22 -07:00
Michael Bolin
b50e4f8225 tests: cover sandbox link write behavior 2026-05-08 17:13:22 -07:00
Michael Bolin
2cc24cb6c2 merge commit for archive created by Sapling 2026-05-08 17:07:18 -07:00
Michael Bolin
e798cf3e40 tests: characterize macOS sandbox link writes 2026-05-08 17:06:58 -07:00
Michael Bolin
953e2ae5a6 Merge 5b07ee6141 into sapling-pr-archive-bolinfest 2026-05-08 17:04:06 -07:00
Michael Bolin
5b07ee6141 tests: cover sandbox link write behavior 2026-05-08 17:03:33 -07:00
Celia Chen
bd42660cb4 feat: add Bedrock Mantle client agent header (#21840)
## Why

Amazon Bedrock Mantle needs a stable client-agent header so requests
from the built-in Bedrock provider can be identified as coming from
Codex for safety stack.

## What changed

- Added `x-amzn-mantle-client-agent: codex` to the built-in Amazon
Bedrock provider default HTTP headers.
2026-05-08 23:58:41 +00:00
Ruslan Nigmatullin
0c8d42525e [daemon] Add app-server daemon lifecycle management (#20718)
## Why

Desktop and mobile Codex clients need a machine-readable way to
bootstrap and manage `codex app-server` on remote machines reached over
SSH. The same flow is also useful for bringing up app-server with
`remote_control` enabled on a fresh developer machine and keeping that
managed install current without requiring a human session.

## What changed

- add the new experimental `codex-app-server-daemon` crate and wire it
into `codex app-server daemon` lifecycle commands: `start`, `restart`,
`stop`, `version`, and `bootstrap`
- add explicit `enable-remote-control` and `disable-remote-control`
commands that persist the launch setting and restart a running managed
daemon so the change takes effect immediately
- emit JSON success responses for daemon commands so remote callers can
consume them directly
- support a Unix-only pidfile-backed detached backend for lifecycle
management
- assume the standalone `install.sh` layout for daemon-managed binaries
and always launch `CODEX_HOME/packages/standalone/current/codex`
- add bootstrap support for the standalone managed install plus a
detached hourly updater loop
- harden lifecycle management around concurrent operations, pidfile
ownership, stale state cleanup, updater ownership, managed-binary
preflight, Unix-only rejection, forced shutdown after the graceful
window, and updater process-group tracking/cleanup
- document the experimental Unix-only support boundary plus the
standalone bootstrap/update flow in
`codex-rs/app-server-daemon/README.md`

## Verification

- `cargo test -p codex-app-server-daemon -p codex-cli`
- live pid validation on `cb4`: `bootstrap --remote-control`, `restart`,
`version`, `stop`

## Follow-up

- Add updater self-refresh so the long-lived `pid-update-loop` can
replace its own executable image after installing a newer managed Codex
binary.
2026-05-08 16:51:16 -07:00
starr-openai
faa5d4a5e2 Increase exec-server environment transport timeouts (#21825)
## Why

The environment-backed exec-server transport currently hardcodes 5
second connect and initialize timeouts in `client_transport.rs`. That is
short for SSH-backed stdio environments and remote websocket
environments, and there is currently no way to raise those values from
`CODEX_HOME/environments.toml`.

This stacked follow-up raises the default environment transport timeouts
and lets each configured environment override them in
`environments.toml`.

## What Changed

- raise the default environment transport connect and initialize
timeouts from 5s to 10s
- store concrete timeout values on `ExecServerTransportParams` instead
of hardcoding them in `connect_for_transport(...)`
- add `connect_timeout_sec` and `initialize_timeout_sec` to
`[[environments]]` entries in `environments.toml`
- apply parse-time defaults so runtime transport code receives fully
resolved timeout values
- reject `connect_timeout_sec` on stdio environments because it only
applies to websocket transports
- extend parser tests to cover the new fields and defaults

## Stack

- base: https://github.com/openai/codex/pull/21794
- this PR: configurable environment transport timeouts

## Validation

- `cd
/Users/starr/code/codex-worktrees/exec-env-timeouts-config-20260508/codex-rs
&& just fmt`
- not run: tests

---------

Co-authored-by: Codex <noreply@openai.com>
2026-05-08 16:33:29 -07:00
Michael Zeng
8f4020846e [codex] support executor registry remote environments (#21323)
## Summary

Support registry-backed remote executors end to end so downstream
services can resolve an executor id into an exec-server URL and make
that environment available to Codex without relying on the legacy cloud
environments flow.

## What changed

- switch remote executor registration to the executor registry bootstrap
contract
- allow named remote environments to be inserted into
`EnvironmentManager` at runtime
- add the experimental app-server RPC `environment/add` so initialized
experimental clients can register those remote environments for later
`thread/start` and `turn/start` selection

## Validation

Ran focused validation locally:
- `cargo test -p codex-exec-server environment_manager_`
- `cargo test -p codex-exec-server
register_executor_posts_with_bearer_token_header`
- `cargo test -p codex-app-server-protocol`
2026-05-08 16:30:07 -07:00
Michael Bolin
626a07b5a0 Merge 8874edd5b3 into sapling-pr-archive-bolinfest 2026-05-08 16:15:29 -07:00
Michael Bolin
8874edd5b3 tests: demonstrate macos hard-link sandbox escape 2026-05-08 16:15:11 -07:00
lt-oai
80a408e201 Support openai library tool (#20293)
Support chatgpt library tool
2026-05-08 22:56:13 +00:00
Ruslan Nigmatullin
1b86906fa1 app-server: support daemon-safe restart handling (#21831)
## Why

The app-server daemon work needs two app-server behaviors to be safe
when lifecycle management is driven by a helper process:

- a readiness probe must not become the process-wide client identity
just because it connects first
- a graceful reload signal needs to keep draining active turns even if
it is delivered more than once

## What changed

- Treat `codex_app_server_daemon` initialization as a probe-only client
for process-global originator and user-agent suffix state.
- Distinguish forceable shutdown signals from graceful-only ones, and
treat Unix `SIGHUP` as graceful-only while leaving `SIGTERM` and Ctrl-C
forceable.
- Add regression coverage for daemon probe initialization and repeated
`SIGHUP` delivery while a turn is still running.

## Testing

- `cargo test -p codex-app-server`
  - The new daemon-probe and repeated-`SIGHUP` coverage passed.
- The run still failed in the existing
`suite::conversation_summary::get_conversation_summary_by_relative_rollout_path_resolves_from_codex_home`
and
`suite::conversation_summary::get_conversation_summary_by_thread_id_reads_rollout`
tests because their initialize handshake timed out.
- `cargo test -p codex-app-server --test all
suite::conversation_summary::`
- Reproduced the same two existing initialize-timeout failures in
isolation.
2026-05-08 15:47:51 -07:00
starr-openai
dac108f2f1 Make environment provider snapshots path-free (#21794)
## Summary
- make EnvironmentProvider::snapshot path-free and keep providers
focused on provider-owned remote environments
- let provider snapshots request local inclusion via include_local, with
environments.toml including local and CODEX_EXEC_SERVER_URL excluding
local
- move reserved local environment construction into EnvironmentManager
using ExecServerRuntimePaths

Follow-up to https://github.com/openai/codex/pull/20667

## Testing
- just fmt
- git diff --check
- devbox: bazel build --bes_backend= --bes_results_url=
//codex-rs/exec-server:exec-server
- devbox: bazel test --bes_backend= --bes_results_url=
//codex-rs/exec-server:exec-server-unit-tests

Co-authored-by: Codex <noreply@openai.com>
2026-05-08 15:30:00 -07:00
Michael Bolin
24111790f0 ci: check out PR head commits in workflows (#21835)
## Why

PR CI should test the exact commit that was pushed to the PR branch. By
default, GitHub's `pull_request` event checks out a synthetic merge
commit from `refs/pull/<number>/merge`, so the tested tree can include
an implicit merge with the current base branch instead of matching the
pushed head SHA.

Using the PR head SHA makes each check result correspond to a concrete
commit the author submitted. This also behaves better for stacked PR
workflows, including Sapling stacks and other Git stack tooling: a
middle PR's head commit already contains the lower stack changes in its
tree, without pulling in commits above it or GitHub's temporary merge
ref.

## What Changed

- Set every `actions/checkout` in `pull_request` workflows under
`.github/workflows` to use `github.event.pull_request.head.sha` on PR
events and `github.sha` otherwise.
- Updated `blob-size-policy` to compare
`github.event.pull_request.base.sha` and
`github.event.pull_request.head.sha`, since it no longer checks out
GitHub's merge commit where `HEAD^1`/`HEAD^2` represented the PR range.

## Verification

- Parsed the edited workflow YAML files with Ruby.
- Checked that every checkout block in the `pull_request` workflows has
the PR-head `ref`.
2026-05-08 15:14:33 -07:00
Michael Bolin
024a36c118 merge commit for archive created by Sapling 2026-05-08 14:51:59 -07:00
Michael Bolin
da5505c670 Move workspace roots onto thread/session state and stop using active permission profile modifications as an overlay for writable roots. Existing app-server threads now preserve their persisted PermissionProfile value across resume, fork, and turn updates; permissions requests on existing threads only update the active named profile after validating it exists. Workspace roots can be updated independently, and SandboxPolicy::WorkspaceWrite no longer stores its own writable_roots. 2026-05-08 14:51:41 -07:00
Michael Bolin
fdc31c7e15 Merge 998a82daf0 into sapling-pr-archive-bolinfest 2026-05-08 14:51:29 -07:00
Michael Bolin
998a82daf0 ci: check out PR head commits in workflows 2026-05-08 14:51:15 -07:00
Michael Bolin
da5889957c Merge b6dc57e035 into sapling-pr-archive-bolinfest 2026-05-08 14:45:28 -07:00
Michael Bolin
b6dc57e035 apply-patch: avoid sandbox link write-through 2026-05-08 14:41:27 -07:00
Michael Bolin
c56590d0a9 merge commit for archive created by Sapling 2026-05-08 14:40:43 -07:00
Michael Bolin
7402d29902 apply-patch: avoid sandbox link write-through 2026-05-08 14:40:24 -07:00
Michael Bolin
c8910c8233 merge commit for archive created by Sapling 2026-05-08 14:28:40 -07:00
Michael Bolin
3ebbcbca31 apply-patch: avoid sandbox link write-through 2026-05-08 14:28:19 -07:00
Michael Bolin
7a7756ce40 Merge 63d0cc87d3 into sapling-pr-archive-bolinfest 2026-05-08 14:27:56 -07:00
Michael Bolin
63d0cc87d3 Move workspace roots onto thread/session state and stop using active permission profile modifications as an overlay for writable roots. Existing app-server threads now preserve their persisted PermissionProfile value across resume, fork, and turn updates; permissions requests on existing threads only update the active named profile after validating it exists. Workspace roots can be updated independently, and SandboxPolicy::WorkspaceWrite no longer stores its own writable_roots. 2026-05-08 14:27:42 -07:00
Michael Bolin
67f7f4bad9 Merge 841a91d42d into sapling-pr-archive-bolinfest 2026-05-08 14:21:49 -07:00
Michael Bolin
841a91d42d apply-patch: avoid sandbox link write-through 2026-05-08 14:21:31 -07:00