Compare commits

...

111 Commits

Author SHA1 Message Date
jimmyfraiture
8882f8427c Add a few tests 2025-09-29 15:28:06 +01:00
jimmyfraiture
c30e4d3969 Clippy 2025-09-29 14:15:15 +01:00
jimmyfraiture
36298cf734 P2 2025-09-29 14:09:40 +01:00
jimmyfraiture
4562946b3b P1 2025-09-29 13:56:57 +01:00
jimmyfraiture
8a7f75eeef Just fix 2025-09-29 13:08:17 +01:00
jimmyfraiture
6283dc42f8 Rename 2025-09-29 12:58:54 +01:00
jimmyfraiture
0d340b1bec P5 2025-09-29 12:03:57 +01:00
jimmyfraiture
c9f6b5dffc P4 2025-09-29 11:06:44 +01:00
jimmyfraiture
2efe961ac1 P3 2025-09-29 10:49:19 +01:00
jimmyfraiture
491ba05f71 P2 2025-09-29 10:30:24 +01:00
jimmyfraiture
cd7e37c6b0 P1 2025-09-29 09:48:56 +01:00
jimmyfraiture
3cdf35e198 Merge remote-tracking branch 'origin/main' into jif/sandbox-1 2025-09-26 15:50:40 +02:00
jif-oai
1fc3413a46 ref: state - 2 (#4229)
Extracting tasks in a module and start abstraction behind a Trait (more
to come on this but each task will be tackled in a dedicated PR)
The goal was to drop the ActiveTask and to have a (potentially) set of
tasks during each turn
2025-09-26 13:49:08 +00:00
jimmyfraiture
caab5a19ee Move some stuff around 2025-09-26 14:46:07 +02:00
jimmyfraiture
a29380cdff Isolate apply patch adapter 2025-09-26 14:02:38 +02:00
jimmyfraiture
805de19381 V1 2025-09-26 13:42:58 +02:00
iceweasel-oai
eb2b739d6a core: add potentially dangerous command check (#4211)
Certain shell commands are potentially dangerous, and we want to check
for them.
Unless the user has explicitly approved a command, we will *always* ask
them for approval
when one of these commands is encountered, regardless of whether they
are in a sandbox, or what their approval policy is.

The first (of probably many) such examples is `git reset --hard`. We
will be conservative and check for any `git reset`
2025-09-25 19:46:20 -07:00
pakrym-oai
a10403d697 Actually mount sse once (#4264)
Mock server was responding with the same result many times.
2025-09-26 01:17:51 +00:00
pakrym-oai
8e3a048fec Add codex exec testing helpers (#4254)
Add a shortcut to create working directories and run codex exec with
fake server.
2025-09-25 17:12:45 -07:00
Eric Traut
9f2ab97fbc Fixed login failure with API key in IDE extension when a .codex directory doesn't exist (#4258)
This addresses bug #4092

Testing:
* Confirmed error occurs prior to fix if logging in using API key and no
`~/.codex` directory exists
* Confirmed after fix that `~/.codex` directory is properly created and
error doesn't occur
2025-09-25 16:53:28 -07:00
iceweasel-oai
38c9d7dca1 fix typo in sandbox doc (#4256)
just fixes a simple typo I noticed.
2025-09-25 16:03:44 -07:00
pakrym-oai
67aab04c66 [codex exec] Add item.started and support it for command execution (#4250)
Adds a new `item.started` event to `codex exec` and implements it for
command_execution item type.

```jsonl
{"type":"session.created","session_id":"019982d1-75f0-7920-b051-e0d3731a5ed8"}
{"type":"item.completed","item":{"id":"item_0","item_type":"reasoning","text":"**Executing commands securely**\n\nI'm thinking about how the default harness typically uses \"bash -lc,\" while historically \"bash\" is what we've been using. The command should be executed as a string in our CLI, so using \"bash -lc 'echo hello'\" is optimal but calling \"echo hello\" directly feels safer. The sandbox makes sure environment variables like CODEX_SANDBOX_NETWORK_DISABLED=1 are set, so I won't ask for approval. I just need to run \"echo hello\" and correctly present the output."}}
{"type":"item.completed","item":{"id":"item_1","item_type":"reasoning","text":"**Preparing for tool calls**\n\nI realize that I need to include a preamble before making any tool calls. So, I'll first state the preamble in the commentary channel, then proceed with the tool call. After that, I need to present the final message along with the output. It's possible that the CLI will show the output inline, but I must ensure that I present the result clearly regardless. Let's move forward and get this organized!"}}
{"type":"item.completed","item":{"id":"item_2","item_type":"assistant_message","text":"Running `echo` to confirm shell access and print output."}}
{"type":"item.started","item":{"id":"item_3","item_type":"command_execution","command":"bash -lc echo hello","aggregated_output":"","exit_code":null,"status":"in_progress"}}
{"type":"item.completed","item":{"id":"item_3","item_type":"command_execution","command":"bash -lc echo hello","aggregated_output":"hello\n","exit_code":0,"status":"completed"}}
{"type":"item.completed","item":{"id":"item_4","item_type":"assistant_message","text":"hello"}}
```
2025-09-25 22:25:02 +00:00
Ahmed Ibrahim
7355ca48c5 fix (#4251)
# External (non-OpenAI) Pull Request Requirements

Before opening this Pull Request, please read the dedicated
"Contributing" markdown file or your PR may be closed:
https://github.com/openai/codex/blob/main/docs/contributing.md

If your PR conforms to our contribution guidelines, replace this text
with a detailed and high quality description of your changes.
2025-09-25 15:12:25 -07:00
Jeremy Rose
affb5fc1d0 fix bug when resizing to a smaller width (#4248)
The composer and key hint lines were using line styles, causing ratatui
to print spaces all the way to the right side of the terminal. this
meant that resizing the terminal to be narrower would result in
rewrapping those lines, causing the bottom area to rerender and push all
content up.

Before


https://github.com/user-attachments/assets/8b14555a-1fc5-4f78-8df7-1410ee25e07a

After


https://github.com/user-attachments/assets/707645ab-89c7-4c7f-b556-02f53cef8a2f
2025-09-25 14:17:13 -07:00
Jeremy Rose
4a5f05c136 make tests pass cleanly in sandbox (#4067)
This changes the reqwest client used in tests to be sandbox-friendly,
and skips a bunch of other tests that don't work inside the
sandbox/without network.
2025-09-25 13:11:14 -07:00
pakrym-oai
acc2b63dfb Fix error message (#4204)
Co-authored-by: Ahmed Ibrahim <aibrahim@openai.com>
2025-09-25 11:10:40 -07:00
pakrym-oai
344d4a1d68 Add explicit codex exec events (#4177)
This pull request add a new experimental format of JSON output.

You can try it using `codex exec --experimental-json`.

Design takes a lot of inspiration from Responses API items and stream
format.

# Session and items
Each invocation of `codex exec` starts or resumes a session. 

Session contains multiple high-level item types:
1. Assistant message 
2. Assistant thinking 
3. Command execution 
4. File changes
5. To-do lists
6. etc.

# Events 
Session and items are going through their life cycles which is
represented by events.

Session is `session.created` or `session.resumed`
Items are `item.added`, `item.updated`, `item.completed`,
`item.require_approval` (or other item types like `item.output_delta`
when we need streaming).

So a typical session can look like:

<details>

```
{
  "type": "session.created",
  "session_id": "01997dac-9581-7de3-b6a0-1df8256f2752"
}
{
  "type": "item.completed",
  "item": {
    "id": "itm_0",
    "item_type": "assistant_message",
    "text": "I’ll locate the top-level README and remove its first line. Then I’ll show a quick summary of what changed."
  }
}
{
  "type": "item.completed",
  "item": {
    "id": "itm_1",
    "item_type": "command_execution",
    "command": "bash -lc ls -la | sed -n '1,200p'",
    "aggregated_output": "pyenv: cannot rehash: /Users/pakrym/.pyenv/shims isn't writable\ntotal 192\ndrwxr-xr-x@  33 pakrym  staff   1056 Sep 24 14:36 .\ndrwxr-xr-x   41 pakrym  staff   1312 Sep 24 09:17 ..\n-rw-r--r--@   1 pakrym  staff      6 Jul  9 16:16 .codespellignore\n-rw-r--r--@   1 pakrym  staff    258 Aug 13 09:40 .codespellrc\ndrwxr-xr-x@   5 pakrym  staff    160 Jul 23 08:26 .devcontainer\n-rw-r--r--@   1 pakrym  staff   6148 Jul 22 10:03 .DS_Store\ndrwxr-xr-x@  15 pakrym  staff    480 Sep 24 14:38 .git\ndrwxr-xr-x@  12 pakrym  staff    384 Sep  2 16:00 .github\n-rw-r--r--@   1 pakrym  staff    778 Jul  9 16:16 .gitignore\ndrwxr-xr-x@   3 pakrym  staff     96 Aug 11 09:37 .husky\n-rw-r--r--@   1 pakrym  staff    104 Jul  9 16:16 .npmrc\n-rw-r--r--@   1 pakrym  staff     96 Sep  2 08:52 .prettierignore\n-rw-r--r--@   1 pakrym  staff    170 Jul  9 16:16 .prettierrc.toml\ndrwxr-xr-x@   5 pakrym  staff    160 Sep 14 17:43 .vscode\ndrwxr-xr-x@   2 pakrym  staff     64 Sep 11 11:37 2025-09-11\n-rw-r--r--@   1 pakrym  staff   5505 Sep 18 09:28 AGENTS.md\n-rw-r--r--@   1 pakrym  staff     92 Sep  2 08:52 CHANGELOG.md\n-rw-r--r--@   1 pakrym  staff   1145 Jul  9 16:16 cliff.toml\ndrwxr-xr-x@  11 pakrym  staff    352 Sep 24 13:03 codex-cli\ndrwxr-xr-x@  38 pakrym  staff   1216 Sep 24 14:38 codex-rs\ndrwxr-xr-x@  18 pakrym  staff    576 Sep 23 11:01 docs\n-rw-r--r--@   1 pakrym  staff   2038 Jul  9 16:16 flake.lock\n-rw-r--r--@   1 pakrym  staff   1434 Jul  9 16:16 flake.nix\n-rw-r--r--@   1 pakrym  staff  10926 Jul  9 16:16 LICENSE\ndrwxr-xr-x@ 465 pakrym  staff  14880 Jul 15 07:36 node_modules\n-rw-r--r--@   1 pakrym  staff    242 Aug  5 08:25 NOTICE\n-rw-r--r--@   1 pakrym  staff    578 Aug 14 12:31 package.json\n-rw-r--r--@   1 pakrym  staff    498 Aug 11 09:37 pnpm-lock.yaml\n-rw-r--r--@   1 pakrym  staff     58 Aug 11 09:37 pnpm-workspace.yaml\n-rw-r--r--@   1 pakrym  staff   2402 Jul  9 16:16 PNPM.md\n-rw-r--r--@   1 pakrym  staff   4393 Sep 12 14:36 README.md\ndrwxr-xr-x@   4 pakrym  staff    128 Sep 18 09:28 scripts\ndrwxr-xr-x@   2 pakrym  staff     64 Sep 11 11:34 tmp\n",
    "exit_code": 0,
    "status": "completed"
  }
}
{
  "type": "item.completed",
  "item": {
    "id": "itm_2",
    "item_type": "reasoning",
    "text": "**Reviewing README.md file**\n\nI've located the README.md file at the root, and it’s 4393 bytes. Now, I need to remove the first line, but first, I should check its content to make sure I’m patching it correctly. I’ll use sed to display the first 20 lines. By reviewing those lines, I can determine exactly what needs to be removed before I proceed with the editing. Let's do this carefully!"
  }
}
{
  "type": "item.completed",
  "item": {
    "id": "itm_3",
    "item_type": "command_execution",
    "command": "bash -lc sed -n '1,40p' README.md",
    "aggregated_output": "<h1 align=\"center\">OpenAI Codex CLI</h1>\n\n<p align=\"center\"><code>npm i -g @openai/codex</code><br />or <code>brew install codex</code></p>\n\n<p align=\"center\"><strong>Codex CLI</strong> is a coding agent from OpenAI that runs locally on your computer.\n</br>\n</br>If you want Codex in your code editor (VS Code, Cursor, Windsurf), <a href=\"https://developers.openai.com/codex/ide\">install in your IDE</a>\n</br>If you are looking for the <em>cloud-based agent</em> from OpenAI, <strong>Codex Web</strong>, go to <a href=\"https://chatgpt.com/codex\">chatgpt.com/codex</a></p>\n\n<p align=\"center\">\n  <img src=\"./.github/codex-cli-splash.png\" alt=\"Codex CLI splash\" width=\"80%\" />\n  </p>\n\n---\n\n## Quickstart\n\n### Installing and running Codex CLI\n\nInstall globally with your preferred package manager. If you use npm:\n\n```shell\nnpm install -g @openai/codex\n```\n\nAlternatively, if you use Homebrew:\n\n```shell\nbrew install codex\n```\n\nThen simply run `codex` to get started:\n\n```shell\ncodex\n```\n\n<details>\n<summary>You can also go to the <a href=\"https://github.com/openai/codex/releases/latest\">latest GitHub Release</a> and download the appropriate binary for your platform.</summary>\n\n",
    "exit_code": 0,
    "status": "completed"
  }
}
{
  "type": "item.completed",
  "item": {
    "id": "itm_4",
    "item_type": "reasoning",
    "text": "**Updating README file**\n\nThe user wants to update the README by removing the first line, which is an HTML h1 tag. I'll proceed to delete that line while keeping the rest intact. To do this, I need to use the apply_patch tool and create a patch in the diff format. I’ll take care to include context lines around the change. After removing the header, I’m considering whether to delete the following blank line as well to keep it tidy."
  }
}
{
  "type": "item.completed",
  "item": {
    "id": "itm_5",
    "item_type": "reasoning",
    "text": "**Crafting the patch**\n\nThe user has asked to remove just the first line, without mentioning the blank lines. Technically, leaving a blank line at the start is acceptable, but it's not ideal. Ideally, I'd prefer to remove the first line while maintaining the subsequent blank line. However, if I can also remove that blank line, it could tidy things up even more. Still, since the user's request is strictly to remove only the first line, I'll stick to that and craft the patch accordingly."
  }
}
{
  "type": "item.completed",
  "item": {
    "id": "itm_6",
    "item_type": "assistant_message",
    "text": "Applying the change by removing the first line from `README.md`."
  }
}
{
  "type": "item.completed",
  "item": {
    "id": "itm_7",
    "item_type": "file_change",
    "changes": [
      {
        "path": "/Users/pakrym/code/codex/README.md",
        "kind": "update"
      }
    ],
    "status": "completed"
  }
}
{
  "type": "item.completed",
  "item": {
    "id": "itm_8",
    "item_type": "reasoning",
    "text": "**Validating README Update**\n\nI updated the README.md and now I need to validate that the first line has been successfully removed. To do this, I'll check the first 10 lines of the document to ensure everything looks correct. It’s always good to double-check these kinds of changes, so I’m going to take that extra step to make sure the update was executed properly. Let's see what those initial lines say!"
  }
}
{
  "type": "item.completed",
  "item": {
    "id": "itm_9",
    "item_type": "command_execution",
    "command": "bash -lc sed -n '1,20p' README.md",
    "aggregated_output": "\n<p align=\"center\"><code>npm i -g @openai/codex</code><br />or <code>brew install codex</code></p>\n\n<p align=\"center\"><strong>Codex CLI</strong> is a coding agent from OpenAI that runs locally on your computer.\n</br>\n</br>If you want Codex in your code editor (VS Code, Cursor, Windsurf), <a href=\"https://developers.openai.com/codex/ide\">install in your IDE</a>\n</br>If you are looking for the <em>cloud-based agent</em> from OpenAI, <strong>Codex Web</strong>, go to <a href=\"https://chatgpt.com/codex\">chatgpt.com/codex</a></p>\n\n<p align=\"center\">\n  <img src=\"./.github/codex-cli-splash.png\" alt=\"Codex CLI splash\" width=\"80%\" />\n  </p>\n\n---\n\n## Quickstart\n\n### Installing and running Codex CLI\n\nInstall globally with your preferred package manager. If you use npm:\n\n",
    "exit_code": 0,
    "status": "completed"
  }
}
{
  "type": "item.completed",
  "item": {
    "id": "itm_10",
    "item_type": "assistant_message",
    "text": "- Updated `README.md:1` by removing the opening H1 line.\n- The README now starts with the install instructions paragraph.\n\nWant me to also remove the leading blank line at the top?"
  }
}
```

</details>

The idea is to give users fully formatted items they can use directly in
their rendering/application logic and avoid having them building up
items manually based on events (unless they want to for streaming).

This PR implements only the `item.completed` payload for some event
types, more event types and item types to come.

---------

Co-authored-by: Michael Bolin <mbolin@openai.com>
2025-09-25 17:47:09 +00:00
Michael Bolin
a0c37f5d07 chore: refactor attempt_stream_responses() out of stream_responses() (#4194)
I would like to be able to swap in a different way to resolve model
sampling requests, so this refactoring consolidates things behind
`attempt_stream_responses()` to make that easier. Ideally, we would
support an in-memory backend that we can use in our integration tests,
for example.
2025-09-25 10:34:07 -07:00
Jeremy Rose
103adcdf2d fix: esc w/ queued messages overwrites draft in composer (#4237)
Instead of overwriting the contents of the composer when pressing
<kbd>Esc</kbd> when there's a queued message, prepend the queued
message(s) to the composer draft.
2025-09-25 10:07:27 -07:00
Michael Bolin
d61dea6fe6 feat: add support for CODEX_SECURE_MODE=1 to restrict process observability (#4220)
Because the `codex` process could contain sensitive information in
memory, such as API keys, we add logic so that when
`CODEX_SECURE_MODE=1` is specified, we avail ourselves of whatever the
operating system provides to restrict observability/tampering, which
includes:

- disabling `ptrace(2)`, so it is not possible to attach to the process
with a debugger, such as `gdb`
- disabling core dumps

Admittedly, a user with root privileges can defeat these safeguards.

For now, we only add support for this in the `codex` multitool, but we
may ultimately want to support this in some of the smaller CLIs that are
buildable out of our Cargo workspace.
2025-09-25 10:02:28 -07:00
Ahmed Ibrahim
e363dac249 revamp /status (#4196)
<img width="543" height="520" alt="image"
src="https://github.com/user-attachments/assets/bbc0eec0-e40b-45e7-bcd0-a997f8eeffa2"
/>
2025-09-25 15:38:50 +00:00
jif-oai
250b244ab4 ref: full state refactor (#4174)
## Current State Observations
- `Session` currently holds many unrelated responsibilities (history,
approval queues, task handles, rollout recorder, shell discovery, token
tracking, etc.), making it hard to reason about ownership and lifetimes.
- The anonymous `State` struct inside `codex.rs` mixes session-long data
with turn-scoped queues and approval bookkeeping.
- Turn execution (`run_task`) relies on ad-hoc local variables that
should conceptually belong to a per-turn state object.
- External modules (`codex::compact`, tests) frequently poke the raw
`Session.state` mutex, which couples them to implementation details.
- Interrupts, approvals, and rollout persistence all have bespoke
cleanup paths, contributing to subtle bugs when a turn is aborted
mid-flight.

## Desired End State
- Keep a slim `Session` object that acts as the orchestrator and façade.
It should expose a focused API (submit, approvals, interrupts, event
emission) without storing unrelated fields directly.
- Introduce a `state` module that encapsulates all mutable data
structures:
- `SessionState`: session-persistent data (history, approved commands,
token/rate-limit info, maybe user preferences).
- `ActiveTurn`: metadata for the currently running turn (sub-id, task
kind, abort handle) and an `Arc<TurnState>`.
- `TurnState`: all turn-scoped pieces (pending inputs, approval waiters,
diff tracker, review history, auto-compact flags, last agent message,
outstanding tool call bookkeeping).
- Group long-lived helpers/managers into a dedicated `SessionServices`
struct so `Session` does not accumulate "random" fields.
- Provide clear, lock-safe APIs so other modules never touch raw
mutexes.
- Ensure every turn creates/drops a `TurnState` and that
interrupts/finishes delegate cleanup to it.
2025-09-25 12:16:06 +02:00
pakrym-oai
d1ed3a4cef github: update codespell action to v2.1 in workflow (#4205)
Old version fails to find python 3.8 docker image
2025-09-25 04:05:00 +00:00
pakrym-oai
e85742635f Send text parameter for non-gpt-5 models (#4195)
We had a hardcoded check for gpt-5 before.

Fixes: https://github.com/openai/codex/issues/4181
2025-09-24 22:00:06 +00:00
Michael Bolin
87b299aa3f chore: drop unused values from env_flags (#4188)
For the most part, we try to avoid environment variables in favor of
config options so the environment variables do not leak into child
processes. These environment variables are no longer honored, so let's
delete them to be clear.

Ultimately, I would also like to eliminate `CODEX_RS_SSE_FIXTURE` in
favor of something cleaner.
2025-09-24 14:29:51 -07:00
iceweasel-oai
0e58870634 adds a windows-specific method to check if a command is safe (#4119)
refactors command_safety files into its own package, so we can add
platform-specific ones
Also creates a windows-specific of `is_known_safe_command` that just
returns false always, since that is what happens today.
2025-09-24 14:03:43 -07:00
Jeremy Rose
42847baaf7 pageless session list (#3194) 2025-09-24 13:44:48 -07:00
Jeremy Rose
6032d784ee improve MCP tool call styling (#3871)
<img width="760" height="213" alt="Screenshot 2025-09-18 at 12 29 15 PM"
src="https://github.com/user-attachments/assets/48a205b7-b95a-4988-8c76-efceb998dee7"
/>
2025-09-24 13:36:01 -07:00
Jeremy Rose
7bff8df10e hide the status indicator when the answer stream starts (#4101)
This eliminates a "bounce" at the end of streaming where we hide the
status indicator at the end of the turn and the composer moves up two
lines.

Also, simplify streaming further by removing the HistorySink and
inverting control, and collapsing a few single-element structures.
2025-09-24 11:51:48 -07:00
pakrym-oai
addc946d13 Simplify tool implemetations (#4160)
Use Result<String, FunctionCallError> for all tool handling code and
rely on error propagation instead of creating failed items everywhere.
2025-09-24 17:27:35 +00:00
dependabot[bot]
bffdbec2c5 chore(deps): bump chrono from 0.4.41 to 0.4.42 in /codex-rs (#4028)
Bumps [chrono](https://github.com/chronotope/chrono) from 0.4.41 to
0.4.42.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/chronotope/chrono/releases">chrono's
releases</a>.</em></p>
<blockquote>
<h2>0.4.42</h2>
<h2>What's Changed</h2>
<ul>
<li>Add fuzzer for DateTime::parse_from_str by <a
href="https://github.com/tyler92"><code>@​tyler92</code></a> in <a
href="https://redirect.github.com/chronotope/chrono/pull/1700">chronotope/chrono#1700</a></li>
<li>Fix wrong amount of micro/milliseconds by <a
href="https://github.com/nmlt"><code>@​nmlt</code></a> in <a
href="https://redirect.github.com/chronotope/chrono/pull/1703">chronotope/chrono#1703</a></li>
<li>Add warning about MappedLocalTime and wasm by <a
href="https://github.com/lutzky"><code>@​lutzky</code></a> in <a
href="https://redirect.github.com/chronotope/chrono/pull/1702">chronotope/chrono#1702</a></li>
<li>Fix incorrect parsing of fixed-length second fractions by <a
href="https://github.com/chris-leach"><code>@​chris-leach</code></a> in
<a
href="https://redirect.github.com/chronotope/chrono/pull/1705">chronotope/chrono#1705</a></li>
<li>Fix cfgs for <code>wasm32-linux</code> support by <a
href="https://github.com/arjunr2"><code>@​arjunr2</code></a> in <a
href="https://redirect.github.com/chronotope/chrono/pull/1707">chronotope/chrono#1707</a></li>
<li>Fix OpenHarmony's <code>tzdata</code> parsing by <a
href="https://github.com/ldm0"><code>@​ldm0</code></a> in <a
href="https://redirect.github.com/chronotope/chrono/pull/1679">chronotope/chrono#1679</a></li>
<li>Convert NaiveDate to/from days since unix epoch by <a
href="https://github.com/findepi"><code>@​findepi</code></a> in <a
href="https://redirect.github.com/chronotope/chrono/pull/1715">chronotope/chrono#1715</a></li>
<li>Add <code>?Sized</code> bound to related methods of
<code>DelayedFormat::write_to</code> by <a
href="https://github.com/Huliiiiii"><code>@​Huliiiiii</code></a> in <a
href="https://redirect.github.com/chronotope/chrono/pull/1721">chronotope/chrono#1721</a></li>
<li>Add <code>from_timestamp_secs</code> method to <code>DateTime</code>
by <a href="https://github.com/jasonaowen"><code>@​jasonaowen</code></a>
in <a
href="https://redirect.github.com/chronotope/chrono/pull/1719">chronotope/chrono#1719</a></li>
<li>Migrate to core::error::Error by <a
href="https://github.com/benbrittain"><code>@​benbrittain</code></a> in
<a
href="https://redirect.github.com/chronotope/chrono/pull/1704">chronotope/chrono#1704</a></li>
<li>Upgrade to windows-bindgen 0.63 by <a
href="https://github.com/djc"><code>@​djc</code></a> in <a
href="https://redirect.github.com/chronotope/chrono/pull/1730">chronotope/chrono#1730</a></li>
<li>strftime: simplify error handling by <a
href="https://github.com/djc"><code>@​djc</code></a> in <a
href="https://redirect.github.com/chronotope/chrono/pull/1731">chronotope/chrono#1731</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="f3fd15f976"><code>f3fd15f</code></a>
Bump version to 0.4.42</li>
<li><a
href="5cf5603500"><code>5cf5603</code></a>
strftime: add regression test case</li>
<li><a
href="a6231701ee"><code>a623170</code></a>
strftime: simplify error handling</li>
<li><a
href="36fbfb1221"><code>36fbfb1</code></a>
strftime: move specifier handling out of match to reduce rightward
drift</li>
<li><a
href="7f413c363b"><code>7f413c3</code></a>
strftime: yield None early</li>
<li><a
href="9d5dfe1640"><code>9d5dfe1</code></a>
strftime: outline constants</li>
<li><a
href="e5f6be7db4"><code>e5f6be7</code></a>
strftime: move error() method below caller</li>
<li><a
href="d516c2764d"><code>d516c27</code></a>
strftime: merge impl blocks</li>
<li><a
href="0ee2172fb9"><code>0ee2172</code></a>
strftime: re-order items to keep impls together</li>
<li><a
href="757a8b0226"><code>757a8b0</code></a>
Upgrade to windows-bindgen 0.63</li>
<li>Additional commits viewable in <a
href="https://github.com/chronotope/chrono/compare/v0.4.41...v0.4.42">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=chrono&package-manager=cargo&previous-version=0.4.41&new-version=0.4.42)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-24 16:53:26 +00:00
dependabot[bot]
353a5c2046 chore(deps): bump unicode-width from 0.1.14 to 0.2.1 in /codex-rs (#2156)
Bumps [unicode-width](https://github.com/unicode-rs/unicode-width) from
0.1.14 to 0.2.1.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="0085e91db7"><code>0085e91</code></a>
Publish 0.2.1</li>
<li><a
href="6db0c14cbd"><code>6db0c14</code></a>
Remove <code>compiler-builtins</code> from <code>rustc-dep-of-std</code>
dependencies (<a
href="https://redirect.github.com/unicode-rs/unicode-width/issues/77">#77</a>)</li>
<li><a
href="0bccd3f1b5"><code>0bccd3f</code></a>
update copyright year (<a
href="https://redirect.github.com/unicode-rs/unicode-width/issues/76">#76</a>)</li>
<li><a
href="7a7fcdc813"><code>7a7fcdc</code></a>
Support Unicode 16 (<a
href="https://redirect.github.com/unicode-rs/unicode-width/issues/74">#74</a>)</li>
<li><a
href="82d7136b49"><code>82d7136</code></a>
Advertise and enforce MSRV (<a
href="https://redirect.github.com/unicode-rs/unicode-width/issues/73">#73</a>)</li>
<li><a
href="e77b2929bc"><code>e77b292</code></a>
Make characters with <code>Line_Break=Ambiguous</code> ambiguous (<a
href="https://redirect.github.com/unicode-rs/unicode-width/issues/61">#61</a>)</li>
<li><a
href="5a7fced663"><code>5a7fced</code></a>
Update version number in Readme (<a
href="https://redirect.github.com/unicode-rs/unicode-width/issues/70">#70</a>)</li>
<li><a
href="79eab0d9fc"><code>79eab0d</code></a>
Publish 0.2.0 with newlines treated as width 1 (<a
href="https://redirect.github.com/unicode-rs/unicode-width/issues/68">#68</a>)</li>
<li>See full diff in <a
href="https://github.com/unicode-rs/unicode-width/compare/v0.1.14...v0.2.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=unicode-width&package-manager=cargo&previous-version=0.1.14&new-version=0.2.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

You can trigger a rebase of this PR by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

> **Note**
> Automatic rebases have been disabled on this pull request as it has
been open for over 30 days.

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-24 16:33:46 +00:00
Tien Nguyen
00c7f7a16c chore: remove once_cell dependency from multiple crates (#4154)
This commit removes the `once_cell` dependency from `Cargo.toml` files
in the `codex-rs` and `apply-patch` directories, replacing its usage
with `std::sync::LazyLock` and `std::sync::OnceLock` where applicable.
This change simplifies the dependency tree and utilizes standard library
features for lazy initialization.

# External (non-OpenAI) Pull Request Requirements

Before opening this Pull Request, please read the dedicated
"Contributing" markdown file or your PR may be closed:
https://github.com/openai/codex/blob/main/docs/contributing.md

If your PR conforms to our contribution guidelines, replace this text
with a detailed and high quality description of your changes.
2025-09-24 09:15:57 -07:00
Michael Bolin
82e65975b2 fix: add tolerance for ambiguous behavior in gh run list (#4162)
I am not sure what is going on, as
https://github.com/openai/codex/pull/3660 introduced this new logic and
I swear that CI was green before I merged that PR, but I am seeing
failures in this CI job this morning. This feels like a
non-backwards-compatible change in `gh`, but that feels unlikely...

Nevertheless, this is what I currently see on my laptop:

```
$ gh --version
gh version 2.76.2 (2025-07-30)
https://github.com/cli/cli/releases/tag/v2.76.2
$ gh run list --workflow .github/workflows/rust-release.yml --branch rust-v0.40.0 --json workflowName,url,headSha --jq 'first(.[])'
{
  "headSha": "5268705a69713752adcbd8416ef9e84a683f7aa3",
  "url": "https://github.com/openai/codex/actions/runs/17952349351",
  "workflowName": ".github/workflows/rust-release.yml"
}
```

Looking at sample output from an old GitHub issue
(https://github.com/cli/cli/issues/6678), it appears that, at least at
one point in time, the `workflowName` was _not_ the path to the
workflow.
2025-09-24 09:15:03 -07:00
Michael Bolin
639a6fd2f3 chore: upgrade to Rust 1.90 (#4124)
Inspired by Dependabot's attempt to do this:
https://github.com/openai/codex/pull/4029

The new version of Clippy found some unused structs that are removed in
this PR.

Though nothing stood out to me in the Release Notes in terms of things
we should start to take advantage of:
https://blog.rust-lang.org/2025/09/18/Rust-1.90.0/.
2025-09-24 08:32:00 -07:00
jif-oai
db4aa6f916 nit: 350k tokens (#4156)
350k tokens for gpt-5-codex auto-compaction and update comments for
better description
2025-09-24 15:31:27 +00:00
Ahmed Ibrahim
cb96f4f596 Add Reset in for rate limits (#4111)
- Parse the headers
- Reorganize the struct because it's getting too long
- show the resets at in the tui

<img width="324" height="79" alt="image"
src="https://github.com/user-attachments/assets/ca15cd48-f112-4556-91ab-1e3a9bc4683d"
/>
2025-09-24 15:31:08 +00:00
jif-oai
5b910f1f05 chore: extract readiness in a dedicated utils crate (#4140)
Create an `utils` directory for the small utils crates
2025-09-24 10:15:54 +00:00
jif-oai
af6304c641 nit: drop instruction override for auto-compact (#4137)
drop instruction override for auto-compact as this is not used and
dangerous as it invalidates the cache
2025-09-24 10:47:12 +01:00
jif-oai
b90eeabd74 nit: update auto compact to 250k (#4135)
update auto compact for gpt-5-codex to 250k
2025-09-24 09:41:33 +00:00
dependabot[bot]
f7d2f3e54d chore(deps): bump tempfile from 3.20.0 to 3.22.0 in /codex-rs (#4030)
Bumps [tempfile](https://github.com/Stebalien/tempfile) from 3.20.0 to
3.22.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md">tempfile's
changelog</a>.</em></p>
<blockquote>
<h2>3.22.0</h2>
<ul>
<li>Updated <code>windows-sys</code> requirement to allow version
0.61.x</li>
<li>Remove <code>unstable-windows-keep-open-tempfile</code>
feature.</li>
</ul>
<h2>3.21.0</h2>
<ul>
<li>Updated <code>windows-sys</code> requirement to allow version
0.60.x</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="f720dbe098"><code>f720dbe</code></a>
chore: release 3.22.0</li>
<li><a
href="55d742cb5d"><code>55d742c</code></a>
chore: remove deprecated unstable feature flag</li>
<li><a
href="bc41a0b586"><code>bc41a0b</code></a>
build(deps): update windows-sys requirement from &gt;=0.52, &lt;0.61 to
&gt;=0.52, &lt;0....</li>
<li><a
href="3c55387ede"><code>3c55387</code></a>
test: make sure we don't drop tempdirs early (<a
href="https://redirect.github.com/Stebalien/tempfile/issues/373">#373</a>)</li>
<li><a
href="17bf644406"><code>17bf644</code></a>
doc(builder): clarify permissions (<a
href="https://redirect.github.com/Stebalien/tempfile/issues/372">#372</a>)</li>
<li><a
href="c7423f1761"><code>c7423f1</code></a>
doc(env): document the alternative to setting the tempdir (<a
href="https://redirect.github.com/Stebalien/tempfile/issues/371">#371</a>)</li>
<li><a
href="5af60ca9e3"><code>5af60ca</code></a>
test(wasi): run a few tests that shouldn't have been disabled (<a
href="https://redirect.github.com/Stebalien/tempfile/issues/370">#370</a>)</li>
<li><a
href="6c0c56198a"><code>6c0c561</code></a>
fix(doc): temp_dir doesn't check if writable</li>
<li><a
href="48bff5f54c"><code>48bff5f</code></a>
test(tempdir): configure tempdir on wasi</li>
<li><a
href="704a1d2752"><code>704a1d2</code></a>
test(tempdir): cleanup tempdir tests and run more tests on wasi</li>
<li>Additional commits viewable in <a
href="https://github.com/Stebalien/tempfile/compare/v3.20.0...v3.22.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=tempfile&package-manager=cargo&previous-version=3.20.0&new-version=3.22.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-23 23:41:35 -07:00
dependabot[bot]
3fe3b6328b chore(deps): bump log from 0.4.27 to 0.4.28 in /codex-rs (#4027)
[//]: # (dependabot-start)
⚠️  **Dependabot is rebasing this PR** ⚠️ 

Rebasing might not happen immediately, so don't worry if this takes some
time.

Note: if you make any changes to this PR yourself, they will take
precedence over the rebase.

---

[//]: # (dependabot-end)

Bumps [log](https://github.com/rust-lang/log) from 0.4.27 to 0.4.28.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/rust-lang/log/releases">log's
releases</a>.</em></p>
<blockquote>
<h2>0.4.28</h2>
<h2>What's Changed</h2>
<ul>
<li>ci: drop really old trick and ensure MSRV for all feature combo by
<a href="https://github.com/tisonkun"><code>@​tisonkun</code></a> in <a
href="https://redirect.github.com/rust-lang/log/pull/676">rust-lang/log#676</a></li>
<li>chore: fix some typos in comment by <a
href="https://github.com/xixishidibei"><code>@​xixishidibei</code></a>
in <a
href="https://redirect.github.com/rust-lang/log/pull/677">rust-lang/log#677</a></li>
<li>Unhide <code>#[derive(Debug)]</code> in example by <a
href="https://github.com/ZylosLumen"><code>@​ZylosLumen</code></a> in <a
href="https://redirect.github.com/rust-lang/log/pull/688">rust-lang/log#688</a></li>
<li>Chore: delete compare_exchange method for AtomicUsize on platforms
without atomics by <a
href="https://github.com/HaoliangXu"><code>@​HaoliangXu</code></a> in <a
href="https://redirect.github.com/rust-lang/log/pull/690">rust-lang/log#690</a></li>
<li>Add <code>increment_severity()</code> and
<code>decrement_severity()</code> methods for <code>Level</code> and
<code>LevelFilter</code> by <a
href="https://github.com/nebkor"><code>@​nebkor</code></a> in <a
href="https://redirect.github.com/rust-lang/log/pull/692">rust-lang/log#692</a></li>
<li>Prepare for 0.4.28 release by <a
href="https://github.com/KodrAus"><code>@​KodrAus</code></a> in <a
href="https://redirect.github.com/rust-lang/log/pull/695">rust-lang/log#695</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/xixishidibei"><code>@​xixishidibei</code></a>
made their first contribution in <a
href="https://redirect.github.com/rust-lang/log/pull/677">rust-lang/log#677</a></li>
<li><a
href="https://github.com/ZylosLumen"><code>@​ZylosLumen</code></a> made
their first contribution in <a
href="https://redirect.github.com/rust-lang/log/pull/688">rust-lang/log#688</a></li>
<li><a
href="https://github.com/HaoliangXu"><code>@​HaoliangXu</code></a> made
their first contribution in <a
href="https://redirect.github.com/rust-lang/log/pull/690">rust-lang/log#690</a></li>
<li><a href="https://github.com/nebkor"><code>@​nebkor</code></a> made
their first contribution in <a
href="https://redirect.github.com/rust-lang/log/pull/692">rust-lang/log#692</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/rust-lang/log/compare/0.4.27...0.4.28">https://github.com/rust-lang/log/compare/0.4.27...0.4.28</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/rust-lang/log/blob/master/CHANGELOG.md">log's
changelog</a>.</em></p>
<blockquote>
<h2>[0.4.28] - 2025-09-02</h2>
<h2>What's Changed</h2>
<ul>
<li>ci: drop really old trick and ensure MSRV for all feature combo by
<a href="https://github.com/tisonkun"><code>@​tisonkun</code></a> in <a
href="https://redirect.github.com/rust-lang/log/pull/676">rust-lang/log#676</a></li>
<li>Chore: delete compare_exchange method for AtomicUsize on platforms
without atomics by <a
href="https://github.com/HaoliangXu"><code>@​HaoliangXu</code></a> in <a
href="https://redirect.github.com/rust-lang/log/pull/690">rust-lang/log#690</a></li>
<li>Add <code>increment_severity()</code> and
<code>decrement_severity()</code> methods for <code>Level</code> and
<code>LevelFilter</code> by <a
href="https://github.com/nebkor"><code>@​nebkor</code></a> in <a
href="https://redirect.github.com/rust-lang/log/pull/692">rust-lang/log#692</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/xixishidibei"><code>@​xixishidibei</code></a>
made their first contribution in <a
href="https://redirect.github.com/rust-lang/log/pull/677">rust-lang/log#677</a></li>
<li><a
href="https://github.com/ZylosLumen"><code>@​ZylosLumen</code></a> made
their first contribution in <a
href="https://redirect.github.com/rust-lang/log/pull/688">rust-lang/log#688</a></li>
<li><a
href="https://github.com/HaoliangXu"><code>@​HaoliangXu</code></a> made
their first contribution in <a
href="https://redirect.github.com/rust-lang/log/pull/690">rust-lang/log#690</a></li>
<li><a href="https://github.com/nebkor"><code>@​nebkor</code></a> made
their first contribution in <a
href="https://redirect.github.com/rust-lang/log/pull/692">rust-lang/log#692</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/rust-lang/log/compare/0.4.27...0.4.28">https://github.com/rust-lang/log/compare/0.4.27...0.4.28</a></p>
<h3>Notable Changes</h3>
<ul>
<li>MSRV is bumped to 1.61.0 in <a
href="https://redirect.github.com/rust-lang/log/pull/676">rust-lang/log#676</a></li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="6e1735597b"><code>6e17355</code></a>
Merge pull request <a
href="https://redirect.github.com/rust-lang/log/issues/695">#695</a>
from rust-lang/cargo/0.4.28</li>
<li><a
href="57719dbef5"><code>57719db</code></a>
focus on user-facing source changes in the changelog</li>
<li><a
href="e0630c6485"><code>e0630c6</code></a>
prepare for 0.4.28 release</li>
<li><a
href="60829b11f5"><code>60829b1</code></a>
Merge pull request <a
href="https://redirect.github.com/rust-lang/log/issues/692">#692</a>
from nebkor/up-and-down</li>
<li><a
href="95d44f8af5"><code>95d44f8</code></a>
change names of log-level-changing methods to be more descriptive</li>
<li><a
href="2b63dfada6"><code>2b63dfa</code></a>
Add <code>up()</code> and <code>down()</code> methods for
<code>Level</code> and <code>LevelFilter</code></li>
<li><a
href="3aa1359e92"><code>3aa1359</code></a>
Merge pull request <a
href="https://redirect.github.com/rust-lang/log/issues/690">#690</a>
from HaoliangXu/master</li>
<li><a
href="1091f2cbd2"><code>1091f2c</code></a>
Chore:delete compare_exchange method for AtomicUsize on platforms</li>
<li><a
href="24c5f44efd"><code>24c5f44</code></a>
Merge pull request <a
href="https://redirect.github.com/rust-lang/log/issues/688">#688</a>
from ZylosLumen/patch-1</li>
<li><a
href="4498495467"><code>4498495</code></a>
Unhide <code>#[derive(Debug)]</code> in example</li>
<li>Additional commits viewable in <a
href="https://github.com/rust-lang/log/compare/0.4.27...0.4.28">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=log&package-manager=cargo&previous-version=0.4.27&new-version=0.4.28)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-23 23:07:54 -07:00
dependabot[bot]
8144ddb3da chore(deps): bump serde from 1.0.224 to 1.0.226 in /codex-rs (#4031)
[//]: # (dependabot-start)
⚠️  **Dependabot is rebasing this PR** ⚠️ 

Rebasing might not happen immediately, so don't worry if this takes some
time.

Note: if you make any changes to this PR yourself, they will take
precedence over the rebase.

---

[//]: # (dependabot-end)

Bumps [serde](https://github.com/serde-rs/serde) from 1.0.224 to
1.0.226.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/serde-rs/serde/releases">serde's
releases</a>.</em></p>
<blockquote>
<h2>v1.0.226</h2>
<ul>
<li>Deduplicate variant matching logic inside generated Deserialize impl
for adjacently tagged enums (<a
href="https://redirect.github.com/serde-rs/serde/issues/2935">#2935</a>,
thanks <a
href="https://github.com/Mingun"><code>@​Mingun</code></a>)</li>
</ul>
<h2>v1.0.225</h2>
<ul>
<li>Avoid triggering a deprecation warning in derived Serialize and
Deserialize impls for a data structure that contains its own
deprecations (<a
href="https://redirect.github.com/serde-rs/serde/issues/2879">#2879</a>,
thanks <a
href="https://github.com/rcrisanti"><code>@​rcrisanti</code></a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="1799547846"><code>1799547</code></a>
Release 1.0.226</li>
<li><a
href="2dbeefb11b"><code>2dbeefb</code></a>
Merge pull request <a
href="https://redirect.github.com/serde-rs/serde/issues/2935">#2935</a>
from Mingun/dedupe-adj-enums</li>
<li><a
href="8a3c29ff19"><code>8a3c29f</code></a>
Merge pull request <a
href="https://redirect.github.com/serde-rs/serde/issues/2986">#2986</a>
from dtolnay/didnotwork</li>
<li><a
href="defc24d361"><code>defc24d</code></a>
Remove &quot;did not work&quot; comment from test suite</li>
<li><a
href="2316610760"><code>2316610</code></a>
Merge pull request <a
href="https://redirect.github.com/serde-rs/serde/issues/2929">#2929</a>
from Mingun/flatten-enum-tests</li>
<li><a
href="c09e2bd690"><code>c09e2bd</code></a>
Add tests for flatten unit variant in adjacently tagged (tag + content)
enums</li>
<li><a
href="fe7dcc4cd8"><code>fe7dcc4</code></a>
Test all possible orders of map entries for enum-flatten-in-struct
representa...</li>
<li><a
href="a20e66e131"><code>a20e66e</code></a>
Check serialization in
flatten::enum_::internally_tagged::unit_enum_with_unkn...</li>
<li><a
href="1c1a5d95cd"><code>1c1a5d9</code></a>
Reorder struct_ and newtype tests of adjacently_tagged enums to match
order i...</li>
<li><a
href="ee3c2372fb"><code>ee3c237</code></a>
Opt in to generate-macro-expansion when building on docs.rs</li>
<li>Additional commits viewable in <a
href="https://github.com/serde-rs/serde/compare/v1.0.224...v1.0.226">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=serde&package-manager=cargo&previous-version=1.0.224&new-version=1.0.226)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-23 23:06:30 -07:00
Michael Bolin
9336f2b84b fix: npm publish --tag alpha when building an alpha release (#4112)
This updates our release process so that when we build an alpha of the
Codex CLI (as determined by pushing a tag of the format
`rust-v<cli-version>-alpha.<alpha-version>`), we will now publish the
corresponding npm module publicly, but under the `alpha` tag. As you can
see, this PR adds `--tag alpha` to the `npm publish` command, as
appropriate.
2025-09-23 23:03:43 -07:00
Michael Bolin
af37785bca fix: vendor ripgrep in the npm module (#3660)
We try to ensure ripgrep (`rg`) is provided with Codex.

- For `brew`, we declare it as a dependency of our formula:

08d82d8b00/Formula/c/codex.rb (L24)
- For `npm`, we declare `@vscode/ripgrep` as a dependency, which
installs the platform-specific binary as part of a `postinstall` script:

fdb8dadcae/codex-cli/package.json (L22)
- Users who download the CLI directly from GitHub Releases are on their
own.

In practice, I have seen `@vscode/ripgrep` fail on occasion. Here is a
trace from a GitHub workflow:

```
npm error code 1
npm error path /Users/runner/hostedtoolcache/node/20.19.5/arm64/lib/node_modules/@openai/codex/node_modules/@vscode/ripgrep
npm error command failed
npm error command sh -c node ./lib/postinstall.js
npm error Finding release for v13.0.0-13
npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13
npm error Deleting invalid download cache
npm error Download attempt 1 failed, retrying in 2 seconds...
npm error Finding release for v13.0.0-13
npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13
npm error Deleting invalid download cache
npm error Download attempt 2 failed, retrying in 4 seconds...
npm error Finding release for v13.0.0-13
npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13
npm error Deleting invalid download cache
npm error Download attempt 3 failed, retrying in 8 seconds...
npm error Finding release for v13.0.0-13
npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13
npm error Deleting invalid download cache
npm error Download attempt 4 failed, retrying in 16 seconds...
npm error Finding release for v13.0.0-13
npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13
npm error Deleting invalid download cache
npm error Error: Request failed: 403
```

To eliminate this error, this PR changes things so that we vendor the
`rg` binary into https://www.npmjs.com/package/@openai/codex so it is
guaranteed to be included when a user runs `npm i -g @openai/codex`.

The downside of this approach is the increase in package size: we
include the `rg` executable for six architectures (in addition to the
six copies of `codex` we already include). In a follow-up, I plan to add
support for "slices" of our npm module, so that soon users will be able
to do:

```
npm install -g @openai/codex@aarch64-apple-darwin
```

Admittedly, this is a sizable change and I tried to clean some things up
in the process:

- `install_native_deps.sh` has been replaced by `install_native_deps.py`
- `stage_release.sh` and `stage_rust_release.py` has been replaced by
`build_npm_package.py`

We now vendor in a DotSlash file for ripgrep (as a modest attempt to
facilitate local testing) and then build up the extension by:

- creating a temp directory and copying `package.json` over to it with
the target value for `"version"`
- finding the GitHub workflow that corresponds to the
`--release-version` and copying the various `codex` artifacts to
respective `vendor/TARGET_TRIPLE/codex` folder
- downloading the `rg` artifacts specified in the DotSlash file and
copying them over to the respective `vendor/TARGET_TRIPLE/path` folder
- if `--pack-output` is specified, runs `npm pack` on the temp directory

To test, I downloaded the artifact produced by this CI job:


https://github.com/openai/codex/actions/runs/17961595388/job/51085840022?pr=3660

and verified that `node ./bin/codex.js 'which -a rg'` worked as
intended.
2025-09-23 23:00:33 -07:00
Dylan
594248f415 [exec] add include-plan-tool flag and print it nicely (#3461)
### Summary
Sometimes in exec runs, we want to allow the model to use the
`update_plan` tool, but that's not easily configurable. This change adds
a feature flag for this, and formats the output so it's human-readable

## Test Plan
<img width="1280" height="354" alt="Screenshot 2025-09-11 at 12 39
44 AM"
src="https://github.com/user-attachments/assets/72e11070-fb98-47f5-a784-5123ca7333d9"
/>
2025-09-23 16:50:59 -07:00
Ahmed Ibrahim
8227a5ba1b Send limits when getting rate limited (#4102)
Users need visibility on rate limits when they are rate limited.
2025-09-23 22:56:34 +00:00
pakrym-oai
fdb8dadcae Add exec output-schema parameter (#4079)
Adds structured output to `exec` via the `--structured-output`
parameter.
2025-09-23 13:59:16 -07:00
pakrym-oai
0f9a796617 Use anyhow::Result in tests for error propagation (#4105) 2025-09-23 13:31:36 -07:00
Ahmed Ibrahim
c6e8671b2a Refactor codex card layout (#4069)
Refactor it to be used in status
2025-09-23 17:37:14 +00:00
jif-oai
b84a920067 chore: compact do not modify instructions (#4088)
Keep the developer instruction and insert the summarisation message as a
user message instead
2025-09-23 17:59:17 +01:00
jif-oai
6cd5309d91 feat: readiness tool (#4090)
Readiness flag with token-based subscription and async wait function
that waits for all the subscribers to be ready
2025-09-23 17:27:20 +01:00
Ahmed Ibrahim
664ee07540 Rate limits warning (#4075)
Only show the highest warning rate.
Change the warning threshold
2025-09-23 09:15:16 -07:00
ae
51c465bddc fix: usage data tweaks (#4082)
- Only show the usage data section when signed in with ChatGPT. (Tested
with Chat auth and API auth.)
- Friendlier string change.
- Also removed `.dim()` on the string, since it was the only string in
`/status` that was dim.
2025-09-23 09:14:02 -07:00
jif-oai
e0fbc112c7 feat: git tooling for undo (#3914)
## Summary
Introduces a “ghost commit” workflow that snapshots the tree without
touching refs.
1. git commit-tree writes an unreferenced commit object from the current
index, optionally pointing to the current HEAD as its parent.
2. We then stash that commit id and use git restore --source <ghost> to
roll the worktree (and index) back to the recorded snapshot later on.

## Details
- Ghost commits live only as loose objects—we never update branches or
tags—so the repo history stays untouched while still giving us a full
tree snapshot.
- Force-included paths let us stage otherwise ignored files before
capturing the tree.
- Restoration rehydrates both tracked and force-included files while
leaving untracked/ignored files alone.
2025-09-23 16:59:52 +01:00
pakrym-oai
76ecbb3d8e Use TestCodex builder in stream retry tests (#4096)
## Summary
- refactor the stream retry integration tests to construct conversations
through `TestCodex`
- remove bespoke config and tempdir setup now handled by the shared
builder

## Testing
- cargo test -p codex-core --test all
stream_error_allows_next_turn::continue_after_stream_error
- cargo test -p codex-core --test all
stream_no_completed::retries_on_early_close

------
https://chatgpt.com/codex/tasks/task_i_68d2b94d83888320bc75a0bc3bd77b49
2025-09-23 08:57:08 -07:00
jif-oai
2451b19d13 chore: enable auto-compaction for gpt-5-codex (#4093)
enable auto-compaction for `gpt-5-codex` at 220k tokens
2025-09-23 16:12:36 +01:00
pakrym-oai
5c7d9e27b1 Add notifier tests (#4064)
Proposal:
1. Use anyhow for tests and avoid unwrap
2. Extract a helper for starting a test instance of codex
2025-09-23 14:25:46 +00:00
Thibault Sottiaux
c93e77b68b feat: update default (#4076)
Changes:
- Default model and docs now use gpt-5-codex. 
- Disables the GPT-5 Codex NUX by default.
- Keeps presets available for API key users.
2025-09-22 20:10:52 -07:00
dedrisian-oai
c415827ac2 Truncate potentially long user messages in compact message. (#4068)
If a prior user message is massive, any future `/compact` task would
fail because we're verbatim copying the user message into the new chat.
2025-09-22 23:12:26 +00:00
Jeremy Rose
4e0550b995 fix codex resume message at end of session (#3957)
This was only being printed when running the codex-tui executable
directly, not via the codex-cli wrapper.
2025-09-22 22:24:31 +00:00
Jeremy Rose
f54a49157b Fix pager overlay clear between pages (#3952)
should fix characters sometimes hanging around while scrolling the
transcript.
2025-09-22 15:12:29 -07:00
Ahmed Ibrahim
dd56750612 Change headers and struct of rate limits (#4060) 2025-09-22 21:06:20 +00:00
dedrisian-oai
8bc73a2bfd Fix branch mode prompt for /review (#4061)
Updates `/review` branch mode to review against a branch's upstream.
2025-09-22 12:34:08 -07:00
jif-oai
be366a31ab chore: clippy on redundant closure (#4058)
Add redundant closure clippy rules and let Codex fix it by minimising
FQP
2025-09-22 19:30:16 +00:00
Ahmed Ibrahim
c75920a071 Change limits warning copy (#4059) 2025-09-22 18:52:45 +00:00
dedrisian-oai
8daba53808 feat: Add view stack to BottomPane (#4026)
Adds a "View Stack" to the bottom pane to allow for pushing/popping
bottom panels.

`esc` will go back instead of dismissing.

Benefit: We retain the "selection state" of a parent panel (e.g. the
review panel).
2025-09-22 11:29:39 -07:00
Ahmed Ibrahim
d2940bd4c3 Remove /limits after moving to /status (#4055)
Moved to /status #4053
2025-09-22 18:23:05 +00:00
friel-openai
76a9b11678 Tui: fix backtracking (#4020)
Backtracking multiple times could drop earlier turns. We now derive the
active user-turn positions from the transcript on demand (keying off the
latest session header) instead of caching state. This keeps the replayed
context intact during repeated edits and adds a regression test.
2025-09-22 11:16:25 -07:00
Jeremy Rose
fa80bbb587 simplify StreamController (#3928)
no intended functional change, just simplifying the code.
2025-09-22 11:14:04 -07:00
Ahmed Ibrahim
434eb4fd49 Add limits to /status (#4053)
Add limits to status

<img width="579" height="430" alt="image"
src="https://github.com/user-attachments/assets/d3794d92-ffca-47be-8011-b4452223cc89"
/>
2025-09-22 18:13:34 +00:00
Jeremy Rose
19f46439ae timeouts for mcp tool calls (#3959)
defaults to 60sec, overridable with MCP_TOOL_TIMEOUT or on a per-server
basis in the config.
2025-09-22 10:30:59 -07:00
jif-oai
e258ca61b4 chore: more clippy rules 2 (#4057)
The only file to watch is the cargo.toml
All the others come from just fix + a few manual small fix

The set of rules have been taken from the list of clippy rules
arbitrarily while trying to optimise the learning and style of the code
while limiting the loss of productivity
2025-09-22 17:16:02 +00:00
jif-oai
e5fe50d3ce chore: unify cargo versions (#4044)
Unify cargo versions at root
2025-09-22 16:47:01 +00:00
pakrym-oai
14a115d488 Add non_sandbox_test helper (#3880)
Makes tests shorter
2025-09-22 14:50:41 +00:00
dedrisian-oai
5996ee0e5f feat: Add more /review options (#3961)
Adds the following options:

1. Review current changes
2. Review a specific commit
3. Review against a base branch (PR style)
4. Custom instructions

<img width="487" height="330" alt="Screenshot 2025-09-20 at 2 11 36 PM"
src="https://github.com/user-attachments/assets/edb0aaa5-5747-47fa-881f-cc4c4f7fe8bc"
/>

---

\+ Adds the following UI helpers:

1. Makes list selection searchable
2. Adds navigation to the bottom pane, so you could add a stack of
popups
3. Basic custom prompt view
2025-09-21 20:18:35 -07:00
Ahmed Ibrahim
a4ebd069e5 Tui: Rate limits (#3977)
### /limits: show rate limits graph

<img width="442" height="287" alt="image"
src="https://github.com/user-attachments/assets/3e29a241-a4b0-4df8-bf71-43dc4dd805ca"
/>

### Warning on close to rate limits:

<img width="507" height="96" alt="image"
src="https://github.com/user-attachments/assets/732a958b-d240-4a89-8289-caa92de83537"
/>

Based on #3965
2025-09-21 10:20:49 -07:00
Ahmed Ibrahim
04504d8218 Forward Rate limits to the UI (#3965)
We currently get information about rate limits in the response headers.
We want to forward them to the clients to have better transparency.
UI/UX plans have been discussed and this information is needed.
2025-09-20 21:26:16 -07:00
Jeremy Rose
42d335deb8 Cache keyboard enhancement detection before event streams (#3950)
Hopefully fixes incorrectly showing ^J instead of Shift+Enter in the key
hints occasionally.
2025-09-19 21:38:36 +00:00
Jeremy Rose
ad0c2b4db3 don't clear screen on startup (#3925) 2025-09-19 14:22:58 -07:00
Jeremy Rose
ff389dc52f fix alignment in slash command popup (#3937) 2025-09-19 19:08:04 +00:00
pakrym-oai
9b18875a42 Use helpers instead of fixtures (#3888)
Move to using test helper method everywhere.
2025-09-19 06:46:25 -07:00
pakrym-oai
881c7978f1 Move responses mocking helpers to a shared lib (#3878)
These are generally useful
2025-09-18 17:53:14 -07:00
Ahmed Ibrahim
a7fda70053 Use a unified shell tell to not break cache (#3814)
Currently, we change the tool description according to the sandbox
policy and approval policy. This breaks the cache when the user hits
`/approvals`. This PR does the following:
- Always use the shell with escalation parameter:
- removes `create_shell_tool_for_sandbox` and always uses unified tool
via `create_shell_tool`
- Reject the func call when the model uses escalation parameter when it
cannot.
2025-09-19 00:08:28 +00:00
Michael Bolin
de64f5f007 fix: update try_parse_word_only_commands_sequence() to return commands in order (#3881)
Incidentally, we had a test for this in
`accepts_multiple_commands_with_allowed_operators()`, but it was
verifying the bad behavior. Oops!
2025-09-18 16:07:38 -07:00
Michael Bolin
8595237505 fix: ensure cwd for conversation and sandbox are separate concerns (#3874)
Previous to this PR, both of these functions take a single `cwd`:


71038381aa/codex-rs/core/src/seatbelt.rs (L19-L25)


71038381aa/codex-rs/core/src/landlock.rs (L16-L23)

whereas `cwd` and `sandbox_cwd` should be set independently (fixed in
this PR).

Added `sandbox_distinguishes_command_and_policy_cwds()` to
`codex-rs/exec/tests/suite/sandbox.rs` to verify this.
2025-09-18 14:37:06 -07:00
dedrisian-oai
62258df92f feat: /review (#3774)
Adds `/review` action in TUI

<img width="637" height="370" alt="Screenshot 2025-09-17 at 12 41 19 AM"
src="https://github.com/user-attachments/assets/b1979a6e-844a-4b97-ab20-107c185aec1d"
/>
2025-09-18 14:14:16 -07:00
Jeremy Rose
b34e906396 Reland "refactor transcript view to handle HistoryCells" (#3753)
Reland of #3538
2025-09-18 20:55:53 +00:00
Jeremy Rose
71038381aa fix error on missing notifications in [tui] (#3867)
Fixes #3811.
2025-09-18 11:25:09 -07:00
jif-oai
277fc6254e chore: use tokio mutex and async function to prevent blocking a worker (#3850)
### Why Use `tokio::sync::Mutex`

`std::sync::Mutex` are not _async-aware_. As a result, they will block
the entire thread instead of just yielding the task. Furthermore they
can be poisoned which is not the case of `tokio` Mutex.
This allows the Tokio runtime to continue running other tasks while
waiting for the lock, preventing deadlocks and performance bottlenecks.

In general, this is preferred in async environment
2025-09-18 18:21:52 +01:00
jif-oai
992b531180 fix: some nit Rust reference issues (#3849)
Fix some small references issue. No behavioural change. Just making the
code cleaner
2025-09-18 18:18:06 +01:00
Jeremy Rose
84a0ba9bf5 hint for codex resume on tui exit (#3757)
<img width="931" height="438" alt="Screenshot 2025-09-16 at 4 25 19 PM"
src="https://github.com/user-attachments/assets/ccfb8df1-feaf-45b4-8f7f-56100de916d5"
/>
2025-09-18 09:28:32 -07:00
jif-oai
4a5d6f7c71 Make ESC button work when auto-compaction (#3857)
Only emit a task finished when the compaction comes from a `/compact`
2025-09-18 15:34:16 +00:00
jif-oai
1b3c8b8e94 Unify animations (#3729)
Unify the animation in a single code and add the CTRL + . in the
onboarding
2025-09-18 16:27:15 +01:00
pakrym-oai
d4aba772cb Switch to uuid_v7 and tighten ConversationId usage (#3819)
Make sure conversations have a timestamp.
2025-09-18 14:37:03 +00:00
jif-oai
4c97eeb32a bug: Ignore tests for now (#3777)
Ignore flaky / long tests for now
2025-09-18 10:43:45 +01:00
Thibault Sottiaux
c9505488a1 chore: update "Codex CLI harness, sandboxing, and approvals" section (#3822) 2025-09-17 16:48:20 -07:00
Jeremy Rose
530382db05 Use agent reply text in turn notifications (#3756)
Instead of "Agent turn complete", turn-complete notifications now
include the first handful of chars from the agent's final message.
2025-09-17 11:23:46 -07:00
Abhishek Bhardwaj
208089e58e AGENTS.md: Add instruction to install missing commands (#3807)
This change instructs the model to install any missing command. Else
tokens are wasted when it tries to run
commands that aren't available multiple times before installing them.
2025-09-17 11:06:59 -07:00
Michael Bolin
e5fdb5b0fd fix: specify --repo when calling gh (#3806)
Often, `gh` infers `--repo` when it is run from a Git clone, but our
`publish-npm` step is designed to avoid the overhead of cloning the
repo, so add the `--repo` option explicitly to fix things.
2025-09-17 11:05:22 -07:00
Michael Bolin
5332f6e215 fix: make publish-npm its own job with specific permissions (#3767)
The build for `v0.37.0-alpha.3` failed on the `Create GitHub Release`
step:

https://github.com/openai/codex/actions/runs/17786866086/job/50556513221

with:

```
⚠️ GitHub release failed with status: 403
{"message":"Resource not accessible by integration","documentation_url":"https://docs.github.com/rest/releases/releases#create-a-release","status":"403"}
Skip retry — your GitHub token/PAT does not have the required permission to create a release
```

I believe I should have not introduced a top-level `permissions` for the
workflow in https://github.com/openai/codex/pull/3431 because that
affected the `permissions` for each job in the workflow.

This PR introduces `publish-npm` as its own job, which allows us to:

- consolidate all the Node.js-related steps required for publishing
- limit the reach of the `id-token: write` permission
- skip it altogether if is an alpha build

With this PR, each of `release`, `publish-npm`, and `update-branch` has
an explicit `permissions` block.
2025-09-16 22:55:53 -07:00
299 changed files with 18192 additions and 8405 deletions

View File

@@ -27,12 +27,26 @@ jobs:
- name: Install dependencies
run: pnpm install --frozen-lockfile
# Run all tasks using workspace filters
# build_npm_package.py requires DotSlash when staging releases.
- uses: facebook/install-dotslash@v2
- name: Ensure staging a release works.
- name: Stage npm package
env:
GH_TOKEN: ${{ github.token }}
run: ./codex-cli/scripts/stage_release.sh
run: |
set -euo pipefail
CODEX_VERSION=0.40.0
PACK_OUTPUT="${RUNNER_TEMP}/codex-npm.tgz"
python3 ./codex-cli/scripts/build_npm_package.py \
--release-version "$CODEX_VERSION" \
--pack-output "$PACK_OUTPUT"
echo "PACK_OUTPUT=$PACK_OUTPUT" >> "$GITHUB_ENV"
- name: Upload staged npm package artifact
uses: actions/upload-artifact@v4
with:
name: codex-npm-staging
path: ${{ env.PACK_OUTPUT }}
- name: Ensure root README.md contains only ASCII and certain Unicode code points
run: ./scripts/asciicheck.py README.md

View File

@@ -22,7 +22,7 @@ jobs:
- name: Annotate locations with typos
uses: codespell-project/codespell-problem-matcher@b80729f885d32f78a716c2f107b4db1025001c42 # v1
- name: Codespell
uses: codespell-project/actions-codespell@406322ec52dd7b488e48c1c4b82e2a8b3a1bf630 # v2
uses: codespell-project/actions-codespell@406322ec52dd7b488e48c1c4b82e2a8b3a1bf630 # v2.1
with:
ignore_words_file: .codespellignore
skip: frame*.txt

View File

@@ -57,7 +57,7 @@ jobs:
working-directory: codex-rs
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.89
- uses: dtolnay/rust-toolchain@1.90
with:
components: rustfmt
- name: cargo fmt
@@ -75,7 +75,7 @@ jobs:
working-directory: codex-rs
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.89
- uses: dtolnay/rust-toolchain@1.90
- uses: taiki-e/install-action@0c5db7f7f897c03b771660e91d065338615679f4 # v2
with:
tool: cargo-shear
@@ -143,7 +143,7 @@ jobs:
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.89
- uses: dtolnay/rust-toolchain@1.90
with:
targets: ${{ matrix.target }}
components: clippy

View File

@@ -11,9 +11,6 @@ on:
tags:
- "rust-v*.*.*"
permissions:
id-token: write # Required for OIDC
concurrency:
group: ${{ github.workflow }}
cancel-in-progress: true
@@ -80,7 +77,7 @@ jobs:
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.89
- uses: dtolnay/rust-toolchain@1.90
with:
targets: ${{ matrix.target }}
@@ -170,6 +167,14 @@ jobs:
needs: build
name: release
runs-on: ubuntu-latest
permissions:
contents: write
actions: read
outputs:
version: ${{ steps.release_name.outputs.name }}
tag: ${{ github.ref_name }}
should_publish_npm: ${{ steps.npm_publish_settings.outputs.should_publish }}
npm_tag: ${{ steps.npm_publish_settings.outputs.npm_tag }}
steps:
- name: Checkout repository
@@ -190,43 +195,37 @@ jobs:
version="${GITHUB_REF_NAME#rust-v}"
echo "name=${version}" >> $GITHUB_OUTPUT
# Publish to npm using OIDC authentication.
# July 31, 2025: https://github.blog/changelog/2025-07-31-npm-trusted-publishing-with-oidc-is-generally-available/
# npm docs: https://docs.npmjs.com/trusted-publishers
- name: Determine npm publish settings
id: npm_publish_settings
env:
VERSION: ${{ steps.release_name.outputs.name }}
run: |
set -euo pipefail
version="${VERSION}"
# package.json has `packageManager: "pnpm@`, so we must get pnpm on the
# PATH before setting up Node.js.
- name: Setup pnpm
uses: pnpm/action-setup@v4
with:
run_install: false
- name: Setup Node.js
uses: actions/setup-node@v5
with:
node-version: 22
registry-url: "https://registry.npmjs.org"
scope: "@openai"
# Trusted publishing requires npm CLI version 11.5.1 or later.
- name: Update npm
run: npm install -g npm@latest
if [[ "${version}" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo "should_publish=true" >> "$GITHUB_OUTPUT"
echo "npm_tag=" >> "$GITHUB_OUTPUT"
elif [[ "${version}" =~ ^[0-9]+\.[0-9]+\.[0-9]+-alpha\.[0-9]+$ ]]; then
echo "should_publish=true" >> "$GITHUB_OUTPUT"
echo "npm_tag=alpha" >> "$GITHUB_OUTPUT"
else
echo "should_publish=false" >> "$GITHUB_OUTPUT"
echo "npm_tag=" >> "$GITHUB_OUTPUT"
fi
# build_npm_package.py requires DotSlash when staging releases.
- uses: facebook/install-dotslash@v2
- name: Stage npm package
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
TMP_DIR="${RUNNER_TEMP}/npm-stage"
python3 codex-cli/scripts/stage_rust_release.py \
./codex-cli/scripts/build_npm_package.py \
--release-version "${{ steps.release_name.outputs.name }}" \
--tmp "${TMP_DIR}"
mkdir -p dist/npm
# Produce an npm-ready tarball using `npm pack` and store it in dist/npm.
# We then rename it to a stable name used by our publishing script.
(cd "$TMP_DIR" && npm pack --pack-destination "${GITHUB_WORKSPACE}/dist/npm")
mv "${GITHUB_WORKSPACE}"/dist/npm/*.tgz \
"${GITHUB_WORKSPACE}/dist/npm/codex-npm-${{ steps.release_name.outputs.name }}.tgz"
--staging-dir "${TMP_DIR}" \
--pack-output "${GITHUB_WORKSPACE}/dist/npm/codex-npm-${{ steps.release_name.outputs.name }}.tgz"
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
@@ -245,11 +244,57 @@ jobs:
tag: ${{ github.ref_name }}
config: .github/dotslash-config.json
# Publish to npm using OIDC authentication.
# July 31, 2025: https://github.blog/changelog/2025-07-31-npm-trusted-publishing-with-oidc-is-generally-available/
# npm docs: https://docs.npmjs.com/trusted-publishers
publish-npm:
# Publish to npm for stable releases and alpha pre-releases with numeric suffixes.
if: ${{ needs.release.outputs.should_publish_npm == 'true' }}
name: publish-npm
needs: release
runs-on: ubuntu-latest
permissions:
id-token: write # Required for OIDC
contents: read
steps:
- name: Setup Node.js
uses: actions/setup-node@v5
with:
node-version: 22
registry-url: "https://registry.npmjs.org"
scope: "@openai"
# Trusted publishing requires npm CLI version 11.5.1 or later.
- name: Update npm
run: npm install -g npm@latest
- name: Download npm tarball from release
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
set -euo pipefail
version="${{ needs.release.outputs.version }}"
tag="${{ needs.release.outputs.tag }}"
mkdir -p dist/npm
gh release download "$tag" \
--repo "${GITHUB_REPOSITORY}" \
--pattern "codex-npm-${version}.tgz" \
--dir dist/npm
# No NODE_AUTH_TOKEN needed because we use OIDC.
- name: Publish to npm
# Do not publish alphas to npm.
if: ${{ !contains(steps.release_name.outputs.name, '-') }}
run: npm publish "${GITHUB_WORKSPACE}/dist/npm/codex-npm-${{ steps.release_name.outputs.name }}.tgz"
env:
VERSION: ${{ needs.release.outputs.version }}
NPM_TAG: ${{ needs.release.outputs.npm_tag }}
run: |
set -euo pipefail
tag_args=()
if [[ -n "${NPM_TAG}" ]]; then
tag_args+=(--tag "${NPM_TAG}")
fi
npm publish "${GITHUB_WORKSPACE}/dist/npm/codex-npm-${VERSION}.tgz" "${tag_args[@]}"
update-branch:
name: Update latest-alpha-cli branch

View File

@@ -4,6 +4,7 @@ In the codex-rs folder where the rust code lives:
- Crate names are prefixed with `codex-`. For example, the `core` folder's crate is named `codex-core`
- When using format! and you can inline variables into {}, always do that.
- Install any commands the repo relies on (for example `just`, `rg`, or `cargo-insta`) if they aren't already available before running instructions here.
- Never add or modify any code related to `CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR` or `CODEX_SANDBOX_ENV_VAR`.
- You operate in a sandbox where `CODEX_SANDBOX_NETWORK_DISABLED=1` will be set whenever you use the `shell` tool. Any existing code that uses `CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR` was authored with this fact in mind. It is often used to early exit out of tests that the author knew you would not be able to run given your sandbox limitations.
- Similarly, when you spawn a process using Seatbelt (`/usr/bin/sandbox-exec`), `CODEX_SANDBOX=seatbelt` will be set on the child process. Integration tests that want to run Seatbelt themselves cannot be run under Seatbelt, so checks for `CODEX_SANDBOX=seatbelt` are also often used to early exit out of tests, as appropriate.

View File

@@ -1,7 +1 @@
# Added by ./scripts/install_native_deps.sh
/bin/codex-aarch64-apple-darwin
/bin/codex-aarch64-unknown-linux-musl
/bin/codex-linux-sandbox-arm64
/bin/codex-linux-sandbox-x64
/bin/codex-x86_64-apple-darwin
/bin/codex-x86_64-unknown-linux-musl
/vendor/

View File

@@ -1,6 +1,7 @@
#!/usr/bin/env node
// Unified entry point for the Codex CLI.
import { existsSync } from "fs";
import path from "path";
import { fileURLToPath } from "url";
@@ -40,10 +41,10 @@ switch (platform) {
case "win32":
switch (arch) {
case "x64":
targetTriple = "x86_64-pc-windows-msvc.exe";
targetTriple = "x86_64-pc-windows-msvc";
break;
case "arm64":
targetTriple = "aarch64-pc-windows-msvc.exe";
targetTriple = "aarch64-pc-windows-msvc";
break;
default:
break;
@@ -57,7 +58,10 @@ if (!targetTriple) {
throw new Error(`Unsupported platform: ${platform} (${arch})`);
}
const binaryPath = path.join(__dirname, "..", "bin", `codex-${targetTriple}`);
const vendorRoot = path.join(__dirname, "..", "vendor");
const archRoot = path.join(vendorRoot, targetTriple);
const codexBinaryName = process.platform === "win32" ? "codex.exe" : "codex";
const binaryPath = path.join(archRoot, "codex", codexBinaryName);
// Use an asynchronous spawn instead of spawnSync so that Node is able to
// respond to signals (e.g. Ctrl-C / SIGINT) while the native binary is
@@ -66,23 +70,6 @@ const binaryPath = path.join(__dirname, "..", "bin", `codex-${targetTriple}`);
// receives a fatal signal, both processes exit in a predictable manner.
const { spawn } = await import("child_process");
async function tryImport(moduleName) {
try {
// eslint-disable-next-line node/no-unsupported-features/es-syntax
return await import(moduleName);
} catch (err) {
return null;
}
}
async function resolveRgDir() {
const ripgrep = await tryImport("@vscode/ripgrep");
if (!ripgrep?.rgPath) {
return null;
}
return path.dirname(ripgrep.rgPath);
}
function getUpdatedPath(newDirs) {
const pathSep = process.platform === "win32" ? ";" : ":";
const existingPath = process.env.PATH || "";
@@ -94,9 +81,9 @@ function getUpdatedPath(newDirs) {
}
const additionalDirs = [];
const rgDir = await resolveRgDir();
if (rgDir) {
additionalDirs.push(rgDir);
const pathDir = path.join(archRoot, "path");
if (existsSync(pathDir)) {
additionalDirs.push(pathDir);
}
const updatedPath = getUpdatedPath(additionalDirs);

79
codex-cli/bin/rg Executable file
View File

@@ -0,0 +1,79 @@
#!/usr/bin/env dotslash
{
"name": "rg",
"platforms": {
"macos-aarch64": {
"size": 1787248,
"hash": "blake3",
"digest": "8d9942032585ea8ee805937634238d9aee7b210069f4703c88fbe568e26fb78a",
"format": "tar.gz",
"path": "ripgrep-14.1.1-aarch64-apple-darwin/rg",
"providers": [
{
"url": "https://github.com/BurntSushi/ripgrep/releases/download/14.1.1/ripgrep-14.1.1-aarch64-apple-darwin.tar.gz"
}
]
},
"linux-aarch64": {
"size": 2047405,
"hash": "blake3",
"digest": "0b670b8fa0a3df2762af2fc82cc4932f684ca4c02dbd1260d4f3133fd4b2a515",
"format": "tar.gz",
"path": "ripgrep-14.1.1-aarch64-unknown-linux-gnu/rg",
"providers": [
{
"url": "https://github.com/BurntSushi/ripgrep/releases/download/14.1.1/ripgrep-14.1.1-aarch64-unknown-linux-gnu.tar.gz"
}
]
},
"macos-x86_64": {
"size": 2082672,
"hash": "blake3",
"digest": "e9b862fc8da3127f92791f0ff6a799504154ca9d36c98bf3e60a81c6b1f7289e",
"format": "tar.gz",
"path": "ripgrep-14.1.1-x86_64-apple-darwin/rg",
"providers": [
{
"url": "https://github.com/BurntSushi/ripgrep/releases/download/14.1.1/ripgrep-14.1.1-x86_64-apple-darwin.tar.gz"
}
]
},
"linux-x86_64": {
"size": 2566310,
"hash": "blake3",
"digest": "f73cca4e54d78c31f832c7f6e2c0b4db8b04fa3eaa747915727d570893dbee76",
"format": "tar.gz",
"path": "ripgrep-14.1.1-x86_64-unknown-linux-musl/rg",
"providers": [
{
"url": "https://github.com/BurntSushi/ripgrep/releases/download/14.1.1/ripgrep-14.1.1-x86_64-unknown-linux-musl.tar.gz"
}
]
},
"windows-x86_64": {
"size": 2058893,
"hash": "blake3",
"digest": "a8ce1a6fed4f8093ee997e57f33254e94b2cd18e26358b09db599c89882eadbd",
"format": "zip",
"path": "ripgrep-14.1.1-x86_64-pc-windows-msvc/rg.exe",
"providers": [
{
"url": "https://github.com/BurntSushi/ripgrep/releases/download/14.1.1/ripgrep-14.1.1-x86_64-pc-windows-msvc.zip"
}
]
},
"windows-aarch64": {
"size": 1667740,
"hash": "blake3",
"digest": "47b971a8c4fca1d23a4e7c19bd4d88465ebc395598458133139406d3bf85f3fa",
"format": "zip",
"path": "rg.exe",
"providers": [
{
"url": "https://github.com/microsoft/ripgrep-prebuilt/releases/download/v13.0.0-13/ripgrep-v13.0.0-13-aarch64-pc-windows-msvc.zip"
}
]
}
}
}

View File

@@ -2,118 +2,17 @@
"name": "@openai/codex",
"version": "0.0.0-dev",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "@openai/codex",
"version": "0.0.0-dev",
"license": "Apache-2.0",
"dependencies": {
"@vscode/ripgrep": "^1.15.14"
},
"bin": {
"codex": "bin/codex.js"
},
"engines": {
"node": ">=20"
}
},
"node_modules/@vscode/ripgrep": {
"version": "1.15.14",
"resolved": "https://registry.npmjs.org/@vscode/ripgrep/-/ripgrep-1.15.14.tgz",
"integrity": "sha512-/G1UJPYlm+trBWQ6cMO3sv6b8D1+G16WaJH1/DSqw32JOVlzgZbLkDxRyzIpTpv30AcYGMkCf5tUqGlW6HbDWw==",
"hasInstallScript": true,
"license": "MIT",
"dependencies": {
"https-proxy-agent": "^7.0.2",
"proxy-from-env": "^1.1.0",
"yauzl": "^2.9.2"
}
},
"node_modules/agent-base": {
"version": "7.1.4",
"resolved": "https://registry.npmjs.org/agent-base/-/agent-base-7.1.4.tgz",
"integrity": "sha512-MnA+YT8fwfJPgBx3m60MNqakm30XOkyIoH1y6huTQvC0PwZG7ki8NacLBcrPbNoo8vEZy7Jpuk7+jMO+CUovTQ==",
"license": "MIT",
"engines": {
"node": ">= 14"
}
},
"node_modules/buffer-crc32": {
"version": "0.2.13",
"resolved": "https://registry.npmjs.org/buffer-crc32/-/buffer-crc32-0.2.13.tgz",
"integrity": "sha512-VO9Ht/+p3SN7SKWqcrgEzjGbRSJYTx+Q1pTQC0wrWqHx0vpJraQ6GtHx8tvcg1rlK1byhU5gccxgOgj7B0TDkQ==",
"license": "MIT",
"engines": {
"node": "*"
}
},
"node_modules/debug": {
"version": "4.4.1",
"resolved": "https://registry.npmjs.org/debug/-/debug-4.4.1.tgz",
"integrity": "sha512-KcKCqiftBJcZr++7ykoDIEwSa3XWowTfNPo92BYxjXiyYEVrUQh2aLyhxBCwww+heortUFxEJYcRzosstTEBYQ==",
"license": "MIT",
"dependencies": {
"ms": "^2.1.3"
},
"engines": {
"node": ">=6.0"
},
"peerDependenciesMeta": {
"supports-color": {
"optional": true
}
}
},
"node_modules/fd-slicer": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/fd-slicer/-/fd-slicer-1.1.0.tgz",
"integrity": "sha512-cE1qsB/VwyQozZ+q1dGxR8LBYNZeofhEdUNGSMbQD3Gw2lAzX9Zb3uIU6Ebc/Fmyjo9AWWfnn0AUCHqtevs/8g==",
"license": "MIT",
"dependencies": {
"pend": "~1.2.0"
}
},
"node_modules/https-proxy-agent": {
"version": "7.0.6",
"resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-7.0.6.tgz",
"integrity": "sha512-vK9P5/iUfdl95AI+JVyUuIcVtd4ofvtrOr3HNtM2yxC9bnMbEdp3x01OhQNnjb8IJYi38VlTE3mBXwcfvywuSw==",
"license": "MIT",
"dependencies": {
"agent-base": "^7.1.2",
"debug": "4"
},
"engines": {
"node": ">= 14"
}
},
"node_modules/ms": {
"version": "2.1.3",
"resolved": "https://registry.npmjs.org/ms/-/ms-2.1.3.tgz",
"integrity": "sha512-6FlzubTLZG3J2a/NVCAleEhjzq5oxgHyaCU9yYXvcLsvoVaHJq/s5xXI6/XXP6tz7R9xAOtHnSO/tXtF3WRTlA==",
"license": "MIT"
},
"node_modules/pend": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/pend/-/pend-1.2.0.tgz",
"integrity": "sha512-F3asv42UuXchdzt+xXqfW1OGlVBe+mxa2mqI0pg5yAHZPvFmY3Y6drSf/GQ1A86WgWEN9Kzh/WrgKa6iGcHXLg==",
"license": "MIT"
},
"node_modules/proxy-from-env": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/proxy-from-env/-/proxy-from-env-1.1.0.tgz",
"integrity": "sha512-D+zkORCbA9f1tdWRK0RaCR3GPv50cMxcrz4X8k5LTSUD1Dkw47mKJEZQNunItRTkWwgtaUSo1RVFRIG9ZXiFYg==",
"license": "MIT"
},
"node_modules/yauzl": {
"version": "2.10.0",
"resolved": "https://registry.npmjs.org/yauzl/-/yauzl-2.10.0.tgz",
"integrity": "sha512-p4a9I6X6nu6IhoGmBqAcbJy1mlC4j27vEPZX9F4L4/vZT3Lyq1VkFHw/V/PUcB9Buo+DG3iHkT0x3Qya58zc3g==",
"license": "MIT",
"dependencies": {
"buffer-crc32": "~0.2.3",
"fd-slicer": "~1.1.0"
}
}
}
}

View File

@@ -11,17 +11,11 @@
},
"files": [
"bin",
"dist"
"vendor"
],
"repository": {
"type": "git",
"url": "git+https://github.com/openai/codex.git",
"directory": "codex-cli"
},
"dependencies": {
"@vscode/ripgrep": "^1.15.14"
},
"devDependencies": {
"prettier": "^3.3.3"
}
}

View File

@@ -5,5 +5,7 @@ Run the following:
To build the 0.2.x or later version of the npm module, which runs the Rust version of the CLI, build it as follows:
```bash
./codex-cli/scripts/stage_rust_release.py --release-version 0.6.0
./codex-cli/scripts/build_npm_package.py --release-version 0.6.0
```
Note this will create `./codex-cli/vendor/` as a side-effect.

View File

@@ -0,0 +1,269 @@
#!/usr/bin/env python3
"""Stage and optionally package the @openai/codex npm module."""
import argparse
import json
import re
import shutil
import subprocess
import sys
import tempfile
from pathlib import Path
SCRIPT_DIR = Path(__file__).resolve().parent
CODEX_CLI_ROOT = SCRIPT_DIR.parent
REPO_ROOT = CODEX_CLI_ROOT.parent
GITHUB_REPO = "openai/codex"
# The docs are not clear on what the expected value/format of
# workflow/workflowName is:
# https://cli.github.com/manual/gh_run_list
WORKFLOW_NAME = ".github/workflows/rust-release.yml"
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Build or stage the Codex CLI npm package.")
parser.add_argument(
"--version",
help="Version number to write to package.json inside the staged package.",
)
parser.add_argument(
"--release-version",
help=(
"Version to stage for npm release. When provided, the script also resolves the "
"matching rust-release workflow unless --workflow-url is supplied."
),
)
parser.add_argument(
"--workflow-url",
help="Optional GitHub Actions workflow run URL used to download native binaries.",
)
parser.add_argument(
"--staging-dir",
type=Path,
help=(
"Directory to stage the package contents. Defaults to a new temporary directory "
"if omitted. The directory must be empty when provided."
),
)
parser.add_argument(
"--tmp",
dest="staging_dir",
type=Path,
help=argparse.SUPPRESS,
)
parser.add_argument(
"--pack-output",
type=Path,
help="Path where the generated npm tarball should be written.",
)
return parser.parse_args()
def main() -> int:
args = parse_args()
version = args.version
release_version = args.release_version
if release_version:
if version and version != release_version:
raise RuntimeError("--version and --release-version must match when both are provided.")
version = release_version
if not version:
raise RuntimeError("Must specify --version or --release-version.")
staging_dir, created_temp = prepare_staging_dir(args.staging_dir)
try:
stage_sources(staging_dir, version)
workflow_url = args.workflow_url
resolved_head_sha: str | None = None
if not workflow_url:
if release_version:
workflow = resolve_release_workflow(version)
workflow_url = workflow["url"]
resolved_head_sha = workflow.get("headSha")
else:
workflow_url = resolve_latest_alpha_workflow_url()
elif release_version:
try:
workflow = resolve_release_workflow(version)
resolved_head_sha = workflow.get("headSha")
except Exception:
resolved_head_sha = None
if release_version and resolved_head_sha:
print(f"should `git checkout {resolved_head_sha}`")
if not workflow_url:
raise RuntimeError("Unable to determine workflow URL for native binaries.")
install_native_binaries(staging_dir, workflow_url)
if release_version:
staging_dir_str = str(staging_dir)
print(
f"Staged version {version} for release in {staging_dir_str}\n\n"
"Verify the CLI:\n"
f" node {staging_dir_str}/bin/codex.js --version\n"
f" node {staging_dir_str}/bin/codex.js --help\n\n"
)
else:
print(f"Staged package in {staging_dir}")
if args.pack_output is not None:
output_path = run_npm_pack(staging_dir, args.pack_output)
print(f"npm pack output written to {output_path}")
finally:
if created_temp:
# Preserve the staging directory for further inspection.
pass
return 0
def prepare_staging_dir(staging_dir: Path | None) -> tuple[Path, bool]:
if staging_dir is not None:
staging_dir = staging_dir.resolve()
staging_dir.mkdir(parents=True, exist_ok=True)
if any(staging_dir.iterdir()):
raise RuntimeError(f"Staging directory {staging_dir} is not empty.")
return staging_dir, False
temp_dir = Path(tempfile.mkdtemp(prefix="codex-npm-stage-"))
return temp_dir, True
def stage_sources(staging_dir: Path, version: str) -> None:
bin_dir = staging_dir / "bin"
bin_dir.mkdir(parents=True, exist_ok=True)
shutil.copy2(CODEX_CLI_ROOT / "bin" / "codex.js", bin_dir / "codex.js")
rg_manifest = CODEX_CLI_ROOT / "bin" / "rg"
if rg_manifest.exists():
shutil.copy2(rg_manifest, bin_dir / "rg")
readme_src = REPO_ROOT / "README.md"
if readme_src.exists():
shutil.copy2(readme_src, staging_dir / "README.md")
with open(CODEX_CLI_ROOT / "package.json", "r", encoding="utf-8") as fh:
package_json = json.load(fh)
package_json["version"] = version
with open(staging_dir / "package.json", "w", encoding="utf-8") as out:
json.dump(package_json, out, indent=2)
out.write("\n")
def install_native_binaries(staging_dir: Path, workflow_url: str | None) -> None:
cmd = ["./scripts/install_native_deps.py"]
if workflow_url:
cmd.extend(["--workflow-url", workflow_url])
cmd.append(str(staging_dir))
subprocess.check_call(cmd, cwd=CODEX_CLI_ROOT)
def resolve_latest_alpha_workflow_url() -> str:
version = determine_latest_alpha_version()
workflow = resolve_release_workflow(version)
return workflow["url"]
def determine_latest_alpha_version() -> str:
releases = list_releases()
best_key: tuple[int, int, int, int] | None = None
best_version: str | None = None
pattern = re.compile(r"^rust-v(\d+)\.(\d+)\.(\d+)-alpha\.(\d+)$")
for release in releases:
tag = release.get("tag_name", "")
match = pattern.match(tag)
if not match:
continue
key = tuple(int(match.group(i)) for i in range(1, 5))
if best_key is None or key > best_key:
best_key = key
best_version = (
f"{match.group(1)}.{match.group(2)}.{match.group(3)}-alpha.{match.group(4)}"
)
if best_version is None:
raise RuntimeError("No alpha releases found when resolving workflow URL.")
return best_version
def list_releases() -> list[dict]:
stdout = subprocess.check_output(
["gh", "api", f"/repos/{GITHUB_REPO}/releases?per_page=100"],
text=True,
)
try:
releases = json.loads(stdout or "[]")
except json.JSONDecodeError as exc:
raise RuntimeError("Unable to parse releases JSON.") from exc
if not isinstance(releases, list):
raise RuntimeError("Unexpected response when listing releases.")
return releases
def resolve_release_workflow(version: str) -> dict:
stdout = subprocess.check_output(
[
"gh",
"run",
"list",
"--branch",
f"rust-v{version}",
"--json",
"workflowName,url,headSha",
"--workflow",
WORKFLOW_NAME,
"--jq",
"first(.[])",
],
text=True,
)
workflow = json.loads(stdout or "[]")
if not workflow:
raise RuntimeError(f"Unable to find rust-release workflow for version {version}.")
return workflow
def run_npm_pack(staging_dir: Path, output_path: Path) -> Path:
output_path = output_path.resolve()
output_path.parent.mkdir(parents=True, exist_ok=True)
with tempfile.TemporaryDirectory(prefix="codex-npm-pack-") as pack_dir_str:
pack_dir = Path(pack_dir_str)
stdout = subprocess.check_output(
["npm", "pack", "--json", "--pack-destination", str(pack_dir)],
cwd=staging_dir,
text=True,
)
try:
pack_output = json.loads(stdout)
except json.JSONDecodeError as exc:
raise RuntimeError("Failed to parse npm pack output.") from exc
if not pack_output:
raise RuntimeError("npm pack did not produce an output tarball.")
tarball_name = pack_output[0].get("filename") or pack_output[0].get("name")
if not tarball_name:
raise RuntimeError("Unable to determine npm pack output filename.")
tarball_path = pack_dir / tarball_name
if not tarball_path.exists():
raise RuntimeError(f"Expected npm pack output not found: {tarball_path}")
shutil.move(str(tarball_path), output_path)
return output_path
if __name__ == "__main__":
import sys
sys.exit(main())

View File

@@ -0,0 +1,318 @@
#!/usr/bin/env python3
"""Install Codex native binaries (Rust CLI plus ripgrep helpers)."""
import argparse
import json
import os
import shutil
import subprocess
import tarfile
import tempfile
import zipfile
from concurrent.futures import ThreadPoolExecutor, as_completed
from pathlib import Path
from typing import Iterable, Sequence
from urllib.parse import urlparse
from urllib.request import urlopen
SCRIPT_DIR = Path(__file__).resolve().parent
CODEX_CLI_ROOT = SCRIPT_DIR.parent
DEFAULT_WORKFLOW_URL = "https://github.com/openai/codex/actions/runs/17952349351" # rust-v0.40.0
VENDOR_DIR_NAME = "vendor"
RG_MANIFEST = CODEX_CLI_ROOT / "bin" / "rg"
CODEX_TARGETS = (
"x86_64-unknown-linux-musl",
"aarch64-unknown-linux-musl",
"x86_64-apple-darwin",
"aarch64-apple-darwin",
"x86_64-pc-windows-msvc",
"aarch64-pc-windows-msvc",
)
RG_TARGET_PLATFORM_PAIRS: list[tuple[str, str]] = [
("x86_64-unknown-linux-musl", "linux-x86_64"),
("aarch64-unknown-linux-musl", "linux-aarch64"),
("x86_64-apple-darwin", "macos-x86_64"),
("aarch64-apple-darwin", "macos-aarch64"),
("x86_64-pc-windows-msvc", "windows-x86_64"),
("aarch64-pc-windows-msvc", "windows-aarch64"),
]
RG_TARGET_TO_PLATFORM = {target: platform for target, platform in RG_TARGET_PLATFORM_PAIRS}
DEFAULT_RG_TARGETS = [target for target, _ in RG_TARGET_PLATFORM_PAIRS]
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Install native Codex binaries.")
parser.add_argument(
"--workflow-url",
help=(
"GitHub Actions workflow URL that produced the artifacts. Defaults to a "
"known good run when omitted."
),
)
parser.add_argument(
"root",
nargs="?",
type=Path,
help=(
"Directory containing package.json for the staged package. If omitted, the "
"repository checkout is used."
),
)
return parser.parse_args()
def main() -> int:
args = parse_args()
codex_cli_root = (args.root or CODEX_CLI_ROOT).resolve()
vendor_dir = codex_cli_root / VENDOR_DIR_NAME
vendor_dir.mkdir(parents=True, exist_ok=True)
workflow_url = (args.workflow_url or DEFAULT_WORKFLOW_URL).strip()
if not workflow_url:
workflow_url = DEFAULT_WORKFLOW_URL
workflow_id = workflow_url.rstrip("/").split("/")[-1]
with tempfile.TemporaryDirectory(prefix="codex-native-artifacts-") as artifacts_dir_str:
artifacts_dir = Path(artifacts_dir_str)
_download_artifacts(workflow_id, artifacts_dir)
install_codex_binaries(artifacts_dir, vendor_dir, CODEX_TARGETS)
fetch_rg(vendor_dir, DEFAULT_RG_TARGETS, manifest_path=RG_MANIFEST)
print(f"Installed native dependencies into {vendor_dir}")
return 0
def fetch_rg(
vendor_dir: Path,
targets: Sequence[str] | None = None,
*,
manifest_path: Path,
) -> list[Path]:
"""Download ripgrep binaries described by the DotSlash manifest."""
if targets is None:
targets = DEFAULT_RG_TARGETS
if not manifest_path.exists():
raise FileNotFoundError(f"DotSlash manifest not found: {manifest_path}")
manifest = _load_manifest(manifest_path)
platforms = manifest.get("platforms", {})
vendor_dir.mkdir(parents=True, exist_ok=True)
targets = list(targets)
if not targets:
return []
task_configs: list[tuple[str, str, dict]] = []
for target in targets:
platform_key = RG_TARGET_TO_PLATFORM.get(target)
if platform_key is None:
raise ValueError(f"Unsupported ripgrep target '{target}'.")
platform_info = platforms.get(platform_key)
if platform_info is None:
raise RuntimeError(f"Platform '{platform_key}' not found in manifest {manifest_path}.")
task_configs.append((target, platform_key, platform_info))
results: dict[str, Path] = {}
max_workers = min(len(task_configs), max(1, (os.cpu_count() or 1)))
with ThreadPoolExecutor(max_workers=max_workers) as executor:
future_map = {
executor.submit(
_fetch_single_rg,
vendor_dir,
target,
platform_key,
platform_info,
manifest_path,
): target
for target, platform_key, platform_info in task_configs
}
for future in as_completed(future_map):
target = future_map[future]
results[target] = future.result()
return [results[target] for target in targets]
def _download_artifacts(workflow_id: str, dest_dir: Path) -> None:
cmd = [
"gh",
"run",
"download",
"--dir",
str(dest_dir),
"--repo",
"openai/codex",
workflow_id,
]
subprocess.check_call(cmd)
def install_codex_binaries(
artifacts_dir: Path, vendor_dir: Path, targets: Iterable[str]
) -> list[Path]:
targets = list(targets)
if not targets:
return []
results: dict[str, Path] = {}
max_workers = min(len(targets), max(1, (os.cpu_count() or 1)))
with ThreadPoolExecutor(max_workers=max_workers) as executor:
future_map = {
executor.submit(_install_single_codex_binary, artifacts_dir, vendor_dir, target): target
for target in targets
}
for future in as_completed(future_map):
target = future_map[future]
results[target] = future.result()
return [results[target] for target in targets]
def _install_single_codex_binary(artifacts_dir: Path, vendor_dir: Path, target: str) -> Path:
artifact_subdir = artifacts_dir / target
archive_name = _archive_name_for_target(target)
archive_path = artifact_subdir / archive_name
if not archive_path.exists():
raise FileNotFoundError(f"Expected artifact not found: {archive_path}")
dest_dir = vendor_dir / target / "codex"
dest_dir.mkdir(parents=True, exist_ok=True)
binary_name = "codex.exe" if "windows" in target else "codex"
dest = dest_dir / binary_name
dest.unlink(missing_ok=True)
extract_archive(archive_path, "zst", None, dest)
if "windows" not in target:
dest.chmod(0o755)
return dest
def _archive_name_for_target(target: str) -> str:
if "windows" in target:
return f"codex-{target}.exe.zst"
return f"codex-{target}.zst"
def _fetch_single_rg(
vendor_dir: Path,
target: str,
platform_key: str,
platform_info: dict,
manifest_path: Path,
) -> Path:
providers = platform_info.get("providers", [])
if not providers:
raise RuntimeError(f"No providers listed for platform '{platform_key}' in {manifest_path}.")
url = providers[0]["url"]
archive_format = platform_info.get("format", "zst")
archive_member = platform_info.get("path")
dest_dir = vendor_dir / target / "path"
dest_dir.mkdir(parents=True, exist_ok=True)
is_windows = platform_key.startswith("win")
binary_name = "rg.exe" if is_windows else "rg"
dest = dest_dir / binary_name
with tempfile.TemporaryDirectory() as tmp_dir_str:
tmp_dir = Path(tmp_dir_str)
archive_filename = os.path.basename(urlparse(url).path)
download_path = tmp_dir / archive_filename
_download_file(url, download_path)
dest.unlink(missing_ok=True)
extract_archive(download_path, archive_format, archive_member, dest)
if not is_windows:
dest.chmod(0o755)
return dest
def _download_file(url: str, dest: Path) -> None:
dest.parent.mkdir(parents=True, exist_ok=True)
with urlopen(url) as response, open(dest, "wb") as out:
shutil.copyfileobj(response, out)
def extract_archive(
archive_path: Path,
archive_format: str,
archive_member: str | None,
dest: Path,
) -> None:
dest.parent.mkdir(parents=True, exist_ok=True)
if archive_format == "zst":
output_path = archive_path.parent / dest.name
subprocess.check_call(
["zstd", "-f", "-d", str(archive_path), "-o", str(output_path)]
)
shutil.move(str(output_path), dest)
return
if archive_format == "tar.gz":
if not archive_member:
raise RuntimeError("Missing 'path' for tar.gz archive in DotSlash manifest.")
with tarfile.open(archive_path, "r:gz") as tar:
try:
member = tar.getmember(archive_member)
except KeyError as exc:
raise RuntimeError(
f"Entry '{archive_member}' not found in archive {archive_path}."
) from exc
tar.extract(member, path=archive_path.parent, filter="data")
extracted = archive_path.parent / archive_member
shutil.move(str(extracted), dest)
return
if archive_format == "zip":
if not archive_member:
raise RuntimeError("Missing 'path' for zip archive in DotSlash manifest.")
with zipfile.ZipFile(archive_path) as archive:
try:
with archive.open(archive_member) as src, open(dest, "wb") as out:
shutil.copyfileobj(src, out)
except KeyError as exc:
raise RuntimeError(
f"Entry '{archive_member}' not found in archive {archive_path}."
) from exc
return
raise RuntimeError(f"Unsupported archive format '{archive_format}'.")
def _load_manifest(manifest_path: Path) -> dict:
cmd = ["dotslash", "--", "parse", str(manifest_path)]
stdout = subprocess.check_output(cmd, text=True)
try:
manifest = json.loads(stdout)
except json.JSONDecodeError as exc:
raise RuntimeError(f"Invalid DotSlash manifest output from {manifest_path}.") from exc
if not isinstance(manifest, dict):
raise RuntimeError(
f"Unexpected DotSlash manifest structure for {manifest_path}: {type(manifest)!r}"
)
return manifest
if __name__ == "__main__":
import sys
sys.exit(main())

View File

@@ -1,94 +0,0 @@
#!/usr/bin/env bash
# Install native runtime dependencies for codex-cli.
#
# Usage
# install_native_deps.sh [--workflow-url URL] [CODEX_CLI_ROOT]
#
# The optional RELEASE_ROOT is the path that contains package.json. Omitting
# it installs the binaries into the repository's own bin/ folder to support
# local development.
set -euo pipefail
# ------------------
# Parse arguments
# ------------------
CODEX_CLI_ROOT=""
# Until we start publishing stable GitHub releases, we have to grab the binaries
# from the GitHub Action that created them. Update the URL below to point to the
# appropriate workflow run:
WORKFLOW_URL="https://github.com/openai/codex/actions/runs/17417194663" # rust-v0.28.0
while [[ $# -gt 0 ]]; do
case "$1" in
--workflow-url)
shift || { echo "--workflow-url requires an argument"; exit 1; }
if [ -n "$1" ]; then
WORKFLOW_URL="$1"
fi
;;
*)
if [[ -z "$CODEX_CLI_ROOT" ]]; then
CODEX_CLI_ROOT="$1"
else
echo "Unexpected argument: $1" >&2
exit 1
fi
;;
esac
shift
done
# ----------------------------------------------------------------------------
# Determine where the binaries should be installed.
# ----------------------------------------------------------------------------
if [ -n "$CODEX_CLI_ROOT" ]; then
# The caller supplied a release root directory.
BIN_DIR="$CODEX_CLI_ROOT/bin"
else
# No argument; fall back to the repos own bin directory.
# Resolve the path of this script, then walk up to the repo root.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
CODEX_CLI_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
BIN_DIR="$CODEX_CLI_ROOT/bin"
fi
# Make sure the destination directory exists.
mkdir -p "$BIN_DIR"
# ----------------------------------------------------------------------------
# Download and decompress the artifacts from the GitHub Actions workflow.
# ----------------------------------------------------------------------------
WORKFLOW_ID="${WORKFLOW_URL##*/}"
ARTIFACTS_DIR="$(mktemp -d)"
trap 'rm -rf "$ARTIFACTS_DIR"' EXIT
# NB: The GitHub CLI `gh` must be installed and authenticated.
gh run download --dir "$ARTIFACTS_DIR" --repo openai/codex "$WORKFLOW_ID"
# x64 Linux
zstd -d "$ARTIFACTS_DIR/x86_64-unknown-linux-musl/codex-x86_64-unknown-linux-musl.zst" \
-o "$BIN_DIR/codex-x86_64-unknown-linux-musl"
# ARM64 Linux
zstd -d "$ARTIFACTS_DIR/aarch64-unknown-linux-musl/codex-aarch64-unknown-linux-musl.zst" \
-o "$BIN_DIR/codex-aarch64-unknown-linux-musl"
# x64 macOS
zstd -d "$ARTIFACTS_DIR/x86_64-apple-darwin/codex-x86_64-apple-darwin.zst" \
-o "$BIN_DIR/codex-x86_64-apple-darwin"
# ARM64 macOS
zstd -d "$ARTIFACTS_DIR/aarch64-apple-darwin/codex-aarch64-apple-darwin.zst" \
-o "$BIN_DIR/codex-aarch64-apple-darwin"
# x64 Windows
zstd -d "$ARTIFACTS_DIR/x86_64-pc-windows-msvc/codex-x86_64-pc-windows-msvc.exe.zst" \
-o "$BIN_DIR/codex-x86_64-pc-windows-msvc.exe"
# ARM64 Windows
zstd -d "$ARTIFACTS_DIR/aarch64-pc-windows-msvc/codex-aarch64-pc-windows-msvc.exe.zst" \
-o "$BIN_DIR/codex-aarch64-pc-windows-msvc.exe"
echo "Installed native dependencies into $BIN_DIR"

View File

@@ -1,120 +0,0 @@
#!/usr/bin/env bash
# -----------------------------------------------------------------------------
# stage_release.sh
# -----------------------------------------------------------------------------
# Stages an npm release for @openai/codex.
#
# Usage:
#
# --tmp <dir> : Use <dir> instead of a freshly created temp directory.
# -h|--help : Print usage.
#
# -----------------------------------------------------------------------------
set -euo pipefail
# Helper - usage / flag parsing
usage() {
cat <<EOF
Usage: $(basename "$0") [--tmp DIR] [--version VERSION]
Options
--tmp DIR Use DIR to stage the release (defaults to a fresh mktemp dir)
--version Specify the version to release (defaults to a timestamp-based version)
-h, --help Show this help
Legacy positional argument: the first non-flag argument is still interpreted
as the temporary directory (for backwards compatibility) but is deprecated.
EOF
exit "${1:-0}"
}
TMPDIR=""
# Default to a timestamp-based version (keep same scheme as before)
VERSION="$(printf '0.1.%d' "$(date +%y%m%d%H%M)")"
WORKFLOW_URL=""
# Manual flag parser - Bash getopts does not handle GNU long options well.
while [[ $# -gt 0 ]]; do
case "$1" in
--tmp)
shift || { echo "--tmp requires an argument"; usage 1; }
TMPDIR="$1"
;;
--tmp=*)
TMPDIR="${1#*=}"
;;
--version)
shift || { echo "--version requires an argument"; usage 1; }
VERSION="$1"
;;
--workflow-url)
shift || { echo "--workflow-url requires an argument"; exit 1; }
WORKFLOW_URL="$1"
;;
-h|--help)
usage 0
;;
--*)
echo "Unknown option: $1" >&2
usage 1
;;
*)
echo "Unexpected extra argument: $1" >&2
usage 1
;;
esac
shift
done
# Fallback when the caller did not specify a directory.
# If no directory was specified create a fresh temporary one.
if [[ -z "$TMPDIR" ]]; then
TMPDIR="$(mktemp -d)"
fi
# Ensure the directory exists, then resolve to an absolute path.
mkdir -p "$TMPDIR"
TMPDIR="$(cd "$TMPDIR" && pwd)"
# Main build logic
echo "Staging release in $TMPDIR"
# The script lives in codex-cli/scripts/ - change into codex-cli root so that
# relative paths keep working.
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
CODEX_CLI_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
pushd "$CODEX_CLI_ROOT" >/dev/null
# 1. Build the JS artifacts ---------------------------------------------------
# Paths inside the staged package
mkdir -p "$TMPDIR/bin"
cp -r bin/codex.js "$TMPDIR/bin/codex.js"
cp ../README.md "$TMPDIR" || true # README is one level up - ignore if missing
# Modify package.json - bump version and optionally add the native directory to
# the files array so that the binaries are published to npm.
jq --arg version "$VERSION" \
'.version = $version' \
package.json > "$TMPDIR/package.json"
# 2. Native runtime deps (sandbox plus optional Rust binaries)
./scripts/install_native_deps.sh --workflow-url "$WORKFLOW_URL" "$TMPDIR"
popd >/dev/null
echo "Staged version $VERSION for release in $TMPDIR"
echo "Verify the CLI:"
echo " node ${TMPDIR}/bin/codex.js --version"
echo " node ${TMPDIR}/bin/codex.js --help"
# Print final hint for convenience
echo "Next: cd \"$TMPDIR\" && npm publish"

View File

@@ -1,70 +0,0 @@
#!/usr/bin/env python3
import json
import subprocess
import sys
import argparse
from pathlib import Path
def main() -> int:
parser = argparse.ArgumentParser(
description="""Stage a release for the npm module.
Run this after the GitHub Release has been created and use
`--release-version` to specify the version to release.
Optionally pass `--tmp` to control the temporary staging directory that will be
forwarded to stage_release.sh.
"""
)
parser.add_argument(
"--release-version", required=True, help="Version to release, e.g., 0.3.0"
)
parser.add_argument(
"--tmp",
help="Optional path to stage the npm package; forwarded to stage_release.sh",
)
args = parser.parse_args()
version = args.release_version
gh_run = subprocess.run(
[
"gh",
"run",
"list",
"--branch",
f"rust-v{version}",
"--json",
"workflowName,url,headSha",
"--jq",
'first(.[] | select(.workflowName == "rust-release"))',
],
stdout=subprocess.PIPE,
check=True,
)
gh_run.check_returncode()
workflow = json.loads(gh_run.stdout)
sha = workflow["headSha"]
print(f"should `git checkout {sha}`")
current_dir = Path(__file__).parent.resolve()
cmd = [
str(current_dir / "stage_release.sh"),
"--version",
version,
"--workflow-url",
workflow["url"],
]
if args.tmp:
cmd.extend(["--tmp", args.tmp])
stage_release = subprocess.run(cmd)
stage_release.check_returncode()
return 0
if __name__ == "__main__":
sys.exit(main())

230
codex-rs/Cargo.lock generated
View File

@@ -56,7 +56,7 @@ checksum = "8fac2ce611db8b8cee9b2aa886ca03c924e9da5e5295d0dbd0526e5d0b0710f7"
dependencies = [
"allocative_derive",
"bumpalo",
"ctor",
"ctor 0.1.26",
"hashbrown 0.14.5",
"num-bigint",
]
@@ -78,12 +78,6 @@ version = "0.2.21"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "683d7910e743518b0e34f1186f92494becacb047c7b6bf616c96772180fef923"
[[package]]
name = "android-tzdata"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e999941b234f3131b00bc13c22d06e8c5ff726d1b6318ac7eb276997bbb4fef0"
[[package]]
name = "android_system_properties"
version = "0.1.5"
@@ -316,6 +310,17 @@ dependencies = [
"syn 2.0.104",
]
[[package]]
name = "async-trait"
version = "0.1.89"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9035ad2d096bed7955a320ee7e2230574d28fd3c3a0f186cbea1ff3c7eed5dbb"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.104",
]
[[package]]
name = "atomic-waker"
version = "1.1.2"
@@ -485,17 +490,16 @@ checksum = "fd16c4719339c4530435d38e511904438d07cce7950afa3718a84ac36c10e89e"
[[package]]
name = "chrono"
version = "0.4.41"
version = "0.4.42"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c469d952047f47f91b68d1cba3f10d63c11d73e4636f24f08daf0278abf01c4d"
checksum = "145052bdd345b87320e369255277e3fb5152762ad123a901ef5c262dd38fe8d2"
dependencies = [
"android-tzdata",
"iana-time-zone",
"js-sys",
"num-traits",
"serde",
"wasm-bindgen",
"windows-link",
"windows-link 0.2.0",
]
[[package]]
@@ -569,6 +573,39 @@ version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e9b18233253483ce2f65329a24072ec414db782531bdbb7d0bbc4bd2ce6b7e21"
[[package]]
name = "codex-agent"
version = "0.0.0"
dependencies = [
"anyhow",
"async-trait",
"base64",
"codex-apply-patch",
"codex-file-search",
"codex-protocol",
"core_test_support",
"futures",
"libc",
"mcp-types",
"portable-pty",
"pretty_assertions",
"serde",
"serde_json",
"sha1",
"shlex",
"similar",
"tempfile",
"thiserror 2.0.16",
"time",
"tokio",
"tracing",
"tree-sitter",
"tree-sitter-bash",
"uuid",
"which",
"wildmatch",
]
[[package]]
name = "codex-ansi-escape"
version = "0.0.0"
@@ -584,7 +621,6 @@ version = "0.0.0"
dependencies = [
"anyhow",
"assert_cmd",
"once_cell",
"pretty_assertions",
"similar",
"tempfile",
@@ -638,9 +674,13 @@ dependencies = [
"codex-protocol",
"codex-protocol-ts",
"codex-tui",
"ctor 0.5.0",
"libc",
"owo-colors",
"predicates",
"pretty_assertions",
"serde_json",
"supports-color",
"tempfile",
"tokio",
"tracing",
@@ -666,9 +706,11 @@ dependencies = [
"askama",
"assert_cmd",
"async-channel",
"async-trait",
"base64",
"bytes",
"chrono",
"codex-agent",
"codex-apply-patch",
"codex-file-search",
"codex-mcp-client",
@@ -678,6 +720,7 @@ dependencies = [
"env-flags",
"eventsource-stream",
"futures",
"indexmap 2.10.0",
"landlock",
"libc",
"maplit",
@@ -687,7 +730,7 @@ dependencies = [
"portable-pty",
"predicates",
"pretty_assertions",
"rand 0.9.2",
"rand",
"regex-lite",
"reqwest",
"seccompiler",
@@ -706,8 +749,6 @@ dependencies = [
"toml",
"toml_edit",
"tracing",
"tree-sitter",
"tree-sitter-bash",
"uuid",
"walkdir",
"which",
@@ -732,12 +773,15 @@ dependencies = [
"libc",
"owo-colors",
"predicates",
"pretty_assertions",
"serde",
"serde_json",
"shlex",
"tempfile",
"tokio",
"tracing",
"tracing-subscriber",
"ts-rs",
"uuid",
"walkdir",
"wiremock",
@@ -776,6 +820,16 @@ dependencies = [
"tokio",
]
[[package]]
name = "codex-git-tooling"
version = "0.0.0"
dependencies = [
"pretty_assertions",
"tempfile",
"thiserror 2.0.16",
"walkdir",
]
[[package]]
name = "codex-linux-sandbox"
version = "0.0.0"
@@ -793,11 +847,13 @@ dependencies = [
name = "codex-login"
version = "0.0.0"
dependencies = [
"anyhow",
"base64",
"chrono",
"codex-core",
"codex-protocol",
"rand 0.8.5",
"core_test_support",
"rand",
"reqwest",
"serde",
"serde_json",
@@ -835,6 +891,7 @@ dependencies = [
"codex-core",
"codex-login",
"codex-protocol",
"core_test_support",
"mcp-types",
"mcp_test_support",
"os_info",
@@ -871,6 +928,7 @@ dependencies = [
name = "codex-protocol"
version = "0.0.0"
dependencies = [
"anyhow",
"base64",
"icu_decimal",
"icu_locale_core",
@@ -915,6 +973,7 @@ dependencies = [
"codex-common",
"codex-core",
"codex-file-search",
"codex-git-tooling",
"codex-login",
"codex-ollama",
"codex-protocol",
@@ -928,12 +987,11 @@ dependencies = [
"lazy_static",
"libc",
"mcp-types",
"once_cell",
"path-clean",
"pathdiff",
"pretty_assertions",
"pulldown-cmark",
"rand 0.9.2",
"rand",
"ratatui",
"regex-lite",
"serde",
@@ -950,11 +1008,21 @@ dependencies = [
"tracing-appender",
"tracing-subscriber",
"unicode-segmentation",
"unicode-width 0.1.14",
"unicode-width 0.2.1",
"url",
"vt100",
]
[[package]]
name = "codex-utils-readiness"
version = "0.0.0"
dependencies = [
"async-trait",
"thiserror 2.0.16",
"time",
"tokio",
]
[[package]]
name = "color-eyre"
version = "0.6.5"
@@ -1072,10 +1140,13 @@ checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b"
name = "core_test_support"
version = "0.0.0"
dependencies = [
"anyhow",
"assert_cmd",
"codex-core",
"serde_json",
"tempfile",
"tokio",
"wiremock",
]
[[package]]
@@ -1182,6 +1253,22 @@ dependencies = [
"syn 1.0.109",
]
[[package]]
name = "ctor"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67773048316103656a637612c4a62477603b777d91d9c62ff2290f9cde178fdb"
dependencies = [
"ctor-proc-macro",
"dtor",
]
[[package]]
name = "ctor-proc-macro"
version = "0.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e2931af7e13dc045d8e9d26afccc6fa115d64e115c9c84b1166288b46f6782c2"
[[package]]
name = "darling"
version = "0.20.11"
@@ -1248,12 +1335,12 @@ dependencies = [
[[package]]
name = "deranged"
version = "0.4.0"
version = "0.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9c9e6a11ca8224451684bc0d7d5a7adbf8f2fd6887261a1cfc3c0432f9d4068e"
checksum = "a41953f86f8a05768a6cda24def994fd2f424b04ec5c719cf89989779f199071"
dependencies = [
"powerfmt",
"serde",
"serde_core",
]
[[package]]
@@ -1432,6 +1519,21 @@ version = "1.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75b325c5dbd37f80359721ad39aca5a29fb04c89279657cffdda8736d0c0b9d2"
[[package]]
name = "dtor"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e58a0764cddb55ab28955347b45be00ade43d4d6f3ba4bf3dc354e4ec9432934"
dependencies = [
"dtor-proc-macro",
]
[[package]]
name = "dtor-proc-macro"
version = "0.0.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f678cf4a922c215c63e0de95eb1ff08a958a81d47e485cf9da1e27bf6305cfa5"
[[package]]
name = "dupe"
version = "0.9.1"
@@ -2653,9 +2755,9 @@ dependencies = [
[[package]]
name = "log"
version = "0.4.27"
version = "0.4.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "13dc2df351e3202783a1fe0d44375f7295ffb4049267b0f3018346dc122a1d94"
checksum = "34080505efa8e45a4b816c349525ebe327ceaa8559756f0356cba97ef3bf7432"
[[package]]
name = "logos"
@@ -3450,35 +3552,14 @@ dependencies = [
"nibble_vec",
]
[[package]]
name = "rand"
version = "0.8.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34af8d1a0e25924bc5b7c43c079c942339d8f0a8b57c39049bef581b46327404"
dependencies = [
"libc",
"rand_chacha 0.3.1",
"rand_core 0.6.4",
]
[[package]]
name = "rand"
version = "0.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6db2770f06117d490610c7488547d543617b21bfa07796d7a12f6f1bd53850d1"
dependencies = [
"rand_chacha 0.9.0",
"rand_core 0.9.3",
]
[[package]]
name = "rand_chacha"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6c10a63a0fa32252be49d21e7709d4d4baf8d231c2dbce1eaa8141b9b127d88"
dependencies = [
"ppv-lite86",
"rand_core 0.6.4",
"rand_chacha",
"rand_core",
]
[[package]]
@@ -3488,16 +3569,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d3022b5f1df60f26e1ffddd6c66e8aa15de382ae63b3a0c1bfc0e4d3e3f325cb"
dependencies = [
"ppv-lite86",
"rand_core 0.9.3",
]
[[package]]
name = "rand_core"
version = "0.6.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ec0be4795e2f6a28069bec0b5ff3e2ac9bafc99e6a9a7dc3547996c5c816922c"
dependencies = [
"getrandom 0.2.16",
"rand_core",
]
[[package]]
@@ -3926,9 +3998,9 @@ dependencies = [
[[package]]
name = "serde"
version = "1.0.224"
version = "1.0.226"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6aaeb1e94f53b16384af593c71e20b095e958dab1d26939c1b70645c5cfbcc0b"
checksum = "0dca6411025b24b60bfa7ec1fe1f8e710ac09782dca409ee8237ba74b51295fd"
dependencies = [
"serde_core",
"serde_derive",
@@ -3936,18 +4008,18 @@ dependencies = [
[[package]]
name = "serde_core"
version = "1.0.224"
version = "1.0.226"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32f39390fa6346e24defbcdd3d9544ba8a19985d0af74df8501fbfe9a64341ab"
checksum = "ba2ba63999edb9dac981fb34b3e5c0d111a69b0924e253ed29d83f7c99e966a4"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.224"
version = "1.0.226"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "87ff78ab5e8561c9a675bfc1785cb07ae721f0ee53329a595cefd8c04c2ac4e0"
checksum = "8db53ae22f34573731bafa1db20f04027b2d25e02d8205921b569171699cdb33"
dependencies = [
"proc-macro2",
"quote",
@@ -4450,15 +4522,15 @@ dependencies = [
[[package]]
name = "tempfile"
version = "3.20.0"
version = "3.23.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8a64e3985349f2441a1a9ef0b853f869006c3855f2cda6862a94d26ebb9d6a1"
checksum = "2d31c77bdf42a745371d260a26ca7163f1e0924b64afa0b688e61b5a9fa02f16"
dependencies = [
"fastrand",
"getrandom 0.3.3",
"once_cell",
"rustix 1.0.8",
"windows-sys 0.59.0",
"windows-sys 0.60.2",
]
[[package]]
@@ -4582,9 +4654,9 @@ dependencies = [
[[package]]
name = "time"
version = "0.3.41"
version = "0.3.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8a7619e19bc266e0f9c5e6686659d394bc57973859340060a69221e57dbc0c40"
checksum = "91e7d9e3bb61134e77bde20dd4825b97c010155709965fedf0f49bb138e52a9d"
dependencies = [
"deranged",
"itoa",
@@ -4599,15 +4671,15 @@ dependencies = [
[[package]]
name = "time-core"
version = "0.1.4"
version = "0.1.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c9e9a38711f559d9e3ce1cdb06dd7c5b8ea546bc90052da6d06bb76da74bb07c"
checksum = "40868e7c1d2f0b8d73e4a8c7f0ff63af4f6d19be117e90bd73eb1d62cf831c6b"
[[package]]
name = "time-macros"
version = "0.2.22"
version = "0.2.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3526739392ec93fd8b359c8e98514cb3e8e021beb4e5f597b00a0221f8ed8a49"
checksum = "30cfb0125f12d9c277f35663a0a33f8c30190f4e4574868a330595412d34ebf3"
dependencies = [
"num-conv",
"time-core",
@@ -5335,7 +5407,7 @@ checksum = "c0fdd3ddb90610c7638aa2b3a3ab2904fb9e5cdbecc643ddb3647212781c4ae3"
dependencies = [
"windows-implement",
"windows-interface",
"windows-link",
"windows-link 0.1.3",
"windows-result",
"windows-strings",
]
@@ -5368,13 +5440,19 @@ version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e6ad25900d524eaabdbbb96d20b4311e1e7ae1699af4fb28c17ae66c80d798a"
[[package]]
name = "windows-link"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "45e46c0661abb7180e7b9c281db115305d49ca1709ab8242adf09666d2173c65"
[[package]]
name = "windows-registry"
version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5b8a9ed28765efc97bbc954883f4e6796c33a06546ebafacbabee9696967499e"
dependencies = [
"windows-link",
"windows-link 0.1.3",
"windows-result",
"windows-strings",
]
@@ -5385,7 +5463,7 @@ version = "0.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "56f42bd332cc6c8eac5af113fc0c1fd6a8fd2aa08a0119358686e5160d0586c6"
dependencies = [
"windows-link",
"windows-link 0.1.3",
]
[[package]]
@@ -5394,7 +5472,7 @@ version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "56e6c93f3a0c3b36176cb1327a4958a0353d5d166c2a35cb268ace15e91d3b57"
dependencies = [
"windows-link",
"windows-link 0.1.3",
]
[[package]]

View File

@@ -1,5 +1,6 @@
[workspace]
members = [
"agent",
"ansi-escape",
"apply-patch",
"arg0",
@@ -9,6 +10,7 @@ members = [
"exec",
"execpolicy",
"file-search",
"git-tooling",
"linux-sandbox",
"login",
"mcp-client",
@@ -18,6 +20,7 @@ members = [
"protocol",
"protocol-ts",
"tui",
"utils/readiness",
]
resolver = "2"
@@ -29,15 +32,172 @@ version = "0.0.0"
# edition.
edition = "2024"
[workspace.dependencies]
# Internal
codex-agent = { path = "agent" }
codex-ansi-escape = { path = "ansi-escape" }
codex-apply-patch = { path = "apply-patch" }
codex-arg0 = { path = "arg0" }
codex-chatgpt = { path = "chatgpt" }
codex-common = { path = "common" }
codex-core = { path = "core" }
codex-exec = { path = "exec" }
codex-file-search = { path = "file-search" }
codex-git-tooling = { path = "git-tooling" }
codex-linux-sandbox = { path = "linux-sandbox" }
codex-login = { path = "login" }
codex-mcp-client = { path = "mcp-client" }
codex-mcp-server = { path = "mcp-server" }
codex-ollama = { path = "ollama" }
codex-protocol = { path = "protocol" }
codex-protocol-ts = { path = "protocol-ts" }
codex-tui = { path = "tui" }
codex-utils-readiness = { path = "utils/readiness" }
core_test_support = { path = "core/tests/common" }
mcp-types = { path = "mcp-types" }
mcp_test_support = { path = "mcp-server/tests/common" }
# External
allocative = "0.3.3"
ansi-to-tui = "7.0.0"
anyhow = "1"
arboard = "3"
askama = "0.12"
assert_cmd = "2"
async-channel = "2.3.1"
async-stream = "0.3.6"
async-trait = "0.1.89"
base64 = "0.22.1"
bytes = "1.10.1"
chrono = "0.4.42"
clap = "4"
clap_complete = "4"
color-eyre = "0.6.3"
crossterm = "0.28.1"
ctor = "0.5.0"
derive_more = "2"
diffy = "0.4.2"
dirs = "6"
dotenvy = "0.15.7"
env-flags = "0.1.1"
env_logger = "0.11.5"
eventsource-stream = "0.2.3"
futures = "0.3"
icu_decimal = "2.0.0"
icu_locale_core = "2.0.0"
ignore = "0.4.23"
image = { version = "^0.25.8", default-features = false }
indexmap = "2.6.0"
insta = "1.43.2"
itertools = "0.14.0"
landlock = "0.4.1"
lazy_static = "1"
libc = "0.2.175"
log = "0.4"
maplit = "1.0.2"
mime_guess = "2.0.5"
multimap = "0.10.0"
nucleo-matcher = "0.3.1"
openssl-sys = "*"
os_info = "3.12.0"
owo-colors = "4.2.0"
path-absolutize = "3.1.1"
path-clean = "1.0.1"
pathdiff = "0.2"
portable-pty = "0.9.0"
predicates = "3"
pretty_assertions = "1.4.1"
pulldown-cmark = "0.10"
rand = "0.9"
ratatui = "0.29.0"
regex-lite = "0.1.7"
reqwest = "0.12"
schemars = "0.8.22"
seccompiler = "0.5.0"
serde = "1"
serde_json = "1"
serde_with = "3.14"
sha1 = "0.10.6"
sha2 = "0.10"
shlex = "1.3.0"
similar = "2.7.0"
starlark = "0.13.0"
strum = "0.27.2"
strum_macros = "0.27.2"
supports-color = "3.0.2"
sys-locale = "0.3.2"
tempfile = "3.23.0"
textwrap = "0.16.2"
thiserror = "2.0.16"
time = "0.3"
tiny_http = "0.12"
tokio = "1"
tokio-stream = "0.1.17"
tokio-test = "0.4"
tokio-util = "0.7.16"
toml = "0.9.5"
toml_edit = "0.23.4"
tracing = "0.1.41"
tracing-appender = "0.2.3"
tracing-subscriber = "0.3.20"
tree-sitter = "0.25.9"
tree-sitter-bash = "0.25.0"
ts-rs = "11"
unicode-segmentation = "1.12.0"
unicode-width = "0.2"
url = "2"
urlencoding = "2.1"
uuid = "1"
vt100 = "0.16.2"
walkdir = "2.5.0"
webbrowser = "1.0"
which = "6"
wildmatch = "2.5.0"
wiremock = "0.6"
[workspace.lints]
rust = {}
[workspace.lints.clippy]
expect_used = "deny"
identity_op = "deny"
manual_clamp = "deny"
manual_filter = "deny"
manual_find = "deny"
manual_flatten = "deny"
manual_map = "deny"
manual_memcpy = "deny"
manual_non_exhaustive = "deny"
manual_ok_or = "deny"
manual_range_contains = "deny"
manual_retain = "deny"
manual_strip = "deny"
manual_try_fold = "deny"
manual_unwrap_or = "deny"
needless_borrow = "deny"
needless_borrowed_reference = "deny"
needless_collect = "deny"
needless_late_init = "deny"
needless_option_as_deref = "deny"
needless_question_mark = "deny"
needless_update = "deny"
redundant_clone = "deny"
redundant_closure = "deny"
redundant_closure_for_method_calls = "deny"
redundant_static_lifetimes = "deny"
trivially_copy_pass_by_ref = "deny"
uninlined_format_args = "deny"
unnecessary_filter_map = "deny"
unnecessary_lazy_evaluations = "deny"
unnecessary_sort_by = "deny"
unnecessary_to_owned = "deny"
unwrap_used = "deny"
# cargo-shear cannot see the platform-specific openssl-sys usage, so we
# silence the false positive here instead of deleting a real dependency.
[workspace.metadata.cargo-shear]
ignored = ["openssl-sys", "codex-utils-readiness"]
[profile.release]
lto = "fat"
# Because we bundle some of these executables with the TypeScript CLI, we

View File

@@ -97,6 +97,7 @@ The same setting can be persisted in `~/.codex/config.toml` via the top-level `s
This folder is the root of a Cargo workspace. It contains quite a bit of experimental code, but here are the key crates:
- [`core/`](./core) contains the business logic for Codex. Ultimately, we hope this to be a library crate that is generally useful for building other Rust/native applications that use Codex.
- [`docs/agent_runtime_baseline.md`](./docs/agent_runtime_baseline.md) documents the current agent runtime interfaces (`Codex`, `Session`, `SessionTask`) and links to the ongoing refactor plan in `agent_refactor.md`.
- [`exec/`](./exec) "headless" CLI for use in automation.
- [`tui/`](./tui) CLI that launches a fullscreen TUI built with [Ratatui](https://ratatui.rs/).
- [`cli/`](./cli) CLI multitool that provides the aforementioned CLIs via subcommands.

38
codex-rs/agent/Cargo.toml Normal file
View File

@@ -0,0 +1,38 @@
[package]
name = "codex-agent"
version.workspace = true
edition.workspace = true
[dependencies]
anyhow = { workspace = true }
async-trait = { workspace = true }
codex-protocol = { workspace = true }
codex-apply-patch = { workspace = true }
mcp-types = { workspace = true }
base64 = { workspace = true }
serde_json = { workspace = true }
libc = { workspace = true }
portable-pty = { workspace = true }
serde = { workspace = true, features = ["derive"] }
sha1 = { workspace = true }
shlex = { workspace = true }
similar = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true, features = ["macros", "process", "rt-multi-thread", "sync", "time"] }
uuid = { workspace = true, features = ["serde", "v4"] }
which = { workspace = true }
wildmatch = { workspace = true }
codex-file-search = { workspace = true }
time = { workspace = true, features = ["formatting", "parsing", "local-offset", "macros"] }
tracing = { workspace = true }
tree-sitter = { workspace = true }
tree-sitter-bash = { workspace = true }
futures = { workspace = true }
[dev-dependencies]
core_test_support = { workspace = true }
tempfile = { workspace = true }
pretty_assertions = { workspace = true }
[lints]
workspace = true

View File

@@ -1,23 +1,26 @@
use crate::codex::Session;
use crate::codex::TurnContext;
use crate::protocol::FileChange;
use crate::protocol::ReviewDecision;
use crate::safety::SafetyCheck;
use crate::safety::assess_patch_safety;
use std::collections::HashMap;
use std::path::Path;
use std::path::PathBuf;
use codex_apply_patch::ApplyPatchAction;
use codex_apply_patch::ApplyPatchFileChange;
use codex_protocol::models::FunctionCallOutputPayload;
use codex_protocol::models::ResponseInputItem;
use std::collections::HashMap;
use std::path::PathBuf;
use codex_protocol::protocol::AskForApproval;
use codex_protocol::protocol::FileChange;
use codex_protocol::protocol::ReviewDecision;
use codex_protocol::protocol::SandboxPolicy;
use crate::function_tool::FunctionCallError;
use crate::safety::SafetyCheck;
use crate::safety::assess_patch_safety;
use crate::services::ApprovalCoordinator;
pub const CODEX_APPLY_PATCH_ARG1: &str = "--codex-run-as-apply-patch";
pub(crate) enum InternalApplyPatchInvocation {
pub enum InternalApplyPatchInvocation {
/// The `apply_patch` call was handled programmatically, without any sort
/// of sandbox, because the user explicitly approved it. This is the
/// result to use with the `shell` function call that contained `apply_patch`.
Output(ResponseInputItem),
Output(Result<String, FunctionCallError>),
/// The `apply_patch` call was approved, either automatically because it
/// appears that it should be allowed based on the user's sandbox policy
@@ -28,29 +31,30 @@ pub(crate) enum InternalApplyPatchInvocation {
DelegateToExec(ApplyPatchExec),
}
pub(crate) struct ApplyPatchExec {
pub(crate) action: ApplyPatchAction,
pub(crate) user_explicitly_approved_this_action: bool,
#[derive(Debug)]
pub struct ApplyPatchExec {
pub action: ApplyPatchAction,
pub user_explicitly_approved_this_action: bool,
}
impl From<ResponseInputItem> for InternalApplyPatchInvocation {
fn from(item: ResponseInputItem) -> Self {
InternalApplyPatchInvocation::Output(item)
}
pub struct ApplyPatchContext<'a> {
pub approval_policy: AskForApproval,
pub sandbox_policy: &'a SandboxPolicy,
pub cwd: &'a Path,
}
pub(crate) async fn apply_patch(
sess: &Session,
turn_context: &TurnContext,
pub async fn apply_patch(
approvals: &dyn ApprovalCoordinator,
context: ApplyPatchContext<'_>,
sub_id: &str,
call_id: &str,
action: ApplyPatchAction,
) -> InternalApplyPatchInvocation {
match assess_patch_safety(
&action,
turn_context.approval_policy,
&turn_context.sandbox_policy,
&turn_context.cwd,
context.approval_policy,
context.sandbox_policy,
context.cwd,
) {
SafetyCheck::AutoApprove { .. } => {
InternalApplyPatchInvocation::DelegateToExec(ApplyPatchExec {
@@ -59,17 +63,11 @@ pub(crate) async fn apply_patch(
})
}
SafetyCheck::AskUser => {
// Compute a readable summary of path changes to include in the
// approval request so the user can make an informed decision.
//
// Note that it might be worth expanding this approval request to
// give the user the option to expand the set of writable roots so
// that similar patches can be auto-approved in the future during
// this session.
let rx_approve = sess
let approval = approvals
.request_patch_approval(sub_id.to_owned(), call_id.to_owned(), &action, None, None)
.await;
match rx_approve.await.unwrap_or_default() {
match approval {
ReviewDecision::Approved | ReviewDecision::ApprovedForSession => {
InternalApplyPatchInvocation::DelegateToExec(ApplyPatchExec {
action,
@@ -77,31 +75,19 @@ pub(crate) async fn apply_patch(
})
}
ReviewDecision::Denied | ReviewDecision::Abort => {
ResponseInputItem::FunctionCallOutput {
call_id: call_id.to_owned(),
output: FunctionCallOutputPayload {
content: "patch rejected by user".to_string(),
success: Some(false),
},
}
.into()
InternalApplyPatchInvocation::Output(Err(FunctionCallError::RespondToModel(
"patch rejected by user".to_string(),
)))
}
}
}
SafetyCheck::Reject { reason } => ResponseInputItem::FunctionCallOutput {
call_id: call_id.to_owned(),
output: FunctionCallOutputPayload {
content: format!("patch rejected: {reason}"),
success: Some(false),
},
}
.into(),
SafetyCheck::Reject { reason } => InternalApplyPatchInvocation::Output(Err(
FunctionCallError::RespondToModel(format!("patch rejected: {reason}")),
)),
}
}
pub(crate) fn convert_apply_patch_to_protocol(
action: &ApplyPatchAction,
) -> HashMap<PathBuf, FileChange> {
pub fn convert_apply_patch_to_protocol(action: &ApplyPatchAction) -> HashMap<PathBuf, FileChange> {
let changes = action.changes();
let mut result = HashMap::with_capacity(changes.len());
for (path, change) in changes {

View File

@@ -1,3 +1,4 @@
use tree_sitter::Node;
use tree_sitter::Parser;
use tree_sitter::Tree;
use tree_sitter_bash::LANGUAGE as BASH;
@@ -73,6 +74,9 @@ pub fn try_parse_word_only_commands_sequence(tree: &Tree, src: &str) -> Option<V
}
}
// Walk uses a stack (LIFO), so re-sort by position to restore source order.
command_nodes.sort_by_key(Node::start_byte);
let mut commands = Vec::new();
for node in command_nodes {
if let Some(words) = parse_plain_command_from_node(node, src) {
@@ -84,6 +88,21 @@ pub fn try_parse_word_only_commands_sequence(tree: &Tree, src: &str) -> Option<V
Some(commands)
}
/// Returns the sequence of plain commands within a `bash -lc "..."` invocation
/// when the script only contains word-only commands joined by safe operators.
pub fn parse_bash_lc_plain_commands(command: &[String]) -> Option<Vec<Vec<String>>> {
let [bash, flag, script] = command else {
return None;
};
if bash != "bash" || flag != "-lc" {
return None;
}
let tree = try_parse_bash(script)?;
try_parse_word_only_commands_sequence(&tree, script)
}
fn parse_plain_command_from_node(cmd: tree_sitter::Node, src: &str) -> Option<Vec<String>> {
if cmd.kind() != "command" {
return None;
@@ -150,10 +169,10 @@ mod tests {
let src = "ls && pwd; echo 'hi there' | wc -l";
let cmds = parse_seq(src).unwrap();
let expected: Vec<Vec<String>> = vec![
vec!["wc".to_string(), "-l".to_string()],
vec!["echo".to_string(), "hi there".to_string()],
vec!["pwd".to_string()],
vec!["ls".to_string()],
vec!["pwd".to_string()],
vec!["echo".to_string(), "hi there".to_string()],
vec!["wc".to_string(), "-l".to_string()],
];
assert_eq!(cmds, expected);
}

View File

@@ -0,0 +1,363 @@
use crate::model_family::ModelFamily;
use crate::tool_schema::OpenAiTool;
use codex_apply_patch::APPLY_PATCH_TOOL_INSTRUCTIONS;
use codex_protocol::config_types::ReasoningEffort as ReasoningEffortConfig;
use codex_protocol::config_types::ReasoningSummary as ReasoningSummaryConfig;
use codex_protocol::config_types::Verbosity as VerbosityConfig;
use codex_protocol::models::ResponseItem;
use codex_protocol::protocol::RateLimitSnapshot;
use codex_protocol::protocol::TokenUsage;
use futures::Stream;
use serde::Serialize;
use serde_json::Value;
use std::borrow::Cow;
use std::ops::Deref;
use std::pin::Pin;
use std::task::Context;
use std::task::Poll;
use tokio::sync::mpsc;
/// Review thread system prompt. Edit `agent/review_prompt.md` to customize.
pub const REVIEW_PROMPT: &str = include_str!("../review_prompt.md");
/// API request payload for a single model turn
#[derive(Default, Debug, Clone)]
pub struct Prompt {
/// Conversation context input items.
pub input: Vec<ResponseItem>,
/// Tools available to the model, including additional tools sourced from
/// external MCP servers.
pub tools: Vec<OpenAiTool>,
/// Optional override for the built-in BASE_INSTRUCTIONS.
pub base_instructions_override: Option<String>,
/// Optional the output schema for the model's response.
pub output_schema: Option<Value>,
}
impl Prompt {
pub fn get_full_instructions<'a>(&'a self, model: &'a ModelFamily) -> Cow<'a, str> {
let base = self
.base_instructions_override
.as_deref()
.unwrap_or(model.base_instructions.deref());
// When there are no custom instructions, add apply_patch_tool_instructions if:
// - the model needs special instructions (4.1)
// AND
// - there is no apply_patch tool present
let is_apply_patch_tool_present = self.tools.iter().any(|tool| match tool {
OpenAiTool::Function(f) => f.name == "apply_patch",
OpenAiTool::Freeform(f) => f.name == "apply_patch",
_ => false,
});
if self.base_instructions_override.is_none()
&& model.needs_special_apply_patch_instructions
&& !is_apply_patch_tool_present
{
Cow::Owned(format!("{base}\n{APPLY_PATCH_TOOL_INSTRUCTIONS}"))
} else {
Cow::Borrowed(base)
}
}
pub fn get_formatted_input(&self) -> Vec<ResponseItem> {
self.input.clone()
}
}
#[derive(Debug)]
pub enum ResponseEvent {
Created,
OutputItemDone(ResponseItem),
Completed {
response_id: String,
token_usage: Option<TokenUsage>,
},
OutputTextDelta(String),
ReasoningSummaryDelta(String),
ReasoningContentDelta(String),
ReasoningSummaryPartAdded,
WebSearchCallBegin {
call_id: String,
},
RateLimits(RateLimitSnapshot),
}
#[derive(Debug, Serialize)]
pub struct Reasoning {
#[serde(skip_serializing_if = "Option::is_none")]
pub effort: Option<ReasoningEffortConfig>,
#[serde(skip_serializing_if = "Option::is_none")]
pub summary: Option<ReasoningSummaryConfig>,
}
#[derive(Debug, Serialize, Default, Clone)]
#[serde(rename_all = "snake_case")]
pub enum TextFormatType {
#[default]
JsonSchema,
}
#[derive(Debug, Serialize, Default, Clone)]
pub struct TextFormat {
pub r#type: TextFormatType,
pub strict: bool,
pub schema: Value,
pub name: String,
}
/// Controls under the `text` field in the Responses API for GPT-5.
#[derive(Debug, Serialize, Default, Clone)]
pub struct TextControls {
#[serde(skip_serializing_if = "Option::is_none")]
pub verbosity: Option<OpenAiVerbosity>,
#[serde(skip_serializing_if = "Option::is_none")]
pub format: Option<TextFormat>,
}
#[derive(Debug, Serialize, Default, Clone)]
#[serde(rename_all = "lowercase")]
pub enum OpenAiVerbosity {
Low,
#[default]
Medium,
High,
}
impl From<VerbosityConfig> for OpenAiVerbosity {
fn from(v: VerbosityConfig) -> Self {
match v {
VerbosityConfig::Low => OpenAiVerbosity::Low,
VerbosityConfig::Medium => OpenAiVerbosity::Medium,
VerbosityConfig::High => OpenAiVerbosity::High,
}
}
}
/// Request object that is serialized as JSON and POST'ed when using the
/// Responses API.
#[derive(Debug, Serialize)]
pub struct ResponsesApiRequest<'a> {
pub model: &'a str,
pub instructions: &'a str,
// TODO(mbolin): ResponseItem::Other should not be serialized. Currently,
// we code defensively to avoid this case, but perhaps we should use a
// separate enum for serialization.
pub input: &'a Vec<ResponseItem>,
pub tools: &'a [serde_json::Value],
pub tool_choice: &'static str,
pub parallel_tool_calls: bool,
pub reasoning: Option<Reasoning>,
pub store: bool,
pub stream: bool,
pub include: Vec<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub prompt_cache_key: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub text: Option<TextControls>,
}
pub fn create_reasoning_param_for_request(
model_family: &ModelFamily,
effort: Option<ReasoningEffortConfig>,
summary: ReasoningSummaryConfig,
) -> Option<Reasoning> {
if !model_family.supports_reasoning_summaries {
return None;
}
Some(Reasoning {
effort,
summary: Some(summary),
})
}
pub fn create_text_param_for_request(
verbosity: Option<VerbosityConfig>,
output_schema: &Option<Value>,
) -> Option<TextControls> {
if verbosity.is_none() && output_schema.is_none() {
return None;
}
Some(TextControls {
verbosity: verbosity.map(std::convert::Into::into),
format: output_schema.as_ref().map(|schema| TextFormat {
r#type: TextFormatType::JsonSchema,
strict: true,
schema: schema.clone(),
name: "codex_output_schema".to_string(),
}),
})
}
pub struct ResponseStream<E> {
pub rx_event: mpsc::Receiver<std::result::Result<ResponseEvent, E>>,
}
impl<E> Stream for ResponseStream<E> {
type Item = std::result::Result<ResponseEvent, E>;
fn poll_next(
mut self: Pin<&mut Self>,
cx: &mut Context<'_>,
) -> Poll<Option<std::result::Result<ResponseEvent, E>>> {
self.rx_event.poll_recv(cx)
}
}
#[cfg(test)]
mod tests {
use crate::config_types::ReasoningSummaryFormat;
use crate::tooling::ApplyPatchToolType;
use pretty_assertions::assert_eq;
use super::*;
struct InstructionsTestCase {
pub slug: &'static str,
pub expects_apply_patch_instructions: bool,
}
#[test]
fn get_full_instructions_no_user_content() {
let prompt = Prompt::default();
let base_instructions = "Base instructions".to_string();
let test_cases = vec![
InstructionsTestCase {
slug: "needs-apply-patch",
expects_apply_patch_instructions: true,
},
InstructionsTestCase {
slug: "no-apply-patch",
expects_apply_patch_instructions: false,
},
];
for test_case in test_cases {
let model_family = ModelFamily {
slug: test_case.slug.to_string(),
family: "test".to_string(),
needs_special_apply_patch_instructions: test_case.expects_apply_patch_instructions,
supports_reasoning_summaries: false,
reasoning_summary_format: ReasoningSummaryFormat::None,
uses_local_shell_tool: false,
apply_patch_tool_type: Some(ApplyPatchToolType::Function),
base_instructions: base_instructions.clone(),
};
let expected = if test_case.expects_apply_patch_instructions {
format!(
"{}\n{}",
model_family.base_instructions, APPLY_PATCH_TOOL_INSTRUCTIONS
)
} else {
model_family.base_instructions.clone()
};
let full = prompt.get_full_instructions(&model_family);
assert_eq!(full, expected);
}
}
#[test]
fn serializes_text_verbosity_when_set() {
let input: Vec<ResponseItem> = vec![];
let tools: Vec<serde_json::Value> = vec![];
let req = ResponsesApiRequest {
model: "gpt-5",
instructions: "i",
input: &input,
tools: &tools,
tool_choice: "auto",
parallel_tool_calls: false,
reasoning: None,
store: false,
stream: true,
include: vec![],
prompt_cache_key: None,
text: Some(TextControls {
verbosity: Some(OpenAiVerbosity::Low),
format: None,
}),
};
let v = serde_json::to_value(&req).expect("json");
assert_eq!(
v.get("text")
.and_then(|t| t.get("verbosity"))
.and_then(|s| s.as_str()),
Some("low")
);
}
#[test]
fn serializes_text_schema_with_strict_format() {
let input: Vec<ResponseItem> = vec![];
let tools: Vec<serde_json::Value> = vec![];
let schema = serde_json::json!({
"type": "object",
"properties": {
"answer": {"type": "string"}
},
"required": ["answer"],
});
let text_controls =
create_text_param_for_request(None, &Some(schema.clone())).expect("text controls");
let req = ResponsesApiRequest {
model: "gpt-5",
instructions: "i",
input: &input,
tools: &tools,
tool_choice: "auto",
parallel_tool_calls: false,
reasoning: None,
store: false,
stream: true,
include: vec![],
prompt_cache_key: None,
text: Some(text_controls),
};
let v = serde_json::to_value(&req).expect("json");
let text = v.get("text").expect("text field");
assert!(text.get("verbosity").is_none());
let format = text.get("format").expect("format field");
assert_eq!(
format.get("name"),
Some(&serde_json::Value::String("codex_output_schema".into()))
);
assert_eq!(
format.get("type"),
Some(&serde_json::Value::String("json_schema".into()))
);
assert_eq!(format.get("strict"), Some(&serde_json::Value::Bool(true)));
assert_eq!(format.get("schema"), Some(&schema));
}
#[test]
fn omits_text_when_not_set() {
let input: Vec<ResponseItem> = vec![];
let tools: Vec<serde_json::Value> = vec![];
let req = ResponsesApiRequest {
model: "gpt-5",
instructions: "i",
input: &input,
tools: &tools,
tool_choice: "auto",
parallel_tool_calls: false,
reasoning: None,
store: false,
stream: true,
include: vec![],
prompt_cache_key: None,
text: None,
};
let v = serde_json::to_value(&req).expect("json");
assert!(v.get("text").is_none());
}
}

View File

@@ -0,0 +1,99 @@
use crate::bash::parse_bash_lc_plain_commands;
pub fn command_might_be_dangerous(command: &[String]) -> bool {
if is_dangerous_to_call_with_exec(command) {
return true;
}
// Support `bash -lc "<script>"` where the any part of the script might contain a dangerous command.
if let Some(all_commands) = parse_bash_lc_plain_commands(command)
&& all_commands
.iter()
.any(|cmd| is_dangerous_to_call_with_exec(cmd))
{
return true;
}
false
}
fn is_dangerous_to_call_with_exec(command: &[String]) -> bool {
let cmd0 = command.first().map(String::as_str);
match cmd0 {
Some(cmd) if cmd.ends_with("git") || cmd.ends_with("/git") => {
matches!(command.get(1).map(String::as_str), Some("reset" | "rm"))
}
Some("rm") => matches!(command.get(1).map(String::as_str), Some("-f" | "-rf")),
// for sudo <cmd> simply do the check for <cmd>
Some("sudo") => is_dangerous_to_call_with_exec(&command[1..]),
// ── anything else ─────────────────────────────────────────────────
_ => false,
}
}
#[cfg(test)]
mod tests {
use super::*;
fn vec_str(items: &[&str]) -> Vec<String> {
items.iter().map(std::string::ToString::to_string).collect()
}
#[test]
fn git_reset_is_dangerous() {
assert!(command_might_be_dangerous(&vec_str(&["git", "reset"])));
}
#[test]
fn bash_git_reset_is_dangerous() {
assert!(command_might_be_dangerous(&vec_str(&[
"bash",
"-lc",
"git reset --hard"
])));
}
#[test]
fn git_status_is_not_dangerous() {
assert!(!command_might_be_dangerous(&vec_str(&["git", "status"])));
}
#[test]
fn bash_git_status_is_not_dangerous() {
assert!(!command_might_be_dangerous(&vec_str(&[
"bash",
"-lc",
"git status"
])));
}
#[test]
fn sudo_git_reset_is_dangerous() {
assert!(command_might_be_dangerous(&vec_str(&[
"sudo", "git", "reset", "--hard"
])));
}
#[test]
fn usr_bin_git_is_dangerous() {
assert!(command_might_be_dangerous(&vec_str(&[
"/usr/bin/git",
"reset",
"--hard"
])));
}
#[test]
fn rm_rf_is_dangerous() {
assert!(command_might_be_dangerous(&vec_str(&["rm", "-rf", "/"])));
}
#[test]
fn rm_f_is_dangerous() {
assert!(command_might_be_dangerous(&vec_str(&["rm", "-f", "/"])));
}
}

View File

@@ -1,7 +1,14 @@
use crate::bash::try_parse_bash;
use crate::bash::try_parse_word_only_commands_sequence;
use crate::bash::parse_bash_lc_plain_commands;
pub fn is_known_safe_command(command: &[String]) -> bool {
#[cfg(target_os = "windows")]
{
use super::windows_safe_commands::is_safe_command_windows;
if is_safe_command_windows(command) {
return true;
}
}
if is_safe_to_call_with_exec(command) {
return true;
}
@@ -12,11 +19,7 @@ pub fn is_known_safe_command(command: &[String]) -> bool {
// introduce side effects ( "&&", "||", ";", and "|" ). If every
// individual command in the script is itself a knownsafe command, then
// the composite expression is considered safe.
if let [bash, flag, script] = command
&& bash == "bash"
&& flag == "-lc"
&& let Some(tree) = try_parse_bash(script)
&& let Some(all_commands) = try_parse_word_only_commands_sequence(&tree, script)
if let Some(all_commands) = parse_bash_lc_plain_commands(command)
&& !all_commands.is_empty()
&& all_commands
.iter()
@@ -24,7 +27,6 @@ pub fn is_known_safe_command(command: &[String]) -> bool {
{
return true;
}
false
}
@@ -160,9 +162,10 @@ fn is_valid_sed_n_arg(arg: Option<&str>) -> bool {
#[cfg(test)]
mod tests {
use super::*;
use std::string::ToString;
fn vec_str(args: &[&str]) -> Vec<String> {
args.iter().map(|s| s.to_string()).collect()
args.iter().map(ToString::to_string).collect()
}
#[test]

View File

@@ -0,0 +1,4 @@
pub mod is_dangerous_command;
pub mod is_safe_command;
#[cfg(target_os = "windows")]
pub mod windows_safe_commands;

View File

@@ -0,0 +1,25 @@
// This is a WIP. This will eventually contain a real list of common safe Windows commands.
pub fn is_safe_command_windows(_command: &[String]) -> bool {
false
}
#[cfg(test)]
mod tests {
use super::is_safe_command_windows;
fn vec_str(args: &[&str]) -> Vec<String> {
args.iter().map(ToString::to_string).collect()
}
#[test]
fn everything_is_unsafe() {
for cmd in [
vec_str(&["powershell.exe", "-NoLogo", "-Command", "echo hello"]),
vec_str(&["copy", "foo", "bar"]),
vec_str(&["del", "file.txt"]),
vec_str(&["powershell.exe", "Get-ChildItem"]),
] {
assert!(!is_safe_command_windows(&cmd));
}
}
}

View File

@@ -0,0 +1,305 @@
//! Shared configuration data structures for Codex runtime and hosts.
//
// This module intentionally focuses on simple data containers without
// business logic so they can be reused across crates.
use std::collections::HashMap;
use std::path::PathBuf;
use std::time::Duration;
use wildmatch::WildMatchPattern;
use serde::Deserialize;
use serde::Deserializer;
use serde::Serialize;
use serde::de::Error as SerdeError;
#[derive(Serialize, Debug, Clone, PartialEq)]
pub struct McpServerConfig {
pub command: String,
#[serde(default)]
pub args: Vec<String>,
#[serde(default)]
pub env: Option<HashMap<String, String>>,
/// Startup timeout in seconds for initializing MCP server & initially listing tools.
#[serde(
default,
with = "option_duration_secs",
skip_serializing_if = "Option::is_none"
)]
pub startup_timeout_sec: Option<Duration>,
/// Default timeout for MCP tool calls initiated via this server.
#[serde(default, with = "option_duration_secs")]
pub tool_timeout_sec: Option<Duration>,
}
impl<'de> Deserialize<'de> for McpServerConfig {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
#[derive(Deserialize)]
struct RawMcpServerConfig {
command: String,
#[serde(default)]
args: Vec<String>,
#[serde(default)]
env: Option<HashMap<String, String>>,
#[serde(default)]
startup_timeout_sec: Option<f64>,
#[serde(default)]
startup_timeout_ms: Option<u64>,
#[serde(default, with = "option_duration_secs")]
tool_timeout_sec: Option<Duration>,
}
let raw = RawMcpServerConfig::deserialize(deserializer)?;
let startup_timeout_sec = match (raw.startup_timeout_sec, raw.startup_timeout_ms) {
(Some(sec), _) => {
let duration = Duration::try_from_secs_f64(sec).map_err(SerdeError::custom)?;
Some(duration)
}
(None, Some(ms)) => Some(Duration::from_millis(ms)),
(None, None) => None,
};
Ok(Self {
command: raw.command,
args: raw.args,
env: raw.env,
startup_timeout_sec,
tool_timeout_sec: raw.tool_timeout_sec,
})
}
}
mod option_duration_secs {
use serde::Deserialize;
use serde::Deserializer;
use serde::Serializer;
use std::time::Duration;
pub fn serialize<S>(value: &Option<Duration>, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
match value {
Some(duration) => serializer.serialize_some(&duration.as_secs_f64()),
None => serializer.serialize_none(),
}
}
pub fn deserialize<'de, D>(deserializer: D) -> Result<Option<Duration>, D::Error>
where
D: Deserializer<'de>,
{
let secs = Option::<f64>::deserialize(deserializer)?;
secs.map(|secs| Duration::try_from_secs_f64(secs).map_err(serde::de::Error::custom))
.transpose()
}
}
#[derive(Deserialize, Debug, Copy, Clone, PartialEq)]
pub enum UriBasedFileOpener {
#[serde(rename = "vscode")]
VsCode,
#[serde(rename = "vscode-insiders")]
VsCodeInsiders,
#[serde(rename = "windsurf")]
Windsurf,
#[serde(rename = "cursor")]
Cursor,
/// Option to disable the URI-based file opener.
#[serde(rename = "none")]
None,
}
impl UriBasedFileOpener {
pub fn get_scheme(&self) -> Option<&str> {
match self {
UriBasedFileOpener::VsCode => Some("vscode"),
UriBasedFileOpener::VsCodeInsiders => Some("vscode-insiders"),
UriBasedFileOpener::Windsurf => Some("windsurf"),
UriBasedFileOpener::Cursor => Some("cursor"),
UriBasedFileOpener::None => None,
}
}
}
/// Settings that govern if and what will be written to `~/.codex/history.jsonl`.
#[derive(Deserialize, Debug, Clone, PartialEq, Default)]
pub struct History {
/// If true, history entries will not be written to disk.
pub persistence: HistoryPersistence,
/// If set, the maximum size of the history file in bytes.
/// TODO(mbolin): Not currently honored.
pub max_bytes: Option<usize>,
}
#[derive(Deserialize, Debug, Copy, Clone, PartialEq, Default)]
#[serde(rename_all = "kebab-case")]
pub enum HistoryPersistence {
/// Save all history entries to disk.
#[default]
SaveAll,
/// Do not write history to disk.
None,
}
#[derive(Debug, Clone, PartialEq, Eq, Deserialize)]
#[serde(untagged)]
pub enum Notifications {
Enabled(bool),
Custom(Vec<String>),
}
impl Default for Notifications {
fn default() -> Self {
Self::Enabled(false)
}
}
/// Collection of settings that are specific to the TUI.
#[derive(Deserialize, Debug, Clone, PartialEq, Default)]
pub struct Tui {
/// Enable desktop notifications from the TUI when the terminal is unfocused.
/// Defaults to `false`.
#[serde(default)]
pub notifications: Notifications,
}
#[derive(Deserialize, Debug, Clone, PartialEq, Default)]
pub struct SandboxWorkspaceWrite {
#[serde(default)]
pub writable_roots: Vec<PathBuf>,
#[serde(default)]
pub network_access: bool,
#[serde(default)]
pub exclude_tmpdir_env_var: bool,
#[serde(default)]
pub exclude_slash_tmp: bool,
}
impl From<SandboxWorkspaceWrite> for codex_protocol::mcp_protocol::SandboxSettings {
fn from(sandbox_workspace_write: SandboxWorkspaceWrite) -> Self {
Self {
writable_roots: sandbox_workspace_write.writable_roots,
network_access: Some(sandbox_workspace_write.network_access),
exclude_tmpdir_env_var: Some(sandbox_workspace_write.exclude_tmpdir_env_var),
exclude_slash_tmp: Some(sandbox_workspace_write.exclude_slash_tmp),
}
}
}
#[derive(Deserialize, Debug, Clone, PartialEq, Default)]
#[serde(rename_all = "kebab-case")]
pub enum ShellEnvironmentPolicyInherit {
/// "Core" environment variables for the platform. On UNIX, this would
/// include HOME, LOGNAME, PATH, SHELL, and USER, among others.
Core,
/// Inherits the full environment from the parent process.
#[default]
All,
/// Do not inherit any environment variables from the parent process.
None,
}
/// Policy for building the `env` when spawning a process via either the
/// `shell` or `local_shell` tool.
#[derive(Deserialize, Debug, Clone, PartialEq, Default)]
pub struct ShellEnvironmentPolicyToml {
pub inherit: Option<ShellEnvironmentPolicyInherit>,
pub ignore_default_excludes: Option<bool>,
/// List of regular expressions.
pub exclude: Option<Vec<String>>,
pub r#set: Option<HashMap<String, String>>,
/// List of regular expressions.
pub include_only: Option<Vec<String>>,
pub experimental_use_profile: Option<bool>,
}
pub type EnvironmentVariablePattern = WildMatchPattern<'*', '?'>;
/// Deriving the `env` based on this policy works as follows:
/// 1. Create an initial map based on the `inherit` policy.
/// 2. If `ignore_default_excludes` is false, filter the map using the default
/// exclude pattern(s), which are: `"*KEY*"` and `"*TOKEN*"`.
/// 3. If `exclude` is not empty, filter the map using the provided patterns.
/// 4. Insert any entries from `r#set` into the map.
/// 5. If non-empty, filter the map using the `include_only` patterns.
#[derive(Debug, Clone, PartialEq, Default)]
pub struct ShellEnvironmentPolicy {
/// Starting point when building the environment.
pub inherit: ShellEnvironmentPolicyInherit,
/// True to skip the check to exclude default environment variables that
/// contain "KEY" or "TOKEN" in their name.
pub ignore_default_excludes: bool,
/// Environment variable names to exclude from the environment.
pub exclude: Vec<EnvironmentVariablePattern>,
/// (key, value) pairs to insert in the environment.
pub r#set: HashMap<String, String>,
/// Environment variable names to retain in the environment.
pub include_only: Vec<EnvironmentVariablePattern>,
/// If true, the shell profile will be used to run the command.
pub use_profile: bool,
}
impl From<ShellEnvironmentPolicyToml> for ShellEnvironmentPolicy {
fn from(toml: ShellEnvironmentPolicyToml) -> Self {
// Default to inheriting the full environment when not specified.
let inherit = toml.inherit.unwrap_or(ShellEnvironmentPolicyInherit::All);
let ignore_default_excludes = toml.ignore_default_excludes.unwrap_or(false);
let exclude = toml
.exclude
.unwrap_or_default()
.into_iter()
.map(|s| EnvironmentVariablePattern::new_case_insensitive(&s))
.collect();
let r#set = toml.r#set.unwrap_or_default();
let include_only = toml
.include_only
.unwrap_or_default()
.into_iter()
.map(|s| EnvironmentVariablePattern::new_case_insensitive(&s))
.collect();
let use_profile = toml.experimental_use_profile.unwrap_or(false);
Self {
inherit,
ignore_default_excludes,
exclude,
r#set,
include_only,
use_profile,
}
}
}
#[derive(Deserialize, Debug, Clone, PartialEq, Eq, Default, Hash)]
#[serde(rename_all = "kebab-case")]
pub enum ReasoningSummaryFormat {
#[default]
None,
Experimental,
}

View File

@@ -0,0 +1,117 @@
use codex_protocol::models::ResponseItem;
/// Transcript of conversation history shared across agent hosts.
#[derive(Debug, Clone, Default)]
pub struct ConversationHistory {
/// Oldest items appear at the start of the vector.
items: Vec<ResponseItem>,
}
impl ConversationHistory {
pub fn new() -> Self {
Self { items: Vec::new() }
}
/// Returns a clone of the stored transcript.
pub fn contents(&self) -> Vec<ResponseItem> {
self.items.clone()
}
/// Records additional response items, filtering out non-API messages.
pub fn record_items<I>(&mut self, items: I)
where
I: IntoIterator,
I::Item: std::ops::Deref<Target = ResponseItem>,
{
for item in items {
if !is_api_message(&item) {
continue;
}
self.items.push(item.clone());
}
}
pub fn replace(&mut self, items: Vec<ResponseItem>) {
self.items = items;
}
}
/// Detects whether the given message should be persisted to history.
fn is_api_message(message: &ResponseItem) -> bool {
match message {
ResponseItem::Message { role, .. } => role.as_str() != "system",
ResponseItem::FunctionCallOutput { .. }
| ResponseItem::FunctionCall { .. }
| ResponseItem::CustomToolCall { .. }
| ResponseItem::CustomToolCallOutput { .. }
| ResponseItem::LocalShellCall { .. }
| ResponseItem::Reasoning { .. }
| ResponseItem::WebSearchCall { .. } => true,
ResponseItem::Other => false,
}
}
#[cfg(test)]
mod tests {
use super::*;
use codex_protocol::models::ContentItem;
fn assistant_msg(text: &str) -> ResponseItem {
ResponseItem::Message {
id: None,
role: "assistant".to_string(),
content: vec![ContentItem::OutputText {
text: text.to_string(),
}],
}
}
fn user_msg(text: &str) -> ResponseItem {
ResponseItem::Message {
id: None,
role: "user".to_string(),
content: vec![ContentItem::OutputText {
text: text.to_string(),
}],
}
}
#[test]
fn filters_non_api_messages() {
let mut h = ConversationHistory::default();
let system = ResponseItem::Message {
id: None,
role: "system".to_string(),
content: vec![ContentItem::OutputText {
text: "ignored".to_string(),
}],
};
h.record_items([&system, &ResponseItem::Other]);
let u = user_msg("hi");
let a = assistant_msg("hello");
h.record_items([&u, &a]);
let items = h.contents();
assert_eq!(
items,
vec![
ResponseItem::Message {
id: None,
role: "user".to_string(),
content: vec![ContentItem::OutputText {
text: "hi".to_string()
}]
},
ResponseItem::Message {
id: None,
role: "assistant".to_string(),
content: vec![ContentItem::OutputText {
text: "hello".to_string()
}]
}
]
);
}
}

View File

@@ -0,0 +1,21 @@
use std::collections::HashMap;
use std::path::PathBuf;
use std::time::Duration;
const DEFAULT_TIMEOUT_MS: u64 = 10_000;
#[derive(Clone, Debug)]
pub struct ExecParams {
pub command: Vec<String>,
pub cwd: PathBuf,
pub timeout_ms: Option<u64>,
pub env: HashMap<String, String>,
pub with_escalated_permissions: Option<bool>,
pub justification: Option<String>,
}
impl ExecParams {
pub fn timeout_duration(&self) -> Duration {
Duration::from_millis(self.timeout_ms.unwrap_or(DEFAULT_TIMEOUT_MS))
}
}

View File

@@ -5,7 +5,8 @@ use tokio::sync::mpsc;
use tokio::task::JoinHandle;
#[derive(Debug)]
pub(crate) struct ExecCommandSession {
#[allow(dead_code)]
pub struct ExecCommandSession {
/// Queue for writing bytes to the process stdin (PTY master write side).
writer_tx: mpsc::Sender<Vec<u8>>,
/// Broadcast stream of output chunks read from the PTY. New subscribers
@@ -29,8 +30,9 @@ pub(crate) struct ExecCommandSession {
exit_status: std::sync::Arc<std::sync::atomic::AtomicBool>,
}
#[allow(dead_code)]
impl ExecCommandSession {
pub(crate) fn new(
pub fn new(
writer_tx: mpsc::Sender<Vec<u8>>,
output_tx: broadcast::Sender<Vec<u8>>,
killer: Box<dyn portable_pty::ChildKiller + Send + Sync>,
@@ -54,7 +56,7 @@ impl ExecCommandSession {
)
}
pub(crate) fn writer_sender(&self) -> mpsc::Sender<Vec<u8>> {
pub fn writer_sender(&self) -> mpsc::Sender<Vec<u8>> {
self.writer_tx.clone()
}
@@ -62,7 +64,7 @@ impl ExecCommandSession {
self.output_tx.subscribe()
}
pub(crate) fn has_exited(&self) -> bool {
pub fn has_exited(&self) -> bool {
self.exit_status.load(std::sync::atomic::Ordering::SeqCst)
}
}

View File

@@ -0,0 +1,11 @@
mod exec_command_params;
mod exec_command_session;
mod session_id;
mod session_manager;
pub use exec_command_params::ExecCommandParams;
pub use exec_command_params::WriteStdinParams;
pub use exec_command_session::ExecCommandSession;
pub use session_id::SessionId;
pub use session_manager::ExecCommandOutput;
pub use session_manager::SessionManager as ExecSessionManager;

View File

@@ -2,4 +2,4 @@ use serde::Deserialize;
use serde::Serialize;
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]
pub(crate) struct SessionId(pub u32);
pub struct SessionId(pub u32);

View File

@@ -5,6 +5,7 @@ use std::sync::Arc;
use std::sync::Mutex as StdMutex;
use std::sync::atomic::AtomicBool;
use std::sync::atomic::AtomicU32;
use std::vec::Vec;
use portable_pty::CommandBuilder;
use portable_pty::PtySize;
@@ -21,7 +22,6 @@ use crate::exec_command::exec_command_params::WriteStdinParams;
use crate::exec_command::exec_command_session::ExecCommandSession;
use crate::exec_command::session_id::SessionId;
use crate::truncate::truncate_middle;
use codex_protocol::models::FunctionCallOutputPayload;
#[derive(Debug, Default)]
pub struct SessionManager {
@@ -29,6 +29,7 @@ pub struct SessionManager {
sessions: Mutex<HashMap<SessionId, ExecCommandSession>>,
}
#[allow(dead_code)]
#[derive(Debug)]
pub struct ExecCommandOutput {
wall_time: Duration,
@@ -38,7 +39,7 @@ pub struct ExecCommandOutput {
}
impl ExecCommandOutput {
fn to_text_output(&self) -> String {
pub fn to_text_output(&self) -> String {
let wall_time_secs = self.wall_time.as_secs_f32();
let termination_status = match self.exit_status {
ExitStatus::Exited(code) => format!("Process exited with code {code}"),
@@ -62,25 +63,13 @@ Output:
}
}
#[allow(dead_code)]
#[derive(Debug)]
pub enum ExitStatus {
Exited(i32),
Ongoing(SessionId),
}
pub fn result_into_payload(result: Result<ExecCommandOutput, String>) -> FunctionCallOutputPayload {
match result {
Ok(output) => FunctionCallOutputPayload {
content: output.to_text_output(),
success: Some(true),
},
Err(err) => FunctionCallOutputPayload {
content: err,
success: Some(false),
},
}
}
impl SessionManager {
/// Processes the request and is required to send a response via `outgoing`.
pub async fn handle_exec_command_request(
@@ -93,7 +82,11 @@ impl SessionManager {
.fetch_add(1, std::sync::atomic::Ordering::SeqCst),
);
let (session, mut output_rx, mut exit_rx) = create_exec_command_session(params.clone())
let (session, mut output_rx, mut exit_rx): (
ExecCommandSession,
tokio::sync::broadcast::Receiver<Vec<u8>>,
tokio::sync::oneshot::Receiver<i32>,
) = create_exec_command_session(params.clone())
.await
.map_err(|err| {
format!(

View File

@@ -0,0 +1,7 @@
use thiserror::Error;
#[derive(Debug, Error, PartialEq)]
pub enum FunctionCallError {
#[error("{0}")]
RespondToModel(String),
}

58
codex-rs/agent/src/lib.rs Normal file
View File

@@ -0,0 +1,58 @@
pub mod apply_patch;
pub mod bash;
pub mod client_common;
pub mod command_safety;
pub mod config_types;
pub mod conversation_history;
pub mod exec;
pub mod exec_command;
pub mod function_tool;
pub mod model_client;
pub mod model_family;
pub mod model_provider;
pub mod notifications;
pub mod rollout;
pub mod runtime;
pub mod runtime_config;
pub mod safety;
pub mod sandbox;
pub mod services;
pub mod session_services;
pub mod session_state;
pub mod shell;
pub mod token_data;
pub mod tool_schema;
pub mod tooling;
pub mod tools_config;
pub mod truncate;
pub mod turn_diff_tracker;
pub mod unified_exec;
pub use apply_patch::*;
pub use bash::*;
pub use client_common::*;
pub use command_safety::*;
pub use config_types::*;
pub use conversation_history::*;
pub use exec::*;
pub use function_tool::*;
pub use model_client::*;
pub use model_family::*;
pub use model_provider::*;
pub use notifications::*;
pub use rollout::*;
pub use runtime::*;
pub use runtime_config::*;
pub use safety::*;
pub use sandbox::*;
pub use services::*;
pub use session_services::*;
pub use session_state::*;
pub use shell::*;
pub use token_data::*;
pub use tool_schema::*;
pub use tooling::*;
pub use tools_config::*;
pub use truncate::*;
pub use turn_diff_tracker::*;
pub use unified_exec::*;

View File

@@ -0,0 +1,34 @@
use std::sync::Arc;
use async_trait::async_trait;
use codex_protocol::config_types::ReasoningEffort as ReasoningEffortConfig;
use codex_protocol::config_types::ReasoningSummary as ReasoningSummaryConfig;
use crate::client_common::Prompt;
use crate::client_common::ResponseStream;
use crate::model_family::ModelFamily;
use crate::model_provider::ModelProviderInfo;
use crate::services::CredentialsProvider;
#[async_trait]
pub trait ModelClientAdapter: Send + Sync {
type Error: std::error::Error + Send + Sync + 'static;
fn get_model_context_window(&self) -> Option<u64>;
fn get_auto_compact_token_limit(&self) -> Option<i64>;
fn get_provider(&self) -> ModelProviderInfo;
fn get_model(&self) -> String;
fn get_model_family(&self) -> ModelFamily;
fn get_reasoning_effort(&self) -> Option<ReasoningEffortConfig>;
fn get_reasoning_summary(&self) -> ReasoningSummaryConfig;
fn get_auth_manager(&self) -> Option<Arc<dyn CredentialsProvider>>;
async fn stream(&self, prompt: &Prompt) -> Result<ResponseStream<Self::Error>, Self::Error>;
}

View File

@@ -0,0 +1,15 @@
use crate::config_types::ReasoningSummaryFormat;
use crate::tooling::ApplyPatchToolType;
/// Metadata describing consistent behaviour across a family of models.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct ModelFamily {
pub slug: String,
pub family: String,
pub needs_special_apply_patch_instructions: bool,
pub supports_reasoning_summaries: bool,
pub reasoning_summary_format: ReasoningSummaryFormat,
pub uses_local_shell_tool: bool,
pub apply_patch_tool_type: Option<ApplyPatchToolType>,
pub base_instructions: String,
}

View File

@@ -0,0 +1,54 @@
use std::collections::HashMap;
use codex_protocol::mcp_protocol::AuthMode;
use serde::Deserialize;
use serde::Serialize;
/// Wire protocol variants supported by model providers.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum WireApi {
Responses,
#[default]
Chat,
}
/// Serializable representation of a provider definition shared across hosts.
#[derive(Debug, Clone, Deserialize, Serialize, PartialEq)]
pub struct ModelProviderInfo {
pub name: String,
pub base_url: Option<String>,
pub env_key: Option<String>,
pub env_key_instructions: Option<String>,
#[serde(default)]
pub wire_api: WireApi,
pub query_params: Option<HashMap<String, String>>,
pub http_headers: Option<HashMap<String, String>>,
pub env_http_headers: Option<HashMap<String, String>>,
pub request_max_retries: Option<u64>,
pub stream_max_retries: Option<u64>,
pub stream_idle_timeout_ms: Option<u64>,
#[serde(default)]
pub requires_openai_auth: bool,
}
impl ModelProviderInfo {
pub fn wire_api(&self) -> WireApi {
self.wire_api
}
pub fn requires_auth(&self) -> bool {
self.requires_openai_auth
}
pub fn base_url(&self, auth_mode: AuthMode) -> String {
let fallback = if auth_mode == AuthMode::ChatGPT {
"https://chatgpt.com/backend-api/codex"
} else {
"https://api.openai.com/v1"
};
self.base_url
.clone()
.unwrap_or_else(|| fallback.to_string())
}
}

View File

@@ -0,0 +1,15 @@
use serde::Serialize;
/// Cross-host notification payloads emitted by the agent runtime.
#[derive(Debug, Clone, PartialEq, Serialize)]
#[serde(tag = "type", rename_all = "kebab-case")]
pub enum UserNotification {
#[serde(rename_all = "kebab-case")]
AgentTurnComplete {
turn_id: String,
/// Messages submitted by the user to start the turn.
input_messages: Vec<String>,
/// Final assistant message emitted at turn completion.
last_assistant_message: Option<String>,
},
}

View File

@@ -1,9 +1,13 @@
use std::cmp::Reverse;
use std::io::{self};
use std::io;
use std::path::Path;
use std::path::PathBuf;
use codex_file_search as file_search;
use codex_protocol::protocol::EventMsg;
use codex_protocol::protocol::RolloutItem;
use codex_protocol::protocol::RolloutLine;
use serde_json::Value;
use std::num::NonZero;
use std::sync::Arc;
use std::sync::atomic::AtomicBool;
@@ -11,40 +15,29 @@ use time::OffsetDateTime;
use time::PrimitiveDateTime;
use time::format_description::FormatItem;
use time::macros::format_description;
use tokio::fs;
use tokio::io::AsyncBufReadExt;
use uuid::Uuid;
use super::SESSIONS_SUBDIR;
use crate::protocol::EventMsg;
use codex_protocol::protocol::RolloutItem;
use codex_protocol::protocol::RolloutLine;
/// Returned page of conversation summaries.
#[derive(Debug, Default, PartialEq)]
pub struct ConversationsPage {
/// Conversation summaries ordered newest first.
pub items: Vec<ConversationItem>,
/// Opaque pagination token to resume after the last item, or `None` if end.
pub next_cursor: Option<Cursor>,
/// Total number of files touched while scanning this request.
pub num_scanned_files: usize,
/// True if a hard scan cap was hit; consider resuming with `next_cursor`.
pub reached_scan_cap: bool,
}
/// Summary information for a conversation rollout file.
#[derive(Debug, PartialEq)]
pub struct ConversationItem {
/// Absolute path to the rollout file.
pub path: PathBuf,
/// First up to 5 JSONL records parsed as JSON (includes meta line).
pub head: Vec<serde_json::Value>,
pub head: Vec<Value>,
}
/// Hard cap to bound worstcase work per request.
const MAX_SCAN_FILES: usize = 100;
const HEAD_RECORD_LIMIT: usize = 10;
/// Pagination cursor identifying a file by timestamp and UUID.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Cursor {
ts: OffsetDateTime,
@@ -82,10 +75,7 @@ impl<'de> serde::Deserialize<'de> for Cursor {
}
}
/// Retrieve recorded conversation file paths with token pagination. The returned `next_cursor`
/// can be supplied on the next call to resume after the last returned item, resilient to
/// concurrent new sessions being appended. Ordering is stable by timestamp desc, then UUID desc.
pub(crate) async fn get_conversations(
pub async fn get_conversations(
codex_home: &Path,
page_size: usize,
cursor: Option<&Cursor>,
@@ -94,31 +84,57 @@ pub(crate) async fn get_conversations(
root.push(SESSIONS_SUBDIR);
if !root.exists() {
return Ok(ConversationsPage {
items: Vec::new(),
next_cursor: None,
num_scanned_files: 0,
reached_scan_cap: false,
});
return Ok(ConversationsPage::default());
}
let anchor = cursor.cloned();
let result = traverse_directories_for_paths(root.clone(), page_size, anchor).await?;
Ok(result)
traverse_directories_for_paths(root, page_size, anchor).await
}
/// Load the full contents of a single conversation session file at `path`.
/// Returns the entire file contents as a String.
#[allow(dead_code)]
pub(crate) async fn get_conversation(path: &Path) -> io::Result<String> {
tokio::fs::read_to_string(path).await
pub async fn get_conversation(path: &Path) -> io::Result<String> {
fs::read_to_string(path).await
}
pub async fn find_conversation_path_by_id_str(
codex_home: &Path,
id_str: &str,
) -> io::Result<Option<PathBuf>> {
if Uuid::parse_str(id_str).is_err() {
return Ok(None);
}
let mut root = codex_home.to_path_buf();
root.push(SESSIONS_SUBDIR);
if !root.exists() {
return Ok(None);
}
let limit = NonZero::new(1).ok_or_else(|| io::Error::other("search limit must be non-zero"))?;
let threads =
NonZero::new(2).ok_or_else(|| io::Error::other("thread pool size must be non-zero"))?;
let cancel = Arc::new(AtomicBool::new(false));
let exclude: Vec<String> = Vec::new();
let compute_indices = false;
let results = file_search::run(
id_str,
limit,
&root,
exclude,
threads,
cancel,
compute_indices,
)
.map_err(|e| io::Error::other(format!("file search failed: {e}")))?;
Ok(results
.matches
.into_iter()
.next()
.map(|m| root.join(m.path)))
}
/// Load conversation file paths from disk using directory traversal.
///
/// Directory layout: `~/.codex/sessions/YYYY/MM/DD/rollout-YYYY-MM-DDThh-mm-ss-<uuid>.jsonl`
/// Returned newest (latest) first.
async fn traverse_directories_for_paths(
root: PathBuf,
page_size: usize,
@@ -157,8 +173,7 @@ async fn traverse_directories_for_paths(
.map(|(ts, id)| (ts, id, name_str.to_string(), path.to_path_buf()))
})
.await?;
// Stable ordering within the same second: (timestamp desc, uuid desc)
day_files.sort_by_key(|(ts, sid, _name_str, _path)| (Reverse(*ts), Reverse(*sid)));
day_files.sort_by_key(|(ts, sid, _, _)| (Reverse(*ts), Reverse(*sid)));
for (ts, sid, _name_str, path) in day_files.into_iter() {
scanned_files += 1;
if scanned_files >= MAX_SCAN_FILES && items.len() >= page_size {
@@ -174,13 +189,10 @@ async fn traverse_directories_for_paths(
if items.len() == page_size {
break 'outer;
}
// Read head and simultaneously detect message events within the same
// first N JSONL records to avoid a second file read.
let (head, saw_session_meta, saw_user_event) =
read_head_and_flags(&path, HEAD_RECORD_LIMIT)
.await
.unwrap_or((Vec::new(), false, false));
// Apply filters: must have session meta and at least one user message event
if saw_session_meta && saw_user_event {
items.push(ConversationItem { path, head });
}
@@ -198,23 +210,6 @@ async fn traverse_directories_for_paths(
})
}
/// Pagination cursor token format: "<file_ts>|<uuid>" where `file_ts` matches the
/// filename timestamp portion (YYYY-MM-DDThh-mm-ss) used in rollout filenames.
/// The cursor orders files by timestamp desc, then UUID desc.
fn parse_cursor(token: &str) -> Option<Cursor> {
let (file_ts, uuid_str) = token.split_once('|')?;
let Ok(uuid) = Uuid::parse_str(uuid_str) else {
return None;
};
let format: &[FormatItem] =
format_description!("[year]-[month]-[day]T[hour]-[minute]-[second]");
let ts = PrimitiveDateTime::parse(file_ts, format).ok()?.assume_utc();
Some(Cursor::new(ts, uuid))
}
fn build_next_cursor(items: &[ConversationItem]) -> Option<Cursor> {
let last = items.last()?;
let file_name = last.path.file_name()?.to_string_lossy();
@@ -222,14 +217,12 @@ fn build_next_cursor(items: &[ConversationItem]) -> Option<Cursor> {
Some(Cursor::new(ts, id))
}
/// Collects immediate subdirectories of `parent`, parses their (string) names with `parse`,
/// and returns them sorted descending by the parsed key.
async fn collect_dirs_desc<T, F>(parent: &Path, parse: F) -> io::Result<Vec<(T, PathBuf)>>
where
T: Ord + Copy,
F: Fn(&str) -> Option<T>,
{
let mut dir = tokio::fs::read_dir(parent).await?;
let mut dir = fs::read_dir(parent).await?;
let mut vec: Vec<(T, PathBuf)> = Vec::new();
while let Some(entry) = dir.next_entry().await? {
if entry
@@ -247,12 +240,11 @@ where
Ok(vec)
}
/// Collects files in a directory and parses them with `parse`.
async fn collect_files<T, F>(parent: &Path, parse: F) -> io::Result<Vec<T>>
where
F: Fn(&str, &Path) -> Option<T>,
{
let mut dir = tokio::fs::read_dir(parent).await?;
let mut dir = fs::read_dir(parent).await?;
let mut collected: Vec<T> = Vec::new();
while let Some(entry) = dir.next_entry().await? {
if entry
@@ -270,15 +262,11 @@ where
}
fn parse_timestamp_uuid_from_filename(name: &str) -> Option<(OffsetDateTime, Uuid)> {
// Expected: rollout-YYYY-MM-DDThh-mm-ss-<uuid>.jsonl
let core = name.strip_prefix("rollout-")?.strip_suffix(".jsonl")?;
// Scan from the right for a '-' such that the suffix parses as a UUID.
let (sep_idx, uuid) = core
.match_indices('-')
.rev()
.find_map(|(i, _)| Uuid::parse_str(&core[i + 1..]).ok().map(|u| (i, u)))?;
let ts_str = &core[..sep_idx];
let format: &[FormatItem] =
format_description!("[year]-[month]-[day]T[hour]-[minute]-[second]");
@@ -286,16 +274,23 @@ fn parse_timestamp_uuid_from_filename(name: &str) -> Option<(OffsetDateTime, Uui
Some((ts, uuid))
}
fn parse_cursor(token: &str) -> Option<Cursor> {
let (file_ts, uuid_str) = token.split_once('|')?;
let uuid = Uuid::parse_str(uuid_str).ok()?;
let format: &[FormatItem] =
format_description!("[year]-[month]-[day]T[hour]-[minute]-[second]");
let ts = PrimitiveDateTime::parse(file_ts, format).ok()?.assume_utc();
Some(Cursor::new(ts, uuid))
}
async fn read_head_and_flags(
path: &Path,
max_records: usize,
) -> io::Result<(Vec<serde_json::Value>, bool, bool)> {
use tokio::io::AsyncBufReadExt;
) -> io::Result<(Vec<Value>, bool, bool)> {
let file = tokio::fs::File::open(path).await?;
let reader = tokio::io::BufReader::new(file);
let mut lines = reader.lines();
let mut head: Vec<serde_json::Value> = Vec::new();
let mut head: Vec<Value> = Vec::new();
let mut saw_session_meta = false;
let mut saw_user_event = false;
@@ -322,12 +317,7 @@ async fn read_head_and_flags(
head.push(val);
}
}
RolloutItem::TurnContext(_) => {
// Not included in `head`; skip.
}
RolloutItem::Compacted(_) => {
// Not included in `head`; skip.
}
RolloutItem::TurnContext(_) | RolloutItem::Compacted(_) => {}
RolloutItem::EventMsg(ev) => {
if matches!(ev, EventMsg::UserMessage(_)) {
saw_user_event = true;
@@ -338,48 +328,3 @@ async fn read_head_and_flags(
Ok((head, saw_session_meta, saw_user_event))
}
/// Locate a recorded conversation rollout file by its UUID string using the existing
/// paginated listing implementation. Returns `Ok(Some(path))` if found, `Ok(None)` if not present
/// or the id is invalid.
pub async fn find_conversation_path_by_id_str(
codex_home: &Path,
id_str: &str,
) -> io::Result<Option<PathBuf>> {
// Validate UUID format early.
if Uuid::parse_str(id_str).is_err() {
return Ok(None);
}
let mut root = codex_home.to_path_buf();
root.push(SESSIONS_SUBDIR);
if !root.exists() {
return Ok(None);
}
// This is safe because we know the values are valid.
#[allow(clippy::unwrap_used)]
let limit = NonZero::new(1).unwrap();
// This is safe because we know the values are valid.
#[allow(clippy::unwrap_used)]
let threads = NonZero::new(2).unwrap();
let cancel = Arc::new(AtomicBool::new(false));
let exclude: Vec<String> = Vec::new();
let compute_indices = false;
let results = file_search::run(
id_str,
limit,
&root,
exclude,
threads,
cancel,
compute_indices,
)
.map_err(|e| io::Error::other(format!("file search failed: {e}")))?;
Ok(results
.matches
.into_iter()
.next()
.map(|m| root.join(m.path)))
}

View File

@@ -0,0 +1,11 @@
pub const SESSIONS_SUBDIR: &str = "sessions";
pub const ARCHIVED_SESSIONS_SUBDIR: &str = "archived_sessions";
pub mod list;
pub mod policy;
pub mod recorder;
pub use recorder::GitInfoCollector;
pub use recorder::RolloutConfig;
pub use recorder::RolloutRecorder;
pub use recorder::RolloutRecorderParams;

View File

@@ -1,14 +1,13 @@
use crate::protocol::EventMsg;
use crate::protocol::RolloutItem;
use codex_protocol::models::ResponseItem;
use codex_protocol::protocol::EventMsg;
use codex_protocol::protocol::RolloutItem;
/// Whether a rollout `item` should be persisted in rollout files.
#[inline]
pub(crate) fn is_persisted_response_item(item: &RolloutItem) -> bool {
pub fn is_persisted_response_item(item: &RolloutItem) -> bool {
match item {
RolloutItem::ResponseItem(item) => should_persist_response_item(item),
RolloutItem::EventMsg(ev) => should_persist_event_msg(ev),
// Persist Codex executive markers so we can analyze flows (e.g., compaction, API turns).
RolloutItem::Compacted(_) | RolloutItem::TurnContext(_) | RolloutItem::SessionMeta(_) => {
true
}
@@ -17,7 +16,7 @@ pub(crate) fn is_persisted_response_item(item: &RolloutItem) -> bool {
/// Whether a `ResponseItem` should be persisted in rollout files.
#[inline]
pub(crate) fn should_persist_response_item(item: &ResponseItem) -> bool {
pub fn should_persist_response_item(item: &ResponseItem) -> bool {
match item {
ResponseItem::Message { .. }
| ResponseItem::Reasoning { .. }
@@ -33,7 +32,7 @@ pub(crate) fn should_persist_response_item(item: &ResponseItem) -> bool {
/// Whether an `EventMsg` should be persisted in rollout files.
#[inline]
pub(crate) fn should_persist_event_msg(ev: &EventMsg) -> bool {
pub fn should_persist_event_msg(ev: &EventMsg) -> bool {
match ev {
EventMsg::UserMessage(_)
| EventMsg::AgentMessage(_)

View File

@@ -1,21 +1,26 @@
//! Persist Codex session rollouts (.jsonl) so sessions can be replayed or inspected later.
use std::fs;
use std::fs::File;
use std::fs::{self};
use std::io::Error as IoError;
use std::path::Path;
use std::path::PathBuf;
use std::sync::Arc;
use async_trait::async_trait;
use codex_protocol::mcp_protocol::ConversationId;
use serde::Deserialize;
use serde::Serialize;
use codex_protocol::protocol::GitInfo;
use codex_protocol::protocol::InitialHistory;
use codex_protocol::protocol::ResumedHistory;
use codex_protocol::protocol::RolloutItem;
use codex_protocol::protocol::RolloutLine;
use codex_protocol::protocol::SessionMeta;
use codex_protocol::protocol::SessionMetaLine;
use serde_json::Value;
use time::OffsetDateTime;
use time::format_description::FormatItem;
use time::macros::format_description;
use tokio::io::AsyncWriteExt;
use tokio::sync::mpsc;
use tokio::sync::mpsc::Sender;
use tokio::sync::mpsc::{self};
use tokio::sync::oneshot;
use tracing::info;
use tracing::warn;
@@ -25,49 +30,31 @@ use super::list::ConversationsPage;
use super::list::Cursor;
use super::list::get_conversations;
use super::policy::is_persisted_response_item;
use crate::config::Config;
use crate::default_client::ORIGINATOR;
use crate::git_info::collect_git_info;
use codex_protocol::models::ResponseItem;
use codex_protocol::protocol::InitialHistory;
use codex_protocol::protocol::ResumedHistory;
use codex_protocol::protocol::RolloutItem;
use codex_protocol::protocol::RolloutLine;
use codex_protocol::protocol::SessionMeta;
use codex_protocol::protocol::SessionMetaLine;
#[derive(Serialize, Deserialize, Default, Clone)]
pub struct SessionStateSnapshot {}
#[derive(Serialize, Deserialize, Default, Clone)]
pub struct SavedSession {
pub session: SessionMeta,
#[serde(default)]
pub items: Vec<ResponseItem>,
#[serde(default)]
pub state: SessionStateSnapshot,
pub session_id: ConversationId,
#[async_trait]
pub trait GitInfoCollector: Send + Sync {
async fn collect(&self, cwd: &Path) -> Option<GitInfo>;
}
#[derive(Clone)]
pub struct RolloutConfig {
pub codex_home: PathBuf,
pub originator: String,
pub cli_version: String,
pub git_info_collector: Option<Arc<dyn GitInfoCollector>>,
}
/// Records all [`ResponseItem`]s for a session and flushes them to disk after
/// every update.
///
/// Rollouts are recorded as JSONL and can be inspected with tools such as:
///
/// ```ignore
/// $ jq -C . ~/.codex/sessions/rollout-2025-05-07T17-24-21-5973b6c0-94b8-487b-a530-2aeb6098ae0e.jsonl
/// $ fx ~/.codex/sessions/rollout-2025-05-07T17-24-21-5973b6c0-94b8-487b-a530-2aeb6098ae0e.jsonl
/// ```
#[derive(Clone)]
pub struct RolloutRecorder {
tx: Sender<RolloutCmd>,
pub(crate) rollout_path: PathBuf,
rollout_path: PathBuf,
}
#[derive(Clone)]
pub enum RolloutRecorderParams {
Create {
conversation_id: ConversationId,
cwd: PathBuf,
instructions: Option<String>,
},
Resume {
@@ -77,19 +64,19 @@ pub enum RolloutRecorderParams {
enum RolloutCmd {
AddItems(Vec<RolloutItem>),
/// Ensure all prior writes are processed; respond when flushed.
Flush {
ack: oneshot::Sender<()>,
},
Shutdown {
ack: oneshot::Sender<()>,
},
Flush { ack: oneshot::Sender<()> },
Shutdown { ack: oneshot::Sender<()> },
}
impl RolloutRecorderParams {
pub fn new(conversation_id: ConversationId, instructions: Option<String>) -> Self {
pub fn new(
conversation_id: ConversationId,
cwd: PathBuf,
instructions: Option<String>,
) -> Self {
Self::Create {
conversation_id,
cwd,
instructions,
}
}
@@ -100,7 +87,6 @@ impl RolloutRecorderParams {
}
impl RolloutRecorder {
/// List conversations (rollout files) under the provided Codex home directory.
pub async fn list_conversations(
codex_home: &Path,
page_size: usize,
@@ -109,13 +95,14 @@ impl RolloutRecorder {
get_conversations(codex_home, page_size, cursor).await
}
/// Attempt to create a new [`RolloutRecorder`]. If the sessions directory
/// cannot be created or the rollout file cannot be opened we return the
/// error so the caller can decide whether to disable persistence.
pub async fn new(config: &Config, params: RolloutRecorderParams) -> std::io::Result<Self> {
let (file, rollout_path, meta) = match params {
pub async fn new(
config: &RolloutConfig,
params: RolloutRecorderParams,
) -> std::io::Result<Self> {
let (file, rollout_path, meta, cwd) = match params {
RolloutRecorderParams::Create {
conversation_id,
cwd,
instructions,
} => {
let LogFileInfo {
@@ -123,7 +110,7 @@ impl RolloutRecorder {
path,
conversation_id: session_id,
timestamp,
} = create_log_file(config, conversation_id)?;
} = create_log_file(&config.codex_home, conversation_id)?;
let timestamp_format: &[FormatItem] = format_description!(
"[year]-[month]-[day]T[hour]:[minute]:[second].[subsecond digits:3]Z"
@@ -133,18 +120,16 @@ impl RolloutRecorder {
.format(timestamp_format)
.map_err(|e| IoError::other(format!("failed to format timestamp: {e}")))?;
(
tokio::fs::File::from_std(file),
path,
Some(SessionMeta {
id: session_id,
timestamp,
cwd: config.cwd.clone(),
originator: ORIGINATOR.value.clone(),
cli_version: env!("CARGO_PKG_VERSION").to_string(),
instructions,
}),
)
let meta = SessionMeta {
id: session_id,
timestamp,
cwd: cwd.clone(),
originator: config.originator.clone(),
cli_version: config.cli_version.clone(),
instructions,
};
(tokio::fs::File::from_std(file), path, Some(meta), Some(cwd))
}
RolloutRecorderParams::Resume { path } => (
tokio::fs::OpenOptions::new()
@@ -153,31 +138,21 @@ impl RolloutRecorder {
.await?,
path,
None,
None,
),
};
// Clone the cwd for the spawned task to collect git info asynchronously
let cwd = config.cwd.clone();
// A reasonably-sized bounded channel. If the buffer fills up the send
// future will yield, which is fine we only need to ensure we do not
// perform *blocking* I/O on the caller's thread.
let (tx, rx) = mpsc::channel::<RolloutCmd>(256);
let collector = config.git_info_collector.clone();
// Spawn a Tokio task that owns the file handle and performs async
// writes. Using `tokio::fs::File` keeps everything on the async I/O
// driver instead of blocking the runtime.
tokio::task::spawn(rollout_writer(file, rx, meta, cwd));
tokio::task::spawn(rollout_writer(file, rx, meta, cwd, collector));
Ok(Self { tx, rollout_path })
}
pub(crate) async fn record_items(&self, items: &[RolloutItem]) -> std::io::Result<()> {
pub async fn record_items(&self, items: &[RolloutItem]) -> std::io::Result<()> {
let mut filtered = Vec::new();
for item in items {
// Note that function calls may look a bit strange if they are
// "fully qualified MCP tool calls," so we could consider
// reformatting them in that case.
if is_persisted_response_item(item) {
filtered.push(item.clone());
}
@@ -191,7 +166,6 @@ impl RolloutRecorder {
.map_err(|e| IoError::other(format!("failed to queue rollout items: {e}")))
}
/// Flush all queued writes and wait until they are committed by the writer task.
pub async fn flush(&self) -> std::io::Result<()> {
let (tx, rx) = oneshot::channel();
self.tx
@@ -202,7 +176,26 @@ impl RolloutRecorder {
.map_err(|e| IoError::other(format!("failed waiting for rollout flush: {e}")))
}
pub(crate) async fn get_rollout_history(path: &Path) -> std::io::Result<InitialHistory> {
pub async fn shutdown(&self) -> std::io::Result<()> {
let (tx_done, rx_done) = oneshot::channel();
match self.tx.send(RolloutCmd::Shutdown { ack: tx_done }).await {
Ok(_) => rx_done
.await
.map_err(|e| IoError::other(format!("failed waiting for rollout shutdown: {e}"))),
Err(e) => {
warn!("failed to send rollout shutdown command: {e}");
Err(IoError::other(format!(
"failed to send rollout shutdown command: {e}"
)))
}
}
}
pub fn get_rollout_path(&self) -> PathBuf {
self.rollout_path.clone()
}
pub async fn get_rollout_history(path: &Path) -> std::io::Result<InitialHistory> {
info!("Resuming rollout from {path:?}");
let text = tokio::fs::read_to_string(path).await?;
if text.trim().is_empty() {
@@ -223,33 +216,17 @@ impl RolloutRecorder {
}
};
// Parse the rollout line structure
match serde_json::from_value::<RolloutLine>(v.clone()) {
Ok(rollout_line) => match rollout_line.item {
RolloutItem::SessionMeta(session_meta_line) => {
// Use the FIRST SessionMeta encountered in the file as the canonical
// conversation id and main session information. Keep all items intact.
if conversation_id.is_none() {
conversation_id = Some(session_meta_line.meta.id);
}
items.push(RolloutItem::SessionMeta(session_meta_line));
}
RolloutItem::ResponseItem(item) => {
items.push(RolloutItem::ResponseItem(item));
}
RolloutItem::Compacted(item) => {
items.push(RolloutItem::Compacted(item));
}
RolloutItem::TurnContext(item) => {
items.push(RolloutItem::TurnContext(item));
}
RolloutItem::EventMsg(_ev) => {
items.push(RolloutItem::EventMsg(_ev));
}
other => items.push(other),
},
Err(e) => {
warn!("failed to parse rollout line: {v:?}, error: {e}");
}
Err(e) => warn!("failed to parse rollout line: {v:?}, error: {e}"),
}
}
@@ -272,57 +249,28 @@ impl RolloutRecorder {
rollout_path: path.to_path_buf(),
}))
}
pub(crate) fn get_rollout_path(&self) -> PathBuf {
self.rollout_path.clone()
}
pub async fn shutdown(&self) -> std::io::Result<()> {
let (tx_done, rx_done) = oneshot::channel();
match self.tx.send(RolloutCmd::Shutdown { ack: tx_done }).await {
Ok(_) => rx_done
.await
.map_err(|e| IoError::other(format!("failed waiting for rollout shutdown: {e}"))),
Err(e) => {
warn!("failed to send rollout shutdown command: {e}");
Err(IoError::other(format!(
"failed to send rollout shutdown command: {e}"
)))
}
}
}
}
struct LogFileInfo {
/// Opened file handle to the rollout file.
file: File,
/// Full path to the rollout file.
path: PathBuf,
/// Session ID (also embedded in filename).
conversation_id: ConversationId,
/// Timestamp for the start of the session.
timestamp: OffsetDateTime,
}
fn create_log_file(
config: &Config,
codex_home: &Path,
conversation_id: ConversationId,
) -> std::io::Result<LogFileInfo> {
// Resolve ~/.codex/sessions/YYYY/MM/DD and create it if missing.
let timestamp = OffsetDateTime::now_local()
.map_err(|e| IoError::other(format!("failed to get local time: {e}")))?;
let mut dir = config.codex_home.clone();
let mut dir = codex_home.to_path_buf();
dir.push(SESSIONS_SUBDIR);
dir.push(timestamp.year().to_string());
dir.push(format!("{:02}", u8::from(timestamp.month())));
dir.push(format!("{:02}", timestamp.day()));
fs::create_dir_all(&dir)?;
// Custom format for YYYY-MM-DDThh-mm-ss. Use `-` instead of `:` for
// compatibility with filesystems that do not allow colons in filenames.
let format: &[FormatItem] =
format_description!("[year]-[month]-[day]T[hour]-[minute]-[second]");
let date_str = timestamp
@@ -330,7 +278,6 @@ fn create_log_file(
.map_err(|e| IoError::other(format!("failed to format timestamp: {e}")))?;
let filename = format!("rollout-{date_str}-{conversation_id}.jsonl");
let path = dir.join(filename);
let file = std::fs::OpenOptions::new()
.append(true)
@@ -349,25 +296,27 @@ async fn rollout_writer(
file: tokio::fs::File,
mut rx: mpsc::Receiver<RolloutCmd>,
mut meta: Option<SessionMeta>,
cwd: std::path::PathBuf,
cwd: Option<PathBuf>,
git_info_collector: Option<Arc<dyn GitInfoCollector>>,
) -> std::io::Result<()> {
let mut writer = JsonlWriter { file };
// If we have a meta, collect git info asynchronously and write meta first
if let Some(session_meta) = meta.take() {
let git_info = collect_git_info(&cwd).await;
let git_info =
if let (Some(provider), Some(cwd)) = (git_info_collector.as_ref(), cwd.as_ref()) {
provider.collect(cwd.as_path()).await
} else {
None
};
let session_meta_line = SessionMetaLine {
meta: session_meta,
git: git_info,
};
// Write the SessionMeta as the first item in the file, wrapped in a rollout line
writer
.write_rollout_item(RolloutItem::SessionMeta(session_meta_line))
.await?;
}
// Process rollout commands
while let Some(cmd) = rx.recv().await {
match cmd {
RolloutCmd::AddItems(items) => {
@@ -378,7 +327,6 @@ async fn rollout_writer(
}
}
RolloutCmd::Flush { ack } => {
// Ensure underlying file is flushed and then ack.
if let Err(e) = writer.file.flush().await {
let _ = ack.send(());
return Err(e);
@@ -413,11 +361,14 @@ impl JsonlWriter {
};
self.write_line(&line).await
}
async fn write_line(&mut self, item: &impl serde::Serialize) -> std::io::Result<()> {
let mut json = serde_json::to_string(item)?;
json.push('\n');
self.file.write_all(json.as_bytes()).await?;
self.file.flush().await?;
Ok(())
let mut buf = serde_json::to_vec(item)
.map_err(|e| IoError::other(format!("failed to serialise rollout line: {e}")))?;
buf.push(b'\n');
self.file
.write_all(&buf)
.await
.map_err(|e| IoError::other(format!("failed to write rollout line: {e}")))
}
}

View File

@@ -0,0 +1,21 @@
use async_trait::async_trait;
use codex_protocol::protocol::Event;
use codex_protocol::protocol::Op;
use codex_protocol::protocol::Submission;
pub mod turn;
pub use turn::ProcessedResponseItem;
pub use turn::TurnRunResult;
/// Minimal async interface for interacting with an agent runtime.
#[async_trait]
pub trait AgentRuntime: Send + Sync {
type Error: std::error::Error + Send + Sync + 'static;
async fn submit(&self, op: Op) -> Result<String, Self::Error>;
async fn submit_with_id(&self, submission: Submission) -> Result<(), Self::Error>;
async fn next_event(&self) -> Result<Event, Self::Error>;
}

View File

@@ -0,0 +1,15 @@
use codex_protocol::models::ResponseInputItem;
use codex_protocol::models::ResponseItem;
use codex_protocol::protocol::TokenUsage;
#[derive(Debug, Clone)]
pub struct ProcessedResponseItem {
pub item: ResponseItem,
pub response: Option<ResponseInputItem>,
}
#[derive(Debug, Clone)]
pub struct TurnRunResult {
pub processed_items: Vec<ProcessedResponseItem>,
pub total_token_usage: Option<TokenUsage>,
}

View File

@@ -0,0 +1,46 @@
use std::collections::HashMap;
use std::path::PathBuf;
use crate::config_types::History;
use crate::config_types::McpServerConfig;
use crate::config_types::ShellEnvironmentPolicy;
use crate::model_family::ModelFamily;
use crate::model_provider::ModelProviderInfo;
use codex_protocol::config_types::ReasoningEffort;
use codex_protocol::config_types::ReasoningSummary;
use codex_protocol::config_types::Verbosity;
use codex_protocol::protocol::AskForApproval;
use codex_protocol::protocol::SandboxPolicy;
/// Configuration surface consumed by the agent runtime regardless of host.
#[derive(Debug, Clone, PartialEq)]
pub struct AgentConfig {
pub model: String,
pub review_model: String,
pub model_family: ModelFamily,
pub model_context_window: Option<u64>,
pub model_auto_compact_token_limit: Option<i64>,
pub model_reasoning_effort: Option<ReasoningEffort>,
pub model_reasoning_summary: ReasoningSummary,
pub model_verbosity: Option<Verbosity>,
pub model_provider: ModelProviderInfo,
pub approval_policy: AskForApproval,
pub sandbox_policy: SandboxPolicy,
pub shell_environment_policy: ShellEnvironmentPolicy,
pub user_instructions: Option<String>,
pub base_instructions: Option<String>,
pub notify: Option<Vec<String>>,
pub cwd: PathBuf,
pub codex_home: PathBuf,
pub history: History,
pub mcp_servers: HashMap<String, McpServerConfig>,
pub include_plan_tool: bool,
pub include_apply_patch_tool: bool,
pub include_view_image_tool: bool,
pub tools_web_search_request: bool,
pub use_experimental_streamable_shell_tool: bool,
pub use_experimental_unified_exec_tool: bool,
pub show_raw_agent_reasoning: bool,
pub codex_linux_sandbox_exe: Option<PathBuf>,
pub project_doc_max_bytes: usize,
}

View File

@@ -1,15 +1,14 @@
use std::collections::HashSet;
use std::path::Component;
use std::path::Path;
use std::path::PathBuf;
use codex_apply_patch::ApplyPatchAction;
use codex_apply_patch::ApplyPatchFileChange;
use codex_protocol::protocol::AskForApproval;
use codex_protocol::protocol::SandboxPolicy;
use crate::exec::SandboxType;
use crate::is_safe_command::is_known_safe_command;
use crate::protocol::AskForApproval;
use crate::protocol::SandboxPolicy;
use crate::command_safety::is_dangerous_command::command_might_be_dangerous;
use crate::command_safety::is_safe_command::is_known_safe_command;
use crate::sandbox::SandboxType;
#[derive(Debug, PartialEq)]
pub enum SafetyCheck {
@@ -85,6 +84,13 @@ pub fn assess_command_safety(
approved: &HashSet<Vec<String>>,
with_escalated_permissions: bool,
) -> SafetyCheck {
// Some commands look dangerous. Even if they are run inside a sandbox,
// unless the user has explicitly approved them, we should ask,
// regardless of the approval policy and sandbox policy.
if command_might_be_dangerous(command) && !approved.contains(command) {
return SafetyCheck::AskUser;
}
// A command is "trusted" because either:
// - it belongs to a set of commands we consider "safe" by default, or
// - the user has explicitly approved the command for this session
@@ -98,6 +104,7 @@ pub fn assess_command_safety(
// would probably be fine to run the command in a sandbox, but when
// `approved.contains(command)` is `true`, the user may have approved it for
// the session _because_ they know it needs to run outside a sandbox.
if is_known_safe_command(command) || approved.contains(command) {
return SafetyCheck::AutoApprove {
sandbox_type: SandboxType::None,
@@ -189,81 +196,196 @@ fn is_write_patch_constrained_to_writable_paths(
SandboxPolicy::DangerFullAccess => {
return true;
}
SandboxPolicy::WorkspaceWrite { .. } => sandbox_policy.get_writable_roots_with_cwd(cwd),
SandboxPolicy::WorkspaceWrite {
writable_roots,
exclude_slash_tmp: _exclude_slash_tmp,
exclude_tmpdir_env_var: _exclude_tmpdir,
network_access: _network_access,
} => writable_roots,
};
// Normalize a path by removing `.` and resolving `..` without touching the
// filesystem (works even if the file does not exist).
fn normalize(path: &Path) -> Option<PathBuf> {
let mut out = PathBuf::new();
for comp in path.components() {
match comp {
Component::ParentDir => {
out.pop();
}
Component::CurDir => { /* skip */ }
other => out.push(other.as_os_str()),
}
}
Some(out)
// If the policy allows writes outside the workspace (DangerFullAccess),
// we've already returned true above. At this point we only have
// `WorkspaceWrite`, which includes the cwd implicitly, so first check if
// the patch fully lives within the cwd. If it does then we're fine.
let workspace_root = cwd.canonicalize().unwrap_or_else(|_| cwd.to_path_buf());
if all_changes_within_root(action, &workspace_root) {
return true;
}
// Determine whether `path` is inside **any** writable root. Both `path`
// and roots are converted to absolute, normalized forms before the
// prefix check.
let is_path_writable = |p: &PathBuf| {
let abs = if p.is_absolute() {
p.clone()
} else {
cwd.join(p)
};
let abs = match normalize(&abs) {
Some(v) => v,
None => return false,
};
if writable_roots.is_empty() {
return false;
}
writable_roots
.iter()
.any(|writable_root| writable_root.is_path_writable(&abs))
};
// When `/tmp` is excluded, filter it out of writable roots. Some patch commands write
// temporary files there even for workspace-only updates.
let mut writable_roots: Vec<&PathBuf> = writable_roots.iter().collect();
if matches!(
sandbox_policy,
SandboxPolicy::WorkspaceWrite {
exclude_slash_tmp: true,
..
}
) {
writable_roots.retain(|path| !path.as_path().starts_with("/tmp"));
}
for (path, change) in action.changes() {
match change {
ApplyPatchFileChange::Add { .. } | ApplyPatchFileChange::Delete { .. } => {
if !is_path_writable(path) {
return false;
let mut all_within_declared_root = true;
for change in action.changes() {
match change.0.strip_prefix(&workspace_root) {
Ok(relative_path) => {
if !is_within_any_root(relative_path, &writable_roots) {
all_within_declared_root = false;
break;
}
}
ApplyPatchFileChange::Update { move_path, .. } => {
if !is_path_writable(path) {
return false;
}
if let Some(dest) = move_path
&& !is_path_writable(dest)
{
return false;
}
Err(_) => {
all_within_declared_root = false;
break;
}
}
}
true
all_within_declared_root
}
#[cfg(test)]
fn all_changes_within_root(action: &ApplyPatchAction, root: &Path) -> bool {
action
.changes()
.iter()
.all(|(path, _)| path.starts_with(root))
}
fn is_within_any_root(path: &Path, roots: &[&PathBuf]) -> bool {
roots.iter().any(|root| path.starts_with(root.as_path()))
}
#[cfg(any())]
mod tests {
use super::*;
use tempfile::TempDir;
#[test]
fn test_writable_roots_constraint() {
// Use a temporary directory as our workspace to avoid touching
// the real current working directory.
let tmp = TempDir::new().unwrap();
let cwd = tmp.path().to_path_buf();
fn reject_empty_patch() {
let action = ApplyPatchAction::new_for_test(vec![]);
let sandbox_policy = SandboxPolicy::ReadOnly;
let cwd = Path::new(".");
assert_eq!(
assess_patch_safety(&action, AskForApproval::OnRequest, &sandbox_policy, cwd),
SafetyCheck::Reject {
reason: "empty patch".to_string(),
}
);
}
#[test]
fn auto_allow_patch_in_workspace_write_sandbox() {
let patch_action = ApplyPatchAction::new_for_test(vec![ApplyPatchFileChange::new_update(
PathBuf::from("src/main.rs"),
"diff --git a/src/main.rs b/src/main.rs\n".to_string(),
None,
"".to_string(),
)]);
let sandbox_policy = SandboxPolicy::WorkspaceWrite {
writable_roots: vec![],
network_access: false,
exclude_tmpdir_env_var: false,
exclude_slash_tmp: false,
};
assert_eq!(
assess_patch_safety(
&patch_action,
AskForApproval::OnRequest,
&sandbox_policy,
Path::new("."),
),
SafetyCheck::AutoApprove {
sandbox_type: get_platform_sandbox().unwrap_or(SandboxType::None),
}
);
}
#[test]
fn reject_patch_if_policy_is_never_and_writes_outside_of_workspace() {
let patch_action = ApplyPatchAction::new_for_test(vec![ApplyPatchFileChange::new_update(
PathBuf::from("../outside_file.txt"),
"diff --git a/../outside_file.txt b/../outside_file.txt\n".to_string(),
None,
"".to_string(),
)]);
let sandbox_policy = SandboxPolicy::WorkspaceWrite {
writable_roots: vec![],
network_access: false,
exclude_tmpdir_env_var: false,
exclude_slash_tmp: false,
};
assert_eq!(
assess_patch_safety(
&patch_action,
AskForApproval::Never,
&sandbox_policy,
Path::new("."),
),
SafetyCheck::Reject {
reason: "writing outside of the project; rejected by user approval settings"
.to_string(),
}
);
}
#[test]
fn assess_command_safety_known_safe_command() {
let command = vec!["ls".to_string()];
let approval_policy = AskForApproval::OnRequest;
let sandbox_policy = SandboxPolicy::ReadOnly;
let approved = HashSet::new();
let request_escalated_privileges = false;
let safety_check = assess_command_safety(
&command,
approval_policy,
&sandbox_policy,
&approved,
request_escalated_privileges,
);
assert_eq!(
safety_check,
SafetyCheck::AutoApprove {
sandbox_type: SandboxType::None
}
);
}
#[test]
fn assess_command_safety_dangerous_command_to_reject() {
let command = vec!["rm".to_string(), "-rf".to_string(), "/".to_string()];
let approval_policy = AskForApproval::OnRequest;
let sandbox_policy = SandboxPolicy::ReadOnly;
let approved = HashSet::new();
let request_escalated_privileges = false;
let safety_check = assess_command_safety(
&command,
approval_policy,
&sandbox_policy,
&approved,
request_escalated_privileges,
);
assert_eq!(safety_check, SafetyCheck::AskUser);
}
#[test]
fn patch_within_declared_root() {
let tempdir = tempfile::tempdir().unwrap();
let cwd = tempdir.path().to_path_buf();
let parent = cwd.parent().unwrap().to_path_buf();
// Helper to build a singleentry patch that adds a file at `p`.
let make_add_change = |p: PathBuf| ApplyPatchAction::new_add_for_test(&p, "".to_string());
let add_inside = make_add_change(cwd.join("inner.txt"));
@@ -325,6 +447,50 @@ mod tests {
assert_eq!(safety_check, SafetyCheck::AskUser);
}
#[test]
fn dangerous_command_allowed_if_explicitly_approved() {
let command = vec!["git".to_string(), "reset".to_string(), "--hard".to_string()];
let approval_policy = AskForApproval::OnRequest;
let sandbox_policy = SandboxPolicy::ReadOnly;
let mut approved: HashSet<Vec<String>> = HashSet::new();
approved.insert(command.clone());
let request_escalated_privileges = false;
let safety_check = assess_command_safety(
&command,
approval_policy,
&sandbox_policy,
&approved,
request_escalated_privileges,
);
assert_eq!(
safety_check,
SafetyCheck::AutoApprove {
sandbox_type: SandboxType::None
}
);
}
#[test]
fn dangerous_command_not_allowed_if_not_explicitly_approved() {
let command = vec!["git".to_string(), "reset".to_string(), "--hard".to_string()];
let approval_policy = AskForApproval::Never;
let sandbox_policy = SandboxPolicy::ReadOnly;
let approved: HashSet<Vec<String>> = HashSet::new();
let request_escalated_privileges = false;
let safety_check = assess_command_safety(
&command,
approval_policy,
&sandbox_policy,
&approved,
request_escalated_privileges,
);
assert_eq!(safety_check, SafetyCheck::AskUser);
}
#[test]
fn test_request_escalated_privileges_no_sandbox_fallback() {
let command = vec!["git".to_string(), "commit".to_string()];

View File

@@ -0,0 +1,10 @@
pub mod planner;
pub mod types;
pub use planner::CommandPlanRequest;
pub use planner::ExecPlan;
pub use planner::PatchPlanRequest;
pub use planner::plan_apply_patch;
pub use planner::plan_exec;
pub use planner::should_escalate_on_failure;
pub use types::SandboxType;

View File

@@ -0,0 +1,106 @@
use std::collections::HashSet;
use std::path::Path;
use codex_apply_patch::ApplyPatchAction;
use codex_protocol::protocol::AskForApproval;
use codex_protocol::protocol::SandboxPolicy;
use crate::safety::SafetyCheck;
use crate::safety::assess_command_safety;
use crate::safety::assess_patch_safety;
use super::SandboxType;
#[derive(Clone, Debug)]
pub enum ExecPlan {
Reject {
reason: String,
},
AskUser {
reason: Option<String>,
},
Approved {
sandbox: SandboxType,
on_failure_escalate: bool,
approved_by_user: bool,
},
}
impl ExecPlan {
pub fn approved(
sandbox: SandboxType,
on_failure_escalate: bool,
approved_by_user: bool,
) -> Self {
ExecPlan::Approved {
sandbox,
on_failure_escalate,
approved_by_user,
}
}
}
pub struct CommandPlanRequest<'a> {
pub command: &'a [String],
pub approval: AskForApproval,
pub policy: &'a SandboxPolicy,
pub approved_session_commands: &'a HashSet<Vec<String>>,
pub with_escalated_permissions: bool,
pub justification: Option<&'a String>,
}
pub struct PatchPlanRequest<'a> {
pub action: &'a ApplyPatchAction,
pub approval: AskForApproval,
pub policy: &'a SandboxPolicy,
pub cwd: &'a Path,
pub user_explicitly_approved: bool,
}
pub fn plan_exec(req: &CommandPlanRequest<'_>) -> ExecPlan {
let safety = assess_command_safety(
req.command,
req.approval,
req.policy,
req.approved_session_commands,
req.with_escalated_permissions,
);
match safety {
SafetyCheck::AutoApprove { sandbox_type } => ExecPlan::approved(
sandbox_type,
should_escalate_on_failure(req.approval, sandbox_type),
false,
),
SafetyCheck::AskUser => ExecPlan::AskUser {
reason: req.justification.map(ToOwned::to_owned),
},
SafetyCheck::Reject { reason } => ExecPlan::Reject { reason },
}
}
pub fn plan_apply_patch(req: &PatchPlanRequest<'_>) -> ExecPlan {
if req.user_explicitly_approved {
return ExecPlan::approved(SandboxType::None, false, true);
}
match assess_patch_safety(req.action, req.approval, req.policy, req.cwd) {
SafetyCheck::AutoApprove { sandbox_type } => ExecPlan::approved(
sandbox_type,
should_escalate_on_failure(req.approval, sandbox_type),
false,
),
SafetyCheck::AskUser => ExecPlan::AskUser { reason: None },
SafetyCheck::Reject { reason } => ExecPlan::Reject { reason },
}
}
pub fn should_escalate_on_failure(approval: AskForApproval, sandbox: SandboxType) -> bool {
matches!(
(approval, sandbox),
(
AskForApproval::UnlessTrusted | AskForApproval::OnFailure,
SandboxType::MacosSeatbelt | SandboxType::LinuxSeccomp
)
)
}

View File

@@ -0,0 +1,10 @@
#[derive(Clone, Copy, Debug, PartialEq)]
pub enum SandboxType {
None,
/// Only available on macOS.
MacosSeatbelt,
/// Only available on Linux.
LinuxSeccomp,
}

View File

@@ -0,0 +1,138 @@
use std::collections::HashMap;
use std::path::PathBuf;
use async_trait::async_trait;
use codex_apply_patch::ApplyPatchAction;
use codex_protocol::mcp_protocol::AuthMode;
use codex_protocol::protocol::ReviewDecision;
use codex_protocol::protocol::RolloutItem;
use mcp_types::Tool;
use serde_json::Value;
use crate::exec_command::ExecCommandOutput;
use crate::exec_command::ExecCommandParams;
use crate::exec_command::WriteStdinParams;
use crate::notifications::UserNotification;
use crate::rollout::RolloutRecorder;
use crate::token_data::PlanType;
use crate::unified_exec::UnifiedExecError;
use crate::unified_exec::UnifiedExecRequest;
use crate::unified_exec::UnifiedExecResult;
/// Authentication context made available to the provider layer.
#[async_trait]
pub trait ProviderAuth: Send + Sync {
fn mode(&self) -> AuthMode;
async fn access_token(&self) -> std::io::Result<String>;
fn account_id(&self) -> Option<String>;
fn plan_type(&self) -> Option<PlanType>;
}
/// Provides access to credentials required when talking to model providers.
#[async_trait]
pub trait CredentialsProvider: Send + Sync {
fn auth(&self) -> Option<std::sync::Arc<dyn ProviderAuth>>;
async fn refresh_token(&self) -> std::io::Result<Option<String>>;
}
/// Emits user-facing notifications for turn completion or other events.
pub trait Notifier: Send + Sync {
fn notify(&self, notification: &UserNotification);
}
/// Runtime callbacks for user approval workflows.
#[async_trait]
pub trait ApprovalCoordinator: Send + Sync {
async fn request_patch_approval(
&self,
sub_id: String,
call_id: String,
action: &ApplyPatchAction,
reason: Option<String>,
grant_root: Option<PathBuf>,
) -> ReviewDecision;
async fn request_command_approval(
&self,
sub_id: String,
call_id: String,
command: Vec<String>,
cwd: PathBuf,
reason: Option<String>,
) -> ReviewDecision;
async fn add_approved_command(&self, command: Vec<String>);
}
/// Aggregates and dispatches MCP tool calls across configured servers.
#[async_trait]
pub trait McpInterface: Send + Sync {
fn list_all_tools(&self) -> HashMap<String, Tool>;
fn parse_tool_name(&self, tool_name: &str) -> Option<(String, String)>;
async fn call_tool(
&self,
server: &str,
tool: &str,
arguments: Option<Value>,
) -> anyhow::Result<mcp_types::CallToolResult>;
}
/// Persists rollout events for later inspection or replay.
#[async_trait]
pub trait RolloutSink: Send + Sync {
async fn record_items(&self, items: &[RolloutItem]) -> std::io::Result<()>;
async fn flush(&self) -> std::io::Result<()>;
async fn shutdown(&self) -> std::io::Result<()>;
fn get_rollout_path(&self) -> PathBuf;
}
#[async_trait]
impl RolloutSink for RolloutRecorder {
async fn record_items(&self, items: &[RolloutItem]) -> std::io::Result<()> {
RolloutRecorder::record_items(self, items).await
}
async fn flush(&self) -> std::io::Result<()> {
RolloutRecorder::flush(self).await
}
async fn shutdown(&self) -> std::io::Result<()> {
RolloutRecorder::shutdown(self).await
}
fn get_rollout_path(&self) -> PathBuf {
RolloutRecorder::get_rollout_path(self)
}
}
/// Handles sandboxed exec orchestration, including long-running sessions.
#[async_trait]
pub trait SandboxManager: Send + Sync {
async fn handle_exec_command_request(
&self,
params: ExecCommandParams,
) -> Result<ExecCommandOutput, String>;
async fn handle_write_stdin_request(
&self,
params: WriteStdinParams,
) -> Result<ExecCommandOutput, String>;
async fn handle_unified_exec_request(
&self,
request: UnifiedExecRequest<'_>,
) -> Result<UnifiedExecResult, UnifiedExecError>;
fn codex_linux_sandbox_exe(&self) -> &Option<PathBuf>;
fn user_shell(&self) -> &crate::shell::Shell;
}

View File

@@ -0,0 +1,18 @@
use std::sync::Arc;
use tokio::sync::Mutex;
use crate::services::McpInterface;
use crate::services::Notifier;
use crate::services::RolloutSink;
use crate::services::SandboxManager;
/// Aggregated services that back a running agent session. Hosts provide
/// implementations for these traits and hand them to the runtime at spawn.
pub struct SessionServices {
pub mcp: Arc<dyn McpInterface>,
pub notifier: Arc<dyn Notifier>,
pub sandbox: Arc<dyn SandboxManager>,
pub rollout: Mutex<Option<Arc<dyn RolloutSink>>>,
pub show_raw_agent_reasoning: bool,
}

View File

@@ -0,0 +1,76 @@
use std::collections::HashSet;
use codex_protocol::models::ResponseItem;
use codex_protocol::protocol::RateLimitSnapshot;
use codex_protocol::protocol::TokenUsage;
use codex_protocol::protocol::TokenUsageInfo;
use crate::conversation_history::ConversationHistory;
/// Persistent, session-scoped state previously stored directly on `Session`.
#[derive(Default)]
pub struct SessionState {
approved_commands: HashSet<Vec<String>>,
history: ConversationHistory,
token_info: Option<TokenUsageInfo>,
latest_rate_limits: Option<RateLimitSnapshot>,
}
impl SessionState {
/// Create a new session state mirroring previous `State::default()` semantics.
pub fn new() -> Self {
Self {
history: ConversationHistory::new(),
..Default::default()
}
}
// History helpers
pub fn record_items<I>(&mut self, items: I)
where
I: IntoIterator,
I::Item: std::ops::Deref<Target = ResponseItem>,
{
self.history.record_items(items)
}
pub fn history_snapshot(&self) -> Vec<ResponseItem> {
self.history.contents()
}
pub fn replace_history(&mut self, items: Vec<ResponseItem>) {
self.history.replace(items);
}
// Approved command helpers
pub fn add_approved_command(&mut self, cmd: Vec<String>) {
self.approved_commands.insert(cmd);
}
pub fn approved_commands_ref(&self) -> &HashSet<Vec<String>> {
&self.approved_commands
}
// Token/rate limit helpers
pub fn update_token_info_from_usage(
&mut self,
usage: &TokenUsage,
model_context_window: Option<u64>,
) {
self.token_info = TokenUsageInfo::new_or_append(
&self.token_info,
&Some(usage.clone()),
model_context_window,
);
}
pub fn set_rate_limits(&mut self, snapshot: RateLimitSnapshot) {
self.latest_rate_limits = Some(snapshot);
}
pub fn token_info_and_rate_limits(
&self,
) -> (Option<TokenUsageInfo>, Option<RateLimitSnapshot>) {
(self.token_info.clone(), self.latest_rate_limits.clone())
}
}

271
codex-rs/agent/src/shell.rs Normal file
View File

@@ -0,0 +1,271 @@
use serde::Deserialize;
use serde::Serialize;
use shlex;
use std::path::PathBuf;
#[derive(Debug, PartialEq, Eq, Clone, Serialize, Deserialize)]
pub struct ZshShell {
pub(crate) shell_path: String,
pub(crate) zshrc_path: String,
}
impl ZshShell {
pub fn new(shell_path: impl Into<String>, zshrc_path: impl Into<String>) -> Self {
Self {
shell_path: shell_path.into(),
zshrc_path: zshrc_path.into(),
}
}
pub fn shell_path(&self) -> &str {
&self.shell_path
}
pub fn zshrc_path(&self) -> &str {
&self.zshrc_path
}
}
#[derive(Debug, PartialEq, Eq, Clone, Serialize, Deserialize)]
pub struct BashShell {
pub(crate) shell_path: String,
pub(crate) bashrc_path: String,
}
impl BashShell {
pub fn new(shell_path: impl Into<String>, bashrc_path: impl Into<String>) -> Self {
Self {
shell_path: shell_path.into(),
bashrc_path: bashrc_path.into(),
}
}
pub fn shell_path(&self) -> &str {
&self.shell_path
}
pub fn bashrc_path(&self) -> &str {
&self.bashrc_path
}
}
#[derive(Debug, PartialEq, Eq, Clone, Serialize, Deserialize)]
pub struct PowerShellConfig {
pub(crate) exe: String, // Executable name or path, e.g. "pwsh" or "powershell.exe".
pub(crate) bash_exe_fallback: Option<PathBuf>, // In case the model generates a bash command.
}
impl PowerShellConfig {
pub fn new(exe: impl Into<String>, bash_exe_fallback: Option<PathBuf>) -> Self {
Self {
exe: exe.into(),
bash_exe_fallback,
}
}
pub fn exe(&self) -> &str {
&self.exe
}
pub fn bash_exe_fallback(&self) -> Option<&PathBuf> {
self.bash_exe_fallback.as_ref()
}
}
#[derive(Debug, PartialEq, Eq, Clone, Serialize, Deserialize)]
pub enum Shell {
Zsh(ZshShell),
Bash(BashShell),
PowerShell(PowerShellConfig),
Unknown,
}
impl Shell {
pub fn format_default_shell_invocation(&self, command: Vec<String>) -> Option<Vec<String>> {
match self {
Shell::Zsh(zsh) => format_shell_invocation_with_rc(
command.as_slice(),
&zsh.shell_path,
&zsh.zshrc_path,
),
Shell::Bash(bash) => format_shell_invocation_with_rc(
command.as_slice(),
&bash.shell_path,
&bash.bashrc_path,
),
Shell::PowerShell(ps) => {
// If model generated a bash command, prefer a detected bash fallback
if let Some(script) = strip_bash_lc(command.as_slice()) {
return match &ps.bash_exe_fallback {
Some(bash) => Some(vec![
bash.to_string_lossy().to_string(),
"-lc".to_string(),
script,
]),
// No bash fallback → run the script under PowerShell.
// It will likely fail (except for some simple commands), but the error
// should give a clue to the model to fix upon retry that it's running under PowerShell.
None => Some(vec![
ps.exe.clone(),
"-NoProfile".to_string(),
"-Command".to_string(),
script,
]),
};
}
// Not a bash command. If model did not generate a PowerShell command,
// turn it into a PowerShell command.
let first = command.first().map(String::as_str);
if first != Some(ps.exe.as_str()) {
// TODO (CODEX_2900): Handle escaping newlines.
if command.iter().any(|a| a.contains('\n') || a.contains('\r')) {
return Some(command);
}
let joined = shlex::try_join(command.iter().map(String::as_str)).ok();
return joined.map(|arg| {
vec![
ps.exe.clone(),
"-NoProfile".to_string(),
"-Command".to_string(),
arg,
]
});
}
// Model generated a PowerShell command. Run it.
Some(command)
}
Shell::Unknown => None,
}
}
pub fn name(&self) -> Option<String> {
match self {
Shell::Zsh(zsh) => std::path::Path::new(&zsh.shell_path)
.file_name()
.map(|s| s.to_string_lossy().to_string()),
Shell::Bash(bash) => std::path::Path::new(&bash.shell_path)
.file_name()
.map(|s| s.to_string_lossy().to_string()),
Shell::PowerShell(ps) => Some(ps.exe.clone()),
Shell::Unknown => None,
}
}
}
fn format_shell_invocation_with_rc(
command: &[String],
shell_path: &str,
rc_path: &str,
) -> Option<Vec<String>> {
let joined = strip_bash_lc(command)
.or_else(|| shlex::try_join(command.iter().map(String::as_str)).ok())?;
let rc_command = if std::path::Path::new(rc_path).exists() {
format!("source {rc_path} && ({joined})")
} else {
joined
};
Some(vec![shell_path.to_string(), "-lc".to_string(), rc_command])
}
fn strip_bash_lc(command: &[String]) -> Option<String> {
match command {
// exactly three items
[first, second, third]
// first two must be "bash", "-lc"
if first == "bash" && second == "-lc" =>
{
Some(third.clone())
}
_ => None,
}
}
#[cfg(unix)]
fn detect_default_user_shell() -> Shell {
use libc::getpwuid;
use libc::getuid;
use std::ffi::CStr;
unsafe {
let uid = getuid();
let pw = getpwuid(uid);
if !pw.is_null() {
let shell_path = CStr::from_ptr((*pw).pw_shell)
.to_string_lossy()
.into_owned();
let home_path = CStr::from_ptr((*pw).pw_dir).to_string_lossy().into_owned();
if shell_path.ends_with("/zsh") {
return Shell::Zsh(ZshShell {
shell_path,
zshrc_path: format!("{home_path}/.zshrc"),
});
}
if shell_path.ends_with("/bash") {
return Shell::Bash(BashShell {
shell_path,
bashrc_path: format!("{home_path}/.bashrc"),
});
}
}
}
Shell::Unknown
}
#[cfg(unix)]
pub async fn default_user_shell() -> Shell {
detect_default_user_shell()
}
#[cfg(target_os = "windows")]
pub async fn default_user_shell() -> Shell {
use tokio::process::Command;
// Prefer PowerShell 7+ (`pwsh`) if available, otherwise fall back to Windows PowerShell.
let has_pwsh = Command::new("pwsh")
.arg("-NoLogo")
.arg("-NoProfile")
.arg("-Command")
.arg("$PSVersionTable.PSVersion.Major")
.output()
.await
.map(|o| o.status.success())
.unwrap_or(false);
let bash_exe = if Command::new("bash.exe")
.arg("--version")
.output()
.await
.ok()
.map(|o| o.status.success())
.unwrap_or(false)
{
which::which("bash.exe").ok()
} else {
None
};
if has_pwsh {
Shell::PowerShell(PowerShellConfig {
exe: "pwsh.exe".to_string(),
bash_exe_fallback: bash_exe,
})
} else {
Shell::PowerShell(PowerShellConfig {
exe: "powershell.exe".to_string(),
bash_exe_fallback: bash_exe,
})
}
}
#[cfg(all(not(target_os = "windows"), not(unix)))]
pub async fn default_user_shell() -> Shell {
Shell::Unknown
}

View File

@@ -0,0 +1,182 @@
use base64::Engine;
use serde::Deserialize;
use serde::Serialize;
use thiserror::Error;
#[derive(Deserialize, Serialize, Clone, Debug, PartialEq, Default)]
pub struct TokenData {
/// Flat info parsed from the JWT in auth.json.
#[serde(
deserialize_with = "deserialize_id_token",
serialize_with = "serialize_id_token"
)]
pub id_token: IdTokenInfo,
/// This is a JWT.
pub access_token: String,
pub refresh_token: String,
pub account_id: Option<String>,
}
/// Flat subset of useful claims in id_token from auth.json.
#[derive(Debug, Clone, PartialEq, Eq, Default, Serialize, Deserialize)]
pub struct IdTokenInfo {
pub email: Option<String>,
/// The ChatGPT subscription plan type
/// (e.g., "free", "plus", "pro", "business", "enterprise", "edu").
/// (Note: values may vary by backend.)
pub chatgpt_plan_type: Option<PlanType>,
pub raw_jwt: String,
}
impl IdTokenInfo {
pub fn get_chatgpt_plan_type(&self) -> Option<String> {
self.chatgpt_plan_type.as_ref().map(|t| match t {
PlanType::Known(plan) => format!("{plan:?}"),
PlanType::Unknown(s) => s.clone(),
})
}
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(untagged)]
pub enum PlanType {
Known(KnownPlan),
Unknown(String),
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum KnownPlan {
Free,
Plus,
Pro,
Team,
Business,
Enterprise,
Edu,
}
#[derive(Deserialize)]
struct IdClaims {
#[serde(default)]
email: Option<String>,
#[serde(rename = "https://api.openai.com/auth", default)]
auth: Option<AuthClaims>,
}
#[derive(Deserialize)]
struct AuthClaims {
#[serde(default)]
chatgpt_plan_type: Option<PlanType>,
}
#[derive(Debug, Error)]
pub enum IdTokenInfoError {
#[error("invalid ID token format")]
InvalidFormat,
#[error(transparent)]
Base64(#[from] base64::DecodeError),
#[error(transparent)]
Json(#[from] serde_json::Error),
}
pub fn parse_id_token(id_token: &str) -> Result<IdTokenInfo, IdTokenInfoError> {
// JWT format: header.payload.signature
let mut parts = id_token.split('.');
let (_header_b64, payload_b64, _sig_b64) = match (parts.next(), parts.next(), parts.next()) {
(Some(h), Some(p), Some(s)) if !h.is_empty() && !p.is_empty() && !s.is_empty() => (h, p, s),
_ => return Err(IdTokenInfoError::InvalidFormat),
};
let payload_bytes = base64::engine::general_purpose::URL_SAFE_NO_PAD.decode(payload_b64)?;
let claims: IdClaims = serde_json::from_slice(&payload_bytes)?;
Ok(IdTokenInfo {
email: claims.email,
chatgpt_plan_type: claims.auth.and_then(|a| a.chatgpt_plan_type),
raw_jwt: id_token.to_string(),
})
}
fn deserialize_id_token<'de, D>(deserializer: D) -> Result<IdTokenInfo, D::Error>
where
D: serde::Deserializer<'de>,
{
let s = String::deserialize(deserializer)?;
parse_id_token(&s).map_err(serde::de::Error::custom)
}
fn serialize_id_token<S>(id_token: &IdTokenInfo, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
serializer.serialize_str(&id_token.raw_jwt)
}
#[cfg(test)]
mod tests {
use super::*;
use serde::Serialize;
#[test]
fn id_token_info_parses_email_and_plan() {
#[derive(Serialize)]
struct Header {
alg: &'static str,
typ: &'static str,
}
let header = Header {
alg: "none",
typ: "JWT",
};
let payload = serde_json::json!({
"email": "user@example.com",
"https://api.openai.com/auth": {
"chatgpt_plan_type": "pro"
}
});
fn b64url_no_pad(bytes: &[u8]) -> String {
base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(bytes)
}
let header_b64 = b64url_no_pad(&serde_json::to_vec(&header).unwrap());
let payload_b64 = b64url_no_pad(&serde_json::to_vec(&payload).unwrap());
let signature_b64 = b64url_no_pad(b"sig");
let fake_jwt = format!("{header_b64}.{payload_b64}.{signature_b64}");
let info = parse_id_token(&fake_jwt).expect("should parse");
assert_eq!(info.email.as_deref(), Some("user@example.com"));
assert_eq!(info.get_chatgpt_plan_type().as_deref(), Some("Pro"));
}
#[test]
fn id_token_info_handles_missing_fields() {
#[derive(Serialize)]
struct Header {
alg: &'static str,
typ: &'static str,
}
let header = Header {
alg: "none",
typ: "JWT",
};
let payload = serde_json::json!({ "sub": "123" });
fn b64url_no_pad(bytes: &[u8]) -> String {
base64::engine::general_purpose::URL_SAFE_NO_PAD.encode(bytes)
}
let header_b64 = b64url_no_pad(&serde_json::to_vec(&header).unwrap());
let payload_b64 = b64url_no_pad(&serde_json::to_vec(&payload).unwrap());
let signature_b64 = b64url_no_pad(b"sig");
let fake_jwt = format!("{header_b64}.{payload_b64}.{signature_b64}");
let info = parse_id_token(&fake_jwt).expect("should parse");
assert!(info.email.is_none());
assert!(info.get_chatgpt_plan_type().is_none());
}
}

View File

@@ -0,0 +1,79 @@
use serde::Deserialize;
use serde::Serialize;
#[derive(Debug, Clone, Serialize, PartialEq)]
pub struct ResponsesApiTool {
pub name: String,
pub description: String,
/// TODO: Validation. When strict is set to true, the JSON schema,
/// `required` and `additional_properties` must be present. All fields in
/// `properties` must be present in `required`.
pub strict: bool,
pub parameters: JsonSchema,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub struct FreeformTool {
pub name: String,
pub description: String,
pub format: FreeformToolFormat,
}
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
pub struct FreeformToolFormat {
pub r#type: String,
pub syntax: String,
pub definition: String,
}
/// Generic JSON-Schema subset needed for our tool definitions
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq)]
#[serde(tag = "type", rename_all = "lowercase")]
pub enum JsonSchema {
Boolean {
#[serde(skip_serializing_if = "Option::is_none")]
description: Option<String>,
},
String {
#[serde(skip_serializing_if = "Option::is_none")]
description: Option<String>,
},
/// MCP schema allows "number" | "integer" for Number
#[serde(alias = "integer")]
Number {
#[serde(skip_serializing_if = "Option::is_none")]
description: Option<String>,
},
Array {
items: Box<JsonSchema>,
#[serde(skip_serializing_if = "Option::is_none")]
description: Option<String>,
},
Object {
properties: std::collections::BTreeMap<String, JsonSchema>,
#[serde(skip_serializing_if = "Option::is_none")]
required: Option<Vec<String>>,
#[serde(
rename = "additionalProperties",
skip_serializing_if = "Option::is_none"
)]
additional_properties: Option<bool>,
},
}
/// When serialized as JSON, this produces a valid "Tool" in the OpenAI Responses API.
#[derive(Debug, Clone, Serialize, PartialEq)]
#[serde(tag = "type")]
pub enum OpenAiTool {
#[serde(rename = "function")]
Function(ResponsesApiTool),
#[serde(rename = "local_shell")]
LocalShell {},
// TODO: Understand why we get an error on web_search although the API docs say it's supported.
// https://platform.openai.com/docs/guides/tools-web-search?api-mode=responses#:~:text=%7B%20type%3A%20%22web_search%22%20%7D%2C
#[serde(rename = "web_search")]
WebSearch {},
#[serde(rename = "custom")]
Freeform(FreeformTool),
}

View File

@@ -0,0 +1,10 @@
use serde::Deserialize;
use serde::Serialize;
/// Represents which apply_patch tool variant a model expects.
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq, Hash)]
#[serde(rename_all = "snake_case")]
pub enum ApplyPatchToolType {
Freeform,
Function,
}

View File

@@ -0,0 +1,71 @@
use crate::model_family::ModelFamily;
use crate::tooling::ApplyPatchToolType;
#[derive(Debug, Clone)]
pub enum ConfigShellToolType {
Default,
Local,
Streamable,
}
#[derive(Debug, Clone)]
pub struct ToolsConfig {
pub shell_type: ConfigShellToolType,
pub plan_tool: bool,
pub apply_patch_tool_type: Option<ApplyPatchToolType>,
pub web_search_request: bool,
pub include_view_image_tool: bool,
pub experimental_unified_exec_tool: bool,
}
pub struct ToolsConfigParams<'a> {
pub model_family: &'a ModelFamily,
pub include_plan_tool: bool,
pub include_apply_patch_tool: bool,
pub include_web_search_request: bool,
pub use_streamable_shell_tool: bool,
pub include_view_image_tool: bool,
pub experimental_unified_exec_tool: bool,
}
impl ToolsConfig {
pub fn new(params: &ToolsConfigParams) -> Self {
let ToolsConfigParams {
model_family,
include_plan_tool,
include_apply_patch_tool,
include_web_search_request,
use_streamable_shell_tool,
include_view_image_tool,
experimental_unified_exec_tool,
} = params;
let shell_type = if *use_streamable_shell_tool {
ConfigShellToolType::Streamable
} else if model_family.uses_local_shell_tool {
ConfigShellToolType::Local
} else {
ConfigShellToolType::Default
};
let apply_patch_tool_type = match model_family.apply_patch_tool_type {
Some(ApplyPatchToolType::Freeform) => Some(ApplyPatchToolType::Freeform),
Some(ApplyPatchToolType::Function) => Some(ApplyPatchToolType::Function),
None => {
if *include_apply_patch_tool {
Some(ApplyPatchToolType::Freeform)
} else {
None
}
}
};
Self {
shell_type,
plan_tool: *include_plan_tool,
apply_patch_tool_type,
web_search_request: *include_web_search_request,
include_view_image_tool: *include_view_image_tool,
experimental_unified_exec_tool: *experimental_unified_exec_tool,
}
}
}

View File

@@ -0,0 +1,180 @@
//! Utilities for truncating large chunks of output while preserving a prefix
//! and suffix on UTF-8 boundaries.
/// Truncate the middle of a UTF-8 string to at most `max_bytes` bytes,
/// preserving the beginning and the end. Returns the possibly truncated
/// string and `Some(original_token_count)` (estimated at 4 bytes/token)
/// if truncation occurred; otherwise returns the original string and `None`.
pub fn truncate_middle(s: &str, max_bytes: usize) -> (String, Option<u64>) {
if s.len() <= max_bytes {
return (s.to_string(), None);
}
let est_tokens = (s.len() as u64).div_ceil(4);
if max_bytes == 0 {
return (format!("{est_tokens} tokens truncated…"), Some(est_tokens));
}
fn truncate_on_boundary(input: &str, max_len: usize) -> &str {
if input.len() <= max_len {
return input;
}
let mut end = max_len;
while end > 0 && !input.is_char_boundary(end) {
end -= 1;
}
&input[..end]
}
fn pick_prefix_end(s: &str, left_budget: usize) -> usize {
if let Some(head) = s.get(..left_budget)
&& let Some(i) = head.rfind('\n')
{
return i + 1;
}
truncate_on_boundary(s, left_budget).len()
}
fn pick_suffix_start(s: &str, right_budget: usize) -> usize {
let start_tail = s.len().saturating_sub(right_budget);
if let Some(tail) = s.get(start_tail..)
&& let Some(i) = tail.find('\n')
{
return start_tail + i + 1;
}
let mut idx = start_tail.min(s.len());
while idx < s.len() && !s.is_char_boundary(idx) {
idx += 1;
}
idx
}
let mut guess_tokens = est_tokens;
for _ in 0..4 {
let marker = format!("{guess_tokens} tokens truncated…");
let marker_len = marker.len();
let keep_budget = max_bytes.saturating_sub(marker_len);
if keep_budget == 0 {
return (format!("{est_tokens} tokens truncated…"), Some(est_tokens));
}
let left_budget = keep_budget / 2;
let right_budget = keep_budget - left_budget;
let prefix_end = pick_prefix_end(s, left_budget);
let mut suffix_start = pick_suffix_start(s, right_budget);
if suffix_start < prefix_end {
suffix_start = prefix_end;
}
let kept_content_bytes = prefix_end + (s.len() - suffix_start);
let truncated_content_bytes = s.len().saturating_sub(kept_content_bytes);
let new_tokens = (truncated_content_bytes as u64).div_ceil(4);
if new_tokens == guess_tokens {
let mut out = String::with_capacity(marker_len + kept_content_bytes + 1);
out.push_str(&s[..prefix_end]);
out.push_str(&marker);
out.push('\n');
out.push_str(&s[suffix_start..]);
return (out, Some(est_tokens));
}
guess_tokens = new_tokens;
}
let marker = format!("{guess_tokens} tokens truncated…");
let marker_len = marker.len();
let keep_budget = max_bytes.saturating_sub(marker_len);
if keep_budget == 0 {
return (format!("{est_tokens} tokens truncated…"), Some(est_tokens));
}
let left_budget = keep_budget / 2;
let right_budget = keep_budget - left_budget;
let prefix_end = pick_prefix_end(s, left_budget);
let suffix_start = pick_suffix_start(s, right_budget);
let mut out = String::with_capacity(marker_len + prefix_end + (s.len() - suffix_start) + 1);
out.push_str(&s[..prefix_end]);
out.push_str(&marker);
out.push('\n');
out.push_str(&s[suffix_start..]);
(out, Some(est_tokens))
}
#[cfg(test)]
mod tests {
use super::truncate_middle;
#[test]
fn truncate_middle_no_newlines_fallback() {
let s = "abcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ*";
let max_bytes = 32;
let (out, original) = truncate_middle(s, max_bytes);
assert!(out.starts_with("abc"));
assert!(out.contains("tokens truncated"));
assert!(out.ends_with("XYZ*"));
assert_eq!(original, Some((s.len() as u64).div_ceil(4)));
}
#[test]
fn truncate_middle_prefers_newline_boundaries() {
let mut s = String::new();
for i in 1..=20 {
s.push_str(&format!("{i:03}\n"));
}
assert_eq!(s.len(), 80);
let max_bytes = 64;
let (out, tokens) = truncate_middle(&s, max_bytes);
assert!(out.starts_with("001\n002\n003\n004\n"));
assert!(out.contains("tokens truncated"));
assert!(out.ends_with("017\n018\n019\n020\n"));
assert_eq!(tokens, Some(20));
}
#[test]
fn truncate_middle_handles_utf8_content() {
let s = "😀😀😀😀😀😀😀😀😀😀\nsecond line with ascii text\n";
let max_bytes = 32;
let (out, tokens) = truncate_middle(s, max_bytes);
assert!(out.contains("tokens truncated"));
assert!(!out.contains('\u{fffd}'));
assert_eq!(tokens, Some((s.len() as u64).div_ceil(4)));
}
#[test]
fn truncate_middle_prefers_newline_boundaries_2() {
// Build a multi-line string of 20 numbered lines (each "NNN\n").
let mut s = String::new();
for i in 1..=20 {
s.push_str(&format!("{i:03}\n"));
}
// Total length: 20 lines * 4 bytes per line = 80 bytes.
assert_eq!(s.len(), 80);
// Choose a cap that forces truncation while leaving room for
// a few lines on each side after accounting for the marker.
let max_bytes = 64;
// Expect exact output: first 4 lines, marker, last 4 lines, and correct token estimate (80/4 = 20).
assert_eq!(
truncate_middle(&s, max_bytes),
(
r#"001
002
003
004
…12 tokens truncated…
017
018
019
020
"#
.to_string(),
Some(20)
)
);
}
}

View File

@@ -0,0 +1,896 @@
use std::collections::HashMap;
use std::fs;
use std::path::Path;
use std::path::PathBuf;
use std::process::Command;
use anyhow::Context;
use anyhow::Result;
use anyhow::anyhow;
use sha1::digest::Output;
use uuid::Uuid;
use codex_protocol::protocol::FileChange;
const ZERO_OID: &str = "0000000000000000000000000000000000000000";
const DEV_NULL: &str = "/dev/null";
struct BaselineFileInfo {
path: PathBuf,
content: Vec<u8>,
mode: FileMode,
oid: String,
}
/// Tracks sets of changes to files and exposes the overall unified diff.
/// Internally, the way this works is now:
/// 1. Maintain an in-memory baseline snapshot of files when they are first seen.
/// For new additions, do not create a baseline so that diffs are shown as proper additions (using /dev/null).
/// 2. Keep a stable internal filename (uuid) per external path for rename tracking.
/// 3. To compute the aggregated unified diff, compare each baseline snapshot to the current file on disk entirely in-memory
/// using the `similar` crate and emit unified diffs with rewritten external paths.
#[derive(Default)]
pub struct TurnDiffTracker {
/// Map external path -> internal filename (uuid).
external_to_temp_name: HashMap<PathBuf, String>,
/// Internal filename -> baseline file info.
baseline_file_info: HashMap<String, BaselineFileInfo>,
/// Internal filename -> external path as of current accumulated state (after applying all changes).
/// This is where renames are tracked.
temp_name_to_current_path: HashMap<String, PathBuf>,
/// Cache of known git worktree roots to avoid repeated filesystem walks.
git_root_cache: Vec<PathBuf>,
}
impl TurnDiffTracker {
pub fn new() -> Self {
Self::default()
}
/// Front-run apply patch calls to track the starting contents of any modified files.
/// - Creates an in-memory baseline snapshot for files that already exist on disk when first seen.
/// - For additions, we intentionally do not create a baseline snapshot so that diffs are proper additions.
/// - Also updates internal mappings for move/rename events.
pub fn on_patch_begin(&mut self, changes: &HashMap<PathBuf, FileChange>) {
for (path, change) in changes.iter() {
// Ensure a stable internal filename exists for this external path.
if !self.external_to_temp_name.contains_key(path.as_path()) {
let internal = Uuid::new_v4().to_string();
self.external_to_temp_name
.insert(path.clone(), internal.clone());
self.temp_name_to_current_path
.insert(internal.clone(), path.clone());
// If the file exists on disk now, snapshot as baseline; else leave missing to represent /dev/null.
let baseline_file_info = if path.exists() {
let mode = file_mode_for_path(path);
let mode_val = mode.unwrap_or(FileMode::Regular);
let content = blob_bytes(path, mode_val).unwrap_or_default();
let oid = if mode == Some(FileMode::Symlink) {
format!("{:x}", git_blob_sha1_hex_bytes(&content))
} else {
self.git_blob_oid_for_path(path)
.unwrap_or_else(|| format!("{:x}", git_blob_sha1_hex_bytes(&content)))
};
Some(BaselineFileInfo {
path: path.clone(),
content,
mode: mode_val,
oid,
})
} else {
Some(BaselineFileInfo {
path: path.clone(),
content: vec![],
mode: FileMode::Regular,
oid: ZERO_OID.to_string(),
})
};
if let Some(baseline_file_info) = baseline_file_info {
self.baseline_file_info
.insert(internal.clone(), baseline_file_info);
}
}
// Track rename/move in current mapping if provided in an Update.
if let FileChange::Update {
move_path: Some(dest),
..
} = change
{
let uuid_filename = match self.external_to_temp_name.get(path.as_path()) {
Some(i) => i.clone(),
None => {
// This should be rare, but if we haven't mapped the source, create it with no baseline.
let i = Uuid::new_v4().to_string();
self.baseline_file_info.insert(
i.clone(),
BaselineFileInfo {
path: path.clone(),
content: vec![],
mode: FileMode::Regular,
oid: ZERO_OID.to_string(),
},
);
i
}
};
// Update current external mapping for temp file name.
self.temp_name_to_current_path
.insert(uuid_filename.clone(), dest.clone());
// Update forward file_mapping: external current -> internal name.
self.external_to_temp_name.remove(path);
self.external_to_temp_name
.insert(dest.clone(), uuid_filename);
};
}
}
fn get_path_for_internal(&self, internal: &str) -> Option<PathBuf> {
self.temp_name_to_current_path
.get(internal)
.cloned()
.or_else(|| {
self.baseline_file_info
.get(internal)
.map(|info| info.path.clone())
})
}
/// Find the git worktree root for a file/directory by walking up to the first ancestor containing a `.git` entry.
/// Uses a simple cache of known roots and avoids negative-result caching for simplicity.
fn find_git_root_cached(&mut self, start: &Path) -> Option<PathBuf> {
let dir = if start.is_dir() {
start
} else {
start.parent()?
};
// Fast path: if any cached root is an ancestor of this path, use it.
if let Some(root) = self
.git_root_cache
.iter()
.find(|r| dir.starts_with(r))
.cloned()
{
return Some(root);
}
// Walk up to find a `.git` marker.
let mut cur = dir.to_path_buf();
loop {
let git_marker = cur.join(".git");
if git_marker.is_dir() || git_marker.is_file() {
if !self.git_root_cache.iter().any(|r| r == &cur) {
self.git_root_cache.push(cur.clone());
}
return Some(cur);
}
// On Windows, avoid walking above the drive or UNC share root.
#[cfg(windows)]
{
if is_windows_drive_or_unc_root(&cur) {
return None;
}
}
if let Some(parent) = cur.parent() {
cur = parent.to_path_buf();
} else {
return None;
}
}
}
/// Return a display string for `path` relative to its git root if found, else absolute.
fn relative_to_git_root_str(&mut self, path: &Path) -> String {
let s = if let Some(root) = self.find_git_root_cached(path) {
if let Ok(rel) = path.strip_prefix(&root) {
rel.display().to_string()
} else {
path.display().to_string()
}
} else {
path.display().to_string()
};
s.replace('\\', "/")
}
/// Ask git to compute the blob SHA-1 for the file at `path` within its repository.
/// Returns None if no repository is found or git invocation fails.
fn git_blob_oid_for_path(&mut self, path: &Path) -> Option<String> {
let root = self.find_git_root_cached(path)?;
// Compute a path relative to the repo root for better portability across platforms.
let rel = path.strip_prefix(&root).unwrap_or(path);
let output = Command::new("git")
.arg("-C")
.arg(&root)
.arg("hash-object")
.arg("--")
.arg(rel)
.output()
.ok()?;
if !output.status.success() {
return None;
}
let s = String::from_utf8_lossy(&output.stdout).trim().to_string();
if s.len() == 40 { Some(s) } else { None }
}
/// Recompute the aggregated unified diff by comparing all of the in-memory snapshots that were
/// collected before the first time they were touched by apply_patch during this turn with
/// the current repo state.
pub fn get_unified_diff(&mut self) -> Result<Option<String>> {
let mut aggregated = String::new();
// Compute diffs per tracked internal file in a stable order by external path.
let mut baseline_file_names: Vec<String> =
self.baseline_file_info.keys().cloned().collect();
// Sort lexicographically by full repo-relative path to match git behavior.
baseline_file_names.sort_by_key(|internal| {
self.get_path_for_internal(internal)
.map(|p| self.relative_to_git_root_str(&p))
.unwrap_or_default()
});
for internal in baseline_file_names {
aggregated.push_str(self.get_file_diff(&internal).as_str());
if !aggregated.ends_with('\n') {
aggregated.push('\n');
}
}
if aggregated.trim().is_empty() {
Ok(None)
} else {
Ok(Some(aggregated))
}
}
fn get_file_diff(&mut self, internal_file_name: &str) -> String {
let mut aggregated = String::new();
// Snapshot lightweight fields only.
let (baseline_external_path, baseline_mode, left_oid) = {
if let Some(info) = self.baseline_file_info.get(internal_file_name) {
(info.path.clone(), info.mode, info.oid.clone())
} else {
(PathBuf::new(), FileMode::Regular, ZERO_OID.to_string())
}
};
let current_external_path = match self.get_path_for_internal(internal_file_name) {
Some(p) => p,
None => return aggregated,
};
let current_mode = file_mode_for_path(&current_external_path).unwrap_or(FileMode::Regular);
let right_bytes = blob_bytes(&current_external_path, current_mode);
// Compute displays with &mut self before borrowing any baseline content.
let left_display = self.relative_to_git_root_str(&baseline_external_path);
let right_display = self.relative_to_git_root_str(&current_external_path);
// Compute right oid before borrowing baseline content.
let right_oid = if let Some(b) = right_bytes.as_ref() {
if current_mode == FileMode::Symlink {
format!("{:x}", git_blob_sha1_hex_bytes(b))
} else {
self.git_blob_oid_for_path(&current_external_path)
.unwrap_or_else(|| format!("{:x}", git_blob_sha1_hex_bytes(b)))
}
} else {
ZERO_OID.to_string()
};
// Borrow baseline content only after all &mut self uses are done.
let left_present = left_oid.as_str() != ZERO_OID;
let left_bytes: Option<&[u8]> = if left_present {
self.baseline_file_info
.get(internal_file_name)
.map(|i| i.content.as_slice())
} else {
None
};
// Fast path: identical bytes or both missing.
if left_bytes == right_bytes.as_deref() {
return aggregated;
}
aggregated.push_str(&format!("diff --git a/{left_display} b/{right_display}\n"));
let is_add = !left_present && right_bytes.is_some();
let is_delete = left_present && right_bytes.is_none();
if is_add {
aggregated.push_str(&format!("new file mode {current_mode}\n"));
} else if is_delete {
aggregated.push_str(&format!("deleted file mode {baseline_mode}\n"));
} else if baseline_mode != current_mode {
aggregated.push_str(&format!("old mode {baseline_mode}\n"));
aggregated.push_str(&format!("new mode {current_mode}\n"));
}
let left_text = left_bytes.and_then(|b| std::str::from_utf8(b).ok());
let right_text = right_bytes
.as_deref()
.and_then(|b| std::str::from_utf8(b).ok());
let can_text_diff = matches!(
(left_text, right_text, is_add, is_delete),
(Some(_), Some(_), _, _) | (_, Some(_), true, _) | (Some(_), _, _, true)
);
if can_text_diff {
let l = left_text.unwrap_or("");
let r = right_text.unwrap_or("");
aggregated.push_str(&format!("index {left_oid}..{right_oid}\n"));
let old_header = if left_present {
format!("a/{left_display}")
} else {
DEV_NULL.to_string()
};
let new_header = if right_bytes.is_some() {
format!("b/{right_display}")
} else {
DEV_NULL.to_string()
};
let diff = similar::TextDiff::from_lines(l, r);
let unified = diff
.unified_diff()
.context_radius(3)
.header(&old_header, &new_header)
.to_string();
aggregated.push_str(&unified);
} else {
aggregated.push_str(&format!("index {left_oid}..{right_oid}\n"));
let old_header = if left_present {
format!("a/{left_display}")
} else {
DEV_NULL.to_string()
};
let new_header = if right_bytes.is_some() {
format!("b/{right_display}")
} else {
DEV_NULL.to_string()
};
aggregated.push_str(&format!("--- {old_header}\n"));
aggregated.push_str(&format!("+++ {new_header}\n"));
aggregated.push_str("Binary files differ\n");
}
aggregated
}
}
/// Compute the Git SHA-1 blob object ID for the given content (bytes).
fn git_blob_sha1_hex_bytes(data: &[u8]) -> Output<sha1::Sha1> {
// Git blob hash is sha1 of: "blob <len>\0<data>"
let header = format!("blob {}\0", data.len());
use sha1::Digest;
let mut hasher = sha1::Sha1::new();
hasher.update(header.as_bytes());
hasher.update(data);
hasher.finalize()
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum FileMode {
Regular,
#[cfg(unix)]
Executable,
Symlink,
}
impl FileMode {
fn as_str(self) -> &'static str {
match self {
FileMode::Regular => "100644",
#[cfg(unix)]
FileMode::Executable => "100755",
FileMode::Symlink => "120000",
}
}
}
impl std::fmt::Display for FileMode {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
f.write_str(self.as_str())
}
}
#[cfg(unix)]
fn file_mode_for_path(path: &Path) -> Option<FileMode> {
use std::os::unix::fs::PermissionsExt;
let meta = fs::symlink_metadata(path).ok()?;
let ft = meta.file_type();
if ft.is_symlink() {
return Some(FileMode::Symlink);
}
let mode = meta.permissions().mode();
let is_exec = (mode & 0o111) != 0;
Some(if is_exec {
FileMode::Executable
} else {
FileMode::Regular
})
}
#[cfg(not(unix))]
fn file_mode_for_path(_path: &Path) -> Option<FileMode> {
// Default to non-executable on non-unix.
Some(FileMode::Regular)
}
fn blob_bytes(path: &Path, mode: FileMode) -> Option<Vec<u8>> {
if path.exists() {
let contents = if mode == FileMode::Symlink {
symlink_blob_bytes(path)
.ok_or_else(|| anyhow!("failed to read symlink target for {}", path.display()))
} else {
fs::read(path)
.with_context(|| format!("failed to read current file for diff {}", path.display()))
};
contents.ok()
} else {
None
}
}
#[cfg(unix)]
fn symlink_blob_bytes(path: &Path) -> Option<Vec<u8>> {
use std::os::unix::ffi::OsStrExt;
let target = std::fs::read_link(path).ok()?;
Some(target.as_os_str().as_bytes().to_vec())
}
#[cfg(not(unix))]
fn symlink_blob_bytes(_path: &Path) -> Option<Vec<u8>> {
None
}
#[cfg(windows)]
fn is_windows_drive_or_unc_root(p: &std::path::Path) -> bool {
use std::path::Component;
let mut comps = p.components();
matches!(
(comps.next(), comps.next(), comps.next()),
(Some(Component::Prefix(_)), Some(Component::RootDir), None)
)
}
#[cfg(test)]
mod tests {
use super::*;
use pretty_assertions::assert_eq;
use tempfile::tempdir;
/// Compute the Git SHA-1 blob object ID for the given content (string).
/// This delegates to the bytes version to avoid UTF-8 lossy conversions here.
fn git_blob_sha1_hex(data: &str) -> String {
format!("{:x}", git_blob_sha1_hex_bytes(data.as_bytes()))
}
fn normalize_diff_for_test(input: &str, root: &Path) -> String {
let root_str = root.display().to_string().replace('\\', "/");
let replaced = input.replace(&root_str, "<TMP>");
// Split into blocks on lines starting with "diff --git ", sort blocks for determinism, and rejoin
let mut blocks: Vec<String> = Vec::new();
let mut current = String::new();
for line in replaced.lines() {
if line.starts_with("diff --git ") && !current.is_empty() {
blocks.push(current);
current = String::new();
}
if !current.is_empty() {
current.push('\n');
}
current.push_str(line);
}
if !current.is_empty() {
blocks.push(current);
}
blocks.sort();
let mut out = blocks.join("\n");
if !out.ends_with('\n') {
out.push('\n');
}
out
}
#[test]
fn accumulates_add_and_update() {
let mut acc = TurnDiffTracker::new();
let dir = tempdir().unwrap();
let file = dir.path().join("a.txt");
// First patch: add file (baseline should be /dev/null).
let add_changes = HashMap::from([(
file.clone(),
FileChange::Add {
content: "foo\n".to_string(),
},
)]);
acc.on_patch_begin(&add_changes);
// Simulate apply: create the file on disk.
fs::write(&file, "foo\n").unwrap();
let first = acc.get_unified_diff().unwrap().unwrap();
let first = normalize_diff_for_test(&first, dir.path());
let expected_first = {
let mode = file_mode_for_path(&file).unwrap_or(FileMode::Regular);
let right_oid = git_blob_sha1_hex("foo\n");
format!(
r#"diff --git a/<TMP>/a.txt b/<TMP>/a.txt
new file mode {mode}
index {ZERO_OID}..{right_oid}
--- {DEV_NULL}
+++ b/<TMP>/a.txt
@@ -0,0 +1 @@
+foo
"#,
)
};
assert_eq!(first, expected_first);
// Second patch: update the file on disk.
let update_changes = HashMap::from([(
file.clone(),
FileChange::Update {
unified_diff: "".to_owned(),
move_path: None,
},
)]);
acc.on_patch_begin(&update_changes);
// Simulate apply: append a new line.
fs::write(&file, "foo\nbar\n").unwrap();
let combined = acc.get_unified_diff().unwrap().unwrap();
let combined = normalize_diff_for_test(&combined, dir.path());
let expected_combined = {
let mode = file_mode_for_path(&file).unwrap_or(FileMode::Regular);
let right_oid = git_blob_sha1_hex("foo\nbar\n");
format!(
r#"diff --git a/<TMP>/a.txt b/<TMP>/a.txt
new file mode {mode}
index {ZERO_OID}..{right_oid}
--- {DEV_NULL}
+++ b/<TMP>/a.txt
@@ -0,0 +1,2 @@
+foo
+bar
"#,
)
};
assert_eq!(combined, expected_combined);
}
#[test]
fn accumulates_delete() {
let dir = tempdir().unwrap();
let file = dir.path().join("b.txt");
fs::write(&file, "x\n").unwrap();
let mut acc = TurnDiffTracker::new();
let del_changes = HashMap::from([(
file.clone(),
FileChange::Delete {
content: "x\n".to_string(),
},
)]);
acc.on_patch_begin(&del_changes);
// Simulate apply: delete the file from disk.
let baseline_mode = file_mode_for_path(&file).unwrap_or(FileMode::Regular);
fs::remove_file(&file).unwrap();
let diff = acc.get_unified_diff().unwrap().unwrap();
let diff = normalize_diff_for_test(&diff, dir.path());
let expected = {
let left_oid = git_blob_sha1_hex("x\n");
format!(
r#"diff --git a/<TMP>/b.txt b/<TMP>/b.txt
deleted file mode {baseline_mode}
index {left_oid}..{ZERO_OID}
--- a/<TMP>/b.txt
+++ {DEV_NULL}
@@ -1 +0,0 @@
-x
"#,
)
};
assert_eq!(diff, expected);
}
#[test]
fn accumulates_move_and_update() {
let dir = tempdir().unwrap();
let src = dir.path().join("src.txt");
let dest = dir.path().join("dst.txt");
fs::write(&src, "line\n").unwrap();
let mut acc = TurnDiffTracker::new();
let mv_changes = HashMap::from([(
src.clone(),
FileChange::Update {
unified_diff: "".to_owned(),
move_path: Some(dest.clone()),
},
)]);
acc.on_patch_begin(&mv_changes);
// Simulate apply: move and update content.
fs::rename(&src, &dest).unwrap();
fs::write(&dest, "line2\n").unwrap();
let out = acc.get_unified_diff().unwrap().unwrap();
let out = normalize_diff_for_test(&out, dir.path());
let expected = {
let left_oid = git_blob_sha1_hex("line\n");
let right_oid = git_blob_sha1_hex("line2\n");
format!(
r#"diff --git a/<TMP>/src.txt b/<TMP>/dst.txt
index {left_oid}..{right_oid}
--- a/<TMP>/src.txt
+++ b/<TMP>/dst.txt
@@ -1 +1 @@
-line
+line2
"#
)
};
assert_eq!(out, expected);
}
#[test]
fn move_without_1change_yields_no_diff() {
let dir = tempdir().unwrap();
let src = dir.path().join("moved.txt");
let dest = dir.path().join("renamed.txt");
fs::write(&src, "same\n").unwrap();
let mut acc = TurnDiffTracker::new();
let mv_changes = HashMap::from([(
src.clone(),
FileChange::Update {
unified_diff: "".to_owned(),
move_path: Some(dest.clone()),
},
)]);
acc.on_patch_begin(&mv_changes);
// Simulate apply: move only, no content change.
fs::rename(&src, &dest).unwrap();
let diff = acc.get_unified_diff().unwrap();
assert_eq!(diff, None);
}
#[test]
fn move_declared_but_file_only_appears_at_dest_is_add() {
let dir = tempdir().unwrap();
let src = dir.path().join("src.txt");
let dest = dir.path().join("dest.txt");
let mut acc = TurnDiffTracker::new();
let mv = HashMap::from([(
src,
FileChange::Update {
unified_diff: "".into(),
move_path: Some(dest.clone()),
},
)]);
acc.on_patch_begin(&mv);
// No file existed initially; create only dest
fs::write(&dest, "hello\n").unwrap();
let diff = acc.get_unified_diff().unwrap().unwrap();
let diff = normalize_diff_for_test(&diff, dir.path());
let expected = {
let mode = file_mode_for_path(&dest).unwrap_or(FileMode::Regular);
let right_oid = git_blob_sha1_hex("hello\n");
format!(
r#"diff --git a/<TMP>/src.txt b/<TMP>/dest.txt
new file mode {mode}
index {ZERO_OID}..{right_oid}
--- {DEV_NULL}
+++ b/<TMP>/dest.txt
@@ -0,0 +1 @@
+hello
"#,
)
};
assert_eq!(diff, expected);
}
#[test]
fn update_persists_across_new_baseline_for_new_file() {
let dir = tempdir().unwrap();
let a = dir.path().join("a.txt");
let b = dir.path().join("b.txt");
fs::write(&a, "foo\n").unwrap();
fs::write(&b, "z\n").unwrap();
let mut acc = TurnDiffTracker::new();
// First: update existing a.txt (baseline snapshot is created for a).
let update_a = HashMap::from([(
a.clone(),
FileChange::Update {
unified_diff: "".to_owned(),
move_path: None,
},
)]);
acc.on_patch_begin(&update_a);
// Simulate apply: modify a.txt on disk.
fs::write(&a, "foo\nbar\n").unwrap();
let first = acc.get_unified_diff().unwrap().unwrap();
let first = normalize_diff_for_test(&first, dir.path());
let expected_first = {
let left_oid = git_blob_sha1_hex("foo\n");
let right_oid = git_blob_sha1_hex("foo\nbar\n");
format!(
r#"diff --git a/<TMP>/a.txt b/<TMP>/a.txt
index {left_oid}..{right_oid}
--- a/<TMP>/a.txt
+++ b/<TMP>/a.txt
@@ -1 +1,2 @@
foo
+bar
"#
)
};
assert_eq!(first, expected_first);
// Next: introduce a brand-new path b.txt into baseline snapshots via a delete change.
let del_b = HashMap::from([(
b.clone(),
FileChange::Delete {
content: "z\n".to_string(),
},
)]);
acc.on_patch_begin(&del_b);
// Simulate apply: delete b.txt.
let baseline_mode = file_mode_for_path(&b).unwrap_or(FileMode::Regular);
fs::remove_file(&b).unwrap();
let combined = acc.get_unified_diff().unwrap().unwrap();
let combined = normalize_diff_for_test(&combined, dir.path());
let expected = {
let left_oid_a = git_blob_sha1_hex("foo\n");
let right_oid_a = git_blob_sha1_hex("foo\nbar\n");
let left_oid_b = git_blob_sha1_hex("z\n");
format!(
r#"diff --git a/<TMP>/a.txt b/<TMP>/a.txt
index {left_oid_a}..{right_oid_a}
--- a/<TMP>/a.txt
+++ b/<TMP>/a.txt
@@ -1 +1,2 @@
foo
+bar
diff --git a/<TMP>/b.txt b/<TMP>/b.txt
deleted file mode {baseline_mode}
index {left_oid_b}..{ZERO_OID}
--- a/<TMP>/b.txt
+++ {DEV_NULL}
@@ -1 +0,0 @@
-z
"#,
)
};
assert_eq!(combined, expected);
}
#[test]
fn binary_files_differ_update() {
let dir = tempdir().unwrap();
let file = dir.path().join("bin.dat");
// Initial non-UTF8 bytes
let left_bytes: Vec<u8> = vec![0xff, 0xfe, 0xfd, 0x00];
// Updated non-UTF8 bytes
let right_bytes: Vec<u8> = vec![0x01, 0x02, 0x03, 0x00];
fs::write(&file, &left_bytes).unwrap();
let mut acc = TurnDiffTracker::new();
let update_changes = HashMap::from([(
file.clone(),
FileChange::Update {
unified_diff: "".to_owned(),
move_path: None,
},
)]);
acc.on_patch_begin(&update_changes);
// Apply update on disk
fs::write(&file, &right_bytes).unwrap();
let diff = acc.get_unified_diff().unwrap().unwrap();
let diff = normalize_diff_for_test(&diff, dir.path());
let expected = {
let left_oid = format!("{:x}", git_blob_sha1_hex_bytes(&left_bytes));
let right_oid = format!("{:x}", git_blob_sha1_hex_bytes(&right_bytes));
format!(
r#"diff --git a/<TMP>/bin.dat b/<TMP>/bin.dat
index {left_oid}..{right_oid}
--- a/<TMP>/bin.dat
+++ b/<TMP>/bin.dat
Binary files differ
"#
)
};
assert_eq!(diff, expected);
}
#[test]
fn filenames_with_spaces_add_and_update() {
let mut acc = TurnDiffTracker::new();
let dir = tempdir().unwrap();
let file = dir.path().join("name with spaces.txt");
// First patch: add file (baseline should be /dev/null).
let add_changes = HashMap::from([(
file.clone(),
FileChange::Add {
content: "foo\n".to_string(),
},
)]);
acc.on_patch_begin(&add_changes);
// Simulate apply: create the file on disk.
fs::write(&file, "foo\n").unwrap();
let first = acc.get_unified_diff().unwrap().unwrap();
let first = normalize_diff_for_test(&first, dir.path());
let expected_first = {
let mode = file_mode_for_path(&file).unwrap_or(FileMode::Regular);
let right_oid = git_blob_sha1_hex("foo\n");
format!(
r#"diff --git a/<TMP>/name with spaces.txt b/<TMP>/name with spaces.txt
new file mode {mode}
index {ZERO_OID}..{right_oid}
--- {DEV_NULL}
+++ b/<TMP>/name with spaces.txt
@@ -0,0 +1 @@
+foo
"#,
)
};
assert_eq!(first, expected_first);
// Second patch: update the file on disk.
let update_changes = HashMap::from([(
file.clone(),
FileChange::Update {
unified_diff: "".to_owned(),
move_path: None,
},
)]);
acc.on_patch_begin(&update_changes);
// Simulate apply: append a new line with a space.
fs::write(&file, "foo\nbar baz\n").unwrap();
let combined = acc.get_unified_diff().unwrap().unwrap();
let combined = normalize_diff_for_test(&combined, dir.path());
let expected_combined = {
let mode = file_mode_for_path(&file).unwrap_or(FileMode::Regular);
let right_oid = git_blob_sha1_hex("foo\nbar baz\n");
format!(
r#"diff --git a/<TMP>/name with spaces.txt b/<TMP>/name with spaces.txt
new file mode {mode}
index {ZERO_OID}..{right_oid}
--- {DEV_NULL}
+++ b/<TMP>/name with spaces.txt
@@ -0,0 +1,2 @@
+foo
+bar baz
"#,
)
};
assert_eq!(combined, expected_combined);
}
}

View File

@@ -1,7 +1,7 @@
use thiserror::Error;
#[derive(Debug, Error)]
pub(crate) enum UnifiedExecError {
pub enum UnifiedExecError {
#[error("Failed to create unified exec session: {pty_error}")]
CreateSession {
#[source]

View File

@@ -22,27 +22,27 @@ use crate::truncate::truncate_middle;
mod errors;
pub(crate) use errors::UnifiedExecError;
pub use errors::UnifiedExecError;
const DEFAULT_TIMEOUT_MS: u64 = 1_000;
const MAX_TIMEOUT_MS: u64 = 60_000;
const UNIFIED_EXEC_OUTPUT_MAX_BYTES: usize = 128 * 1024; // 128 KiB
#[derive(Debug)]
pub(crate) struct UnifiedExecRequest<'a> {
pub struct UnifiedExecRequest<'a> {
pub session_id: Option<i32>,
pub input_chunks: &'a [String],
pub timeout_ms: Option<u64>,
}
#[derive(Debug, Clone, PartialEq)]
pub(crate) struct UnifiedExecResult {
pub struct UnifiedExecResult {
pub session_id: Option<i32>,
pub output: String,
}
#[derive(Debug, Default)]
pub(crate) struct UnifiedExecSessionManager {
pub struct UnifiedExecSessionManager {
next_session_id: AtomicI32,
sessions: Mutex<HashMap<i32, ManagedUnifiedExecSession>>,
}
@@ -404,6 +404,8 @@ async fn create_unified_exec_session(
#[cfg(test)]
mod tests {
use super::*;
#[cfg(unix)]
use core_test_support::skip_if_sandbox;
#[test]
fn push_chunk_trims_only_excess_bytes() {
@@ -425,6 +427,8 @@ mod tests {
#[cfg(unix)]
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn unified_exec_persists_across_requests_jif() -> Result<(), UnifiedExecError> {
skip_if_sandbox!(Ok(()));
let manager = UnifiedExecSessionManager::default();
let open_shell = manager
@@ -462,6 +466,8 @@ mod tests {
#[cfg(unix)]
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn multi_unified_exec_sessions() -> Result<(), UnifiedExecError> {
skip_if_sandbox!(Ok(()));
let manager = UnifiedExecSessionManager::default();
let shell_a = manager
@@ -508,6 +514,8 @@ mod tests {
#[cfg(unix)]
#[tokio::test]
async fn unified_exec_timeouts() -> Result<(), UnifiedExecError> {
skip_if_sandbox!(Ok(()));
let manager = UnifiedExecSessionManager::default();
let open_shell = manager
@@ -557,6 +565,7 @@ mod tests {
#[cfg(unix)]
#[tokio::test]
#[ignore] // Ignored while we have a better way to test this.
async fn requests_with_large_timeout_are_capped() -> Result<(), UnifiedExecError> {
let manager = UnifiedExecSessionManager::default();
@@ -578,6 +587,7 @@ mod tests {
#[cfg(unix)]
#[tokio::test]
#[ignore] // Ignored while we have a better way to test this.
async fn completed_commands_do_not_persist_sessions() -> Result<(), UnifiedExecError> {
let manager = UnifiedExecSessionManager::default();
let result = manager
@@ -599,6 +609,8 @@ mod tests {
#[cfg(unix)]
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn reusing_completed_session_returns_unknown_session() -> Result<(), UnifiedExecError> {
skip_if_sandbox!(Ok(()));
let manager = UnifiedExecSessionManager::default();
let open_shell = manager

View File

@@ -0,0 +1,348 @@
use codex_protocol::protocol::AgentMessageDeltaEvent;
use codex_protocol::protocol::AgentReasoningDeltaEvent;
use codex_protocol::protocol::AgentReasoningSectionBreakEvent;
use codex_protocol::protocol::EventMsg;
use codex_protocol::protocol::InputItem;
use codex_protocol::protocol::Op;
use codex_protocol::protocol::Submission;
use codex_protocol::protocol::TaskStartedEvent;
use codex_protocol::protocol::TokenCountEvent;
use codex_protocol::protocol::TokenUsageInfo;
use core_test_support::responses::ev_assistant_message;
use core_test_support::responses::ev_completed;
use core_test_support::responses::mount_sse_sequence;
use core_test_support::responses::sse;
use core_test_support::responses::start_mock_server;
use core_test_support::test_codex::test_codex;
use serde_json::json;
fn build_user_input(text: &str) -> Op {
Op::UserInput {
items: vec![InputItem::Text {
text: text.to_string(),
}],
}
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn submit_returns_id_and_events_share_it() {
core_test_support::skip_if_no_network!();
let server = start_mock_server().await;
let body = sse(vec![
ev_assistant_message("resp-item", "First turn complete."),
ev_completed("resp-complete"),
]);
mount_sse_sequence(&server, vec![body]).await;
let mut builder = test_codex();
let codex = builder.build(&server).await.unwrap().codex;
let submission_id = codex
.submit(build_user_input("first turn"))
.await
.expect("submit succeeds");
assert!(!submission_id.is_empty(), "submit should return id");
let mut saw_agent_message = false;
loop {
let event = codex.next_event().await.expect("event available");
assert_eq!(event.id, submission_id, "event id should match submission");
match event.msg {
EventMsg::AgentMessage(ev) => {
saw_agent_message = true;
assert_eq!(ev.message, "First turn complete.");
}
EventMsg::TaskComplete(ev) => {
assert_eq!(
ev.last_agent_message.as_deref(),
Some("First turn complete."),
);
break;
}
_ => {}
}
}
assert!(saw_agent_message, "expected AgentMessage event");
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn submit_with_id_uses_caller_supplied_id() {
core_test_support::skip_if_no_network!();
let server = start_mock_server().await;
let body = sse(vec![
ev_assistant_message("resp-item", "Acknowledged."),
ev_completed("resp-complete"),
]);
mount_sse_sequence(&server, vec![body]).await;
let mut builder = test_codex();
let codex = builder.build(&server).await.unwrap().codex;
let custom_id = "custom-submission-id".to_string();
codex
.submit_with_id(Submission {
id: custom_id.clone(),
op: build_user_input("please acknowledge"),
})
.await
.expect("submit_with_id succeeds");
loop {
let event = codex.next_event().await.expect("event available");
assert_eq!(event.id, custom_id, "event id should match provided id");
if let EventMsg::TaskComplete(ev) = event.msg {
assert_eq!(ev.last_agent_message.as_deref(), Some("Acknowledged."));
break;
}
}
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn submit_streams_reasoning_and_token_usage_events() {
core_test_support::skip_if_no_network!();
let server = start_mock_server().await;
let response_id = "resp-stream";
let body = sse(vec![
json!({
"type": "response.created",
"response": {"id": response_id}
}),
json!({
"type": "response.output_text.delta",
"delta": "Partial "
}),
json!({
"type": "response.reasoning_summary_text.delta",
"delta": "Drafting plan"
}),
json!({
"type": "response.reasoning_summary_part.added"
}),
ev_assistant_message("resp-item", "Partial Final output."),
json!({
"type": "response.completed",
"response": {
"id": response_id,
"usage": {
"input_tokens": 40,
"input_tokens_details": {"cached_tokens": 5},
"output_tokens": 12,
"output_tokens_details": {"reasoning_tokens": 3},
"total_tokens": 52
}
}
}),
]);
mount_sse_sequence(&server, vec![body]).await;
let mut builder = test_codex();
let codex = builder.build(&server).await.unwrap().codex;
let submission_id = codex
.submit(build_user_input("stream detailed output"))
.await
.expect("submit succeeds");
let mut saw_text_delta = false;
let mut saw_reasoning_delta = false;
let mut saw_agent_message = false;
let mut final_message: Option<String> = None;
let mut token_count: Option<TokenCountEvent> = None;
loop {
let event = codex.next_event().await.expect("event available");
assert_eq!(event.id, submission_id);
match event.msg {
EventMsg::AgentMessageDelta(AgentMessageDeltaEvent { delta }) => {
saw_text_delta = true;
assert_eq!(delta, "Partial ");
}
EventMsg::AgentReasoningDelta(AgentReasoningDeltaEvent { delta }) => {
saw_reasoning_delta = true;
assert_eq!(delta, "Drafting plan");
}
EventMsg::AgentReasoningSectionBreak(AgentReasoningSectionBreakEvent {}) => {}
EventMsg::AgentMessage(ev) => {
saw_agent_message = true;
final_message = Some(ev.message);
}
EventMsg::TokenCount(ev) => {
token_count = Some(ev);
}
EventMsg::TaskComplete(ev) => {
assert_eq!(
ev.last_agent_message.as_deref(),
Some("Partial Final output."),
);
break;
}
_ => {}
}
}
assert!(saw_text_delta, "expected streaming text delta event");
assert!(saw_reasoning_delta, "expected reasoning summary delta");
// Some model responses may omit explicit reasoning section boundaries even when
// reasoning deltas stream, so treat the section break as optional.
assert!(saw_agent_message, "expected agent message event");
assert_eq!(final_message.as_deref(), Some("Partial Final output."));
let token_count = token_count.expect("expected token count event");
let info: &TokenUsageInfo = token_count
.info
.as_ref()
.expect("token usage info should be present");
assert_eq!(info.last_token_usage.total_tokens, 52);
assert_eq!(info.last_token_usage.input_tokens, 40);
assert_eq!(info.last_token_usage.output_tokens, 12);
assert_eq!(info.last_token_usage.reasoning_output_tokens, 3);
assert_eq!(info.last_token_usage.cached_input_tokens, 5);
assert_eq!(info.total_token_usage.total_tokens, 52);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn sequential_submissions_emit_distinct_ids_and_token_totals() {
core_test_support::skip_if_no_network!();
let server = start_mock_server().await;
let first_body = sse(vec![
json!({"type": "response.created", "response": {"id": "resp-first"}}),
ev_assistant_message("resp-first-item", "Turn one done."),
json!({
"type": "response.completed",
"response": {
"id": "resp-first",
"usage": {
"input_tokens": 10,
"input_tokens_details": {"cached_tokens": 0},
"output_tokens": 4,
"output_tokens_details": {"reasoning_tokens": 1},
"total_tokens": 14
}
}
}),
]);
let second_body = sse(vec![
json!({"type": "response.created", "response": {"id": "resp-second"}}),
json!({
"type": "response.output_text.delta",
"delta": "Streaming second "
}),
ev_assistant_message("resp-second-item", "Turn two complete."),
json!({
"type": "response.completed",
"response": {
"id": "resp-second",
"usage": {
"input_tokens": 12,
"input_tokens_details": {"cached_tokens": 2},
"output_tokens": 6,
"output_tokens_details": {"reasoning_tokens": 2},
"total_tokens": 20
}
}
}),
]);
mount_sse_sequence(&server, vec![first_body, second_body]).await;
let mut builder = test_codex();
let codex = builder.build(&server).await.unwrap().codex;
let first_id = codex
.submit(build_user_input("first turn"))
.await
.expect("first submit succeeds");
assert!(
!first_id.is_empty(),
"first submission id should be present"
);
let mut saw_first_task_started = false;
let mut saw_first_token_count: Option<TokenCountEvent> = None;
let mut first_completed = false;
while !first_completed {
let event = codex.next_event().await.expect("first turn event");
assert_eq!(event.id, first_id);
match event.msg {
EventMsg::TaskStarted(TaskStartedEvent { .. }) => {
saw_first_task_started = true;
}
EventMsg::AgentMessage(message) => {
assert_eq!(message.message, "Turn one done.");
}
EventMsg::TokenCount(ev) => {
saw_first_token_count = Some(ev);
}
EventMsg::TaskComplete(_) => {
first_completed = true;
}
_ => {}
}
}
assert!(saw_first_task_started, "first turn should emit TaskStarted");
let first_token = saw_first_token_count.expect("first turn should emit TokenCount");
let first_usage: &TokenUsageInfo = first_token
.info
.as_ref()
.expect("usage info should exist for first turn");
assert_eq!(first_usage.last_token_usage.total_tokens, 14);
let custom_id = "manual-second".to_string();
codex
.submit_with_id(Submission {
id: custom_id.clone(),
op: build_user_input("second turn"),
})
.await
.expect("second submit succeeds");
let mut saw_second_task_started = false;
let mut saw_second_token_count: Option<TokenCountEvent> = None;
loop {
let event = codex.next_event().await.expect("second turn event");
assert_eq!(event.id, custom_id);
match event.msg {
EventMsg::TaskStarted(TaskStartedEvent { .. }) => {
saw_second_task_started = true;
}
EventMsg::AgentMessage(message) => {
assert_eq!(message.message, "Turn two complete.");
}
EventMsg::TokenCount(ev) => {
saw_second_token_count = Some(ev);
}
EventMsg::TaskComplete(ev) => {
assert_eq!(ev.last_agent_message.as_deref(), Some("Turn two complete."),);
break;
}
_ => {}
}
}
assert!(
saw_second_task_started,
"second turn should emit TaskStarted"
);
let second_token = saw_second_token_count.expect("second turn should emit TokenCount");
let second_usage: &TokenUsageInfo = second_token
.info
.as_ref()
.expect("usage info should exist for second turn");
assert_eq!(second_usage.last_token_usage.total_tokens, 20);
assert_eq!(second_usage.total_token_usage.total_tokens, 34);
assert!(
second_usage.total_token_usage.total_tokens > first_usage.total_token_usage.total_tokens
);
}

112
codex-rs/agent_refactor.md Normal file
View File

@@ -0,0 +1,112 @@
# Agent Runtime Refactor
## Goals
- Decouple the Codex agent loop from CLI-specific wiring so it can run as a reusable library or standalone binary.
- Preserve the current behaviour of `codex-core` (tooling, approvals, sandboxing, MCP integration) while providing a cleaner embedding surface.
- Enable specialised hosts—CLI, training harnesses, response API bridges—to share the same runtime with minimal glue code.
## Proposed Architecture
### 1. `codex-agent` crate (new)
- Owns the session runtime: `AgentRuntime`, `AgentHandle`, the states, and the task runners now under `core/src/tasks`.
- Exposes a queue-like API: `AgentHandle::submit(Op/Submission)` and `AgentHandle::next_event()` mirroring todays behaviour.
- Re-exports protocol types from `codex-protocol` so consumers do not depend on the entire `codex-core` tree.
- Houses the agent loop (`run_task`, `run_turn`, exec/safety plumbing) together with the sandbox planner (`ExecPlan`, `PreparedExec`, etc.).
### 2. Shared configuration surface
- Introduce `AgentConfig` as the minimal runtime configuration (model, provider, approvals, sandbox defaults, cwd, user/base instructions, feature flags relevant to the loop).
- Provide `From<&Config>` for CLI compatibility; training/other hosts construct `AgentConfig` directly.
- CLI-only concerns (logging, auth prompts, workspace presets) stay inside `codex-core` and are translated before spawning the runtime.
### 3. Service abstraction layer
- Define traits that the runtime depends on instead of concrete CLI structs:
- `CredentialsProvider` (wraps `AuthManager`).
- `Notifier` (reuses `UserNotifier` contract).
- `McpInterface` (start/list tools, dispatch tool calls).
- `SandboxManager` (wraps `BackendRegistry`/`prepare_exec_invocation` wiring).
- `RolloutSink` (write/flush rollout items; default no-op).
- Provide default implementations in `codex-core` that simply wrap the existing services (`SessionServices`).
### 4. Task subsystem consolidation
- Keep the new `SessionTask` trait and concrete tasks (`RegularTask`, `ReviewTask`, `CompactTask`) inside `codex-agent` so custom hosts can opt into additional tasks without touching CLI crates.
- Ensure task lifecycle management (`spawn_task`, `abort_all_tasks`, `ActiveTurn`) stays encapsulated in the runtime and surfaces only high-level signals (events, cancellation APIs).
### 5. Sandbox execution layer
- Move the recently created `core/src/sandbox` module into `codex-agent` (or re-export) so runtime owns exec planning.
- Runtime exposes an injectable `SandboxRuntimeConfig` (paths, seatbelt binary, stdout streaming choice) and calls into `SandboxManager` to execute plans.
- Respect existing environment variables and approval policies; no semantic changes to seatbelt handling.
### 6. Host integrations
- CLI crate: replaces direct usage of `Codex::spawn` with `AgentRuntime::spawn`, adapting CLI config/auth providers to runtime traits. Behaviour remains identical.
- Training binary (`codex-agent-bin`): thin crate that parses CLI flags (Response API URL, auth token, optional instructions) and bridges remote Ops/Events to the runtime via chosen transport (MCP channel, HTTP/WebSocket bridge).
- Additional hosts can embed the runtime by implementing the service traits and providing transport glue.
### 7. Transport adapters
- Internally keep `async_channel` for runtime queues.
- Provide helper adapters (`AgentTransport` trait) so callers can hook streams (local channel, TCP bridge, etc.) while keeping backpressure and graceful shutdown semantics consistent.
## Guidelines
- **Config boundary**: new code must depend on `AgentConfig`; only CLI/front-ends may use the broader `Config` struct. Avoid adding CLI-specific fields to the runtime config.
- **Trait-based services**: any runtime dependency that could vary across hosts (MCP, rollout persistence, sandbox execution, notifications) should be expressed as a trait with a default implementation living in `codex-core`.
- **Task authoring**: additional tasks must implement `SessionTask`; tasks are responsible for calling `run_task`/`exit_review_mode` helpers and returning final assistant output for `TaskComplete` events.
- **Sandbox safety**: all exec/patch calls must flow through `plan_exec`/`plan_apply_patch` (now under `codex-agent::sandbox`) to preserve approval semantics. Never bypass `SandboxManager`.
- **MCP usage**: runtime talks only through `McpInterface`; hosts provide concrete connectors (existing CLI manager, lightweight training stub, etc.).
- **Rollout handling**: default `RolloutSink` should no-op; hosts that require persistence (CLI, evaluation harness) supply an implementation that wraps existing recorder.
- **Transport/backpressure**: treat the runtime queue as bounded and handle cancellations; adapters must propagate `Op::Shutdown` promptly.
- **Observability**: keep tracing instrumentation intact; new modules should use existing `tracing` spans for start/end of tasks, exec calls, and MCP interactions.
- **Code quality**: write minimalist idiomatic code. Leverage the capacity of Rust
## Current Scope Snapshot
- `codex-agent` owns the execution/runtime surface: conversation history, rollout recording, function tool plumbing, sandbox planning, command/apply_patch safety, and the new `ApprovalCoordinator` trait that abstracts user approvals. Host-agnostic helpers such as shell formatting, bash parsing, and command safety now live here.
- `codex-core` focuses on CLI integration: loading user configuration, wiring concrete services (auth, MCP, sandbox manager), translating CLI policies into runtime configs, and exposing the embedded runtime to front-ends. It re-exports runtime modules needed by existing callers but should avoid hosting new agent logic.
- Session bootstrap now flows through a host-provided `prepare_session_bootstrap` helper: the CLI constructs rollout/MCP/sandbox services, builds the new `codex_agent::SessionServices` + `SessionState`, pre-builds the initial `TurnContext` (model client + tool config), and hands them to `Session::new` instead of constructing them inline.
## Implementation Plan
1. **Baseline & documentation**
- Capture current interfaces (`Codex`, `Session`, `SessionTask`) and update developer docs to reference this refactor plan.
- Add smoke tests covering multi-task scenarios (regular + review + compact) to guard against regressions during extraction.
2. **Introduce `AgentConfig`**
- Define struct + conversion helpers inside `codex-core`.
- Refactor internal `Session::new` / `TurnContext` builders to accept `AgentConfig` without changing external behaviour.
3. **Service trait extraction**
- Carve out trait definitions (`CredentialsProvider`, `McpInterface`, `SandboxManager`, `RolloutSink`, `Notifier`).
- Provide adapters backed by existing `SessionServices`.
- Update `Session` and helper modules to depend on traits rather than concrete structs.
4. **Create `codex-agent` crate**
- Scaffold crate, move runtime modules (`codex.rs`, `state`, `tasks`, `sandbox`) while keeping module paths stable via `pub use` re-exports.
- Resolve module imports to reference trait abstractions / helper crates (e.g., `codex_protocol`, `codex-apply-patch`).
- Ensure crate exposes `AgentRuntime`, `AgentHandle`, and service traits.
5. **Adapt `codex-core`**
- Replace `Codex::spawn` with thin wrapper that constructs `AgentConfig`, runtime service adapters, and delegates to `codex-agent`.
- Update public API to re-export runtime types if downstream crates expect them.
- Confirm unit tests continue to pass.
6. **Update front-ends**
- CLI crate: switch to new runtime API; verify login/auth flows, approvals, and sandbox invocations.
- Other binaries (`chatgpt`, etc.) migrate similarly, adjusting imports/config conversions.
7. **Add training binary**
- Implement new `codex-agent-bin` crate providing CLI for Response API URL + auth.
- Reuse existing MCP client logic where possible; otherwise, provide minimal HTTP bridge translating Ops/Events.
- Add integration tests using mocked Response API.
8. **Refine transport adapters**
- Add optional helper module offering channel/TCP/WebSocket adapters along with graceful shutdown behaviour.
- Document how hosts select or implement transports.
9. **Finalize rollout persistence strategy**
- Implement `RolloutSink` adapters (file-based, in-memory, disabled).
- Ensure CLI wires existing recorder; training binary can opt in/out via flags.
10. **Docs & polish**
- Update repository documentation (`README`, architecture docs) to reference the new crates and APIs.
- Record migration notes for downstream consumers.
- Run `just fmt`, scoped `just fix -p`, and targeted tests for touched crates before merging.
11. **Validation**
- Execute `cargo test -p codex-agent`, `cargo test -p codex-core`, and full suite (`cargo test --all-features`) once shared crates change.
- Perform manual verification: CLI session, review task, training binary against mock Response API, ensuring approvals and sandboxing behave identically.

View File

@@ -8,9 +8,9 @@ name = "codex_ansi_escape"
path = "src/lib.rs"
[dependencies]
ansi-to-tui = "7.0.0"
ratatui = { version = "0.29.0", features = [
ansi-to-tui = { workspace = true }
ratatui = { workspace = true, features = [
"unstable-rendered-line-info",
"unstable-widget-ref",
] }
tracing = { version = "0.1.41", features = ["log"] }
tracing = { workspace = true, features = ["log"] }

View File

@@ -15,14 +15,13 @@ path = "src/main.rs"
workspace = true
[dependencies]
anyhow = "1"
similar = "2.7.0"
thiserror = "2.0.16"
tree-sitter = "0.25.9"
tree-sitter-bash = "0.25.0"
once_cell = "1"
anyhow = { workspace = true }
similar = { workspace = true }
thiserror = { workspace = true }
tree-sitter = { workspace = true }
tree-sitter-bash = { workspace = true }
[dev-dependencies]
assert_cmd = "2"
pretty_assertions = "1.4.1"
tempfile = "3.13.0"
assert_cmd = { workspace = true }
pretty_assertions = { workspace = true }
tempfile = { workspace = true }

View File

@@ -6,10 +6,10 @@ use std::collections::HashMap;
use std::path::Path;
use std::path::PathBuf;
use std::str::Utf8Error;
use std::sync::LazyLock;
use anyhow::Context;
use anyhow::Result;
use once_cell::sync::Lazy;
pub use parser::Hunk;
pub use parser::ParseError;
use parser::ParseError::*;
@@ -351,7 +351,7 @@ fn extract_apply_patch_from_bash(
// also run an arbitrary query against the AST. This is useful for understanding
// how tree-sitter parses the script and whether the query syntax is correct. Be sure
// to test both positive and negative cases.
static APPLY_PATCH_QUERY: Lazy<Query> = Lazy::new(|| {
static APPLY_PATCH_QUERY: LazyLock<Query> = LazyLock::new(|| {
let language = BASH.into();
#[expect(clippy::expect_used)]
Query::new(
@@ -648,21 +648,18 @@ fn derive_new_contents_from_chunks(
}
};
let mut original_lines: Vec<String> = original_contents
.split('\n')
.map(|s| s.to_string())
.collect();
let mut original_lines: Vec<String> = original_contents.split('\n').map(String::from).collect();
// Drop the trailing empty element that results from the final newline so
// that line counts match the behaviour of standard `diff`.
if original_lines.last().is_some_and(|s| s.is_empty()) {
if original_lines.last().is_some_and(String::is_empty) {
original_lines.pop();
}
let replacements = compute_replacements(&original_lines, path, chunks)?;
let new_lines = apply_replacements(original_lines, &replacements);
let mut new_lines = new_lines;
if !new_lines.last().is_some_and(|s| s.is_empty()) {
if !new_lines.last().is_some_and(String::is_empty) {
new_lines.push(String::new());
}
let new_contents = new_lines.join("\n");
@@ -706,7 +703,7 @@ fn compute_replacements(
if chunk.old_lines.is_empty() {
// Pure addition (no old lines). We'll add them at the end or just
// before the final empty line if one exists.
let insertion_idx = if original_lines.last().is_some_and(|s| s.is_empty()) {
let insertion_idx = if original_lines.last().is_some_and(String::is_empty) {
original_lines.len() - 1
} else {
original_lines.len()
@@ -732,11 +729,11 @@ fn compute_replacements(
let mut new_slice: &[String] = &chunk.new_lines;
if found.is_none() && pattern.last().is_some_and(|s| s.is_empty()) {
if found.is_none() && pattern.last().is_some_and(String::is_empty) {
// Retry without the trailing empty line which represents the final
// newline in the file.
pattern = &pattern[..pattern.len() - 1];
if new_slice.last().is_some_and(|s| s.is_empty()) {
if new_slice.last().is_some_and(String::is_empty) {
new_slice = &new_slice[..new_slice.len() - 1];
}
@@ -848,6 +845,7 @@ mod tests {
use super::*;
use pretty_assertions::assert_eq;
use std::fs;
use std::string::ToString;
use tempfile::tempdir;
/// Helper to construct a patch with the given body.
@@ -856,7 +854,7 @@ mod tests {
}
fn strs_to_strings(strs: &[&str]) -> Vec<String> {
strs.iter().map(|s| s.to_string()).collect()
strs.iter().map(ToString::to_string).collect()
}
// Test helpers to reduce repetition when building bash -lc heredoc scripts

View File

@@ -112,9 +112,10 @@ pub(crate) fn seek_sequence(
#[cfg(test)]
mod tests {
use super::seek_sequence;
use std::string::ToString;
fn to_vec(strings: &[&str]) -> Vec<String> {
strings.iter().map(|s| s.to_string()).collect()
strings.iter().map(ToString::to_string).collect()
}
#[test]

View File

@@ -11,10 +11,10 @@ path = "src/lib.rs"
workspace = true
[dependencies]
anyhow = "1"
codex-apply-patch = { path = "../apply-patch" }
codex-core = { path = "../core" }
codex-linux-sandbox = { path = "../linux-sandbox" }
dotenvy = "0.15.7"
tempfile = "3"
tokio = { version = "1", features = ["rt-multi-thread"] }
anyhow = { workspace = true }
codex-apply-patch = { workspace = true }
codex-core = { workspace = true }
codex-linux-sandbox = { workspace = true }
dotenvy = { workspace = true }
tempfile = { workspace = true }
tokio = { workspace = true, features = ["rt-multi-thread"] }

View File

@@ -54,7 +54,7 @@ where
let argv1 = args.next().unwrap_or_default();
if argv1 == CODEX_APPLY_PATCH_ARG1 {
let patch_arg = args.next().and_then(|s| s.to_str().map(|s| s.to_owned()));
let patch_arg = args.next().and_then(|s| s.to_str().map(str::to_owned));
let exit_code = match patch_arg {
Some(patch_arg) => {
let mut stdout = std::io::stdout();

View File

@@ -7,13 +7,13 @@ version = { workspace = true }
workspace = true
[dependencies]
anyhow = "1"
clap = { version = "4", features = ["derive"] }
codex-common = { path = "../common", features = ["cli"] }
codex-core = { path = "../core" }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1", features = ["full"] }
anyhow = { workspace = true }
clap = { workspace = true, features = ["derive"] }
codex-common = { workspace = true, features = ["cli"] }
codex-core = { workspace = true }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
tokio = { workspace = true, features = ["full"] }
[dev-dependencies]
tempfile = "3"
tempfile = { workspace = true }

View File

@@ -15,32 +15,44 @@ path = "src/lib.rs"
workspace = true
[dependencies]
anyhow = "1"
clap = { version = "4", features = ["derive"] }
clap_complete = "4"
codex-arg0 = { path = "../arg0" }
codex-chatgpt = { path = "../chatgpt" }
codex-common = { path = "../common", features = ["cli"] }
codex-core = { path = "../core" }
codex-exec = { path = "../exec" }
codex-login = { path = "../login" }
codex-mcp-server = { path = "../mcp-server" }
codex-protocol = { path = "../protocol" }
codex-tui = { path = "../tui" }
serde_json = "1"
tokio = { version = "1", features = [
anyhow = { workspace = true }
clap = { workspace = true, features = ["derive"] }
clap_complete = { workspace = true }
codex-arg0 = { workspace = true }
codex-chatgpt = { workspace = true }
codex-common = { workspace = true, features = ["cli"] }
codex-core = { workspace = true }
codex-exec = { workspace = true }
codex-login = { workspace = true }
codex-mcp-server = { workspace = true }
codex-protocol = { workspace = true }
codex-protocol-ts = { workspace = true }
codex-tui = { workspace = true }
ctor = { workspace = true }
owo-colors = { workspace = true }
serde_json = { workspace = true }
supports-color = { workspace = true }
tokio = { workspace = true, features = [
"io-std",
"macros",
"process",
"rt-multi-thread",
"signal",
] }
tracing = "0.1.41"
tracing-subscriber = "0.3.20"
codex-protocol-ts = { path = "../protocol-ts" }
tracing = { workspace = true }
tracing-subscriber = { workspace = true }
[target.'cfg(target_os = "linux")'.dependencies]
libc = { workspace = true }
[target.'cfg(target_os = "android")'.dependencies]
libc = { workspace = true }
[target.'cfg(target_os = "macos")'.dependencies]
libc = { workspace = true }
[dev-dependencies]
assert_cmd = "2"
predicates = "3"
pretty_assertions = "1"
tempfile = "3"
assert_cmd = { workspace = true }
predicates = { workspace = true }
pretty_assertions = { workspace = true }
tempfile = { workspace = true }

View File

@@ -64,7 +64,6 @@ async fn run_command_under_sandbox(
sandbox_type: SandboxType,
) -> anyhow::Result<()> {
let sandbox_mode = create_sandbox_mode(full_auto);
let cwd = std::env::current_dir()?;
let config = Config::load_with_cli_overrides(
config_overrides
.parse_overrides()
@@ -75,13 +74,29 @@ async fn run_command_under_sandbox(
..Default::default()
},
)?;
// In practice, this should be `std::env::current_dir()` because this CLI
// does not support `--cwd`, but let's use the config value for consistency.
let cwd = config.cwd.clone();
// For now, we always use the same cwd for both the command and the
// sandbox policy. In the future, we could add a CLI option to set them
// separately.
let sandbox_policy_cwd = cwd.clone();
let stdio_policy = StdioPolicy::Inherit;
let env = create_env(&config.shell_environment_policy);
let mut child = match sandbox_type {
SandboxType::Seatbelt => {
spawn_command_under_seatbelt(command, &config.sandbox_policy, cwd, stdio_policy, env)
.await?
spawn_command_under_seatbelt(
command,
cwd,
&config.sandbox_policy,
sandbox_policy_cwd.as_path(),
stdio_policy,
env,
)
.await?
}
SandboxType::Landlock => {
#[expect(clippy::expect_used)]
@@ -91,8 +106,9 @@ async fn run_command_under_sandbox(
spawn_command_under_linux_sandbox(
codex_linux_sandbox_exe,
command,
&config.sandbox_policy,
cwd,
&config.sandbox_policy,
sandbox_policy_cwd.as_path(),
stdio_policy,
env,
)

View File

@@ -14,10 +14,14 @@ use codex_cli::login::run_logout;
use codex_cli::proto;
use codex_common::CliConfigOverrides;
use codex_exec::Cli as ExecCli;
use codex_tui::AppExitInfo;
use codex_tui::Cli as TuiCli;
use owo_colors::OwoColorize;
use std::path::PathBuf;
use supports_color::Stream;
mod mcp_cmd;
mod pre_main_hardening;
use crate::mcp_cmd::McpCli;
use crate::proto::ProtoCli;
@@ -156,6 +160,69 @@ struct GenerateTsCommand {
prettier: Option<PathBuf>,
}
fn format_exit_messages(exit_info: AppExitInfo, color_enabled: bool) -> Vec<String> {
let AppExitInfo {
token_usage,
conversation_id,
} = exit_info;
if token_usage.is_zero() {
return Vec::new();
}
let mut lines = vec![format!(
"{}",
codex_core::protocol::FinalOutput::from(token_usage)
)];
if let Some(session_id) = conversation_id {
let resume_cmd = format!("codex resume {session_id}");
let command = if color_enabled {
resume_cmd.cyan().to_string()
} else {
resume_cmd
};
lines.push(format!("To continue this session, run {command}."));
}
lines
}
fn print_exit_messages(exit_info: AppExitInfo) {
let color_enabled = supports_color::on(Stream::Stdout).is_some();
for line in format_exit_messages(exit_info, color_enabled) {
println!("{line}");
}
}
pub(crate) const CODEX_SECURE_MODE_ENV_VAR: &str = "CODEX_SECURE_MODE";
/// As early as possible in the process lifecycle, apply hardening measures
/// if the CODEX_SECURE_MODE environment variable is set to "1".
#[ctor::ctor]
fn pre_main_hardening() {
let secure_mode = match std::env::var(CODEX_SECURE_MODE_ENV_VAR) {
Ok(value) => value,
Err(_) => return,
};
if secure_mode == "1" {
#[cfg(any(target_os = "linux", target_os = "android"))]
crate::pre_main_hardening::pre_main_hardening_linux();
#[cfg(target_os = "macos")]
crate::pre_main_hardening::pre_main_hardening_macos();
#[cfg(windows)]
crate::pre_main_hardening::pre_main_hardening_windows();
}
// Always clear this env var so child processes don't inherit it.
unsafe {
std::env::remove_var(CODEX_SECURE_MODE_ENV_VAR);
}
}
fn main() -> anyhow::Result<()> {
arg0_dispatch_or_else(|codex_linux_sandbox_exe| async move {
cli_main(codex_linux_sandbox_exe).await?;
@@ -176,10 +243,8 @@ async fn cli_main(codex_linux_sandbox_exe: Option<PathBuf>) -> anyhow::Result<()
&mut interactive.config_overrides,
root_config_overrides.clone(),
);
let usage = codex_tui::run_main(interactive, codex_linux_sandbox_exe).await?;
if !usage.is_zero() {
println!("{}", codex_core::protocol::FinalOutput::from(usage));
}
let exit_info = codex_tui::run_main(interactive, codex_linux_sandbox_exe).await?;
print_exit_messages(exit_info);
}
Some(Subcommand::Exec(mut exec_cli)) => {
prepend_config_flags(
@@ -369,6 +434,8 @@ fn print_completion(cmd: CompletionCommand) {
#[cfg(test)]
mod tests {
use super::*;
use codex_core::protocol::TokenUsage;
use codex_protocol::mcp_protocol::ConversationId;
fn finalize_from_args(args: &[&str]) -> TuiCli {
let cli = MultitoolCli::try_parse_from(args).expect("parse");
@@ -390,6 +457,52 @@ mod tests {
finalize_resume_interactive(interactive, root_overrides, session_id, last, resume_cli)
}
fn sample_exit_info(conversation: Option<&str>) -> AppExitInfo {
let token_usage = TokenUsage {
output_tokens: 2,
total_tokens: 2,
..Default::default()
};
AppExitInfo {
token_usage,
conversation_id: conversation
.map(ConversationId::from_string)
.map(Result::unwrap),
}
}
#[test]
fn format_exit_messages_skips_zero_usage() {
let exit_info = AppExitInfo {
token_usage: TokenUsage::default(),
conversation_id: None,
};
let lines = format_exit_messages(exit_info, false);
assert!(lines.is_empty());
}
#[test]
fn format_exit_messages_includes_resume_hint_without_color() {
let exit_info = sample_exit_info(Some("123e4567-e89b-12d3-a456-426614174000"));
let lines = format_exit_messages(exit_info, false);
assert_eq!(
lines,
vec![
"Token usage: total=2 input=0 output=2".to_string(),
"To continue this session, run codex resume 123e4567-e89b-12d3-a456-426614174000."
.to_string(),
]
);
}
#[test]
fn format_exit_messages_applies_color_when_enabled() {
let exit_info = sample_exit_info(Some("123e4567-e89b-12d3-a456-426614174000"));
let lines = format_exit_messages(exit_info, true);
assert_eq!(lines.len(), 2);
assert!(lines[1].contains("\u{1b}[36m"));
}
#[test]
fn resume_model_flag_applies_when_no_root_flags() {
let interactive = finalize_from_args(["codex", "resume", "-m", "gpt-5-test"].as_ref());

View File

@@ -148,7 +148,8 @@ fn run_add(config_overrides: &CliConfigOverrides, add_args: AddArgs) -> Result<(
command: command_bin,
args: command_args,
env: env_map,
startup_timeout_ms: None,
startup_timeout_sec: None,
tool_timeout_sec: None,
};
servers.insert(name.clone(), new_entry);
@@ -210,7 +211,12 @@ fn run_list(config_overrides: &CliConfigOverrides, list_args: ListArgs) -> Resul
"command": cfg.command,
"args": cfg.args,
"env": env,
"startup_timeout_ms": cfg.startup_timeout_ms,
"startup_timeout_sec": cfg
.startup_timeout_sec
.map(|timeout| timeout.as_secs_f64()),
"tool_timeout_sec": cfg
.tool_timeout_sec
.map(|timeout| timeout.as_secs_f64()),
})
})
.collect();
@@ -305,7 +311,12 @@ fn run_get(config_overrides: &CliConfigOverrides, get_args: GetArgs) -> Result<(
"command": server.command,
"args": server.args,
"env": env,
"startup_timeout_ms": server.startup_timeout_ms,
"startup_timeout_sec": server
.startup_timeout_sec
.map(|timeout| timeout.as_secs_f64()),
"tool_timeout_sec": server
.tool_timeout_sec
.map(|timeout| timeout.as_secs_f64()),
}))?;
println!("{output}");
return Ok(());
@@ -333,8 +344,11 @@ fn run_get(config_overrides: &CliConfigOverrides, get_args: GetArgs) -> Result<(
}
};
println!(" env: {env_display}");
if let Some(timeout) = server.startup_timeout_ms {
println!(" startup_timeout_ms: {timeout}");
if let Some(timeout) = server.startup_timeout_sec {
println!(" startup_timeout_sec: {}", timeout.as_secs_f64());
}
if let Some(timeout) = server.tool_timeout_sec {
println!(" tool_timeout_sec: {}", timeout.as_secs_f64());
}
println!(" remove: codex mcp remove {}", get_args.name);

View File

@@ -0,0 +1,98 @@
#[cfg(any(target_os = "linux", target_os = "android"))]
const PRCTL_FAILED_EXIT_CODE: i32 = 5;
#[cfg(target_os = "macos")]
const PTRACE_DENY_ATTACH_FAILED_EXIT_CODE: i32 = 6;
#[cfg(any(target_os = "linux", target_os = "android", target_os = "macos"))]
const SET_RLIMIT_CORE_FAILED_EXIT_CODE: i32 = 7;
#[cfg(any(target_os = "linux", target_os = "android"))]
pub(crate) fn pre_main_hardening_linux() {
// Disable ptrace attach / mark process non-dumpable.
let ret_code = unsafe { libc::prctl(libc::PR_SET_DUMPABLE, 0, 0, 0, 0) };
if ret_code != 0 {
eprintln!(
"ERROR: prctl(PR_SET_DUMPABLE, 0) failed: {}",
std::io::Error::last_os_error()
);
std::process::exit(PRCTL_FAILED_EXIT_CODE);
}
// For "defense in depth," set the core file size limit to 0.
set_core_file_size_limit_to_zero();
// Official Codex releases are MUSL-linked, which means that variables such
// as LD_PRELOAD are ignored anyway, but just to be sure, clear them here.
let ld_keys: Vec<String> = std::env::vars()
.filter_map(|(key, _)| {
if key.starts_with("LD_") {
Some(key)
} else {
None
}
})
.collect();
for key in ld_keys {
unsafe {
std::env::remove_var(key);
}
}
}
#[cfg(target_os = "macos")]
pub(crate) fn pre_main_hardening_macos() {
// Prevent debuggers from attaching to this process.
let ret_code = unsafe { libc::ptrace(libc::PT_DENY_ATTACH, 0, std::ptr::null_mut(), 0) };
if ret_code == -1 {
eprintln!(
"ERROR: ptrace(PT_DENY_ATTACH) failed: {}",
std::io::Error::last_os_error()
);
std::process::exit(PTRACE_DENY_ATTACH_FAILED_EXIT_CODE);
}
// Set the core file size limit to 0 to prevent core dumps.
set_core_file_size_limit_to_zero();
// Remove all DYLD_ environment variables, which can be used to subvert
// library loading.
let dyld_keys: Vec<String> = std::env::vars()
.filter_map(|(key, _)| {
if key.starts_with("DYLD_") {
Some(key)
} else {
None
}
})
.collect();
for key in dyld_keys {
unsafe {
std::env::remove_var(key);
}
}
}
#[cfg(unix)]
fn set_core_file_size_limit_to_zero() {
let rlim = libc::rlimit {
rlim_cur: 0,
rlim_max: 0,
};
let ret_code = unsafe { libc::setrlimit(libc::RLIMIT_CORE, &rlim) };
if ret_code != 0 {
eprintln!(
"ERROR: setrlimit(RLIMIT_CORE) failed: {}",
std::io::Error::last_os_error()
);
std::process::exit(SET_RLIMIT_CORE_FAILED_EXIT_CODE);
}
}
#[cfg(windows)]
pub(crate) fn pre_main_hardening_windows() {
// TODO(mbolin): Perform the appropriate configuration for Windows.
}

0
codex-rs/code Normal file
View File

View File

@@ -7,11 +7,11 @@ version = { workspace = true }
workspace = true
[dependencies]
clap = { version = "4", features = ["derive", "wrap_help"], optional = true }
codex-core = { path = "../core" }
codex-protocol = { path = "../protocol" }
serde = { version = "1", optional = true }
toml = { version = "0.9", optional = true }
clap = { workspace = true, features = ["derive", "wrap_help"], optional = true }
codex-core = { workspace = true }
codex-protocol = { workspace = true }
serde = { workspace = true, optional = true }
toml = { workspace = true, optional = true }
[features]
# Separate feature so that `clap` is not a mandatory dependency.

View File

@@ -1,4 +1,3 @@
use codex_core::config::GPT_5_CODEX_MEDIUM_MODEL;
use codex_core::protocol_config_types::ReasoningEffort;
use codex_protocol::mcp_protocol::AuthMode;
@@ -69,13 +68,6 @@ const PRESETS: &[ModelPreset] = &[
},
];
pub fn builtin_model_presets(auth_mode: Option<AuthMode>) -> Vec<ModelPreset> {
match auth_mode {
Some(AuthMode::ApiKey) => PRESETS
.iter()
.copied()
.filter(|p| p.model != GPT_5_CODEX_MEDIUM_MODEL)
.collect(),
_ => PRESETS.to_vec(),
}
pub fn builtin_model_presets(_auth_mode: Option<AuthMode>) -> Vec<ModelPreset> {
PRESETS.to_vec()
}

View File

@@ -4,84 +4,90 @@ name = "codex-core"
version = { workspace = true }
[lib]
doctest = false
name = "codex_core"
path = "src/lib.rs"
doctest = false
[lints]
workspace = true
[dependencies]
anyhow = "1"
askama = "0.12"
async-channel = "2.3.1"
base64 = "0.22"
bytes = "1.10.1"
chrono = { version = "0.4", features = ["serde"] }
codex-apply-patch = { path = "../apply-patch" }
codex-file-search = { path = "../file-search" }
codex-mcp-client = { path = "../mcp-client" }
codex-protocol = { path = "../protocol" }
dirs = "6"
env-flags = "0.1.1"
eventsource-stream = "0.2.3"
futures = "0.3"
libc = "0.2.175"
mcp-types = { path = "../mcp-types" }
os_info = "3.12.0"
portable-pty = "0.9.0"
rand = "0.9"
regex-lite = "0.1.7"
reqwest = { version = "0.12", features = ["json", "stream"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
sha1 = "0.10.6"
shlex = "1.3.0"
similar = "2.7.0"
strum_macros = "0.27.2"
tempfile = "3"
thiserror = "2.0.16"
time = { version = "0.3", features = ["formatting", "parsing", "local-offset", "macros"] }
tokio = { version = "1", features = [
anyhow = { workspace = true }
askama = { workspace = true }
async-channel = { workspace = true }
async-trait = { workspace = true }
base64 = { workspace = true }
bytes = { workspace = true }
chrono = { workspace = true, features = ["serde"] }
codex-apply-patch = { workspace = true }
codex-agent = { workspace = true }
codex-file-search = { workspace = true }
codex-mcp-client = { workspace = true }
codex-protocol = { workspace = true }
dirs = { workspace = true }
env-flags = { workspace = true }
eventsource-stream = { workspace = true }
futures = { workspace = true }
indexmap = { workspace = true }
libc = { workspace = true }
mcp-types = { workspace = true }
os_info = { workspace = true }
portable-pty = { workspace = true }
rand = { workspace = true }
regex-lite = { workspace = true }
reqwest = { workspace = true, features = ["json", "stream"] }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
sha1 = { workspace = true }
shlex = { workspace = true }
similar = { workspace = true }
strum_macros = { workspace = true }
tempfile = { workspace = true }
thiserror = { workspace = true }
time = { workspace = true, features = [
"formatting",
"parsing",
"local-offset",
"macros",
] }
tokio = { workspace = true, features = [
"io-std",
"macros",
"process",
"rt-multi-thread",
"signal",
] }
tokio-util = "0.7.16"
toml = "0.9.5"
toml_edit = "0.23.4"
tracing = { version = "0.1.41", features = ["log"] }
tree-sitter = "0.25.9"
tree-sitter-bash = "0.25.0"
uuid = { version = "1", features = ["serde", "v4"] }
which = "6"
wildmatch = "2.5.0"
tokio-util = { workspace = true }
toml = { workspace = true }
toml_edit = { workspace = true }
tracing = { workspace = true, features = ["log"] }
uuid = { workspace = true, features = ["serde", "v4"] }
which = { workspace = true }
wildmatch = { workspace = true }
[target.'cfg(target_os = "linux")'.dependencies]
landlock = "0.4.1"
seccompiler = "0.5.0"
landlock = { workspace = true }
seccompiler = { workspace = true }
# Build OpenSSL from source for musl builds.
[target.x86_64-unknown-linux-musl.dependencies]
openssl-sys = { version = "*", features = ["vendored"] }
openssl-sys = { workspace = true, features = ["vendored"] }
# Build OpenSSL from source for musl builds.
[target.aarch64-unknown-linux-musl.dependencies]
openssl-sys = { version = "*", features = ["vendored"] }
openssl-sys = { workspace = true, features = ["vendored"] }
[dev-dependencies]
assert_cmd = "2"
core_test_support = { path = "tests/common" }
maplit = "1.0.2"
predicates = "3"
pretty_assertions = "1.4.1"
tempfile = "3"
tokio-test = "0.4"
walkdir = "2.5.0"
wiremock = "0.6"
assert_cmd = { workspace = true }
core_test_support = { workspace = true }
maplit = { workspace = true }
predicates = { workspace = true }
pretty_assertions = { workspace = true }
tempfile = { workspace = true }
tokio-test = { workspace = true }
walkdir = { workspace = true }
wiremock = { workspace = true }
[package.metadata.cargo-shear]
ignored = ["openssl-sys"]

View File

@@ -26,37 +26,41 @@ When using the planning tool:
## Codex CLI harness, sandboxing, and approvals
The Codex CLI harness supports several different sandboxing, and approval configurations that the user can choose from.
The Codex CLI harness supports several different configurations for sandboxing and escalation approvals that the user can choose from.
Filesystem sandboxing defines which files can be read or written. The options are:
- **read-only**: You can only read files.
- **workspace-write**: You can read files. You can write to files in this folder, but not outside it.
- **danger-full-access**: No filesystem sandboxing.
Filesystem sandboxing defines which files can be read or written. The options for `sandbox_mode` are:
- **read-only**: The sandbox only permits reading files.
- **workspace-write**: The sandbox permits reading files, and editing files in `cwd` and `writable_roots`. Editing files in other directories requires approval.
- **danger-full-access**: No filesystem sandboxing - all commands are permitted.
Network sandboxing defines whether network can be accessed without approval. Options are
Network sandboxing defines whether network can be accessed without approval. Options for `network_access` are:
- **restricted**: Requires approval
- **enabled**: No approval needed
Approvals are your mechanism to get user consent to perform more privileged actions. Although they introduce friction to the user because your work is paused until the user responds, you should leverage them to accomplish your important work. Do not let these settings or the sandbox deter you from attempting to accomplish the user's task unless it is set to "never", in which case never ask for approvals.
Approval options are
Approvals are your mechanism to get user consent to run shell commands without the sandbox. Possible configuration options for `approval_policy` are
- **untrusted**: The harness will escalate most commands for user approval, apart from a limited allowlist of safe "read" commands.
- **on-failure**: The harness will allow all commands to run in the sandbox (if enabled), and failures will be escalated to the user for approval to run again without the sandbox.
- **on-request**: Commands will be run in the sandbox by default, and you can specify in your tool call if you want to escalate a command to run without sandboxing. (Note that this mode is not always available. If it is, you'll see parameters for it in the `shell` command description.)
- **never**: This is a non-interactive mode where you may NEVER ask the user for approval to run commands. Instead, you must always persist and work around constraints to solve the task for the user. You MUST do your utmost best to finish the task and validate your work before yielding. If this mode is paired with `danger-full-access`, take advantage of it to deliver the best outcome for the user. Further, in this mode, your default testing philosophy is overridden: Even if you don't see local patterns for testing, you may add tests and scripts to validate your work. Just remove them before yielding.
When you are running with approvals `on-request`, and sandboxing enabled, here are scenarios where you'll need to request approval:
- You need to run a command that writes to a directory that requires it (e.g. running tests that write to /tmp)
When you are running with `approval_policy == on-request`, and sandboxing enabled, here are scenarios where you'll need to request approval:
- You need to run a command that writes to a directory that requires it (e.g. running tests that write to /var)
- You need to run a GUI app (e.g., open/xdg-open/osascript) to open browsers or files.
- You are running sandboxed and need to run a command that requires network access (e.g. installing packages)
- If you run a command that is important to solving the user's query, but it fails because of sandboxing, rerun the command with approval.
- If you run a command that is important to solving the user's query, but it fails because of sandboxing, rerun the command with approval. ALWAYS proceed to use the `with_escalated_permissions` and `justification` parameters - do not message the user before requesting approval for the command.
- You are about to take a potentially destructive action such as an `rm` or `git reset` that the user did not explicitly ask for
- (for all of these, you should weigh alternative paths that do not require approval)
When sandboxing is set to read-only, you'll need to request approval for any command that isn't a read.
When `sandbox_mode` is set to read-only, you'll need to request approval for any command that isn't a read.
You will be told what filesystem sandboxing, network sandboxing, and approval mode are active in a developer or user message. If you are not told about this, assume that you are running with workspace-write, network sandboxing enabled, and approval on-failure.
Although they introduce friction to the user because your work is paused until the user responds, you should leverage them when necessary to accomplish important work. If the completing the task requires escalated permissions, Do not let these settings or the sandbox deter you from attempting to accomplish the user's task unless it is set to "never", in which case never ask for approvals.
When requesting approval to execute a command that will require escalated privileges:
- Provide the `with_escalated_permissions` parameter with the boolean value true
- Include a short, 1 sentence explanation for why you need to enable `with_escalated_permissions` in the justification parameter
## Special user requests
- If the user makes a simple request (such as asking for the time) which you can fulfill by running a terminal command (such as `date`), you should do so.

View File

@@ -0,0 +1,38 @@
pub use codex_agent::AgentConfig;
use crate::config::Config;
impl From<&Config> for AgentConfig {
fn from(config: &Config) -> Self {
Self {
model: config.model.clone(),
review_model: config.review_model.clone(),
model_family: config.model_family.clone(),
model_context_window: config.model_context_window,
model_auto_compact_token_limit: config.model_auto_compact_token_limit,
model_reasoning_effort: config.model_reasoning_effort,
model_reasoning_summary: config.model_reasoning_summary,
model_verbosity: config.model_verbosity,
model_provider: config.model_provider.clone(),
approval_policy: config.approval_policy,
sandbox_policy: config.sandbox_policy.clone(),
shell_environment_policy: config.shell_environment_policy.clone(),
user_instructions: config.user_instructions.clone(),
base_instructions: config.base_instructions.clone(),
notify: config.notify.clone(),
cwd: config.cwd.clone(),
codex_home: config.codex_home.clone(),
history: config.history.clone(),
mcp_servers: config.mcp_servers.clone(),
include_plan_tool: config.include_plan_tool,
include_apply_patch_tool: config.include_apply_patch_tool,
include_view_image_tool: config.include_view_image_tool,
tools_web_search_request: config.tools_web_search_request,
use_experimental_streamable_shell_tool: config.use_experimental_streamable_shell_tool,
use_experimental_unified_exec_tool: config.use_experimental_unified_exec_tool,
show_raw_agent_reasoning: config.show_raw_agent_reasoning,
codex_linux_sandbox_exe: config.codex_linux_sandbox_exe.clone(),
project_doc_max_bytes: config.project_doc_max_bytes,
}
}
}

View File

@@ -0,0 +1,148 @@
use std::collections::HashMap;
use std::path::PathBuf;
use std::sync::Arc;
use anyhow::Result;
use async_trait::async_trait;
use codex_agent::notifications::UserNotification;
use codex_agent::services::CredentialsProvider;
use codex_agent::services::McpInterface;
use codex_agent::services::Notifier;
use codex_agent::services::ProviderAuth;
use codex_agent::services::SandboxManager;
use codex_agent::token_data::PlanType;
use codex_protocol::mcp_protocol::AuthMode;
use mcp_types::CallToolResult;
use mcp_types::Tool;
use serde_json::Value;
use crate::auth::AuthManager;
use crate::auth::CodexAuth;
use crate::exec_command::ExecCommandOutput;
use crate::exec_command::ExecCommandParams;
use crate::exec_command::ExecSessionManager;
use crate::exec_command::WriteStdinParams;
use crate::mcp_connection_manager::McpConnectionManager;
use crate::unified_exec::UnifiedExecError;
use crate::unified_exec::UnifiedExecRequest;
use crate::unified_exec::UnifiedExecResult;
use crate::unified_exec::UnifiedExecSessionManager;
use crate::user_notification::UserNotifier;
#[async_trait]
impl ProviderAuth for CodexAuth {
fn mode(&self) -> AuthMode {
self.mode
}
async fn access_token(&self) -> std::io::Result<String> {
self.get_token().await
}
fn account_id(&self) -> Option<String> {
self.get_account_id()
}
fn plan_type(&self) -> Option<PlanType> {
self.get_plan_type()
}
}
#[async_trait]
impl CredentialsProvider for AuthManager {
fn auth(&self) -> Option<Arc<dyn ProviderAuth>> {
AuthManager::auth(self).map(|auth| Arc::new(auth) as Arc<dyn ProviderAuth>)
}
async fn refresh_token(&self) -> std::io::Result<Option<String>> {
AuthManager::refresh_token(self).await
}
}
impl Notifier for UserNotifier {
fn notify(&self, notification: &UserNotification) {
UserNotifier::notify(self, notification);
}
}
#[async_trait]
impl McpInterface for McpConnectionManager {
fn list_all_tools(&self) -> HashMap<String, Tool> {
McpConnectionManager::list_all_tools(self)
}
fn parse_tool_name(&self, tool_name: &str) -> Option<(String, String)> {
McpConnectionManager::parse_tool_name(self, tool_name)
}
async fn call_tool(
&self,
server: &str,
tool: &str,
arguments: Option<Value>,
) -> Result<CallToolResult> {
McpConnectionManager::call_tool(self, server, tool, arguments).await
}
}
/// Default [`SandboxManager`] used by the CLI runtime. Wraps the existing exec
/// session managers and exposes their functionality via the trait-based
/// interface so other hosts can substitute different implementations.
pub struct DefaultSandboxManager {
exec_session_manager: ExecSessionManager,
unified_exec_manager: UnifiedExecSessionManager,
codex_linux_sandbox_exe: Option<PathBuf>,
user_shell: crate::shell::Shell,
}
impl DefaultSandboxManager {
pub fn new(
exec_session_manager: ExecSessionManager,
unified_exec_manager: UnifiedExecSessionManager,
codex_linux_sandbox_exe: Option<PathBuf>,
user_shell: crate::shell::Shell,
) -> Self {
Self {
exec_session_manager,
unified_exec_manager,
codex_linux_sandbox_exe,
user_shell,
}
}
}
#[async_trait]
impl SandboxManager for DefaultSandboxManager {
async fn handle_exec_command_request(
&self,
params: ExecCommandParams,
) -> Result<ExecCommandOutput, String> {
self.exec_session_manager
.handle_exec_command_request(params)
.await
}
async fn handle_write_stdin_request(
&self,
params: WriteStdinParams,
) -> Result<ExecCommandOutput, String> {
self.exec_session_manager
.handle_write_stdin_request(params)
.await
}
async fn handle_unified_exec_request(
&self,
request: UnifiedExecRequest<'_>,
) -> Result<UnifiedExecResult, UnifiedExecError> {
self.unified_exec_manager.handle_request(request).await
}
fn codex_linux_sandbox_exe(&self) -> &Option<PathBuf> {
&self.codex_linux_sandbox_exe
}
fn user_shell(&self) -> &crate::shell::Shell {
&self.user_shell
}
}

View File

@@ -135,7 +135,7 @@ impl CodexAuth {
self.get_current_token_data().and_then(|t| t.account_id)
}
pub(crate) fn get_plan_type(&self) -> Option<PlanType> {
pub fn get_plan_type(&self) -> Option<PlanType> {
self.get_current_token_data()
.and_then(|t| t.id_token.chatgpt_plan_type)
}
@@ -267,6 +267,9 @@ pub fn try_read_auth_json(auth_file: &Path) -> std::io::Result<AuthDotJson> {
}
pub fn write_auth_json(auth_file: &Path, auth_dot_json: &AuthDotJson) -> std::io::Result<()> {
if let Some(parent) = auth_file.parent() {
std::fs::create_dir_all(parent)?;
}
let json_data = serde_json::to_string_pretty(auth_dot_json)?;
let mut options = OpenOptions::new();
options.truncate(true).write(true).create(true);

View File

@@ -22,6 +22,7 @@ use crate::client_common::ResponseStream;
use crate::error::CodexErr;
use crate::error::Result;
use crate::model_family::ModelFamily;
use crate::model_provider_info::ModelProviderExt;
use crate::openai_tools::create_tools_json_for_chat_completions_api;
use crate::util::backoff;
use codex_protocol::models::ContentItem;
@@ -35,6 +36,12 @@ pub(crate) async fn stream_chat_completions(
client: &reqwest::Client,
provider: &ModelProviderInfo,
) -> Result<ResponseStream> {
if prompt.output_schema.is_some() {
return Err(CodexErr::UnsupportedOperation(
"output_schema is not supported for Chat Completions API".to_string(),
));
}
// Build messages array
let mut messages = Vec::<serde_json::Value>::new();
@@ -96,7 +103,7 @@ pub(crate) async fn stream_chat_completions(
for c in items {
match c {
ReasoningItemContent::ReasoningText { text: t }
| ReasoningItemContent::Text { text: t } => text.push_str(t),
| ReasoningItemContent::Text { text: t } => text.push_str(t.as_str()),
}
}
if text.trim().is_empty() {
@@ -151,7 +158,7 @@ pub(crate) async fn stream_chat_completions(
match c {
ContentItem::InputText { text: t }
| ContentItem::OutputText { text: t } => {
text.push_str(t);
text.push_str(t.as_str());
}
_ => {}
}
@@ -462,7 +469,7 @@ async fn process_chat_sse<S>(
if let Some(reasoning_val) = choice.get("delta").and_then(|d| d.get("reasoning")) {
let mut maybe_text = reasoning_val
.as_str()
.map(|s| s.to_string())
.map(str::to_string)
.filter(|s| !s.is_empty());
if maybe_text.is_none() && reasoning_val.is_object() {
@@ -716,6 +723,9 @@ where
// Not an assistant message forward immediately.
return Poll::Ready(Some(Ok(ResponseEvent::OutputItemDone(item))));
}
Poll::Ready(Some(Ok(ResponseEvent::RateLimits(snapshot)))) => {
return Poll::Ready(Some(Ok(ResponseEvent::RateLimits(snapshot))));
}
Poll::Ready(Some(Ok(ResponseEvent::Completed {
response_id,
token_usage,

View File

@@ -1,9 +1,10 @@
use std::fmt;
use std::io::BufRead;
use std::path::Path;
use std::sync::OnceLock;
use std::time::Duration;
use crate::AuthManager;
use crate::CredentialsProvider;
use bytes::Bytes;
use codex_protocol::mcp_protocol::AuthMode;
use codex_protocol::mcp_protocol::ConversationId;
@@ -11,6 +12,7 @@ use eventsource_stream::Eventsource;
use futures::prelude::*;
use regex_lite::Regex;
use reqwest::StatusCode;
use reqwest::header::HeaderMap;
use serde::Deserialize;
use serde::Serialize;
use serde_json::Value;
@@ -21,6 +23,9 @@ use tracing::debug;
use tracing::trace;
use tracing::warn;
use crate::ModelProviderInfo;
use crate::WireApi;
use crate::agent_config::AgentConfig;
use crate::chat_completions::AggregateStreamExt;
use crate::chat_completions::stream_chat_completions;
use crate::client_common::Prompt;
@@ -29,17 +34,17 @@ use crate::client_common::ResponseStream;
use crate::client_common::ResponsesApiRequest;
use crate::client_common::create_reasoning_param_for_request;
use crate::client_common::create_text_param_for_request;
use crate::config::Config;
use crate::default_client::create_client;
use crate::error::CodexErr;
use crate::error::Result;
use crate::error::UsageLimitReachedError;
use crate::flags::CODEX_RS_SSE_FIXTURE;
use crate::model_family::ModelFamily;
use crate::model_provider_info::ModelProviderInfo;
use crate::model_provider_info::WireApi;
use crate::model_provider_info::ModelProviderExt;
use crate::openai_model_info::get_model_info;
use crate::openai_tools::create_tools_json_for_responses_api;
use crate::protocol::RateLimitSnapshot;
use crate::protocol::RateLimitWindow;
use crate::protocol::TokenUsage;
use crate::token_data::PlanType;
use crate::util::backoff;
@@ -65,10 +70,10 @@ struct Error {
resets_in_seconds: Option<u64>,
}
#[derive(Debug, Clone)]
#[derive(Clone)]
pub struct ModelClient {
config: Arc<Config>,
auth_manager: Option<Arc<AuthManager>>,
config: Arc<AgentConfig>,
auth_manager: Option<Arc<dyn CredentialsProvider>>,
client: reqwest::Client,
provider: ModelProviderInfo,
conversation_id: ConversationId,
@@ -76,10 +81,22 @@ pub struct ModelClient {
summary: ReasoningSummaryConfig,
}
impl fmt::Debug for ModelClient {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
f.debug_struct("ModelClient")
.field("config", &self.config)
.field("provider", &self.provider)
.field("conversation_id", &self.conversation_id)
.field("effort", &self.effort)
.field("summary", &self.summary)
.finish()
}
}
impl ModelClient {
pub fn new(
config: Arc<Config>,
auth_manager: Option<Arc<AuthManager>>,
config: Arc<AgentConfig>,
auth_manager: Option<Arc<dyn CredentialsProvider>>,
provider: ModelProviderInfo,
effort: Option<ReasoningEffortConfig>,
summary: ReasoningSummaryConfig,
@@ -180,19 +197,23 @@ impl ModelClient {
let input_with_instructions = prompt.get_formatted_input();
// Only include `text.verbosity` for GPT-5 family models
let text = if self.config.model_family.family == "gpt-5" {
create_text_param_for_request(self.config.model_verbosity)
} else {
if self.config.model_verbosity.is_some() {
warn!(
"model_verbosity is set but ignored for non-gpt-5 model family: {}",
self.config.model_family.family
);
let verbosity = match &self.config.model_family.family {
family if family == "gpt-5" => self.config.model_verbosity,
_ => {
if self.config.model_verbosity.is_some() {
warn!(
"model_verbosity is set but ignored for non-gpt-5 model family: {}",
self.config.model_family.family
);
}
None
}
None
};
// Only include `text.verbosity` for GPT-5 family models
let text = create_text_param_for_request(verbosity, &prompt.output_schema);
// In general, we want to explicitly send `store: false` when using the Responses API,
// but in practice, the Azure Responses API rejects `store: false`:
//
@@ -221,144 +242,169 @@ impl ModelClient {
if azure_workaround {
attach_item_ids(&mut payload_json, &input_with_instructions);
}
let payload_body = serde_json::to_string(&payload_json)?;
let mut attempt = 0;
let max_retries = self.provider.request_max_retries();
loop {
attempt += 1;
// Always fetch the latest auth in case a prior attempt refreshed the token.
let auth = auth_manager.as_ref().and_then(|m| m.auth());
trace!(
"POST to {}: {}",
self.provider.get_full_url(&auth),
payload_body.as_str()
);
let mut req_builder = self
.provider
.create_request_builder(&self.client, &auth)
.await?;
req_builder = req_builder
.header("OpenAI-Beta", "responses=experimental")
// Send session_id for compatibility.
.header("conversation_id", self.conversation_id.to_string())
.header("session_id", self.conversation_id.to_string())
.header(reqwest::header::ACCEPT, "text/event-stream")
.json(&payload_json);
if let Some(auth) = auth.as_ref()
&& auth.mode == AuthMode::ChatGPT
&& let Some(account_id) = auth.get_account_id()
let max_attempts = self.provider.request_max_retries();
for attempt in 0..=max_attempts {
match self
.attempt_stream_responses(&payload_json, &auth_manager)
.await
{
req_builder = req_builder.header("chatgpt-account-id", account_id);
}
let res = req_builder.send().await;
if let Ok(resp) = &res {
trace!(
"Response status: {}, cf-ray: {}",
resp.status(),
resp.headers()
.get("cf-ray")
.map(|v| v.to_str().unwrap_or_default())
.unwrap_or_default()
);
}
match res {
Ok(resp) if resp.status().is_success() => {
let (tx_event, rx_event) = mpsc::channel::<Result<ResponseEvent>>(1600);
// spawn task to process SSE
let stream = resp.bytes_stream().map_err(CodexErr::Reqwest);
tokio::spawn(process_sse(
stream,
tx_event,
self.provider.stream_idle_timeout(),
));
return Ok(ResponseStream { rx_event });
Ok(stream) => {
return Ok(stream);
}
Ok(res) => {
let status = res.status();
// Pull out RetryAfter header if present.
let retry_after_secs = res
.headers()
.get(reqwest::header::RETRY_AFTER)
.and_then(|v| v.to_str().ok())
.and_then(|s| s.parse::<u64>().ok());
if status == StatusCode::UNAUTHORIZED
&& let Some(manager) = auth_manager.as_ref()
&& manager.auth().is_some()
{
let _ = manager.refresh_token().await;
Err(StreamAttemptError::Fatal(e)) => {
return Err(e);
}
Err(retryable_attempt_error) => {
if attempt == max_attempts {
return Err(retryable_attempt_error.into_error());
}
// The OpenAI Responses endpoint returns structured JSON bodies even for 4xx/5xx
// errors. When we bubble early with only the HTTP status the caller sees an opaque
// "unexpected status 400 Bad Request" which makes debugging nearly impossible.
// Instead, read (and include) the response text so higher layers and users see the
// exact error message (e.g. "Unknown parameter: 'input[0].metadata'"). The body is
// small and this branch only runs on error paths so the extra allocation is
// negligible.
if !(status == StatusCode::TOO_MANY_REQUESTS
|| status == StatusCode::UNAUTHORIZED
|| status.is_server_error())
{
// Surface the error body to callers. Use `unwrap_or_default` per Clippy.
let body = res.text().await.unwrap_or_default();
return Err(CodexErr::UnexpectedStatus(status, body));
}
tokio::time::sleep(retryable_attempt_error.delay(attempt)).await;
}
}
}
if status == StatusCode::TOO_MANY_REQUESTS {
let body = res.json::<ErrorResponse>().await.ok();
if let Some(ErrorResponse { error }) = body {
if error.r#type.as_deref() == Some("usage_limit_reached") {
// Prefer the plan_type provided in the error message if present
// because it's more up to date than the one encoded in the auth
// token.
let plan_type = error
.plan_type
.or_else(|| auth.as_ref().and_then(|a| a.get_plan_type()));
let resets_in_seconds = error.resets_in_seconds;
return Err(CodexErr::UsageLimitReached(UsageLimitReachedError {
plan_type,
resets_in_seconds,
}));
} else if error.r#type.as_deref() == Some("usage_not_included") {
return Err(CodexErr::UsageNotIncluded);
}
unreachable!("stream_responses_attempt should always return");
}
/// Single attempt to start a streaming Responses API call.
async fn attempt_stream_responses(
&self,
payload_json: &Value,
auth_manager: &Option<Arc<dyn CredentialsProvider>>,
) -> std::result::Result<ResponseStream, StreamAttemptError> {
// Always fetch the latest auth in case a prior attempt refreshed the token.
let auth = auth_manager.as_ref().and_then(|m| m.auth());
trace!(
"POST to {}: {:?}",
self.provider.get_full_url(&auth),
serde_json::to_string(payload_json)
);
let mut req_builder = self
.provider
.create_request_builder(&self.client, &auth)
.await
.map_err(StreamAttemptError::Fatal)?;
req_builder = req_builder
.header("OpenAI-Beta", "responses=experimental")
// Send session_id for compatibility.
.header("conversation_id", self.conversation_id.to_string())
.header("session_id", self.conversation_id.to_string())
.header(reqwest::header::ACCEPT, "text/event-stream")
.json(payload_json);
if let Some(auth) = auth.as_ref()
&& auth.mode() == AuthMode::ChatGPT
&& let Some(account_id) = auth.account_id()
{
req_builder = req_builder.header("chatgpt-account-id", account_id);
}
let res = req_builder.send().await;
if let Ok(resp) = &res {
trace!(
"Response status: {}, cf-ray: {}",
resp.status(),
resp.headers()
.get("cf-ray")
.map(|v| v.to_str().unwrap_or_default())
.unwrap_or_default()
);
}
match res {
Ok(resp) if resp.status().is_success() => {
let (tx_event, rx_event) = mpsc::channel::<Result<ResponseEvent>>(1600);
if let Some(snapshot) = parse_rate_limit_snapshot(resp.headers())
&& tx_event
.send(Ok(ResponseEvent::RateLimits(snapshot)))
.await
.is_err()
{
debug!("receiver dropped rate limit snapshot event");
}
// spawn task to process SSE
let stream = resp.bytes_stream().map_err(CodexErr::Reqwest);
tokio::spawn(process_sse(
stream,
tx_event,
self.provider.stream_idle_timeout(),
));
Ok(ResponseStream { rx_event })
}
Ok(res) => {
let status = res.status();
// Pull out RetryAfter header if present.
let retry_after_secs = res
.headers()
.get(reqwest::header::RETRY_AFTER)
.and_then(|v| v.to_str().ok())
.and_then(|s| s.parse::<u64>().ok());
let retry_after = retry_after_secs.map(|s| Duration::from_millis(s * 1_000));
if status == StatusCode::UNAUTHORIZED
&& let Some(manager) = auth_manager.as_ref()
&& manager.auth().is_some()
{
let _ = manager.refresh_token().await;
}
// The OpenAI Responses endpoint returns structured JSON bodies even for 4xx/5xx
// errors. When we bubble early with only the HTTP status the caller sees an opaque
// "unexpected status 400 Bad Request" which makes debugging nearly impossible.
// Instead, read (and include) the response text so higher layers and users see the
// exact error message (e.g. "Unknown parameter: 'input[0].metadata'"). The body is
// small and this branch only runs on error paths so the extra allocation is
// negligible.
if !(status == StatusCode::TOO_MANY_REQUESTS
|| status == StatusCode::UNAUTHORIZED
|| status.is_server_error())
{
// Surface the error body to callers. Use `unwrap_or_default` per Clippy.
let body = res.text().await.unwrap_or_default();
return Err(StreamAttemptError::Fatal(CodexErr::UnexpectedStatus(
status, body,
)));
}
if status == StatusCode::TOO_MANY_REQUESTS {
let rate_limit_snapshot = parse_rate_limit_snapshot(res.headers());
let body = res.json::<ErrorResponse>().await.ok();
if let Some(ErrorResponse { error }) = body {
if error.r#type.as_deref() == Some("usage_limit_reached") {
// Prefer the plan_type provided in the error message if present
// because it's more up to date than the one encoded in the auth
// token.
let plan_type = error
.plan_type
.or_else(|| auth.as_ref().and_then(|a| a.plan_type()));
let resets_in_seconds = error.resets_in_seconds;
let codex_err = CodexErr::UsageLimitReached(UsageLimitReachedError {
plan_type,
resets_in_seconds,
rate_limits: rate_limit_snapshot,
});
return Err(StreamAttemptError::Fatal(codex_err));
} else if error.r#type.as_deref() == Some("usage_not_included") {
return Err(StreamAttemptError::Fatal(CodexErr::UsageNotIncluded));
}
}
if attempt > max_retries {
if status == StatusCode::INTERNAL_SERVER_ERROR {
return Err(CodexErr::InternalServerError);
}
return Err(CodexErr::RetryLimit(status));
}
let delay = retry_after_secs
.map(|s| Duration::from_millis(s * 1_000))
.unwrap_or_else(|| backoff(attempt));
tokio::time::sleep(delay).await;
}
Err(e) => {
if attempt > max_retries {
return Err(e.into());
}
let delay = backoff(attempt);
tokio::time::sleep(delay).await;
}
Err(StreamAttemptError::RetryableHttpError {
status,
retry_after,
})
}
Err(e) => Err(StreamAttemptError::RetryableTransportError(e.into())),
}
}
@@ -386,11 +432,52 @@ impl ModelClient {
self.summary
}
pub fn get_auth_manager(&self) -> Option<Arc<AuthManager>> {
pub fn get_auth_manager(&self) -> Option<Arc<dyn CredentialsProvider>> {
self.auth_manager.clone()
}
}
enum StreamAttemptError {
RetryableHttpError {
status: StatusCode,
retry_after: Option<Duration>,
},
RetryableTransportError(CodexErr),
Fatal(CodexErr),
}
impl StreamAttemptError {
/// attempt is 0-based.
fn delay(&self, attempt: u64) -> Duration {
// backoff() uses 1-based attempts.
let backoff_attempt = attempt + 1;
match self {
Self::RetryableHttpError { retry_after, .. } => {
retry_after.unwrap_or_else(|| backoff(backoff_attempt))
}
Self::RetryableTransportError { .. } => backoff(backoff_attempt),
Self::Fatal(_) => {
// Should not be called on Fatal errors.
Duration::from_secs(0)
}
}
}
fn into_error(self) -> CodexErr {
match self {
Self::RetryableHttpError { status, .. } => {
if status == StatusCode::INTERNAL_SERVER_ERROR {
CodexErr::InternalServerError
} else {
CodexErr::RetryLimit(status)
}
}
Self::RetryableTransportError(error) => error,
Self::Fatal(error) => error,
}
}
}
#[derive(Debug, Deserialize, Serialize)]
struct SseEvent {
#[serde(rename = "type")]
@@ -400,9 +487,6 @@ struct SseEvent {
delta: Option<String>,
}
#[derive(Debug, Deserialize)]
struct ResponseCreated {}
#[derive(Debug, Deserialize)]
struct ResponseCompleted {
id: String,
@@ -473,6 +557,67 @@ fn attach_item_ids(payload_json: &mut Value, original_items: &[ResponseItem]) {
}
}
fn parse_rate_limit_snapshot(headers: &HeaderMap) -> Option<RateLimitSnapshot> {
let primary = parse_rate_limit_window(
headers,
"x-codex-primary-used-percent",
"x-codex-primary-window-minutes",
"x-codex-primary-reset-after-seconds",
);
let secondary = parse_rate_limit_window(
headers,
"x-codex-secondary-used-percent",
"x-codex-secondary-window-minutes",
"x-codex-secondary-reset-after-seconds",
);
if primary.is_none() && secondary.is_none() {
return None;
}
Some(RateLimitSnapshot { primary, secondary })
}
fn parse_rate_limit_window(
headers: &HeaderMap,
used_percent_header: &str,
window_minutes_header: &str,
resets_header: &str,
) -> Option<RateLimitWindow> {
let used_percent: Option<f64> = parse_header_f64(headers, used_percent_header);
used_percent.and_then(|used_percent| {
let window_minutes = parse_header_u64(headers, window_minutes_header);
let resets_in_seconds = parse_header_u64(headers, resets_header);
let has_data = used_percent != 0.0
|| window_minutes.is_some_and(|minutes| minutes != 0)
|| resets_in_seconds.is_some_and(|seconds| seconds != 0);
has_data.then_some(RateLimitWindow {
used_percent,
window_minutes,
resets_in_seconds,
})
})
}
fn parse_header_f64(headers: &HeaderMap, name: &str) -> Option<f64> {
parse_header_str(headers, name)?
.parse::<f64>()
.ok()
.filter(|v| v.is_finite())
}
fn parse_header_u64(headers: &HeaderMap, name: &str) -> Option<u64> {
parse_header_str(headers, name)?.parse::<u64>().ok()
}
fn parse_header_str<'a>(headers: &'a HeaderMap, name: &str) -> Option<&'a str> {
headers.get(name)?.to_str().ok()
}
async fn process_sse<S>(
stream: S,
tx_event: mpsc::Sender<Result<ResponseEvent>>,

View File

@@ -1,291 +1,5 @@
use crate::error::Result;
use crate::model_family::ModelFamily;
use crate::openai_tools::OpenAiTool;
use crate::protocol::TokenUsage;
use codex_apply_patch::APPLY_PATCH_TOOL_INSTRUCTIONS;
use codex_protocol::config_types::ReasoningEffort as ReasoningEffortConfig;
use codex_protocol::config_types::ReasoningSummary as ReasoningSummaryConfig;
use codex_protocol::config_types::Verbosity as VerbosityConfig;
use codex_protocol::models::ResponseItem;
use futures::Stream;
use serde::Serialize;
use std::borrow::Cow;
use std::ops::Deref;
use std::pin::Pin;
use std::task::Context;
use std::task::Poll;
use tokio::sync::mpsc;
pub use codex_agent::client_common::*;
/// Review thread system prompt. Edit `core/src/review_prompt.md` to customize.
pub const REVIEW_PROMPT: &str = include_str!("../review_prompt.md");
use crate::error::CodexErr;
/// API request payload for a single model turn
#[derive(Default, Debug, Clone)]
pub struct Prompt {
/// Conversation context input items.
pub input: Vec<ResponseItem>,
/// Tools available to the model, including additional tools sourced from
/// external MCP servers.
pub(crate) tools: Vec<OpenAiTool>,
/// Optional override for the built-in BASE_INSTRUCTIONS.
pub base_instructions_override: Option<String>,
}
impl Prompt {
pub(crate) fn get_full_instructions(&self, model: &ModelFamily) -> Cow<'_, str> {
let base = self
.base_instructions_override
.as_deref()
.unwrap_or(model.base_instructions.deref());
let mut sections: Vec<&str> = vec![base];
// When there are no custom instructions, add apply_patch_tool_instructions if:
// - the model needs special instructions (4.1)
// AND
// - there is no apply_patch tool present
let is_apply_patch_tool_present = self.tools.iter().any(|tool| match tool {
OpenAiTool::Function(f) => f.name == "apply_patch",
OpenAiTool::Freeform(f) => f.name == "apply_patch",
_ => false,
});
if self.base_instructions_override.is_none()
&& model.needs_special_apply_patch_instructions
&& !is_apply_patch_tool_present
{
sections.push(APPLY_PATCH_TOOL_INSTRUCTIONS);
}
Cow::Owned(sections.join("\n"))
}
pub(crate) fn get_formatted_input(&self) -> Vec<ResponseItem> {
self.input.clone()
}
}
#[derive(Debug)]
pub enum ResponseEvent {
Created,
OutputItemDone(ResponseItem),
Completed {
response_id: String,
token_usage: Option<TokenUsage>,
},
OutputTextDelta(String),
ReasoningSummaryDelta(String),
ReasoningContentDelta(String),
ReasoningSummaryPartAdded,
WebSearchCallBegin {
call_id: String,
},
}
#[derive(Debug, Serialize)]
pub(crate) struct Reasoning {
#[serde(skip_serializing_if = "Option::is_none")]
pub(crate) effort: Option<ReasoningEffortConfig>,
#[serde(skip_serializing_if = "Option::is_none")]
pub(crate) summary: Option<ReasoningSummaryConfig>,
}
/// Controls under the `text` field in the Responses API for GPT-5.
#[derive(Debug, Serialize, Default, Clone, Copy)]
pub(crate) struct TextControls {
#[serde(skip_serializing_if = "Option::is_none")]
pub(crate) verbosity: Option<OpenAiVerbosity>,
}
#[derive(Debug, Serialize, Default, Clone, Copy)]
#[serde(rename_all = "lowercase")]
pub(crate) enum OpenAiVerbosity {
Low,
#[default]
Medium,
High,
}
impl From<VerbosityConfig> for OpenAiVerbosity {
fn from(v: VerbosityConfig) -> Self {
match v {
VerbosityConfig::Low => OpenAiVerbosity::Low,
VerbosityConfig::Medium => OpenAiVerbosity::Medium,
VerbosityConfig::High => OpenAiVerbosity::High,
}
}
}
/// Request object that is serialized as JSON and POST'ed when using the
/// Responses API.
#[derive(Debug, Serialize)]
pub(crate) struct ResponsesApiRequest<'a> {
pub(crate) model: &'a str,
pub(crate) instructions: &'a str,
// TODO(mbolin): ResponseItem::Other should not be serialized. Currently,
// we code defensively to avoid this case, but perhaps we should use a
// separate enum for serialization.
pub(crate) input: &'a Vec<ResponseItem>,
pub(crate) tools: &'a [serde_json::Value],
pub(crate) tool_choice: &'static str,
pub(crate) parallel_tool_calls: bool,
pub(crate) reasoning: Option<Reasoning>,
pub(crate) store: bool,
pub(crate) stream: bool,
pub(crate) include: Vec<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub(crate) prompt_cache_key: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub(crate) text: Option<TextControls>,
}
pub(crate) fn create_reasoning_param_for_request(
model_family: &ModelFamily,
effort: Option<ReasoningEffortConfig>,
summary: ReasoningSummaryConfig,
) -> Option<Reasoning> {
if !model_family.supports_reasoning_summaries {
return None;
}
Some(Reasoning {
effort,
summary: Some(summary),
})
}
pub(crate) fn create_text_param_for_request(
verbosity: Option<VerbosityConfig>,
) -> Option<TextControls> {
verbosity.map(|v| TextControls {
verbosity: Some(v.into()),
})
}
pub struct ResponseStream {
pub(crate) rx_event: mpsc::Receiver<Result<ResponseEvent>>,
}
impl Stream for ResponseStream {
type Item = Result<ResponseEvent>;
fn poll_next(mut self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Option<Self::Item>> {
self.rx_event.poll_recv(cx)
}
}
#[cfg(test)]
mod tests {
use crate::model_family::find_family_for_model;
use pretty_assertions::assert_eq;
use super::*;
struct InstructionsTestCase {
pub slug: &'static str,
pub expects_apply_patch_instructions: bool,
}
#[test]
fn get_full_instructions_no_user_content() {
let prompt = Prompt {
..Default::default()
};
let test_cases = vec![
InstructionsTestCase {
slug: "gpt-3.5",
expects_apply_patch_instructions: true,
},
InstructionsTestCase {
slug: "gpt-4.1",
expects_apply_patch_instructions: true,
},
InstructionsTestCase {
slug: "gpt-4o",
expects_apply_patch_instructions: true,
},
InstructionsTestCase {
slug: "gpt-5",
expects_apply_patch_instructions: true,
},
InstructionsTestCase {
slug: "codex-mini-latest",
expects_apply_patch_instructions: true,
},
InstructionsTestCase {
slug: "gpt-oss:120b",
expects_apply_patch_instructions: false,
},
InstructionsTestCase {
slug: "gpt-5-codex",
expects_apply_patch_instructions: false,
},
];
for test_case in test_cases {
let model_family = find_family_for_model(test_case.slug).expect("known model slug");
let expected = if test_case.expects_apply_patch_instructions {
format!(
"{}\n{}",
model_family.clone().base_instructions,
APPLY_PATCH_TOOL_INSTRUCTIONS
)
} else {
model_family.clone().base_instructions
};
let full = prompt.get_full_instructions(&model_family);
assert_eq!(full, expected);
}
}
#[test]
fn serializes_text_verbosity_when_set() {
let input: Vec<ResponseItem> = vec![];
let tools: Vec<serde_json::Value> = vec![];
let req = ResponsesApiRequest {
model: "gpt-5",
instructions: "i",
input: &input,
tools: &tools,
tool_choice: "auto",
parallel_tool_calls: false,
reasoning: None,
store: false,
stream: true,
include: vec![],
prompt_cache_key: None,
text: Some(TextControls {
verbosity: Some(OpenAiVerbosity::Low),
}),
};
let v = serde_json::to_value(&req).expect("json");
assert_eq!(
v.get("text")
.and_then(|t| t.get("verbosity"))
.and_then(|s| s.as_str()),
Some("low")
);
}
#[test]
fn omits_text_when_not_set() {
let input: Vec<ResponseItem> = vec![];
let tools: Vec<serde_json::Value> = vec![];
let req = ResponsesApiRequest {
model: "gpt-5",
instructions: "i",
input: &input,
tools: &tools,
tool_choice: "auto",
parallel_tool_calls: false,
reasoning: None,
store: false,
stream: true,
include: vec![],
prompt_cache_key: None,
text: None,
};
let v = serde_json::to_value(&req).expect("json");
assert!(v.get("text").is_none());
}
}
pub type ResponseStream = codex_agent::client_common::ResponseStream<CodexErr>;

File diff suppressed because it is too large Load Diff

View File

@@ -1,7 +1,5 @@
use std::sync::Arc;
use super::AgentTask;
use super::MutexExt;
use super::Session;
use super::TurnContext;
use super::get_last_assistant_message_from_turn;
@@ -9,6 +7,7 @@ use crate::Prompt;
use crate::client_common::ResponseEvent;
use crate::error::CodexErr;
use crate::error::Result as CodexResult;
use crate::model_provider_info::ModelProviderExt;
use crate::protocol::AgentMessageEvent;
use crate::protocol::CompactedItem;
use crate::protocol::ErrorEvent;
@@ -16,9 +15,9 @@ use crate::protocol::Event;
use crate::protocol::EventMsg;
use crate::protocol::InputItem;
use crate::protocol::InputMessageKind;
use crate::protocol::TaskCompleteEvent;
use crate::protocol::TaskStartedEvent;
use crate::protocol::TurnContextItem;
use crate::truncate::truncate_middle;
use crate::util::backoff;
use askama::Template;
use codex_protocol::models::ContentItem;
@@ -27,8 +26,8 @@ use codex_protocol::models::ResponseItem;
use codex_protocol::protocol::RolloutItem;
use futures::prelude::*;
pub(super) const COMPACT_TRIGGER_TEXT: &str = "Start Summarization";
const SUMMARIZATION_PROMPT: &str = include_str!("../../templates/compact/prompt.md");
pub const SUMMARIZATION_PROMPT: &str = include_str!("../../templates/compact/prompt.md");
const COMPACT_USER_MESSAGE_MAX_TOKENS: usize = 20_000;
#[derive(Template)]
#[template(path = "compact/history_bridge.md", escape = "none")]
@@ -37,57 +36,32 @@ struct HistoryBridgeTemplate<'a> {
summary_text: &'a str,
}
pub(super) fn spawn_compact_task(
sess: Arc<Session>,
turn_context: Arc<TurnContext>,
sub_id: String,
input: Vec<InputItem>,
) {
let task = AgentTask::compact(
sess.clone(),
turn_context,
sub_id,
input,
SUMMARIZATION_PROMPT.to_string(),
);
sess.set_task(task);
}
pub(super) async fn run_inline_auto_compact_task(
pub(crate) async fn run_inline_auto_compact_task(
sess: Arc<Session>,
turn_context: Arc<TurnContext>,
) {
let sub_id = sess.next_internal_sub_id();
let input = vec![InputItem::Text {
text: COMPACT_TRIGGER_TEXT.to_string(),
text: SUMMARIZATION_PROMPT.to_string(),
}];
run_compact_task_inner(
sess,
turn_context,
sub_id,
input,
SUMMARIZATION_PROMPT.to_string(),
false,
)
.await;
run_compact_task_inner(sess, turn_context, sub_id, input).await;
}
pub(super) async fn run_compact_task(
pub(crate) async fn run_compact_task(
sess: Arc<Session>,
turn_context: Arc<TurnContext>,
sub_id: String,
input: Vec<InputItem>,
compact_instructions: String,
) {
run_compact_task_inner(
sess,
turn_context,
sub_id,
input,
compact_instructions,
true,
)
.await;
) -> Option<String> {
let start_event = Event {
id: sub_id.clone(),
msg: EventMsg::TaskStarted(TaskStartedEvent {
model_context_window: turn_context.client.get_model_context_window(),
}),
};
sess.send_event(start_event).await;
run_compact_task_inner(sess.clone(), turn_context, sub_id.clone(), input).await;
None
}
async fn run_compact_task_inner(
@@ -95,26 +69,15 @@ async fn run_compact_task_inner(
turn_context: Arc<TurnContext>,
sub_id: String,
input: Vec<InputItem>,
compact_instructions: String,
remove_task_on_completion: bool,
) {
let model_context_window = turn_context.client.get_model_context_window();
let start_event = Event {
id: sub_id.clone(),
msg: EventMsg::TaskStarted(TaskStartedEvent {
model_context_window,
}),
};
sess.send_event(start_event).await;
let initial_input_for_turn: ResponseInputItem = ResponseInputItem::from(input);
let instructions_override = compact_instructions;
let turn_input = sess.turn_input_with_history(vec![initial_input_for_turn.clone().into()]);
let turn_input = sess
.turn_input_with_history(vec![initial_input_for_turn.clone().into()])
.await;
let prompt = Prompt {
input: turn_input,
tools: Vec::new(),
base_instructions_override: Some(instructions_override),
..Default::default()
};
let max_retries = turn_context.client.get_provider().stream_max_retries();
@@ -167,21 +130,12 @@ async fn run_compact_task_inner(
}
}
if remove_task_on_completion {
sess.remove_task(&sub_id);
}
let history_snapshot = {
let state = sess.state.lock_unchecked();
state.history.contents()
};
let history_snapshot = sess.history_snapshot().await;
let summary_text = get_last_assistant_message_from_turn(&history_snapshot).unwrap_or_default();
let user_messages = collect_user_messages(&history_snapshot);
let initial_context = sess.build_initial_context(turn_context.as_ref());
let new_history = build_compacted_history(initial_context, &user_messages, &summary_text);
{
let mut state = sess.state.lock_unchecked();
state.history.replace(new_history);
}
sess.replace_history(new_history).await;
let rollout_item = RolloutItem::Compacted(CompactedItem {
message: summary_text.clone(),
@@ -195,16 +149,9 @@ async fn run_compact_task_inner(
}),
};
sess.send_event(event).await;
let event = Event {
id: sub_id.clone(),
msg: EventMsg::TaskComplete(TaskCompleteEvent {
last_agent_message: None,
}),
};
sess.send_event(event).await;
}
fn content_items_to_text(content: &[ContentItem]) -> Option<String> {
pub fn content_items_to_text(content: &[ContentItem]) -> Option<String> {
let mut pieces = Vec::new();
for item in content {
match item {
@@ -236,7 +183,7 @@ pub(crate) fn collect_user_messages(items: &[ResponseItem]) -> Vec<String> {
.collect()
}
fn is_session_prefix_message(text: &str) -> bool {
pub fn is_session_prefix_message(text: &str) -> bool {
matches!(
InputMessageKind::from(("user", text)),
InputMessageKind::UserInstructions | InputMessageKind::EnvironmentContext
@@ -249,11 +196,17 @@ pub(crate) fn build_compacted_history(
summary_text: &str,
) -> Vec<ResponseItem> {
let mut history = initial_context;
let user_messages_text = if user_messages.is_empty() {
let mut user_messages_text = if user_messages.is_empty() {
"(none)".to_string()
} else {
user_messages.join("\n\n")
};
// Truncate the concatenated prior user messages so the bridge message
// stays well under the context window (approx. 4 bytes/token).
let max_bytes = COMPACT_USER_MESSAGE_MAX_TOKENS * 4;
if user_messages_text.len() > max_bytes {
user_messages_text = truncate_middle(&user_messages_text, max_bytes).0;
}
let summary_text = if summary_text.is_empty() {
"(no summary available)".to_string()
} else {
@@ -290,8 +243,7 @@ async fn drain_to_completed(
};
match event {
Ok(ResponseEvent::OutputItemDone(item)) => {
let mut state = sess.state.lock_unchecked();
state.history.record_items(std::slice::from_ref(&item));
sess.record_into_history(std::slice::from_ref(&item)).await;
}
Ok(ResponseEvent::Completed { .. }) => {
return Ok(());
@@ -397,4 +349,38 @@ mod tests {
assert_eq!(vec!["real user message".to_string()], collected);
}
#[test]
fn build_compacted_history_truncates_overlong_user_messages() {
// Prepare a very large prior user message so the aggregated
// `user_messages_text` exceeds the truncation threshold used by
// `build_compacted_history` (80k bytes).
let big = "X".repeat(200_000);
let history = build_compacted_history(Vec::new(), std::slice::from_ref(&big), "SUMMARY");
// Expect exactly one bridge message added to history (plus any initial context we provided, which is none).
assert_eq!(history.len(), 1);
// Extract the text content of the bridge message.
let bridge_text = match &history[0] {
ResponseItem::Message { role, content, .. } if role == "user" => {
content_items_to_text(content).unwrap_or_default()
}
other => panic!("unexpected item in history: {other:?}"),
};
// The bridge should contain the truncation marker and not the full original payload.
assert!(
bridge_text.contains("tokens truncated"),
"expected truncation marker in bridge message"
);
assert!(
!bridge_text.contains(&big),
"bridge should not include the full oversized user text"
);
assert!(
bridge_text.contains("SUMMARY"),
"bridge should include the provided summary text"
);
}
}

View File

@@ -1,3 +1,4 @@
use crate::ModelProviderInfo;
use crate::config_profile::ConfigProfile;
use crate::config_types::History;
use crate::config_types::McpServerConfig;
@@ -12,7 +13,6 @@ use crate::git_info::resolve_root_git_project_for_trust;
use crate::model_family::ModelFamily;
use crate::model_family::derive_default_model_family;
use crate::model_family::find_family_for_model;
use crate::model_provider_info::ModelProviderInfo;
use crate::model_provider_info::built_in_model_providers;
use crate::openai_model_info::get_model_info;
use crate::protocol::AskForApproval;
@@ -37,7 +37,7 @@ use toml_edit::DocumentMut;
use toml_edit::Item as TomlItem;
use toml_edit::Table as TomlTable;
const OPENAI_DEFAULT_MODEL: &str = "gpt-5";
const OPENAI_DEFAULT_MODEL: &str = "gpt-5-codex";
const OPENAI_DEFAULT_REVIEW_MODEL: &str = "gpt-5-codex";
pub const GPT_5_CODEX_MEDIUM_MODEL: &str = "gpt-5-codex";
@@ -54,7 +54,7 @@ pub struct Config {
/// Optional override of model selection.
pub model: String,
/// Model used specifically for review sessions. Defaults to "gpt-5".
/// Model used specifically for review sessions. Defaults to "gpt-5-codex".
pub review_model: String,
pub model_family: ModelFamily,
@@ -333,14 +333,12 @@ pub fn write_global_mcp_servers(
entry["env"] = TomlItem::Table(env_table);
}
if let Some(timeout) = config.startup_timeout_ms {
let timeout = i64::try_from(timeout).map_err(|_| {
std::io::Error::new(
std::io::ErrorKind::InvalidData,
"startup_timeout_ms exceeds supported range",
)
})?;
entry["startup_timeout_ms"] = toml_edit::value(timeout);
if let Some(timeout) = config.startup_timeout_sec {
entry["startup_timeout_sec"] = toml_edit::value(timeout.as_secs_f64());
}
if let Some(timeout) = config.tool_timeout_sec {
entry["tool_timeout_sec"] = toml_edit::value(timeout.as_secs_f64());
}
doc["mcp_servers"][name.as_str()] = TomlItem::Table(entry);
@@ -1163,10 +1161,12 @@ pub fn log_dir(cfg: &Config) -> std::io::Result<PathBuf> {
#[cfg(test)]
mod tests {
use crate::config_types::HistoryPersistence;
use crate::config_types::Notifications;
use super::*;
use pretty_assertions::assert_eq;
use std::time::Duration;
use tempfile::TempDir;
#[test]
@@ -1201,6 +1201,19 @@ persistence = "none"
);
}
#[test]
fn tui_config_missing_notifications_field_defaults_to_disabled() {
let cfg = r#"
[tui]
"#;
let parsed = toml::from_str::<ConfigToml>(cfg)
.expect("TUI config without notifications should succeed");
let tui = parsed.tui.expect("config should include tui section");
assert_eq!(tui.notifications, Notifications::Enabled(false));
}
#[test]
fn test_sandbox_config_parsing() {
let sandbox_full_access = r#"
@@ -1278,7 +1291,8 @@ exclude_slash_tmp = true
command: "echo".to_string(),
args: vec!["hello".to_string()],
env: None,
startup_timeout_ms: None,
startup_timeout_sec: Some(Duration::from_secs(3)),
tool_timeout_sec: Some(Duration::from_secs(5)),
},
);
@@ -1289,6 +1303,8 @@ exclude_slash_tmp = true
let docs = loaded.get("docs").expect("docs entry");
assert_eq!(docs.command, "echo");
assert_eq!(docs.args, vec!["hello".to_string()]);
assert_eq!(docs.startup_timeout_sec, Some(Duration::from_secs(3)));
assert_eq!(docs.tool_timeout_sec, Some(Duration::from_secs(5)));
let empty = BTreeMap::new();
write_global_mcp_servers(codex_home.path(), &empty)?;
@@ -1298,6 +1314,28 @@ exclude_slash_tmp = true
Ok(())
}
#[test]
fn load_global_mcp_servers_accepts_legacy_ms_field() -> anyhow::Result<()> {
let codex_home = TempDir::new()?;
let config_path = codex_home.path().join(CONFIG_TOML_FILE);
std::fs::write(
&config_path,
r#"
[mcp_servers]
[mcp_servers.docs]
command = "echo"
startup_timeout_ms = 2500
"#,
)?;
let servers = load_global_mcp_servers(codex_home.path())?;
let docs = servers.get("docs").expect("docs entry");
assert_eq!(docs.startup_timeout_sec, Some(Duration::from_millis(2500)));
Ok(())
}
#[tokio::test]
async fn persist_model_selection_updates_defaults() -> anyhow::Result<()> {
let codex_home = TempDir::new()?;
@@ -1328,7 +1366,7 @@ exclude_slash_tmp = true
tokio::fs::write(
&config_path,
r#"
model = "gpt-5"
model = "gpt-5-codex"
model_reasoning_effort = "medium"
[profiles.dev]
@@ -1403,7 +1441,7 @@ model = "gpt-4"
model_reasoning_effort = "medium"
[profiles.prod]
model = "gpt-5"
model = "gpt-5-codex"
"#,
)
.await?;
@@ -1434,7 +1472,7 @@ model = "gpt-5"
.profiles
.get("prod")
.and_then(|profile| profile.model.as_deref()),
Some("gpt-5"),
Some("gpt-5-codex"),
);
Ok(())

View File

@@ -136,7 +136,7 @@ async fn persist_overrides_with_behavior(
} else {
doc.get("profile")
.and_then(|i| i.as_str())
.map(|s| s.to_string())
.map(str::to_string)
};
let mut mutated = false;
@@ -228,7 +228,7 @@ mod tests {
codex_home,
None,
&[
(&[CONFIG_KEY_MODEL], "gpt-5"),
(&[CONFIG_KEY_MODEL], "gpt-5-codex"),
(&[CONFIG_KEY_EFFORT], "high"),
],
)
@@ -236,7 +236,7 @@ mod tests {
.expect("persist");
let contents = read_config(codex_home).await;
let expected = r#"model = "gpt-5"
let expected = r#"model = "gpt-5-codex"
model_reasoning_effort = "high"
"#;
assert_eq!(contents, expected);
@@ -348,7 +348,7 @@ model_reasoning_effort = "high"
&[
(&["a", "b", "c"], "v"),
(&["x"], "y"),
(&["profiles", "p1", CONFIG_KEY_MODEL], "gpt-5"),
(&["profiles", "p1", CONFIG_KEY_MODEL], "gpt-5-codex"),
],
)
.await
@@ -361,7 +361,7 @@ model_reasoning_effort = "high"
c = "v"
[profiles.p1]
model = "gpt-5"
model = "gpt-5-codex"
"#;
assert_eq!(contents, expected);
}
@@ -454,7 +454,7 @@ existing = "keep"
codex_home,
None,
&[
(&[CONFIG_KEY_MODEL], "gpt-5"),
(&[CONFIG_KEY_MODEL], "gpt-5-codex"),
(&[CONFIG_KEY_EFFORT], "minimal"),
],
)
@@ -466,7 +466,7 @@ existing = "keep"
# should be preserved
existing = "keep"
model = "gpt-5"
model = "gpt-5-codex"
model_reasoning_effort = "minimal"
"#;
assert_eq!(contents, expected);
@@ -524,7 +524,7 @@ model = "o3"
let codex_home = tmpdir.path();
// Seed with a model value only
let seed = "model = \"gpt-5\"\n";
let seed = "model = \"gpt-5-codex\"\n";
tokio::fs::write(codex_home.join(CONFIG_TOML_FILE), seed)
.await
.expect("seed write");
@@ -535,7 +535,7 @@ model = "o3"
.expect("persist");
let contents = read_config(codex_home).await;
let expected = r#"model = "gpt-5"
let expected = r#"model = "gpt-5-codex"
model_reasoning_effort = "high"
"#;
assert_eq!(contents, expected);
@@ -579,7 +579,7 @@ model = "o4-mini"
// No active profile key; we'll target an explicit override
let seed = r#"[profiles.team]
model = "gpt-5"
model = "gpt-5-codex"
"#;
tokio::fs::write(codex_home.join(CONFIG_TOML_FILE), seed)
.await
@@ -595,7 +595,7 @@ model = "gpt-5"
let contents = read_config(codex_home).await;
let expected = r#"[profiles.team]
model = "gpt-5"
model = "gpt-5-codex"
model_reasoning_effort = "minimal"
"#;
assert_eq!(contents, expected);
@@ -611,7 +611,7 @@ model_reasoning_effort = "minimal"
codex_home,
None,
&[
(&[CONFIG_KEY_MODEL], Some("gpt-5")),
(&[CONFIG_KEY_MODEL], Some("gpt-5-codex")),
(&[CONFIG_KEY_EFFORT], None),
],
)
@@ -619,7 +619,7 @@ model_reasoning_effort = "minimal"
.expect("persist");
let contents = read_config(codex_home).await;
let expected = "model = \"gpt-5\"\n";
let expected = "model = \"gpt-5-codex\"\n";
assert_eq!(contents, expected);
}
@@ -670,7 +670,7 @@ model = "o3"
let tmpdir = tempdir().expect("tmp");
let codex_home = tmpdir.path();
let seed = r#"model = "gpt-5"
let seed = r#"model = "gpt-5-codex"
model_reasoning_effort = "medium"
"#;
tokio::fs::write(codex_home.join(CONFIG_TOML_FILE), seed)

View File

@@ -1,225 +1,3 @@
//! Types used to define the fields of [`crate::config::Config`].
//! Re-exported configuration data structures now defined in `codex-agent`.
// Note this file should generally be restricted to simple struct/enum
// definitions that do not contain business logic.
use std::collections::HashMap;
use std::path::PathBuf;
use wildmatch::WildMatchPattern;
use serde::Deserialize;
#[derive(Deserialize, Debug, Clone, PartialEq)]
pub struct McpServerConfig {
pub command: String,
#[serde(default)]
pub args: Vec<String>,
#[serde(default)]
pub env: Option<HashMap<String, String>>,
/// Startup timeout in milliseconds for initializing MCP server & initially listing tools.
#[serde(default)]
pub startup_timeout_ms: Option<u64>,
}
#[derive(Deserialize, Debug, Copy, Clone, PartialEq)]
pub enum UriBasedFileOpener {
#[serde(rename = "vscode")]
VsCode,
#[serde(rename = "vscode-insiders")]
VsCodeInsiders,
#[serde(rename = "windsurf")]
Windsurf,
#[serde(rename = "cursor")]
Cursor,
/// Option to disable the URI-based file opener.
#[serde(rename = "none")]
None,
}
impl UriBasedFileOpener {
pub fn get_scheme(&self) -> Option<&str> {
match self {
UriBasedFileOpener::VsCode => Some("vscode"),
UriBasedFileOpener::VsCodeInsiders => Some("vscode-insiders"),
UriBasedFileOpener::Windsurf => Some("windsurf"),
UriBasedFileOpener::Cursor => Some("cursor"),
UriBasedFileOpener::None => None,
}
}
}
/// Settings that govern if and what will be written to `~/.codex/history.jsonl`.
#[derive(Deserialize, Debug, Clone, PartialEq, Default)]
pub struct History {
/// If true, history entries will not be written to disk.
pub persistence: HistoryPersistence,
/// If set, the maximum size of the history file in bytes.
/// TODO(mbolin): Not currently honored.
pub max_bytes: Option<usize>,
}
#[derive(Deserialize, Debug, Copy, Clone, PartialEq, Default)]
#[serde(rename_all = "kebab-case")]
pub enum HistoryPersistence {
/// Save all history entries to disk.
#[default]
SaveAll,
/// Do not write history to disk.
None,
}
#[derive(Debug, Clone, PartialEq, Eq, Deserialize)]
#[serde(untagged)]
pub enum Notifications {
Enabled(bool),
Custom(Vec<String>),
}
impl Default for Notifications {
fn default() -> Self {
Self::Enabled(false)
}
}
/// Collection of settings that are specific to the TUI.
#[derive(Deserialize, Debug, Clone, PartialEq, Default)]
pub struct Tui {
/// Enable desktop notifications from the TUI when the terminal is unfocused.
/// Defaults to `false`.
pub notifications: Notifications,
}
#[derive(Deserialize, Debug, Clone, PartialEq, Default)]
pub struct SandboxWorkspaceWrite {
#[serde(default)]
pub writable_roots: Vec<PathBuf>,
#[serde(default)]
pub network_access: bool,
#[serde(default)]
pub exclude_tmpdir_env_var: bool,
#[serde(default)]
pub exclude_slash_tmp: bool,
}
impl From<SandboxWorkspaceWrite> for codex_protocol::mcp_protocol::SandboxSettings {
fn from(sandbox_workspace_write: SandboxWorkspaceWrite) -> Self {
Self {
writable_roots: sandbox_workspace_write.writable_roots,
network_access: Some(sandbox_workspace_write.network_access),
exclude_tmpdir_env_var: Some(sandbox_workspace_write.exclude_tmpdir_env_var),
exclude_slash_tmp: Some(sandbox_workspace_write.exclude_slash_tmp),
}
}
}
#[derive(Deserialize, Debug, Clone, PartialEq, Default)]
#[serde(rename_all = "kebab-case")]
pub enum ShellEnvironmentPolicyInherit {
/// "Core" environment variables for the platform. On UNIX, this would
/// include HOME, LOGNAME, PATH, SHELL, and USER, among others.
Core,
/// Inherits the full environment from the parent process.
#[default]
All,
/// Do not inherit any environment variables from the parent process.
None,
}
/// Policy for building the `env` when spawning a process via either the
/// `shell` or `local_shell` tool.
#[derive(Deserialize, Debug, Clone, PartialEq, Default)]
pub struct ShellEnvironmentPolicyToml {
pub inherit: Option<ShellEnvironmentPolicyInherit>,
pub ignore_default_excludes: Option<bool>,
/// List of regular expressions.
pub exclude: Option<Vec<String>>,
pub r#set: Option<HashMap<String, String>>,
/// List of regular expressions.
pub include_only: Option<Vec<String>>,
pub experimental_use_profile: Option<bool>,
}
pub type EnvironmentVariablePattern = WildMatchPattern<'*', '?'>;
/// Deriving the `env` based on this policy works as follows:
/// 1. Create an initial map based on the `inherit` policy.
/// 2. If `ignore_default_excludes` is false, filter the map using the default
/// exclude pattern(s), which are: `"*KEY*"` and `"*TOKEN*"`.
/// 3. If `exclude` is not empty, filter the map using the provided patterns.
/// 4. Insert any entries from `r#set` into the map.
/// 5. If non-empty, filter the map using the `include_only` patterns.
#[derive(Debug, Clone, PartialEq, Default)]
pub struct ShellEnvironmentPolicy {
/// Starting point when building the environment.
pub inherit: ShellEnvironmentPolicyInherit,
/// True to skip the check to exclude default environment variables that
/// contain "KEY" or "TOKEN" in their name.
pub ignore_default_excludes: bool,
/// Environment variable names to exclude from the environment.
pub exclude: Vec<EnvironmentVariablePattern>,
/// (key, value) pairs to insert in the environment.
pub r#set: HashMap<String, String>,
/// Environment variable names to retain in the environment.
pub include_only: Vec<EnvironmentVariablePattern>,
/// If true, the shell profile will be used to run the command.
pub use_profile: bool,
}
impl From<ShellEnvironmentPolicyToml> for ShellEnvironmentPolicy {
fn from(toml: ShellEnvironmentPolicyToml) -> Self {
// Default to inheriting the full environment when not specified.
let inherit = toml.inherit.unwrap_or(ShellEnvironmentPolicyInherit::All);
let ignore_default_excludes = toml.ignore_default_excludes.unwrap_or(false);
let exclude = toml
.exclude
.unwrap_or_default()
.into_iter()
.map(|s| EnvironmentVariablePattern::new_case_insensitive(&s))
.collect();
let r#set = toml.r#set.unwrap_or_default();
let include_only = toml
.include_only
.unwrap_or_default()
.into_iter()
.map(|s| EnvironmentVariablePattern::new_case_insensitive(&s))
.collect();
let use_profile = toml.experimental_use_profile.unwrap_or(false);
Self {
inherit,
ignore_default_excludes,
exclude,
r#set,
include_only,
use_profile,
}
}
}
#[derive(Deserialize, Debug, Clone, PartialEq, Eq, Default, Hash)]
#[serde(rename_all = "kebab-case")]
pub enum ReasoningSummaryFormat {
#[default]
None,
Experimental,
}
pub use codex_agent::config_types::*;

View File

@@ -1,120 +1 @@
use codex_protocol::models::ResponseItem;
/// Transcript of conversation history
#[derive(Debug, Clone, Default)]
pub(crate) struct ConversationHistory {
/// The oldest items are at the beginning of the vector.
items: Vec<ResponseItem>,
}
impl ConversationHistory {
pub(crate) fn new() -> Self {
Self { items: Vec::new() }
}
/// Returns a clone of the contents in the transcript.
pub(crate) fn contents(&self) -> Vec<ResponseItem> {
self.items.clone()
}
/// `items` is ordered from oldest to newest.
pub(crate) fn record_items<I>(&mut self, items: I)
where
I: IntoIterator,
I::Item: std::ops::Deref<Target = ResponseItem>,
{
for item in items {
if !is_api_message(&item) {
continue;
}
self.items.push(item.clone());
}
}
pub(crate) fn replace(&mut self, items: Vec<ResponseItem>) {
self.items = items;
}
}
/// Anything that is not a system message or "reasoning" message is considered
/// an API message.
fn is_api_message(message: &ResponseItem) -> bool {
match message {
ResponseItem::Message { role, .. } => role.as_str() != "system",
ResponseItem::FunctionCallOutput { .. }
| ResponseItem::FunctionCall { .. }
| ResponseItem::CustomToolCall { .. }
| ResponseItem::CustomToolCallOutput { .. }
| ResponseItem::LocalShellCall { .. }
| ResponseItem::Reasoning { .. }
| ResponseItem::WebSearchCall { .. } => true,
ResponseItem::Other => false,
}
}
#[cfg(test)]
mod tests {
use super::*;
use codex_protocol::models::ContentItem;
fn assistant_msg(text: &str) -> ResponseItem {
ResponseItem::Message {
id: None,
role: "assistant".to_string(),
content: vec![ContentItem::OutputText {
text: text.to_string(),
}],
}
}
fn user_msg(text: &str) -> ResponseItem {
ResponseItem::Message {
id: None,
role: "user".to_string(),
content: vec![ContentItem::OutputText {
text: text.to_string(),
}],
}
}
#[test]
fn filters_non_api_messages() {
let mut h = ConversationHistory::default();
// System message is not an API message; Other is ignored.
let system = ResponseItem::Message {
id: None,
role: "system".to_string(),
content: vec![ContentItem::OutputText {
text: "ignored".to_string(),
}],
};
h.record_items([&system, &ResponseItem::Other]);
// User and assistant should be retained.
let u = user_msg("hi");
let a = assistant_msg("hello");
h.record_items([&u, &a]);
let items = h.contents();
assert_eq!(
items,
vec![
ResponseItem::Message {
id: None,
role: "user".to_string(),
content: vec![ContentItem::OutputText {
text: "hi".to_string()
}]
},
ResponseItem::Message {
id: None,
role: "assistant".to_string(),
content: vec![ContentItem::OutputText {
text: "hello".to_string()
}]
}
]
);
}
}
pub use codex_agent::ConversationHistory;

View File

@@ -3,6 +3,8 @@ use crate::CodexAuth;
use crate::codex::Codex;
use crate::codex::CodexSpawnOk;
use crate::codex::INITIAL_SUBMIT_ID;
use crate::codex::compact::content_items_to_text;
use crate::codex::compact::is_session_prefix_message;
use crate::codex_conversation::CodexConversation;
use crate::config::Config;
use crate::error::CodexErr;
@@ -134,19 +136,19 @@ impl ConversationManager {
self.conversations.write().await.remove(conversation_id)
}
/// Fork an existing conversation by dropping the last `drop_last_messages`
/// user/assistant messages from its transcript and starting a new
/// Fork an existing conversation by taking messages up to the given position
/// (not including the message at the given position) and starting a new
/// conversation with identical configuration (unless overridden by the
/// caller's `config`). The new conversation will have a fresh id.
pub async fn fork_conversation(
&self,
num_messages_to_drop: usize,
nth_user_message: usize,
config: Config,
path: PathBuf,
) -> CodexResult<NewConversation> {
// Compute the prefix up to the cut point.
let history = RolloutRecorder::get_rollout_history(&path).await?;
let history = truncate_after_dropping_last_messages(history, num_messages_to_drop);
let history = truncate_before_nth_user_message(history, nth_user_message);
// Spawn a new conversation with the computed initial history.
let auth_manager = self.auth_manager.clone();
@@ -159,33 +161,30 @@ impl ConversationManager {
}
}
/// Return a prefix of `items` obtained by dropping the last `n` user messages
/// and all items that follow them.
fn truncate_after_dropping_last_messages(history: InitialHistory, n: usize) -> InitialHistory {
if n == 0 {
return InitialHistory::Forked(history.get_rollout_items());
}
// Work directly on rollout items, and cut the vector at the nth-from-last user message input.
/// Return a prefix of `items` obtained by cutting strictly before the nth user message
/// (0-based) and all items that follow it.
fn truncate_before_nth_user_message(history: InitialHistory, n: usize) -> InitialHistory {
// Work directly on rollout items, and cut the vector at the nth user message input.
let items: Vec<RolloutItem> = history.get_rollout_items();
// Find indices of user message inputs in rollout order.
let mut user_positions: Vec<usize> = Vec::new();
for (idx, item) in items.iter().enumerate() {
if let RolloutItem::ResponseItem(ResponseItem::Message { role, .. }) = item
if let RolloutItem::ResponseItem(ResponseItem::Message { role, content, .. }) = item
&& role == "user"
&& content_items_to_text(content).is_some_and(|text| !is_session_prefix_message(&text))
{
user_positions.push(idx);
}
}
// If fewer than n user messages exist, treat as empty.
if user_positions.len() < n {
// If fewer than or equal to n user messages exist, treat as empty (out of range).
if user_positions.len() <= n {
return InitialHistory::New;
}
// Cut strictly before the nth-from-last user message (do not keep the nth itself).
let cut_idx = user_positions[user_positions.len() - n];
// Cut strictly before the nth user message (do not keep the nth itself).
let cut_idx = user_positions[n];
let rolled: Vec<RolloutItem> = items.into_iter().take(cut_idx).collect();
if rolled.is_empty() {
@@ -198,9 +197,11 @@ fn truncate_after_dropping_last_messages(history: InitialHistory, n: usize) -> I
#[cfg(test)]
mod tests {
use super::*;
use crate::codex::make_session_and_context;
use codex_protocol::models::ContentItem;
use codex_protocol::models::ReasoningItemReasoningSummary;
use codex_protocol::models::ResponseItem;
use pretty_assertions::assert_eq;
fn user_msg(text: &str) -> ResponseItem {
ResponseItem::Message {
@@ -252,7 +253,7 @@ mod tests {
.cloned()
.map(RolloutItem::ResponseItem)
.collect();
let truncated = truncate_after_dropping_last_messages(InitialHistory::Forked(initial), 1);
let truncated = truncate_before_nth_user_message(InitialHistory::Forked(initial), 1);
let got_items = truncated.get_rollout_items();
let expected_items = vec![
RolloutItem::ResponseItem(items[0].clone()),
@@ -269,7 +270,37 @@ mod tests {
.cloned()
.map(RolloutItem::ResponseItem)
.collect();
let truncated2 = truncate_after_dropping_last_messages(InitialHistory::Forked(initial2), 2);
let truncated2 = truncate_before_nth_user_message(InitialHistory::Forked(initial2), 2);
assert!(matches!(truncated2, InitialHistory::New));
}
#[test]
fn ignores_session_prefix_messages_when_truncating() {
let (session, turn_context) = make_session_and_context();
let mut items = session.build_initial_context(&turn_context);
items.push(user_msg("feature request"));
items.push(assistant_msg("ack"));
items.push(user_msg("second question"));
items.push(assistant_msg("answer"));
let rollout_items: Vec<RolloutItem> = items
.iter()
.cloned()
.map(RolloutItem::ResponseItem)
.collect();
let truncated = truncate_before_nth_user_message(InitialHistory::Forked(rollout_items), 1);
let got_items = truncated.get_rollout_items();
let expected: Vec<RolloutItem> = vec![
RolloutItem::ResponseItem(items[0].clone()),
RolloutItem::ResponseItem(items[1].clone()),
RolloutItem::ResponseItem(items[2].clone()),
];
assert_eq!(
serde_json::to_value(&got_items).unwrap(),
serde_json::to_value(&expected).unwrap()
);
}
}

View File

@@ -52,7 +52,7 @@ pub async fn discover_prompts_in_excluding(
let Some(name) = path
.file_stem()
.and_then(|s| s.to_str())
.map(|s| s.to_string())
.map(str::to_string)
else {
continue;
};

View File

@@ -1,3 +1,4 @@
use crate::spawn::CODEX_SANDBOX_ENV_VAR;
use reqwest::header::HeaderValue;
use std::sync::LazyLock;
use std::sync::Mutex;
@@ -20,7 +21,6 @@ use std::sync::Mutex;
pub static USER_AGENT_SUFFIX: LazyLock<Mutex<Option<String>>> = LazyLock::new(|| Mutex::new(None));
pub const CODEX_INTERNAL_ORIGINATOR_OVERRIDE_ENV_VAR: &str = "CODEX_INTERNAL_ORIGINATOR_OVERRIDE";
#[derive(Debug, Clone)]
pub struct Originator {
pub value: String,
@@ -112,17 +112,25 @@ pub fn create_client() -> reqwest::Client {
headers.insert("originator", ORIGINATOR.header_value.clone());
let ua = get_codex_user_agent();
reqwest::Client::builder()
let mut builder = reqwest::Client::builder()
// Set UA via dedicated helper to avoid header validation pitfalls
.user_agent(ua)
.default_headers(headers)
.build()
.unwrap_or_else(|_| reqwest::Client::new())
.default_headers(headers);
if is_sandboxed() {
builder = builder.no_proxy();
}
builder.build().unwrap_or_else(|_| reqwest::Client::new())
}
fn is_sandboxed() -> bool {
std::env::var(CODEX_SANDBOX_ENV_VAR).as_deref() == Ok("seatbelt")
}
#[cfg(test)]
mod tests {
use super::*;
use core_test_support::skip_if_no_network;
#[test]
fn test_get_codex_user_agent() {
@@ -132,6 +140,8 @@ mod tests {
#[tokio::test]
async fn test_create_client_sets_default_headers() {
skip_if_no_network!();
use wiremock::Mock;
use wiremock::MockServer;
use wiremock::ResponseTemplate;

View File

@@ -308,19 +308,16 @@ mod tests {
Some(PathBuf::from("/repo")),
Some(AskForApproval::OnRequest),
Some(workspace_write_policy(vec!["/repo"], false)),
Some(Shell::Bash(BashShell {
shell_path: "/bin/bash".into(),
bashrc_path: "/home/user/.bashrc".into(),
})),
Some(Shell::Bash(BashShell::new(
"/bin/bash",
"/home/user/.bashrc",
))),
);
let context2 = EnvironmentContext::new(
Some(PathBuf::from("/repo")),
Some(AskForApproval::OnRequest),
Some(workspace_write_policy(vec!["/repo"], false)),
Some(Shell::Zsh(ZshShell {
shell_path: "/bin/zsh".into(),
zshrc_path: "/home/user/.zshrc".into(),
})),
Some(Shell::Zsh(ZshShell::new("/bin/zsh", "/home/user/.zshrc"))),
);
assert!(context1.equals_except_shell(&context2));

Some files were not shown because too many files have changed in this diff Show More