Compare commits

..

62 Commits

Author SHA1 Message Date
Ahmed Ibrahim
9f63b77fb2 tests 2025-10-01 17:52:13 -07:00
Ahmed Ibrahim
7750d859ef tests 2025-10-01 17:49:56 -07:00
Ahmed Ibrahim
cd70f68240 tests 2025-10-01 17:49:34 -07:00
Ahmed Ibrahim
0c3e584af6 plan is on by default 2025-10-01 17:30:49 -07:00
pakrym-oai
e899ae7d8a Include request ID in the error message (#4572)
To help with issue debugging
<img width="1414" height="253" alt="image"
src="https://github.com/user-attachments/assets/254732df-44ac-4252-997a-6c5e0927355b"
/>
2025-10-01 15:36:04 -07:00
iceweasel-oai
6f97ec4990 canonicalize display of Agents.md paths on Windows. (#4577)
Canonicalize path on Windows to 
- remove unattractive path prefixes such as `\\?\`
- simplify it (`../AGENTS.md` vs
`C:\Users\iceweasel\code\coded\Agents.md`)
before: <img width="1110" height="45" alt="Screenshot 2025-10-01 123520"
src="https://github.com/user-attachments/assets/48920ae6-d89c-41b8-b4ea-df5c18fb5fad"
/>

after: 
<img width="585" height="46" alt="Screenshot 2025-10-01 123612"
src="https://github.com/user-attachments/assets/70a1761a-9d97-4836-b14c-670b6f13e608"
/>
2025-10-01 14:33:19 -07:00
Jeremy Rose
07c1db351a rework patch/exec approval UI (#4573)
| Scenario | Screenshot |
| ---------------------- |
----------------------------------------------------------------------------------------------------------------------------------------------------
|
| short patch | <img width="1096" height="533" alt="short patch"
src="https://github.com/user-attachments/assets/8a883429-0965-4c0b-9002-217b3759b557"
/> |
| short command | <img width="1096" height="533" alt="short command"
src="https://github.com/user-attachments/assets/901abde8-2494-4e86-b98a-7cabaf87ca9c"
/> |
| long patch | <img width="1129" height="892" alt="long patch"
src="https://github.com/user-attachments/assets/fa799a29-a0d6-48e6-b2ef-10302a7916d3"
/> |
| long command | <img width="1096" height="892" alt="long command"
src="https://github.com/user-attachments/assets/11ddf79b-98cb-4b60-ac22-49dfa7779343"
/> |
| viewing complete patch | <img width="1129" height="892" alt="viewing
complete patch"
src="https://github.com/user-attachments/assets/81666958-af94-420e-aa66-b60d0a42b9db"
/> |
2025-10-01 14:29:05 -07:00
pakrym-oai
31102af54b Add initial set of doc comments to the SDK (#4513)
Also perform minor code cleanup.
2025-10-01 13:12:59 -07:00
Thibault Sottiaux
5d78c1edd3 Revert "chore: prompt update to enforce good usage of apply_patch" (#4576)
Reverts openai/codex#3846
2025-10-01 20:11:36 +00:00
pakrym-oai
170c685882 Explicit node imports (#4567)
To help with compatibility
2025-10-01 12:39:04 -07:00
Eric Traut
609f75acec Fix hang on second oauth login attempt (#4568)
This PR fixes a bug that results in a hang in the oauth login flow if a
user logs in, then logs out, then logs in again without first closing
the browser window.

Root cause of problem: We use a local web server for the oauth flow, and
it's implemented using the `tiny_http` rust crate. During the first
login, a socket is created between the browser and the server. The
`tiny_http` library creates worker threads that persist for as long as
this socket remains open. Currently, there's no way to close the
connection on the server side — the library provides no API to do this.
The library also filters all "Connect: close" headers, which makes it
difficult to tell the client browser to close the connection. On the
second login attempt, the browser uses the existing connection rather
than creating a new one. Since that connection is associated with a
server instance that no longer exists, it is effectively ignored.

I considered switching from `tiny_http` to a different web server
library, but that would have been a big change with significant
regression risk. This PR includes a more surgical fix that works around
the limitation of `tiny_http` and sends a "Connect: close" header on the
last "success" page of the oauth flow.
2025-10-01 12:26:28 -07:00
Michael Bolin
eabe18714f fix: use number instead of bigint for the generated TS for RequestId (#4575)
Before this PR:

```typescript
export type RequestId = string | bigint;
```

After:

```typescript
export type RequestId = string | number;
```

`bigint` introduces headaches in TypeScript without providing any real
value.
2025-10-01 12:10:20 -07:00
easong-openai
ceaba36c7f fix ctr-n hint (#4566)
don't show or enable ctr-n to choose best of n while not in the composer
2025-10-01 18:42:04 +00:00
Michael Bolin
d94e8bad8b feat: add --emergency-version-override option to create_github_release script (#4556)
I just had to use this like so:

```
./codex-rs/scripts/create_github_release --publish-alpha --emergency-version-override 0.43.0-alpha.10
```

because the build for `0.43.0-alpha.9` failed:

https://github.com/openai/codex/actions/runs/18167317356
2025-10-01 11:40:04 -07:00
pakrym-oai
8a367ef6bf SDK: support working directory and skipGitRepoCheck options (#4563)
Make options not required, add support for working directory and
skipGitRepoCheck options on the turn
2025-10-01 11:26:49 -07:00
easong-openai
400a5a90bf Fall back to configured instruction files if AGENTS.md isn't available (#4544)
Allow users to configure an agents.md alternative to consume, but warn
the user it may degrade model performance.

Fixes #4376
2025-10-01 18:19:59 +00:00
Ahmed Ibrahim
2f370e946d Show context window usage while tasks run (#4536)
## Summary
- show the remaining context window percentage in `/status` alongside
existing token usage details
- replace the composer shortcut prompt with the context window
percentage (or an unavailable message) while a task is running
- update TUI snapshots to reflect the new context window line

## Testing
- cargo test -p codex-tui

------
https://chatgpt.com/codex/tasks/task_i_68dc6e7397ac8321909d7daff25a396c
2025-10-01 18:03:05 +00:00
Ahmed Ibrahim
751b3b50ac Show placeholder for commands with no output (#4509)
## Summary
- show a dim “(no output)” placeholder when an executed command produces
no stdout or stderr so empty runs are visible
- update TUI snapshots to include the new placeholder in history
renderings

## Testing
- cargo test -p codex-tui


------
https://chatgpt.com/codex/tasks/task_i_68dc056c1d5883218fe8d9929e9b1657
2025-10-01 10:42:30 -07:00
Ahmed Ibrahim
d78d0764aa Add Updated at time in resume picker (#4468)
<img width="639" height="281" alt="image"
src="https://github.com/user-attachments/assets/92b2ad2b-9e18-4485-9b8d-d7056eb98651"
/>
2025-10-01 10:40:43 -07:00
rakesh-oai
699c121606 Handle trailing backslash properly (#4559)
**Summary**

This PR fixes an issue in the device code login flow where trailing
slashes in the issuer URL could cause malformed URLs during codex token
exchange step


**Test**


Before the changes

`Error logging in with device code: device code exchange failed: error
decoding response body`

After the changes

`Successfully logged in`
2025-10-01 10:32:09 -07:00
iceweasel-oai
dde615f482 implement command safety for PowerShell commands (#4269)
Implement command safety for PowerShell commands on Windows

This change adds a new Windows-specific command-safety module under
`codex-rs/core/src/command_safety/windows_safe_commands.rs` to strictly
sanitise PowerShell invocations. Key points:

- Introduce `is_safe_command_windows()` to only allow explicitly
read-only PowerShell calls.
- Parse and split PowerShell invocations (including inline `-Command`
scripts and pipelines).
- Block unsafe switches (`-File`, `-EncodedCommand`, `-ExecutionPolicy`,
unknown flags, call operators, redirections, separators).
- Whitelist only read-only cmdlets (`Get-ChildItem`, `Get-Content`,
`Select-Object`, etc.), safe Git subcommands (`status`, `log`, `show`,
`diff`, `cat-file`), and ripgrep without unsafe options.
- Add comprehensive unit tests covering allowed and rejected command
patterns (nested calls, side effects, chaining, redirections).

This ensures Codex on Windows can safely execute discover-only
PowerShell workflows without risking destructive operations.
2025-10-01 09:56:48 -07:00
Michael Bolin
325fad1d92 fix: pnpm/action-setup@v4 should run before actions/setup-node@v5 (#4555)
`rust-release.yml` just failed:

https://github.com/openai/codex/actions/runs/18167317356/job/51714366768

The error is:

> Error: Unable to locate executable file: pnpm. Please verify either
the file path exists or the file can be found within a directory
specified by the PATH environment variable. Also check the file mode to
verify the file is executable.

We need to install `pnpm` first like we do in `ci.yml`:


f815157dd9/.github/workflows/ci.yml (L17-L25)
2025-10-01 09:04:14 -07:00
Michael Bolin
f815157dd9 chore: introduce publishing logic for @openai/codex-sdk (#4543)
There was a bit of copypasta I put up with when were publishing two
packages to npm, but now that it's three, I created some more scripts to
consolidate things.

With this change, I ran:

```shell
./scripts/stage_npm_packages.py --release-version 0.43.0-alpha.8 --package codex --package codex-responses-api-proxy --package codex-sdk
```

Indeed when it finished, I ended up with:

```shell
$ tree dist
dist
└── npm
    ├── codex-npm-0.43.0-alpha.8.tgz
    ├── codex-responses-api-proxy-npm-0.43.0-alpha.8.tgz
    └── codex-sdk-npm-0.43.0-alpha.8.tgz
$ tar tzvf dist/npm/codex-sdk-npm-0.43.0-alpha.8.tgz
-rwxr-xr-x  0 0      0    25476720 Oct 26  1985 package/vendor/aarch64-apple-darwin/codex/codex
-rwxr-xr-x  0 0      0    29871400 Oct 26  1985 package/vendor/aarch64-unknown-linux-musl/codex/codex
-rwxr-xr-x  0 0      0    28368096 Oct 26  1985 package/vendor/x86_64-apple-darwin/codex/codex
-rwxr-xr-x  0 0      0    36029472 Oct 26  1985 package/vendor/x86_64-unknown-linux-musl/codex/codex
-rw-r--r--  0 0      0       10926 Oct 26  1985 package/LICENSE
-rw-r--r--  0 0      0    30187520 Oct 26  1985 package/vendor/aarch64-pc-windows-msvc/codex/codex.exe
-rw-r--r--  0 0      0    35277824 Oct 26  1985 package/vendor/x86_64-pc-windows-msvc/codex/codex.exe
-rw-r--r--  0 0      0        4842 Oct 26  1985 package/dist/index.js
-rw-r--r--  0 0      0        1347 Oct 26  1985 package/package.json
-rw-r--r--  0 0      0        9867 Oct 26  1985 package/dist/index.js.map
-rw-r--r--  0 0      0          12 Oct 26  1985 package/README.md
-rw-r--r--  0 0      0        4287 Oct 26  1985 package/dist/index.d.ts
```
2025-10-01 08:29:59 -07:00
jif-oai
b8195a17e5 chore: sanbox extraction (#4286)
# Extract and Centralize Sandboxing
- Goal: Improve safety and clarity by centralizing sandbox planning and
execution.
  - Approach:
- Add planner (ExecPlan) and backend registry (Direct/Seatbelt/Linux)
with run_with_plan.
- Refactor codex.rs to plan-then-execute; handle failures/escalation via
the plan.
- Delegate apply_patch to the codex binary and run it with an empty env
for determinism.
2025-10-01 12:05:12 +01:00
rakesh-oai
349ef7edc6 Fix Callback URL for staging and prod environments (#4533)
# External (non-OpenAI) Pull Request Requirements

Before opening this Pull Request, please read the dedicated
"Contributing" markdown file or your PR may be closed:
https://github.com/openai/codex/blob/main/docs/contributing.md

If your PR conforms to our contribution guidelines, replace this text
with a detailed and high quality description of your changes.
2025-10-01 02:57:37 +00:00
Michael Bolin
5881c0d6d4 fix: remove mcp-types from app server protocol (#4537)
We continue the separation between `codex app-server` and `codex
mcp-server`.

In particular, we introduce a new crate, `codex-app-server-protocol`,
and migrate `codex-rs/protocol/src/mcp_protocol.rs` into it, renaming it
`codex-rs/app-server-protocol/src/protocol.rs`.

Because `ConversationId` was defined in `mcp_protocol.rs`, we move it
into its own file, `codex-rs/protocol/src/conversation_id.rs`, and
because it is referenced in a ton of places, we have to touch a lot of
files as part of this PR.

We also decide to get away from proper JSON-RPC 2.0 semantics, so we
also introduce `codex-rs/app-server-protocol/src/jsonrpc_lite.rs`, which
is basically the same `JSONRPCMessage` type defined in `mcp-types`
except with all of the `"jsonrpc": "2.0"` removed.

Getting rid of `"jsonrpc": "2.0"` makes our serialization logic
considerably simpler, as we can lean heavier on serde to serialize
directly into the wire format that we use now.
2025-10-01 02:16:26 +00:00
pakrym-oai
8dd771d217 Add executable detection and export Codex from the SDK (#4532)
Executable detection uses the same rules as the codex wrapper.
2025-09-30 18:06:16 -07:00
Michael Bolin
32853ecbc5 fix: use macros to ensure request/response symmetry (#4529)
Manually curating `protocol-ts/src/lib.rs` was error-prone, as expected.
I finally asked Codex to write some Rust macros so we can ensure that:

- For every variant of `ClientRequest` and `ServerRequest`, there is an
associated `params` and `response` type.
- All response types are included automatically in the output of `codex
generate-ts`.
2025-09-30 18:06:05 -07:00
pakrym-oai
7fc3edf8a7 Remove legacy codex exec --json format (#4525)
`codex exec --json` now maps to the behavior of `codex exec
--experimental-json` with new event and item shapes.

Thread events:
- thread.started
- turn.started
- turn.completed
- turn.failed
- item.started
- item.updated
- item.completed

Item types: 
- assistant_message
- reasoning
- command_execution
- file_change
- mcp_tool_call
- web_search
- todo_list
- error

Sample output:

<details>
`codex exec "list my assigned github issues"  --json | jq`

```
{
  "type": "thread.started",
  "thread_id": "01999ce5-f229-7661-8570-53312bd47ea3"
}
{
  "type": "turn.started"
}
{
  "type": "item.completed",
  "item": {
    "id": "item_0",
    "item_type": "reasoning",
    "text": "**Planning to list assigned GitHub issues**"
  }
}
{
  "type": "item.started",
  "item": {
    "id": "item_1",
    "item_type": "mcp_tool_call",
    "server": "github",
    "tool": "search_issues",
    "status": "in_progress"
  }
}
{
  "type": "item.completed",
  "item": {
    "id": "item_1",
    "item_type": "mcp_tool_call",
    "server": "github",
    "tool": "search_issues",
    "status": "completed"
  }
}
{
  "type": "item.completed",
  "item": {
    "id": "item_2",
    "item_type": "reasoning",
    "text": "**Organizing final message structure**"
  }
}
{
  "type": "item.completed",
  "item": {
    "id": "item_3",
    "item_type": "assistant_message",
    "text": "**Assigned Issues**\n- openai/codex#3267 – “stream error: stream disconnected before completion…” (bug) – last update 2025-09-08\n- openai/codex#3257 – “You've hit your usage limit. Try again in 4 days 20 hours 9 minutes.” – last update 2025-09-23\n- openai/codex#3054 – “reqwest SSL panic (library has no ciphers)” (bug) – last update 2025-09-03\n- openai/codex#3051 – “thread 'main' panicked at linux-sandbox/src/linux_run_main.rs:53:5:” (bug) – last update 2025-09-10\n- openai/codex#3004 – “Auto-compact when approaching context limit” (enhancement) – last update 2025-09-26\n- openai/codex#2916 – “Feature request: Add OpenAI service tier support for cost optimization” – last update 2025-09-12\n- openai/codex#1581 – “stream error: stream disconnected before completion: stream closed before response.complete; retrying...” (bug) – last update 2025-09-17"
  }
}
{
  "type": "turn.completed",
  "usage": {
    "input_tokens": 34785,
    "cached_input_tokens": 12544,
    "output_tokens": 560
  }
}
```

</details>
2025-09-30 17:21:37 -07:00
Jeremy Rose
01e6503672 wrap markdown at render time (#4506)
This results in correctly indenting list items with long lines.

<img width="1006" height="251" alt="Screenshot 2025-09-30 at 10 00
48 AM"
src="https://github.com/user-attachments/assets/0a076cf6-ca3c-4efb-b3af-dc07617cdb6f"
/>
2025-09-30 23:13:55 +00:00
pakrym-oai
9c259737d3 Delete codex proto (#4520) 2025-09-30 22:33:28 +00:00
Michael Bolin
b8e1fe60c5 fix: enable process hardening in Codex CLI for release builds (#4521)
I don't believe there is any upside in making process hardening opt-in
for Codex CLI releases. If you want to tinker with Codex CLI, then build
from source (or run as `root`)?
2025-09-30 14:34:35 -07:00
Michael Bolin
ddfb7eb548 fix: clean up TypeScript exports (#4518)
Fixes:

- Removed overdeclaration of types that were unnecessary because they
were already included by induction.
- Reordered list of response types to match the enum order, making it
easier to identify what was missing.
- Added `ExecArbitraryCommandResponse` because it was missing.
- Leveraged `use codex_protocol::mcp_protocol::*;` to make the file more
readable.
- Removed crate dependency on `mcp-types` now that we have separate the
app server from the MCP server:
https://github.com/openai/codex/pull/4471

My next move is to come up with some scheme that ensures request types
always have a response type and that the response type is automatically
included with the output of `codex generate-ts`.
2025-09-30 14:08:43 -07:00
Michael Bolin
6910be3224 fix: ensure every variant of ClientRequest has a params field (#4512)
This ensures changes the generated TypeScript type for `ClientRequest`
so that instead of this:

```typescript
/**
 * Request from the client to the server.
 */
export type ClientRequest =
  | { method: "initialize"; id: RequestId; params: InitializeParams }
  | { method: "newConversation"; id: RequestId; params: NewConversationParams }
  // ...
  | { method: "getUserAgent"; id: RequestId }
  | { method: "userInfo"; id: RequestId }
  // ...
```

we have this:

```typescript
/**
 * Request from the client to the server.
 */
export type ClientRequest =
  | { method: "initialize"; id: RequestId; params: InitializeParams }
  | { method: "newConversation"; id: RequestId; params: NewConversationParams }
  // ...
  | { method: "getUserAgent"; id: RequestId; params: undefined }
  | { method: "userInfo"; id: RequestId; params: undefined }
  // ...
```

which makes TypeScript happier when it comes to destructuring instances
of `ClientRequest` because it does not complain about `params` not being
guaranteed to exist anymore.
2025-09-30 12:03:32 -07:00
pakrym-oai
a534356fe1 Wire up web search item (#4511)
Add handling for web search events.
2025-09-30 12:01:17 -07:00
pakrym-oai
c89b0e1235 [SDK] Test that a tread can be continued with extra params (#4508) 2025-09-30 17:22:14 +00:00
jif-oai
f6a152848a chore: prompt update to enforce good usage of apply_patch (#3846)
Update prompt to prevent codex to use Python script or fancy commands to
edit files.

## Testing:
3 scenarios have been considered:
1. Rename codex to meca_code. Proceed to the whole refactor file by
file. Don't ask for approval at each step
2. Add a description to every single function you can find in the repo
3. Rewrite codex.rs in a more idiomatic way. Make sure to touch ONLY
this file and that clippy does not complain at the end

Before this update, 22% (estimation as it's sometimes hard to find all
the creative way the model find to edit files) of the file editions
where made using something else than a raw `apply_patch`

After this update, not a single edition without `apply_patch` was found

[EDIT]
I managed to have a few `["bash", "-lc", "apply_path"]` when reaching <
10% context left
2025-09-30 10:18:59 -07:00
dedrisian-oai
3592ecb23c Named args for custom prompts (#4474)
Here's the logic:

1. If text is empty and selector is open:
- Enter on a prompt without args should autosubmit the prompt
- Enter on a prompt with numeric args should add `/prompts:name ` to the
text input
- Enter on a prompt with named args should add `/prompts:name ARG1=""
ARG2=""` to the text input
2. If text is not empty but no args are passed:
- For prompts with numeric args -> we allow it to submit (params are
optional)
- For prompts with named args -> we throw an error (all params should
have values)

<img width="454" height="246" alt="Screenshot 2025-09-23 at 2 23 21 PM"
src="https://github.com/user-attachments/assets/fd180a1b-7d17-42ec-b231-8da48828b811"
/>
2025-09-30 10:06:41 -07:00
pakrym-oai
516acc030b Support model and sandbox mode in the sdk (#4503) 2025-09-30 09:00:39 -07:00
easong-openai
5b038135de Add cloud tasks (#3197)
Adds a TUI for managing, applying, and creating cloud tasks
2025-09-30 10:10:33 +00:00
Michael Bolin
d9dbf48828 fix: separate codex mcp into codex mcp-server and codex app-server (#4471)
This is a very large PR with some non-backwards-compatible changes.

Historically, `codex mcp` (or `codex mcp serve`) started a JSON-RPC-ish
server that had two overlapping responsibilities:

- Running an MCP server, providing some basic tool calls.
- Running the app server used to power experiences such as the VS Code
extension.

This PR aims to separate these into distinct concepts:

- `codex mcp-server` for the MCP server
- `codex app-server` for the "application server"

Note `codex mcp` still exists because it already has its own subcommands
for MCP management (`list`, `add`, etc.)

The MCP logic continues to live in `codex-rs/mcp-server` whereas the
refactored app server logic is in the new `codex-rs/app-server` folder.
Note that most of the existing integration tests in
`codex-rs/mcp-server/tests/suite` were actually for the app server, so
all the tests have been moved with the exception of
`codex-rs/mcp-server/tests/suite/mod.rs`.

Because this is already a large diff, I tried not to change more than I
had to, so `codex-rs/app-server/tests/common/mcp_process.rs` still uses
the name `McpProcess` for now, but I will do some mechanical renamings
to things like `AppServer` in subsequent PRs.

While `mcp-server` and `app-server` share some overlapping functionality
(like reading streams of JSONL and dispatching based on message types)
and some differences (completely different message types), I ended up
doing a bit of copypasta between the two crates, as both have somewhat
similar `message_processor.rs` and `outgoing_message.rs` files for now,
though I expect them to diverge more in the near future.

One material change is that of the initialize handshake for `codex
app-server`, as we no longer use the MCP types for that handshake.
Instead, we update `codex-rs/protocol/src/mcp_protocol.rs` to add an
`Initialize` variant to `ClientRequest`, which takes the `ClientInfo`
object we need to update the `USER_AGENT_SUFFIX` in
`codex-rs/app-server/src/message_processor.rs`.

One other material change is in
`codex-rs/app-server/src/codex_message_processor.rs` where I eliminated
a use of the `send_event_as_notification()` method I am generally trying
to deprecate (because it blindly maps an `EventMsg` into a
`JSONNotification`) in favor of `send_server_notification()`, which
takes a `ServerNotification`, as that is intended to be a custom enum of
all notification types supported by the app server. So to make this
update, I had to introduce a new variant of `ServerNotification`,
`SessionConfigured`, which is a non-backwards compatible change with the
old `codex mcp`, and clients will have to be updated after the next
release that contains this PR. Note that
`codex-rs/app-server/tests/suite/list_resume.rs` also had to be update
to reflect this change.

I introduced `codex-rs/utils/json-to-toml/src/lib.rs` as a small utility
crate to avoid some of the copying between `mcp-server` and
`app-server`.
2025-09-30 07:06:18 +00:00
Gabriel Peal
2e95e5602d Update MCP docs to reference experimental RMCP client (#4422) 2025-09-30 02:44:16 -04:00
dedrisian-oai
87a654cf6b Move PR-style review to top (#4486)
<img width="469" height="330" alt="Screenshot 2025-09-29 at 10 31 22 PM"
src="https://github.com/user-attachments/assets/b5e20a08-85b4-4095-8a7f-0f58d1195b7e"
/>
2025-09-30 06:03:37 +00:00
pakrym-oai
27c6c5d7a7 SDK CI (#4483)
Build debug codex in SDK configuration
2025-09-29 21:15:02 -07:00
pakrym-oai
c09e131653 Set originator for codex exec (#4485)
Distinct from the main CLI.
2025-09-29 20:59:19 -07:00
pakrym-oai
ea82f86662 Rename conversation to thread in codex exec (#4482) 2025-09-29 20:18:30 -07:00
pakrym-oai
a8edc57740 Add MCP tool call item to codex exec (#4481)
No arguments/results for now.
```
{
  "type": "item.started",
  "item": {
    "id": "item_1",
    "item_type": "mcp_tool_call",
    "server": "github",
    "tool": "search_issues",
    "status": "in_progress"
  }
}
{
  "type": "item.completed",
  "item": {
    "id": "item_1",
    "item_type": "mcp_tool_call",
    "server": "github",
    "tool": "search_issues",
    "status": "completed"
  }
}
```
2025-09-29 19:45:11 -07:00
pakrym-oai
52e591ce60 Add some types and a basic test to the SDK (#4472) 2025-09-29 19:40:08 -07:00
rakesh-oai
079303091f Rakesh/support device auth (#3531)
# External (non-OpenAI) Pull Request Requirements

Before opening this Pull Request, please read the dedicated
"Contributing" markdown file or your PR may be closed:
https://github.com/openai/codex/blob/main/docs/contributing.md

If your PR conforms to our contribution guidelines, replace this text
with a detailed and high quality description of your changes.

# test

```
codex-rs % export CODEX_DEVICE_AUTH_BASE_URL=http://localhost:3007
codex-rs % cargo run --bin codex login --experimental_use-device-code
   Compiling codex-login v0.0.0 (/Users/rakesh/code/codex/codex-rs/login)
   Compiling codex-mcp-server v0.0.0 (/Users/rakesh/code/codex/codex-rs/mcp-server)
   Compiling codex-tui v0.0.0 (/Users/rakesh/code/codex/codex-rs/tui)
   Compiling codex-cli v0.0.0 (/Users/rakesh/code/codex/codex-rs/cli)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 2.90s
     Running `target/debug/codex login --experimental_use-device-code`
To authenticate, enter this code when prompted: 6Q27-KBVRF with interval 5
^C

```

The error in the last line is since the poll endpoint is not yet
implemented
2025-09-30 02:34:57 +00:00
pakrym-oai
4a80059b1b Add turn.failed and rename session created to thread started (#4478)
Don't produce completed when turn failed.
2025-09-29 18:38:04 -07:00
dedrisian-oai
bf76258cdc Custom prompts begin with /prompts: (#4476)
<img width="608" height="354" alt="Screenshot 2025-09-29 at 4 41 08 PM"
src="https://github.com/user-attachments/assets/162508eb-c1ac-4bc0-95f2-5e23cb4ae428"
/>
2025-09-29 17:58:16 -07:00
Ahmed Ibrahim
c64da4ff71 Fixes (#4458)
Fixing the "? for shortcuts"

- Only show the hint when composer is empty
- Don't reset footer on new task updates
- Reorder the elements
- Align the "?" and "/" with overlay on and off

Based on #4364
2025-09-30 00:10:04 +00:00
Ahmed Ibrahim
98efd352ae reintroduce "? for shortcuts" (#4364)
Reverts openai/codex#4362
2025-09-29 23:35:47 +00:00
dedrisian-oai
80ccec6530 Custom prompt args (numeric) (#4470)
[Cherry picked from /pull/3565]

Adds $1, $2, $3, $ARGUMENTS param parsing for custom prompts.
2025-09-29 16:14:37 -07:00
Jeremy Rose
c81baaabda no background for /command or @file popup (#4469)
before:

<img width="855" height="270" alt="Screenshot 2025-09-29 at 3 42 53 PM"
src="https://github.com/user-attachments/assets/eb247e1f-0947-4830-93c4-d4ecb2992b32"
/>


after:

<img width="855" height="270" alt="Screenshot 2025-09-29 at 3 43 04 PM"
src="https://github.com/user-attachments/assets/46717844-6066-47a4-a34a-1a75508ea2c3"
/>
2025-09-29 22:58:15 +00:00
Jeremy Rose
55b74c95e2 render • as dim (#4467)
<img width="988" height="686" alt="Screenshot 2025-09-29 at 3 28 30 PM"
src="https://github.com/user-attachments/assets/634a6e6f-cdc0-49af-97c1-096e871414bb"
/>
2025-09-29 15:46:47 -07:00
Ahmed Ibrahim
16057e76b0 [Core]: add tail in the rollout data (#4461)
This will help us show the conversation tail and last updated timestamp.
2025-09-29 14:32:26 -07:00
pakrym-oai
adbc38a978 TypeScript SDK scaffold (#4455) 2025-09-29 13:27:13 -07:00
dedrisian-oai
83a4d4d8ed Parse out frontmatter for custom prompts (#4456)
[Cherry picked from https://github.com/openai/codex/pull/3565]

Removes the frontmatter description/args from custom prompt files and
only includes body.
2025-09-29 13:06:08 -07:00
Dylan
197f45a3be [mcp-server] Expose fuzzy file search in MCP (#2677)
## Summary
Expose a simple fuzzy file search implementation for mcp clients to work
with

## Testing
- [x] Tested locally
2025-09-29 12:19:09 -07:00
vishnu-oai
04c1782e52 OpenTelemetry events (#2103)
### Title

## otel

Codex can emit [OpenTelemetry](https://opentelemetry.io/) **log events**
that
describe each run: outbound API requests, streamed responses, user
input,
tool-approval decisions, and the result of every tool invocation. Export
is
**disabled by default** so local runs remain self-contained. Opt in by
adding an
`[otel]` table and choosing an exporter.

```toml
[otel]
environment = "staging"   # defaults to "dev"
exporter = "none"          # defaults to "none"; set to otlp-http or otlp-grpc to send events
log_user_prompt = false    # defaults to false; redact prompt text unless explicitly enabled
```

Codex tags every exported event with `service.name = "codex-cli"`, the
CLI
version, and an `env` attribute so downstream collectors can distinguish
dev/staging/prod traffic. Only telemetry produced inside the
`codex_otel`
crate—the events listed below—is forwarded to the exporter.

### Event catalog

Every event shares a common set of metadata fields: `event.timestamp`,
`conversation.id`, `app.version`, `auth_mode` (when available),
`user.account_id` (when available), `terminal.type`, `model`, and
`slug`.

With OTEL enabled Codex emits the following event types (in addition to
the
metadata above):

- `codex.api_request`
  - `cf_ray` (optional)
  - `attempt`
  - `duration_ms`
  - `http.response.status_code` (optional)
  - `error.message` (failures)
- `codex.sse_event`
  - `event.kind`
  - `duration_ms`
  - `error.message` (failures)
  - `input_token_count` (completion only)
  - `output_token_count` (completion only)
  - `cached_token_count` (completion only, optional)
  - `reasoning_token_count` (completion only, optional)
  - `tool_token_count` (completion only)
- `codex.user_prompt`
  - `prompt_length`
  - `prompt` (redacted unless `log_user_prompt = true`)
- `codex.tool_decision`
  - `tool_name`
  - `call_id`
- `decision` (`approved`, `approved_for_session`, `denied`, or `abort`)
  - `source` (`config` or `user`)
- `codex.tool_result`
  - `tool_name`
  - `call_id`
  - `arguments`
  - `duration_ms` (execution time for the tool)
  - `success` (`"true"` or `"false"`)
  - `output`

### Choosing an exporter

Set `otel.exporter` to control where events go:

- `none` – leaves instrumentation active but skips exporting. This is
the
  default.
- `otlp-http` – posts OTLP log records to an OTLP/HTTP collector.
Specify the
  endpoint, protocol, and headers your collector expects:

  ```toml
  [otel]
  exporter = { otlp-http = {
    endpoint = "https://otel.example.com/v1/logs",
    protocol = "binary",
    headers = { "x-otlp-api-key" = "${OTLP_TOKEN}" }
  }}
  ```

- `otlp-grpc` – streams OTLP log records over gRPC. Provide the endpoint
and any
  metadata headers:

  ```toml
  [otel]
  exporter = { otlp-grpc = {
    endpoint = "https://otel.example.com:4317",
    headers = { "x-otlp-meta" = "abc123" }
  }}
  ```

If the exporter is `none` nothing is written anywhere; otherwise you
must run or point to your
own collector. All exporters run on a background batch worker that is
flushed on
shutdown.

If you build Codex from source the OTEL crate is still behind an `otel`
feature
flag; the official prebuilt binaries ship with the feature enabled. When
the
feature is disabled the telemetry hooks become no-ops so the CLI
continues to
function without the extra dependencies.

---------

Co-authored-by: Anton Panasenko <apanasenko@openai.com>
2025-09-29 11:30:55 -07:00
Jeremy Rose
d15253415a fix clear-to-end being emitted at the end of a row (#4447)
This was causing glitchy behavior when a line in the input was the exact
width of the terminal.
2025-09-29 16:52:35 +00:00
291 changed files with 26824 additions and 4612 deletions

View File

@@ -1,6 +1,6 @@
[codespell]
# Ref: https://github.com/codespell-project/codespell#using-a-config-file
skip = .git*,vendor,*-lock.yaml,*.lock,.codespellrc,*test.ts,*.jsonl
skip = .git*,vendor,*-lock.yaml,*.lock,.codespellrc,*test.ts,*.jsonl,frame*.txt
check-hidden = true
ignore-regex = ^\s*"image/\S+": ".*|\b(afterAll)\b
ignore-words-list = ratatui,ser

View File

@@ -27,7 +27,7 @@ jobs:
- name: Install dependencies
run: pnpm install --frozen-lockfile
# build_npm_package.py requires DotSlash when staging releases.
# stage_npm_packages.py requires DotSlash when staging releases.
- uses: facebook/install-dotslash@v2
- name: Stage npm package
@@ -37,10 +37,12 @@ jobs:
run: |
set -euo pipefail
CODEX_VERSION=0.40.0
PACK_OUTPUT="${RUNNER_TEMP}/codex-npm.tgz"
python3 ./codex-cli/scripts/build_npm_package.py \
OUTPUT_DIR="${RUNNER_TEMP}"
python3 ./scripts/stage_npm_packages.py \
--release-version "$CODEX_VERSION" \
--pack-output "$PACK_OUTPUT"
--package codex \
--output-dir "$OUTPUT_DIR"
PACK_OUTPUT="${OUTPUT_DIR}/codex-npm-${CODEX_VERSION}.tgz"
echo "pack_output=$PACK_OUTPUT" >> "$GITHUB_OUTPUT"
- name: Upload staged npm package artifact

View File

@@ -25,4 +25,3 @@ jobs:
uses: codespell-project/actions-codespell@406322ec52dd7b488e48c1c4b82e2a8b3a1bf630 # v2.1
with:
ignore_words_file: .codespellignore
skip: frame*.txt

View File

@@ -216,31 +216,30 @@ jobs:
echo "npm_tag=" >> "$GITHUB_OUTPUT"
fi
# build_npm_package.py requires DotSlash when staging releases.
- uses: facebook/install-dotslash@v2
- name: Stage codex CLI npm package
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
TMP_DIR="${RUNNER_TEMP}/npm-stage"
./codex-cli/scripts/build_npm_package.py \
--package codex \
--release-version "${{ steps.release_name.outputs.name }}" \
--staging-dir "${TMP_DIR}" \
--pack-output "${GITHUB_WORKSPACE}/dist/npm/codex-npm-${{ steps.release_name.outputs.name }}.tgz"
- name: Setup pnpm
uses: pnpm/action-setup@v4
with:
run_install: false
- name: Stage responses API proxy npm package
- name: Setup Node.js for npm packaging
uses: actions/setup-node@v5
with:
node-version: 22
- name: Install dependencies
run: pnpm install --frozen-lockfile
# stage_npm_packages.py requires DotSlash when staging releases.
- uses: facebook/install-dotslash@v2
- name: Stage npm packages
env:
GH_TOKEN: ${{ github.token }}
run: |
set -euo pipefail
TMP_DIR="${RUNNER_TEMP}/npm-stage-responses"
./codex-cli/scripts/build_npm_package.py \
--package codex-responses-api-proxy \
./scripts/stage_npm_packages.py \
--release-version "${{ steps.release_name.outputs.name }}" \
--staging-dir "${TMP_DIR}" \
--pack-output "${GITHUB_WORKSPACE}/dist/npm/codex-responses-api-proxy-npm-${{ steps.release_name.outputs.name }}.tgz"
--package codex \
--package codex-responses-api-proxy \
--package codex-sdk
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
@@ -300,6 +299,10 @@ jobs:
--repo "${GITHUB_REPOSITORY}" \
--pattern "codex-responses-api-proxy-npm-${version}.tgz" \
--dir dist/npm
gh release download "$tag" \
--repo "${GITHUB_REPOSITORY}" \
--pattern "codex-sdk-npm-${version}.tgz" \
--dir dist/npm
# No NODE_AUTH_TOKEN needed because we use OIDC.
- name: Publish to npm
@@ -316,6 +319,7 @@ jobs:
tarballs=(
"codex-npm-${VERSION}.tgz"
"codex-responses-api-proxy-npm-${VERSION}.tgz"
"codex-sdk-npm-${VERSION}.tgz"
)
for tarball in "${tarballs[@]}"; do

43
.github/workflows/sdk.yml vendored Normal file
View File

@@ -0,0 +1,43 @@
name: sdk
on:
push:
branches: [main]
pull_request: {}
jobs:
sdks:
runs-on: ubuntu-latest
timeout-minutes: 10
steps:
- name: Checkout repository
uses: actions/checkout@v5
- name: Setup pnpm
uses: pnpm/action-setup@v4
with:
run_install: false
- name: Setup Node.js
uses: actions/setup-node@v5
with:
node-version: 22
cache: pnpm
- uses: dtolnay/rust-toolchain@1.90
- name: build codex
run: cargo build --bin codex
working-directory: codex-rs
- name: Install dependencies
run: pnpm install --frozen-lockfile
- name: Build SDK packages
run: pnpm -r --filter ./sdk/typescript run build
- name: Lint SDK packages
run: pnpm -r --filter ./sdk/typescript run lint
- name: Test SDK packages
run: pnpm -r --filter ./sdk/typescript run test

View File

@@ -1,11 +1,19 @@
# npm releases
Run the following:
To build the 0.2.x or later version of the npm module, which runs the Rust version of the CLI, build it as follows:
Use the staging helper in the repo root to generate npm tarballs for a release. For
example, to stage the CLI, responses proxy, and SDK packages for version `0.6.0`:
```bash
./codex-cli/scripts/build_npm_package.py --release-version 0.6.0
./scripts/stage_npm_packages.py \
--release-version 0.6.0 \
--package codex \
--package codex-responses-api-proxy \
--package codex-sdk
```
Note this will create `./codex-cli/vendor/` as a side-effect.
This downloads the native artifacts once, hydrates `vendor/` for each package, and writes
tarballs to `dist/npm/`.
If you need to invoke `build_npm_package.py` directly, run
`codex-cli/scripts/install_native_deps.py` first and pass `--vendor-src` pointing to the
directory that contains the populated `vendor/` tree.

View File

@@ -3,7 +3,6 @@
import argparse
import json
import re
import shutil
import subprocess
import sys
@@ -14,19 +13,25 @@ SCRIPT_DIR = Path(__file__).resolve().parent
CODEX_CLI_ROOT = SCRIPT_DIR.parent
REPO_ROOT = CODEX_CLI_ROOT.parent
RESPONSES_API_PROXY_NPM_ROOT = REPO_ROOT / "codex-rs" / "responses-api-proxy" / "npm"
GITHUB_REPO = "openai/codex"
CODEX_SDK_ROOT = REPO_ROOT / "sdk" / "typescript"
# The docs are not clear on what the expected value/format of
# workflow/workflowName is:
# https://cli.github.com/manual/gh_run_list
WORKFLOW_NAME = ".github/workflows/rust-release.yml"
PACKAGE_NATIVE_COMPONENTS: dict[str, list[str]] = {
"codex": ["codex", "rg"],
"codex-responses-api-proxy": ["codex-responses-api-proxy"],
"codex-sdk": ["codex"],
}
COMPONENT_DEST_DIR: dict[str, str] = {
"codex": "codex",
"codex-responses-api-proxy": "codex-responses-api-proxy",
"rg": "path",
}
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Build or stage the Codex CLI npm package.")
parser.add_argument(
"--package",
choices=("codex", "codex-responses-api-proxy"),
choices=("codex", "codex-responses-api-proxy", "codex-sdk"),
default="codex",
help="Which npm package to stage (default: codex).",
)
@@ -37,14 +42,9 @@ def parse_args() -> argparse.Namespace:
parser.add_argument(
"--release-version",
help=(
"Version to stage for npm release. When provided, the script also resolves the "
"matching rust-release workflow unless --workflow-url is supplied."
"Version to stage for npm release."
),
)
parser.add_argument(
"--workflow-url",
help="Optional GitHub Actions workflow run URL used to download native binaries.",
)
parser.add_argument(
"--staging-dir",
type=Path,
@@ -64,6 +64,11 @@ def parse_args() -> argparse.Namespace:
type=Path,
help="Path where the generated npm tarball should be written.",
)
parser.add_argument(
"--vendor-src",
type=Path,
help="Directory containing pre-installed native binaries to bundle (vendor root).",
)
return parser.parse_args()
@@ -86,29 +91,19 @@ def main() -> int:
try:
stage_sources(staging_dir, version, package)
workflow_url = args.workflow_url
resolved_head_sha: str | None = None
if not workflow_url:
if release_version:
workflow = resolve_release_workflow(version)
workflow_url = workflow["url"]
resolved_head_sha = workflow.get("headSha")
else:
workflow_url = resolve_latest_alpha_workflow_url()
elif release_version:
try:
workflow = resolve_release_workflow(version)
resolved_head_sha = workflow.get("headSha")
except Exception:
resolved_head_sha = None
vendor_src = args.vendor_src.resolve() if args.vendor_src else None
native_components = PACKAGE_NATIVE_COMPONENTS.get(package, [])
if release_version and resolved_head_sha:
print(f"should `git checkout {resolved_head_sha}`")
if native_components:
if vendor_src is None:
components_str = ", ".join(native_components)
raise RuntimeError(
"Native components "
f"({components_str}) required for package '{package}'. Provide --vendor-src "
"pointing to a directory containing pre-installed binaries."
)
if not workflow_url:
raise RuntimeError("Unable to determine workflow URL for native binaries.")
install_native_binaries(staging_dir, workflow_url, package)
copy_native_binaries(vendor_src, staging_dir, native_components)
if release_version:
staging_dir_str = str(staging_dir)
@@ -119,12 +114,20 @@ def main() -> int:
f" node {staging_dir_str}/bin/codex.js --version\n"
f" node {staging_dir_str}/bin/codex.js --help\n\n"
)
else:
elif package == "codex-responses-api-proxy":
print(
f"Staged version {version} for release in {staging_dir_str}\n\n"
"Verify the responses API proxy:\n"
f" node {staging_dir_str}/bin/codex-responses-api-proxy.js --help\n\n"
)
else:
print(
f"Staged version {version} for release in {staging_dir_str}\n\n"
"Verify the SDK contents:\n"
f" ls {staging_dir_str}/dist\n"
f" ls {staging_dir_str}/vendor\n"
" node -e \"import('./dist/index.js').then(() => console.log('ok'))\"\n\n"
)
else:
print(f"Staged package in {staging_dir}")
@@ -152,10 +155,9 @@ def prepare_staging_dir(staging_dir: Path | None) -> tuple[Path, bool]:
def stage_sources(staging_dir: Path, version: str, package: str) -> None:
bin_dir = staging_dir / "bin"
bin_dir.mkdir(parents=True, exist_ok=True)
if package == "codex":
bin_dir = staging_dir / "bin"
bin_dir.mkdir(parents=True, exist_ok=True)
shutil.copy2(CODEX_CLI_ROOT / "bin" / "codex.js", bin_dir / "codex.js")
rg_manifest = CODEX_CLI_ROOT / "bin" / "rg"
if rg_manifest.exists():
@@ -167,6 +169,8 @@ def stage_sources(staging_dir: Path, version: str, package: str) -> None:
package_json_path = CODEX_CLI_ROOT / "package.json"
elif package == "codex-responses-api-proxy":
bin_dir = staging_dir / "bin"
bin_dir.mkdir(parents=True, exist_ok=True)
launcher_src = RESPONSES_API_PROXY_NPM_ROOT / "bin" / "codex-responses-api-proxy.js"
shutil.copy2(launcher_src, bin_dir / "codex-responses-api-proxy.js")
@@ -175,6 +179,9 @@ def stage_sources(staging_dir: Path, version: str, package: str) -> None:
shutil.copy2(readme_src, staging_dir / "README.md")
package_json_path = RESPONSES_API_PROXY_NPM_ROOT / "package.json"
elif package == "codex-sdk":
package_json_path = CODEX_SDK_ROOT / "package.json"
stage_codex_sdk_sources(staging_dir)
else:
raise RuntimeError(f"Unknown package '{package}'.")
@@ -182,91 +189,85 @@ def stage_sources(staging_dir: Path, version: str, package: str) -> None:
package_json = json.load(fh)
package_json["version"] = version
if package == "codex-sdk":
scripts = package_json.get("scripts")
if isinstance(scripts, dict):
scripts.pop("prepare", None)
files = package_json.get("files")
if isinstance(files, list):
if "vendor" not in files:
files.append("vendor")
else:
package_json["files"] = ["dist", "vendor"]
with open(staging_dir / "package.json", "w", encoding="utf-8") as out:
json.dump(package_json, out, indent=2)
out.write("\n")
def install_native_binaries(staging_dir: Path, workflow_url: str, package: str) -> None:
package_components = {
"codex": ["codex", "rg"],
"codex-responses-api-proxy": ["codex-responses-api-proxy"],
}
components = package_components.get(package)
if components is None:
raise RuntimeError(f"Unknown package '{package}'.")
cmd = ["./scripts/install_native_deps.py", "--workflow-url", workflow_url]
for component in components:
cmd.extend(["--component", component])
cmd.append(str(staging_dir))
subprocess.check_call(cmd, cwd=CODEX_CLI_ROOT)
def run_command(cmd: list[str], cwd: Path | None = None) -> None:
print("+", " ".join(cmd))
subprocess.run(cmd, cwd=cwd, check=True)
def resolve_latest_alpha_workflow_url() -> str:
version = determine_latest_alpha_version()
workflow = resolve_release_workflow(version)
return workflow["url"]
def stage_codex_sdk_sources(staging_dir: Path) -> None:
package_root = CODEX_SDK_ROOT
run_command(["pnpm", "install", "--frozen-lockfile"], cwd=package_root)
run_command(["pnpm", "run", "build"], cwd=package_root)
dist_src = package_root / "dist"
if not dist_src.exists():
raise RuntimeError("codex-sdk build did not produce a dist directory.")
shutil.copytree(dist_src, staging_dir / "dist")
readme_src = package_root / "README.md"
if readme_src.exists():
shutil.copy2(readme_src, staging_dir / "README.md")
license_src = REPO_ROOT / "LICENSE"
if license_src.exists():
shutil.copy2(license_src, staging_dir / "LICENSE")
def determine_latest_alpha_version() -> str:
releases = list_releases()
best_key: tuple[int, int, int, int] | None = None
best_version: str | None = None
pattern = re.compile(r"^rust-v(\d+)\.(\d+)\.(\d+)-alpha\.(\d+)$")
for release in releases:
tag = release.get("tag_name", "")
match = pattern.match(tag)
if not match:
def copy_native_binaries(vendor_src: Path, staging_dir: Path, components: list[str]) -> None:
vendor_src = vendor_src.resolve()
if not vendor_src.exists():
raise RuntimeError(f"Vendor source directory not found: {vendor_src}")
components_set = {component for component in components if component in COMPONENT_DEST_DIR}
if not components_set:
return
vendor_dest = staging_dir / "vendor"
if vendor_dest.exists():
shutil.rmtree(vendor_dest)
vendor_dest.mkdir(parents=True, exist_ok=True)
for target_dir in vendor_src.iterdir():
if not target_dir.is_dir():
continue
key = tuple(int(match.group(i)) for i in range(1, 5))
if best_key is None or key > best_key:
best_key = key
best_version = (
f"{match.group(1)}.{match.group(2)}.{match.group(3)}-alpha.{match.group(4)}"
)
if best_version is None:
raise RuntimeError("No alpha releases found when resolving workflow URL.")
return best_version
dest_target_dir = vendor_dest / target_dir.name
dest_target_dir.mkdir(parents=True, exist_ok=True)
for component in components_set:
dest_dir_name = COMPONENT_DEST_DIR.get(component)
if dest_dir_name is None:
continue
def list_releases() -> list[dict]:
stdout = subprocess.check_output(
["gh", "api", f"/repos/{GITHUB_REPO}/releases?per_page=100"],
text=True,
)
try:
releases = json.loads(stdout or "[]")
except json.JSONDecodeError as exc:
raise RuntimeError("Unable to parse releases JSON.") from exc
if not isinstance(releases, list):
raise RuntimeError("Unexpected response when listing releases.")
return releases
src_component_dir = target_dir / dest_dir_name
if not src_component_dir.exists():
raise RuntimeError(
f"Missing native component '{component}' in vendor source: {src_component_dir}"
)
def resolve_release_workflow(version: str) -> dict:
stdout = subprocess.check_output(
[
"gh",
"run",
"list",
"--branch",
f"rust-v{version}",
"--json",
"workflowName,url,headSha",
"--workflow",
WORKFLOW_NAME,
"--jq",
"first(.[])",
],
text=True,
)
workflow = json.loads(stdout or "[]")
if not workflow:
raise RuntimeError(f"Unable to find rust-release workflow for version {version}.")
return workflow
dest_component_dir = dest_target_dir / dest_dir_name
if dest_component_dir.exists():
shutil.rmtree(dest_component_dir)
shutil.copytree(src_component_dir, dest_component_dir)
def run_npm_pack(staging_dir: Path, output_path: Path) -> Path:

1271
codex-rs/Cargo.lock generated

File diff suppressed because it is too large Load Diff

View File

@@ -1,8 +1,14 @@
[workspace]
members = [
"backend-client",
"ansi-escape",
"app-server",
"app-server-protocol",
"apply-patch",
"arg0",
"codex-backend-openapi-models",
"cloud-tasks",
"cloud-tasks-client",
"cli",
"common",
"core",
@@ -21,7 +27,10 @@ members = [
"protocol-ts",
"rmcp-client",
"responses-api-proxy",
"otel",
"tui",
"git-apply",
"utils/json-to-toml",
"utils/readiness",
]
resolver = "2"
@@ -36,7 +45,10 @@ edition = "2024"
[workspace.dependencies]
# Internal
app_test_support = { path = "app-server/tests/common" }
codex-ansi-escape = { path = "ansi-escape" }
codex-app-server = { path = "app-server" }
codex-app-server-protocol = { path = "app-server-protocol" }
codex-apply-patch = { path = "apply-patch" }
codex-arg0 = { path = "arg0" }
codex-chatgpt = { path = "chatgpt" }
@@ -50,11 +62,14 @@ codex-login = { path = "login" }
codex-mcp-client = { path = "mcp-client" }
codex-mcp-server = { path = "mcp-server" }
codex-ollama = { path = "ollama" }
codex-otel = { path = "otel" }
codex-process-hardening = { path = "process-hardening" }
codex-protocol = { path = "protocol" }
codex-protocol-ts = { path = "protocol-ts" }
codex-responses-api-proxy = { path = "responses-api-proxy" }
codex-rmcp-client = { path = "rmcp-client" }
codex-tui = { path = "tui" }
codex-utils-json-to-toml = { path = "utils/json-to-toml" }
codex-utils-readiness = { path = "utils/readiness" }
core_test_support = { path = "core/tests/common" }
mcp-types = { path = "mcp-types" }
@@ -82,6 +97,7 @@ derive_more = "2"
diffy = "0.4.2"
dirs = "6"
dotenvy = "0.15.7"
dunce = "1.0.4"
env-flags = "0.1.1"
env_logger = "0.11.5"
escargot = "0.5"
@@ -103,8 +119,14 @@ mime_guess = "2.0.5"
multimap = "0.10.0"
nucleo-matcher = "0.3.1"
openssl-sys = "*"
opentelemetry = "0.30.0"
opentelemetry-appender-tracing = "0.30.0"
opentelemetry-otlp = "0.30.0"
opentelemetry-semantic-conventions = "0.30.0"
opentelemetry_sdk = "0.30.0"
os_info = "3.12.0"
owo-colors = "4.2.0"
paste = "1.0.15"
path-absolutize = "3.1.1"
path-clean = "1.0.1"
pathdiff = "0.2"
@@ -141,9 +163,11 @@ tokio-test = "0.4"
tokio-util = "0.7.16"
toml = "0.9.5"
toml_edit = "0.23.4"
tonic = "0.13.1"
tracing = "0.1.41"
tracing-appender = "0.2.3"
tracing-subscriber = "0.3.20"
tracing-test = "0.2.5"
tree-sitter = "0.25.9"
tree-sitter-bash = "0.25.0"
ts-rs = "11"

View File

@@ -25,12 +25,14 @@ Codex supports a rich set of configuration options. Note that the Rust CLI uses
Codex CLI functions as an MCP client that can connect to MCP servers on startup. See the [`mcp_servers`](../docs/config.md#mcp_servers) section in the configuration documentation for details.
It is still experimental, but you can also launch Codex as an MCP _server_ by running `codex mcp`. Use the [`@modelcontextprotocol/inspector`](https://github.com/modelcontextprotocol/inspector) to try it out:
It is still experimental, but you can also launch Codex as an MCP _server_ by running `codex mcp-server`. Use the [`@modelcontextprotocol/inspector`](https://github.com/modelcontextprotocol/inspector) to try it out:
```shell
npx @modelcontextprotocol/inspector codex mcp
npx @modelcontextprotocol/inspector codex mcp-server
```
Use `codex mcp` to add/list/get/remove MCP server launchers defined in `config.toml`, and `codex mcp-server` to run the MCP server directly.
### Notifications
You can enable notifications by configuring a script that is run whenever the agent finishes a turn. The [notify documentation](../docs/config.md#notify) includes a detailed example that explains how to get desktop notifications via [terminal-notifier](https://github.com/julienXX/terminal-notifier) on macOS.

View File

@@ -0,0 +1,24 @@
[package]
edition = "2024"
name = "codex-app-server-protocol"
version = { workspace = true }
[lib]
name = "codex_app_server_protocol"
path = "src/lib.rs"
[lints]
workspace = true
[dependencies]
codex-protocol = { workspace = true }
paste = { workspace = true }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
strum_macros = { workspace = true }
ts-rs = { workspace = true }
uuid = { workspace = true, features = ["serde", "v7"] }
[dev-dependencies]
anyhow = { workspace = true }
pretty_assertions = { workspace = true }

View File

@@ -0,0 +1,67 @@
//! We do not do true JSON-RPC 2.0, as we neither send nor expect the
//! "jsonrpc": "2.0" field.
use serde::Deserialize;
use serde::Serialize;
use ts_rs::TS;
pub const JSONRPC_VERSION: &str = "2.0";
#[derive(Debug, Clone, PartialEq, Deserialize, Serialize, Hash, Eq, TS)]
#[serde(untagged)]
pub enum RequestId {
String(String),
#[ts(type = "number")]
Integer(i64),
}
pub type Result = serde_json::Value;
/// Refers to any valid JSON-RPC object that can be decoded off the wire, or encoded to be sent.
#[derive(Debug, Clone, PartialEq, Deserialize, Serialize, TS)]
#[serde(untagged)]
pub enum JSONRPCMessage {
Request(JSONRPCRequest),
Notification(JSONRPCNotification),
Response(JSONRPCResponse),
Error(JSONRPCError),
}
/// A request that expects a response.
#[derive(Debug, Clone, PartialEq, Deserialize, Serialize, TS)]
pub struct JSONRPCRequest {
pub id: RequestId,
pub method: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub params: Option<serde_json::Value>,
}
/// A notification which does not expect a response.
#[derive(Debug, Clone, PartialEq, Deserialize, Serialize, TS)]
pub struct JSONRPCNotification {
pub method: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub params: Option<serde_json::Value>,
}
/// A successful (non-error) response to a request.
#[derive(Debug, Clone, PartialEq, Deserialize, Serialize, TS)]
pub struct JSONRPCResponse {
pub id: RequestId,
pub result: Result,
}
/// A response to a request that indicates an error occurred.
#[derive(Debug, Clone, PartialEq, Deserialize, Serialize, TS)]
pub struct JSONRPCError {
pub error: JSONRPCErrorError,
pub id: RequestId,
}
#[derive(Debug, Clone, PartialEq, Deserialize, Serialize, TS)]
pub struct JSONRPCErrorError {
pub code: i64,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub data: Option<serde_json::Value>,
pub message: String,
}

View File

@@ -0,0 +1,5 @@
mod jsonrpc_lite;
mod protocol;
pub use jsonrpc_lite::*;
pub use protocol::*;

View File

@@ -1,76 +1,27 @@
use std::collections::HashMap;
use std::fmt::Display;
use std::path::PathBuf;
use crate::config_types::ReasoningEffort;
use crate::config_types::ReasoningSummary;
use crate::config_types::SandboxMode;
use crate::config_types::Verbosity;
use crate::protocol::AskForApproval;
use crate::protocol::EventMsg;
use crate::protocol::FileChange;
use crate::protocol::ReviewDecision;
use crate::protocol::SandboxPolicy;
use crate::protocol::TurnAbortReason;
use mcp_types::RequestId;
use crate::JSONRPCNotification;
use crate::JSONRPCRequest;
use crate::RequestId;
use codex_protocol::ConversationId;
use codex_protocol::config_types::ReasoningEffort;
use codex_protocol::config_types::ReasoningSummary;
use codex_protocol::config_types::SandboxMode;
use codex_protocol::config_types::Verbosity;
use codex_protocol::protocol::AskForApproval;
use codex_protocol::protocol::EventMsg;
use codex_protocol::protocol::FileChange;
use codex_protocol::protocol::ReviewDecision;
use codex_protocol::protocol::SandboxPolicy;
use codex_protocol::protocol::TurnAbortReason;
use paste::paste;
use serde::Deserialize;
use serde::Serialize;
use strum_macros::Display;
use ts_rs::TS;
use uuid::Uuid;
#[derive(Debug, Clone, Copy, PartialEq, Eq, TS, Hash)]
#[ts(type = "string")]
pub struct ConversationId {
uuid: Uuid,
}
impl ConversationId {
pub fn new() -> Self {
Self {
uuid: Uuid::now_v7(),
}
}
pub fn from_string(s: &str) -> Result<Self, uuid::Error> {
Ok(Self {
uuid: Uuid::parse_str(s)?,
})
}
}
impl Default for ConversationId {
fn default() -> Self {
Self::new()
}
}
impl Display for ConversationId {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.uuid)
}
}
impl Serialize for ConversationId {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
serializer.collect_str(&self.uuid)
}
}
impl<'de> Deserialize<'de> for ConversationId {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
let value = String::deserialize(deserializer)?;
let uuid = Uuid::parse_str(&value).map_err(serde::de::Error::custom)?;
Ok(Self { uuid })
}
}
#[derive(Serialize, Deserialize, Clone, Debug, PartialEq, TS)]
#[ts(type = "string")]
pub struct GitSha(pub String);
@@ -81,117 +32,168 @@ impl GitSha {
}
}
#[derive(Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq, TS)]
#[derive(Serialize, Deserialize, Debug, Clone, Copy, PartialEq, Eq, Display, TS)]
#[serde(rename_all = "lowercase")]
pub enum AuthMode {
ApiKey,
ChatGPT,
}
/// Request from the client to the server.
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, TS)]
#[serde(tag = "method", rename_all = "camelCase")]
pub enum ClientRequest {
/// Generates an `enum ClientRequest` where each variant is a request that the
/// client can send to the server. Each variant has associated `params` and
/// `response` types. Also generates a `export_client_responses()` function to
/// export all response types to TypeScript.
macro_rules! client_request_definitions {
(
$(
$(#[$variant_meta:meta])*
$variant:ident {
params: $(#[$params_meta:meta])* $params:ty,
response: $response:ty,
}
),* $(,)?
) => {
/// Request from the client to the server.
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, TS)]
#[serde(tag = "method", rename_all = "camelCase")]
pub enum ClientRequest {
$(
$(#[$variant_meta])*
$variant {
#[serde(rename = "id")]
request_id: RequestId,
$(#[$params_meta])*
params: $params,
},
)*
}
pub fn export_client_responses(
out_dir: &::std::path::Path,
) -> ::std::result::Result<(), ::ts_rs::ExportError> {
$(
<$response as ::ts_rs::TS>::export_all_to(out_dir)?;
)*
Ok(())
}
};
}
client_request_definitions! {
Initialize {
params: InitializeParams,
response: InitializeResponse,
},
NewConversation {
#[serde(rename = "id")]
request_id: RequestId,
params: NewConversationParams,
response: NewConversationResponse,
},
/// List recorded Codex conversations (rollouts) with optional pagination and search.
ListConversations {
#[serde(rename = "id")]
request_id: RequestId,
params: ListConversationsParams,
response: ListConversationsResponse,
},
/// Resume a recorded Codex conversation from a rollout file.
ResumeConversation {
#[serde(rename = "id")]
request_id: RequestId,
params: ResumeConversationParams,
response: ResumeConversationResponse,
},
ArchiveConversation {
#[serde(rename = "id")]
request_id: RequestId,
params: ArchiveConversationParams,
response: ArchiveConversationResponse,
},
SendUserMessage {
#[serde(rename = "id")]
request_id: RequestId,
params: SendUserMessageParams,
response: SendUserMessageResponse,
},
SendUserTurn {
#[serde(rename = "id")]
request_id: RequestId,
params: SendUserTurnParams,
response: SendUserTurnResponse,
},
InterruptConversation {
#[serde(rename = "id")]
request_id: RequestId,
params: InterruptConversationParams,
response: InterruptConversationResponse,
},
AddConversationListener {
#[serde(rename = "id")]
request_id: RequestId,
params: AddConversationListenerParams,
response: AddConversationSubscriptionResponse,
},
RemoveConversationListener {
#[serde(rename = "id")]
request_id: RequestId,
params: RemoveConversationListenerParams,
response: RemoveConversationSubscriptionResponse,
},
GitDiffToRemote {
#[serde(rename = "id")]
request_id: RequestId,
params: GitDiffToRemoteParams,
response: GitDiffToRemoteResponse,
},
LoginApiKey {
#[serde(rename = "id")]
request_id: RequestId,
params: LoginApiKeyParams,
response: LoginApiKeyResponse,
},
LoginChatGpt {
#[serde(rename = "id")]
request_id: RequestId,
params: #[ts(type = "undefined")] #[serde(skip_serializing_if = "Option::is_none")] Option<()>,
response: LoginChatGptResponse,
},
CancelLoginChatGpt {
#[serde(rename = "id")]
request_id: RequestId,
params: CancelLoginChatGptParams,
response: CancelLoginChatGptResponse,
},
LogoutChatGpt {
#[serde(rename = "id")]
request_id: RequestId,
params: #[ts(type = "undefined")] #[serde(skip_serializing_if = "Option::is_none")] Option<()>,
response: LogoutChatGptResponse,
},
GetAuthStatus {
#[serde(rename = "id")]
request_id: RequestId,
params: GetAuthStatusParams,
response: GetAuthStatusResponse,
},
GetUserSavedConfig {
#[serde(rename = "id")]
request_id: RequestId,
params: #[ts(type = "undefined")] #[serde(skip_serializing_if = "Option::is_none")] Option<()>,
response: GetUserSavedConfigResponse,
},
SetDefaultModel {
#[serde(rename = "id")]
request_id: RequestId,
params: SetDefaultModelParams,
response: SetDefaultModelResponse,
},
GetUserAgent {
#[serde(rename = "id")]
request_id: RequestId,
params: #[ts(type = "undefined")] #[serde(skip_serializing_if = "Option::is_none")] Option<()>,
response: GetUserAgentResponse,
},
UserInfo {
#[serde(rename = "id")]
request_id: RequestId,
params: #[ts(type = "undefined")] #[serde(skip_serializing_if = "Option::is_none")] Option<()>,
response: UserInfoResponse,
},
FuzzyFileSearch {
params: FuzzyFileSearchParams,
response: FuzzyFileSearchResponse,
},
/// Execute a command (argv vector) under the server's sandbox.
ExecOneOffCommand {
#[serde(rename = "id")]
request_id: RequestId,
params: ExecOneOffCommandParams,
response: ExecOneOffCommandResponse,
},
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Default, TS)]
#[serde(rename_all = "camelCase")]
pub struct InitializeParams {
pub client_info: ClientInfo,
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Default, TS)]
#[serde(rename_all = "camelCase")]
pub struct ClientInfo {
pub name: String,
#[serde(skip_serializing_if = "Option::is_none")]
pub title: Option<String>,
pub version: String,
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, TS)]
#[serde(rename_all = "camelCase")]
pub struct InitializeResponse {
pub user_agent: String,
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, Default, TS)]
#[serde(rename_all = "camelCase")]
pub struct NewConversationParams {
@@ -397,7 +399,7 @@ pub struct ExecOneOffCommandParams {
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, TS)]
#[serde(rename_all = "camelCase")]
pub struct ExecArbitraryCommandResponse {
pub struct ExecOneOffCommandResponse {
pub exit_code: i32,
pub stdout: String,
pub stderr: String,
@@ -601,30 +603,74 @@ pub enum InputItem {
},
}
// TODO(mbolin): Need test to ensure these constants match the enum variants.
/// Generates an `enum ServerRequest` where each variant is a request that the
/// server can send to the client along with the corresponding params and
/// response types. It also generates helper types used by the app/server
/// infrastructure (payload enum, request constructor, and export helpers).
macro_rules! server_request_definitions {
(
$(
$(#[$variant_meta:meta])*
$variant:ident
),* $(,)?
) => {
paste! {
/// Request initiated from the server and sent to the client.
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, TS)]
#[serde(tag = "method", rename_all = "camelCase")]
pub enum ServerRequest {
$(
$(#[$variant_meta])*
$variant {
#[serde(rename = "id")]
request_id: RequestId,
params: [<$variant Params>],
},
)*
}
pub const APPLY_PATCH_APPROVAL_METHOD: &str = "applyPatchApproval";
pub const EXEC_COMMAND_APPROVAL_METHOD: &str = "execCommandApproval";
#[derive(Debug, Clone, PartialEq)]
pub enum ServerRequestPayload {
$( $variant([<$variant Params>]), )*
}
/// Request initiated from the server and sent to the client.
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, TS)]
#[serde(tag = "method", rename_all = "camelCase")]
pub enum ServerRequest {
impl ServerRequestPayload {
pub fn request_with_id(self, request_id: RequestId) -> ServerRequest {
match self {
$(Self::$variant(params) => ServerRequest::$variant { request_id, params },)*
}
}
}
}
pub fn export_server_responses(
out_dir: &::std::path::Path,
) -> ::std::result::Result<(), ::ts_rs::ExportError> {
paste! {
$(<[<$variant Response>] as ::ts_rs::TS>::export_all_to(out_dir)?;)*
}
Ok(())
}
};
}
impl TryFrom<JSONRPCRequest> for ServerRequest {
type Error = serde_json::Error;
fn try_from(value: JSONRPCRequest) -> Result<Self, Self::Error> {
serde_json::from_value(serde_json::to_value(value)?)
}
}
server_request_definitions! {
/// Request to approve a patch.
ApplyPatchApproval {
#[serde(rename = "id")]
request_id: RequestId,
params: ApplyPatchApprovalParams,
},
ApplyPatchApproval,
/// Request to exec a command.
ExecCommandApproval {
#[serde(rename = "id")]
request_id: RequestId,
params: ExecCommandApprovalParams,
},
ExecCommandApproval,
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, TS)]
#[serde(rename_all = "camelCase")]
pub struct ApplyPatchApprovalParams {
pub conversation_id: ConversationId,
/// Use to correlate this with [codex_core::protocol::PatchApplyBeginEvent]
@@ -641,6 +687,7 @@ pub struct ApplyPatchApprovalParams {
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, TS)]
#[serde(rename_all = "camelCase")]
pub struct ExecCommandApprovalParams {
pub conversation_id: ConversationId,
/// Use to correlate this with [codex_core::protocol::ExecCommandBeginEvent]
@@ -662,6 +709,32 @@ pub struct ApplyPatchApprovalResponse {
pub decision: ReviewDecision,
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, TS)]
#[serde(rename_all = "camelCase")]
#[ts(rename_all = "camelCase")]
pub struct FuzzyFileSearchParams {
pub query: String,
pub roots: Vec<String>,
// if provided, will cancel any previous request that used the same value
#[serde(skip_serializing_if = "Option::is_none")]
pub cancellation_token: Option<String>,
}
/// Superset of [`codex_file_search::FileMatch`]
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, TS)]
pub struct FuzzyFileSearchResult {
pub root: String,
pub path: String,
pub score: u32,
#[serde(skip_serializing_if = "Option::is_none")]
pub indices: Option<Vec<u32>>,
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, TS)]
pub struct FuzzyFileSearchResponse {
pub files: Vec<FuzzyFileSearchResult>,
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, TS)]
#[serde(rename_all = "camelCase")]
pub struct LoginChatGptCompleteNotification {
@@ -671,6 +744,34 @@ pub struct LoginChatGptCompleteNotification {
pub error: Option<String>,
}
#[derive(Serialize, Deserialize, Debug, Clone, TS)]
#[serde(rename_all = "camelCase")]
pub struct SessionConfiguredNotification {
/// Name left as session_id instead of conversation_id for backwards compatibility.
pub session_id: ConversationId,
/// Tell the client what model is being queried.
pub model: String,
/// The effort the model is putting into reasoning about the user's request.
#[serde(skip_serializing_if = "Option::is_none")]
pub reasoning_effort: Option<ReasoningEffort>,
/// Identifier of the history log file (inode on Unix, 0 otherwise).
pub history_log_id: u64,
/// Current number of entries in the history log.
#[ts(type = "number")]
pub history_entry_count: usize,
/// Optional initial messages (as events) for resumed sessions.
/// When present, UIs can use these to seed the history.
#[serde(skip_serializing_if = "Option::is_none")]
pub initial_messages: Option<Vec<EventMsg>>,
pub rollout_path: PathBuf,
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, TS)]
#[serde(rename_all = "camelCase")]
pub struct AuthStatusChangeNotification {
@@ -679,7 +780,8 @@ pub struct AuthStatusChangeNotification {
pub auth_method: Option<AuthMode>,
}
#[derive(Serialize, Deserialize, Debug, Clone, PartialEq, TS, Display)]
/// Notification sent from the server to the client.
#[derive(Serialize, Deserialize, Debug, Clone, TS, Display)]
#[serde(tag = "method", content = "params", rename_all = "camelCase")]
#[strum(serialize_all = "camelCase")]
pub enum ServerNotification {
@@ -688,6 +790,9 @@ pub enum ServerNotification {
/// ChatGPT login flow completed
LoginChatGptComplete(LoginChatGptCompleteNotification),
/// The special session configured event for a new or resumed conversation.
SessionConfigured(SessionConfiguredNotification),
}
impl ServerNotification {
@@ -695,10 +800,27 @@ impl ServerNotification {
match self {
ServerNotification::AuthStatusChange(params) => serde_json::to_value(params),
ServerNotification::LoginChatGptComplete(params) => serde_json::to_value(params),
ServerNotification::SessionConfigured(params) => serde_json::to_value(params),
}
}
}
impl TryFrom<JSONRPCNotification> for ServerNotification {
type Error = serde_json::Error;
fn try_from(value: JSONRPCNotification) -> Result<Self, Self::Error> {
serde_json::from_value(serde_json::to_value(value)?)
}
}
/// Notification sent from the client to the server.
#[derive(Serialize, Deserialize, Debug, Clone, TS, Display)]
#[serde(tag = "method", content = "params", rename_all = "camelCase")]
#[strum(serialize_all = "camelCase")]
pub enum ClientNotification {
Initialized,
}
#[cfg(test)]
mod tests {
use super::*;
@@ -736,12 +858,6 @@ mod tests {
Ok(())
}
#[test]
fn test_conversation_id_default_is_not_zeroes() {
let id = ConversationId::default();
assert_ne!(id.uuid, Uuid::nil());
}
#[test]
fn conversation_id_serializes_as_plain_string() -> Result<()> {
let id = ConversationId::from_string("67e55044-10b1-426f-9247-bb680e5fe0c8")?;
@@ -764,4 +880,52 @@ mod tests {
);
Ok(())
}
#[test]
fn serialize_client_notification() -> Result<()> {
let notification = ClientNotification::Initialized;
// Note there is no "params" field for this notification.
assert_eq!(
json!({
"method": "initialized",
}),
serde_json::to_value(&notification)?,
);
Ok(())
}
#[test]
fn serialize_server_request() -> Result<()> {
let conversation_id = ConversationId::from_string("67e55044-10b1-426f-9247-bb680e5fe0c8")?;
let params = ExecCommandApprovalParams {
conversation_id,
call_id: "call-42".to_string(),
command: vec!["echo".to_string(), "hello".to_string()],
cwd: PathBuf::from("/tmp"),
reason: Some("because tests".to_string()),
};
let request = ServerRequest::ExecCommandApproval {
request_id: RequestId::Integer(7),
params: params.clone(),
};
assert_eq!(
json!({
"method": "execCommandApproval",
"id": 7,
"params": {
"conversationId": "67e55044-10b1-426f-9247-bb680e5fe0c8",
"callId": "call-42",
"command": ["echo", "hello"],
"cwd": "/tmp",
"reason": "because tests",
}
}),
serde_json::to_value(&request)?,
);
let payload = ServerRequestPayload::ExecCommandApproval(params);
assert_eq!(payload.request_with_id(RequestId::Integer(7)), request);
Ok(())
}
}

View File

@@ -0,0 +1,49 @@
[package]
edition = "2024"
name = "codex-app-server"
version = { workspace = true }
[[bin]]
name = "codex-app-server"
path = "src/main.rs"
[lib]
name = "codex_app_server"
path = "src/lib.rs"
[lints]
workspace = true
[dependencies]
anyhow = { workspace = true }
codex-arg0 = { workspace = true }
codex-common = { workspace = true, features = ["cli"] }
codex-core = { workspace = true }
codex-file-search = { workspace = true }
codex-login = { workspace = true }
codex-protocol = { workspace = true }
codex-app-server-protocol = { workspace = true }
codex-utils-json-to-toml = { workspace = true }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
tokio = { workspace = true, features = [
"io-std",
"macros",
"process",
"rt-multi-thread",
"signal",
] }
tracing = { workspace = true, features = ["log"] }
tracing-subscriber = { workspace = true, features = ["env-filter", "fmt"] }
uuid = { workspace = true, features = ["serde", "v7"] }
[dev-dependencies]
app_test_support = { workspace = true }
assert_cmd = { workspace = true }
base64 = { workspace = true }
core_test_support = { workspace = true }
os_info = { workspace = true }
pretty_assertions = { workspace = true }
tempfile = { workspace = true }
toml = { workspace = true }
wiremock = { workspace = true }

View File

@@ -1,8 +1,54 @@
use crate::error_code::INTERNAL_ERROR_CODE;
use crate::error_code::INVALID_REQUEST_ERROR_CODE;
use crate::json_to_toml::json_to_toml;
use crate::fuzzy_file_search::run_fuzzy_file_search;
use crate::outgoing_message::OutgoingMessageSender;
use crate::outgoing_message::OutgoingNotification;
use codex_app_server_protocol::AddConversationListenerParams;
use codex_app_server_protocol::AddConversationSubscriptionResponse;
use codex_app_server_protocol::ApplyPatchApprovalParams;
use codex_app_server_protocol::ApplyPatchApprovalResponse;
use codex_app_server_protocol::ArchiveConversationParams;
use codex_app_server_protocol::ArchiveConversationResponse;
use codex_app_server_protocol::AuthStatusChangeNotification;
use codex_app_server_protocol::ClientRequest;
use codex_app_server_protocol::ConversationSummary;
use codex_app_server_protocol::ExecCommandApprovalParams;
use codex_app_server_protocol::ExecCommandApprovalResponse;
use codex_app_server_protocol::ExecOneOffCommandParams;
use codex_app_server_protocol::ExecOneOffCommandResponse;
use codex_app_server_protocol::FuzzyFileSearchParams;
use codex_app_server_protocol::FuzzyFileSearchResponse;
use codex_app_server_protocol::GetUserAgentResponse;
use codex_app_server_protocol::GetUserSavedConfigResponse;
use codex_app_server_protocol::GitDiffToRemoteResponse;
use codex_app_server_protocol::InputItem as WireInputItem;
use codex_app_server_protocol::InterruptConversationParams;
use codex_app_server_protocol::InterruptConversationResponse;
use codex_app_server_protocol::JSONRPCErrorError;
use codex_app_server_protocol::ListConversationsParams;
use codex_app_server_protocol::ListConversationsResponse;
use codex_app_server_protocol::LoginApiKeyParams;
use codex_app_server_protocol::LoginApiKeyResponse;
use codex_app_server_protocol::LoginChatGptCompleteNotification;
use codex_app_server_protocol::LoginChatGptResponse;
use codex_app_server_protocol::NewConversationParams;
use codex_app_server_protocol::NewConversationResponse;
use codex_app_server_protocol::RemoveConversationListenerParams;
use codex_app_server_protocol::RemoveConversationSubscriptionResponse;
use codex_app_server_protocol::RequestId;
use codex_app_server_protocol::Result as JsonRpcResult;
use codex_app_server_protocol::ResumeConversationParams;
use codex_app_server_protocol::SendUserMessageParams;
use codex_app_server_protocol::SendUserMessageResponse;
use codex_app_server_protocol::SendUserTurnParams;
use codex_app_server_protocol::SendUserTurnResponse;
use codex_app_server_protocol::ServerNotification;
use codex_app_server_protocol::ServerRequestPayload;
use codex_app_server_protocol::SessionConfiguredNotification;
use codex_app_server_protocol::SetDefaultModelParams;
use codex_app_server_protocol::SetDefaultModelResponse;
use codex_app_server_protocol::UserInfoResponse;
use codex_app_server_protocol::UserSavedConfig;
use codex_core::AuthManager;
use codex_core::CodexConversation;
use codex_core::ConversationManager;
@@ -36,58 +82,18 @@ use codex_core::protocol::ReviewDecision;
use codex_login::ServerOptions as LoginServerOptions;
use codex_login::ShutdownHandle;
use codex_login::run_login_server;
use codex_protocol::mcp_protocol::APPLY_PATCH_APPROVAL_METHOD;
use codex_protocol::mcp_protocol::AddConversationListenerParams;
use codex_protocol::mcp_protocol::AddConversationSubscriptionResponse;
use codex_protocol::mcp_protocol::ApplyPatchApprovalParams;
use codex_protocol::mcp_protocol::ApplyPatchApprovalResponse;
use codex_protocol::mcp_protocol::ArchiveConversationParams;
use codex_protocol::mcp_protocol::ArchiveConversationResponse;
use codex_protocol::mcp_protocol::AuthStatusChangeNotification;
use codex_protocol::mcp_protocol::ClientRequest;
use codex_protocol::mcp_protocol::ConversationId;
use codex_protocol::mcp_protocol::ConversationSummary;
use codex_protocol::mcp_protocol::EXEC_COMMAND_APPROVAL_METHOD;
use codex_protocol::mcp_protocol::ExecArbitraryCommandResponse;
use codex_protocol::mcp_protocol::ExecCommandApprovalParams;
use codex_protocol::mcp_protocol::ExecCommandApprovalResponse;
use codex_protocol::mcp_protocol::ExecOneOffCommandParams;
use codex_protocol::mcp_protocol::GetUserAgentResponse;
use codex_protocol::mcp_protocol::GetUserSavedConfigResponse;
use codex_protocol::mcp_protocol::GitDiffToRemoteResponse;
use codex_protocol::mcp_protocol::InputItem as WireInputItem;
use codex_protocol::mcp_protocol::InterruptConversationParams;
use codex_protocol::mcp_protocol::InterruptConversationResponse;
use codex_protocol::mcp_protocol::ListConversationsParams;
use codex_protocol::mcp_protocol::ListConversationsResponse;
use codex_protocol::mcp_protocol::LoginApiKeyParams;
use codex_protocol::mcp_protocol::LoginApiKeyResponse;
use codex_protocol::mcp_protocol::LoginChatGptCompleteNotification;
use codex_protocol::mcp_protocol::LoginChatGptResponse;
use codex_protocol::mcp_protocol::NewConversationParams;
use codex_protocol::mcp_protocol::NewConversationResponse;
use codex_protocol::mcp_protocol::RemoveConversationListenerParams;
use codex_protocol::mcp_protocol::RemoveConversationSubscriptionResponse;
use codex_protocol::mcp_protocol::ResumeConversationParams;
use codex_protocol::mcp_protocol::SendUserMessageParams;
use codex_protocol::mcp_protocol::SendUserMessageResponse;
use codex_protocol::mcp_protocol::SendUserTurnParams;
use codex_protocol::mcp_protocol::SendUserTurnResponse;
use codex_protocol::mcp_protocol::ServerNotification;
use codex_protocol::mcp_protocol::SetDefaultModelParams;
use codex_protocol::mcp_protocol::SetDefaultModelResponse;
use codex_protocol::mcp_protocol::UserInfoResponse;
use codex_protocol::mcp_protocol::UserSavedConfig;
use codex_protocol::ConversationId;
use codex_protocol::models::ContentItem;
use codex_protocol::models::ResponseItem;
use codex_protocol::protocol::InputMessageKind;
use codex_protocol::protocol::USER_MESSAGE_BEGIN;
use mcp_types::JSONRPCErrorError;
use mcp_types::RequestId;
use codex_utils_json_to_toml::json_to_toml;
use std::collections::HashMap;
use std::ffi::OsStr;
use std::path::PathBuf;
use std::sync::Arc;
use std::sync::atomic::AtomicBool;
use std::sync::atomic::Ordering;
use std::time::Duration;
use tokio::select;
use tokio::sync::Mutex;
@@ -122,6 +128,7 @@ pub(crate) struct CodexMessageProcessor {
active_login: Arc<Mutex<Option<ActiveLogin>>>,
// Queue of pending interrupt requests per conversation. We reply when TurnAborted arrives.
pending_interrupts: Arc<Mutex<HashMap<ConversationId, Vec<RequestId>>>>,
pending_fuzzy_searches: Arc<Mutex<HashMap<String, Arc<AtomicBool>>>>,
}
impl CodexMessageProcessor {
@@ -141,11 +148,15 @@ impl CodexMessageProcessor {
conversation_listeners: HashMap::new(),
active_login: Arc::new(Mutex::new(None)),
pending_interrupts: Arc::new(Mutex::new(HashMap::new())),
pending_fuzzy_searches: Arc::new(Mutex::new(HashMap::new())),
}
}
pub async fn process_request(&mut self, request: ClientRequest) {
match request {
ClientRequest::Initialize { .. } => {
panic!("Initialize should be handled in MessageProcessor");
}
ClientRequest::NewConversation { request_id, params } => {
// Do not tokio::spawn() to process new_conversation()
// asynchronously because we need to ensure the conversation is
@@ -182,30 +193,48 @@ impl CodexMessageProcessor {
ClientRequest::LoginApiKey { request_id, params } => {
self.login_api_key(request_id, params).await;
}
ClientRequest::LoginChatGpt { request_id } => {
ClientRequest::LoginChatGpt {
request_id,
params: _,
} => {
self.login_chatgpt(request_id).await;
}
ClientRequest::CancelLoginChatGpt { request_id, params } => {
self.cancel_login_chatgpt(request_id, params.login_id).await;
}
ClientRequest::LogoutChatGpt { request_id } => {
ClientRequest::LogoutChatGpt {
request_id,
params: _,
} => {
self.logout_chatgpt(request_id).await;
}
ClientRequest::GetAuthStatus { request_id, params } => {
self.get_auth_status(request_id, params).await;
}
ClientRequest::GetUserSavedConfig { request_id } => {
ClientRequest::GetUserSavedConfig {
request_id,
params: _,
} => {
self.get_user_saved_config(request_id).await;
}
ClientRequest::SetDefaultModel { request_id, params } => {
self.set_default_model(request_id, params).await;
}
ClientRequest::GetUserAgent { request_id } => {
ClientRequest::GetUserAgent {
request_id,
params: _,
} => {
self.get_user_agent(request_id).await;
}
ClientRequest::UserInfo { request_id } => {
ClientRequest::UserInfo {
request_id,
params: _,
} => {
self.get_user_info(request_id).await;
}
ClientRequest::FuzzyFileSearch { request_id, params } => {
self.fuzzy_file_search(request_id, params).await;
}
ClientRequest::ExecOneOffCommand { request_id, params } => {
self.exec_one_off_command(request_id, params).await;
}
@@ -357,7 +386,7 @@ impl CodexMessageProcessor {
self.outgoing
.send_response(
request_id,
codex_protocol::mcp_protocol::CancelLoginChatGptResponse {},
codex_app_server_protocol::CancelLoginChatGptResponse {},
)
.await;
} else {
@@ -393,7 +422,7 @@ impl CodexMessageProcessor {
self.outgoing
.send_response(
request_id,
codex_protocol::mcp_protocol::LogoutChatGptResponse {},
codex_app_server_protocol::LogoutChatGptResponse {},
)
.await;
@@ -411,7 +440,7 @@ impl CodexMessageProcessor {
async fn get_auth_status(
&self,
request_id: RequestId,
params: codex_protocol::mcp_protocol::GetAuthStatusParams,
params: codex_app_server_protocol::GetAuthStatusParams,
) {
let include_token = params.include_token.unwrap_or(false);
let do_refresh = params.refresh_token.unwrap_or(false);
@@ -426,7 +455,7 @@ impl CodexMessageProcessor {
let requires_openai_auth = self.config.model_provider.requires_openai_auth;
let response = if !requires_openai_auth {
codex_protocol::mcp_protocol::GetAuthStatusResponse {
codex_app_server_protocol::GetAuthStatusResponse {
auth_method: None,
auth_token: None,
requires_openai_auth: Some(false),
@@ -446,13 +475,13 @@ impl CodexMessageProcessor {
(None, None)
}
};
codex_protocol::mcp_protocol::GetAuthStatusResponse {
codex_app_server_protocol::GetAuthStatusResponse {
auth_method: reported_auth_method,
auth_token: token_opt,
requires_openai_auth: Some(true),
}
}
None => codex_protocol::mcp_protocol::GetAuthStatusResponse {
None => codex_app_server_protocol::GetAuthStatusResponse {
auth_method: None,
auth_token: None,
requires_openai_auth: Some(true),
@@ -603,7 +632,7 @@ impl CodexMessageProcessor {
.await
{
Ok(output) => {
let response = ExecArbitraryCommandResponse {
let response = ExecOneOffCommandResponse {
exit_code: output.exit_code,
stdout: output.stdout.text,
stderr: output.stderr.text,
@@ -752,11 +781,19 @@ impl CodexMessageProcessor {
session_configured,
..
}) => {
let event = Event {
id: "".to_string(),
msg: EventMsg::SessionConfigured(session_configured.clone()),
};
self.outgoing.send_event_as_notification(&event, None).await;
self.outgoing
.send_server_notification(ServerNotification::SessionConfigured(
SessionConfiguredNotification {
session_id: session_configured.session_id,
model: session_configured.model.clone(),
reasoning_effort: session_configured.reasoning_effort,
history_log_id: session_configured.history_log_id,
history_entry_count: session_configured.history_entry_count,
initial_messages: session_configured.initial_messages.clone(),
rollout_path: session_configured.rollout_path.clone(),
},
))
.await;
let initial_messages = session_configured.initial_messages.map(|msgs| {
msgs.into_iter()
.filter(|event| {
@@ -771,7 +808,7 @@ impl CodexMessageProcessor {
});
// Reply with conversation id + model and initial messages (when present)
let response = codex_protocol::mcp_protocol::ResumeConversationResponse {
let response = codex_app_server_protocol::ResumeConversationResponse {
conversation_id,
model: session_configured.model.clone(),
initial_messages,
@@ -1167,6 +1204,46 @@ impl CodexMessageProcessor {
}
}
}
async fn fuzzy_file_search(&mut self, request_id: RequestId, params: FuzzyFileSearchParams) {
let FuzzyFileSearchParams {
query,
roots,
cancellation_token,
} = params;
let cancel_flag = match cancellation_token.clone() {
Some(token) => {
let mut pending_fuzzy_searches = self.pending_fuzzy_searches.lock().await;
// if a cancellation_token is provided and a pending_request exists for
// that token, cancel it
if let Some(existing) = pending_fuzzy_searches.get(&token) {
existing.store(true, Ordering::Relaxed);
}
let flag = Arc::new(AtomicBool::new(false));
pending_fuzzy_searches.insert(token.clone(), flag.clone());
flag
}
None => Arc::new(AtomicBool::new(false)),
};
let results = match query.as_str() {
"" => vec![],
_ => run_fuzzy_file_search(query, roots, cancel_flag.clone()).await,
};
if let Some(token) = cancellation_token {
let mut pending_fuzzy_searches = self.pending_fuzzy_searches.lock().await;
if let Some(current_flag) = pending_fuzzy_searches.get(&token)
&& Arc::ptr_eq(current_flag, &cancel_flag)
{
pending_fuzzy_searches.remove(&token);
}
}
let response = FuzzyFileSearchResponse { files: results };
self.outgoing.send_response(request_id, response).await;
}
}
async fn apply_bespoke_event_handling(
@@ -1191,9 +1268,8 @@ async fn apply_bespoke_event_handling(
reason,
grant_root,
};
let value = serde_json::to_value(&params).unwrap_or_default();
let rx = outgoing
.send_request(APPLY_PATCH_APPROVAL_METHOD, Some(value))
.send_request(ServerRequestPayload::ApplyPatchApproval(params))
.await;
// TODO(mbolin): Enforce a timeout so this task does not live indefinitely?
tokio::spawn(async move {
@@ -1213,9 +1289,8 @@ async fn apply_bespoke_event_handling(
cwd,
reason,
};
let value = serde_json::to_value(&params).unwrap_or_default();
let rx = outgoing
.send_request(EXEC_COMMAND_APPROVAL_METHOD, Some(value))
.send_request(ServerRequestPayload::ExecCommandApproval(params))
.await;
// TODO(mbolin): Enforce a timeout so this task does not live indefinitely?
@@ -1273,7 +1348,6 @@ fn derive_config_from_params(
include_view_image_tool: None,
show_raw_agent_reasoning: None,
tools_web_search_request: None,
enable_git_worktree: None,
};
let cli_overrides = cli_overrides
@@ -1287,7 +1361,7 @@ fn derive_config_from_params(
async fn on_patch_approval_response(
event_id: String,
receiver: oneshot::Receiver<mcp_types::Result>,
receiver: oneshot::Receiver<JsonRpcResult>,
codex: Arc<CodexConversation>,
) {
let response = receiver.await;
@@ -1329,7 +1403,7 @@ async fn on_patch_approval_response(
async fn on_exec_approval_response(
event_id: String,
receiver: oneshot::Receiver<mcp_types::Result>,
receiver: oneshot::Receiver<JsonRpcResult>,
conversation: Arc<CodexConversation>,
) {
let response = receiver.await;

View File

@@ -0,0 +1,2 @@
pub(crate) const INVALID_REQUEST_ERROR_CODE: i64 = -32600;
pub(crate) const INTERNAL_ERROR_CODE: i64 = -32603;

View File

@@ -0,0 +1,84 @@
use std::num::NonZero;
use std::num::NonZeroUsize;
use std::path::PathBuf;
use std::sync::Arc;
use std::sync::atomic::AtomicBool;
use codex_app_server_protocol::FuzzyFileSearchResult;
use codex_file_search as file_search;
use tokio::task::JoinSet;
use tracing::warn;
const LIMIT_PER_ROOT: usize = 50;
const MAX_THREADS: usize = 12;
const COMPUTE_INDICES: bool = true;
pub(crate) async fn run_fuzzy_file_search(
query: String,
roots: Vec<String>,
cancellation_flag: Arc<AtomicBool>,
) -> Vec<FuzzyFileSearchResult> {
#[expect(clippy::expect_used)]
let limit_per_root =
NonZero::new(LIMIT_PER_ROOT).expect("LIMIT_PER_ROOT should be a valid non-zero usize");
let cores = std::thread::available_parallelism()
.map(std::num::NonZero::get)
.unwrap_or(1);
let threads = cores.min(MAX_THREADS);
let threads_per_root = (threads / roots.len()).max(1);
let threads = NonZero::new(threads_per_root).unwrap_or(NonZeroUsize::MIN);
let mut files: Vec<FuzzyFileSearchResult> = Vec::new();
let mut join_set = JoinSet::new();
for root in roots {
let search_dir = PathBuf::from(&root);
let query = query.clone();
let cancel_flag = cancellation_flag.clone();
join_set.spawn_blocking(move || {
match file_search::run(
query.as_str(),
limit_per_root,
&search_dir,
Vec::new(),
threads,
cancel_flag,
COMPUTE_INDICES,
) {
Ok(res) => Ok((root, res)),
Err(err) => Err((root, err)),
}
});
}
while let Some(res) = join_set.join_next().await {
match res {
Ok(Ok((root, res))) => {
for m in res.matches {
let result = FuzzyFileSearchResult {
root: root.clone(),
path: m.path,
score: m.score,
indices: m.indices,
};
files.push(result);
}
}
Ok(Err((root, err))) => {
warn!("fuzzy-file-search in dir '{root}' failed: {err}");
}
Err(err) => {
warn!("fuzzy-file-search join_next failed: {err}");
}
}
}
files.sort_by(file_search::cmp_by_score_desc_then_path_asc::<
FuzzyFileSearchResult,
_,
_,
>(|f| f.score, |f| f.path.as_str()));
files
}

View File

@@ -0,0 +1,139 @@
#![deny(clippy::print_stdout, clippy::print_stderr)]
use std::io::ErrorKind;
use std::io::Result as IoResult;
use std::path::PathBuf;
use codex_common::CliConfigOverrides;
use codex_core::config::Config;
use codex_core::config::ConfigOverrides;
use codex_app_server_protocol::JSONRPCMessage;
use tokio::io::AsyncBufReadExt;
use tokio::io::AsyncWriteExt;
use tokio::io::BufReader;
use tokio::io::{self};
use tokio::sync::mpsc;
use tracing::debug;
use tracing::error;
use tracing::info;
use tracing_subscriber::EnvFilter;
use crate::message_processor::MessageProcessor;
use crate::outgoing_message::OutgoingMessage;
use crate::outgoing_message::OutgoingMessageSender;
mod codex_message_processor;
mod error_code;
mod fuzzy_file_search;
mod message_processor;
mod outgoing_message;
/// Size of the bounded channels used to communicate between tasks. The value
/// is a balance between throughput and memory usage 128 messages should be
/// plenty for an interactive CLI.
const CHANNEL_CAPACITY: usize = 128;
pub async fn run_main(
codex_linux_sandbox_exe: Option<PathBuf>,
cli_config_overrides: CliConfigOverrides,
) -> IoResult<()> {
// Install a simple subscriber so `tracing` output is visible. Users can
// control the log level with `RUST_LOG`.
tracing_subscriber::fmt()
.with_writer(std::io::stderr)
.with_env_filter(EnvFilter::from_default_env())
.init();
// Set up channels.
let (incoming_tx, mut incoming_rx) = mpsc::channel::<JSONRPCMessage>(CHANNEL_CAPACITY);
let (outgoing_tx, mut outgoing_rx) = mpsc::unbounded_channel::<OutgoingMessage>();
// Task: read from stdin, push to `incoming_tx`.
let stdin_reader_handle = tokio::spawn({
async move {
let stdin = io::stdin();
let reader = BufReader::new(stdin);
let mut lines = reader.lines();
while let Some(line) = lines.next_line().await.unwrap_or_default() {
match serde_json::from_str::<JSONRPCMessage>(&line) {
Ok(msg) => {
if incoming_tx.send(msg).await.is_err() {
// Receiver gone nothing left to do.
break;
}
}
Err(e) => error!("Failed to deserialize JSONRPCMessage: {e}"),
}
}
debug!("stdin reader finished (EOF)");
}
});
// Parse CLI overrides once and derive the base Config eagerly so later
// components do not need to work with raw TOML values.
let cli_kv_overrides = cli_config_overrides.parse_overrides().map_err(|e| {
std::io::Error::new(
ErrorKind::InvalidInput,
format!("error parsing -c overrides: {e}"),
)
})?;
let config = Config::load_with_cli_overrides(cli_kv_overrides, ConfigOverrides::default())
.map_err(|e| {
std::io::Error::new(ErrorKind::InvalidData, format!("error loading config: {e}"))
})?;
// Task: process incoming messages.
let processor_handle = tokio::spawn({
let outgoing_message_sender = OutgoingMessageSender::new(outgoing_tx);
let mut processor = MessageProcessor::new(
outgoing_message_sender,
codex_linux_sandbox_exe,
std::sync::Arc::new(config),
);
async move {
while let Some(msg) = incoming_rx.recv().await {
match msg {
JSONRPCMessage::Request(r) => processor.process_request(r).await,
JSONRPCMessage::Response(r) => processor.process_response(r).await,
JSONRPCMessage::Notification(n) => processor.process_notification(n).await,
JSONRPCMessage::Error(e) => processor.process_error(e),
}
}
info!("processor task exited (channel closed)");
}
});
// Task: write outgoing messages to stdout.
let stdout_writer_handle = tokio::spawn(async move {
let mut stdout = io::stdout();
while let Some(outgoing_message) = outgoing_rx.recv().await {
let Ok(value) = serde_json::to_value(outgoing_message) else {
error!("Failed to convert OutgoingMessage to JSON value");
continue;
};
match serde_json::to_string(&value) {
Ok(mut json) => {
json.push('\n');
if let Err(e) = stdout.write_all(json.as_bytes()).await {
error!("Failed to write to stdout: {e}");
break;
}
}
Err(e) => error!("Failed to serialize JSONRPCMessage: {e}"),
}
}
info!("stdout writer exited (channel closed)");
});
// Wait for all tasks to finish. The typical exit path is the stdin reader
// hitting EOF which, once it drops `incoming_tx`, propagates shutdown to
// the processor and then to the stdout task.
let _ = tokio::join!(stdin_reader_handle, processor_handle, stdout_writer_handle);
Ok(())
}

View File

@@ -0,0 +1,10 @@
use codex_app_server::run_main;
use codex_arg0::arg0_dispatch_or_else;
use codex_common::CliConfigOverrides;
fn main() -> anyhow::Result<()> {
arg0_dispatch_or_else(|codex_linux_sandbox_exe| async move {
run_main(codex_linux_sandbox_exe, CliConfigOverrides::default()).await?;
Ok(())
})
}

View File

@@ -0,0 +1,133 @@
use std::path::PathBuf;
use crate::codex_message_processor::CodexMessageProcessor;
use crate::error_code::INVALID_REQUEST_ERROR_CODE;
use crate::outgoing_message::OutgoingMessageSender;
use codex_app_server_protocol::ClientInfo;
use codex_app_server_protocol::ClientRequest;
use codex_app_server_protocol::InitializeResponse;
use codex_app_server_protocol::JSONRPCError;
use codex_app_server_protocol::JSONRPCErrorError;
use codex_app_server_protocol::JSONRPCNotification;
use codex_app_server_protocol::JSONRPCRequest;
use codex_app_server_protocol::JSONRPCResponse;
use codex_core::AuthManager;
use codex_core::ConversationManager;
use codex_core::config::Config;
use codex_core::default_client::USER_AGENT_SUFFIX;
use codex_core::default_client::get_codex_user_agent;
use std::sync::Arc;
pub(crate) struct MessageProcessor {
outgoing: Arc<OutgoingMessageSender>,
codex_message_processor: CodexMessageProcessor,
initialized: bool,
}
impl MessageProcessor {
/// Create a new `MessageProcessor`, retaining a handle to the outgoing
/// `Sender` so handlers can enqueue messages to be written to stdout.
pub(crate) fn new(
outgoing: OutgoingMessageSender,
codex_linux_sandbox_exe: Option<PathBuf>,
config: Arc<Config>,
) -> Self {
let outgoing = Arc::new(outgoing);
let auth_manager = AuthManager::shared(config.codex_home.clone());
let conversation_manager = Arc::new(ConversationManager::new(auth_manager.clone()));
let codex_message_processor = CodexMessageProcessor::new(
auth_manager,
conversation_manager,
outgoing.clone(),
codex_linux_sandbox_exe,
config,
);
Self {
outgoing,
codex_message_processor,
initialized: false,
}
}
pub(crate) async fn process_request(&mut self, request: JSONRPCRequest) {
let request_id = request.id.clone();
if let Ok(request_json) = serde_json::to_value(request)
&& let Ok(codex_request) = serde_json::from_value::<ClientRequest>(request_json)
{
match codex_request {
// Handle Initialize internally so CodexMessageProcessor does not have to concern
// itself with the `initialized` bool.
ClientRequest::Initialize { request_id, params } => {
if self.initialized {
let error = JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message: "Already initialized".to_string(),
data: None,
};
self.outgoing.send_error(request_id, error).await;
return;
} else {
let ClientInfo {
name,
title: _title,
version,
} = params.client_info;
let user_agent_suffix = format!("{name}; {version}");
if let Ok(mut suffix) = USER_AGENT_SUFFIX.lock() {
*suffix = Some(user_agent_suffix);
}
let user_agent = get_codex_user_agent();
let response = InitializeResponse { user_agent };
self.outgoing.send_response(request_id, response).await;
self.initialized = true;
return;
}
}
_ => {
if !self.initialized {
let error = JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message: "Not initialized".to_string(),
data: None,
};
self.outgoing.send_error(request_id, error).await;
return;
}
}
}
self.codex_message_processor
.process_request(codex_request)
.await;
} else {
let error = JSONRPCErrorError {
code: INVALID_REQUEST_ERROR_CODE,
message: "Invalid request".to_string(),
data: None,
};
self.outgoing.send_error(request_id, error).await;
}
}
pub(crate) async fn process_notification(&self, notification: JSONRPCNotification) {
// Currently, we do not expect to receive any notifications from the
// client, so we just log them.
tracing::info!("<- notification: {:?}", notification);
}
/// Handle a standalone JSON-RPC response originating from the peer.
pub(crate) async fn process_response(&mut self, response: JSONRPCResponse) {
tracing::info!("<- response: {:?}", response);
let JSONRPCResponse { id, result, .. } = response;
self.outgoing.notify_client_response(id, result).await
}
/// Handle an error object received from the peer.
pub(crate) fn process_error(&mut self, err: JSONRPCError) {
tracing::error!("<- error: {:?}", err);
}
}

View File

@@ -0,0 +1,174 @@
use std::collections::HashMap;
use std::sync::atomic::AtomicI64;
use std::sync::atomic::Ordering;
use codex_app_server_protocol::JSONRPCErrorError;
use codex_app_server_protocol::RequestId;
use codex_app_server_protocol::Result;
use codex_app_server_protocol::ServerNotification;
use codex_app_server_protocol::ServerRequest;
use codex_app_server_protocol::ServerRequestPayload;
use serde::Serialize;
use tokio::sync::Mutex;
use tokio::sync::mpsc;
use tokio::sync::oneshot;
use tracing::warn;
use crate::error_code::INTERNAL_ERROR_CODE;
/// Sends messages to the client and manages request callbacks.
pub(crate) struct OutgoingMessageSender {
next_request_id: AtomicI64,
sender: mpsc::UnboundedSender<OutgoingMessage>,
request_id_to_callback: Mutex<HashMap<RequestId, oneshot::Sender<Result>>>,
}
impl OutgoingMessageSender {
pub(crate) fn new(sender: mpsc::UnboundedSender<OutgoingMessage>) -> Self {
Self {
next_request_id: AtomicI64::new(0),
sender,
request_id_to_callback: Mutex::new(HashMap::new()),
}
}
pub(crate) async fn send_request(
&self,
request: ServerRequestPayload,
) -> oneshot::Receiver<Result> {
let id = RequestId::Integer(self.next_request_id.fetch_add(1, Ordering::Relaxed));
let outgoing_message_id = id.clone();
let (tx_approve, rx_approve) = oneshot::channel();
{
let mut request_id_to_callback = self.request_id_to_callback.lock().await;
request_id_to_callback.insert(id, tx_approve);
}
let outgoing_message =
OutgoingMessage::Request(request.request_with_id(outgoing_message_id));
let _ = self.sender.send(outgoing_message);
rx_approve
}
pub(crate) async fn notify_client_response(&self, id: RequestId, result: Result) {
let entry = {
let mut request_id_to_callback = self.request_id_to_callback.lock().await;
request_id_to_callback.remove_entry(&id)
};
match entry {
Some((id, sender)) => {
if let Err(err) = sender.send(result) {
warn!("could not notify callback for {id:?} due to: {err:?}");
}
}
None => {
warn!("could not find callback for {id:?}");
}
}
}
pub(crate) async fn send_response<T: Serialize>(&self, id: RequestId, response: T) {
match serde_json::to_value(response) {
Ok(result) => {
let outgoing_message = OutgoingMessage::Response(OutgoingResponse { id, result });
let _ = self.sender.send(outgoing_message);
}
Err(err) => {
self.send_error(
id,
JSONRPCErrorError {
code: INTERNAL_ERROR_CODE,
message: format!("failed to serialize response: {err}"),
data: None,
},
)
.await;
}
}
}
pub(crate) async fn send_server_notification(&self, notification: ServerNotification) {
let _ = self
.sender
.send(OutgoingMessage::AppServerNotification(notification));
}
/// All notifications should be migrated to [`ServerNotification`] and
/// [`OutgoingMessage::Notification`] should be removed.
pub(crate) async fn send_notification(&self, notification: OutgoingNotification) {
let outgoing_message = OutgoingMessage::Notification(notification);
let _ = self.sender.send(outgoing_message);
}
pub(crate) async fn send_error(&self, id: RequestId, error: JSONRPCErrorError) {
let outgoing_message = OutgoingMessage::Error(OutgoingError { id, error });
let _ = self.sender.send(outgoing_message);
}
}
/// Outgoing message from the server to the client.
#[derive(Debug, Clone, Serialize)]
#[serde(untagged)]
pub(crate) enum OutgoingMessage {
Request(ServerRequest),
Notification(OutgoingNotification),
/// AppServerNotification is specific to the case where this is run as an
/// "app server" as opposed to an MCP server.
AppServerNotification(ServerNotification),
Response(OutgoingResponse),
Error(OutgoingError),
}
#[derive(Debug, Clone, PartialEq, Serialize)]
pub(crate) struct OutgoingNotification {
pub method: String,
#[serde(default, skip_serializing_if = "Option::is_none")]
pub params: Option<serde_json::Value>,
}
#[derive(Debug, Clone, PartialEq, Serialize)]
pub(crate) struct OutgoingResponse {
pub id: RequestId,
pub result: Result,
}
#[derive(Debug, Clone, PartialEq, Serialize)]
pub(crate) struct OutgoingError {
pub error: JSONRPCErrorError,
pub id: RequestId,
}
#[cfg(test)]
mod tests {
use codex_app_server_protocol::LoginChatGptCompleteNotification;
use pretty_assertions::assert_eq;
use serde_json::json;
use uuid::Uuid;
use super::*;
#[test]
fn verify_server_notification_serialization() {
let notification =
ServerNotification::LoginChatGptComplete(LoginChatGptCompleteNotification {
login_id: Uuid::nil(),
success: true,
error: None,
});
let jsonrpc_notification = OutgoingMessage::AppServerNotification(notification);
assert_eq!(
json!({
"method": "loginChatGptComplete",
"params": {
"loginId": Uuid::nil(),
"success": true,
},
}),
serde_json::to_value(jsonrpc_notification)
.expect("ensure the strum macros serialize the method field correctly"),
"ensure the strum macros serialize the method field correctly"
);
}
}

View File

@@ -0,0 +1,3 @@
// Single integration test binary that aggregates all test modules.
// The submodules live in `tests/suite/`.
mod suite;

View File

@@ -0,0 +1,21 @@
[package]
edition = "2024"
name = "app_test_support"
version = { workspace = true }
[lib]
path = "lib.rs"
[dependencies]
anyhow = { workspace = true }
assert_cmd = { workspace = true }
codex-app-server-protocol = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
tokio = { workspace = true, features = [
"io-std",
"macros",
"process",
"rt-multi-thread",
] }
wiremock = { workspace = true }

View File

@@ -0,0 +1,17 @@
mod mcp_process;
mod mock_model_server;
mod responses;
use codex_app_server_protocol::JSONRPCResponse;
pub use mcp_process::McpProcess;
pub use mock_model_server::create_mock_chat_completions_server;
pub use responses::create_apply_patch_sse_response;
pub use responses::create_final_assistant_message_sse_response;
pub use responses::create_shell_sse_response;
use serde::de::DeserializeOwned;
pub fn to_response<T: DeserializeOwned>(response: JSONRPCResponse) -> anyhow::Result<T> {
let value = serde_json::to_value(response.result)?;
let codex_response = serde_json::from_value(value)?;
Ok(codex_response)
}

View File

@@ -0,0 +1,502 @@
use std::collections::VecDeque;
use std::path::Path;
use std::process::Stdio;
use std::sync::atomic::AtomicI64;
use std::sync::atomic::Ordering;
use tokio::io::AsyncBufReadExt;
use tokio::io::AsyncWriteExt;
use tokio::io::BufReader;
use tokio::process::Child;
use tokio::process::ChildStdin;
use tokio::process::ChildStdout;
use anyhow::Context;
use assert_cmd::prelude::*;
use codex_app_server_protocol::AddConversationListenerParams;
use codex_app_server_protocol::ArchiveConversationParams;
use codex_app_server_protocol::CancelLoginChatGptParams;
use codex_app_server_protocol::ClientInfo;
use codex_app_server_protocol::ClientNotification;
use codex_app_server_protocol::GetAuthStatusParams;
use codex_app_server_protocol::InitializeParams;
use codex_app_server_protocol::InterruptConversationParams;
use codex_app_server_protocol::ListConversationsParams;
use codex_app_server_protocol::LoginApiKeyParams;
use codex_app_server_protocol::NewConversationParams;
use codex_app_server_protocol::RemoveConversationListenerParams;
use codex_app_server_protocol::ResumeConversationParams;
use codex_app_server_protocol::SendUserMessageParams;
use codex_app_server_protocol::SendUserTurnParams;
use codex_app_server_protocol::ServerRequest;
use codex_app_server_protocol::SetDefaultModelParams;
use codex_app_server_protocol::JSONRPCError;
use codex_app_server_protocol::JSONRPCMessage;
use codex_app_server_protocol::JSONRPCNotification;
use codex_app_server_protocol::JSONRPCRequest;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::RequestId;
use std::process::Command as StdCommand;
use tokio::process::Command;
pub struct McpProcess {
next_request_id: AtomicI64,
/// Retain this child process until the client is dropped. The Tokio runtime
/// will make a "best effort" to reap the process after it exits, but it is
/// not a guarantee. See the `kill_on_drop` documentation for details.
#[allow(dead_code)]
process: Child,
stdin: ChildStdin,
stdout: BufReader<ChildStdout>,
pending_user_messages: VecDeque<JSONRPCNotification>,
}
impl McpProcess {
pub async fn new(codex_home: &Path) -> anyhow::Result<Self> {
Self::new_with_env(codex_home, &[]).await
}
/// Creates a new MCP process, allowing tests to override or remove
/// specific environment variables for the child process only.
///
/// Pass a tuple of (key, Some(value)) to set/override, or (key, None) to
/// remove a variable from the child's environment.
pub async fn new_with_env(
codex_home: &Path,
env_overrides: &[(&str, Option<&str>)],
) -> anyhow::Result<Self> {
// Use assert_cmd to locate the binary path and then switch to tokio::process::Command
let std_cmd = StdCommand::cargo_bin("codex-app-server")
.context("should find binary for codex-mcp-server")?;
let program = std_cmd.get_program().to_owned();
let mut cmd = Command::new(program);
cmd.stdin(Stdio::piped());
cmd.stdout(Stdio::piped());
cmd.stderr(Stdio::piped());
cmd.env("CODEX_HOME", codex_home);
cmd.env("RUST_LOG", "debug");
for (k, v) in env_overrides {
match v {
Some(val) => {
cmd.env(k, val);
}
None => {
cmd.env_remove(k);
}
}
}
let mut process = cmd
.kill_on_drop(true)
.spawn()
.context("codex-mcp-server proc should start")?;
let stdin = process
.stdin
.take()
.ok_or_else(|| anyhow::format_err!("mcp should have stdin fd"))?;
let stdout = process
.stdout
.take()
.ok_or_else(|| anyhow::format_err!("mcp should have stdout fd"))?;
let stdout = BufReader::new(stdout);
// Forward child's stderr to our stderr so failures are visible even
// when stdout/stderr are captured by the test harness.
if let Some(stderr) = process.stderr.take() {
let mut stderr_reader = BufReader::new(stderr).lines();
tokio::spawn(async move {
while let Ok(Some(line)) = stderr_reader.next_line().await {
eprintln!("[mcp stderr] {line}");
}
});
}
Ok(Self {
next_request_id: AtomicI64::new(0),
process,
stdin,
stdout,
pending_user_messages: VecDeque::new(),
})
}
/// Performs the initialization handshake with the MCP server.
pub async fn initialize(&mut self) -> anyhow::Result<()> {
let params = Some(serde_json::to_value(InitializeParams {
client_info: ClientInfo {
name: "codex-app-server-tests".to_string(),
title: None,
version: "0.1.0".to_string(),
},
})?);
let req_id = self.send_request("initialize", params).await?;
let initialized = self.read_jsonrpc_message().await?;
let JSONRPCMessage::Response(response) = initialized else {
unreachable!("expected JSONRPCMessage::Response for initialize, got {initialized:?}");
};
if response.id != RequestId::Integer(req_id) {
anyhow::bail!(
"initialize response id mismatch: expected {}, got {:?}",
req_id,
response.id
);
}
// Send notifications/initialized to ack the response.
self.send_notification(ClientNotification::Initialized)
.await?;
Ok(())
}
/// Send a `newConversation` JSON-RPC request.
pub async fn send_new_conversation_request(
&mut self,
params: NewConversationParams,
) -> anyhow::Result<i64> {
let params = Some(serde_json::to_value(params)?);
self.send_request("newConversation", params).await
}
/// Send an `archiveConversation` JSON-RPC request.
pub async fn send_archive_conversation_request(
&mut self,
params: ArchiveConversationParams,
) -> anyhow::Result<i64> {
let params = Some(serde_json::to_value(params)?);
self.send_request("archiveConversation", params).await
}
/// Send an `addConversationListener` JSON-RPC request.
pub async fn send_add_conversation_listener_request(
&mut self,
params: AddConversationListenerParams,
) -> anyhow::Result<i64> {
let params = Some(serde_json::to_value(params)?);
self.send_request("addConversationListener", params).await
}
/// Send a `sendUserMessage` JSON-RPC request with a single text item.
pub async fn send_send_user_message_request(
&mut self,
params: SendUserMessageParams,
) -> anyhow::Result<i64> {
// Wire format expects variants in camelCase; text item uses external tagging.
let params = Some(serde_json::to_value(params)?);
self.send_request("sendUserMessage", params).await
}
/// Send a `removeConversationListener` JSON-RPC request.
pub async fn send_remove_conversation_listener_request(
&mut self,
params: RemoveConversationListenerParams,
) -> anyhow::Result<i64> {
let params = Some(serde_json::to_value(params)?);
self.send_request("removeConversationListener", params)
.await
}
/// Send a `sendUserTurn` JSON-RPC request.
pub async fn send_send_user_turn_request(
&mut self,
params: SendUserTurnParams,
) -> anyhow::Result<i64> {
let params = Some(serde_json::to_value(params)?);
self.send_request("sendUserTurn", params).await
}
/// Send a `interruptConversation` JSON-RPC request.
pub async fn send_interrupt_conversation_request(
&mut self,
params: InterruptConversationParams,
) -> anyhow::Result<i64> {
let params = Some(serde_json::to_value(params)?);
self.send_request("interruptConversation", params).await
}
/// Send a `getAuthStatus` JSON-RPC request.
pub async fn send_get_auth_status_request(
&mut self,
params: GetAuthStatusParams,
) -> anyhow::Result<i64> {
let params = Some(serde_json::to_value(params)?);
self.send_request("getAuthStatus", params).await
}
/// Send a `getUserSavedConfig` JSON-RPC request.
pub async fn send_get_user_saved_config_request(&mut self) -> anyhow::Result<i64> {
self.send_request("getUserSavedConfig", None).await
}
/// Send a `getUserAgent` JSON-RPC request.
pub async fn send_get_user_agent_request(&mut self) -> anyhow::Result<i64> {
self.send_request("getUserAgent", None).await
}
/// Send a `userInfo` JSON-RPC request.
pub async fn send_user_info_request(&mut self) -> anyhow::Result<i64> {
self.send_request("userInfo", None).await
}
/// Send a `setDefaultModel` JSON-RPC request.
pub async fn send_set_default_model_request(
&mut self,
params: SetDefaultModelParams,
) -> anyhow::Result<i64> {
let params = Some(serde_json::to_value(params)?);
self.send_request("setDefaultModel", params).await
}
/// Send a `listConversations` JSON-RPC request.
pub async fn send_list_conversations_request(
&mut self,
params: ListConversationsParams,
) -> anyhow::Result<i64> {
let params = Some(serde_json::to_value(params)?);
self.send_request("listConversations", params).await
}
/// Send a `resumeConversation` JSON-RPC request.
pub async fn send_resume_conversation_request(
&mut self,
params: ResumeConversationParams,
) -> anyhow::Result<i64> {
let params = Some(serde_json::to_value(params)?);
self.send_request("resumeConversation", params).await
}
/// Send a `loginApiKey` JSON-RPC request.
pub async fn send_login_api_key_request(
&mut self,
params: LoginApiKeyParams,
) -> anyhow::Result<i64> {
let params = Some(serde_json::to_value(params)?);
self.send_request("loginApiKey", params).await
}
/// Send a `loginChatGpt` JSON-RPC request.
pub async fn send_login_chat_gpt_request(&mut self) -> anyhow::Result<i64> {
self.send_request("loginChatGpt", None).await
}
/// Send a `cancelLoginChatGpt` JSON-RPC request.
pub async fn send_cancel_login_chat_gpt_request(
&mut self,
params: CancelLoginChatGptParams,
) -> anyhow::Result<i64> {
let params = Some(serde_json::to_value(params)?);
self.send_request("cancelLoginChatGpt", params).await
}
/// Send a `logoutChatGpt` JSON-RPC request.
pub async fn send_logout_chat_gpt_request(&mut self) -> anyhow::Result<i64> {
self.send_request("logoutChatGpt", None).await
}
/// Send a `fuzzyFileSearch` JSON-RPC request.
pub async fn send_fuzzy_file_search_request(
&mut self,
query: &str,
roots: Vec<String>,
cancellation_token: Option<String>,
) -> anyhow::Result<i64> {
let mut params = serde_json::json!({
"query": query,
"roots": roots,
});
if let Some(token) = cancellation_token {
params["cancellationToken"] = serde_json::json!(token);
}
self.send_request("fuzzyFileSearch", Some(params)).await
}
async fn send_request(
&mut self,
method: &str,
params: Option<serde_json::Value>,
) -> anyhow::Result<i64> {
let request_id = self.next_request_id.fetch_add(1, Ordering::Relaxed);
let message = JSONRPCMessage::Request(JSONRPCRequest {
id: RequestId::Integer(request_id),
method: method.to_string(),
params,
});
self.send_jsonrpc_message(message).await?;
Ok(request_id)
}
pub async fn send_response(
&mut self,
id: RequestId,
result: serde_json::Value,
) -> anyhow::Result<()> {
self.send_jsonrpc_message(JSONRPCMessage::Response(JSONRPCResponse { id, result }))
.await
}
pub async fn send_notification(
&mut self,
notification: ClientNotification,
) -> anyhow::Result<()> {
let value = serde_json::to_value(notification)?;
self.send_jsonrpc_message(JSONRPCMessage::Notification(JSONRPCNotification {
method: value
.get("method")
.and_then(|m| m.as_str())
.ok_or_else(|| anyhow::format_err!("notification missing method field"))?
.to_string(),
params: value.get("params").cloned(),
}))
.await
}
async fn send_jsonrpc_message(&mut self, message: JSONRPCMessage) -> anyhow::Result<()> {
eprintln!("writing message to stdin: {message:?}");
let payload = serde_json::to_string(&message)?;
self.stdin.write_all(payload.as_bytes()).await?;
self.stdin.write_all(b"\n").await?;
self.stdin.flush().await?;
Ok(())
}
async fn read_jsonrpc_message(&mut self) -> anyhow::Result<JSONRPCMessage> {
let mut line = String::new();
self.stdout.read_line(&mut line).await?;
let message = serde_json::from_str::<JSONRPCMessage>(&line)?;
eprintln!("read message from stdout: {message:?}");
Ok(message)
}
pub async fn read_stream_until_request_message(&mut self) -> anyhow::Result<ServerRequest> {
eprintln!("in read_stream_until_request_message()");
loop {
let message = self.read_jsonrpc_message().await?;
match message {
JSONRPCMessage::Notification(notification) => {
eprintln!("notification: {notification:?}");
self.enqueue_user_message(notification);
}
JSONRPCMessage::Request(jsonrpc_request) => {
return jsonrpc_request.try_into().with_context(
|| "failed to deserialize ServerRequest from JSONRPCRequest",
);
}
JSONRPCMessage::Error(_) => {
anyhow::bail!("unexpected JSONRPCMessage::Error: {message:?}");
}
JSONRPCMessage::Response(_) => {
anyhow::bail!("unexpected JSONRPCMessage::Response: {message:?}");
}
}
}
}
pub async fn read_stream_until_response_message(
&mut self,
request_id: RequestId,
) -> anyhow::Result<JSONRPCResponse> {
eprintln!("in read_stream_until_response_message({request_id:?})");
loop {
let message = self.read_jsonrpc_message().await?;
match message {
JSONRPCMessage::Notification(notification) => {
eprintln!("notification: {notification:?}");
self.enqueue_user_message(notification);
}
JSONRPCMessage::Request(_) => {
anyhow::bail!("unexpected JSONRPCMessage::Request: {message:?}");
}
JSONRPCMessage::Error(_) => {
anyhow::bail!("unexpected JSONRPCMessage::Error: {message:?}");
}
JSONRPCMessage::Response(jsonrpc_response) => {
if jsonrpc_response.id == request_id {
return Ok(jsonrpc_response);
}
}
}
}
}
pub async fn read_stream_until_error_message(
&mut self,
request_id: RequestId,
) -> anyhow::Result<JSONRPCError> {
loop {
let message = self.read_jsonrpc_message().await?;
match message {
JSONRPCMessage::Notification(notification) => {
eprintln!("notification: {notification:?}");
self.enqueue_user_message(notification);
}
JSONRPCMessage::Request(_) => {
anyhow::bail!("unexpected JSONRPCMessage::Request: {message:?}");
}
JSONRPCMessage::Response(_) => {
// Keep scanning; we're waiting for an error with matching id.
}
JSONRPCMessage::Error(err) => {
if err.id == request_id {
return Ok(err);
}
}
}
}
}
pub async fn read_stream_until_notification_message(
&mut self,
method: &str,
) -> anyhow::Result<JSONRPCNotification> {
eprintln!("in read_stream_until_notification_message({method})");
if let Some(notification) = self.take_pending_notification_by_method(method) {
return Ok(notification);
}
loop {
let message = self.read_jsonrpc_message().await?;
match message {
JSONRPCMessage::Notification(notification) => {
if notification.method == method {
return Ok(notification);
}
self.enqueue_user_message(notification);
}
JSONRPCMessage::Request(_) => {
anyhow::bail!("unexpected JSONRPCMessage::Request: {message:?}");
}
JSONRPCMessage::Error(_) => {
anyhow::bail!("unexpected JSONRPCMessage::Error: {message:?}");
}
JSONRPCMessage::Response(_) => {
anyhow::bail!("unexpected JSONRPCMessage::Response: {message:?}");
}
}
}
}
fn take_pending_notification_by_method(&mut self, method: &str) -> Option<JSONRPCNotification> {
if let Some(pos) = self
.pending_user_messages
.iter()
.position(|notification| notification.method == method)
{
return self.pending_user_messages.remove(pos);
}
None
}
fn enqueue_user_message(&mut self, notification: JSONRPCNotification) {
if notification.method == "codex/event/user_message" {
self.pending_user_messages.push_back(notification);
}
}
}

View File

@@ -0,0 +1,47 @@
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering;
use wiremock::Mock;
use wiremock::MockServer;
use wiremock::Respond;
use wiremock::ResponseTemplate;
use wiremock::matchers::method;
use wiremock::matchers::path;
/// Create a mock server that will provide the responses, in order, for
/// requests to the `/v1/chat/completions` endpoint.
pub async fn create_mock_chat_completions_server(responses: Vec<String>) -> MockServer {
let server = MockServer::start().await;
let num_calls = responses.len();
let seq_responder = SeqResponder {
num_calls: AtomicUsize::new(0),
responses,
};
Mock::given(method("POST"))
.and(path("/v1/chat/completions"))
.respond_with(seq_responder)
.expect(num_calls as u64)
.mount(&server)
.await;
server
}
struct SeqResponder {
num_calls: AtomicUsize,
responses: Vec<String>,
}
impl Respond for SeqResponder {
fn respond(&self, _: &wiremock::Request) -> ResponseTemplate {
let call_num = self.num_calls.fetch_add(1, Ordering::SeqCst);
match self.responses.get(call_num) {
Some(response) => ResponseTemplate::new(200)
.insert_header("content-type", "text/event-stream")
.set_body_raw(response.clone(), "text/event-stream"),
None => panic!("no response for {call_num}"),
}
}
}

View File

@@ -0,0 +1,95 @@
use serde_json::json;
use std::path::Path;
pub fn create_shell_sse_response(
command: Vec<String>,
workdir: Option<&Path>,
timeout_ms: Option<u64>,
call_id: &str,
) -> anyhow::Result<String> {
// The `arguments`` for the `shell` tool is a serialized JSON object.
let tool_call_arguments = serde_json::to_string(&json!({
"command": command,
"workdir": workdir.map(|w| w.to_string_lossy()),
"timeout": timeout_ms
}))?;
let tool_call = json!({
"choices": [
{
"delta": {
"tool_calls": [
{
"id": call_id,
"function": {
"name": "shell",
"arguments": tool_call_arguments
}
}
]
},
"finish_reason": "tool_calls"
}
]
});
let sse = format!(
"data: {}\n\ndata: DONE\n\n",
serde_json::to_string(&tool_call)?
);
Ok(sse)
}
pub fn create_final_assistant_message_sse_response(message: &str) -> anyhow::Result<String> {
let assistant_message = json!({
"choices": [
{
"delta": {
"content": message
},
"finish_reason": "stop"
}
]
});
let sse = format!(
"data: {}\n\ndata: DONE\n\n",
serde_json::to_string(&assistant_message)?
);
Ok(sse)
}
pub fn create_apply_patch_sse_response(
patch_content: &str,
call_id: &str,
) -> anyhow::Result<String> {
// Use shell command to call apply_patch with heredoc format
let shell_command = format!("apply_patch <<'EOF'\n{patch_content}\nEOF");
let tool_call_arguments = serde_json::to_string(&json!({
"command": ["bash", "-lc", shell_command]
}))?;
let tool_call = json!({
"choices": [
{
"delta": {
"tool_calls": [
{
"id": call_id,
"function": {
"name": "shell",
"arguments": tool_call_arguments
}
}
]
},
"finish_reason": "tool_calls"
}
]
});
let sse = format!(
"data: {}\n\ndata: DONE\n\n",
serde_json::to_string(&tool_call)?
);
Ok(sse)
}

View File

@@ -1,14 +1,14 @@
use std::path::Path;
use app_test_support::McpProcess;
use app_test_support::to_response;
use codex_app_server_protocol::ArchiveConversationParams;
use codex_app_server_protocol::ArchiveConversationResponse;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::NewConversationParams;
use codex_app_server_protocol::NewConversationResponse;
use codex_app_server_protocol::RequestId;
use codex_core::ARCHIVED_SESSIONS_SUBDIR;
use codex_protocol::mcp_protocol::ArchiveConversationParams;
use codex_protocol::mcp_protocol::ArchiveConversationResponse;
use codex_protocol::mcp_protocol::NewConversationParams;
use codex_protocol::mcp_protocol::NewConversationResponse;
use mcp_test_support::McpProcess;
use mcp_test_support::to_response;
use mcp_types::JSONRPCResponse;
use mcp_types::RequestId;
use tempfile::TempDir;
use tokio::time::timeout;

View File

@@ -1,14 +1,14 @@
use std::path::Path;
use codex_protocol::mcp_protocol::AuthMode;
use codex_protocol::mcp_protocol::GetAuthStatusParams;
use codex_protocol::mcp_protocol::GetAuthStatusResponse;
use codex_protocol::mcp_protocol::LoginApiKeyParams;
use codex_protocol::mcp_protocol::LoginApiKeyResponse;
use mcp_test_support::McpProcess;
use mcp_test_support::to_response;
use mcp_types::JSONRPCResponse;
use mcp_types::RequestId;
use app_test_support::McpProcess;
use app_test_support::to_response;
use codex_app_server_protocol::AuthMode;
use codex_app_server_protocol::GetAuthStatusParams;
use codex_app_server_protocol::GetAuthStatusResponse;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::LoginApiKeyParams;
use codex_app_server_protocol::LoginApiKeyResponse;
use codex_app_server_protocol::RequestId;
use pretty_assertions::assert_eq;
use tempfile::TempDir;
use tokio::time::timeout;

View File

@@ -1,29 +1,35 @@
use std::path::Path;
use app_test_support::McpProcess;
use app_test_support::create_final_assistant_message_sse_response;
use app_test_support::create_mock_chat_completions_server;
use app_test_support::create_shell_sse_response;
use app_test_support::to_response;
use codex_app_server_protocol::AddConversationListenerParams;
use codex_app_server_protocol::AddConversationSubscriptionResponse;
use codex_app_server_protocol::ExecCommandApprovalParams;
use codex_app_server_protocol::InputItem;
use codex_app_server_protocol::JSONRPCNotification;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::NewConversationParams;
use codex_app_server_protocol::NewConversationResponse;
use codex_app_server_protocol::RemoveConversationListenerParams;
use codex_app_server_protocol::RemoveConversationSubscriptionResponse;
use codex_app_server_protocol::RequestId;
use codex_app_server_protocol::SendUserMessageParams;
use codex_app_server_protocol::SendUserMessageResponse;
use codex_app_server_protocol::SendUserTurnParams;
use codex_app_server_protocol::SendUserTurnResponse;
use codex_app_server_protocol::ServerRequest;
use codex_core::protocol::AskForApproval;
use codex_core::protocol::SandboxPolicy;
use codex_core::protocol_config_types::ReasoningEffort;
use codex_core::protocol_config_types::ReasoningSummary;
use codex_core::spawn::CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR;
use codex_protocol::mcp_protocol::AddConversationListenerParams;
use codex_protocol::mcp_protocol::AddConversationSubscriptionResponse;
use codex_protocol::mcp_protocol::EXEC_COMMAND_APPROVAL_METHOD;
use codex_protocol::mcp_protocol::NewConversationParams;
use codex_protocol::mcp_protocol::NewConversationResponse;
use codex_protocol::mcp_protocol::RemoveConversationListenerParams;
use codex_protocol::mcp_protocol::RemoveConversationSubscriptionResponse;
use codex_protocol::mcp_protocol::SendUserMessageParams;
use codex_protocol::mcp_protocol::SendUserMessageResponse;
use codex_protocol::mcp_protocol::SendUserTurnParams;
use codex_protocol::mcp_protocol::SendUserTurnResponse;
use mcp_test_support::McpProcess;
use mcp_test_support::create_final_assistant_message_sse_response;
use mcp_test_support::create_mock_chat_completions_server;
use mcp_test_support::create_shell_sse_response;
use mcp_test_support::to_response;
use mcp_types::JSONRPCNotification;
use mcp_types::JSONRPCResponse;
use mcp_types::RequestId;
use codex_protocol::config_types::SandboxMode;
use codex_protocol::protocol::Event;
use codex_protocol::protocol::EventMsg;
use codex_protocol::protocol::InputMessageKind;
use pretty_assertions::assert_eq;
use std::env;
use tempfile::TempDir;
@@ -115,7 +121,7 @@ async fn test_codex_jsonrpc_conversation_flow() {
let send_user_id = mcp
.send_send_user_message_request(SendUserMessageParams {
conversation_id,
items: vec![codex_protocol::mcp_protocol::InputItem::Text {
items: vec![codex_app_server_protocol::InputItem::Text {
text: "text".to_string(),
}],
})
@@ -265,7 +271,7 @@ async fn test_send_user_turn_changes_approval_policy_behavior() {
let send_user_id = mcp
.send_send_user_message_request(SendUserMessageParams {
conversation_id,
items: vec![codex_protocol::mcp_protocol::InputItem::Text {
items: vec![codex_app_server_protocol::InputItem::Text {
text: "run python".to_string(),
}],
})
@@ -290,11 +296,28 @@ async fn test_send_user_turn_changes_approval_policy_behavior() {
.await
.expect("waiting for exec approval request timeout")
.expect("exec approval request");
assert_eq!(request.method, EXEC_COMMAND_APPROVAL_METHOD);
let ServerRequest::ExecCommandApproval { request_id, params } = request else {
panic!("expected ExecCommandApproval request, got: {request:?}");
};
assert_eq!(
ExecCommandApprovalParams {
conversation_id,
call_id: "call1".to_string(),
command: vec![
"python3".to_string(),
"-c".to_string(),
"print(42)".to_string(),
],
cwd: working_directory.clone(),
reason: None,
},
params
);
// Approve so the first turn can complete
mcp.send_response(
request.id,
request_id,
serde_json::json!({ "decision": codex_core::protocol::ReviewDecision::Approved }),
)
.await
@@ -313,7 +336,7 @@ async fn test_send_user_turn_changes_approval_policy_behavior() {
let send_turn_id = mcp
.send_send_user_turn_request(SendUserTurnParams {
conversation_id,
items: vec![codex_protocol::mcp_protocol::InputItem::Text {
items: vec![codex_app_server_protocol::InputItem::Text {
text: "run python again".to_string(),
}],
cwd: working_directory.clone(),
@@ -349,6 +372,234 @@ async fn test_send_user_turn_changes_approval_policy_behavior() {
}
// Helper: minimal config.toml pointing at mock provider.
#[tokio::test(flavor = "multi_thread", worker_threads = 4)]
async fn test_send_user_turn_updates_sandbox_and_cwd_between_turns() {
if env::var(CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR).is_ok() {
println!(
"Skipping test because it cannot execute when network is disabled in a Codex sandbox."
);
return;
}
let tmp = TempDir::new().expect("tmp dir");
let codex_home = tmp.path().join("codex_home");
std::fs::create_dir(&codex_home).expect("create codex home dir");
let workspace_root = tmp.path().join("workspace");
std::fs::create_dir(&workspace_root).expect("create workspace root");
let first_cwd = workspace_root.join("turn1");
let second_cwd = workspace_root.join("turn2");
std::fs::create_dir(&first_cwd).expect("create first cwd");
std::fs::create_dir(&second_cwd).expect("create second cwd");
let responses = vec![
create_shell_sse_response(
vec![
"bash".to_string(),
"-lc".to_string(),
"echo first turn".to_string(),
],
None,
Some(5000),
"call-first",
)
.expect("create first shell response"),
create_final_assistant_message_sse_response("done first")
.expect("create first final assistant message"),
create_shell_sse_response(
vec![
"bash".to_string(),
"-lc".to_string(),
"echo second turn".to_string(),
],
None,
Some(5000),
"call-second",
)
.expect("create second shell response"),
create_final_assistant_message_sse_response("done second")
.expect("create second final assistant message"),
];
let server = create_mock_chat_completions_server(responses).await;
create_config_toml(&codex_home, &server.uri()).expect("write config");
let mut mcp = McpProcess::new(&codex_home)
.await
.expect("spawn mcp process");
timeout(DEFAULT_READ_TIMEOUT, mcp.initialize())
.await
.expect("init timeout")
.expect("init failed");
let new_conv_id = mcp
.send_new_conversation_request(NewConversationParams {
cwd: Some(first_cwd.to_string_lossy().into_owned()),
approval_policy: Some(AskForApproval::Never),
sandbox: Some(SandboxMode::WorkspaceWrite),
..Default::default()
})
.await
.expect("send newConversation");
let new_conv_resp: JSONRPCResponse = timeout(
DEFAULT_READ_TIMEOUT,
mcp.read_stream_until_response_message(RequestId::Integer(new_conv_id)),
)
.await
.expect("newConversation timeout")
.expect("newConversation resp");
let NewConversationResponse {
conversation_id,
model,
..
} = to_response::<NewConversationResponse>(new_conv_resp)
.expect("deserialize newConversation response");
let add_listener_id = mcp
.send_add_conversation_listener_request(AddConversationListenerParams { conversation_id })
.await
.expect("send addConversationListener");
timeout(
DEFAULT_READ_TIMEOUT,
mcp.read_stream_until_response_message(RequestId::Integer(add_listener_id)),
)
.await
.expect("addConversationListener timeout")
.expect("addConversationListener resp");
let first_turn_id = mcp
.send_send_user_turn_request(SendUserTurnParams {
conversation_id,
items: vec![InputItem::Text {
text: "first turn".to_string(),
}],
cwd: first_cwd.clone(),
approval_policy: AskForApproval::Never,
sandbox_policy: SandboxPolicy::WorkspaceWrite {
writable_roots: vec![first_cwd.clone()],
network_access: false,
exclude_tmpdir_env_var: false,
exclude_slash_tmp: false,
},
model: model.clone(),
effort: Some(ReasoningEffort::Medium),
summary: ReasoningSummary::Auto,
})
.await
.expect("send first sendUserTurn");
timeout(
DEFAULT_READ_TIMEOUT,
mcp.read_stream_until_response_message(RequestId::Integer(first_turn_id)),
)
.await
.expect("sendUserTurn 1 timeout")
.expect("sendUserTurn 1 resp");
timeout(
DEFAULT_READ_TIMEOUT,
mcp.read_stream_until_notification_message("codex/event/task_complete"),
)
.await
.expect("task_complete 1 timeout")
.expect("task_complete 1 notification");
let second_turn_id = mcp
.send_send_user_turn_request(SendUserTurnParams {
conversation_id,
items: vec![InputItem::Text {
text: "second turn".to_string(),
}],
cwd: second_cwd.clone(),
approval_policy: AskForApproval::Never,
sandbox_policy: SandboxPolicy::DangerFullAccess,
model: model.clone(),
effort: Some(ReasoningEffort::Medium),
summary: ReasoningSummary::Auto,
})
.await
.expect("send second sendUserTurn");
timeout(
DEFAULT_READ_TIMEOUT,
mcp.read_stream_until_response_message(RequestId::Integer(second_turn_id)),
)
.await
.expect("sendUserTurn 2 timeout")
.expect("sendUserTurn 2 resp");
let mut env_message: Option<String> = None;
let second_cwd_str = second_cwd.to_string_lossy().into_owned();
for _ in 0..10 {
let notification = timeout(
DEFAULT_READ_TIMEOUT,
mcp.read_stream_until_notification_message("codex/event/user_message"),
)
.await
.expect("user_message timeout")
.expect("user_message notification");
let params = notification
.params
.clone()
.expect("user_message should include params");
let event: Event = serde_json::from_value(params).expect("deserialize user_message event");
if let EventMsg::UserMessage(user) = event.msg
&& matches!(user.kind, Some(InputMessageKind::EnvironmentContext))
&& user.message.contains(&second_cwd_str)
{
env_message = Some(user.message);
break;
}
}
let env_message = env_message.expect("expected environment context update");
assert!(
env_message.contains("<sandbox_mode>danger-full-access</sandbox_mode>"),
"env context should reflect new sandbox mode: {env_message}"
);
assert!(
env_message.contains("<network_access>enabled</network_access>"),
"env context should enable network access for danger-full-access policy: {env_message}"
);
assert!(
env_message.contains(&second_cwd_str),
"env context should include updated cwd: {env_message}"
);
let exec_begin_notification = timeout(
DEFAULT_READ_TIMEOUT,
mcp.read_stream_until_notification_message("codex/event/exec_command_begin"),
)
.await
.expect("exec_command_begin timeout")
.expect("exec_command_begin notification");
let params = exec_begin_notification
.params
.clone()
.expect("exec_command_begin params");
let event: Event = serde_json::from_value(params).expect("deserialize exec begin event");
let exec_begin = match event.msg {
EventMsg::ExecCommandBegin(exec_begin) => exec_begin,
other => panic!("expected ExecCommandBegin event, got {other:?}"),
};
assert_eq!(
exec_begin.cwd, second_cwd,
"exec turn should run from updated cwd"
);
assert_eq!(
exec_begin.command,
vec![
"bash".to_string(),
"-lc".to_string(),
"echo second turn".to_string()
],
"exec turn should run expected command"
);
timeout(
DEFAULT_READ_TIMEOUT,
mcp.read_stream_until_notification_message("codex/event/task_complete"),
)
.await
.expect("task_complete 2 timeout")
.expect("task_complete 2 notification");
}
fn create_config_toml(codex_home: &Path, server_uri: &str) -> std::io::Result<()> {
let config_toml = codex_home.join("config.toml");
std::fs::write(

View File

@@ -1,20 +1,20 @@
use std::collections::HashMap;
use std::path::Path;
use app_test_support::McpProcess;
use app_test_support::to_response;
use codex_app_server_protocol::GetUserSavedConfigResponse;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::Profile;
use codex_app_server_protocol::RequestId;
use codex_app_server_protocol::SandboxSettings;
use codex_app_server_protocol::Tools;
use codex_app_server_protocol::UserSavedConfig;
use codex_core::protocol::AskForApproval;
use codex_protocol::config_types::ReasoningEffort;
use codex_protocol::config_types::ReasoningSummary;
use codex_protocol::config_types::SandboxMode;
use codex_protocol::config_types::Verbosity;
use codex_protocol::mcp_protocol::GetUserSavedConfigResponse;
use codex_protocol::mcp_protocol::Profile;
use codex_protocol::mcp_protocol::SandboxSettings;
use codex_protocol::mcp_protocol::Tools;
use codex_protocol::mcp_protocol::UserSavedConfig;
use mcp_test_support::McpProcess;
use mcp_test_support::to_response;
use mcp_types::JSONRPCResponse;
use mcp_types::RequestId;
use pretty_assertions::assert_eq;
use tempfile::TempDir;
use tokio::time::timeout;

View File

@@ -1,18 +1,18 @@
use std::path::Path;
use codex_protocol::mcp_protocol::AddConversationListenerParams;
use codex_protocol::mcp_protocol::AddConversationSubscriptionResponse;
use codex_protocol::mcp_protocol::InputItem;
use codex_protocol::mcp_protocol::NewConversationParams;
use codex_protocol::mcp_protocol::NewConversationResponse;
use codex_protocol::mcp_protocol::SendUserMessageParams;
use codex_protocol::mcp_protocol::SendUserMessageResponse;
use mcp_test_support::McpProcess;
use mcp_test_support::create_final_assistant_message_sse_response;
use mcp_test_support::create_mock_chat_completions_server;
use mcp_test_support::to_response;
use mcp_types::JSONRPCResponse;
use mcp_types::RequestId;
use app_test_support::McpProcess;
use app_test_support::create_final_assistant_message_sse_response;
use app_test_support::create_mock_chat_completions_server;
use app_test_support::to_response;
use codex_app_server_protocol::AddConversationListenerParams;
use codex_app_server_protocol::AddConversationSubscriptionResponse;
use codex_app_server_protocol::InputItem;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::NewConversationParams;
use codex_app_server_protocol::NewConversationResponse;
use codex_app_server_protocol::RequestId;
use codex_app_server_protocol::SendUserMessageParams;
use codex_app_server_protocol::SendUserMessageResponse;
use pretty_assertions::assert_eq;
use serde_json::json;
use tempfile::TempDir;

View File

@@ -0,0 +1,104 @@
use app_test_support::McpProcess;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::RequestId;
use pretty_assertions::assert_eq;
use serde_json::json;
use tempfile::TempDir;
use tokio::time::timeout;
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_fuzzy_file_search_sorts_and_includes_indices() {
// Prepare a temporary Codex home and a separate root with test files.
let codex_home = TempDir::new().expect("create temp codex home");
let root = TempDir::new().expect("create temp search root");
// Create files designed to have deterministic ordering for query "abc".
std::fs::write(root.path().join("abc"), "x").expect("write file abc");
std::fs::write(root.path().join("abcde"), "x").expect("write file abcx");
std::fs::write(root.path().join("abexy"), "x").expect("write file abcx");
std::fs::write(root.path().join("zzz.txt"), "x").expect("write file zzz");
// Start MCP server and initialize.
let mut mcp = McpProcess::new(codex_home.path()).await.expect("spawn mcp");
timeout(DEFAULT_READ_TIMEOUT, mcp.initialize())
.await
.expect("init timeout")
.expect("init failed");
let root_path = root.path().to_string_lossy().to_string();
// Send fuzzyFileSearch request.
let request_id = mcp
.send_fuzzy_file_search_request("abe", vec![root_path.clone()], None)
.await
.expect("send fuzzyFileSearch");
// Read response and verify shape and ordering.
let resp: JSONRPCResponse = timeout(
DEFAULT_READ_TIMEOUT,
mcp.read_stream_until_response_message(RequestId::Integer(request_id)),
)
.await
.expect("fuzzyFileSearch timeout")
.expect("fuzzyFileSearch resp");
let value = resp.result;
assert_eq!(
value,
json!({
"files": [
{ "root": root_path.clone(), "path": "abexy", "score": 88, "indices": [0, 1, 2] },
{ "root": root_path.clone(), "path": "abcde", "score": 74, "indices": [0, 1, 4] },
]
})
);
}
#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
async fn test_fuzzy_file_search_accepts_cancellation_token() {
let codex_home = TempDir::new().expect("create temp codex home");
let root = TempDir::new().expect("create temp search root");
std::fs::write(root.path().join("alpha.txt"), "contents").expect("write alpha");
let mut mcp = McpProcess::new(codex_home.path()).await.expect("spawn mcp");
timeout(DEFAULT_READ_TIMEOUT, mcp.initialize())
.await
.expect("init timeout")
.expect("init failed");
let root_path = root.path().to_string_lossy().to_string();
let request_id = mcp
.send_fuzzy_file_search_request("alp", vec![root_path.clone()], None)
.await
.expect("send fuzzyFileSearch");
let request_id_2 = mcp
.send_fuzzy_file_search_request(
"alp",
vec![root_path.clone()],
Some(request_id.to_string()),
)
.await
.expect("send fuzzyFileSearch");
let resp: JSONRPCResponse = timeout(
DEFAULT_READ_TIMEOUT,
mcp.read_stream_until_response_message(RequestId::Integer(request_id_2)),
)
.await
.expect("fuzzyFileSearch timeout")
.expect("fuzzyFileSearch resp");
let files = resp
.result
.get("files")
.and_then(|value| value.as_array())
.cloned()
.expect("files array");
assert_eq!(files.len(), 1);
assert_eq!(files[0]["root"], root_path);
assert_eq!(files[0]["path"], "alpha.txt");
}

View File

@@ -1,26 +1,26 @@
#![cfg(unix)]
// Support code lives in the `mcp_test_support` crate under tests/common.
// Support code lives in the `app_test_support` crate under tests/common.
use std::path::Path;
use codex_app_server_protocol::AddConversationListenerParams;
use codex_app_server_protocol::InterruptConversationParams;
use codex_app_server_protocol::InterruptConversationResponse;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::NewConversationParams;
use codex_app_server_protocol::NewConversationResponse;
use codex_app_server_protocol::RequestId;
use codex_app_server_protocol::SendUserMessageParams;
use codex_app_server_protocol::SendUserMessageResponse;
use codex_core::protocol::TurnAbortReason;
use codex_protocol::mcp_protocol::AddConversationListenerParams;
use codex_protocol::mcp_protocol::InterruptConversationParams;
use codex_protocol::mcp_protocol::InterruptConversationResponse;
use codex_protocol::mcp_protocol::NewConversationParams;
use codex_protocol::mcp_protocol::NewConversationResponse;
use codex_protocol::mcp_protocol::SendUserMessageParams;
use codex_protocol::mcp_protocol::SendUserMessageResponse;
use core_test_support::skip_if_no_network;
use mcp_types::JSONRPCResponse;
use mcp_types::RequestId;
use tempfile::TempDir;
use tokio::time::timeout;
use mcp_test_support::McpProcess;
use mcp_test_support::create_mock_chat_completions_server;
use mcp_test_support::create_shell_sse_response;
use mcp_test_support::to_response;
use app_test_support::McpProcess;
use app_test_support::create_mock_chat_completions_server;
use app_test_support::create_shell_sse_response;
use app_test_support::to_response;
const DEFAULT_READ_TIMEOUT: std::time::Duration = std::time::Duration::from_secs(10);
@@ -100,7 +100,7 @@ async fn shell_command_interruption() -> anyhow::Result<()> {
let send_user_id = mcp
.send_send_user_message_request(SendUserMessageParams {
conversation_id,
items: vec![codex_protocol::mcp_protocol::InputItem::Text {
items: vec![codex_app_server_protocol::InputItem::Text {
text: "run first sleep command".to_string(),
}],
})

View File

@@ -1,16 +1,18 @@
use std::fs;
use std::path::Path;
use codex_protocol::mcp_protocol::ListConversationsParams;
use codex_protocol::mcp_protocol::ListConversationsResponse;
use codex_protocol::mcp_protocol::NewConversationParams; // reused for overrides shape
use codex_protocol::mcp_protocol::ResumeConversationParams;
use codex_protocol::mcp_protocol::ResumeConversationResponse;
use mcp_test_support::McpProcess;
use mcp_test_support::to_response;
use mcp_types::JSONRPCNotification;
use mcp_types::JSONRPCResponse;
use mcp_types::RequestId;
use app_test_support::McpProcess;
use app_test_support::to_response;
use codex_app_server_protocol::JSONRPCNotification;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::ListConversationsParams;
use codex_app_server_protocol::ListConversationsResponse;
use codex_app_server_protocol::NewConversationParams; // reused for overrides shape
use codex_app_server_protocol::RequestId;
use codex_app_server_protocol::ResumeConversationParams;
use codex_app_server_protocol::ResumeConversationResponse;
use codex_app_server_protocol::ServerNotification;
use codex_app_server_protocol::SessionConfiguredNotification;
use pretty_assertions::assert_eq;
use serde_json::json;
use tempfile::TempDir;
@@ -111,23 +113,28 @@ async fn test_list_and_resume_conversations() {
.await
.expect("send resumeConversation");
// Expect a codex/event notification with msg.type == session_configured
// Expect a codex/event notification with msg.type == sessionConfigured
let notification: JSONRPCNotification = timeout(
DEFAULT_READ_TIMEOUT,
mcp.read_stream_until_notification_message("codex/event"),
mcp.read_stream_until_notification_message("sessionConfigured"),
)
.await
.expect("session_configured notification timeout")
.expect("session_configured notification");
// Basic shape assertion: ensure event type is session_configured
let msg_type = notification
.params
.as_ref()
.and_then(|p| p.get("msg"))
.and_then(|m| m.get("type"))
.and_then(|t| t.as_str())
.unwrap_or("");
assert_eq!(msg_type, "session_configured");
.expect("sessionConfigured notification timeout")
.expect("sessionConfigured notification");
let session_configured: ServerNotification = notification
.try_into()
.expect("deserialize sessionConfigured notification");
// Basic shape assertion: ensure event type is sessionConfigured
let ServerNotification::SessionConfigured(SessionConfiguredNotification {
model,
rollout_path,
..
}) = session_configured
else {
unreachable!("expected sessionConfigured notification");
};
assert_eq!(model, "o3");
assert_eq!(items[0].path.clone(), rollout_path);
// Then the response for resumeConversation
let resume_resp: JSONRPCResponse = timeout(

View File

@@ -1,17 +1,17 @@
use std::path::Path;
use std::time::Duration;
use app_test_support::McpProcess;
use app_test_support::to_response;
use codex_app_server_protocol::CancelLoginChatGptParams;
use codex_app_server_protocol::CancelLoginChatGptResponse;
use codex_app_server_protocol::GetAuthStatusParams;
use codex_app_server_protocol::GetAuthStatusResponse;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::LoginChatGptResponse;
use codex_app_server_protocol::LogoutChatGptResponse;
use codex_app_server_protocol::RequestId;
use codex_login::login_with_api_key;
use codex_protocol::mcp_protocol::CancelLoginChatGptParams;
use codex_protocol::mcp_protocol::CancelLoginChatGptResponse;
use codex_protocol::mcp_protocol::GetAuthStatusParams;
use codex_protocol::mcp_protocol::GetAuthStatusResponse;
use codex_protocol::mcp_protocol::LoginChatGptResponse;
use codex_protocol::mcp_protocol::LogoutChatGptResponse;
use mcp_test_support::McpProcess;
use mcp_test_support::to_response;
use mcp_types::JSONRPCResponse;
use mcp_types::RequestId;
use tempfile::TempDir;
use tokio::time::timeout;

View File

@@ -0,0 +1,13 @@
mod archive_conversation;
mod auth;
mod codex_message_processor_flow;
mod config;
mod create_conversation;
mod fuzzy_file_search;
mod interrupt;
mod list_resume;
mod login;
mod send_message;
mod set_default_model;
mod user_agent;
mod user_info;

View File

@@ -1,20 +1,20 @@
use std::path::Path;
use codex_protocol::mcp_protocol::AddConversationListenerParams;
use codex_protocol::mcp_protocol::AddConversationSubscriptionResponse;
use codex_protocol::mcp_protocol::ConversationId;
use codex_protocol::mcp_protocol::InputItem;
use codex_protocol::mcp_protocol::NewConversationParams;
use codex_protocol::mcp_protocol::NewConversationResponse;
use codex_protocol::mcp_protocol::SendUserMessageParams;
use codex_protocol::mcp_protocol::SendUserMessageResponse;
use mcp_test_support::McpProcess;
use mcp_test_support::create_final_assistant_message_sse_response;
use mcp_test_support::create_mock_chat_completions_server;
use mcp_test_support::to_response;
use mcp_types::JSONRPCNotification;
use mcp_types::JSONRPCResponse;
use mcp_types::RequestId;
use app_test_support::McpProcess;
use app_test_support::create_final_assistant_message_sse_response;
use app_test_support::create_mock_chat_completions_server;
use app_test_support::to_response;
use codex_app_server_protocol::AddConversationListenerParams;
use codex_app_server_protocol::AddConversationSubscriptionResponse;
use codex_app_server_protocol::InputItem;
use codex_app_server_protocol::JSONRPCNotification;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::NewConversationParams;
use codex_app_server_protocol::NewConversationResponse;
use codex_app_server_protocol::RequestId;
use codex_app_server_protocol::SendUserMessageParams;
use codex_app_server_protocol::SendUserMessageResponse;
use codex_protocol::ConversationId;
use pretty_assertions::assert_eq;
use tempfile::TempDir;
use tokio::time::timeout;

View File

@@ -1,12 +1,12 @@
use std::path::Path;
use app_test_support::McpProcess;
use app_test_support::to_response;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::RequestId;
use codex_app_server_protocol::SetDefaultModelParams;
use codex_app_server_protocol::SetDefaultModelResponse;
use codex_core::config::ConfigToml;
use codex_protocol::mcp_protocol::SetDefaultModelParams;
use codex_protocol::mcp_protocol::SetDefaultModelResponse;
use mcp_test_support::McpProcess;
use mcp_test_support::to_response;
use mcp_types::JSONRPCResponse;
use mcp_types::RequestId;
use pretty_assertions::assert_eq;
use tempfile::TempDir;
use tokio::time::timeout;

View File

@@ -1,8 +1,8 @@
use codex_protocol::mcp_protocol::GetUserAgentResponse;
use mcp_test_support::McpProcess;
use mcp_test_support::to_response;
use mcp_types::JSONRPCResponse;
use mcp_types::RequestId;
use app_test_support::McpProcess;
use app_test_support::to_response;
use codex_app_server_protocol::GetUserAgentResponse;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::RequestId;
use pretty_assertions::assert_eq;
use tempfile::TempDir;
use tokio::time::timeout;
@@ -35,7 +35,7 @@ async fn get_user_agent_returns_current_codex_user_agent() {
let os_info = os_info::get();
let user_agent = format!(
"codex_cli_rs/0.0.0 ({} {}; {}) {} (elicitation test; 0.0.0)",
"codex_cli_rs/0.0.0 ({} {}; {}) {} (codex-app-server-tests; 0.1.0)",
os_info.os_type(),
os_info.version(),
os_info.architecture().unwrap_or("unknown"),

View File

@@ -1,18 +1,18 @@
use std::time::Duration;
use anyhow::Context;
use app_test_support::McpProcess;
use app_test_support::to_response;
use base64::Engine;
use base64::engine::general_purpose::URL_SAFE_NO_PAD;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::RequestId;
use codex_app_server_protocol::UserInfoResponse;
use codex_core::auth::AuthDotJson;
use codex_core::auth::get_auth_file;
use codex_core::auth::write_auth_json;
use codex_core::token_data::IdTokenInfo;
use codex_core::token_data::TokenData;
use codex_protocol::mcp_protocol::UserInfoResponse;
use mcp_test_support::McpProcess;
use mcp_test_support::to_response;
use mcp_types::JSONRPCResponse;
use mcp_types::RequestId;
use pretty_assertions::assert_eq;
use serde_json::json;
use tempfile::TempDir;

View File

@@ -0,0 +1,18 @@
[package]
name = "codex-backend-client"
version = "0.0.0"
edition = "2024"
publish = false
[lib]
path = "src/lib.rs"
[dependencies]
anyhow = "1"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
reqwest = { version = "0.12", default-features = false, features = ["json", "rustls-tls"] }
codex-backend-openapi-models = { path = "../codex-backend-openapi-models" }
[dev-dependencies]
pretty_assertions = "1"

View File

@@ -0,0 +1,244 @@
use crate::types::CodeTaskDetailsResponse;
use crate::types::PaginatedListTaskListItem;
use crate::types::TurnAttemptsSiblingTurnsResponse;
use anyhow::Result;
use reqwest::header::AUTHORIZATION;
use reqwest::header::CONTENT_TYPE;
use reqwest::header::HeaderMap;
use reqwest::header::HeaderName;
use reqwest::header::HeaderValue;
use reqwest::header::USER_AGENT;
use serde::de::DeserializeOwned;
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum PathStyle {
/// /api/codex/…
CodexApi,
/// /wham/…
ChatGptApi,
}
impl PathStyle {
pub fn from_base_url(base_url: &str) -> Self {
if base_url.contains("/backend-api") {
PathStyle::ChatGptApi
} else {
PathStyle::CodexApi
}
}
}
#[derive(Clone, Debug)]
pub struct Client {
base_url: String,
http: reqwest::Client,
bearer_token: Option<String>,
user_agent: Option<HeaderValue>,
chatgpt_account_id: Option<String>,
path_style: PathStyle,
}
impl Client {
pub fn new(base_url: impl Into<String>) -> Result<Self> {
let mut base_url = base_url.into();
// Normalize common ChatGPT hostnames to include /backend-api so we hit the WHAM paths.
// Also trim trailing slashes for consistent URL building.
while base_url.ends_with('/') {
base_url.pop();
}
if (base_url.starts_with("https://chatgpt.com")
|| base_url.starts_with("https://chat.openai.com"))
&& !base_url.contains("/backend-api")
{
base_url = format!("{base_url}/backend-api");
}
let http = reqwest::Client::builder().build()?;
let path_style = PathStyle::from_base_url(&base_url);
Ok(Self {
base_url,
http,
bearer_token: None,
user_agent: None,
chatgpt_account_id: None,
path_style,
})
}
pub fn with_bearer_token(mut self, token: impl Into<String>) -> Self {
self.bearer_token = Some(token.into());
self
}
pub fn with_user_agent(mut self, ua: impl Into<String>) -> Self {
if let Ok(hv) = HeaderValue::from_str(&ua.into()) {
self.user_agent = Some(hv);
}
self
}
pub fn with_chatgpt_account_id(mut self, account_id: impl Into<String>) -> Self {
self.chatgpt_account_id = Some(account_id.into());
self
}
pub fn with_path_style(mut self, style: PathStyle) -> Self {
self.path_style = style;
self
}
fn headers(&self) -> HeaderMap {
let mut h = HeaderMap::new();
if let Some(ua) = &self.user_agent {
h.insert(USER_AGENT, ua.clone());
} else {
h.insert(USER_AGENT, HeaderValue::from_static("codex-cli"));
}
if let Some(token) = &self.bearer_token {
let value = format!("Bearer {token}");
if let Ok(hv) = HeaderValue::from_str(&value) {
h.insert(AUTHORIZATION, hv);
}
}
if let Some(acc) = &self.chatgpt_account_id
&& let Ok(name) = HeaderName::from_bytes(b"ChatGPT-Account-Id")
&& let Ok(hv) = HeaderValue::from_str(acc)
{
h.insert(name, hv);
}
h
}
async fn exec_request(
&self,
req: reqwest::RequestBuilder,
method: &str,
url: &str,
) -> Result<(String, String)> {
let res = req.send().await?;
let status = res.status();
let ct = res
.headers()
.get(CONTENT_TYPE)
.and_then(|v| v.to_str().ok())
.unwrap_or("")
.to_string();
let body = res.text().await.unwrap_or_default();
if !status.is_success() {
anyhow::bail!("{method} {url} failed: {status}; content-type={ct}; body={body}");
}
Ok((body, ct))
}
fn decode_json<T: DeserializeOwned>(&self, url: &str, ct: &str, body: &str) -> Result<T> {
match serde_json::from_str::<T>(body) {
Ok(v) => Ok(v),
Err(e) => {
anyhow::bail!("Decode error for {url}: {e}; content-type={ct}; body={body}");
}
}
}
pub async fn list_tasks(
&self,
limit: Option<i32>,
task_filter: Option<&str>,
environment_id: Option<&str>,
) -> Result<PaginatedListTaskListItem> {
let url = match self.path_style {
PathStyle::CodexApi => format!("{}/api/codex/tasks/list", self.base_url),
PathStyle::ChatGptApi => format!("{}/wham/tasks/list", self.base_url),
};
let req = self.http.get(&url).headers(self.headers());
let req = if let Some(lim) = limit {
req.query(&[("limit", lim)])
} else {
req
};
let req = if let Some(tf) = task_filter {
req.query(&[("task_filter", tf)])
} else {
req
};
let req = if let Some(id) = environment_id {
req.query(&[("environment_id", id)])
} else {
req
};
let (body, ct) = self.exec_request(req, "GET", &url).await?;
self.decode_json::<PaginatedListTaskListItem>(&url, &ct, &body)
}
pub async fn get_task_details(&self, task_id: &str) -> Result<CodeTaskDetailsResponse> {
let (parsed, _body, _ct) = self.get_task_details_with_body(task_id).await?;
Ok(parsed)
}
pub async fn get_task_details_with_body(
&self,
task_id: &str,
) -> Result<(CodeTaskDetailsResponse, String, String)> {
let url = match self.path_style {
PathStyle::CodexApi => format!("{}/api/codex/tasks/{}", self.base_url, task_id),
PathStyle::ChatGptApi => format!("{}/wham/tasks/{}", self.base_url, task_id),
};
let req = self.http.get(&url).headers(self.headers());
let (body, ct) = self.exec_request(req, "GET", &url).await?;
let parsed: CodeTaskDetailsResponse = self.decode_json(&url, &ct, &body)?;
Ok((parsed, body, ct))
}
pub async fn list_sibling_turns(
&self,
task_id: &str,
turn_id: &str,
) -> Result<TurnAttemptsSiblingTurnsResponse> {
let url = match self.path_style {
PathStyle::CodexApi => format!(
"{}/api/codex/tasks/{}/turns/{}/sibling_turns",
self.base_url, task_id, turn_id
),
PathStyle::ChatGptApi => format!(
"{}/wham/tasks/{}/turns/{}/sibling_turns",
self.base_url, task_id, turn_id
),
};
let req = self.http.get(&url).headers(self.headers());
let (body, ct) = self.exec_request(req, "GET", &url).await?;
self.decode_json::<TurnAttemptsSiblingTurnsResponse>(&url, &ct, &body)
}
/// Create a new task (user turn) by POSTing to the appropriate backend path
/// based on `path_style`. Returns the created task id.
pub async fn create_task(&self, request_body: serde_json::Value) -> Result<String> {
let url = match self.path_style {
PathStyle::CodexApi => format!("{}/api/codex/tasks", self.base_url),
PathStyle::ChatGptApi => format!("{}/wham/tasks", self.base_url),
};
let req = self
.http
.post(&url)
.headers(self.headers())
.header(CONTENT_TYPE, HeaderValue::from_static("application/json"))
.json(&request_body);
let (body, ct) = self.exec_request(req, "POST", &url).await?;
// Extract id from JSON: prefer `task.id`; fallback to top-level `id` when present.
match serde_json::from_str::<serde_json::Value>(&body) {
Ok(v) => {
if let Some(id) = v
.get("task")
.and_then(|t| t.get("id"))
.and_then(|s| s.as_str())
{
Ok(id.to_string())
} else if let Some(id) = v.get("id").and_then(|s| s.as_str()) {
Ok(id.to_string())
} else {
anyhow::bail!(
"POST {url} succeeded but no task id found; content-type={ct}; body={body}"
);
}
}
Err(e) => anyhow::bail!("Decode error for {url}: {e}; content-type={ct}; body={body}"),
}
}
}

View File

@@ -0,0 +1,9 @@
mod client;
pub mod types;
pub use client::Client;
pub use types::CodeTaskDetailsResponse;
pub use types::CodeTaskDetailsResponseExt;
pub use types::PaginatedListTaskListItem;
pub use types::TaskListItem;
pub use types::TurnAttemptsSiblingTurnsResponse;

View File

@@ -0,0 +1,369 @@
pub use codex_backend_openapi_models::models::PaginatedListTaskListItem;
pub use codex_backend_openapi_models::models::TaskListItem;
use serde::Deserialize;
use serde::de::Deserializer;
use serde_json::Value;
use std::collections::HashMap;
/// Hand-rolled models for the Cloud Tasks task-details response.
/// The generated OpenAPI models are pretty bad. This is a half-step
/// towards hand-rolling them.
#[derive(Clone, Debug, Deserialize)]
pub struct CodeTaskDetailsResponse {
#[serde(default)]
pub current_user_turn: Option<Turn>,
#[serde(default)]
pub current_assistant_turn: Option<Turn>,
#[serde(default)]
pub current_diff_task_turn: Option<Turn>,
}
#[derive(Clone, Debug, Default, Deserialize)]
pub struct Turn {
#[serde(default)]
pub id: Option<String>,
#[serde(default)]
pub attempt_placement: Option<i64>,
#[serde(default, rename = "turn_status")]
pub turn_status: Option<String>,
#[serde(default, deserialize_with = "deserialize_vec")]
pub sibling_turn_ids: Vec<String>,
#[serde(default, deserialize_with = "deserialize_vec")]
pub input_items: Vec<TurnItem>,
#[serde(default, deserialize_with = "deserialize_vec")]
pub output_items: Vec<TurnItem>,
#[serde(default)]
pub worklog: Option<Worklog>,
#[serde(default)]
pub error: Option<TurnError>,
}
#[derive(Clone, Debug, Default, Deserialize)]
pub struct TurnItem {
#[serde(rename = "type", default)]
pub kind: String,
#[serde(default)]
pub role: Option<String>,
#[serde(default, deserialize_with = "deserialize_vec")]
pub content: Vec<ContentFragment>,
#[serde(default)]
pub diff: Option<String>,
#[serde(default)]
pub output_diff: Option<DiffPayload>,
}
#[derive(Clone, Debug, Deserialize)]
#[serde(untagged)]
pub enum ContentFragment {
Structured(StructuredContent),
Text(String),
}
#[derive(Clone, Debug, Default, Deserialize)]
pub struct StructuredContent {
#[serde(rename = "content_type", default)]
pub content_type: Option<String>,
#[serde(default)]
pub text: Option<String>,
}
#[derive(Clone, Debug, Default, Deserialize)]
pub struct DiffPayload {
#[serde(default)]
pub diff: Option<String>,
}
#[derive(Clone, Debug, Default, Deserialize)]
pub struct Worklog {
#[serde(default, deserialize_with = "deserialize_vec")]
pub messages: Vec<WorklogMessage>,
}
#[derive(Clone, Debug, Default, Deserialize)]
pub struct WorklogMessage {
#[serde(default)]
pub author: Option<Author>,
#[serde(default)]
pub content: Option<WorklogContent>,
}
#[derive(Clone, Debug, Default, Deserialize)]
pub struct Author {
#[serde(default)]
pub role: Option<String>,
}
#[derive(Clone, Debug, Default, Deserialize)]
pub struct WorklogContent {
#[serde(default)]
pub parts: Vec<ContentFragment>,
}
#[derive(Clone, Debug, Default, Deserialize)]
pub struct TurnError {
#[serde(default)]
pub code: Option<String>,
#[serde(default)]
pub message: Option<String>,
}
impl ContentFragment {
fn text(&self) -> Option<&str> {
match self {
ContentFragment::Structured(inner) => {
if inner
.content_type
.as_deref()
.map(|ct| ct.eq_ignore_ascii_case("text"))
.unwrap_or(false)
{
inner.text.as_deref().filter(|s| !s.is_empty())
} else {
None
}
}
ContentFragment::Text(raw) => {
if raw.trim().is_empty() {
None
} else {
Some(raw.as_str())
}
}
}
}
}
impl TurnItem {
fn text_values(&self) -> Vec<String> {
self.content
.iter()
.filter_map(|fragment| fragment.text().map(str::to_string))
.collect()
}
fn diff_text(&self) -> Option<String> {
if self.kind == "output_diff" {
if let Some(diff) = &self.diff
&& !diff.is_empty()
{
return Some(diff.clone());
}
} else if self.kind == "pr"
&& let Some(payload) = &self.output_diff
&& let Some(diff) = &payload.diff
&& !diff.is_empty()
{
return Some(diff.clone());
}
None
}
}
impl Turn {
fn unified_diff(&self) -> Option<String> {
self.output_items.iter().find_map(TurnItem::diff_text)
}
fn message_texts(&self) -> Vec<String> {
let mut out: Vec<String> = self
.output_items
.iter()
.filter(|item| item.kind == "message")
.flat_map(TurnItem::text_values)
.collect();
if let Some(log) = &self.worklog {
for message in &log.messages {
if message.is_assistant() {
out.extend(message.text_values());
}
}
}
out
}
fn user_prompt(&self) -> Option<String> {
let parts: Vec<String> = self
.input_items
.iter()
.filter(|item| item.kind == "message")
.filter(|item| {
item.role
.as_deref()
.map(|r| r.eq_ignore_ascii_case("user"))
.unwrap_or(true)
})
.flat_map(TurnItem::text_values)
.collect();
if parts.is_empty() {
None
} else {
Some(parts.join(
"
",
))
}
}
fn error_summary(&self) -> Option<String> {
self.error.as_ref().and_then(TurnError::summary)
}
}
impl WorklogMessage {
fn is_assistant(&self) -> bool {
self.author
.as_ref()
.and_then(|a| a.role.as_deref())
.map(|role| role.eq_ignore_ascii_case("assistant"))
.unwrap_or(false)
}
fn text_values(&self) -> Vec<String> {
self.content
.as_ref()
.map(|content| {
content
.parts
.iter()
.filter_map(|fragment| fragment.text().map(str::to_string))
.collect()
})
.unwrap_or_default()
}
}
impl TurnError {
fn summary(&self) -> Option<String> {
let code = self.code.as_deref().unwrap_or("");
let message = self.message.as_deref().unwrap_or("");
match (code.is_empty(), message.is_empty()) {
(true, true) => None,
(false, true) => Some(code.to_string()),
(true, false) => Some(message.to_string()),
(false, false) => Some(format!("{code}: {message}")),
}
}
}
pub trait CodeTaskDetailsResponseExt {
/// Attempt to extract a unified diff string from the assistant or diff turn.
fn unified_diff(&self) -> Option<String>;
/// Extract assistant text output messages (no diff) from current turns.
fn assistant_text_messages(&self) -> Vec<String>;
/// Extract the user's prompt text from the current user turn, when present.
fn user_text_prompt(&self) -> Option<String>;
/// Extract an assistant error message (if the turn failed and provided one).
fn assistant_error_message(&self) -> Option<String>;
}
impl CodeTaskDetailsResponseExt for CodeTaskDetailsResponse {
fn unified_diff(&self) -> Option<String> {
[
self.current_diff_task_turn.as_ref(),
self.current_assistant_turn.as_ref(),
]
.into_iter()
.flatten()
.find_map(Turn::unified_diff)
}
fn assistant_text_messages(&self) -> Vec<String> {
let mut out = Vec::new();
for turn in [
self.current_diff_task_turn.as_ref(),
self.current_assistant_turn.as_ref(),
]
.into_iter()
.flatten()
{
out.extend(turn.message_texts());
}
out
}
fn user_text_prompt(&self) -> Option<String> {
self.current_user_turn.as_ref().and_then(Turn::user_prompt)
}
fn assistant_error_message(&self) -> Option<String> {
self.current_assistant_turn
.as_ref()
.and_then(Turn::error_summary)
}
}
fn deserialize_vec<'de, D, T>(deserializer: D) -> Result<Vec<T>, D::Error>
where
D: Deserializer<'de>,
T: Deserialize<'de>,
{
Option::<Vec<T>>::deserialize(deserializer).map(|opt| opt.unwrap_or_default())
}
#[derive(Clone, Debug, Deserialize)]
pub struct TurnAttemptsSiblingTurnsResponse {
#[serde(default)]
pub sibling_turns: Vec<HashMap<String, Value>>,
}
#[cfg(test)]
mod tests {
use super::*;
use pretty_assertions::assert_eq;
fn fixture(name: &str) -> CodeTaskDetailsResponse {
let json = match name {
"diff" => include_str!("../tests/fixtures/task_details_with_diff.json"),
"error" => include_str!("../tests/fixtures/task_details_with_error.json"),
other => panic!("unknown fixture {other}"),
};
serde_json::from_str(json).expect("fixture should deserialize")
}
#[test]
fn unified_diff_prefers_current_diff_task_turn() {
let details = fixture("diff");
let diff = details.unified_diff().expect("diff present");
assert!(diff.contains("diff --git"));
}
#[test]
fn unified_diff_falls_back_to_pr_output_diff() {
let details = fixture("error");
let diff = details.unified_diff().expect("diff from pr output");
assert!(diff.contains("lib.rs"));
}
#[test]
fn assistant_text_messages_extracts_text_content() {
let details = fixture("diff");
let messages = details.assistant_text_messages();
assert_eq!(messages, vec!["Assistant response".to_string()]);
}
#[test]
fn user_text_prompt_joins_parts_with_spacing() {
let details = fixture("diff");
let prompt = details.user_text_prompt().expect("prompt present");
assert_eq!(
prompt,
"First line
Second line"
);
}
#[test]
fn assistant_error_message_combines_code_and_message() {
let details = fixture("error");
let msg = details
.assistant_error_message()
.expect("error should be present");
assert_eq!(msg, "APPLY_FAILED: Patch could not be applied");
}
}

View File

@@ -0,0 +1,38 @@
{
"task": {
"id": "task_123",
"title": "Refactor cloud task client",
"archived": false,
"external_pull_requests": []
},
"current_user_turn": {
"input_items": [
{
"type": "message",
"role": "user",
"content": [
{ "content_type": "text", "text": "First line" },
{ "content_type": "text", "text": "Second line" }
]
}
]
},
"current_assistant_turn": {
"output_items": [
{
"type": "message",
"content": [
{ "content_type": "text", "text": "Assistant response" }
]
}
]
},
"current_diff_task_turn": {
"output_items": [
{
"type": "output_diff",
"diff": "diff --git a/src/main.rs b/src/main.rs\n+fn main() { println!(\"hi\"); }\n"
}
]
}
}

View File

@@ -0,0 +1,22 @@
{
"task": {
"id": "task_456",
"title": "Investigate failure",
"archived": false,
"external_pull_requests": []
},
"current_assistant_turn": {
"output_items": [
{
"type": "pr",
"output_diff": {
"diff": "diff --git a/lib.rs b/lib.rs\n+pub fn hello() {}\n"
}
}
],
"error": {
"code": "APPLY_FAILED",
"message": "Patch could not be applied"
}
}
}

View File

@@ -14,6 +14,7 @@ codex-core = { workspace = true }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
tokio = { workspace = true, features = ["full"] }
codex-git-apply = { path = "../git-apply" }
[dev-dependencies]
tempfile = { workspace = true }

View File

@@ -56,46 +56,24 @@ pub async fn apply_diff_from_task(
}
async fn apply_diff(diff: &str, cwd: Option<PathBuf>) -> anyhow::Result<()> {
let mut cmd = tokio::process::Command::new("git");
if let Some(cwd) = cwd {
cmd.current_dir(cwd);
}
let toplevel_output = cmd
.args(vec!["rev-parse", "--show-toplevel"])
.output()
.await?;
if !toplevel_output.status.success() {
anyhow::bail!("apply must be run from a git repository.");
}
let repo_root = String::from_utf8(toplevel_output.stdout)?
.trim()
.to_string();
let mut git_apply_cmd = tokio::process::Command::new("git")
.args(vec!["apply", "--3way"])
.current_dir(&repo_root)
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()?;
if let Some(mut stdin) = git_apply_cmd.stdin.take() {
tokio::io::AsyncWriteExt::write_all(&mut stdin, diff.as_bytes()).await?;
drop(stdin);
}
let output = git_apply_cmd.wait_with_output().await?;
if !output.status.success() {
let cwd = cwd.unwrap_or(std::env::current_dir().unwrap_or_else(|_| std::env::temp_dir()));
let req = codex_git_apply::ApplyGitRequest {
cwd,
diff: diff.to_string(),
revert: false,
preflight: false,
};
let res = codex_git_apply::apply_git_patch(&req)?;
if res.exit_code != 0 {
anyhow::bail!(
"Git apply failed with status {}: {}",
output.status,
String::from_utf8_lossy(&output.stderr)
"Git apply failed (applied={}, skipped={}, conflicts={})\nstdout:\n{}\nstderr:\n{}",
res.applied_paths.len(),
res.skipped_paths.len(),
res.conflicted_paths.len(),
res.stdout,
res.stderr
);
}
println!("Successfully applied diff");
Ok(())
}

View File

@@ -44,6 +44,6 @@ pub(crate) async fn chatgpt_get_request<T: DeserializeOwned>(
} else {
let status = response.status();
let body = response.text().await.unwrap_or_default();
anyhow::bail!("Request failed with status {}: {}", status, body)
anyhow::bail!("Request failed with status {status}: {body}")
}
}

View File

@@ -18,6 +18,7 @@ workspace = true
anyhow = { workspace = true }
clap = { workspace = true, features = ["derive"] }
clap_complete = { workspace = true }
codex-app-server = { workspace = true }
codex-arg0 = { workspace = true }
codex-chatgpt = { workspace = true }
codex-common = { workspace = true, features = ["cli"] }
@@ -27,8 +28,11 @@ codex-login = { workspace = true }
codex-mcp-server = { workspace = true }
codex-process-hardening = { workspace = true }
codex-protocol = { workspace = true }
codex-app-server-protocol = { workspace = true }
codex-protocol-ts = { workspace = true }
codex-responses-api-proxy = { workspace = true }
codex-tui = { workspace = true }
codex-cloud-tasks = { path = "../cloud-tasks" }
ctor = { workspace = true }
owo-colors = { workspace = true }
serde_json = { workspace = true }
@@ -40,8 +44,6 @@ tokio = { workspace = true, features = [
"rt-multi-thread",
"signal",
] }
tracing = { workspace = true }
tracing-subscriber = { workspace = true }
[dev-dependencies]
assert_cmd = { workspace = true }

View File

@@ -1,7 +1,6 @@
pub mod debug_sandbox;
mod exit_status;
pub mod login;
pub mod proto;
use clap::Parser;
use codex_common::CliConfigOverrides;

View File

@@ -1,3 +1,4 @@
use codex_app_server_protocol::AuthMode;
use codex_common::CliConfigOverrides;
use codex_core::CodexAuth;
use codex_core::auth::CLIENT_ID;
@@ -6,8 +7,8 @@ use codex_core::auth::logout;
use codex_core::config::Config;
use codex_core::config::ConfigOverrides;
use codex_login::ServerOptions;
use codex_login::run_device_code_login;
use codex_login::run_login_server;
use codex_protocol::mcp_protocol::AuthMode;
use std::path::PathBuf;
pub async fn login_with_chatgpt(codex_home: PathBuf) -> std::io::Result<()> {
@@ -55,6 +56,32 @@ pub async fn run_login_with_api_key(
}
}
/// Login using the OAuth device code flow.
pub async fn run_login_with_device_code(
cli_config_overrides: CliConfigOverrides,
issuer_base_url: Option<String>,
client_id: Option<String>,
) -> ! {
let config = load_config_or_exit(cli_config_overrides);
let mut opts = ServerOptions::new(
config.codex_home,
client_id.unwrap_or(CLIENT_ID.to_string()),
);
if let Some(iss) = issuer_base_url {
opts.issuer = iss;
}
match run_device_code_login(opts).await {
Ok(()) => {
eprintln!("Successfully logged in");
std::process::exit(0);
}
Err(e) => {
eprintln!("Error logging in with device code: {e}");
std::process::exit(1);
}
}
}
pub async fn run_login_status(cli_config_overrides: CliConfigOverrides) -> ! {
let config = load_config_or_exit(cli_config_overrides);

View File

@@ -10,10 +10,12 @@ use codex_cli::SeatbeltCommand;
use codex_cli::login::run_login_status;
use codex_cli::login::run_login_with_api_key;
use codex_cli::login::run_login_with_chatgpt;
use codex_cli::login::run_login_with_device_code;
use codex_cli::login::run_logout;
use codex_cli::proto;
use codex_cloud_tasks::Cli as CloudTasksCli;
use codex_common::CliConfigOverrides;
use codex_exec::Cli as ExecCli;
use codex_responses_api_proxy::Args as ResponsesApiProxyArgs;
use codex_tui::AppExitInfo;
use codex_tui::Cli as TuiCli;
use owo_colors::OwoColorize;
@@ -23,7 +25,6 @@ use supports_color::Stream;
mod mcp_cmd;
use crate::mcp_cmd::McpCli;
use crate::proto::ProtoCli;
/// Codex CLI
///
@@ -65,9 +66,11 @@ enum Subcommand {
/// [experimental] Run Codex as an MCP server and manage MCP servers.
Mcp(McpCli),
/// Run the Protocol stream via stdin/stdout
#[clap(visible_alias = "p")]
Proto(ProtoCli),
/// [experimental] Run the Codex MCP server (stdio transport).
McpServer,
/// [experimental] Run the app server.
AppServer,
/// Generate shell completion scripts.
Completion(CompletionCommand),
@@ -85,6 +88,13 @@ enum Subcommand {
/// Internal: generate TypeScript protocol bindings.
#[clap(hide = true)]
GenerateTs(GenerateTsCommand),
/// [EXPERIMENTAL] Browse tasks from Codex Cloud and apply changes locally.
#[clap(name = "cloud", alias = "cloud-tasks")]
Cloud(CloudTasksCli),
/// Internal: run the responses API proxy.
#[clap(hide = true)]
ResponsesApiProxy(ResponsesApiProxyArgs),
}
#[derive(Debug, Parser)]
@@ -132,6 +142,20 @@ struct LoginCommand {
#[arg(long = "api-key", value_name = "API_KEY")]
api_key: Option<String>,
/// EXPERIMENTAL: Use device code flow (not yet supported)
/// This feature is experimental and may changed in future releases.
#[arg(long = "experimental_use-device-code", hide = true)]
use_device_code: bool,
/// EXPERIMENTAL: Use custom OAuth issuer base URL (advanced)
/// Override the OAuth issuer base URL (advanced)
#[arg(long = "experimental_issuer", value_name = "URL", hide = true)]
issuer_base_url: Option<String>,
/// EXPERIMENTAL: Use custom OAuth client ID (advanced)
#[arg(long = "experimental_client-id", value_name = "CLIENT_ID", hide = true)]
client_id: Option<String>,
#[command(subcommand)]
action: Option<LoginSubcommand>,
}
@@ -194,25 +218,12 @@ fn print_exit_messages(exit_info: AppExitInfo) {
}
}
pub(crate) const CODEX_SECURE_MODE_ENV_VAR: &str = "CODEX_SECURE_MODE";
/// As early as possible in the process lifecycle, apply hardening measures
/// if the CODEX_SECURE_MODE environment variable is set to "1".
/// As early as possible in the process lifecycle, apply hardening measures. We
/// skip this in debug builds to avoid interfering with debugging.
#[ctor::ctor]
#[cfg(not(debug_assertions))]
fn pre_main_hardening() {
let secure_mode = match std::env::var(CODEX_SECURE_MODE_ENV_VAR) {
Ok(value) => value,
Err(_) => return,
};
if secure_mode == "1" {
codex_process_hardening::pre_main_hardening();
}
// Always clear this env var so child processes don't inherit it.
unsafe {
std::env::remove_var(CODEX_SECURE_MODE_ENV_VAR);
}
codex_process_hardening::pre_main_hardening();
}
fn main() -> anyhow::Result<()> {
@@ -245,10 +256,16 @@ async fn cli_main(codex_linux_sandbox_exe: Option<PathBuf>) -> anyhow::Result<()
);
codex_exec::run_main(exec_cli, codex_linux_sandbox_exe).await?;
}
Some(Subcommand::McpServer) => {
codex_mcp_server::run_main(codex_linux_sandbox_exe, root_config_overrides).await?;
}
Some(Subcommand::Mcp(mut mcp_cli)) => {
// Propagate any root-level config overrides (e.g. `-c key=value`).
prepend_config_flags(&mut mcp_cli.config_overrides, root_config_overrides.clone());
mcp_cli.run(codex_linux_sandbox_exe).await?;
mcp_cli.run().await?;
}
Some(Subcommand::AppServer) => {
codex_app_server::run_main(codex_linux_sandbox_exe, root_config_overrides).await?;
}
Some(Subcommand::Resume(ResumeCommand {
session_id,
@@ -274,7 +291,14 @@ async fn cli_main(codex_linux_sandbox_exe: Option<PathBuf>) -> anyhow::Result<()
run_login_status(login_cli.config_overrides).await;
}
None => {
if let Some(api_key) = login_cli.api_key {
if login_cli.use_device_code {
run_login_with_device_code(
login_cli.config_overrides,
login_cli.issuer_base_url,
login_cli.client_id,
)
.await;
} else if let Some(api_key) = login_cli.api_key {
run_login_with_api_key(login_cli.config_overrides, api_key).await;
} else {
run_login_with_chatgpt(login_cli.config_overrides).await;
@@ -289,16 +313,16 @@ async fn cli_main(codex_linux_sandbox_exe: Option<PathBuf>) -> anyhow::Result<()
);
run_logout(logout_cli.config_overrides).await;
}
Some(Subcommand::Proto(mut proto_cli)) => {
prepend_config_flags(
&mut proto_cli.config_overrides,
root_config_overrides.clone(),
);
proto::run_main(proto_cli).await?;
}
Some(Subcommand::Completion(completion_cli)) => {
print_completion(completion_cli);
}
Some(Subcommand::Cloud(mut cloud_cli)) => {
prepend_config_flags(
&mut cloud_cli.config_overrides,
root_config_overrides.clone(),
);
codex_cloud_tasks::run_main(cloud_cli, codex_linux_sandbox_exe).await?;
}
Some(Subcommand::Debug(debug_args)) => match debug_args.cmd {
DebugCommand::Seatbelt(mut seatbelt_cli) => {
prepend_config_flags(
@@ -330,6 +354,10 @@ async fn cli_main(codex_linux_sandbox_exe: Option<PathBuf>) -> anyhow::Result<()
);
run_apply_command(apply_cli, None).await?;
}
Some(Subcommand::ResponsesApiProxy(args)) => {
tokio::task::spawn_blocking(move || codex_responses_api_proxy::run_main(args))
.await??;
}
Some(Subcommand::GenerateTs(gen_cli)) => {
codex_protocol_ts::generate_ts(&gen_cli.out_dir, gen_cli.prettier.as_deref())?;
}
@@ -427,7 +455,7 @@ fn print_completion(cmd: CompletionCommand) {
mod tests {
use super::*;
use codex_core::protocol::TokenUsage;
use codex_protocol::mcp_protocol::ConversationId;
use codex_protocol::ConversationId;
fn finalize_from_args(args: &[&str]) -> TuiCli {
let cli = MultitoolCli::try_parse_from(args).expect("parse");

View File

@@ -1,5 +1,4 @@
use std::collections::HashMap;
use std::path::PathBuf;
use anyhow::Context;
use anyhow::Result;
@@ -28,14 +27,11 @@ pub struct McpCli {
pub config_overrides: CliConfigOverrides,
#[command(subcommand)]
pub cmd: Option<McpSubcommand>,
pub subcommand: McpSubcommand,
}
#[derive(Debug, clap::Subcommand)]
pub enum McpSubcommand {
/// [experimental] Run the Codex MCP server (stdio transport).
Serve,
/// [experimental] List configured MCP servers.
List(ListArgs),
@@ -87,17 +83,13 @@ pub struct RemoveArgs {
}
impl McpCli {
pub async fn run(self, codex_linux_sandbox_exe: Option<PathBuf>) -> Result<()> {
pub async fn run(self) -> Result<()> {
let McpCli {
config_overrides,
cmd,
subcommand,
} = self;
let subcommand = cmd.unwrap_or(McpSubcommand::Serve);
match subcommand {
McpSubcommand::Serve => {
codex_mcp_server::run_main(codex_linux_sandbox_exe, config_overrides).await?;
}
McpSubcommand::List(args) => {
run_list(&config_overrides, args)?;
}

View File

@@ -1,133 +0,0 @@
use std::io::IsTerminal;
use clap::Parser;
use codex_common::CliConfigOverrides;
use codex_core::AuthManager;
use codex_core::ConversationManager;
use codex_core::NewConversation;
use codex_core::config::Config;
use codex_core::config::ConfigOverrides;
use codex_core::protocol::Event;
use codex_core::protocol::EventMsg;
use codex_core::protocol::Submission;
use tokio::io::AsyncBufReadExt;
use tokio::io::BufReader;
use tracing::error;
use tracing::info;
#[derive(Debug, Parser)]
pub struct ProtoCli {
#[clap(skip)]
pub config_overrides: CliConfigOverrides,
}
pub async fn run_main(opts: ProtoCli) -> anyhow::Result<()> {
if std::io::stdin().is_terminal() {
anyhow::bail!("Protocol mode expects stdin to be a pipe, not a terminal");
}
tracing_subscriber::fmt()
.with_writer(std::io::stderr)
.init();
let ProtoCli { config_overrides } = opts;
let overrides_vec = config_overrides
.parse_overrides()
.map_err(anyhow::Error::msg)?;
let config = Config::load_with_cli_overrides(overrides_vec, ConfigOverrides::default())?;
// Use conversation_manager API to start a conversation
let conversation_manager =
ConversationManager::new(AuthManager::shared(config.codex_home.clone()));
let NewConversation {
conversation_id: _,
conversation,
session_configured,
} = conversation_manager.new_conversation(config).await?;
// Simulate streaming the session_configured event.
let synthetic_event = Event {
// Fake id value.
id: "".to_string(),
msg: EventMsg::SessionConfigured(session_configured),
};
let session_configured_event = match serde_json::to_string(&synthetic_event) {
Ok(s) => s,
Err(e) => {
error!("Failed to serialize session_configured: {e}");
return Err(anyhow::Error::from(e));
}
};
println!("{session_configured_event}");
// Task that reads JSON lines from stdin and forwards to Submission Queue
let sq_fut = {
let conversation = conversation.clone();
async move {
let stdin = BufReader::new(tokio::io::stdin());
let mut lines = stdin.lines();
loop {
let result = tokio::select! {
_ = tokio::signal::ctrl_c() => {
break
},
res = lines.next_line() => res,
};
match result {
Ok(Some(line)) => {
let line = line.trim();
if line.is_empty() {
continue;
}
match serde_json::from_str::<Submission>(line) {
Ok(sub) => {
if let Err(e) = conversation.submit_with_id(sub).await {
error!("{e:#}");
break;
}
}
Err(e) => {
error!("invalid submission: {e}");
}
}
}
_ => {
info!("Submission queue closed");
break;
}
}
}
}
};
// Task that reads events from the agent and prints them as JSON lines to stdout
let eq_fut = async move {
loop {
let event = tokio::select! {
_ = tokio::signal::ctrl_c() => break,
event = conversation.next_event() => event,
};
match event {
Ok(event) => {
let event_str = match serde_json::to_string(&event) {
Ok(s) => s,
Err(e) => {
error!("Failed to serialize event: {e}");
continue;
}
};
println!("{event_str}");
}
Err(e) => {
error!("{e:#}");
break;
}
}
}
info!("Event queue closed");
};
tokio::join!(sq_fut, eq_fut);
Ok(())
}

View File

@@ -0,0 +1,27 @@
[package]
name = "codex-cloud-tasks-client"
version = { workspace = true }
edition = "2024"
[lib]
name = "codex_cloud_tasks_client"
path = "src/lib.rs"
[lints]
workspace = true
[features]
default = ["online"]
online = ["dep:codex-backend-client"]
mock = []
[dependencies]
anyhow = "1"
async-trait = "0.1"
chrono = { version = "0.4", features = ["serde"] }
diffy = "0.4.2"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
thiserror = "2.0.12"
codex-backend-client = { path = "../backend-client", optional = true }
codex-git-apply = { path = "../git-apply" }

View File

@@ -0,0 +1,158 @@
use chrono::DateTime;
use chrono::Utc;
use serde::Deserialize;
use serde::Serialize;
pub type Result<T> = std::result::Result<T, CloudTaskError>;
#[derive(Debug, thiserror::Error)]
pub enum CloudTaskError {
#[error("unimplemented: {0}")]
Unimplemented(&'static str),
#[error("http error: {0}")]
Http(String),
#[error("io error: {0}")]
Io(String),
#[error("{0}")]
Msg(String),
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(transparent)]
pub struct TaskId(pub String);
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "kebab-case")]
pub enum TaskStatus {
Pending,
Ready,
Applied,
Error,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct TaskSummary {
pub id: TaskId,
pub title: String,
pub status: TaskStatus,
pub updated_at: DateTime<Utc>,
/// Backend environment identifier (when available)
pub environment_id: Option<String>,
/// Human-friendly environment label (when available)
pub environment_label: Option<String>,
pub summary: DiffSummary,
/// True when the backend reports this task as a code review.
#[serde(default)]
pub is_review: bool,
/// Number of assistant attempts (best-of-N), when reported by the backend.
#[serde(default)]
pub attempt_total: Option<usize>,
}
#[derive(Clone, Copy, Debug, PartialEq, Eq, Default)]
pub enum AttemptStatus {
Pending,
InProgress,
Completed,
Failed,
Cancelled,
#[default]
Unknown,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct TurnAttempt {
pub turn_id: String,
pub attempt_placement: Option<i64>,
pub created_at: Option<DateTime<Utc>>,
pub status: AttemptStatus,
pub diff: Option<String>,
pub messages: Vec<String>,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "lowercase")]
pub enum ApplyStatus {
Success,
Partial,
Error,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct ApplyOutcome {
pub applied: bool,
pub status: ApplyStatus,
pub message: String,
#[serde(default)]
pub skipped_paths: Vec<String>,
#[serde(default)]
pub conflict_paths: Vec<String>,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
pub struct CreatedTask {
pub id: TaskId,
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize, Default)]
pub struct DiffSummary {
pub files_changed: usize,
pub lines_added: usize,
pub lines_removed: usize,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct TaskText {
pub prompt: Option<String>,
pub messages: Vec<String>,
pub turn_id: Option<String>,
pub sibling_turn_ids: Vec<String>,
pub attempt_placement: Option<i64>,
pub attempt_status: AttemptStatus,
}
impl Default for TaskText {
fn default() -> Self {
Self {
prompt: None,
messages: Vec::new(),
turn_id: None,
sibling_turn_ids: Vec::new(),
attempt_placement: None,
attempt_status: AttemptStatus::Unknown,
}
}
}
#[async_trait::async_trait]
pub trait CloudBackend: Send + Sync {
async fn list_tasks(&self, env: Option<&str>) -> Result<Vec<TaskSummary>>;
async fn get_task_diff(&self, id: TaskId) -> Result<Option<String>>;
/// Return assistant output messages (no diff) when available.
async fn get_task_messages(&self, id: TaskId) -> Result<Vec<String>>;
/// Return the creating prompt and assistant messages (when available).
async fn get_task_text(&self, id: TaskId) -> Result<TaskText>;
/// Return any sibling attempts (best-of-N) for the given assistant turn.
async fn list_sibling_attempts(
&self,
task: TaskId,
turn_id: String,
) -> Result<Vec<TurnAttempt>>;
/// Dry-run apply (preflight) that validates whether the patch would apply cleanly.
/// Never modifies the working tree. When `diff_override` is supplied, the provided diff is
/// used instead of re-fetching the task details so callers can apply alternate attempts.
async fn apply_task_preflight(
&self,
id: TaskId,
diff_override: Option<String>,
) -> Result<ApplyOutcome>;
async fn apply_task(&self, id: TaskId, diff_override: Option<String>) -> Result<ApplyOutcome>;
async fn create_task(
&self,
env_id: &str,
prompt: &str,
git_ref: &str,
qa_mode: bool,
best_of_n: usize,
) -> Result<CreatedTask>;
}

View File

@@ -0,0 +1,769 @@
use crate::ApplyOutcome;
use crate::ApplyStatus;
use crate::AttemptStatus;
use crate::CloudBackend;
use crate::CloudTaskError;
use crate::DiffSummary;
use crate::Result;
use crate::TaskId;
use crate::TaskStatus;
use crate::TaskSummary;
use crate::TurnAttempt;
use crate::api::TaskText;
use chrono::DateTime;
use chrono::Utc;
use codex_backend_client as backend;
use codex_backend_client::CodeTaskDetailsResponseExt;
#[derive(Clone)]
pub struct HttpClient {
pub base_url: String,
backend: backend::Client,
}
impl HttpClient {
pub fn new(base_url: impl Into<String>) -> anyhow::Result<Self> {
let base_url = base_url.into();
let backend = backend::Client::new(base_url.clone())?;
Ok(Self { base_url, backend })
}
pub fn with_bearer_token(mut self, token: impl Into<String>) -> Self {
self.backend = self.backend.clone().with_bearer_token(token);
self
}
pub fn with_user_agent(mut self, ua: impl Into<String>) -> Self {
self.backend = self.backend.clone().with_user_agent(ua);
self
}
pub fn with_chatgpt_account_id(mut self, account_id: impl Into<String>) -> Self {
self.backend = self.backend.clone().with_chatgpt_account_id(account_id);
self
}
fn tasks_api(&self) -> api::Tasks<'_> {
api::Tasks::new(self)
}
fn attempts_api(&self) -> api::Attempts<'_> {
api::Attempts::new(self)
}
fn apply_api(&self) -> api::Apply<'_> {
api::Apply::new(self)
}
}
#[async_trait::async_trait]
impl CloudBackend for HttpClient {
async fn list_tasks(&self, env: Option<&str>) -> Result<Vec<TaskSummary>> {
self.tasks_api().list(env).await
}
async fn get_task_diff(&self, id: TaskId) -> Result<Option<String>> {
self.tasks_api().diff(id).await
}
async fn get_task_messages(&self, id: TaskId) -> Result<Vec<String>> {
self.tasks_api().messages(id).await
}
async fn get_task_text(&self, id: TaskId) -> Result<TaskText> {
self.tasks_api().task_text(id).await
}
async fn list_sibling_attempts(
&self,
task: TaskId,
turn_id: String,
) -> Result<Vec<TurnAttempt>> {
self.attempts_api().list(task, turn_id).await
}
async fn apply_task(&self, id: TaskId, diff_override: Option<String>) -> Result<ApplyOutcome> {
self.apply_api().run(id, diff_override, false).await
}
async fn apply_task_preflight(
&self,
id: TaskId,
diff_override: Option<String>,
) -> Result<ApplyOutcome> {
self.apply_api().run(id, diff_override, true).await
}
async fn create_task(
&self,
env_id: &str,
prompt: &str,
git_ref: &str,
qa_mode: bool,
best_of_n: usize,
) -> Result<crate::CreatedTask> {
self.tasks_api()
.create(env_id, prompt, git_ref, qa_mode, best_of_n)
.await
}
}
mod api {
use super::*;
use serde_json::Value;
use std::cmp::Ordering;
use std::collections::HashMap;
pub(crate) struct Tasks<'a> {
base_url: &'a str,
backend: &'a backend::Client,
}
impl<'a> Tasks<'a> {
pub(crate) fn new(client: &'a HttpClient) -> Self {
Self {
base_url: &client.base_url,
backend: &client.backend,
}
}
pub(crate) async fn list(&self, env: Option<&str>) -> Result<Vec<TaskSummary>> {
let resp = self
.backend
.list_tasks(Some(20), Some("current"), env)
.await
.map_err(|e| CloudTaskError::Http(format!("list_tasks failed: {e}")))?;
let tasks: Vec<TaskSummary> = resp
.items
.into_iter()
.map(map_task_list_item_to_summary)
.collect();
append_error_log(&format!(
"http.list_tasks: env={} items={}",
env.unwrap_or("<all>"),
tasks.len()
));
Ok(tasks)
}
pub(crate) async fn diff(&self, id: TaskId) -> Result<Option<String>> {
let (details, body, ct) = self
.details_with_body(&id.0)
.await
.map_err(|e| CloudTaskError::Http(format!("get_task_details failed: {e}")))?;
if let Some(diff) = details.unified_diff() {
return Ok(Some(diff));
}
let _ = (body, ct);
Ok(None)
}
pub(crate) async fn messages(&self, id: TaskId) -> Result<Vec<String>> {
let (details, body, ct) = self
.details_with_body(&id.0)
.await
.map_err(|e| CloudTaskError::Http(format!("get_task_details failed: {e}")))?;
let mut msgs = details.assistant_text_messages();
if msgs.is_empty() {
msgs.extend(extract_assistant_messages_from_body(&body));
}
if !msgs.is_empty() {
return Ok(msgs);
}
if let Some(err) = details.assistant_error_message() {
return Ok(vec![format!("Task failed: {err}")]);
}
let url = match details_path(self.base_url, &id.0) {
Some(url) => url,
None => format!("{}/api/codex/tasks/{}", self.base_url, id.0),
};
Err(CloudTaskError::Http(format!(
"No assistant text messages in response. GET {url}; content-type={ct}; body={body}"
)))
}
pub(crate) async fn task_text(&self, id: TaskId) -> Result<TaskText> {
let (details, body, _ct) = self
.details_with_body(&id.0)
.await
.map_err(|e| CloudTaskError::Http(format!("get_task_details failed: {e}")))?;
let prompt = details.user_text_prompt();
let mut messages = details.assistant_text_messages();
if messages.is_empty() {
messages.extend(extract_assistant_messages_from_body(&body));
}
let assistant_turn = details.current_assistant_turn.as_ref();
let turn_id = assistant_turn.and_then(|turn| turn.id.clone());
let sibling_turn_ids = assistant_turn
.map(|turn| turn.sibling_turn_ids.clone())
.unwrap_or_default();
let attempt_placement = assistant_turn.and_then(|turn| turn.attempt_placement);
let attempt_status = attempt_status_from_str(
assistant_turn.and_then(|turn| turn.turn_status.as_deref()),
);
Ok(TaskText {
prompt,
messages,
turn_id,
sibling_turn_ids,
attempt_placement,
attempt_status,
})
}
pub(crate) async fn create(
&self,
env_id: &str,
prompt: &str,
git_ref: &str,
qa_mode: bool,
best_of_n: usize,
) -> Result<crate::CreatedTask> {
let mut input_items: Vec<serde_json::Value> = Vec::new();
input_items.push(serde_json::json!({
"type": "message",
"role": "user",
"content": [{ "content_type": "text", "text": prompt }]
}));
if let Ok(diff) = std::env::var("CODEX_STARTING_DIFF")
&& !diff.is_empty()
{
input_items.push(serde_json::json!({
"type": "pre_apply_patch",
"output_diff": { "diff": diff }
}));
}
let mut request_body = serde_json::json!({
"new_task": {
"environment_id": env_id,
"branch": git_ref,
"run_environment_in_qa_mode": qa_mode,
},
"input_items": input_items,
});
if best_of_n > 1
&& let Some(obj) = request_body.as_object_mut()
{
obj.insert(
"metadata".to_string(),
serde_json::json!({ "best_of_n": best_of_n }),
);
}
match self.backend.create_task(request_body).await {
Ok(id) => {
append_error_log(&format!(
"new_task: created id={id} env={} prompt_chars={}",
env_id,
prompt.chars().count()
));
Ok(crate::CreatedTask { id: TaskId(id) })
}
Err(e) => {
append_error_log(&format!(
"new_task: create failed env={} prompt_chars={}: {}",
env_id,
prompt.chars().count(),
e
));
Err(CloudTaskError::Http(format!("create_task failed: {e}")))
}
}
}
async fn details_with_body(
&self,
id: &str,
) -> anyhow::Result<(backend::CodeTaskDetailsResponse, String, String)> {
let (parsed, body, ct) = self.backend.get_task_details_with_body(id).await?;
Ok((parsed, body, ct))
}
}
pub(crate) struct Attempts<'a> {
backend: &'a backend::Client,
}
impl<'a> Attempts<'a> {
pub(crate) fn new(client: &'a HttpClient) -> Self {
Self {
backend: &client.backend,
}
}
pub(crate) async fn list(&self, task: TaskId, turn_id: String) -> Result<Vec<TurnAttempt>> {
let resp = self
.backend
.list_sibling_turns(&task.0, &turn_id)
.await
.map_err(|e| CloudTaskError::Http(format!("list_sibling_turns failed: {e}")))?;
let mut attempts: Vec<TurnAttempt> = resp
.sibling_turns
.iter()
.filter_map(turn_attempt_from_map)
.collect();
attempts.sort_by(compare_attempts);
Ok(attempts)
}
}
pub(crate) struct Apply<'a> {
backend: &'a backend::Client,
}
impl<'a> Apply<'a> {
pub(crate) fn new(client: &'a HttpClient) -> Self {
Self {
backend: &client.backend,
}
}
pub(crate) async fn run(
&self,
task_id: TaskId,
diff_override: Option<String>,
preflight: bool,
) -> Result<ApplyOutcome> {
let id = task_id.0.clone();
let diff = match diff_override {
Some(diff) => diff,
None => {
let details = self.backend.get_task_details(&id).await.map_err(|e| {
CloudTaskError::Http(format!("get_task_details failed: {e}"))
})?;
details.unified_diff().ok_or_else(|| {
CloudTaskError::Msg(format!("No diff available for task {id}"))
})?
}
};
if !is_unified_diff(&diff) {
let summary = summarize_patch_for_logging(&diff);
let mode = if preflight { "preflight" } else { "apply" };
append_error_log(&format!(
"apply_error: id={id} mode={mode} format=non-unified; {summary}"
));
return Ok(ApplyOutcome {
applied: false,
status: ApplyStatus::Error,
message: "Expected unified git diff; backend returned an incompatible format."
.to_string(),
skipped_paths: Vec::new(),
conflict_paths: Vec::new(),
});
}
let req = codex_git_apply::ApplyGitRequest {
cwd: std::env::current_dir().unwrap_or_else(|_| std::env::temp_dir()),
diff: diff.clone(),
revert: false,
preflight,
};
let r = codex_git_apply::apply_git_patch(&req)
.map_err(|e| CloudTaskError::Io(format!("git apply failed to run: {e}")))?;
let status = if r.exit_code == 0 {
ApplyStatus::Success
} else if !r.applied_paths.is_empty() || !r.conflicted_paths.is_empty() {
ApplyStatus::Partial
} else {
ApplyStatus::Error
};
let applied = matches!(status, ApplyStatus::Success) && !preflight;
let message = if preflight {
match status {
ApplyStatus::Success => {
format!("Preflight passed for task {id} (applies cleanly)")
}
ApplyStatus::Partial => format!(
"Preflight: patch does not fully apply for task {id} (applied={}, skipped={}, conflicts={})",
r.applied_paths.len(),
r.skipped_paths.len(),
r.conflicted_paths.len()
),
ApplyStatus::Error => format!(
"Preflight failed for task {id} (applied={}, skipped={}, conflicts={})",
r.applied_paths.len(),
r.skipped_paths.len(),
r.conflicted_paths.len()
),
}
} else {
match status {
ApplyStatus::Success => format!(
"Applied task {id} locally ({} files)",
r.applied_paths.len()
),
ApplyStatus::Partial => format!(
"Apply partially succeeded for task {id} (applied={}, skipped={}, conflicts={})",
r.applied_paths.len(),
r.skipped_paths.len(),
r.conflicted_paths.len()
),
ApplyStatus::Error => format!(
"Apply failed for task {id} (applied={}, skipped={}, conflicts={})",
r.applied_paths.len(),
r.skipped_paths.len(),
r.conflicted_paths.len()
),
}
};
if matches!(status, ApplyStatus::Partial | ApplyStatus::Error)
|| (preflight && !matches!(status, ApplyStatus::Success))
{
let mut log = String::new();
let summary = summarize_patch_for_logging(&diff);
let mode = if preflight { "preflight" } else { "apply" };
use std::fmt::Write as _;
let _ = writeln!(
&mut log,
"apply_result: mode={} id={} status={:?} applied={} skipped={} conflicts={} cmd={}",
mode,
id,
status,
r.applied_paths.len(),
r.skipped_paths.len(),
r.conflicted_paths.len(),
r.cmd_for_log
);
let _ = writeln!(
&mut log,
"stdout_tail=\n{}\nstderr_tail=\n{}",
tail(&r.stdout, 2000),
tail(&r.stderr, 2000)
);
let _ = writeln!(&mut log, "{summary}");
let _ = writeln!(
&mut log,
"----- PATCH BEGIN -----\n{diff}\n----- PATCH END -----"
);
append_error_log(&log);
}
Ok(ApplyOutcome {
applied,
status,
message,
skipped_paths: r.skipped_paths,
conflict_paths: r.conflicted_paths,
})
}
}
fn details_path(base_url: &str, id: &str) -> Option<String> {
if base_url.contains("/backend-api") {
Some(format!("{base_url}/wham/tasks/{id}"))
} else if base_url.contains("/api/codex") {
Some(format!("{base_url}/tasks/{id}"))
} else {
None
}
}
fn extract_assistant_messages_from_body(body: &str) -> Vec<String> {
let mut msgs = Vec::new();
if let Ok(full) = serde_json::from_str::<serde_json::Value>(body)
&& let Some(arr) = full
.get("current_assistant_turn")
.and_then(|v| v.get("worklog"))
.and_then(|v| v.get("messages"))
.and_then(|v| v.as_array())
{
for m in arr {
let is_assistant = m
.get("author")
.and_then(|a| a.get("role"))
.and_then(|r| r.as_str())
== Some("assistant");
if !is_assistant {
continue;
}
if let Some(parts) = m
.get("content")
.and_then(|c| c.get("parts"))
.and_then(|p| p.as_array())
{
for p in parts {
if let Some(s) = p.as_str() {
if !s.is_empty() {
msgs.push(s.to_string());
}
continue;
}
if let Some(obj) = p.as_object()
&& obj.get("content_type").and_then(|t| t.as_str()) == Some("text")
&& let Some(txt) = obj.get("text").and_then(|t| t.as_str())
{
msgs.push(txt.to_string());
}
}
}
}
}
msgs
}
fn turn_attempt_from_map(turn: &HashMap<String, Value>) -> Option<TurnAttempt> {
let turn_id = turn.get("id").and_then(Value::as_str)?.to_string();
let attempt_placement = turn.get("attempt_placement").and_then(Value::as_i64);
let created_at = parse_timestamp_value(turn.get("created_at"));
let status = attempt_status_from_str(turn.get("turn_status").and_then(Value::as_str));
let diff = extract_diff_from_turn(turn);
let messages = extract_assistant_messages_from_turn(turn);
Some(TurnAttempt {
turn_id,
attempt_placement,
created_at,
status,
diff,
messages,
})
}
fn compare_attempts(a: &TurnAttempt, b: &TurnAttempt) -> Ordering {
match (a.attempt_placement, b.attempt_placement) {
(Some(lhs), Some(rhs)) => lhs.cmp(&rhs),
(Some(_), None) => Ordering::Less,
(None, Some(_)) => Ordering::Greater,
(None, None) => match (a.created_at, b.created_at) {
(Some(lhs), Some(rhs)) => lhs.cmp(&rhs),
(Some(_), None) => Ordering::Less,
(None, Some(_)) => Ordering::Greater,
(None, None) => a.turn_id.cmp(&b.turn_id),
},
}
}
fn extract_diff_from_turn(turn: &HashMap<String, Value>) -> Option<String> {
let items = turn.get("output_items").and_then(Value::as_array)?;
for item in items {
match item.get("type").and_then(Value::as_str) {
Some("output_diff") => {
if let Some(diff) = item.get("diff").and_then(Value::as_str)
&& !diff.is_empty()
{
return Some(diff.to_string());
}
}
Some("pr") => {
if let Some(diff) = item
.get("output_diff")
.and_then(Value::as_object)
.and_then(|od| od.get("diff"))
.and_then(Value::as_str)
&& !diff.is_empty()
{
return Some(diff.to_string());
}
}
_ => {}
}
}
None
}
fn extract_assistant_messages_from_turn(turn: &HashMap<String, Value>) -> Vec<String> {
let mut msgs = Vec::new();
if let Some(items) = turn.get("output_items").and_then(Value::as_array) {
for item in items {
if item.get("type").and_then(Value::as_str) != Some("message") {
continue;
}
if let Some(content) = item.get("content").and_then(Value::as_array) {
for part in content {
if part.get("content_type").and_then(Value::as_str) == Some("text")
&& let Some(txt) = part.get("text").and_then(Value::as_str)
&& !txt.is_empty()
{
msgs.push(txt.to_string());
}
}
}
}
}
msgs
}
fn attempt_status_from_str(raw: Option<&str>) -> AttemptStatus {
match raw.unwrap_or_default() {
"failed" => AttemptStatus::Failed,
"completed" => AttemptStatus::Completed,
"in_progress" => AttemptStatus::InProgress,
"pending" => AttemptStatus::Pending,
_ => AttemptStatus::Pending,
}
}
fn parse_timestamp_value(v: Option<&Value>) -> Option<DateTime<Utc>> {
let ts = v?.as_f64()?;
let secs = ts as i64;
let nanos = ((ts - secs as f64) * 1_000_000_000.0) as u32;
Some(DateTime::<Utc>::from(
std::time::UNIX_EPOCH + std::time::Duration::new(secs.max(0) as u64, nanos),
))
}
fn map_task_list_item_to_summary(src: backend::TaskListItem) -> TaskSummary {
let status_display = src.task_status_display.as_ref();
TaskSummary {
id: TaskId(src.id),
title: src.title,
status: map_status(status_display),
updated_at: parse_updated_at(src.updated_at.as_ref()),
environment_id: None,
environment_label: env_label_from_status_display(status_display),
summary: diff_summary_from_status_display(status_display),
is_review: src
.pull_requests
.as_ref()
.is_some_and(|prs| !prs.is_empty()),
attempt_total: attempt_total_from_status_display(status_display),
}
}
fn map_status(v: Option<&HashMap<String, Value>>) -> TaskStatus {
if let Some(val) = v {
if let Some(turn) = val
.get("latest_turn_status_display")
.and_then(Value::as_object)
&& let Some(s) = turn.get("turn_status").and_then(Value::as_str)
{
return match s {
"failed" => TaskStatus::Error,
"completed" => TaskStatus::Ready,
"in_progress" => TaskStatus::Pending,
"pending" => TaskStatus::Pending,
"cancelled" => TaskStatus::Error,
_ => TaskStatus::Pending,
};
}
if let Some(state) = val.get("state").and_then(Value::as_str) {
return match state {
"pending" => TaskStatus::Pending,
"ready" => TaskStatus::Ready,
"applied" => TaskStatus::Applied,
"error" => TaskStatus::Error,
_ => TaskStatus::Pending,
};
}
}
TaskStatus::Pending
}
fn parse_updated_at(ts: Option<&f64>) -> DateTime<Utc> {
if let Some(v) = ts {
let secs = *v as i64;
let nanos = ((*v - secs as f64) * 1_000_000_000.0) as u32;
return DateTime::<Utc>::from(
std::time::UNIX_EPOCH + std::time::Duration::new(secs.max(0) as u64, nanos),
);
}
Utc::now()
}
fn env_label_from_status_display(v: Option<&HashMap<String, Value>>) -> Option<String> {
let map = v?;
map.get("environment_label")
.and_then(Value::as_str)
.map(str::to_string)
}
fn diff_summary_from_status_display(v: Option<&HashMap<String, Value>>) -> DiffSummary {
let mut out = DiffSummary::default();
let Some(map) = v else { return out };
let latest = map
.get("latest_turn_status_display")
.and_then(Value::as_object);
let Some(latest) = latest else { return out };
if let Some(ds) = latest.get("diff_stats").and_then(Value::as_object) {
if let Some(n) = ds.get("files_modified").and_then(Value::as_i64) {
out.files_changed = n.max(0) as usize;
}
if let Some(n) = ds.get("lines_added").and_then(Value::as_i64) {
out.lines_added = n.max(0) as usize;
}
if let Some(n) = ds.get("lines_removed").and_then(Value::as_i64) {
out.lines_removed = n.max(0) as usize;
}
}
out
}
fn attempt_total_from_status_display(v: Option<&HashMap<String, Value>>) -> Option<usize> {
let map = v?;
let latest = map
.get("latest_turn_status_display")
.and_then(Value::as_object)?;
let siblings = latest.get("sibling_turn_ids").and_then(Value::as_array)?;
Some(siblings.len().saturating_add(1))
}
fn is_unified_diff(diff: &str) -> bool {
let t = diff.trim_start();
if t.starts_with("diff --git ") {
return true;
}
let has_dash_headers = diff.contains("\n--- ") && diff.contains("\n+++ ");
let has_hunk = diff.contains("\n@@ ") || diff.starts_with("@@ ");
has_dash_headers && has_hunk
}
fn tail(s: &str, max: usize) -> String {
if s.len() <= max {
s.to_string()
} else {
s[s.len() - max..].to_string()
}
}
fn summarize_patch_for_logging(patch: &str) -> String {
let trimmed = patch.trim_start();
let kind = if trimmed.starts_with("*** Begin Patch") {
"codex-patch"
} else if trimmed.starts_with("diff --git ") || trimmed.contains("\n*** End Patch\n") {
"git-diff"
} else if trimmed.starts_with("@@ ") || trimmed.contains("\n@@ ") {
"unified-diff"
} else {
"unknown"
};
let lines = patch.lines().count();
let chars = patch.len();
let cwd = std::env::current_dir()
.ok()
.map(|p| p.display().to_string())
.unwrap_or_else(|| "<unknown>".to_string());
let head: String = patch.lines().take(20).collect::<Vec<&str>>().join("\n");
let head_trunc = if head.len() > 800 {
format!("{}", &head[..800])
} else {
head
};
format!(
"patch_summary: kind={kind} lines={lines} chars={chars} cwd={cwd} ; head=\n{head_trunc}"
)
}
}
fn append_error_log(message: &str) {
let ts = Utc::now().to_rfc3339();
if let Ok(mut f) = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open("error.log")
{
use std::io::Write as _;
let _ = writeln!(f, "[{ts}] {message}");
}
}

View File

@@ -0,0 +1,29 @@
mod api;
pub use api::ApplyOutcome;
pub use api::ApplyStatus;
pub use api::AttemptStatus;
pub use api::CloudBackend;
pub use api::CloudTaskError;
pub use api::CreatedTask;
pub use api::DiffSummary;
pub use api::Result;
pub use api::TaskId;
pub use api::TaskStatus;
pub use api::TaskSummary;
pub use api::TaskText;
pub use api::TurnAttempt;
#[cfg(feature = "mock")]
mod mock;
#[cfg(feature = "online")]
mod http;
#[cfg(feature = "mock")]
pub use mock::MockClient;
#[cfg(feature = "online")]
pub use http::HttpClient;
// Reusable apply engine now lives in the shared crate `codex-git-apply`.

View File

@@ -0,0 +1,180 @@
use crate::ApplyOutcome;
use crate::AttemptStatus;
use crate::CloudBackend;
use crate::DiffSummary;
use crate::Result;
use crate::TaskId;
use crate::TaskStatus;
use crate::TaskSummary;
use crate::TurnAttempt;
use crate::api::TaskText;
use chrono::Utc;
#[derive(Clone, Default)]
pub struct MockClient;
#[async_trait::async_trait]
impl CloudBackend for MockClient {
async fn list_tasks(&self, _env: Option<&str>) -> Result<Vec<TaskSummary>> {
// Slightly vary content by env to aid tests that rely on the mock
let rows = match _env {
Some("env-A") => vec![("T-2000", "A: First", TaskStatus::Ready)],
Some("env-B") => vec![
("T-3000", "B: One", TaskStatus::Ready),
("T-3001", "B: Two", TaskStatus::Pending),
],
_ => vec![
("T-1000", "Update README formatting", TaskStatus::Ready),
("T-1001", "Fix clippy warnings in core", TaskStatus::Pending),
("T-1002", "Add contributing guide", TaskStatus::Ready),
],
};
let environment_id = _env.map(str::to_string);
let environment_label = match _env {
Some("env-A") => Some("Env A".to_string()),
Some("env-B") => Some("Env B".to_string()),
Some(other) => Some(other.to_string()),
None => Some("Global".to_string()),
};
let mut out = Vec::new();
for (id_str, title, status) in rows {
let id = TaskId(id_str.to_string());
let diff = mock_diff_for(&id);
let (a, d) = count_from_unified(&diff);
out.push(TaskSummary {
id,
title: title.to_string(),
status,
updated_at: Utc::now(),
environment_id: environment_id.clone(),
environment_label: environment_label.clone(),
summary: DiffSummary {
files_changed: 1,
lines_added: a,
lines_removed: d,
},
is_review: false,
attempt_total: Some(if id_str == "T-1000" { 2 } else { 1 }),
});
}
Ok(out)
}
async fn get_task_diff(&self, id: TaskId) -> Result<Option<String>> {
Ok(Some(mock_diff_for(&id)))
}
async fn get_task_messages(&self, _id: TaskId) -> Result<Vec<String>> {
Ok(vec![
"Mock assistant output: this task contains no diff.".to_string(),
])
}
async fn get_task_text(&self, _id: TaskId) -> Result<TaskText> {
Ok(TaskText {
prompt: Some("Why is there no diff?".to_string()),
messages: vec!["Mock assistant output: this task contains no diff.".to_string()],
turn_id: Some("mock-turn".to_string()),
sibling_turn_ids: Vec::new(),
attempt_placement: Some(0),
attempt_status: AttemptStatus::Completed,
})
}
async fn apply_task(&self, id: TaskId, _diff_override: Option<String>) -> Result<ApplyOutcome> {
Ok(ApplyOutcome {
applied: true,
status: crate::ApplyStatus::Success,
message: format!("Applied task {} locally (mock)", id.0),
skipped_paths: Vec::new(),
conflict_paths: Vec::new(),
})
}
async fn apply_task_preflight(
&self,
id: TaskId,
_diff_override: Option<String>,
) -> Result<ApplyOutcome> {
Ok(ApplyOutcome {
applied: false,
status: crate::ApplyStatus::Success,
message: format!("Preflight passed for task {} (mock)", id.0),
skipped_paths: Vec::new(),
conflict_paths: Vec::new(),
})
}
async fn list_sibling_attempts(
&self,
task: TaskId,
_turn_id: String,
) -> Result<Vec<TurnAttempt>> {
if task.0 == "T-1000" {
return Ok(vec![TurnAttempt {
turn_id: "T-1000-attempt-2".to_string(),
attempt_placement: Some(1),
created_at: Some(Utc::now()),
status: AttemptStatus::Completed,
diff: Some(mock_diff_for(&task)),
messages: vec!["Mock alternate attempt".to_string()],
}]);
}
Ok(Vec::new())
}
async fn create_task(
&self,
env_id: &str,
prompt: &str,
git_ref: &str,
qa_mode: bool,
best_of_n: usize,
) -> Result<crate::CreatedTask> {
let _ = (env_id, prompt, git_ref, qa_mode, best_of_n);
let id = format!("task_local_{}", chrono::Utc::now().timestamp_millis());
Ok(crate::CreatedTask { id: TaskId(id) })
}
}
fn mock_diff_for(id: &TaskId) -> String {
match id.0.as_str() {
"T-1000" => {
"diff --git a/README.md b/README.md\nindex 000000..111111 100644\n--- a/README.md\n+++ b/README.md\n@@ -1,2 +1,3 @@\n Intro\n-Hello\n+Hello, world!\n+Task: T-1000\n".to_string()
}
"T-1001" => {
"diff --git a/core/src/lib.rs b/core/src/lib.rs\nindex 000000..111111 100644\n--- a/core/src/lib.rs\n+++ b/core/src/lib.rs\n@@ -1,2 +1,1 @@\n-use foo;\n use bar;\n".to_string()
}
_ => {
"diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md\nindex 000000..111111 100644\n--- /dev/null\n+++ b/CONTRIBUTING.md\n@@ -0,0 +1,3 @@\n+## Contributing\n+Please open PRs.\n+Thanks!\n".to_string()
}
}
}
fn count_from_unified(diff: &str) -> (usize, usize) {
if let Ok(patch) = diffy::Patch::from_str(diff) {
patch
.hunks()
.iter()
.flat_map(diffy::Hunk::lines)
.fold((0, 0), |(a, d), l| match l {
diffy::Line::Insert(_) => (a + 1, d),
diffy::Line::Delete(_) => (a, d + 1),
_ => (a, d),
})
} else {
let mut a = 0;
let mut d = 0;
for l in diff.lines() {
if l.starts_with("+++") || l.starts_with("---") || l.starts_with("@@") {
continue;
}
match l.as_bytes().first() {
Some(b'+') => a += 1,
Some(b'-') => d += 1,
_ => {}
}
}
(a, d)
}
}

View File

@@ -0,0 +1,36 @@
[package]
name = "codex-cloud-tasks"
version = { workspace = true }
edition = "2024"
[lib]
name = "codex_cloud_tasks"
path = "src/lib.rs"
[lints]
workspace = true
[dependencies]
anyhow = "1"
clap = { version = "4", features = ["derive"] }
codex-common = { path = "../common", features = ["cli"] }
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
tracing = { version = "0.1.41", features = ["log"] }
tracing-subscriber = { version = "0.3.19", features = ["env-filter"] }
codex-cloud-tasks-client = { path = "../cloud-tasks-client", features = ["mock", "online"] }
ratatui = { version = "0.29.0" }
crossterm = { version = "0.28.1", features = ["event-stream"] }
tokio-stream = "0.1.17"
chrono = { version = "0.4", features = ["serde"] }
codex-login = { path = "../login" }
codex-core = { path = "../core" }
throbber-widgets-tui = "0.8.0"
base64 = "0.22"
serde_json = "1"
reqwest = { version = "0.12", features = ["json"] }
serde = { version = "1", features = ["derive"] }
unicode-width = "0.1"
codex-tui = { path = "../tui" }
[dev-dependencies]
async-trait = "0.1"

View File

@@ -0,0 +1,482 @@
use std::time::Duration;
// Environment filter data models for the TUI
#[derive(Clone, Debug, Default)]
pub struct EnvironmentRow {
pub id: String,
pub label: Option<String>,
pub is_pinned: bool,
pub repo_hints: Option<String>, // e.g., "openai/codex"
}
#[derive(Clone, Debug, Default)]
pub struct EnvModalState {
pub query: String,
pub selected: usize,
}
#[derive(Clone, Debug, Default)]
pub struct BestOfModalState {
pub selected: usize,
}
#[derive(Clone, Debug, Copy, PartialEq, Eq)]
pub enum ApplyResultLevel {
Success,
Partial,
Error,
}
#[derive(Clone, Debug)]
pub struct ApplyModalState {
pub task_id: TaskId,
pub title: String,
pub result_message: Option<String>,
pub result_level: Option<ApplyResultLevel>,
pub skipped_paths: Vec<String>,
pub conflict_paths: Vec<String>,
pub diff_override: Option<String>,
}
use crate::scrollable_diff::ScrollableDiff;
use codex_cloud_tasks_client::CloudBackend;
use codex_cloud_tasks_client::TaskId;
use codex_cloud_tasks_client::TaskSummary;
use throbber_widgets_tui::ThrobberState;
#[derive(Default)]
pub struct App {
pub tasks: Vec<TaskSummary>,
pub selected: usize,
pub status: String,
pub diff_overlay: Option<DiffOverlay>,
pub throbber: ThrobberState,
pub refresh_inflight: bool,
pub details_inflight: bool,
// Environment filter state
pub env_filter: Option<String>,
pub env_modal: Option<EnvModalState>,
pub apply_modal: Option<ApplyModalState>,
pub best_of_modal: Option<BestOfModalState>,
pub environments: Vec<EnvironmentRow>,
pub env_last_loaded: Option<std::time::Instant>,
pub env_loading: bool,
pub env_error: Option<String>,
// New Task page
pub new_task: Option<crate::new_task::NewTaskPage>,
pub best_of_n: usize,
// Apply preflight spinner state
pub apply_preflight_inflight: bool,
// Apply action spinner state
pub apply_inflight: bool,
// Background enrichment coordination
pub list_generation: u64,
pub in_flight: std::collections::HashSet<String>,
// Background enrichment caches were planned; currently unused.
}
impl App {
pub fn new() -> Self {
Self {
tasks: Vec::new(),
selected: 0,
status: "Press r to refresh".to_string(),
diff_overlay: None,
throbber: ThrobberState::default(),
refresh_inflight: false,
details_inflight: false,
env_filter: None,
env_modal: None,
apply_modal: None,
best_of_modal: None,
environments: Vec::new(),
env_last_loaded: None,
env_loading: false,
env_error: None,
new_task: None,
best_of_n: 1,
apply_preflight_inflight: false,
apply_inflight: false,
list_generation: 0,
in_flight: std::collections::HashSet::new(),
}
}
pub fn next(&mut self) {
if self.tasks.is_empty() {
return;
}
self.selected = (self.selected + 1).min(self.tasks.len().saturating_sub(1));
}
pub fn prev(&mut self) {
if self.tasks.is_empty() {
return;
}
if self.selected > 0 {
self.selected -= 1;
}
}
}
pub async fn load_tasks(
backend: &dyn CloudBackend,
env: Option<&str>,
) -> anyhow::Result<Vec<TaskSummary>> {
// In later milestones, add a small debounce, spinner, and error display.
let tasks = tokio::time::timeout(Duration::from_secs(5), backend.list_tasks(env)).await??;
// Hide review-only tasks from the main list.
let filtered: Vec<TaskSummary> = tasks.into_iter().filter(|t| !t.is_review).collect();
Ok(filtered)
}
pub struct DiffOverlay {
pub title: String,
pub task_id: TaskId,
pub sd: ScrollableDiff,
pub base_can_apply: bool,
pub diff_lines: Vec<String>,
pub text_lines: Vec<String>,
pub prompt: Option<String>,
pub attempts: Vec<AttemptView>,
pub selected_attempt: usize,
pub current_view: DetailView,
pub base_turn_id: Option<String>,
pub sibling_turn_ids: Vec<String>,
pub attempt_total_hint: Option<usize>,
}
#[derive(Clone, Debug, Default)]
pub struct AttemptView {
pub turn_id: Option<String>,
pub status: codex_cloud_tasks_client::AttemptStatus,
pub attempt_placement: Option<i64>,
pub diff_lines: Vec<String>,
pub text_lines: Vec<String>,
pub prompt: Option<String>,
pub diff_raw: Option<String>,
}
impl AttemptView {
pub fn has_diff(&self) -> bool {
!self.diff_lines.is_empty()
}
pub fn has_text(&self) -> bool {
!self.text_lines.is_empty() || self.prompt.is_some()
}
}
impl DiffOverlay {
pub fn new(task_id: TaskId, title: String, attempt_total_hint: Option<usize>) -> Self {
let mut sd = ScrollableDiff::new();
sd.set_content(Vec::new());
Self {
title,
task_id,
sd,
base_can_apply: false,
diff_lines: Vec::new(),
text_lines: Vec::new(),
prompt: None,
attempts: vec![AttemptView::default()],
selected_attempt: 0,
current_view: DetailView::Prompt,
base_turn_id: None,
sibling_turn_ids: Vec::new(),
attempt_total_hint,
}
}
pub fn current_attempt(&self) -> Option<&AttemptView> {
self.attempts.get(self.selected_attempt)
}
pub fn base_attempt_mut(&mut self) -> &mut AttemptView {
if self.attempts.is_empty() {
self.attempts.push(AttemptView::default());
}
&mut self.attempts[0]
}
pub fn set_view(&mut self, view: DetailView) {
self.current_view = view;
self.apply_selection_to_fields();
}
pub fn expected_attempts(&self) -> Option<usize> {
self.attempt_total_hint.or({
if self.attempts.is_empty() {
None
} else {
Some(self.attempts.len())
}
})
}
pub fn attempt_count(&self) -> usize {
self.attempts.len()
}
pub fn attempt_display_total(&self) -> usize {
self.expected_attempts()
.unwrap_or_else(|| self.attempts.len().max(1))
}
pub fn step_attempt(&mut self, delta: isize) -> bool {
let total = self.attempts.len();
if total <= 1 {
return false;
}
let total_isize = total as isize;
let current = self.selected_attempt as isize;
let mut next = current + delta;
next = ((next % total_isize) + total_isize) % total_isize;
let next = next as usize;
self.selected_attempt = next;
self.apply_selection_to_fields();
true
}
pub fn current_can_apply(&self) -> bool {
matches!(self.current_view, DetailView::Diff)
&& self
.current_attempt()
.and_then(|attempt| attempt.diff_raw.as_ref())
.map(|diff| !diff.is_empty())
.unwrap_or(false)
}
pub fn apply_selection_to_fields(&mut self) {
let (diff_lines, text_lines, prompt) = if let Some(attempt) = self.current_attempt() {
(
attempt.diff_lines.clone(),
attempt.text_lines.clone(),
attempt.prompt.clone(),
)
} else {
self.diff_lines.clear();
self.text_lines.clear();
self.prompt = None;
self.sd.set_content(vec!["<loading attempt>".to_string()]);
return;
};
self.diff_lines = diff_lines.clone();
self.text_lines = text_lines.clone();
self.prompt = prompt;
match self.current_view {
DetailView::Diff => {
if diff_lines.is_empty() {
self.sd.set_content(vec!["<no diff available>".to_string()]);
} else {
self.sd.set_content(diff_lines);
}
}
DetailView::Prompt => {
if text_lines.is_empty() {
self.sd.set_content(vec!["<no output>".to_string()]);
} else {
self.sd.set_content(text_lines);
}
}
}
}
}
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum DetailView {
Diff,
Prompt,
}
/// Internal app events delivered from background tasks.
/// These let the UI event loop remain responsive and keep the spinner animating.
#[derive(Debug)]
pub enum AppEvent {
TasksLoaded {
env: Option<String>,
result: anyhow::Result<Vec<TaskSummary>>,
},
// Background diff summary events were planned; removed for now to keep code minimal.
/// Autodetection of a likely environment id finished
EnvironmentAutodetected(anyhow::Result<crate::env_detect::AutodetectSelection>),
/// Background completion of environment list fetch
EnvironmentsLoaded(anyhow::Result<Vec<EnvironmentRow>>),
DetailsDiffLoaded {
id: TaskId,
title: String,
diff: String,
},
DetailsMessagesLoaded {
id: TaskId,
title: String,
messages: Vec<String>,
prompt: Option<String>,
turn_id: Option<String>,
sibling_turn_ids: Vec<String>,
attempt_placement: Option<i64>,
attempt_status: codex_cloud_tasks_client::AttemptStatus,
},
DetailsFailed {
id: TaskId,
title: String,
error: String,
},
AttemptsLoaded {
id: TaskId,
attempts: Vec<codex_cloud_tasks_client::TurnAttempt>,
},
/// Background completion of new task submission
NewTaskSubmitted(Result<codex_cloud_tasks_client::CreatedTask, String>),
/// Background completion of apply preflight when opening modal or on demand
ApplyPreflightFinished {
id: TaskId,
title: String,
message: String,
level: ApplyResultLevel,
skipped: Vec<String>,
conflicts: Vec<String>,
},
/// Background completion of apply action (actual patch application)
ApplyFinished {
id: TaskId,
result: std::result::Result<codex_cloud_tasks_client::ApplyOutcome, String>,
},
}
// Convenience aliases; currently unused.
#[cfg(test)]
mod tests {
use super::*;
use chrono::Utc;
struct FakeBackend {
// maps env key to titles
by_env: std::collections::HashMap<Option<String>, Vec<&'static str>>,
}
#[async_trait::async_trait]
impl codex_cloud_tasks_client::CloudBackend for FakeBackend {
async fn list_tasks(
&self,
env: Option<&str>,
) -> codex_cloud_tasks_client::Result<Vec<TaskSummary>> {
let key = env.map(str::to_string);
let titles = self
.by_env
.get(&key)
.cloned()
.unwrap_or_else(|| vec!["default-a", "default-b"]);
let mut out = Vec::new();
for (i, t) in titles.into_iter().enumerate() {
out.push(TaskSummary {
id: TaskId(format!("T-{i}")),
title: t.to_string(),
status: codex_cloud_tasks_client::TaskStatus::Ready,
updated_at: Utc::now(),
environment_id: env.map(str::to_string),
environment_label: None,
summary: codex_cloud_tasks_client::DiffSummary::default(),
is_review: false,
attempt_total: Some(1),
});
}
Ok(out)
}
async fn get_task_diff(
&self,
_id: TaskId,
) -> codex_cloud_tasks_client::Result<Option<String>> {
Err(codex_cloud_tasks_client::CloudTaskError::Unimplemented(
"not used in test",
))
}
async fn get_task_messages(
&self,
_id: TaskId,
) -> codex_cloud_tasks_client::Result<Vec<String>> {
Ok(vec![])
}
async fn get_task_text(
&self,
_id: TaskId,
) -> codex_cloud_tasks_client::Result<codex_cloud_tasks_client::TaskText> {
Ok(codex_cloud_tasks_client::TaskText {
prompt: Some("Example prompt".to_string()),
messages: Vec::new(),
turn_id: Some("fake-turn".to_string()),
sibling_turn_ids: Vec::new(),
attempt_placement: Some(0),
attempt_status: codex_cloud_tasks_client::AttemptStatus::Completed,
})
}
async fn list_sibling_attempts(
&self,
_task: TaskId,
_turn_id: String,
) -> codex_cloud_tasks_client::Result<Vec<codex_cloud_tasks_client::TurnAttempt>> {
Ok(Vec::new())
}
async fn apply_task(
&self,
_id: TaskId,
_diff_override: Option<String>,
) -> codex_cloud_tasks_client::Result<codex_cloud_tasks_client::ApplyOutcome> {
Err(codex_cloud_tasks_client::CloudTaskError::Unimplemented(
"not used in test",
))
}
async fn apply_task_preflight(
&self,
_id: TaskId,
_diff_override: Option<String>,
) -> codex_cloud_tasks_client::Result<codex_cloud_tasks_client::ApplyOutcome> {
Err(codex_cloud_tasks_client::CloudTaskError::Unimplemented(
"not used in test",
))
}
async fn create_task(
&self,
_env_id: &str,
_prompt: &str,
_git_ref: &str,
_qa_mode: bool,
_best_of_n: usize,
) -> codex_cloud_tasks_client::Result<codex_cloud_tasks_client::CreatedTask> {
Err(codex_cloud_tasks_client::CloudTaskError::Unimplemented(
"not used in test",
))
}
}
#[tokio::test]
async fn load_tasks_uses_env_parameter() {
// Arrange: env-specific task titles
let mut by_env = std::collections::HashMap::new();
by_env.insert(None, vec!["root-1", "root-2"]);
by_env.insert(Some("env-A".to_string()), vec!["A-1"]);
by_env.insert(Some("env-B".to_string()), vec!["B-1", "B-2", "B-3"]);
let backend = FakeBackend { by_env };
// Act + Assert
let root = load_tasks(&backend, None).await.unwrap();
assert_eq!(root.len(), 2);
assert_eq!(root[0].title, "root-1");
let a = load_tasks(&backend, Some("env-A")).await.unwrap();
assert_eq!(a.len(), 1);
assert_eq!(a[0].title, "A-1");
let b = load_tasks(&backend, Some("env-B")).await.unwrap();
assert_eq!(b.len(), 3);
assert_eq!(b[2].title, "B-3");
}
}

View File

@@ -0,0 +1,9 @@
use clap::Parser;
use codex_common::CliConfigOverrides;
#[derive(Parser, Debug, Default)]
#[command(version)]
pub struct Cli {
#[clap(skip)]
pub config_overrides: CliConfigOverrides,
}

View File

@@ -0,0 +1,361 @@
use reqwest::header::CONTENT_TYPE;
use reqwest::header::HeaderMap;
use std::collections::HashMap;
use tracing::info;
use tracing::warn;
#[derive(Debug, Clone, serde::Deserialize)]
struct CodeEnvironment {
id: String,
#[serde(default)]
label: Option<String>,
#[serde(default)]
is_pinned: Option<bool>,
#[serde(default)]
task_count: Option<i64>,
}
#[derive(Debug, Clone)]
pub struct AutodetectSelection {
pub id: String,
pub label: Option<String>,
}
pub async fn autodetect_environment_id(
base_url: &str,
headers: &HeaderMap,
desired_label: Option<String>,
) -> anyhow::Result<AutodetectSelection> {
// 1) Try repo-specific environments based on local git origins (GitHub only, like VSCode)
let origins = get_git_origins();
crate::append_error_log(format!("env: git origins: {origins:?}"));
let mut by_repo_envs: Vec<CodeEnvironment> = Vec::new();
for origin in &origins {
if let Some((owner, repo)) = parse_owner_repo(origin) {
let url = if base_url.contains("/backend-api") {
format!(
"{}/wham/environments/by-repo/{}/{}/{}",
base_url, "github", owner, repo
)
} else {
format!(
"{}/api/codex/environments/by-repo/{}/{}/{}",
base_url, "github", owner, repo
)
};
crate::append_error_log(format!("env: GET {url}"));
match get_json::<Vec<CodeEnvironment>>(&url, headers).await {
Ok(mut list) => {
crate::append_error_log(format!(
"env: by-repo returned {} env(s) for {owner}/{repo}",
list.len(),
));
by_repo_envs.append(&mut list);
}
Err(e) => crate::append_error_log(format!(
"env: by-repo fetch failed for {owner}/{repo}: {e}"
)),
}
}
}
if let Some(env) = pick_environment_row(&by_repo_envs, desired_label.as_deref()) {
return Ok(AutodetectSelection {
id: env.id.clone(),
label: env.label.as_deref().map(str::to_owned),
});
}
// 2) Fallback to the full list
let list_url = if base_url.contains("/backend-api") {
format!("{base_url}/wham/environments")
} else {
format!("{base_url}/api/codex/environments")
};
crate::append_error_log(format!("env: GET {list_url}"));
// Fetch and log the full environments JSON for debugging
let http = reqwest::Client::builder().build()?;
let res = http.get(&list_url).headers(headers.clone()).send().await?;
let status = res.status();
let ct = res
.headers()
.get(CONTENT_TYPE)
.and_then(|v| v.to_str().ok())
.unwrap_or("")
.to_string();
let body = res.text().await.unwrap_or_default();
crate::append_error_log(format!("env: status={status} content-type={ct}"));
match serde_json::from_str::<serde_json::Value>(&body) {
Ok(v) => {
let pretty = serde_json::to_string_pretty(&v).unwrap_or(body.clone());
crate::append_error_log(format!("env: /environments JSON (pretty):\n{pretty}"));
}
Err(_) => crate::append_error_log(format!("env: /environments (raw):\n{body}")),
}
if !status.is_success() {
anyhow::bail!("GET {list_url} failed: {status}; content-type={ct}; body={body}");
}
let all_envs: Vec<CodeEnvironment> = serde_json::from_str(&body).map_err(|e| {
anyhow::anyhow!("Decode error for {list_url}: {e}; content-type={ct}; body={body}")
})?;
if let Some(env) = pick_environment_row(&all_envs, desired_label.as_deref()) {
return Ok(AutodetectSelection {
id: env.id.clone(),
label: env.label.as_deref().map(str::to_owned),
});
}
anyhow::bail!("no environments available")
}
fn pick_environment_row(
envs: &[CodeEnvironment],
desired_label: Option<&str>,
) -> Option<CodeEnvironment> {
if envs.is_empty() {
return None;
}
if let Some(label) = desired_label {
let lc = label.to_lowercase();
if let Some(e) = envs
.iter()
.find(|e| e.label.as_deref().unwrap_or("").to_lowercase() == lc)
{
crate::append_error_log(format!("env: matched by label: {label} -> {}", e.id));
return Some(e.clone());
}
}
if envs.len() == 1 {
crate::append_error_log("env: single environment available; selecting it");
return Some(envs[0].clone());
}
if let Some(e) = envs.iter().find(|e| e.is_pinned.unwrap_or(false)) {
crate::append_error_log(format!("env: selecting pinned environment: {}", e.id));
return Some(e.clone());
}
// Highest task_count as heuristic
if let Some(e) = envs
.iter()
.max_by_key(|e| e.task_count.unwrap_or(0))
.or_else(|| envs.first())
{
crate::append_error_log(format!("env: selecting by task_count/first: {}", e.id));
return Some(e.clone());
}
None
}
async fn get_json<T: serde::de::DeserializeOwned>(
url: &str,
headers: &HeaderMap,
) -> anyhow::Result<T> {
let http = reqwest::Client::builder().build()?;
let res = http.get(url).headers(headers.clone()).send().await?;
let status = res.status();
let ct = res
.headers()
.get(CONTENT_TYPE)
.and_then(|v| v.to_str().ok())
.unwrap_or("")
.to_string();
let body = res.text().await.unwrap_or_default();
crate::append_error_log(format!("env: status={status} content-type={ct}"));
if !status.is_success() {
anyhow::bail!("GET {url} failed: {status}; content-type={ct}; body={body}");
}
let parsed = serde_json::from_str::<T>(&body).map_err(|e| {
anyhow::anyhow!("Decode error for {url}: {e}; content-type={ct}; body={body}")
})?;
Ok(parsed)
}
fn get_git_origins() -> Vec<String> {
// Prefer: git config --get-regexp remote\..*\.url
let out = std::process::Command::new("git")
.args(["config", "--get-regexp", "remote\\..*\\.url"])
.output();
if let Ok(ok) = out
&& ok.status.success()
{
let s = String::from_utf8_lossy(&ok.stdout);
let mut urls = Vec::new();
for line in s.lines() {
if let Some((_, url)) = line.split_once(' ') {
urls.push(url.trim().to_string());
}
}
if !urls.is_empty() {
return uniq(urls);
}
}
// Fallback: git remote -v
let out = std::process::Command::new("git")
.args(["remote", "-v"])
.output();
if let Ok(ok) = out
&& ok.status.success()
{
let s = String::from_utf8_lossy(&ok.stdout);
let mut urls = Vec::new();
for line in s.lines() {
let parts: Vec<&str> = line.split_whitespace().collect();
if parts.len() >= 2 {
urls.push(parts[1].to_string());
}
}
if !urls.is_empty() {
return uniq(urls);
}
}
Vec::new()
}
fn uniq(mut v: Vec<String>) -> Vec<String> {
v.sort();
v.dedup();
v
}
fn parse_owner_repo(url: &str) -> Option<(String, String)> {
// Normalize common prefixes and handle multiple SSH/HTTPS variants.
let mut s = url.trim().to_string();
// Drop protocol scheme for ssh URLs
if let Some(rest) = s.strip_prefix("ssh://") {
s = rest.to_string();
}
// Accept any user before @github.com (e.g., git@, org-123@)
if let Some(idx) = s.find("@github.com:") {
let rest = &s[idx + "@github.com:".len()..];
let rest = rest.trim_start_matches('/').trim_end_matches(".git");
let mut parts = rest.splitn(2, '/');
let owner = parts.next()?.to_string();
let repo = parts.next()?.to_string();
crate::append_error_log(format!("env: parsed SSH GitHub origin => {owner}/{repo}"));
return Some((owner, repo));
}
// HTTPS or git protocol
for prefix in [
"https://github.com/",
"http://github.com/",
"git://github.com/",
"github.com/",
] {
if let Some(rest) = s.strip_prefix(prefix) {
let rest = rest.trim_start_matches('/').trim_end_matches(".git");
let mut parts = rest.splitn(2, '/');
let owner = parts.next()?.to_string();
let repo = parts.next()?.to_string();
crate::append_error_log(format!("env: parsed HTTP GitHub origin => {owner}/{repo}"));
return Some((owner, repo));
}
}
None
}
/// List environments for the current repo(s) with a fallback to the global list.
/// Returns a de-duplicated, sorted set suitable for the TUI modal.
pub async fn list_environments(
base_url: &str,
headers: &HeaderMap,
) -> anyhow::Result<Vec<crate::app::EnvironmentRow>> {
let mut map: HashMap<String, crate::app::EnvironmentRow> = HashMap::new();
// 1) By-repo lookup for each parsed GitHub origin
let origins = get_git_origins();
for origin in &origins {
if let Some((owner, repo)) = parse_owner_repo(origin) {
let url = if base_url.contains("/backend-api") {
format!(
"{}/wham/environments/by-repo/{}/{}/{}",
base_url, "github", owner, repo
)
} else {
format!(
"{}/api/codex/environments/by-repo/{}/{}/{}",
base_url, "github", owner, repo
)
};
match get_json::<Vec<CodeEnvironment>>(&url, headers).await {
Ok(list) => {
info!("env_tui: by-repo {}:{} -> {} envs", owner, repo, list.len());
for e in list {
let entry =
map.entry(e.id.clone())
.or_insert_with(|| crate::app::EnvironmentRow {
id: e.id.clone(),
label: e.label.clone(),
is_pinned: e.is_pinned.unwrap_or(false),
repo_hints: Some(format!("{owner}/{repo}")),
});
// Merge: keep label if present, or use new; accumulate pinned flag
if entry.label.is_none() {
entry.label = e.label.clone();
}
entry.is_pinned = entry.is_pinned || e.is_pinned.unwrap_or(false);
if entry.repo_hints.is_none() {
entry.repo_hints = Some(format!("{owner}/{repo}"));
}
}
}
Err(e) => {
warn!(
"env_tui: by-repo fetch failed for {}/{}: {}",
owner, repo, e
);
}
}
}
}
// 2) Fallback to the full list; on error return what we have if any.
let list_url = if base_url.contains("/backend-api") {
format!("{base_url}/wham/environments")
} else {
format!("{base_url}/api/codex/environments")
};
match get_json::<Vec<CodeEnvironment>>(&list_url, headers).await {
Ok(list) => {
info!("env_tui: global list -> {} envs", list.len());
for e in list {
let entry = map
.entry(e.id.clone())
.or_insert_with(|| crate::app::EnvironmentRow {
id: e.id.clone(),
label: e.label.clone(),
is_pinned: e.is_pinned.unwrap_or(false),
repo_hints: None,
});
if entry.label.is_none() {
entry.label = e.label.clone();
}
entry.is_pinned = entry.is_pinned || e.is_pinned.unwrap_or(false);
}
}
Err(e) => {
if map.is_empty() {
return Err(e);
} else {
warn!(
"env_tui: global list failed; using by-repo results only: {}",
e
);
}
}
}
let mut rows: Vec<crate::app::EnvironmentRow> = map.into_values().collect();
rows.sort_by(|a, b| {
// pinned first
let p = b.is_pinned.cmp(&a.is_pinned);
if p != std::cmp::Ordering::Equal {
return p;
}
// then label (ci), then id
let al = a.label.as_deref().unwrap_or("").to_lowercase();
let bl = b.label.as_deref().unwrap_or("").to_lowercase();
let l = al.cmp(&bl);
if l != std::cmp::Ordering::Equal {
return l;
}
a.id.cmp(&b.id)
});
Ok(rows)
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,35 @@
use codex_tui::ComposerInput;
pub struct NewTaskPage {
pub composer: ComposerInput,
pub submitting: bool,
pub env_id: Option<String>,
pub best_of_n: usize,
}
impl NewTaskPage {
pub fn new(env_id: Option<String>, best_of_n: usize) -> Self {
let mut composer = ComposerInput::new();
composer.set_hint_items(vec![
("", "send"),
("Shift+⏎", "newline"),
("Ctrl+O", "env"),
("Ctrl+N", "attempts"),
("Ctrl+C", "quit"),
]);
Self {
composer,
submitting: false,
env_id,
best_of_n,
}
}
// Additional helpers can be added as usage evolves.
}
impl Default for NewTaskPage {
fn default() -> Self {
Self::new(None, 1)
}
}

View File

@@ -0,0 +1,176 @@
use unicode_width::UnicodeWidthChar;
use unicode_width::UnicodeWidthStr;
/// Scroll position and geometry for a vertical scroll view.
#[derive(Clone, Copy, Debug, Default)]
pub struct ScrollViewState {
pub scroll: u16,
pub viewport_h: u16,
pub content_h: u16,
}
impl ScrollViewState {
pub fn clamp(&mut self) {
let max_scroll = self.content_h.saturating_sub(self.viewport_h);
if self.scroll > max_scroll {
self.scroll = max_scroll;
}
}
}
/// A simple, local scrollable view for diffs or message text.
///
/// Owns raw lines, caches wrapped lines for a given width, and maintains
/// a small scroll state that is clamped whenever geometry shrinks.
#[derive(Clone, Debug, Default)]
pub struct ScrollableDiff {
raw: Vec<String>,
wrapped: Vec<String>,
wrapped_src_idx: Vec<usize>,
wrap_cols: Option<u16>,
pub state: ScrollViewState,
}
impl ScrollableDiff {
pub fn new() -> Self {
Self::default()
}
/// Replace the raw content lines. Does not rewrap immediately; call `set_width` next.
pub fn set_content(&mut self, lines: Vec<String>) {
self.raw = lines;
self.wrapped.clear();
self.wrapped_src_idx.clear();
self.state.content_h = 0;
// Force rewrap on next set_width even if width is unchanged
self.wrap_cols = None;
}
/// Set the wrap width. If changed, rebuild wrapped lines and clamp scroll.
pub fn set_width(&mut self, width: u16) {
if self.wrap_cols == Some(width) {
return;
}
self.wrap_cols = Some(width);
self.rewrap(width);
self.state.clamp();
}
/// Update viewport height and clamp scroll if needed.
pub fn set_viewport(&mut self, height: u16) {
self.state.viewport_h = height;
self.state.clamp();
}
/// Return the cached wrapped lines. Call `set_width` first when area changes.
pub fn wrapped_lines(&self) -> &[String] {
&self.wrapped
}
pub fn wrapped_src_indices(&self) -> &[usize] {
&self.wrapped_src_idx
}
pub fn raw_line_at(&self, idx: usize) -> &str {
self.raw.get(idx).map(String::as_str).unwrap_or("")
}
/// Scroll by a signed delta; clamps to content.
pub fn scroll_by(&mut self, delta: i16) {
let s = self.state.scroll as i32 + delta as i32;
self.state.scroll = s.clamp(0, self.max_scroll() as i32) as u16;
}
/// Page by a signed delta; typically viewport_h - 1.
pub fn page_by(&mut self, delta: i16) {
self.scroll_by(delta);
}
pub fn to_top(&mut self) {
self.state.scroll = 0;
}
pub fn to_bottom(&mut self) {
self.state.scroll = self.max_scroll();
}
/// Optional percent scrolled; None when not enough geometry is known.
pub fn percent_scrolled(&self) -> Option<u8> {
if self.state.content_h == 0 || self.state.viewport_h == 0 {
return None;
}
if self.state.content_h <= self.state.viewport_h {
return None;
}
let visible_bottom = self.state.scroll.saturating_add(self.state.viewport_h) as f32;
let pct = (visible_bottom / self.state.content_h as f32 * 100.0).round();
Some(pct.clamp(0.0, 100.0) as u8)
}
fn max_scroll(&self) -> u16 {
self.state.content_h.saturating_sub(self.state.viewport_h)
}
fn rewrap(&mut self, width: u16) {
if width == 0 {
self.wrapped = self.raw.clone();
self.state.content_h = self.wrapped.len() as u16;
return;
}
let max_cols = width as usize;
let mut out: Vec<String> = Vec::new();
let mut out_idx: Vec<usize> = Vec::new();
for (raw_idx, raw) in self.raw.iter().enumerate() {
// Normalize tabs for width accounting (MVP: 4 spaces).
let raw = raw.replace('\t', " ");
if raw.is_empty() {
out.push(String::new());
out_idx.push(raw_idx);
continue;
}
let mut line = String::new();
let mut line_cols = 0usize;
let mut last_soft_idx: Option<usize> = None; // last whitespace or punctuation break
for (_i, ch) in raw.char_indices() {
if ch == '\n' {
out.push(std::mem::take(&mut line));
out_idx.push(raw_idx);
line_cols = 0;
last_soft_idx = None;
continue;
}
let w = UnicodeWidthChar::width(ch).unwrap_or(0);
if line_cols.saturating_add(w) > max_cols {
if let Some(split) = last_soft_idx {
let (prefix, rest) = line.split_at(split);
out.push(prefix.trim_end().to_string());
out_idx.push(raw_idx);
line = rest.trim_start().to_string();
last_soft_idx = None;
// retry add current ch now that line may be shorter
} else if !line.is_empty() {
out.push(std::mem::take(&mut line));
out_idx.push(raw_idx);
}
}
if ch.is_whitespace()
|| matches!(
ch,
',' | ';' | '.' | ':' | ')' | ']' | '}' | '|' | '/' | '?' | '!' | '-' | '_'
)
{
last_soft_idx = Some(line.len());
}
line.push(ch);
line_cols = UnicodeWidthStr::width(line.as_str());
}
if !line.is_empty() {
out.push(line);
out_idx.push(raw_idx);
}
}
self.wrapped = out;
self.wrapped_src_idx = out_idx;
self.state.content_h = self.wrapped.len() as u16;
}
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,93 @@
use base64::Engine as _;
use chrono::Utc;
use reqwest::header::HeaderMap;
pub fn set_user_agent_suffix(suffix: &str) {
if let Ok(mut guard) = codex_core::default_client::USER_AGENT_SUFFIX.lock() {
guard.replace(suffix.to_string());
}
}
pub fn append_error_log(message: impl AsRef<str>) {
let ts = Utc::now().to_rfc3339();
if let Ok(mut f) = std::fs::OpenOptions::new()
.create(true)
.append(true)
.open("error.log")
{
use std::io::Write as _;
let _ = writeln!(f, "[{ts}] {}", message.as_ref());
}
}
/// Normalize the configured base URL to a canonical form used by the backend client.
/// - trims trailing '/'
/// - appends '/backend-api' for ChatGPT hosts when missing
pub fn normalize_base_url(input: &str) -> String {
let mut base_url = input.to_string();
while base_url.ends_with('/') {
base_url.pop();
}
if (base_url.starts_with("https://chatgpt.com")
|| base_url.starts_with("https://chat.openai.com"))
&& !base_url.contains("/backend-api")
{
base_url = format!("{base_url}/backend-api");
}
base_url
}
/// Extract the ChatGPT account id from a JWT token, when present.
pub fn extract_chatgpt_account_id(token: &str) -> Option<String> {
let mut parts = token.split('.');
let (_h, payload_b64, _s) = match (parts.next(), parts.next(), parts.next()) {
(Some(h), Some(p), Some(s)) if !h.is_empty() && !p.is_empty() && !s.is_empty() => (h, p, s),
_ => return None,
};
let payload_bytes = base64::engine::general_purpose::URL_SAFE_NO_PAD
.decode(payload_b64)
.ok()?;
let v: serde_json::Value = serde_json::from_slice(&payload_bytes).ok()?;
v.get("https://api.openai.com/auth")
.and_then(|auth| auth.get("chatgpt_account_id"))
.and_then(|id| id.as_str())
.map(str::to_string)
}
/// Build headers for ChatGPT-backed requests: `User-Agent`, optional `Authorization`,
/// and optional `ChatGPT-Account-Id`.
pub async fn build_chatgpt_headers() -> HeaderMap {
use reqwest::header::AUTHORIZATION;
use reqwest::header::HeaderName;
use reqwest::header::HeaderValue;
use reqwest::header::USER_AGENT;
set_user_agent_suffix("codex_cloud_tasks_tui");
let ua = codex_core::default_client::get_codex_user_agent();
let mut headers = HeaderMap::new();
headers.insert(
USER_AGENT,
HeaderValue::from_str(&ua).unwrap_or(HeaderValue::from_static("codex-cli")),
);
if let Ok(home) = codex_core::config::find_codex_home() {
let am = codex_login::AuthManager::new(home);
if let Some(auth) = am.auth()
&& let Ok(tok) = auth.get_token().await
&& !tok.is_empty()
{
let v = format!("Bearer {tok}");
if let Ok(hv) = HeaderValue::from_str(&v) {
headers.insert(AUTHORIZATION, hv);
}
if let Some(acc) = auth
.get_account_id()
.or_else(|| extract_chatgpt_account_id(&tok))
&& let Ok(name) = HeaderName::from_bytes(b"ChatGPT-Account-Id")
&& let Ok(hv) = HeaderValue::from_str(&acc)
{
headers.insert(name, hv);
}
}
}
headers
}

View File

@@ -0,0 +1,22 @@
use codex_cloud_tasks_client::CloudBackend;
use codex_cloud_tasks_client::MockClient;
#[tokio::test]
async fn mock_backend_varies_by_env() {
let client = MockClient;
let root = CloudBackend::list_tasks(&client, None).await.unwrap();
assert!(root.iter().any(|t| t.title.contains("Update README")));
let a = CloudBackend::list_tasks(&client, Some("env-A"))
.await
.unwrap();
assert_eq!(a.len(), 1);
assert_eq!(a[0].title, "A: First");
let b = CloudBackend::list_tasks(&client, Some("env-B"))
.await
.unwrap();
assert_eq!(b.len(), 2);
assert!(b[0].title.starts_with("B: "));
}

View File

@@ -0,0 +1,17 @@
[package]
name = "codex-backend-openapi-models"
version = { workspace = true }
edition = "2024"
[lib]
name = "codex_backend_openapi_models"
path = "src/lib.rs"
# Important: generated code often violates our workspace lints.
# Allow unwrap/expect in this crate so the workspace builds cleanly
# after models are regenerated.
# Lint overrides are applied in src/lib.rs via crate attributes
[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"

View File

@@ -0,0 +1,6 @@
#![allow(clippy::unwrap_used, clippy::expect_used)]
// Re-export generated OpenAPI models.
// The regen script populates `src/models/*.rs` and writes `src/models/mod.rs`.
// This module intentionally contains no hand-written types.
pub mod models;

View File

@@ -0,0 +1,42 @@
/*
* codex-backend
*
* codex-backend
*
* The version of the OpenAPI document: 0.0.1
*
* Generated by: https://openapi-generator.tech
*/
use crate::models;
use serde::Deserialize;
use serde::Serialize;
#[derive(Clone, Default, Debug, PartialEq, Serialize, Deserialize)]
pub struct CodeTaskDetailsResponse {
#[serde(rename = "task")]
pub task: Box<models::TaskResponse>,
#[serde(rename = "current_user_turn", skip_serializing_if = "Option::is_none")]
pub current_user_turn: Option<std::collections::HashMap<String, serde_json::Value>>,
#[serde(
rename = "current_assistant_turn",
skip_serializing_if = "Option::is_none"
)]
pub current_assistant_turn: Option<std::collections::HashMap<String, serde_json::Value>>,
#[serde(
rename = "current_diff_task_turn",
skip_serializing_if = "Option::is_none"
)]
pub current_diff_task_turn: Option<std::collections::HashMap<String, serde_json::Value>>,
}
impl CodeTaskDetailsResponse {
pub fn new(task: models::TaskResponse) -> CodeTaskDetailsResponse {
CodeTaskDetailsResponse {
task: Box::new(task),
current_user_turn: None,
current_assistant_turn: None,
current_diff_task_turn: None,
}
}
}

View File

@@ -0,0 +1,40 @@
/*
* codex-backend
*
* codex-backend
*
* The version of the OpenAPI document: 0.0.1
*
* Generated by: https://openapi-generator.tech
*/
use crate::models;
use serde::Deserialize;
use serde::Serialize;
#[derive(Clone, Default, Debug, PartialEq, Serialize, Deserialize)]
pub struct ExternalPullRequestResponse {
#[serde(rename = "id")]
pub id: String,
#[serde(rename = "assistant_turn_id")]
pub assistant_turn_id: String,
#[serde(rename = "pull_request")]
pub pull_request: Box<models::GitPullRequest>,
#[serde(rename = "codex_updated_sha", skip_serializing_if = "Option::is_none")]
pub codex_updated_sha: Option<String>,
}
impl ExternalPullRequestResponse {
pub fn new(
id: String,
assistant_turn_id: String,
pull_request: models::GitPullRequest,
) -> ExternalPullRequestResponse {
ExternalPullRequestResponse {
id,
assistant_turn_id,
pull_request: Box::new(pull_request),
codex_updated_sha: None,
}
}
}

View File

@@ -0,0 +1,77 @@
/*
* codex-backend
*
* codex-backend
*
* The version of the OpenAPI document: 0.0.1
*
* Generated by: https://openapi-generator.tech
*/
use serde::Deserialize;
use serde::Serialize;
#[derive(Clone, Default, Debug, PartialEq, Serialize, Deserialize)]
pub struct GitPullRequest {
#[serde(rename = "number")]
pub number: i32,
#[serde(rename = "url")]
pub url: String,
#[serde(rename = "state")]
pub state: String,
#[serde(rename = "merged")]
pub merged: bool,
#[serde(rename = "mergeable")]
pub mergeable: bool,
#[serde(rename = "draft", skip_serializing_if = "Option::is_none")]
pub draft: Option<bool>,
#[serde(rename = "title", skip_serializing_if = "Option::is_none")]
pub title: Option<String>,
#[serde(rename = "body", skip_serializing_if = "Option::is_none")]
pub body: Option<String>,
#[serde(rename = "base", skip_serializing_if = "Option::is_none")]
pub base: Option<String>,
#[serde(rename = "head", skip_serializing_if = "Option::is_none")]
pub head: Option<String>,
#[serde(rename = "base_sha", skip_serializing_if = "Option::is_none")]
pub base_sha: Option<String>,
#[serde(rename = "head_sha", skip_serializing_if = "Option::is_none")]
pub head_sha: Option<String>,
#[serde(rename = "merge_commit_sha", skip_serializing_if = "Option::is_none")]
pub merge_commit_sha: Option<String>,
#[serde(rename = "comments", skip_serializing_if = "Option::is_none")]
pub comments: Option<serde_json::Value>,
#[serde(rename = "diff", skip_serializing_if = "Option::is_none")]
pub diff: Option<serde_json::Value>,
#[serde(rename = "user", skip_serializing_if = "Option::is_none")]
pub user: Option<serde_json::Value>,
}
impl GitPullRequest {
pub fn new(
number: i32,
url: String,
state: String,
merged: bool,
mergeable: bool,
) -> GitPullRequest {
GitPullRequest {
number,
url,
state,
merged,
mergeable,
draft: None,
title: None,
body: None,
base: None,
head: None,
base_sha: None,
head_sha: None,
merge_commit_sha: None,
comments: None,
diff: None,
user: None,
}
}
}

View File

@@ -0,0 +1,22 @@
// Curated minimal export list for current workspace usage.
// NOTE: This file was previously auto-generated by the OpenAPI generator.
// Currently export only the types referenced by the workspace
// The process for this will change
pub mod code_task_details_response;
pub use self::code_task_details_response::CodeTaskDetailsResponse;
pub mod task_response;
pub use self::task_response::TaskResponse;
pub mod external_pull_request_response;
pub use self::external_pull_request_response::ExternalPullRequestResponse;
pub mod git_pull_request;
pub use self::git_pull_request::GitPullRequest;
pub mod task_list_item;
pub use self::task_list_item::TaskListItem;
pub mod paginated_list_task_list_item_;
pub use self::paginated_list_task_list_item_::PaginatedListTaskListItem;

View File

@@ -0,0 +1,30 @@
/*
* codex-backend
*
* codex-backend
*
* The version of the OpenAPI document: 0.0.1
*
* Generated by: https://openapi-generator.tech
*/
use crate::models;
use serde::Deserialize;
use serde::Serialize;
#[derive(Clone, Default, Debug, PartialEq, Serialize, Deserialize)]
pub struct PaginatedListTaskListItem {
#[serde(rename = "items")]
pub items: Vec<models::TaskListItem>,
#[serde(rename = "cursor", skip_serializing_if = "Option::is_none")]
pub cursor: Option<String>,
}
impl PaginatedListTaskListItem {
pub fn new(items: Vec<models::TaskListItem>) -> PaginatedListTaskListItem {
PaginatedListTaskListItem {
items,
cursor: None,
}
}
}

View File

@@ -0,0 +1,63 @@
/*
* codex-backend
*
* codex-backend
*
* The version of the OpenAPI document: 0.0.1
*
* Generated by: https://openapi-generator.tech
*/
use crate::models;
use serde::Deserialize;
use serde::Serialize;
#[derive(Clone, Default, Debug, PartialEq, Serialize, Deserialize)]
pub struct TaskListItem {
#[serde(rename = "id")]
pub id: String,
#[serde(rename = "title")]
pub title: String,
#[serde(
rename = "has_generated_title",
skip_serializing_if = "Option::is_none"
)]
pub has_generated_title: Option<bool>,
#[serde(rename = "updated_at", skip_serializing_if = "Option::is_none")]
pub updated_at: Option<f64>,
#[serde(rename = "created_at", skip_serializing_if = "Option::is_none")]
pub created_at: Option<f64>,
#[serde(
rename = "task_status_display",
skip_serializing_if = "Option::is_none"
)]
pub task_status_display: Option<std::collections::HashMap<String, serde_json::Value>>,
#[serde(rename = "archived")]
pub archived: bool,
#[serde(rename = "has_unread_turn")]
pub has_unread_turn: bool,
#[serde(rename = "pull_requests", skip_serializing_if = "Option::is_none")]
pub pull_requests: Option<Vec<models::ExternalPullRequestResponse>>,
}
impl TaskListItem {
pub fn new(
id: String,
title: String,
has_generated_title: Option<bool>,
archived: bool,
has_unread_turn: bool,
) -> TaskListItem {
TaskListItem {
id,
title,
has_generated_title,
updated_at: None,
created_at: None,
task_status_display: None,
archived,
has_unread_turn,
pull_requests: None,
}
}
}

View File

@@ -0,0 +1,62 @@
/*
* codex-backend
*
* codex-backend
*
* The version of the OpenAPI document: 0.0.1
*
* Generated by: https://openapi-generator.tech
*/
use crate::models;
use serde::Deserialize;
use serde::Serialize;
#[derive(Clone, Default, Debug, PartialEq, Serialize, Deserialize)]
pub struct TaskResponse {
#[serde(rename = "id")]
pub id: String,
#[serde(rename = "created_at", skip_serializing_if = "Option::is_none")]
pub created_at: Option<f64>,
#[serde(rename = "title")]
pub title: String,
#[serde(
rename = "has_generated_title",
skip_serializing_if = "Option::is_none"
)]
pub has_generated_title: Option<bool>,
#[serde(rename = "current_turn_id", skip_serializing_if = "Option::is_none")]
pub current_turn_id: Option<String>,
#[serde(rename = "has_unread_turn", skip_serializing_if = "Option::is_none")]
pub has_unread_turn: Option<bool>,
#[serde(
rename = "denormalized_metadata",
skip_serializing_if = "Option::is_none"
)]
pub denormalized_metadata: Option<std::collections::HashMap<String, serde_json::Value>>,
#[serde(rename = "archived")]
pub archived: bool,
#[serde(rename = "external_pull_requests")]
pub external_pull_requests: Vec<models::ExternalPullRequestResponse>,
}
impl TaskResponse {
pub fn new(
id: String,
title: String,
archived: bool,
external_pull_requests: Vec<models::ExternalPullRequestResponse>,
) -> TaskResponse {
TaskResponse {
id,
created_at: None,
title,
has_generated_title: None,
current_turn_id: None,
has_unread_turn: None,
denormalized_metadata: None,
archived,
external_pull_requests,
}
}
}

View File

@@ -10,6 +10,7 @@ workspace = true
clap = { workspace = true, features = ["derive", "wrap_help"], optional = true }
codex-core = { workspace = true }
codex-protocol = { workspace = true }
codex-app-server-protocol = { workspace = true }
serde = { workspace = true, optional = true }
toml = { workspace = true, optional = true }

View File

@@ -1,5 +1,5 @@
use codex_app_server_protocol::AuthMode;
use codex_core::protocol_config_types::ReasoningEffort;
use codex_protocol::mcp_protocol::AuthMode;
/// A simple preset pairing a model slug with a reasoning effort.
#[derive(Debug, Clone, Copy)]

View File

@@ -24,7 +24,10 @@ codex-file-search = { workspace = true }
codex-mcp-client = { workspace = true }
codex-rmcp-client = { workspace = true }
codex-protocol = { workspace = true }
codex-app-server-protocol = { workspace = true }
codex-otel = { workspace = true, features = ["otel"] }
dirs = { workspace = true }
dunce = { workspace = true }
env-flags = { workspace = true }
eventsource-stream = { workspace = true }
futures = { workspace = true }
@@ -91,6 +94,7 @@ tempfile = { workspace = true }
tokio-test = { workspace = true }
walkdir = { workspace = true }
wiremock = { workspace = true }
tracing-test = { workspace = true, features = ["no-env-filter"] }
[package.metadata.cargo-shear]
ignored = ["openssl-sys"]

View File

@@ -27,6 +27,7 @@ pub(crate) enum InternalApplyPatchInvocation {
DelegateToExec(ApplyPatchExec),
}
#[derive(Debug)]
pub(crate) struct ApplyPatchExec {
pub(crate) action: ApplyPatchAction,
pub(crate) user_explicitly_approved_this_action: bool,
@@ -45,12 +46,13 @@ pub(crate) async fn apply_patch(
&turn_context.sandbox_policy,
&turn_context.cwd,
) {
SafetyCheck::AutoApprove { .. } => {
InternalApplyPatchInvocation::DelegateToExec(ApplyPatchExec {
action,
user_explicitly_approved_this_action: false,
})
}
SafetyCheck::AutoApprove {
user_explicitly_approved,
..
} => InternalApplyPatchInvocation::DelegateToExec(ApplyPatchExec {
action,
user_explicitly_approved_this_action: user_explicitly_approved,
}),
SafetyCheck::AskUser => {
// Compute a readable summary of path changes to include in the
// approval request so the user can make an informed decision.
@@ -108,3 +110,28 @@ pub(crate) fn convert_apply_patch_to_protocol(
}
result
}
#[cfg(test)]
mod tests {
use super::*;
use pretty_assertions::assert_eq;
use tempfile::tempdir;
#[test]
fn convert_apply_patch_maps_add_variant() {
let tmp = tempdir().expect("tmp");
let p = tmp.path().join("a.txt");
// Create an action with a single Add change
let action = ApplyPatchAction::new_add_for_test(&p, "hello".to_string());
let got = convert_apply_patch_to_protocol(&action);
assert_eq!(
got.get(&p),
Some(&FileChange::Add {
content: "hello".to_string()
})
);
}
}

View File

@@ -15,7 +15,7 @@ use std::sync::Arc;
use std::sync::Mutex;
use std::time::Duration;
use codex_protocol::mcp_protocol::AuthMode;
use codex_app_server_protocol::AuthMode;
use crate::token_data::PlanType;
use crate::token_data::TokenData;

View File

@@ -1,6 +1,21 @@
use std::time::Duration;
use crate::ModelProviderInfo;
use crate::client_common::Prompt;
use crate::client_common::ResponseEvent;
use crate::client_common::ResponseStream;
use crate::error::CodexErr;
use crate::error::Result;
use crate::error::RetryLimitReachedError;
use crate::error::UnexpectedResponseError;
use crate::model_family::ModelFamily;
use crate::openai_tools::create_tools_json_for_chat_completions_api;
use crate::util::backoff;
use bytes::Bytes;
use codex_otel::otel_event_manager::OtelEventManager;
use codex_protocol::models::ContentItem;
use codex_protocol::models::ReasoningItemContent;
use codex_protocol::models::ResponseItem;
use eventsource_stream::Eventsource;
use futures::Stream;
use futures::StreamExt;
@@ -15,25 +30,13 @@ use tokio::time::timeout;
use tracing::debug;
use tracing::trace;
use crate::ModelProviderInfo;
use crate::client_common::Prompt;
use crate::client_common::ResponseEvent;
use crate::client_common::ResponseStream;
use crate::error::CodexErr;
use crate::error::Result;
use crate::model_family::ModelFamily;
use crate::openai_tools::create_tools_json_for_chat_completions_api;
use crate::util::backoff;
use codex_protocol::models::ContentItem;
use codex_protocol::models::ReasoningItemContent;
use codex_protocol::models::ResponseItem;
/// Implementation for the classic Chat Completions API.
pub(crate) async fn stream_chat_completions(
prompt: &Prompt,
model_family: &ModelFamily,
client: &reqwest::Client,
provider: &ModelProviderInfo,
otel_event_manager: &OtelEventManager,
) -> Result<ResponseStream> {
if prompt.output_schema.is_some() {
return Err(CodexErr::UnsupportedOperation(
@@ -294,10 +297,13 @@ pub(crate) async fn stream_chat_completions(
let req_builder = provider.create_request_builder(client, &None).await?;
let res = req_builder
.header(reqwest::header::ACCEPT, "text/event-stream")
.json(&payload)
.send()
let res = otel_event_manager
.log_request(attempt, || {
req_builder
.header(reqwest::header::ACCEPT, "text/event-stream")
.json(&payload)
.send()
})
.await;
match res {
@@ -308,6 +314,7 @@ pub(crate) async fn stream_chat_completions(
stream,
tx_event,
provider.stream_idle_timeout(),
otel_event_manager.clone(),
));
return Ok(ResponseStream { rx_event });
}
@@ -315,11 +322,18 @@ pub(crate) async fn stream_chat_completions(
let status = res.status();
if !(status == StatusCode::TOO_MANY_REQUESTS || status.is_server_error()) {
let body = (res.text().await).unwrap_or_default();
return Err(CodexErr::UnexpectedStatus(status, body));
return Err(CodexErr::UnexpectedStatus(UnexpectedResponseError {
status,
body,
request_id: None,
}));
}
if attempt > max_retries {
return Err(CodexErr::RetryLimit(status));
return Err(CodexErr::RetryLimit(RetryLimitReachedError {
status,
request_id: None,
}));
}
let retry_after_secs = res
@@ -351,6 +365,7 @@ async fn process_chat_sse<S>(
stream: S,
tx_event: mpsc::Sender<Result<ResponseEvent>>,
idle_timeout: Duration,
otel_event_manager: OtelEventManager,
) where
S: Stream<Item = Result<Bytes>> + Unpin,
{
@@ -374,7 +389,10 @@ async fn process_chat_sse<S>(
let mut reasoning_text = String::new();
loop {
let sse = match timeout(idle_timeout, stream.next()).await {
let sse = match otel_event_manager
.log_sse_event(|| timeout(idle_timeout, stream.next()))
.await
{
Ok(Some(Ok(ev))) => ev,
Ok(Some(Err(e))) => {
let _ = tx_event

View File

@@ -5,9 +5,11 @@ use std::time::Duration;
use crate::AuthManager;
use crate::auth::CodexAuth;
use crate::error::RetryLimitReachedError;
use crate::error::UnexpectedResponseError;
use bytes::Bytes;
use codex_protocol::mcp_protocol::AuthMode;
use codex_protocol::mcp_protocol::ConversationId;
use codex_app_server_protocol::AuthMode;
use codex_protocol::ConversationId;
use eventsource_stream::Eventsource;
use futures::prelude::*;
use regex_lite::Regex;
@@ -47,6 +49,7 @@ use crate::protocol::RateLimitWindow;
use crate::protocol::TokenUsage;
use crate::token_data::PlanType;
use crate::util::backoff;
use codex_otel::otel_event_manager::OtelEventManager;
use codex_protocol::config_types::ReasoningEffort as ReasoningEffortConfig;
use codex_protocol::config_types::ReasoningSummary as ReasoningSummaryConfig;
use codex_protocol::models::ResponseItem;
@@ -73,6 +76,7 @@ struct Error {
pub struct ModelClient {
config: Arc<Config>,
auth_manager: Option<Arc<AuthManager>>,
otel_event_manager: OtelEventManager,
client: reqwest::Client,
provider: ModelProviderInfo,
conversation_id: ConversationId,
@@ -84,6 +88,7 @@ impl ModelClient {
pub fn new(
config: Arc<Config>,
auth_manager: Option<Arc<AuthManager>>,
otel_event_manager: OtelEventManager,
provider: ModelProviderInfo,
effort: Option<ReasoningEffortConfig>,
summary: ReasoningSummaryConfig,
@@ -94,6 +99,7 @@ impl ModelClient {
Self {
config,
auth_manager,
otel_event_manager,
client,
provider,
conversation_id,
@@ -127,6 +133,7 @@ impl ModelClient {
&self.config.model_family,
&self.client,
&self.provider,
&self.otel_event_manager,
)
.await?;
@@ -163,7 +170,12 @@ impl ModelClient {
if let Some(path) = &*CODEX_RS_SSE_FIXTURE {
// short circuit for tests
warn!(path, "Streaming from fixture");
return stream_from_fixture(path, self.provider.clone()).await;
return stream_from_fixture(
path,
self.provider.clone(),
self.otel_event_manager.clone(),
)
.await;
}
let auth_manager = self.auth_manager.clone();
@@ -233,7 +245,7 @@ impl ModelClient {
let max_attempts = self.provider.request_max_retries();
for attempt in 0..=max_attempts {
match self
.attempt_stream_responses(&payload_json, &auth_manager)
.attempt_stream_responses(attempt, &payload_json, &auth_manager)
.await
{
Ok(stream) => {
@@ -258,6 +270,7 @@ impl ModelClient {
/// Single attempt to start a streaming Responses API call.
async fn attempt_stream_responses(
&self,
attempt: u64,
payload_json: &Value,
auth_manager: &Option<Arc<AuthManager>>,
) -> std::result::Result<ResponseStream, StreamAttemptError> {
@@ -291,15 +304,22 @@ impl ModelClient {
req_builder = req_builder.header("chatgpt-account-id", account_id);
}
let res = req_builder.send().await;
let res = self
.otel_event_manager
.log_request(attempt, || req_builder.send())
.await;
let mut request_id = None;
if let Ok(resp) = &res {
request_id = resp
.headers()
.get("cf-ray")
.map(|v| v.to_str().unwrap_or_default().to_string());
trace!(
"Response status: {}, cf-ray: {}",
"Response status: {}, cf-ray: {:?}",
resp.status(),
resp.headers()
.get("cf-ray")
.map(|v| v.to_str().unwrap_or_default())
.unwrap_or_default()
request_id
);
}
@@ -322,6 +342,7 @@ impl ModelClient {
stream,
tx_event,
self.provider.stream_idle_timeout(),
self.otel_event_manager.clone(),
));
Ok(ResponseStream { rx_event })
@@ -358,7 +379,11 @@ impl ModelClient {
// Surface the error body to callers. Use `unwrap_or_default` per Clippy.
let body = res.text().await.unwrap_or_default();
return Err(StreamAttemptError::Fatal(CodexErr::UnexpectedStatus(
status, body,
UnexpectedResponseError {
status,
body,
request_id: None,
},
)));
}
@@ -389,6 +414,7 @@ impl ModelClient {
Err(StreamAttemptError::RetryableHttpError {
status,
retry_after,
request_id,
})
}
Err(e) => Err(StreamAttemptError::RetryableTransportError(e.into())),
@@ -399,6 +425,10 @@ impl ModelClient {
self.provider.clone()
}
pub fn get_otel_event_manager(&self) -> OtelEventManager {
self.otel_event_manager.clone()
}
/// Returns the currently configured model slug.
pub fn get_model(&self) -> String {
self.config.model.clone()
@@ -428,6 +458,7 @@ enum StreamAttemptError {
RetryableHttpError {
status: StatusCode,
retry_after: Option<Duration>,
request_id: Option<String>,
},
RetryableTransportError(CodexErr),
Fatal(CodexErr),
@@ -452,11 +483,13 @@ impl StreamAttemptError {
fn into_error(self) -> CodexErr {
match self {
Self::RetryableHttpError { status, .. } => {
Self::RetryableHttpError {
status, request_id, ..
} => {
if status == StatusCode::INTERNAL_SERVER_ERROR {
CodexErr::InternalServerError
} else {
CodexErr::RetryLimit(status)
CodexErr::RetryLimit(RetryLimitReachedError { status, request_id })
}
}
Self::RetryableTransportError(error) => error,
@@ -605,6 +638,7 @@ async fn process_sse<S>(
stream: S,
tx_event: mpsc::Sender<Result<ResponseEvent>>,
idle_timeout: Duration,
otel_event_manager: OtelEventManager,
) where
S: Stream<Item = Result<Bytes>> + Unpin,
{
@@ -616,7 +650,10 @@ async fn process_sse<S>(
let mut response_error: Option<CodexErr> = None;
loop {
let sse = match timeout(idle_timeout, stream.next()).await {
let sse = match otel_event_manager
.log_sse_event(|| timeout(idle_timeout, stream.next()))
.await
{
Ok(Some(Ok(sse))) => sse,
Ok(Some(Err(e))) => {
debug!("SSE Error: {e:#}");
@@ -630,6 +667,21 @@ async fn process_sse<S>(
id: response_id,
usage,
}) => {
if let Some(token_usage) = &usage {
otel_event_manager.sse_event_completed(
token_usage.input_tokens,
token_usage.output_tokens,
token_usage
.input_tokens_details
.as_ref()
.map(|d| d.cached_tokens),
token_usage
.output_tokens_details
.as_ref()
.map(|d| d.reasoning_tokens),
token_usage.total_tokens,
);
}
let event = ResponseEvent::Completed {
response_id,
token_usage: usage.map(Into::into),
@@ -637,12 +689,13 @@ async fn process_sse<S>(
let _ = tx_event.send(Ok(event)).await;
}
None => {
let _ = tx_event
.send(Err(response_error.unwrap_or(CodexErr::Stream(
"stream closed before response.completed".into(),
None,
))))
.await;
let error = response_error.unwrap_or(CodexErr::Stream(
"stream closed before response.completed".into(),
None,
));
otel_event_manager.see_event_completed_failed(&error);
let _ = tx_event.send(Err(error)).await;
}
}
return;
@@ -746,7 +799,9 @@ async fn process_sse<S>(
response_error = Some(CodexErr::Stream(message, delay));
}
Err(e) => {
debug!("failed to parse ErrorResponse: {e}");
let error = format!("failed to parse ErrorResponse: {e}");
debug!(error);
response_error = Some(CodexErr::Stream(error, None))
}
}
}
@@ -760,7 +815,9 @@ async fn process_sse<S>(
response_completed = Some(r);
}
Err(e) => {
debug!("failed to parse ResponseCompleted: {e}");
let error = format!("failed to parse ResponseCompleted: {e}");
debug!(error);
response_error = Some(CodexErr::Stream(error, None));
continue;
}
};
@@ -807,6 +864,7 @@ async fn process_sse<S>(
async fn stream_from_fixture(
path: impl AsRef<Path>,
provider: ModelProviderInfo,
otel_event_manager: OtelEventManager,
) -> Result<ResponseStream> {
let (tx_event, rx_event) = mpsc::channel::<Result<ResponseEvent>>(1600);
let f = std::fs::File::open(path.as_ref())?;
@@ -825,6 +883,7 @@ async fn stream_from_fixture(
stream,
tx_event,
provider.stream_idle_timeout(),
otel_event_manager,
));
Ok(ResponseStream { rx_event })
}
@@ -880,6 +939,7 @@ mod tests {
async fn collect_events(
chunks: &[&[u8]],
provider: ModelProviderInfo,
otel_event_manager: OtelEventManager,
) -> Vec<Result<ResponseEvent>> {
let mut builder = IoBuilder::new();
for chunk in chunks {
@@ -889,7 +949,12 @@ mod tests {
let reader = builder.build();
let stream = ReaderStream::new(reader).map_err(CodexErr::Io);
let (tx, mut rx) = mpsc::channel::<Result<ResponseEvent>>(16);
tokio::spawn(process_sse(stream, tx, provider.stream_idle_timeout()));
tokio::spawn(process_sse(
stream,
tx,
provider.stream_idle_timeout(),
otel_event_manager,
));
let mut events = Vec::new();
while let Some(ev) = rx.recv().await {
@@ -903,6 +968,7 @@ mod tests {
async fn run_sse(
events: Vec<serde_json::Value>,
provider: ModelProviderInfo,
otel_event_manager: OtelEventManager,
) -> Vec<ResponseEvent> {
let mut body = String::new();
for e in events {
@@ -919,7 +985,12 @@ mod tests {
let (tx, mut rx) = mpsc::channel::<Result<ResponseEvent>>(8);
let stream = ReaderStream::new(std::io::Cursor::new(body)).map_err(CodexErr::Io);
tokio::spawn(process_sse(stream, tx, provider.stream_idle_timeout()));
tokio::spawn(process_sse(
stream,
tx,
provider.stream_idle_timeout(),
otel_event_manager,
));
let mut out = Vec::new();
while let Some(ev) = rx.recv().await {
@@ -928,6 +999,18 @@ mod tests {
out
}
fn otel_event_manager() -> OtelEventManager {
OtelEventManager::new(
ConversationId::new(),
"test",
"test",
None,
Some(AuthMode::ChatGPT),
false,
"test".to_string(),
)
}
// ────────────────────────────
// Tests from `implement-test-for-responses-api-sse-parser`
// ────────────────────────────
@@ -979,9 +1062,12 @@ mod tests {
requires_openai_auth: false,
};
let otel_event_manager = otel_event_manager();
let events = collect_events(
&[sse1.as_bytes(), sse2.as_bytes(), sse3.as_bytes()],
provider,
otel_event_manager,
)
.await;
@@ -1039,7 +1125,9 @@ mod tests {
requires_openai_auth: false,
};
let events = collect_events(&[sse1.as_bytes()], provider).await;
let otel_event_manager = otel_event_manager();
let events = collect_events(&[sse1.as_bytes()], provider, otel_event_manager).await;
assert_eq!(events.len(), 2);
@@ -1073,7 +1161,9 @@ mod tests {
requires_openai_auth: false,
};
let events = collect_events(&[sse1.as_bytes()], provider).await;
let otel_event_manager = otel_event_manager();
let events = collect_events(&[sse1.as_bytes()], provider, otel_event_manager).await;
assert_eq!(events.len(), 1);
@@ -1178,7 +1268,9 @@ mod tests {
requires_openai_auth: false,
};
let out = run_sse(evs, provider).await;
let otel_event_manager = otel_event_manager();
let out = run_sse(evs, provider, otel_event_manager).await;
assert_eq!(out.len(), case.expected_len, "case {}", case.name);
assert!(
(case.expect_first)(&out[0]),

File diff suppressed because it is too large Load Diff

View File

@@ -1,25 +1,431 @@
// This is a WIP. This will eventually contain a real list of common safe Windows commands.
pub fn is_safe_command_windows(_command: &[String]) -> bool {
use shlex::split as shlex_split;
/// On Windows, we conservatively allow only clearly read-only PowerShell invocations
/// that match a small safelist. Anything else (including direct CMD commands) is unsafe.
pub fn is_safe_command_windows(command: &[String]) -> bool {
if let Some(commands) = try_parse_powershell_command_sequence(command) {
return commands
.iter()
.all(|cmd| is_safe_powershell_command(cmd.as_slice()));
}
// Only PowerShell invocations are allowed on Windows for now; anything else is unsafe.
false
}
/// Returns each command sequence if the invocation starts with a PowerShell binary.
/// For example, the tokens from `pwsh Get-ChildItem | Measure-Object` become two sequences.
fn try_parse_powershell_command_sequence(command: &[String]) -> Option<Vec<Vec<String>>> {
let (exe, rest) = command.split_first()?;
if !is_powershell_executable(exe) {
return None;
}
parse_powershell_invocation(rest)
}
/// Parses a PowerShell invocation into discrete command vectors, rejecting unsafe patterns.
fn parse_powershell_invocation(args: &[String]) -> Option<Vec<Vec<String>>> {
if args.is_empty() {
// Examples rejected here: "pwsh" and "powershell.exe" with no additional arguments.
return None;
}
let mut idx = 0;
while idx < args.len() {
let arg = &args[idx];
let lower = arg.to_ascii_lowercase();
match lower.as_str() {
"-command" | "/command" | "-c" => {
let script = args.get(idx + 1)?;
if idx + 2 != args.len() {
// Reject if there is more than one token representing the actual command.
// Examples rejected here: "pwsh -Command foo bar" and "powershell -c ls extra".
return None;
}
return parse_powershell_script(script);
}
_ if lower.starts_with("-command:") || lower.starts_with("/command:") => {
if idx + 1 != args.len() {
// Reject if there are more tokens after the command itself.
// Examples rejected here: "pwsh -Command:dir C:\\" and "powershell /Command:dir C:\\" with trailing args.
return None;
}
let script = arg.split_once(':')?.1;
return parse_powershell_script(script);
}
// Benign, no-arg flags we tolerate.
"-nologo" | "-noprofile" | "-noninteractive" | "-mta" | "-sta" => {
idx += 1;
continue;
}
// Explicitly forbidden/opaque or unnecessary for read-only operations.
"-encodedcommand" | "-ec" | "-file" | "/file" | "-windowstyle" | "-executionpolicy"
| "-workingdirectory" => {
// Examples rejected here: "pwsh -EncodedCommand ..." and "powershell -File script.ps1".
return None;
}
// Unknown switch → bail conservatively.
_ if lower.starts_with('-') => {
// Examples rejected here: "pwsh -UnknownFlag" and "powershell -foo bar".
return None;
}
// If we hit non-flag tokens, treat the remainder as a command sequence.
// This happens if powershell is invoked without -Command, e.g.
// ["pwsh", "-NoLogo", "git", "-c", "core.pager=cat", "status"]
_ => {
return split_into_commands(args[idx..].to_vec());
}
}
}
// Examples rejected here: "pwsh" and "powershell.exe -NoLogo" without a script.
None
}
/// Tokenizes an inline PowerShell script and delegates to the command splitter.
/// Examples of when this is called: pwsh.exe -Command '<script>' or pwsh.exe -Command:<script>
fn parse_powershell_script(script: &str) -> Option<Vec<Vec<String>>> {
let tokens = shlex_split(script)?;
split_into_commands(tokens)
}
/// Splits tokens into pipeline segments while ensuring no unsafe separators slip through.
/// e.g. Get-ChildItem | Measure-Object -> [['Get-ChildItem'], ['Measure-Object']]
fn split_into_commands(tokens: Vec<String>) -> Option<Vec<Vec<String>>> {
if tokens.is_empty() {
// Examples rejected here: "pwsh -Command ''" and "powershell -Command \"\"".
return None;
}
let mut commands = Vec::new();
let mut current = Vec::new();
for token in tokens.into_iter() {
match token.as_str() {
"|" | "||" | "&&" | ";" => {
if current.is_empty() {
// Examples rejected here: "pwsh -Command '| Get-ChildItem'" and "pwsh -Command '; dir'".
return None;
}
commands.push(current);
current = Vec::new();
}
// Reject if any token embeds separators, redirection, or call operator characters.
_ if token.contains(['|', ';', '>', '<', '&']) || token.contains("$(") => {
// Examples rejected here: "pwsh -Command 'dir|select'" and "pwsh -Command 'echo hi > out.txt'".
return None;
}
_ => current.push(token),
}
}
if current.is_empty() {
// Examples rejected here: "pwsh -Command 'dir |'" and "pwsh -Command 'Get-ChildItem ;'".
return None;
}
commands.push(current);
Some(commands)
}
/// Returns true when the executable name is one of the supported PowerShell binaries.
fn is_powershell_executable(exe: &str) -> bool {
matches!(
exe.to_ascii_lowercase().as_str(),
"powershell" | "powershell.exe" | "pwsh" | "pwsh.exe"
)
}
/// Validates that a parsed PowerShell command stays within our read-only safelist.
/// Everything before this is parsing, and rejecting things that make us feel uncomfortable.
fn is_safe_powershell_command(words: &[String]) -> bool {
if words.is_empty() {
// Examples rejected here: "pwsh -Command ''" and "pwsh -Command \"\"".
return false;
}
// Reject nested unsafe cmdlets inside parentheses or arguments
for w in words.iter() {
let inner = w
.trim_matches(|c| c == '(' || c == ')')
.trim_start_matches('-')
.to_ascii_lowercase();
if matches!(
inner.as_str(),
"set-content"
| "add-content"
| "out-file"
| "new-item"
| "remove-item"
| "move-item"
| "copy-item"
| "rename-item"
| "start-process"
| "stop-process"
) {
// Examples rejected here: "Write-Output (Set-Content foo6.txt 'abc')" and "Get-Content (New-Item bar.txt)".
return false;
}
}
// Block PowerShell call operator or any redirection explicitly.
if words.iter().any(|w| {
matches!(
w.as_str(),
"&" | ">" | ">>" | "1>" | "2>" | "2>&1" | "*>" | "<" | "<<"
)
}) {
// Examples rejected here: "pwsh -Command '& Remove-Item foo'" and "pwsh -Command 'Get-Content foo > bar'".
return false;
}
let command = words[0]
.trim_matches(|c| c == '(' || c == ')')
.trim_start_matches('-')
.to_ascii_lowercase();
match command.as_str() {
"echo" | "write-output" | "write-host" => true, // (no redirection allowed)
"dir" | "ls" | "get-childitem" | "gci" => true,
"cat" | "type" | "gc" | "get-content" => true,
"select-string" | "sls" | "findstr" => true,
"measure-object" | "measure" => true,
"get-location" | "gl" | "pwd" => true,
"test-path" | "tp" => true,
"resolve-path" | "rvpa" => true,
"select-object" | "select" => true,
"get-item" => true,
"git" => is_safe_git_command(words),
"rg" => is_safe_ripgrep(words),
// Extra safety: explicitly prohibit common side-effecting cmdlets regardless of args.
"set-content" | "add-content" | "out-file" | "new-item" | "remove-item" | "move-item"
| "copy-item" | "rename-item" | "start-process" | "stop-process" => {
// Examples rejected here: "pwsh -Command 'Set-Content notes.txt data'" and "pwsh -Command 'Remove-Item temp.log'".
false
}
_ => {
// Examples rejected here: "pwsh -Command 'Invoke-WebRequest https://example.com'" and "pwsh -Command 'Start-Service Spooler'".
false
}
}
}
/// Checks that an `rg` invocation avoids options that can spawn arbitrary executables.
fn is_safe_ripgrep(words: &[String]) -> bool {
const UNSAFE_RIPGREP_OPTIONS_WITH_ARGS: &[&str] = &["--pre", "--hostname-bin"];
const UNSAFE_RIPGREP_OPTIONS_WITHOUT_ARGS: &[&str] = &["--search-zip", "-z"];
!words.iter().skip(1).any(|arg| {
let arg_lc = arg.to_ascii_lowercase();
// Examples rejected here: "pwsh -Command 'rg --pre cat pattern'" and "pwsh -Command 'rg --search-zip pattern'".
UNSAFE_RIPGREP_OPTIONS_WITHOUT_ARGS.contains(&arg_lc.as_str())
|| UNSAFE_RIPGREP_OPTIONS_WITH_ARGS
.iter()
.any(|opt| arg_lc == *opt || arg_lc.starts_with(&format!("{opt}=")))
})
}
/// Ensures a Git command sticks to whitelisted read-only subcommands and flags.
fn is_safe_git_command(words: &[String]) -> bool {
const SAFE_SUBCOMMANDS: &[&str] = &["status", "log", "show", "diff", "cat-file"];
let mut iter = words.iter().skip(1);
while let Some(arg) = iter.next() {
let arg_lc = arg.to_ascii_lowercase();
if arg.starts_with('-') {
if arg.eq_ignore_ascii_case("-c") || arg.eq_ignore_ascii_case("--config") {
if iter.next().is_none() {
// Examples rejected here: "pwsh -Command 'git -c'" and "pwsh -Command 'git --config'".
return false;
}
continue;
}
if arg_lc.starts_with("-c=")
|| arg_lc.starts_with("--config=")
|| arg_lc.starts_with("--git-dir=")
|| arg_lc.starts_with("--work-tree=")
{
continue;
}
if arg.eq_ignore_ascii_case("--git-dir") || arg.eq_ignore_ascii_case("--work-tree") {
if iter.next().is_none() {
// Examples rejected here: "pwsh -Command 'git --git-dir'" and "pwsh -Command 'git --work-tree'".
return false;
}
continue;
}
continue;
}
return SAFE_SUBCOMMANDS.contains(&arg_lc.as_str());
}
// Examples rejected here: "pwsh -Command 'git'" and "pwsh -Command 'git status --short | Remove-Item foo'".
false
}
#[cfg(test)]
mod tests {
use super::is_safe_command_windows;
use std::string::ToString;
/// Converts a slice of string literals into owned `String`s for the tests.
fn vec_str(args: &[&str]) -> Vec<String> {
args.iter().map(ToString::to_string).collect()
}
#[test]
fn everything_is_unsafe() {
for cmd in [
vec_str(&["powershell.exe", "-NoLogo", "-Command", "echo hello"]),
vec_str(&["copy", "foo", "bar"]),
vec_str(&["del", "file.txt"]),
vec_str(&["powershell.exe", "Get-ChildItem"]),
] {
assert!(!is_safe_command_windows(&cmd));
}
fn recognizes_safe_powershell_wrappers() {
assert!(is_safe_command_windows(&vec_str(&[
"powershell.exe",
"-NoLogo",
"-Command",
"Get-ChildItem -Path .",
])));
assert!(is_safe_command_windows(&vec_str(&[
"powershell.exe",
"-NoProfile",
"-Command",
"git status",
])));
assert!(is_safe_command_windows(&vec_str(&[
"powershell.exe",
"Get-Content",
"Cargo.toml",
])));
// pwsh parity
assert!(is_safe_command_windows(&vec_str(&[
"pwsh.exe",
"-NoProfile",
"-Command",
"Get-ChildItem",
])));
}
#[test]
fn allows_read_only_pipelines_and_git_usage() {
assert!(is_safe_command_windows(&vec_str(&[
"pwsh",
"-NoLogo",
"-NoProfile",
"-Command",
"rg --files-with-matches foo | Measure-Object | Select-Object -ExpandProperty Count",
])));
assert!(is_safe_command_windows(&vec_str(&[
"pwsh",
"-NoLogo",
"-NoProfile",
"-Command",
"Get-Content foo.rs | Select-Object -Skip 200",
])));
assert!(is_safe_command_windows(&vec_str(&[
"pwsh",
"-NoLogo",
"-NoProfile",
"-Command",
"git -c core.pager=cat show HEAD:foo.rs",
])));
assert!(is_safe_command_windows(&vec_str(&[
"pwsh",
"-Command",
"-git cat-file -p HEAD:foo.rs",
])));
assert!(is_safe_command_windows(&vec_str(&[
"pwsh",
"-Command",
"(Get-Content foo.rs -Raw)",
])));
assert!(is_safe_command_windows(&vec_str(&[
"pwsh",
"-Command",
"Get-Item foo.rs | Select-Object Length",
])));
}
#[test]
fn rejects_powershell_commands_with_side_effects() {
assert!(!is_safe_command_windows(&vec_str(&[
"powershell.exe",
"-NoLogo",
"-Command",
"Remove-Item foo.txt",
])));
assert!(!is_safe_command_windows(&vec_str(&[
"powershell.exe",
"-NoProfile",
"-Command",
"rg --pre cat",
])));
assert!(!is_safe_command_windows(&vec_str(&[
"powershell.exe",
"-Command",
"Set-Content foo.txt 'hello'",
])));
// Redirections are blocked
assert!(!is_safe_command_windows(&vec_str(&[
"powershell.exe",
"-Command",
"echo hi > out.txt",
])));
assert!(!is_safe_command_windows(&vec_str(&[
"powershell.exe",
"-Command",
"Get-Content x | Out-File y",
])));
assert!(!is_safe_command_windows(&vec_str(&[
"powershell.exe",
"-Command",
"Write-Output foo 2> err.txt",
])));
// Call operator is blocked
assert!(!is_safe_command_windows(&vec_str(&[
"powershell.exe",
"-Command",
"& Remove-Item foo",
])));
// Chained safe + unsafe must fail
assert!(!is_safe_command_windows(&vec_str(&[
"powershell.exe",
"-Command",
"Get-ChildItem; Remove-Item foo",
])));
// Nested unsafe cmdlet inside safe command must fail
assert!(!is_safe_command_windows(&vec_str(&[
"powershell.exe",
"-Command",
"Write-Output (Set-Content foo6.txt 'abc')",
])));
// Additional nested unsafe cmdlet examples must fail
assert!(!is_safe_command_windows(&vec_str(&[
"powershell.exe",
"-Command",
"Write-Host (Remove-Item foo.txt)",
])));
assert!(!is_safe_command_windows(&vec_str(&[
"powershell.exe",
"-Command",
"Get-Content (New-Item bar.txt)",
])));
}
}

View File

@@ -1,8 +1,12 @@
use crate::config_profile::ConfigProfile;
use crate::config_types::DEFAULT_OTEL_ENVIRONMENT;
use crate::config_types::History;
use crate::config_types::McpServerConfig;
use crate::config_types::McpServerTransportConfig;
use crate::config_types::Notifications;
use crate::config_types::OtelConfig;
use crate::config_types::OtelConfigToml;
use crate::config_types::OtelExporterKind;
use crate::config_types::ReasoningSummaryFormat;
use crate::config_types::SandboxWorkspaceWrite;
use crate::config_types::ShellEnvironmentPolicy;
@@ -19,12 +23,12 @@ use crate::openai_model_info::get_model_info;
use crate::protocol::AskForApproval;
use crate::protocol::SandboxPolicy;
use anyhow::Context;
use codex_app_server_protocol::Tools;
use codex_app_server_protocol::UserSavedConfig;
use codex_protocol::config_types::ReasoningEffort;
use codex_protocol::config_types::ReasoningSummary;
use codex_protocol::config_types::SandboxMode;
use codex_protocol::config_types::Verbosity;
use codex_protocol::mcp_protocol::Tools;
use codex_protocol::mcp_protocol::UserSavedConfig;
use dirs::home_dir;
use serde::Deserialize;
use std::collections::BTreeMap;
@@ -137,6 +141,9 @@ pub struct Config {
/// Maximum number of bytes to include from an AGENTS.md project doc file.
pub project_doc_max_bytes: usize,
/// Additional filenames to try when looking for project-level docs.
pub project_doc_fallback_filenames: Vec<String>,
/// Directory containing all Codex state (defaults to `~/.codex` but can be
/// overridden by the `CODEX_HOME` environment variable).
pub codex_home: PathBuf,
@@ -191,8 +198,6 @@ pub struct Config {
/// Include the `view_image` tool that lets the agent attach a local image path to context.
pub include_view_image_tool: bool,
/// When true, sessions run inside a linked git worktree under `cwd/codex/<conversation>`.
pub enable_git_worktree: bool,
/// The active profile name used to derive this `Config` (if any).
pub active_profile: Option<String>,
@@ -201,6 +206,9 @@ pub struct Config {
/// All characters are inserted as they are received, and no buffering
/// or placeholder replacement will occur for fast keypress bursts.
pub disable_paste_burst: bool,
/// OTEL configuration (exporter type, endpoint, headers, etc.).
pub otel: crate::config_types::OtelConfig,
}
impl Config {
@@ -428,7 +436,7 @@ fn set_project_trusted_inner(doc: &mut DocumentMut, project_path: &Path) -> anyh
.get_mut(project_key.as_str())
.and_then(|i| i.as_table_mut())
else {
return Err(anyhow::anyhow!("project table missing for {}", project_key));
return Err(anyhow::anyhow!("project table missing for {project_key}"));
};
proj_tbl.set_implicit(false);
proj_tbl["trust_level"] = toml_edit::value("trusted");
@@ -665,6 +673,9 @@ pub struct ConfigToml {
/// Maximum number of bytes to include from an AGENTS.md project doc file.
pub project_doc_max_bytes: Option<usize>,
/// Ordered list of fallback filenames to look for when AGENTS.md is missing.
pub project_doc_fallback_filenames: Option<Vec<String>>,
/// Profile to use from the `profiles` map.
pub profile: Option<String>,
@@ -691,9 +702,6 @@ pub struct ConfigToml {
/// Defaults to `false`.
pub show_raw_agent_reasoning: Option<bool>,
/// Enable per-session git worktree checkouts.
pub enable_git_worktree: Option<bool>,
pub model_reasoning_effort: Option<ReasoningEffort>,
pub model_reasoning_summary: Option<ReasoningSummary>,
/// Optional verbosity control for GPT-5 models (Responses API `text.verbosity`).
@@ -724,6 +732,9 @@ pub struct ConfigToml {
/// All characters are inserted as they are received, and no buffering
/// or placeholder replacement will occur for fast keypress bursts.
pub disable_paste_burst: Option<bool>,
/// OTEL configuration.
pub otel: Option<crate::config_types::OtelConfigToml>,
}
impl From<ConfigToml> for UserSavedConfig {
@@ -864,7 +875,6 @@ pub struct ConfigOverrides {
pub include_view_image_tool: Option<bool>,
pub show_raw_agent_reasoning: Option<bool>,
pub tools_web_search_request: Option<bool>,
pub enable_git_worktree: Option<bool>,
}
impl Config {
@@ -893,7 +903,6 @@ impl Config {
include_view_image_tool,
show_raw_agent_reasoning,
tools_web_search_request: override_tools_web_search_request,
enable_git_worktree: override_enable_git_worktree,
} = overrides;
let active_profile_name = config_profile_key
@@ -967,10 +976,6 @@ impl Config {
.or(cfg.tools.as_ref().and_then(|t| t.view_image))
.unwrap_or(true);
let enable_git_worktree = override_enable_git_worktree
.or(cfg.enable_git_worktree)
.unwrap_or(false);
let model = model
.or(config_profile.model)
.or(cfg.model)
@@ -1039,6 +1044,19 @@ impl Config {
mcp_servers: cfg.mcp_servers,
model_providers,
project_doc_max_bytes: cfg.project_doc_max_bytes.unwrap_or(PROJECT_DOC_MAX_BYTES),
project_doc_fallback_filenames: cfg
.project_doc_fallback_filenames
.unwrap_or_default()
.into_iter()
.filter_map(|name| {
let trimmed = name.trim();
if trimmed.is_empty() {
None
} else {
Some(trimmed.to_string())
}
})
.collect(),
codex_home,
history,
file_opener: cfg.file_opener.unwrap_or(UriBasedFileOpener::VsCode),
@@ -1061,7 +1079,7 @@ impl Config {
.chatgpt_base_url
.or(cfg.chatgpt_base_url)
.unwrap_or("https://chatgpt.com/backend-api/".to_string()),
include_plan_tool: include_plan_tool.unwrap_or(false),
include_plan_tool: include_plan_tool.unwrap_or(true),
include_apply_patch_tool: include_apply_patch_tool.unwrap_or(false),
tools_web_search_request,
use_experimental_streamable_shell_tool: cfg
@@ -1072,7 +1090,6 @@ impl Config {
.unwrap_or(false),
use_experimental_use_rmcp_client: cfg.experimental_use_rmcp_client.unwrap_or(false),
include_view_image_tool,
enable_git_worktree,
active_profile: active_profile_name,
disable_paste_burst: cfg.disable_paste_burst.unwrap_or(false),
tui_notifications: cfg
@@ -1080,6 +1097,19 @@ impl Config {
.as_ref()
.map(|t| t.notifications.clone())
.unwrap_or_default(),
otel: {
let t: OtelConfigToml = cfg.otel.unwrap_or_default();
let log_user_prompt = t.log_user_prompt.unwrap_or(false);
let environment = t
.environment
.unwrap_or(DEFAULT_OTEL_ENVIRONMENT.to_string());
let exporter = t.exporter.unwrap_or(OtelExporterKind::None);
OtelConfig {
log_user_prompt,
environment,
exporter,
}
},
};
Ok(config)
}
@@ -1800,6 +1830,7 @@ model_verbosity = "high"
mcp_servers: HashMap::new(),
model_providers: fixture.model_provider_map.clone(),
project_doc_max_bytes: PROJECT_DOC_MAX_BYTES,
project_doc_fallback_filenames: Vec::new(),
codex_home: fixture.codex_home(),
history: History::default(),
file_opener: UriBasedFileOpener::VsCode,
@@ -1811,17 +1842,17 @@ model_verbosity = "high"
model_verbosity: None,
chatgpt_base_url: "https://chatgpt.com/backend-api/".to_string(),
base_instructions: None,
include_plan_tool: false,
include_plan_tool: true,
include_apply_patch_tool: false,
tools_web_search_request: false,
use_experimental_streamable_shell_tool: false,
use_experimental_unified_exec_tool: false,
use_experimental_use_rmcp_client: false,
include_view_image_tool: true,
enable_git_worktree: false,
active_profile: Some("o3".to_string()),
disable_paste_burst: false,
tui_notifications: Default::default(),
otel: OtelConfig::default(),
},
o3_profile_config
);
@@ -1860,6 +1891,7 @@ model_verbosity = "high"
mcp_servers: HashMap::new(),
model_providers: fixture.model_provider_map.clone(),
project_doc_max_bytes: PROJECT_DOC_MAX_BYTES,
project_doc_fallback_filenames: Vec::new(),
codex_home: fixture.codex_home(),
history: History::default(),
file_opener: UriBasedFileOpener::VsCode,
@@ -1871,17 +1903,17 @@ model_verbosity = "high"
model_verbosity: None,
chatgpt_base_url: "https://chatgpt.com/backend-api/".to_string(),
base_instructions: None,
include_plan_tool: false,
include_plan_tool: true,
include_apply_patch_tool: false,
tools_web_search_request: false,
use_experimental_streamable_shell_tool: false,
use_experimental_unified_exec_tool: false,
use_experimental_use_rmcp_client: false,
include_view_image_tool: true,
enable_git_worktree: false,
active_profile: Some("gpt3".to_string()),
disable_paste_burst: false,
tui_notifications: Default::default(),
otel: OtelConfig::default(),
};
assert_eq!(expected_gpt3_profile_config, gpt3_profile_config);
@@ -1935,6 +1967,7 @@ model_verbosity = "high"
mcp_servers: HashMap::new(),
model_providers: fixture.model_provider_map.clone(),
project_doc_max_bytes: PROJECT_DOC_MAX_BYTES,
project_doc_fallback_filenames: Vec::new(),
codex_home: fixture.codex_home(),
history: History::default(),
file_opener: UriBasedFileOpener::VsCode,
@@ -1946,17 +1979,17 @@ model_verbosity = "high"
model_verbosity: None,
chatgpt_base_url: "https://chatgpt.com/backend-api/".to_string(),
base_instructions: None,
include_plan_tool: false,
include_plan_tool: true,
include_apply_patch_tool: false,
tools_web_search_request: false,
use_experimental_streamable_shell_tool: false,
use_experimental_unified_exec_tool: false,
use_experimental_use_rmcp_client: false,
include_view_image_tool: true,
enable_git_worktree: false,
active_profile: Some("zdr".to_string()),
disable_paste_burst: false,
tui_notifications: Default::default(),
otel: OtelConfig::default(),
};
assert_eq!(expected_zdr_profile_config, zdr_profile_config);
@@ -1996,6 +2029,7 @@ model_verbosity = "high"
mcp_servers: HashMap::new(),
model_providers: fixture.model_provider_map.clone(),
project_doc_max_bytes: PROJECT_DOC_MAX_BYTES,
project_doc_fallback_filenames: Vec::new(),
codex_home: fixture.codex_home(),
history: History::default(),
file_opener: UriBasedFileOpener::VsCode,
@@ -2007,17 +2041,17 @@ model_verbosity = "high"
model_verbosity: Some(Verbosity::High),
chatgpt_base_url: "https://chatgpt.com/backend-api/".to_string(),
base_instructions: None,
include_plan_tool: false,
include_plan_tool: true,
include_apply_patch_tool: false,
tools_web_search_request: false,
use_experimental_streamable_shell_tool: false,
use_experimental_unified_exec_tool: false,
use_experimental_use_rmcp_client: false,
include_view_image_tool: true,
enable_git_worktree: false,
active_profile: Some("gpt5".to_string()),
disable_paste_burst: false,
tui_notifications: Default::default(),
otel: OtelConfig::default(),
};
assert_eq!(expected_gpt5_profile_config, gpt5_profile_config);

View File

@@ -22,7 +22,7 @@ pub struct ConfigProfile {
pub experimental_instructions_file: Option<PathBuf>,
}
impl From<ConfigProfile> for codex_protocol::mcp_protocol::Profile {
impl From<ConfigProfile> for codex_app_server_protocol::Profile {
fn from(config_profile: ConfigProfile) -> Self {
Self {
model: config_profile.model,

View File

@@ -13,6 +13,8 @@ use serde::Deserialize;
use serde::Serialize;
use serde::de::Error as SerdeError;
pub const DEFAULT_OTEL_ENVIRONMENT: &str = "dev";
#[derive(Serialize, Debug, Clone, PartialEq)]
pub struct McpServerConfig {
#[serde(flatten)]
@@ -219,6 +221,64 @@ pub enum HistoryPersistence {
None,
}
// ===== OTEL configuration =====
#[derive(Deserialize, Debug, Clone, PartialEq)]
#[serde(rename_all = "kebab-case")]
pub enum OtelHttpProtocol {
/// Binary payload
Binary,
/// JSON payload
Json,
}
/// Which OTEL exporter to use.
#[derive(Deserialize, Debug, Clone, PartialEq)]
#[serde(rename_all = "kebab-case")]
pub enum OtelExporterKind {
None,
OtlpHttp {
endpoint: String,
headers: HashMap<String, String>,
protocol: OtelHttpProtocol,
},
OtlpGrpc {
endpoint: String,
headers: HashMap<String, String>,
},
}
/// OTEL settings loaded from config.toml. Fields are optional so we can apply defaults.
#[derive(Deserialize, Debug, Clone, PartialEq, Default)]
pub struct OtelConfigToml {
/// Log user prompt in traces
pub log_user_prompt: Option<bool>,
/// Mark traces with environment (dev, staging, prod, test). Defaults to dev.
pub environment: Option<String>,
/// Exporter to use. Defaults to `otlp-file`.
pub exporter: Option<OtelExporterKind>,
}
/// Effective OTEL settings after defaults are applied.
#[derive(Debug, Clone, PartialEq)]
pub struct OtelConfig {
pub log_user_prompt: bool,
pub environment: String,
pub exporter: OtelExporterKind,
}
impl Default for OtelConfig {
fn default() -> Self {
OtelConfig {
log_user_prompt: false,
environment: DEFAULT_OTEL_ENVIRONMENT.to_owned(),
exporter: OtelExporterKind::None,
}
}
}
#[derive(Debug, Clone, PartialEq, Eq, Deserialize)]
#[serde(untagged)]
pub enum Notifications {
@@ -253,7 +313,7 @@ pub struct SandboxWorkspaceWrite {
pub exclude_slash_tmp: bool,
}
impl From<SandboxWorkspaceWrite> for codex_protocol::mcp_protocol::SandboxSettings {
impl From<SandboxWorkspaceWrite> for codex_app_server_protocol::SandboxSettings {
fn from(sandbox_workspace_write: SandboxWorkspaceWrite) -> Self {
Self {
writable_roots: sandbox_workspace_write.writable_roots,

View File

@@ -13,7 +13,7 @@ use crate::protocol::Event;
use crate::protocol::EventMsg;
use crate::protocol::SessionConfiguredEvent;
use crate::rollout::RolloutRecorder;
use codex_protocol::mcp_protocol::ConversationId;
use codex_protocol::ConversationId;
use codex_protocol::models::ResponseItem;
use codex_protocol::protocol::InitialHistory;
use codex_protocol::protocol::RolloutItem;

View File

@@ -63,16 +63,88 @@ pub async fn discover_prompts_in_excluding(
Ok(s) => s,
Err(_) => continue,
};
let (description, argument_hint, body) = parse_frontmatter(&content);
out.push(CustomPrompt {
name,
path,
content,
content: body,
description,
argument_hint,
});
}
out.sort_by(|a, b| a.name.cmp(&b.name));
out
}
/// Parse optional YAML-like frontmatter at the beginning of `content`.
/// Supported keys:
/// - `description`: short description shown in the slash popup
/// - `argument-hint` or `argument_hint`: brief hint string shown after the description
/// Returns (description, argument_hint, body_without_frontmatter).
fn parse_frontmatter(content: &str) -> (Option<String>, Option<String>, String) {
let mut segments = content.split_inclusive('\n');
let Some(first_segment) = segments.next() else {
return (None, None, String::new());
};
let first_line = first_segment.trim_end_matches(['\r', '\n']);
if first_line.trim() != "---" {
return (None, None, content.to_string());
}
let mut desc: Option<String> = None;
let mut hint: Option<String> = None;
let mut frontmatter_closed = false;
let mut consumed = first_segment.len();
for segment in segments {
let line = segment.trim_end_matches(['\r', '\n']);
let trimmed = line.trim();
if trimmed == "---" {
frontmatter_closed = true;
consumed += segment.len();
break;
}
if trimmed.is_empty() || trimmed.starts_with('#') {
consumed += segment.len();
continue;
}
if let Some((k, v)) = trimmed.split_once(':') {
let key = k.trim().to_ascii_lowercase();
let mut val = v.trim().to_string();
if val.len() >= 2 {
let bytes = val.as_bytes();
let first = bytes[0];
let last = bytes[bytes.len() - 1];
if (first == b'\"' && last == b'\"') || (first == b'\'' && last == b'\'') {
val = val[1..val.len().saturating_sub(1)].to_string();
}
}
match key.as_str() {
"description" => desc = Some(val),
"argument-hint" | "argument_hint" => hint = Some(val),
_ => {}
}
}
consumed += segment.len();
}
if !frontmatter_closed {
// Unterminated frontmatter: treat input as-is.
return (None, None, content.to_string());
}
let body = if consumed >= content.len() {
String::new()
} else {
content[consumed..].to_string()
};
(desc, hint, body)
}
#[cfg(test)]
mod tests {
use super::*;
@@ -124,4 +196,31 @@ mod tests {
let names: Vec<String> = found.into_iter().map(|e| e.name).collect();
assert_eq!(names, vec!["good"]);
}
#[tokio::test]
async fn parses_frontmatter_and_strips_from_body() {
let tmp = tempdir().expect("create TempDir");
let dir = tmp.path();
let file = dir.join("withmeta.md");
let text = "---\nname: ignored\ndescription: \"Quick review command\"\nargument-hint: \"[file] [priority]\"\n---\nActual body with $1 and $ARGUMENTS";
fs::write(&file, text).unwrap();
let found = discover_prompts_in(dir).await;
assert_eq!(found.len(), 1);
let p = &found[0];
assert_eq!(p.name, "withmeta");
assert_eq!(p.description.as_deref(), Some("Quick review command"));
assert_eq!(p.argument_hint.as_deref(), Some("[file] [priority]"));
// Body should not include the frontmatter delimiters.
assert_eq!(p.content, "Actual body with $1 and $ARGUMENTS");
}
#[test]
fn parse_frontmatter_preserves_body_newlines() {
let content = "---\r\ndescription: \"Line endings\"\r\nargument_hint: \"[arg]\"\r\n---\r\nFirst line\r\nSecond line\r\n";
let (desc, hint, body) = parse_frontmatter(content);
assert_eq!(desc.as_deref(), Some("Line endings"));
assert_eq!(hint.as_deref(), Some("[arg]"));
assert_eq!(body, "First line\r\nSecond line\r\n");
}
}

View File

@@ -2,6 +2,7 @@ use crate::spawn::CODEX_SANDBOX_ENV_VAR;
use reqwest::header::HeaderValue;
use std::sync::LazyLock;
use std::sync::Mutex;
use std::sync::OnceLock;
/// Set this to add a suffix to the User-Agent string.
///
@@ -26,8 +27,15 @@ pub struct Originator {
pub value: String,
pub header_value: HeaderValue,
}
static ORIGINATOR: OnceLock<Originator> = OnceLock::new();
pub static ORIGINATOR: LazyLock<Originator> = LazyLock::new(|| {
#[derive(Debug)]
pub enum SetOriginatorError {
InvalidHeaderValue,
AlreadyInitialized,
}
fn init_originator_from_env() -> Originator {
let default = "codex_cli_rs";
let value = std::env::var(CODEX_INTERNAL_ORIGINATOR_OVERRIDE_ENV_VAR)
.unwrap_or_else(|_| default.to_string());
@@ -45,14 +53,34 @@ pub static ORIGINATOR: LazyLock<Originator> = LazyLock::new(|| {
}
}
}
});
}
fn build_originator(value: String) -> Result<Originator, SetOriginatorError> {
let header_value =
HeaderValue::from_str(&value).map_err(|_| SetOriginatorError::InvalidHeaderValue)?;
Ok(Originator {
value,
header_value,
})
}
pub fn set_default_originator(value: &str) -> Result<(), SetOriginatorError> {
let originator = build_originator(value.to_string())?;
ORIGINATOR
.set(originator)
.map_err(|_| SetOriginatorError::AlreadyInitialized)
}
pub fn originator() -> &'static Originator {
ORIGINATOR.get_or_init(init_originator_from_env)
}
pub fn get_codex_user_agent() -> String {
let build_version = env!("CARGO_PKG_VERSION");
let os_info = os_info::get();
let prefix = format!(
"{}/{build_version} ({} {}; {}) {}",
ORIGINATOR.value.as_str(),
originator().value.as_str(),
os_info.os_type(),
os_info.version(),
os_info.architecture().unwrap_or("unknown"),
@@ -100,7 +128,7 @@ fn sanitize_user_agent(candidate: String, fallback: &str) -> String {
tracing::warn!(
"Falling back to default Codex originator because base user agent string is invalid"
);
ORIGINATOR.value.clone()
originator().value.clone()
}
}
@@ -109,7 +137,7 @@ pub fn create_client() -> reqwest::Client {
use reqwest::header::HeaderMap;
let mut headers = HeaderMap::new();
headers.insert("originator", ORIGINATOR.header_value.clone());
headers.insert("originator", originator().header_value.clone());
let ua = get_codex_user_agent();
let mut builder = reqwest::Client::builder()

View File

@@ -1,7 +1,7 @@
use crate::exec::ExecToolCallOutput;
use crate::token_data::KnownPlan;
use crate::token_data::PlanType;
use codex_protocol::mcp_protocol::ConversationId;
use codex_protocol::ConversationId;
use codex_protocol::protocol::RateLimitSnapshot;
use reqwest::StatusCode;
use serde_json;
@@ -76,8 +76,8 @@ pub enum CodexErr {
Interrupted,
/// Unexpected HTTP status code.
#[error("unexpected status {0}: {1}")]
UnexpectedStatus(StatusCode, String),
#[error("{0}")]
UnexpectedStatus(UnexpectedResponseError),
#[error("{0}")]
UsageLimitReached(UsageLimitReachedError),
@@ -91,8 +91,8 @@ pub enum CodexErr {
InternalServerError,
/// Retry limit exceeded.
#[error("exceeded retry limit, last status: {0}")]
RetryLimit(StatusCode),
#[error("{0}")]
RetryLimit(RetryLimitReachedError),
/// Agent loop died unexpectedly
#[error("internal error; agent loop died unexpectedly")]
@@ -135,6 +135,49 @@ pub enum CodexErr {
EnvVar(EnvVarError),
}
#[derive(Debug)]
pub struct UnexpectedResponseError {
pub status: StatusCode,
pub body: String,
pub request_id: Option<String>,
}
impl std::fmt::Display for UnexpectedResponseError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(
f,
"unexpected status {}: {}{}",
self.status,
self.body,
self.request_id
.as_ref()
.map(|id| format!(", request id: {id}"))
.unwrap_or_default()
)
}
}
impl std::error::Error for UnexpectedResponseError {}
#[derive(Debug)]
pub struct RetryLimitReachedError {
pub status: StatusCode,
pub request_id: Option<String>,
}
impl std::fmt::Display for RetryLimitReachedError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(
f,
"exceeded retry limit, last status: {}{}",
self.status,
self.request_id
.as_ref()
.map(|id| format!(", request id: {id}"))
.unwrap_or_default()
)
}
}
#[derive(Debug)]
pub struct UsageLimitReachedError {
pub(crate) plan_type: Option<PlanType>,

View File

@@ -0,0 +1,101 @@
use std::collections::HashMap;
use std::env;
use async_trait::async_trait;
use crate::CODEX_APPLY_PATCH_ARG1;
use crate::apply_patch::ApplyPatchExec;
use crate::exec::ExecParams;
use crate::function_tool::FunctionCallError;
pub(crate) enum ExecutionMode {
Shell,
ApplyPatch(ApplyPatchExec),
}
#[async_trait]
/// Backend-specific hooks that prepare and post-process execution requests for a
/// given [`ExecutionMode`].
pub(crate) trait ExecutionBackend: Send + Sync {
fn prepare(
&self,
params: ExecParams,
// Required for downcasting the apply_patch.
mode: &ExecutionMode,
) -> Result<ExecParams, FunctionCallError>;
fn stream_stdout(&self, _mode: &ExecutionMode) -> bool {
true
}
}
static SHELL_BACKEND: ShellBackend = ShellBackend;
static APPLY_PATCH_BACKEND: ApplyPatchBackend = ApplyPatchBackend;
pub(crate) fn backend_for_mode(mode: &ExecutionMode) -> &'static dyn ExecutionBackend {
match mode {
ExecutionMode::Shell => &SHELL_BACKEND,
ExecutionMode::ApplyPatch(_) => &APPLY_PATCH_BACKEND,
}
}
struct ShellBackend;
#[async_trait]
impl ExecutionBackend for ShellBackend {
fn prepare(
&self,
params: ExecParams,
mode: &ExecutionMode,
) -> Result<ExecParams, FunctionCallError> {
match mode {
ExecutionMode::Shell => Ok(params),
_ => Err(FunctionCallError::RespondToModel(
"shell backend invoked with non-shell mode".to_string(),
)),
}
}
}
struct ApplyPatchBackend;
#[async_trait]
impl ExecutionBackend for ApplyPatchBackend {
fn prepare(
&self,
params: ExecParams,
mode: &ExecutionMode,
) -> Result<ExecParams, FunctionCallError> {
match mode {
ExecutionMode::ApplyPatch(exec) => {
let path_to_codex = env::current_exe()
.ok()
.map(|p| p.to_string_lossy().to_string())
.ok_or_else(|| {
FunctionCallError::RespondToModel(
"failed to determine path to codex executable".to_string(),
)
})?;
let patch = exec.action.patch.clone();
Ok(ExecParams {
command: vec![path_to_codex, CODEX_APPLY_PATCH_ARG1.to_string(), patch],
cwd: exec.action.cwd.clone(),
timeout_ms: params.timeout_ms,
// Run apply_patch with a minimal environment for determinism and to
// avoid leaking host environment variables into the patch process.
env: HashMap::new(),
with_escalated_permissions: params.with_escalated_permissions,
justification: params.justification,
})
}
ExecutionMode::Shell => Err(FunctionCallError::RespondToModel(
"apply_patch backend invoked without patch context".to_string(),
)),
}
}
fn stream_stdout(&self, _mode: &ExecutionMode) -> bool {
false
}
}

View File

@@ -0,0 +1,51 @@
use std::collections::HashSet;
use std::sync::Arc;
use std::sync::Mutex;
#[derive(Clone, Debug, Default)]
/// Thread-safe store of user approvals so repeated commands can reuse
/// previously granted trust.
pub(crate) struct ApprovalCache {
inner: Arc<Mutex<HashSet<Vec<String>>>>,
}
impl ApprovalCache {
pub(crate) fn insert(&self, command: Vec<String>) {
if command.is_empty() {
return;
}
if let Ok(mut guard) = self.inner.lock() {
guard.insert(command);
}
}
pub(crate) fn snapshot(&self) -> HashSet<Vec<String>> {
self.inner.lock().map(|g| g.clone()).unwrap_or_default()
}
}
#[cfg(test)]
mod tests {
use super::*;
use pretty_assertions::assert_eq;
#[test]
fn insert_ignores_empty_and_dedupes() {
let cache = ApprovalCache::default();
// Empty should be ignored
cache.insert(vec![]);
assert!(cache.snapshot().is_empty());
// Insert a command and verify snapshot contains it
let cmd = vec!["foo".to_string(), "bar".to_string()];
cache.insert(cmd.clone());
let snap1 = cache.snapshot();
assert!(snap1.contains(&cmd));
// Reinserting should not create duplicates
cache.insert(cmd);
let snap2 = cache.snapshot();
assert_eq!(snap1, snap2);
}
}

View File

@@ -0,0 +1,64 @@
mod backends;
mod cache;
mod runner;
mod sandbox;
pub(crate) use backends::ExecutionMode;
pub(crate) use runner::ExecutionRequest;
pub(crate) use runner::Executor;
pub(crate) use runner::ExecutorConfig;
pub(crate) use runner::normalize_exec_result;
pub(crate) mod linkers {
use crate::codex::ExecCommandContext;
use crate::exec::ExecParams;
use crate::exec::StdoutStream;
use crate::executor::backends::ExecutionMode;
use crate::executor::runner::ExecutionRequest;
pub struct PreparedExec {
pub(crate) context: ExecCommandContext,
pub(crate) request: ExecutionRequest,
}
impl PreparedExec {
pub fn new(
context: ExecCommandContext,
params: ExecParams,
approval_command: Vec<String>,
mode: ExecutionMode,
stdout_stream: Option<StdoutStream>,
use_shell_profile: bool,
) -> Self {
let request = ExecutionRequest {
params,
approval_command,
mode,
stdout_stream,
use_shell_profile,
};
Self { context, request }
}
}
}
pub mod errors {
use crate::error::CodexErr;
use crate::function_tool::FunctionCallError;
use thiserror::Error;
#[derive(Debug, Error)]
pub enum ExecError {
#[error(transparent)]
Function(#[from] FunctionCallError),
#[error(transparent)]
Codex(#[from] CodexErr),
}
impl ExecError {
pub(crate) fn rejection(msg: impl Into<String>) -> Self {
FunctionCallError::RespondToModel(msg.into()).into()
}
}
}

Some files were not shown because too many files have changed in this diff Show More