Compare commits

...

10 Commits

Author SHA1 Message Date
pakrym-oai
961a8867f2 Enable supports_parallel_tool_calls for gpt-5 models 2025-11-17 15:27:24 -08:00
Jeremy Rose
ab2e7499f8 core: add a feature to disable the shell tool (#6481)
`--disable shell_tool` disables the built-in shell tool. This is useful
for MCP-only operation.

---------

Co-authored-by: Michael Bolin <mbolin@openai.com>
2025-11-17 22:56:19 +00:00
Dylan Hurd
daf77b8452 chore(core) Update shell instructions (#6679)
## Summary
Consolidates `shell` and `shell_command` tool instructions.
## Testing 
- [x] Updated tests, tested locally
2025-11-17 13:05:15 -08:00
Owen Lin
03a6e853c0 fix: annotate all app server v2 types with camelCase (#6791) 2025-11-17 12:02:52 -08:00
rugvedS07
837bc98a1d LM Studio OSS Support (#2312)
## Overview

Adds LM Studio OSS support. Closes #1883


### Changes
This PR enhances the behavior of `--oss` flag to support LM Studio as a
provider. Additionally, it introduces a new flag`--local-provider` which
can take in `lmstudio` or `ollama` as values if the user wants to
explicitly choose which one to use.

If no provider is specified `codex --oss` will auto-select the provider
based on whichever is running.

#### Additional enhancements 
The default can be set using `oss-provider` in config like:

```
oss_provider = "lmstudio"
```

For non-interactive users, they will need to either provide the provider
as an arg or have it in their `config.toml`

### Notes
For best performance, [set the default context
length](https://lmstudio.ai/docs/app/advanced/per-model) for gpt-oss to
the maximum your machine can support

---------

Co-authored-by: Matt Clayton <matt@lmstudio.ai>
Co-authored-by: Eric Traut <etraut@openai.com>
2025-11-17 11:49:09 -08:00
Celia Chen
842a1b7fe7 [app-server] add events to readme (#6690)
add table of contents, lifecycle and events to readme.
2025-11-17 19:28:05 +00:00
Jeremy Rose
03ffe4d595 core/tui: non-blocking MCP startup (#6334)
This makes MCP startup not block TUI startup. Messages sent while MCPs
are booting will be queued.


https://github.com/user-attachments/assets/96e1d234-5d8f-4932-a935-a675d35c05e0


Fixes #6317

---------

Co-authored-by: pakrym-oai <pakrym@openai.com>
2025-11-17 11:26:11 -08:00
Owen Lin
ae2a084fae chore: delete chatwidget::tests::binary_size_transcript_snapshot tui test (#6759)
We're running into quite a bit of drag maintaining this test, since
every time we add fields to an EventMsg that happened to be dumped into
the `binary-size-log.jsonl` fixture, this test starts to fail. The fix
is usually to either manually update the `binary-size-log.jsonl` fixture
file, or update the `upgrade_event_payload_for_tests` function to map
the data in that file into something workable.

Eason says it's fine to delete this test, so let's just delete it
2025-11-17 11:11:41 -08:00
zhao-oai
a941ae7632 feat: execpolicy v2 (#6467)
## Summary
- Introduces the `codex-execpolicy2` crate.
- This PR covers only the prefix-rule subset of the planned execpolicy
v2 language; a richer language will follow.

## Policy
- Policy language centers on `prefix_rule(pattern=[...], decision?,
match?, not_match?)`, where `pattern` is an ordered list of tokens; any
element may be a list to denote alternatives. `decision` defaults to
`allow`; valid values are `allow`, `prompt`, and `forbidden`. `match` /
`not_match` hold example commands that are tokenized and validated at
load time (think of these as unit tests).

## Policy shapes
- Prefix rules use Starlark syntax:
```starlark
prefix_rule(
    pattern = ["cmd", ["alt1", "alt2"]], # ordered tokens; list entries denote alternatives
    decision = "prompt",                # allow | prompt | forbidden; defaults to allow
    match = [["cmd", "alt1"]],          # examples that must match this rule (enforced at compile time)
    not_match = [["cmd", "oops"]],      # examples that must not match this rule (enforced at compile time)
)
```

## Response shapes
- Match:

```json
{
  "match": {
    "decision": "allow|prompt|forbidden",
    "matchedRules": [
      {
        "prefixRuleMatch": {
          "matchedPrefix": ["<token>", "..."],
          "decision": "allow|prompt|forbidden"
        }
      }
    ]
  }
}
```

- No match:

```json
"noMatch"
```

- `matchedRules` lists every rule whose prefix matched the command;
`matchedPrefix` is the exact prefix that matched.
- The effective `decision` is the strictest severity across all matches
(`forbidden` > `prompt` > `allow`).

---------

Co-authored-by: Michael Bolin <mbolin@openai.com>
2025-11-17 10:15:45 -08:00
jif-oai
2c665fb1dd nit: personal git ignore (#6787) 2025-11-17 17:45:52 +00:00
55 changed files with 3338 additions and 25008 deletions

3
.gitignore vendored
View File

@@ -64,6 +64,9 @@ apply_patch/
# coverage
coverage/
# personal files
personal/
# os
.DS_Store
Thumbs.db

33
codex-rs/Cargo.lock generated
View File

@@ -1061,6 +1061,8 @@ dependencies = [
"clap",
"codex-app-server-protocol",
"codex-core",
"codex-lmstudio",
"codex-ollama",
"codex-protocol",
"once_cell",
"serde",
@@ -1159,7 +1161,6 @@ dependencies = [
"codex-arg0",
"codex-common",
"codex-core",
"codex-ollama",
"codex-protocol",
"core_test_support",
"libc",
@@ -1202,6 +1203,21 @@ dependencies = [
"tempfile",
]
[[package]]
name = "codex-execpolicy2"
version = "0.0.0"
dependencies = [
"anyhow",
"clap",
"multimap",
"pretty_assertions",
"serde",
"serde_json",
"shlex",
"starlark",
"thiserror 2.0.17",
]
[[package]]
name = "codex-feedback"
version = "0.0.0"
@@ -1263,6 +1279,19 @@ dependencies = [
"tokio",
]
[[package]]
name = "codex-lmstudio"
version = "0.0.0"
dependencies = [
"codex-core",
"reqwest",
"serde_json",
"tokio",
"tracing",
"which",
"wiremock",
]
[[package]]
name = "codex-login"
version = "0.0.0"
@@ -1460,7 +1489,6 @@ dependencies = [
"codex-feedback",
"codex-file-search",
"codex-login",
"codex-ollama",
"codex-protocol",
"codex-windows-sandbox",
"color-eyre",
@@ -1483,6 +1511,7 @@ dependencies = [
"ratatui",
"ratatui-macros",
"regex-lite",
"reqwest",
"serde",
"serde_json",
"serial_test",

View File

@@ -17,9 +17,11 @@ members = [
"core",
"exec",
"execpolicy",
"execpolicy2",
"keyring-store",
"file-search",
"linux-sandbox",
"lmstudio",
"login",
"mcp-server",
"mcp-types",
@@ -69,6 +71,7 @@ codex-file-search = { path = "file-search" }
codex-git = { path = "utils/git" }
codex-keyring-store = { path = "keyring-store" }
codex-linux-sandbox = { path = "linux-sandbox" }
codex-lmstudio = { path = "lmstudio" }
codex-login = { path = "login" }
codex-mcp-server = { path = "mcp-server" }
codex-ollama = { path = "ollama" }

View File

@@ -63,6 +63,8 @@ v2_enum_from_core!(
pub enum SandboxPolicy {
DangerFullAccess,
ReadOnly,
#[serde(rename_all = "camelCase")]
#[ts(rename_all = "camelCase")]
WorkspaceWrite {
#[serde(default)]
writable_roots: Vec<PathBuf>,
@@ -506,14 +508,14 @@ impl From<CoreUserInput> for UserInput {
#[ts(tag = "type")]
#[ts(export_to = "v2/")]
pub enum ThreadItem {
UserMessage {
id: String,
content: Vec<UserInput>,
},
AgentMessage {
id: String,
text: String,
},
#[serde(rename_all = "camelCase")]
#[ts(rename_all = "camelCase")]
UserMessage { id: String, content: Vec<UserInput> },
#[serde(rename_all = "camelCase")]
#[ts(rename_all = "camelCase")]
AgentMessage { id: String, text: String },
#[serde(rename_all = "camelCase")]
#[ts(rename_all = "camelCase")]
Reasoning {
id: String,
#[serde(default)]
@@ -521,6 +523,8 @@ pub enum ThreadItem {
#[serde(default)]
content: Vec<String>,
},
#[serde(rename_all = "camelCase")]
#[ts(rename_all = "camelCase")]
CommandExecution {
id: String,
command: String,
@@ -529,11 +533,15 @@ pub enum ThreadItem {
status: CommandExecutionStatus,
duration_ms: Option<i64>,
},
#[serde(rename_all = "camelCase")]
#[ts(rename_all = "camelCase")]
FileChange {
id: String,
changes: Vec<FileUpdateChange>,
status: PatchApplyStatus,
},
#[serde(rename_all = "camelCase")]
#[ts(rename_all = "camelCase")]
McpToolCall {
id: String,
server: String,
@@ -543,22 +551,18 @@ pub enum ThreadItem {
result: Option<McpToolCallResult>,
error: Option<McpToolCallError>,
},
WebSearch {
id: String,
query: String,
},
TodoList {
id: String,
items: Vec<TodoItem>,
},
ImageView {
id: String,
path: String,
},
CodeReview {
id: String,
review: String,
},
#[serde(rename_all = "camelCase")]
#[ts(rename_all = "camelCase")]
WebSearch { id: String, query: String },
#[serde(rename_all = "camelCase")]
#[ts(rename_all = "camelCase")]
TodoList { id: String, items: Vec<TodoItem> },
#[serde(rename_all = "camelCase")]
#[ts(rename_all = "camelCase")]
ImageView { id: String, path: String },
#[serde(rename_all = "camelCase")]
#[ts(rename_all = "camelCase")]
CodeReview { id: String, review: String },
}
impl From<CoreTurnItem> for ThreadItem {

View File

@@ -2,6 +2,16 @@
`codex app-server` is the interface Codex uses to power rich interfaces such as the [Codex VS Code extension](https://marketplace.visualstudio.com/items?itemName=openai.chatgpt). The message schema is currently unstable, but those who wish to build experimental UIs on top of Codex may find it valuable.
## Table of Contents
- [Protocol](#protocol)
- [Message Schema](#message-schema)
- [Lifecycle Overview](#lifecycle-overview)
- [Initialization](#initialization)
- [Core primitives](#core-primitives)
- [Thread & turn endpoints](#thread--turn-endpoints)
- [Auth endpoints](#auth-endpoints)
- [Events (work-in-progress)](#v2-streaming-events-work-in-progress)
## Protocol
Similar to [MCP](https://modelcontextprotocol.io/), `codex app-server` supports bidirectional communication, streaming JSONL over stdio. The protocol is JSON-RPC 2.0, though the `"jsonrpc":"2.0"` header is omitted.
@@ -15,6 +25,14 @@ codex app-server generate-ts --out DIR
codex app-server generate-json-schema --out DIR
```
## Lifecycle Overview
- Initialize once: Immediately after launching the codex app-server process, send an `initialize` request with your client metadata, then emit an `initialized` notification. Any other request before this handshake gets rejected.
- Start (or resume) a thread: Call `thread/start` to open a fresh conversation. The response returns the thread object and youll also get a `thread/started` notification. If youre continuing an existing conversation, call `thread/resume` with its ID instead.
- Begin a turn: To send user input, call `turn/start` with the target `threadId` and the user's input. Optional fields let you override model, cwd, sandbox policy, etc. This immediately returns the new turn object and triggers a `turn/started` notification.
- Stream events: After `turn/start`, keep reading JSON-RPC notifications on stdout. Youll see `item/started`, `item/completed`, deltas like `item/agentMessage/delta`, tool progress, etc. These represent streaming model output plus any side effects (commands, tool calls, reasoning notes).
- Finish the turn: When the model is done (or the turn is interrupted via making the `turn/interrupt` call), the server sends `turn/completed` with the final turn state and token usage.
## Initialization
Clients must send a single `initialize` request before invoking any other method, then acknowledge with an `initialized` notification. The server returns the user agent string it will present to upstream services; subsequent requests issued before initialization receive a `"Not initialized"` error, and repeated `initialize` calls receive an `"Already initialized"` error.
@@ -258,3 +276,33 @@ Field notes:
- `codex app-server generate-ts --out <dir>` emits v2 types under `v2/`.
- `codex app-server generate-json-schema --out <dir>` outputs `codex_app_server_protocol.schemas.json`.
- See [“Authentication and authorization” in the config docs](../../docs/config.md#authentication-and-authorization) for configuration knobs.
## Events (work-in-progress)
Event notifications are the server-initiated event stream for thread lifecycles, turn lifecycles, and the items within them. After you start or resume a thread, keep reading stdout for `thread/started`, `turn/*`, and `item/*` notifications.
### Turn events
The app-server streams JSON-RPC notifications while a turn is running. Each turn starts with `turn/started` (initial `turn`) and ends with `turn/completed` (final `turn` plus token `usage`), and clients subscribe to the events they care about, rendering each item incrementally as updates arrive. The per-item lifecycle is always: `item/started` → zero or more item-specific deltas → `item/completed`.
#### Thread items
`ThreadItem` is the tagged union carried in turn responses and `item/*` notifications. Currently we support events for the following items:
- `userMessage` — `{id, content}` where `content` is a list of user inputs (`text`, `image`, or `localImage`).
- `agentMessage` — `{id, text}` containing the accumulated agent reply.
- `reasoning` — `{id, summary, content}` where `summary` holds streamed reasoning summaries (applicable for most OpenAI models) and `content` holds raw reasoning blocks (applicable for e.g. open source models).
- `mcpToolCall` — `{id, server, tool, status, arguments, result?, error?}` describing MCP calls; `status` is `inProgress`, `completed`, or `failed`.
- `webSearch` — `{id, query}` for a web search request issued by the agent.
All items emit two shared lifecycle events:
- `item/started` — emits the full `item` when a new unit of work begins so the UI can render it immediately; the `item.id` in this payload matches the `itemId` used by deltas.
- `item/completed` — sends the final `item` once that work finishes (e.g., after a tool call or message completes); treat this as the authoritative state.
There are additional item-specific events:
#### agentMessage
- `item/agentMessage/delta` — appends streamed text for the agent message; concatenate `delta` values for the same `itemId` in order to reconstruct the full reply.
#### reasoning
- `item/reasoning/summaryTextDelta` — streams readable reasoning summaries; `summaryIndex` increments when a new summary section opens.
- `item/reasoning/summaryPartAdded` — marks the boundary between reasoning summary sections for an `itemId`; subsequent `summaryTextDelta` entries share the same `summaryIndex`.
- `item/reasoning/textDelta` — streams raw reasoning text (only applicable for e.g. open source models); use `contentIndex` to group deltas that belong together before showing them in the UI.

View File

@@ -10,6 +10,8 @@ workspace = true
clap = { workspace = true, features = ["derive", "wrap_help"], optional = true }
codex-app-server-protocol = { workspace = true }
codex-core = { workspace = true }
codex-lmstudio = { workspace = true }
codex-ollama = { workspace = true }
codex-protocol = { workspace = true }
once_cell = { workspace = true }
serde = { workspace = true, optional = true }

View File

@@ -37,3 +37,5 @@ pub mod model_presets;
// Shared approval presets (AskForApproval + Sandbox) used by TUI and MCP server
// Not to be confused with AskForApproval, which we should probably rename to EscalationPolicy.
pub mod approval_presets;
// Shared OSS provider utilities used by TUI and exec
pub mod oss;

View File

@@ -0,0 +1,60 @@
//! OSS provider utilities shared between TUI and exec.
use codex_core::LMSTUDIO_OSS_PROVIDER_ID;
use codex_core::OLLAMA_OSS_PROVIDER_ID;
use codex_core::config::Config;
/// Returns the default model for a given OSS provider.
pub fn get_default_model_for_oss_provider(provider_id: &str) -> Option<&'static str> {
match provider_id {
LMSTUDIO_OSS_PROVIDER_ID => Some(codex_lmstudio::DEFAULT_OSS_MODEL),
OLLAMA_OSS_PROVIDER_ID => Some(codex_ollama::DEFAULT_OSS_MODEL),
_ => None,
}
}
/// Ensures the specified OSS provider is ready (models downloaded, service reachable).
pub async fn ensure_oss_provider_ready(
provider_id: &str,
config: &Config,
) -> Result<(), std::io::Error> {
match provider_id {
LMSTUDIO_OSS_PROVIDER_ID => {
codex_lmstudio::ensure_oss_ready(config)
.await
.map_err(|e| std::io::Error::other(format!("OSS setup failed: {e}")))?;
}
OLLAMA_OSS_PROVIDER_ID => {
codex_ollama::ensure_oss_ready(config)
.await
.map_err(|e| std::io::Error::other(format!("OSS setup failed: {e}")))?;
}
_ => {
// Unknown provider, skip setup
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_get_default_model_for_provider_lmstudio() {
let result = get_default_model_for_oss_provider(LMSTUDIO_OSS_PROVIDER_ID);
assert_eq!(result, Some(codex_lmstudio::DEFAULT_OSS_MODEL));
}
#[test]
fn test_get_default_model_for_provider_ollama() {
let result = get_default_model_for_oss_provider(OLLAMA_OSS_PROVIDER_ID);
assert_eq!(result, Some(codex_ollama::DEFAULT_OSS_MODEL));
}
#[test]
fn test_get_default_model_for_provider_unknown() {
let result = get_default_model_for_oss_provider("unknown-provider");
assert_eq!(result, None);
}
}

View File

@@ -318,8 +318,6 @@ For casual greetings, acknowledgements, or other one-off conversational messages
When using the shell, you must adhere to the following guidelines:
- The arguments to `shell` will be passed to execvp().
- Always set the `workdir` param when using the shell function. Do not use `cd` unless absolutely necessary.
- When searching for text or files, prefer using `rg` or `rg --files` respectively because `rg` is much faster than alternatives like `grep`. (If the `rg` command is not found, then use alternatives.)
- Read files in chunks with a max chunk size of 250 lines. Do not use python scripts to attempt to output larger chunks of a file. Command line output will be truncated after 10 kilobytes or 256 lines of output, regardless of the command used.

View File

@@ -2,8 +2,6 @@ You are Codex, based on GPT-5. You are running as a coding agent in the Codex CL
## General
- The arguments to `shell` will be passed to execvp(). Most terminal commands should be prefixed with ["bash", "-lc"].
- Always set the `workdir` param when using the shell function. Do not use `cd` unless absolutely necessary.
- When searching for text or files, prefer using `rg` or `rg --files` respectively because `rg` is much faster than alternatives like `grep`. (If the `rg` command is not found, then use alternatives.)
## Editing constraints

View File

@@ -9,8 +9,6 @@ use crate::client_common::REVIEW_PROMPT;
use crate::compact;
use crate::features::Feature;
use crate::function_tool::FunctionCallError;
use crate::mcp::auth::McpAuthStatusEntry;
use crate::mcp_connection_manager::DEFAULT_STARTUP_TIMEOUT;
use crate::parse_command::parse_command;
use crate::parse_turn_item;
use crate::response_processing::process_items;
@@ -45,6 +43,7 @@ use mcp_types::ReadResourceResult;
use serde_json;
use serde_json::Value;
use tokio::sync::Mutex;
use tokio::sync::RwLock;
use tokio::sync::oneshot;
use tokio_util::sync::CancellationToken;
use tracing::debug;
@@ -57,7 +56,6 @@ use crate::client::ModelClient;
use crate::client_common::Prompt;
use crate::client_common::ResponseEvent;
use crate::config::Config;
use crate::config::types::McpServerTransportConfig;
use crate::config::types::ShellEnvironmentPolicy;
use crate::context_manager::ContextManager;
use crate::environment_context::EnvironmentContext;
@@ -476,21 +474,13 @@ impl Session {
),
};
// Error messages to dispatch after SessionConfigured is sent.
let mut post_session_configured_events = Vec::<Event>::new();
// Kick off independent async setup tasks in parallel to reduce startup latency.
//
// - initialize RolloutRecorder with new or resumed session info
// - spin up MCP connection manager
// - perform default shell discovery
// - load history metadata
let rollout_fut = RolloutRecorder::new(&config, rollout_params);
let mcp_fut = McpConnectionManager::new(
config.mcp_servers.clone(),
config.mcp_oauth_credentials_store_mode,
);
let default_shell_fut = shell::default_user_shell();
let history_meta_fut = crate::message_history::history_metadata(&config);
let auth_statuses_fut = compute_auth_statuses(
@@ -499,15 +489,8 @@ impl Session {
);
// Join all independent futures.
let (
rollout_recorder,
mcp_res,
default_shell,
(history_log_id, history_entry_count),
auth_statuses,
) = tokio::join!(
let (rollout_recorder, default_shell, (history_log_id, history_entry_count), auth_statuses) = tokio::join!(
rollout_fut,
mcp_fut,
default_shell_fut,
history_meta_fut,
auth_statuses_fut
@@ -519,34 +502,7 @@ impl Session {
})?;
let rollout_path = rollout_recorder.rollout_path.clone();
// Handle MCP manager result and record any startup failures.
let (mcp_connection_manager, failed_clients) = match mcp_res {
Ok((mgr, failures)) => (mgr, failures),
Err(e) => {
let message = format!("Failed to create MCP connection manager: {e:#}");
error!("{message}");
post_session_configured_events.push(Event {
id: INITIAL_SUBMIT_ID.to_owned(),
msg: EventMsg::Error(ErrorEvent { message }),
});
(McpConnectionManager::default(), Default::default())
}
};
// Surface individual client start-up failures to the user.
if !failed_clients.is_empty() {
for (server_name, err) in failed_clients {
let auth_entry = auth_statuses.get(&server_name);
let display_message = mcp_init_error_display(&server_name, auth_entry, &err);
warn!("MCP client for `{server_name}` failed to start: {err:#}");
post_session_configured_events.push(Event {
id: INITIAL_SUBMIT_ID.to_owned(),
msg: EventMsg::Error(ErrorEvent {
message: display_message,
}),
});
}
}
let mut post_session_configured_events = Vec::<Event>::new();
for (alias, feature) in session_configuration.features.legacy_feature_usages() {
let canonical = feature.key();
@@ -595,7 +551,8 @@ impl Session {
warm_model_cache(&session_configuration.model);
let services = SessionServices {
mcp_connection_manager,
mcp_connection_manager: Arc::new(RwLock::new(McpConnectionManager::default())),
mcp_startup_cancellation_token: CancellationToken::new(),
unified_exec_manager: UnifiedExecSessionManager::default(),
notifier: UserNotifier::new(config.notify.clone()),
rollout: Mutex::new(Some(rollout_recorder)),
@@ -635,6 +592,18 @@ impl Session {
for event in events {
sess.send_event_raw(event).await;
}
sess.services
.mcp_connection_manager
.write()
.await
.initialize(
config.mcp_servers.clone(),
config.mcp_oauth_credentials_store_mode,
auth_statuses.clone(),
tx_event.clone(),
sess.services.mcp_startup_cancellation_token.clone(),
)
.await;
// record_initial_history can emit events. We record only after the SessionConfiguredEvent is emitted.
sess.record_initial_history(initial_history).await;
@@ -1258,6 +1227,8 @@ impl Session {
) -> anyhow::Result<ListResourcesResult> {
self.services
.mcp_connection_manager
.read()
.await
.list_resources(server, params)
.await
}
@@ -1269,6 +1240,8 @@ impl Session {
) -> anyhow::Result<ListResourceTemplatesResult> {
self.services
.mcp_connection_manager
.read()
.await
.list_resource_templates(server, params)
.await
}
@@ -1280,6 +1253,8 @@ impl Session {
) -> anyhow::Result<ReadResourceResult> {
self.services
.mcp_connection_manager
.read()
.await
.read_resource(server, params)
.await
}
@@ -1292,19 +1267,29 @@ impl Session {
) -> anyhow::Result<CallToolResult> {
self.services
.mcp_connection_manager
.read()
.await
.call_tool(server, tool, arguments)
.await
}
pub(crate) fn parse_mcp_tool_name(&self, tool_name: &str) -> Option<(String, String)> {
pub(crate) async fn parse_mcp_tool_name(&self, tool_name: &str) -> Option<(String, String)> {
self.services
.mcp_connection_manager
.read()
.await
.parse_tool_name(tool_name)
.await
}
pub async fn interrupt_task(self: &Arc<Self>) {
info!("interrupt received: abort current task, if any");
self.abort_all_tasks(TurnAbortReason::Interrupted).await;
let has_active_turn = { self.active_turn.lock().await.is_some() };
if has_active_turn {
self.abort_all_tasks(TurnAbortReason::Interrupted).await;
} else {
self.cancel_mcp_startup().await;
}
}
pub(crate) fn notifier(&self) -> &UserNotifier {
@@ -1318,6 +1303,10 @@ impl Session {
fn show_raw_agent_reasoning(&self) -> bool {
self.services.show_raw_agent_reasoning
}
async fn cancel_mcp_startup(&self) {
self.services.mcp_startup_cancellation_token.cancel();
}
}
async fn submission_loop(sess: Arc<Session>, config: Arc<Config>, rx_sub: Receiver<Submission>) {
@@ -1575,17 +1564,15 @@ mod handlers {
}
pub async fn list_mcp_tools(sess: &Session, config: &Arc<Config>, sub_id: String) {
// This is a cheap lookup from the connection manager's cache.
let tools = sess.services.mcp_connection_manager.list_all_tools();
let (auth_status_entries, resources, resource_templates) = tokio::join!(
let mcp_connection_manager = sess.services.mcp_connection_manager.read().await;
let (tools, auth_status_entries, resources, resource_templates) = tokio::join!(
mcp_connection_manager.list_all_tools(),
compute_auth_statuses(
config.mcp_servers.iter(),
config.mcp_oauth_credentials_store_mode,
),
sess.services.mcp_connection_manager.list_all_resources(),
sess.services
.mcp_connection_manager
.list_all_resource_templates()
mcp_connection_manager.list_all_resources(),
mcp_connection_manager.list_all_resource_templates(),
);
let auth_statuses = auth_status_entries
.iter()
@@ -1594,7 +1581,10 @@ mod handlers {
let event = Event {
id: sub_id,
msg: EventMsg::McpListToolsResponse(crate::protocol::McpListToolsResponseEvent {
tools,
tools: tools
.into_iter()
.map(|(name, tool)| (name, tool.tool))
.collect(),
resources,
resource_templates,
auth_statuses,
@@ -1924,10 +1914,22 @@ async fn run_turn(
input: Vec<ResponseItem>,
cancellation_token: CancellationToken,
) -> CodexResult<TurnRunResult> {
let mcp_tools = sess.services.mcp_connection_manager.list_all_tools();
let mcp_tools = sess
.services
.mcp_connection_manager
.read()
.await
.list_all_tools()
.or_cancel(&cancellation_token)
.await?;
let router = Arc::new(ToolRouter::from_config(
&turn_context.tools_config,
Some(mcp_tools),
Some(
mcp_tools
.into_iter()
.map(|(name, tool)| (name, tool.tool))
.collect(),
),
));
let model_supports_parallel = turn_context
@@ -2096,7 +2098,7 @@ async fn try_run_turn(
ResponseEvent::Created => {}
ResponseEvent::OutputItemDone(item) => {
let previously_active_item = active_item.take();
match ToolRouter::build_tool_call(sess.as_ref(), item.clone()) {
match ToolRouter::build_tool_call(sess.as_ref(), item.clone()).await {
Ok(Some(call)) => {
let payload_preview = call.payload.log_payload().into_owned();
tracing::info!("ToolCall: {} {}", call.tool_name, payload_preview);
@@ -2307,59 +2309,6 @@ pub(super) fn get_last_assistant_message_from_turn(responses: &[ResponseItem]) -
})
}
fn mcp_init_error_display(
server_name: &str,
entry: Option<&McpAuthStatusEntry>,
err: &anyhow::Error,
) -> String {
if let Some(McpServerTransportConfig::StreamableHttp {
url,
bearer_token_env_var,
http_headers,
..
}) = &entry.map(|entry| &entry.config.transport)
&& url == "https://api.githubcopilot.com/mcp/"
&& bearer_token_env_var.is_none()
&& http_headers.as_ref().map(HashMap::is_empty).unwrap_or(true)
{
// GitHub only supports OAUth for first party MCP clients.
// That means that the user has to specify a personal access token either via bearer_token_env_var or http_headers.
// https://github.com/github/github-mcp-server/issues/921#issuecomment-3221026448
format!(
"GitHub MCP does not support OAuth. Log in by adding a personal access token (https://github.com/settings/personal-access-tokens) to your environment and config.toml:\n[mcp_servers.{server_name}]\nbearer_token_env_var = CODEX_GITHUB_PERSONAL_ACCESS_TOKEN"
)
} else if is_mcp_client_auth_required_error(err) {
format!(
"The {server_name} MCP server is not logged in. Run `codex mcp login {server_name}`."
)
} else if is_mcp_client_startup_timeout_error(err) {
let startup_timeout_secs = match entry {
Some(entry) => match entry.config.startup_timeout_sec {
Some(timeout) => timeout,
None => DEFAULT_STARTUP_TIMEOUT,
},
None => DEFAULT_STARTUP_TIMEOUT,
}
.as_secs();
format!(
"MCP client for `{server_name}` timed out after {startup_timeout_secs} seconds. Add or adjust `startup_timeout_sec` in your config.toml:\n[mcp_servers.{server_name}]\nstartup_timeout_sec = XX"
)
} else {
format!("MCP client for `{server_name}` failed to start: {err:#}")
}
}
fn is_mcp_client_auth_required_error(error: &anyhow::Error) -> bool {
// StreamableHttpError::AuthRequired from the MCP SDK.
error.to_string().contains("Auth required")
}
fn is_mcp_client_startup_timeout_error(error: &anyhow::Error) -> bool {
let error_message = error.to_string();
error_message.contains("request timed out")
|| error_message.contains("timed out handshaking with MCP server")
}
use crate::features::Features;
#[cfg(test)]
pub(crate) use tests::make_session_and_context;
@@ -2369,10 +2318,7 @@ mod tests {
use super::*;
use crate::config::ConfigOverrides;
use crate::config::ConfigToml;
use crate::config::types::McpServerConfig;
use crate::config::types::McpServerTransportConfig;
use crate::exec::ExecToolCallOutput;
use crate::mcp::auth::McpAuthStatusEntry;
use crate::tools::format_exec_output_str;
use crate::protocol::CompactedItem;
@@ -2392,7 +2338,6 @@ mod tests {
use codex_app_server_protocol::AuthMode;
use codex_protocol::models::ContentItem;
use codex_protocol::models::ResponseItem;
use codex_protocol::protocol::McpAuthStatus;
use std::time::Duration;
use tokio::time::sleep;
@@ -2606,7 +2551,8 @@ mod tests {
let state = SessionState::new(session_configuration.clone());
let services = SessionServices {
mcp_connection_manager: McpConnectionManager::default(),
mcp_connection_manager: Arc::new(RwLock::new(McpConnectionManager::default())),
mcp_startup_cancellation_token: CancellationToken::new(),
unified_exec_manager: UnifiedExecSessionManager::default(),
notifier: UserNotifier::new(None),
rollout: Mutex::new(None),
@@ -2682,7 +2628,8 @@ mod tests {
let state = SessionState::new(session_configuration.clone());
let services = SessionServices {
mcp_connection_manager: McpConnectionManager::default(),
mcp_connection_manager: Arc::new(RwLock::new(McpConnectionManager::default())),
mcp_startup_cancellation_token: CancellationToken::new(),
unified_exec_manager: UnifiedExecSessionManager::default(),
notifier: UserNotifier::new(None),
rollout: Mutex::new(None),
@@ -2863,9 +2810,23 @@ mod tests {
#[tokio::test]
async fn fatal_tool_error_stops_turn_and_reports_error() {
let (session, turn_context, _rx) = make_session_and_context_with_rx();
let tools = {
session
.services
.mcp_connection_manager
.read()
.await
.list_all_tools()
.await
};
let router = ToolRouter::from_config(
&turn_context.tools_config,
Some(session.services.mcp_connection_manager.list_all_tools()),
Some(
tools
.into_iter()
.map(|(name, tool)| (name, tool.tool))
.collect(),
),
);
let item = ResponseItem::CustomToolCall {
id: None,
@@ -2876,6 +2837,7 @@ mod tests {
};
let call = ToolRouter::build_tool_call(session.as_ref(), item.clone())
.await
.expect("build tool call")
.expect("tool call present");
let tracker = Arc::new(tokio::sync::Mutex::new(TurnDiffTracker::new()));
@@ -3125,7 +3087,6 @@ mod tests {
pretty_assertions::assert_eq!(exec_output.metadata, ResponseExecMetadata { exit_code: 0 });
assert!(exec_output.output.contains("hi"));
}
#[tokio::test]
async fn unified_exec_rejects_escalated_permissions_when_policy_not_on_request() {
use crate::protocol::AskForApproval;
@@ -3167,89 +3128,4 @@ mod tests {
pretty_assertions::assert_eq!(output, expected);
}
#[test]
fn mcp_init_error_display_prompts_for_github_pat() {
let server_name = "github";
let entry = McpAuthStatusEntry {
config: McpServerConfig {
transport: McpServerTransportConfig::StreamableHttp {
url: "https://api.githubcopilot.com/mcp/".to_string(),
bearer_token_env_var: None,
http_headers: None,
env_http_headers: None,
},
enabled: true,
startup_timeout_sec: None,
tool_timeout_sec: None,
enabled_tools: None,
disabled_tools: None,
},
auth_status: McpAuthStatus::Unsupported,
};
let err = anyhow::anyhow!("OAuth is unsupported");
let display = mcp_init_error_display(server_name, Some(&entry), &err);
let expected = format!(
"GitHub MCP does not support OAuth. Log in by adding a personal access token (https://github.com/settings/personal-access-tokens) to your environment and config.toml:\n[mcp_servers.{server_name}]\nbearer_token_env_var = CODEX_GITHUB_PERSONAL_ACCESS_TOKEN"
);
assert_eq!(expected, display);
}
#[test]
fn mcp_init_error_display_prompts_for_login_when_auth_required() {
let server_name = "example";
let err = anyhow::anyhow!("Auth required for server");
let display = mcp_init_error_display(server_name, None, &err);
let expected = format!(
"The {server_name} MCP server is not logged in. Run `codex mcp login {server_name}`."
);
assert_eq!(expected, display);
}
#[test]
fn mcp_init_error_display_reports_generic_errors() {
let server_name = "custom";
let entry = McpAuthStatusEntry {
config: McpServerConfig {
transport: McpServerTransportConfig::StreamableHttp {
url: "https://example.com".to_string(),
bearer_token_env_var: Some("TOKEN".to_string()),
http_headers: None,
env_http_headers: None,
},
enabled: true,
startup_timeout_sec: None,
tool_timeout_sec: None,
enabled_tools: None,
disabled_tools: None,
},
auth_status: McpAuthStatus::Unsupported,
};
let err = anyhow::anyhow!("boom");
let display = mcp_init_error_display(server_name, Some(&entry), &err);
let expected = format!("MCP client for `{server_name}` failed to start: {err:#}");
assert_eq!(expected, display);
}
#[test]
fn mcp_init_error_display_includes_startup_timeout_hint() {
let server_name = "slow";
let err = anyhow::anyhow!("request timed out");
let display = mcp_init_error_display(server_name, None, &err);
assert_eq!(
"MCP client for `slow` timed out after 10 seconds. Add or adjust `startup_timeout_sec` in your config.toml:\n[mcp_servers.slow]\nstartup_timeout_sec = XX",
display
);
}
}

View File

@@ -25,7 +25,9 @@ use crate::git_info::resolve_root_git_project_for_trust;
use crate::model_family::ModelFamily;
use crate::model_family::derive_default_model_family;
use crate::model_family::find_family_for_model;
use crate::model_provider_info::LMSTUDIO_OSS_PROVIDER_ID;
use crate::model_provider_info::ModelProviderInfo;
use crate::model_provider_info::OLLAMA_OSS_PROVIDER_ID;
use crate::model_provider_info::built_in_model_providers;
use crate::openai_model_info::get_model_info;
use crate::project_doc::DEFAULT_PROJECT_DOC_FILENAME;
@@ -466,6 +468,48 @@ pub fn set_project_trust_level(
.apply_blocking()
}
/// Save the default OSS provider preference to config.toml
pub fn set_default_oss_provider(codex_home: &Path, provider: &str) -> std::io::Result<()> {
// Validate that the provider is one of the known OSS providers
match provider {
LMSTUDIO_OSS_PROVIDER_ID | OLLAMA_OSS_PROVIDER_ID => {
// Valid provider, continue
}
_ => {
return Err(std::io::Error::new(
std::io::ErrorKind::InvalidInput,
format!(
"Invalid OSS provider '{provider}'. Must be one of: {LMSTUDIO_OSS_PROVIDER_ID}, {OLLAMA_OSS_PROVIDER_ID}"
),
));
}
}
let config_path = codex_home.join(CONFIG_TOML_FILE);
// Read existing config or create empty string if file doesn't exist
let content = match std::fs::read_to_string(&config_path) {
Ok(content) => content,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => String::new(),
Err(e) => return Err(e),
};
// Parse as DocumentMut for editing while preserving structure
let mut doc = content.parse::<DocumentMut>().map_err(|e| {
std::io::Error::new(
std::io::ErrorKind::InvalidData,
format!("failed to parse config.toml: {e}"),
)
})?;
// Set the default_oss_provider at root level
use toml_edit::value;
doc["oss_provider"] = value(provider);
// Write the modified document back
std::fs::write(&config_path, doc.to_string())?;
Ok(())
}
/// Apply a single dotted-path override onto a TOML value.
fn apply_toml_override(root: &mut TomlValue, path: &str, value: TomlValue) {
use toml::value::Table;
@@ -663,6 +707,8 @@ pub struct ConfigToml {
pub experimental_use_rmcp_client: Option<bool>,
pub experimental_use_freeform_apply_patch: Option<bool>,
pub experimental_sandbox_command_assessment: Option<bool>,
/// Preferred OSS provider for local models, e.g. "lmstudio" or "ollama".
pub oss_provider: Option<String>,
}
impl From<ConfigToml> for UserSavedConfig {
@@ -851,6 +897,34 @@ pub struct ConfigOverrides {
pub additional_writable_roots: Vec<PathBuf>,
}
/// Resolves the OSS provider from CLI override, profile config, or global config.
/// Returns `None` if no provider is configured at any level.
pub fn resolve_oss_provider(
explicit_provider: Option<&str>,
config_toml: &ConfigToml,
config_profile: Option<String>,
) -> Option<String> {
if let Some(provider) = explicit_provider {
// Explicit provider specified (e.g., via --local-provider)
Some(provider.to_string())
} else {
// Check profile config first, then global config
let profile = config_toml.get_config_profile(config_profile).ok();
if let Some(profile) = &profile {
// Check if profile has an oss provider
if let Some(profile_oss_provider) = &profile.oss_provider {
Some(profile_oss_provider.clone())
}
// If not then check if the toml has an oss provider
else {
config_toml.oss_provider.clone()
}
} else {
config_toml.oss_provider.clone()
}
}
}
impl Config {
/// Meant to be used exclusively for tests: `load_with_overrides()` should
/// be used in all other cases.
@@ -3265,6 +3339,41 @@ trust_level = "trusted"
Ok(())
}
#[test]
fn test_set_default_oss_provider() -> std::io::Result<()> {
let temp_dir = TempDir::new()?;
let codex_home = temp_dir.path();
let config_path = codex_home.join(CONFIG_TOML_FILE);
// Test setting valid provider on empty config
set_default_oss_provider(codex_home, OLLAMA_OSS_PROVIDER_ID)?;
let content = std::fs::read_to_string(&config_path)?;
assert!(content.contains("oss_provider = \"ollama\""));
// Test updating existing config
std::fs::write(&config_path, "model = \"gpt-4\"\n")?;
set_default_oss_provider(codex_home, LMSTUDIO_OSS_PROVIDER_ID)?;
let content = std::fs::read_to_string(&config_path)?;
assert!(content.contains("oss_provider = \"lmstudio\""));
assert!(content.contains("model = \"gpt-4\""));
// Test overwriting existing oss_provider
set_default_oss_provider(codex_home, OLLAMA_OSS_PROVIDER_ID)?;
let content = std::fs::read_to_string(&config_path)?;
assert!(content.contains("oss_provider = \"ollama\""));
assert!(!content.contains("oss_provider = \"lmstudio\""));
// Test invalid provider
let result = set_default_oss_provider(codex_home, "invalid_provider");
assert!(result.is_err());
let error = result.unwrap_err();
assert_eq!(error.kind(), std::io::ErrorKind::InvalidInput);
assert!(error.to_string().contains("Invalid OSS provider"));
assert!(error.to_string().contains("invalid_provider"));
Ok(())
}
#[test]
fn test_untrusted_project_gets_workspace_write_sandbox() -> anyhow::Result<()> {
let config_with_untrusted = r#"
@@ -3295,6 +3404,85 @@ trust_level = "untrusted"
Ok(())
}
#[test]
fn test_resolve_oss_provider_explicit_override() {
let config_toml = ConfigToml::default();
let result = resolve_oss_provider(Some("custom-provider"), &config_toml, None);
assert_eq!(result, Some("custom-provider".to_string()));
}
#[test]
fn test_resolve_oss_provider_from_profile() {
let mut profiles = std::collections::HashMap::new();
let profile = ConfigProfile {
oss_provider: Some("profile-provider".to_string()),
..Default::default()
};
profiles.insert("test-profile".to_string(), profile);
let config_toml = ConfigToml {
profiles,
..Default::default()
};
let result = resolve_oss_provider(None, &config_toml, Some("test-profile".to_string()));
assert_eq!(result, Some("profile-provider".to_string()));
}
#[test]
fn test_resolve_oss_provider_from_global_config() {
let config_toml = ConfigToml {
oss_provider: Some("global-provider".to_string()),
..Default::default()
};
let result = resolve_oss_provider(None, &config_toml, None);
assert_eq!(result, Some("global-provider".to_string()));
}
#[test]
fn test_resolve_oss_provider_profile_fallback_to_global() {
let mut profiles = std::collections::HashMap::new();
let profile = ConfigProfile::default(); // No oss_provider set
profiles.insert("test-profile".to_string(), profile);
let config_toml = ConfigToml {
oss_provider: Some("global-provider".to_string()),
profiles,
..Default::default()
};
let result = resolve_oss_provider(None, &config_toml, Some("test-profile".to_string()));
assert_eq!(result, Some("global-provider".to_string()));
}
#[test]
fn test_resolve_oss_provider_none_when_not_configured() {
let config_toml = ConfigToml::default();
let result = resolve_oss_provider(None, &config_toml, None);
assert_eq!(result, None);
}
#[test]
fn test_resolve_oss_provider_explicit_overrides_all() {
let mut profiles = std::collections::HashMap::new();
let profile = ConfigProfile {
oss_provider: Some("profile-provider".to_string()),
..Default::default()
};
profiles.insert("test-profile".to_string(), profile);
let config_toml = ConfigToml {
oss_provider: Some("global-provider".to_string()),
profiles,
..Default::default()
};
let result = resolve_oss_provider(
Some("explicit-provider"),
&config_toml,
Some("test-profile".to_string()),
);
assert_eq!(result, Some("explicit-provider".to_string()));
}
#[test]
fn test_untrusted_project_gets_unless_trusted_approval_policy() -> std::io::Result<()> {
let codex_home = TempDir::new()?;

View File

@@ -33,6 +33,7 @@ pub struct ConfigProfile {
/// Optional feature toggles scoped to this profile.
#[serde(default)]
pub features: Option<crate::features::FeaturesToml>,
pub oss_provider: Option<String>,
}
impl From<ConfigProfile> for codex_app_server_protocol::Profile {

View File

@@ -46,6 +46,8 @@ pub enum Feature {
GhostCommit,
/// Enable Windows sandbox (restricted token) on Windows.
WindowsSandbox,
/// Enable the default shell tool.
ShellTool,
}
impl Feature {
@@ -301,4 +303,10 @@ pub const FEATURES: &[FeatureSpec] = &[
stage: Stage::Experimental,
default_enabled: false,
},
FeatureSpec {
id: Feature::ShellTool,
key: "shell_tool",
stage: Stage::Stable,
default_enabled: true,
},
];

View File

@@ -40,8 +40,11 @@ pub mod token_data;
mod truncate;
mod unified_exec;
mod user_instructions;
pub use model_provider_info::BUILT_IN_OSS_MODEL_PROVIDER_ID;
pub use model_provider_info::DEFAULT_LMSTUDIO_PORT;
pub use model_provider_info::DEFAULT_OLLAMA_PORT;
pub use model_provider_info::LMSTUDIO_OSS_PROVIDER_ID;
pub use model_provider_info::ModelProviderInfo;
pub use model_provider_info::OLLAMA_OSS_PROVIDER_ID;
pub use model_provider_info::WireApi;
pub use model_provider_info::built_in_model_providers;
pub use model_provider_info::create_oss_provider_with_base_url;

File diff suppressed because it is too large Load Diff

View File

@@ -177,6 +177,7 @@ pub fn find_family_for_model(slug: &str) -> Option<ModelFamily> {
base_instructions: GPT_5_CODEX_INSTRUCTIONS.to_string(),
apply_patch_tool_type: Some(ApplyPatchToolType::Freeform),
support_verbosity: false,
supports_parallel_tool_calls: true,
)
} else if slug.starts_with("gpt-5.1") {
model_family!(
@@ -187,6 +188,7 @@ pub fn find_family_for_model(slug: &str) -> Option<ModelFamily> {
default_verbosity: Some(Verbosity::Low),
base_instructions: GPT_5_1_INSTRUCTIONS.to_string(),
default_reasoning_effort: Some(ReasoningEffort::Medium),
supports_parallel_tool_calls: true,
)
} else if slug.starts_with("gpt-5") {
model_family!(

View File

@@ -258,9 +258,11 @@ impl ModelProviderInfo {
}
}
const DEFAULT_OLLAMA_PORT: u32 = 11434;
pub const DEFAULT_LMSTUDIO_PORT: u16 = 1234;
pub const DEFAULT_OLLAMA_PORT: u16 = 11434;
pub const BUILT_IN_OSS_MODEL_PROVIDER_ID: &str = "oss";
pub const LMSTUDIO_OSS_PROVIDER_ID: &str = "lmstudio";
pub const OLLAMA_OSS_PROVIDER_ID: &str = "ollama";
/// Built-in default provider list.
pub fn built_in_model_providers() -> HashMap<String, ModelProviderInfo> {
@@ -311,14 +313,21 @@ pub fn built_in_model_providers() -> HashMap<String, ModelProviderInfo> {
requires_openai_auth: true,
},
),
(BUILT_IN_OSS_MODEL_PROVIDER_ID, create_oss_provider()),
(
OLLAMA_OSS_PROVIDER_ID,
create_oss_provider(DEFAULT_OLLAMA_PORT, WireApi::Chat),
),
(
LMSTUDIO_OSS_PROVIDER_ID,
create_oss_provider(DEFAULT_LMSTUDIO_PORT, WireApi::Responses),
),
]
.into_iter()
.map(|(k, v)| (k.to_string(), v))
.collect()
}
pub fn create_oss_provider() -> ModelProviderInfo {
pub fn create_oss_provider(default_provider_port: u16, wire_api: WireApi) -> ModelProviderInfo {
// These CODEX_OSS_ environment variables are experimental: we may
// switch to reading values from config.toml instead.
let codex_oss_base_url = match std::env::var("CODEX_OSS_BASE_URL")
@@ -331,22 +340,21 @@ pub fn create_oss_provider() -> ModelProviderInfo {
port = std::env::var("CODEX_OSS_PORT")
.ok()
.filter(|v| !v.trim().is_empty())
.and_then(|v| v.parse::<u32>().ok())
.unwrap_or(DEFAULT_OLLAMA_PORT)
.and_then(|v| v.parse::<u16>().ok())
.unwrap_or(default_provider_port)
),
};
create_oss_provider_with_base_url(&codex_oss_base_url)
create_oss_provider_with_base_url(&codex_oss_base_url, wire_api)
}
pub fn create_oss_provider_with_base_url(base_url: &str) -> ModelProviderInfo {
pub fn create_oss_provider_with_base_url(base_url: &str, wire_api: WireApi) -> ModelProviderInfo {
ModelProviderInfo {
name: "gpt-oss".into(),
base_url: Some(base_url.into()),
env_key: None,
env_key_instructions: None,
experimental_bearer_token: None,
wire_api: WireApi::Chat,
wire_api,
query_params: None,
http_headers: None,
env_http_headers: None,

View File

@@ -72,6 +72,8 @@ pub(crate) fn should_persist_event_msg(ev: &EventMsg) -> bool {
| EventMsg::GetHistoryEntryResponse(_)
| EventMsg::UndoStarted(_)
| EventMsg::McpListToolsResponse(_)
| EventMsg::McpStartupUpdate(_)
| EventMsg::McpStartupComplete(_)
| EventMsg::ListCustomPromptsResponse(_)
| EventMsg::PlanUpdate(_)
| EventMsg::ShutdownComplete

View File

@@ -8,9 +8,12 @@ use crate::unified_exec::UnifiedExecSessionManager;
use crate::user_notification::UserNotifier;
use codex_otel::otel_event_manager::OtelEventManager;
use tokio::sync::Mutex;
use tokio::sync::RwLock;
use tokio_util::sync::CancellationToken;
pub(crate) struct SessionServices {
pub(crate) mcp_connection_manager: McpConnectionManager,
pub(crate) mcp_connection_manager: Arc<RwLock<McpConnectionManager>>,
pub(crate) mcp_startup_cancellation_token: CancellationToken,
pub(crate) unified_exec_manager: UnifiedExecSessionManager,
pub(crate) notifier: UserNotifier,
pub(crate) rollout: Mutex<Option<RolloutRecorder>>,

View File

@@ -287,6 +287,8 @@ async fn handle_list_resources(
let resources = session
.services
.mcp_connection_manager
.read()
.await
.list_all_resources()
.await;
Ok(ListResourcesPayload::from_all_servers(resources))
@@ -396,6 +398,8 @@ async fn handle_list_resource_templates(
let templates = session
.services
.mcp_connection_manager
.read()
.await
.list_all_resource_templates()
.await;
Ok(ListResourceTemplatesPayload::from_all_servers(templates))

View File

@@ -54,7 +54,7 @@ impl ToolRouter {
.any(|config| config.spec.name() == tool_name)
}
pub fn build_tool_call(
pub async fn build_tool_call(
session: &Session,
item: ResponseItem,
) -> Result<Option<ToolCall>, FunctionCallError> {
@@ -65,7 +65,7 @@ impl ToolRouter {
call_id,
..
} => {
if let Some((server, tool)) = session.parse_mcp_tool_name(&name) {
if let Some((server, tool)) = session.parse_mcp_tool_name(&name).await {
Ok(Some(ToolCall {
tool_name: name,
call_id,

View File

@@ -20,6 +20,11 @@ pub enum ConfigShellToolType {
Default,
Local,
UnifiedExec,
/// Do not include a shell tool by default. Useful when using Codex
/// with tools provided exclusively provided by MCP servers. Often used
/// with `--config base_instructions=CUSTOM_INSTRUCTIONS`
/// to customize agent behavior.
Disabled,
/// Takes a command as a single string to be run in the user's default shell.
ShellCommand,
}
@@ -48,7 +53,9 @@ impl ToolsConfig {
let include_web_search_request = features.enabled(Feature::WebSearchRequest);
let include_view_image_tool = features.enabled(Feature::ViewImageTool);
let shell_type = if features.enabled(Feature::UnifiedExec) {
let shell_type = if !features.enabled(Feature::ShellTool) {
ConfigShellToolType::Disabled
} else if features.enabled(Feature::UnifiedExec) {
ConfigShellToolType::UnifiedExec
} else if features.enabled(Feature::ShellCommandTool) {
ConfigShellToolType::ShellCommand
@@ -294,9 +301,26 @@ fn create_shell_tool() -> ToolSpec {
},
);
let description = if cfg!(windows) {
r#"Runs a Powershell command (Windows) and returns its output. Arguments to `shell` will be passed to CreateProcessW(). Most commands should be prefixed with ["powershell.exe", "-Command"].
Examples of valid command strings:
- ls -a (show hidden): ["powershell.exe", "-Command", "Get-ChildItem -Force"]
- recursive find by name: ["powershell.exe", "-Command", "Get-ChildItem -Recurse -Filter *.py"]
- recursive grep: ["powershell.exe", "-Command", "Get-ChildItem -Path C:\\myrepo -Recurse | Select-String -Pattern 'TODO' -CaseSensitive"]
- ps aux | grep python: ["powershell.exe", "-Command", "Get-Process | Where-Object { $_.ProcessName -like '*python*' }"]
- setting an env var: ["powershell.exe", "-Command", "$env:FOO='bar'; echo $env:FOO"]
- running an inline Python script: ["powershell.exe", "-Command", "@'\\nprint('Hello, world!')\\n'@ | python -"]"#
} else {
r#"Runs a shell command and returns its output.
- The arguments to `shell` will be passed to execvp(). Most terminal commands should be prefixed with ["bash", "-lc"].
- Always set the `workdir` param when using the shell function. Do not use `cd` unless absolutely necessary."#
}.to_string();
ToolSpec::Function(ResponsesApiTool {
name: "shell".to_string(),
description: "Runs a shell command and returns its output.".to_string(),
description,
strict: false,
parameters: JsonSchema::Object {
properties,
@@ -341,9 +365,25 @@ fn create_shell_command_tool() -> ToolSpec {
},
);
let description = if cfg!(windows) {
r#"Runs a Powershell command (Windows) and returns its output.
Examples of valid command strings:
- ls -a (show hidden): "Get-ChildItem -Force"
- recursive find by name: "Get-ChildItem -Recurse -Filter *.py"
- recursive grep: "Get-ChildItem -Path C:\\myrepo -Recurse | Select-String -Pattern 'TODO' -CaseSensitive"
- ps aux | grep python: "Get-Process | Where-Object { $_.ProcessName -like '*python*' }"
- setting an env var: "$env:FOO='bar'; echo $env:FOO"
- running an inline Python script: "@'\\nprint('Hello, world!')\\n'@ | python -"#
} else {
r#"Runs a shell command and returns its output.
- Always set the `workdir` param when using the shell_command function. Do not use `cd` unless absolutely necessary."#
}.to_string();
ToolSpec::Function(ResponsesApiTool {
name: "shell_command".to_string(),
description: "Runs a shell command string and returns its output.".to_string(),
description,
strict: false,
parameters: JsonSchema::Object {
properties,
@@ -973,16 +1013,21 @@ pub(crate) fn build_specs(
builder.register_handler("exec_command", unified_exec_handler.clone());
builder.register_handler("write_stdin", unified_exec_handler);
}
ConfigShellToolType::Disabled => {
// Do nothing.
}
ConfigShellToolType::ShellCommand => {
builder.push_spec(create_shell_command_tool());
}
}
// Always register shell aliases so older prompts remain compatible.
builder.register_handler("shell", shell_handler.clone());
builder.register_handler("container.exec", shell_handler.clone());
builder.register_handler("local_shell", shell_handler);
builder.register_handler("shell_command", shell_command_handler);
if config.shell_type != ConfigShellToolType::Disabled {
// Always register shell aliases so older prompts remain compatible.
builder.register_handler("shell", shell_handler.clone());
builder.register_handler("container.exec", shell_handler.clone());
builder.register_handler("local_shell", shell_handler);
builder.register_handler("shell_command", shell_command_handler);
}
builder.push_spec_with_parallel_support(create_list_mcp_resources_tool(), true);
builder.push_spec_with_parallel_support(create_list_mcp_resource_templates_tool(), true);
@@ -1118,6 +1163,7 @@ mod tests {
ConfigShellToolType::Default => Some("shell"),
ConfigShellToolType::Local => Some("local_shell"),
ConfigShellToolType::UnifiedExec => None,
ConfigShellToolType::Disabled => None,
ConfigShellToolType::ShellCommand => Some("shell_command"),
}
}
@@ -1873,8 +1919,23 @@ mod tests {
};
assert_eq!(name, "shell");
let expected = "Runs a shell command and returns its output.";
assert_eq!(description, expected);
let expected = if cfg!(windows) {
r#"Runs a Powershell command (Windows) and returns its output. Arguments to `shell` will be passed to CreateProcessW(). Most commands should be prefixed with ["powershell.exe", "-Command"].
Examples of valid command strings:
- ls -a (show hidden): ["powershell.exe", "-Command", "Get-ChildItem -Force"]
- recursive find by name: ["powershell.exe", "-Command", "Get-ChildItem -Recurse -Filter *.py"]
- recursive grep: ["powershell.exe", "-Command", "Get-ChildItem -Path C:\\myrepo -Recurse | Select-String -Pattern 'TODO' -CaseSensitive"]
- ps aux | grep python: ["powershell.exe", "-Command", "Get-Process | Where-Object { $_.ProcessName -like '*python*' }"]
- setting an env var: ["powershell.exe", "-Command", "$env:FOO='bar'; echo $env:FOO"]
- running an inline Python script: ["powershell.exe", "-Command", "@'\\nprint('Hello, world!')\\n'@ | python -"]"#
} else {
r#"Runs a shell command and returns its output.
- The arguments to `shell` will be passed to execvp(). Most terminal commands should be prefixed with ["bash", "-lc"].
- Always set the `workdir` param when using the shell function. Do not use `cd` unless absolutely necessary."#
}.to_string();
assert_eq!(description, &expected);
}
#[test]
@@ -1888,8 +1949,22 @@ mod tests {
};
assert_eq!(name, "shell_command");
let expected = "Runs a shell command string and returns its output.";
assert_eq!(description, expected);
let expected = if cfg!(windows) {
r#"Runs a Powershell command (Windows) and returns its output.
Examples of valid command strings:
- ls -a (show hidden): "Get-ChildItem -Force"
- recursive find by name: "Get-ChildItem -Recurse -Filter *.py"
- recursive grep: "Get-ChildItem -Path C:\\myrepo -Recurse | Select-String -Pattern 'TODO' -CaseSensitive"
- ps aux | grep python: "Get-Process | Where-Object { $_.ProcessName -like '*python*' }"
- setting an env var: "$env:FOO='bar'; echo $env:FOO"
- running an inline Python script: "@'\\nprint('Hello, world!')\\n'@ | python -"#.to_string()
} else {
r#"Runs a shell command and returns its output.
- Always set the `workdir` param when using the shell_command function. Do not use `cd` unless absolutely necessary."#.to_string()
};
assert_eq!(description, &expected);
}
#[test]

View File

@@ -24,7 +24,6 @@ codex-common = { workspace = true, features = [
"sandbox_summary",
] }
codex-core = { workspace = true }
codex-ollama = { workspace = true }
codex-protocol = { workspace = true }
mcp-types = { workspace = true }
opentelemetry-appender-tracing = { workspace = true }

View File

@@ -18,9 +18,15 @@ pub struct Cli {
#[arg(long, short = 'm')]
pub model: Option<String>,
/// Use open-source provider.
#[arg(long = "oss", default_value_t = false)]
pub oss: bool,
/// Specify which local provider to use (lmstudio or ollama).
/// If not specified with --oss, will use config default or show selection.
#[arg(long = "local-provider")]
pub oss_provider: Option<String>,
/// Select the sandbox policy to use when executing model-generated shell
/// commands.
#[arg(long = "sandbox", short = 's', value_enum)]

View File

@@ -182,6 +182,42 @@ impl EventProcessor for EventProcessorWithHumanOutput {
ts_msg!(self, " {}", details.style(self.dimmed));
}
}
EventMsg::McpStartupUpdate(update) => {
let status_text = match update.status {
codex_core::protocol::McpStartupStatus::Starting => "starting".to_string(),
codex_core::protocol::McpStartupStatus::Ready => "ready".to_string(),
codex_core::protocol::McpStartupStatus::Cancelled => "cancelled".to_string(),
codex_core::protocol::McpStartupStatus::Failed { ref error } => {
format!("failed: {error}")
}
};
ts_msg!(
self,
"{} {} {}",
"mcp:".style(self.cyan),
update.server,
status_text
);
}
EventMsg::McpStartupComplete(summary) => {
let mut parts = Vec::new();
if !summary.ready.is_empty() {
parts.push(format!("ready: {}", summary.ready.join(", ")));
}
if !summary.failed.is_empty() {
let servers: Vec<_> = summary.failed.iter().map(|f| f.server.clone()).collect();
parts.push(format!("failed: {}", servers.join(", ")));
}
if !summary.cancelled.is_empty() {
parts.push(format!("cancelled: {}", summary.cancelled.join(", ")));
}
let joined = if parts.is_empty() {
"no servers".to_string()
} else {
parts.join("; ")
};
ts_msg!(self, "{} {}", "mcp startup:".style(self.cyan), joined);
}
EventMsg::BackgroundEvent(BackgroundEventEvent { message }) => {
ts_msg!(self, "{}", message.style(self.dimmed));
}

View File

@@ -11,20 +11,25 @@ pub mod event_processor_with_jsonl_output;
pub mod exec_events;
pub use cli::Cli;
use codex_common::oss::ensure_oss_provider_ready;
use codex_common::oss::get_default_model_for_oss_provider;
use codex_core::AuthManager;
use codex_core::BUILT_IN_OSS_MODEL_PROVIDER_ID;
use codex_core::ConversationManager;
use codex_core::LMSTUDIO_OSS_PROVIDER_ID;
use codex_core::NewConversation;
use codex_core::OLLAMA_OSS_PROVIDER_ID;
use codex_core::auth::enforce_login_restrictions;
use codex_core::config::Config;
use codex_core::config::ConfigOverrides;
use codex_core::config::find_codex_home;
use codex_core::config::load_config_as_toml_with_cli_overrides;
use codex_core::config::resolve_oss_provider;
use codex_core::git_info::get_git_repo_root;
use codex_core::protocol::AskForApproval;
use codex_core::protocol::Event;
use codex_core::protocol::EventMsg;
use codex_core::protocol::Op;
use codex_core::protocol::SessionSource;
use codex_ollama::DEFAULT_OSS_MODEL;
use codex_protocol::config_types::SandboxMode;
use codex_protocol::user_input::UserInput;
use event_processor_with_human_output::EventProcessorWithHumanOutput;
@@ -57,6 +62,7 @@ pub async fn run_main(cli: Cli, codex_linux_sandbox_exe: Option<PathBuf>) -> any
images,
model: model_cli_arg,
oss,
oss_provider,
config_profile,
full_auto,
dangerously_bypass_approvals_and_sandbox,
@@ -146,21 +152,64 @@ pub async fn run_main(cli: Cli, codex_linux_sandbox_exe: Option<PathBuf>) -> any
sandbox_mode_cli_arg.map(Into::<SandboxMode>::into)
};
// When using `--oss`, let the bootstrapper pick the model (defaulting to
// gpt-oss:20b) and ensure it is present locally. Also, force the builtin
// `oss` model provider.
let model = if let Some(model) = model_cli_arg {
Some(model)
} else if oss {
Some(DEFAULT_OSS_MODEL.to_owned())
} else {
None // No model specified, will use the default.
// Parse `-c` overrides from the CLI.
let cli_kv_overrides = match config_overrides.parse_overrides() {
Ok(v) => v,
#[allow(clippy::print_stderr)]
Err(e) => {
eprintln!("Error parsing -c overrides: {e}");
std::process::exit(1);
}
};
// we load config.toml here to determine project state.
#[allow(clippy::print_stderr)]
let config_toml = {
let codex_home = match find_codex_home() {
Ok(codex_home) => codex_home,
Err(err) => {
eprintln!("Error finding codex home: {err}");
std::process::exit(1);
}
};
match load_config_as_toml_with_cli_overrides(&codex_home, cli_kv_overrides.clone()).await {
Ok(config_toml) => config_toml,
Err(err) => {
eprintln!("Error loading config.toml: {err}");
std::process::exit(1);
}
}
};
let model_provider = if oss {
Some(BUILT_IN_OSS_MODEL_PROVIDER_ID.to_string())
let resolved = resolve_oss_provider(
oss_provider.as_deref(),
&config_toml,
config_profile.clone(),
);
if let Some(provider) = resolved {
Some(provider)
} else {
return Err(anyhow::anyhow!(
"No default OSS provider configured. Use --local-provider=provider or set oss_provider to either {LMSTUDIO_OSS_PROVIDER_ID} or {OLLAMA_OSS_PROVIDER_ID} in config.toml"
));
}
} else {
None // No specific model provider override.
None // No OSS mode enabled
};
// When using `--oss`, let the bootstrapper pick the model based on selected provider
let model = if let Some(model) = model_cli_arg {
Some(model)
} else if oss {
model_provider
.as_ref()
.and_then(|provider_id| get_default_model_for_oss_provider(provider_id))
.map(std::borrow::ToOwned::to_owned)
} else {
None // No model specified, will use the default.
};
// Load configuration and determine approval policy
@@ -172,7 +221,7 @@ pub async fn run_main(cli: Cli, codex_linux_sandbox_exe: Option<PathBuf>) -> any
approval_policy: Some(AskForApproval::Never),
sandbox_mode,
cwd: cwd.map(|p| p.canonicalize().unwrap_or(p)),
model_provider,
model_provider: model_provider.clone(),
codex_linux_sandbox_exe,
base_instructions: None,
developer_instructions: None,
@@ -183,14 +232,6 @@ pub async fn run_main(cli: Cli, codex_linux_sandbox_exe: Option<PathBuf>) -> any
experimental_sandbox_command_assessment: None,
additional_writable_roots: add_dir,
};
// Parse `-c` overrides.
let cli_kv_overrides = match config_overrides.parse_overrides() {
Ok(v) => v,
Err(e) => {
eprintln!("Error parsing -c overrides: {e}");
std::process::exit(1);
}
};
let config = Config::load_with_cli_overrides(cli_kv_overrides, overrides).await?;
@@ -233,7 +274,18 @@ pub async fn run_main(cli: Cli, codex_linux_sandbox_exe: Option<PathBuf>) -> any
};
if oss {
codex_ollama::ensure_oss_ready(&config)
// We're in the oss section, so provider_id should be Some
// Let's handle None case gracefully though just in case
let provider_id = match model_provider.as_ref() {
Some(id) => id,
None => {
error!("OSS provider unexpectedly not set when oss flag is used");
return Err(anyhow::anyhow!(
"OSS provider not set but oss flag was used"
));
}
};
ensure_oss_provider_ready(provider_id, &config)
.await
.map_err(|e| anyhow::anyhow!("OSS setup failed: {e}"))?;
}

View File

@@ -0,0 +1,29 @@
[package]
name = "codex-execpolicy2"
version = { workspace = true }
edition = "2024"
description = "Codex exec policy v2: prefix-based Starlark rules for command decisions."
[lib]
name = "codex_execpolicy2"
path = "src/lib.rs"
[[bin]]
name = "codex-execpolicy2"
path = "src/main.rs"
[lints]
workspace = true
[dependencies]
anyhow = { workspace = true }
clap = { workspace = true, features = ["derive"] }
multimap = { workspace = true }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
shlex = { workspace = true }
starlark = { workspace = true }
thiserror = { workspace = true }
[dev-dependencies]
pretty_assertions = { workspace = true }

View File

@@ -0,0 +1,54 @@
# codex-execpolicy2
## Overview
- Policy engine and CLI built around `prefix_rule(pattern=[...], decision?, match?, not_match?)`.
- This release covers only the prefix-rule subset of the planned execpolicy v2 language; a richer language will follow.
- Tokens are matched in order; any `pattern` element may be a list to denote alternatives. `decision` defaults to `allow`; valid values: `allow`, `prompt`, `forbidden`.
- `match` / `not_match` supply example invocations that are validated at load time (think of them as unit tests); examples can be token arrays or strings (strings are tokenized with `shlex`).
- The CLI always prints the JSON serialization of the evaluation result (whether a match or not).
## Policy shapes
- Prefix rules use Starlark syntax:
```starlark
prefix_rule(
pattern = ["cmd", ["alt1", "alt2"]], # ordered tokens; list entries denote alternatives
decision = "prompt", # allow | prompt | forbidden; defaults to allow
match = [["cmd", "alt1"], "cmd alt2"], # examples that must match this rule
not_match = [["cmd", "oops"], "cmd alt3"], # examples that must not match this rule
)
```
## Response shapes
- Match:
```json
{
"match": {
"decision": "allow|prompt|forbidden",
"matchedRules": [
{
"prefixRuleMatch": {
"matchedPrefix": ["<token>", "..."],
"decision": "allow|prompt|forbidden"
}
}
]
}
}
```
- No match:
```json
"noMatch"
```
- `matchedRules` lists every rule whose prefix matched the command; `matchedPrefix` is the exact prefix that matched.
- The effective `decision` is the strictest severity across all matches (`forbidden` > `prompt` > `allow`).
## CLI
- Provide a policy file (for example `src/default.codexpolicy`) to check a command:
```bash
cargo run -p codex-execpolicy2 -- check --policy path/to/policy.codexpolicy git status
```
- Example outcomes:
- Match: `{"match": { ... "decision": "allow" ... }}`
- No match: `"noMatch"`

View File

@@ -0,0 +1,77 @@
# Example policy to illustrate syntax; not comprehensive and not recommended for actual use.
prefix_rule(
pattern = ["git", "reset", "--hard"],
decision = "forbidden",
match = [
["git", "reset", "--hard"],
],
not_match = [
["git", "reset", "--keep"],
"git reset --merge",
],
)
prefix_rule(
pattern = ["ls"],
match = [
["ls"],
["ls", "-l"],
["ls", "-a", "."],
],
)
prefix_rule(
pattern = ["cat"],
match = [
["cat", "file.txt"],
["cat", "-n", "README.md"],
],
)
prefix_rule(
pattern = ["cp"],
decision = "prompt",
match = [
["cp", "foo", "bar"],
"cp -r src dest",
],
)
prefix_rule(
pattern = ["head"],
match = [
["head", "README.md"],
["head", "-n", "5", "CHANGELOG.md"],
],
not_match = [
["hea", "-n", "1,5p", "CHANGELOG.md"],
],
)
prefix_rule(
pattern = ["printenv"],
match = [
["printenv"],
["printenv", "PATH"],
],
not_match = [
["print", "-0"],
],
)
prefix_rule(
pattern = ["pwd"],
match = [
["pwd"],
],
)
prefix_rule(
pattern = ["which"],
match = [
["which", "python3"],
["which", "-a", "python3"],
],
)

View File

@@ -0,0 +1,27 @@
use serde::Deserialize;
use serde::Serialize;
use crate::error::Error;
use crate::error::Result;
#[derive(Clone, Copy, Debug, Eq, PartialEq, Ord, PartialOrd, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum Decision {
/// Command may run without further approval.
Allow,
/// Request explicit user approval; rejected outright when running with `approval_policy="never"`.
Prompt,
/// Command is blocked without further consideration.
Forbidden,
}
impl Decision {
pub fn parse(raw: &str) -> Result<Self> {
match raw {
"allow" => Ok(Self::Allow),
"prompt" => Ok(Self::Prompt),
"forbidden" => Ok(Self::Forbidden),
other => Err(Error::InvalidDecision(other.to_string())),
}
}
}

View File

@@ -0,0 +1,26 @@
use starlark::Error as StarlarkError;
use thiserror::Error;
pub type Result<T> = std::result::Result<T, Error>;
#[derive(Debug, Error)]
pub enum Error {
#[error("invalid decision: {0}")]
InvalidDecision(String),
#[error("invalid pattern element: {0}")]
InvalidPattern(String),
#[error("invalid example: {0}")]
InvalidExample(String),
#[error(
"expected every example to match at least one rule. rules: {rules:?}; unmatched examples: \
{examples:?}"
)]
ExampleDidNotMatch {
rules: Vec<String>,
examples: Vec<String>,
},
#[error("expected example to not match rule `{rule}`: {example}")]
ExampleDidMatch { rule: String, example: String },
#[error("starlark error: {0}")]
Starlark(StarlarkError),
}

View File

@@ -0,0 +1,15 @@
pub mod decision;
pub mod error;
pub mod parser;
pub mod policy;
pub mod rule;
pub use decision::Decision;
pub use error::Error;
pub use error::Result;
pub use parser::PolicyParser;
pub use policy::Evaluation;
pub use policy::Policy;
pub use rule::Rule;
pub use rule::RuleMatch;
pub use rule::RuleRef;

View File

@@ -0,0 +1,54 @@
use std::fs;
use std::path::Path;
use std::path::PathBuf;
use anyhow::Context;
use anyhow::Result;
use clap::Parser;
use codex_execpolicy2::PolicyParser;
/// CLI for evaluating exec policies
#[derive(Parser)]
#[command(name = "codex-execpolicy2")]
enum Cli {
/// Evaluate a command against a policy.
Check {
#[arg(short, long, value_name = "PATH")]
policy: PathBuf,
/// Command tokens to check.
#[arg(
value_name = "COMMAND",
required = true,
trailing_var_arg = true,
allow_hyphen_values = true
)]
command: Vec<String>,
},
}
fn main() -> Result<()> {
let cli = Cli::parse();
match cli {
Cli::Check { policy, command } => cmd_check(policy, command),
}
}
fn cmd_check(policy_path: PathBuf, args: Vec<String>) -> Result<()> {
let policy = load_policy(&policy_path)?;
let eval = policy.check(&args);
let json = serde_json::to_string_pretty(&eval)?;
println!("{json}");
Ok(())
}
fn load_policy(policy_path: &Path) -> Result<codex_execpolicy2::Policy> {
let policy_file_contents = fs::read_to_string(policy_path)
.with_context(|| format!("failed to read policy at {}", policy_path.display()))?;
let policy_identifier = policy_path.to_string_lossy();
Ok(PolicyParser::parse(
policy_identifier.as_ref(),
&policy_file_contents,
)?)
}

View File

@@ -0,0 +1,247 @@
use multimap::MultiMap;
use shlex;
use starlark::any::ProvidesStaticType;
use starlark::environment::GlobalsBuilder;
use starlark::environment::Module;
use starlark::eval::Evaluator;
use starlark::starlark_module;
use starlark::syntax::AstModule;
use starlark::syntax::Dialect;
use starlark::values::Value;
use starlark::values::list::ListRef;
use starlark::values::list::UnpackList;
use starlark::values::none::NoneType;
use std::cell::RefCell;
use std::cell::RefMut;
use std::sync::Arc;
use crate::decision::Decision;
use crate::error::Error;
use crate::error::Result;
use crate::rule::PatternToken;
use crate::rule::PrefixPattern;
use crate::rule::PrefixRule;
use crate::rule::RuleRef;
use crate::rule::validate_match_examples;
use crate::rule::validate_not_match_examples;
// todo: support parsing multiple policies
pub struct PolicyParser;
impl PolicyParser {
/// Parses a policy, tagging parser errors with `policy_identifier` so failures include the
/// identifier alongside line numbers.
pub fn parse(
policy_identifier: &str,
policy_file_contents: &str,
) -> Result<crate::policy::Policy> {
let mut dialect = Dialect::Extended.clone();
dialect.enable_f_strings = true;
let ast = AstModule::parse(
policy_identifier,
policy_file_contents.to_string(),
&dialect,
)
.map_err(Error::Starlark)?;
let globals = GlobalsBuilder::standard().with(policy_builtins).build();
let module = Module::new();
let builder = RefCell::new(PolicyBuilder::new());
{
let mut eval = Evaluator::new(&module);
eval.extra = Some(&builder);
eval.eval_module(ast, &globals).map_err(Error::Starlark)?;
}
Ok(builder.into_inner().build())
}
}
#[derive(Debug, ProvidesStaticType)]
struct PolicyBuilder {
rules_by_program: MultiMap<String, RuleRef>,
}
impl PolicyBuilder {
fn new() -> Self {
Self {
rules_by_program: MultiMap::new(),
}
}
fn add_rule(&mut self, rule: RuleRef) {
self.rules_by_program
.insert(rule.program().to_string(), rule);
}
fn build(self) -> crate::policy::Policy {
crate::policy::Policy::new(self.rules_by_program)
}
}
fn parse_pattern<'v>(pattern: UnpackList<Value<'v>>) -> Result<Vec<PatternToken>> {
let tokens: Vec<PatternToken> = pattern
.items
.into_iter()
.map(parse_pattern_token)
.collect::<Result<_>>()?;
if tokens.is_empty() {
Err(Error::InvalidPattern("pattern cannot be empty".to_string()))
} else {
Ok(tokens)
}
}
fn parse_pattern_token<'v>(value: Value<'v>) -> Result<PatternToken> {
if let Some(s) = value.unpack_str() {
Ok(PatternToken::Single(s.to_string()))
} else if let Some(list) = ListRef::from_value(value) {
let tokens: Vec<String> = list
.content()
.iter()
.map(|value| {
value
.unpack_str()
.ok_or_else(|| {
Error::InvalidPattern(format!(
"pattern alternative must be a string (got {})",
value.get_type()
))
})
.map(str::to_string)
})
.collect::<Result<_>>()?;
match tokens.as_slice() {
[] => Err(Error::InvalidPattern(
"pattern alternatives cannot be empty".to_string(),
)),
[single] => Ok(PatternToken::Single(single.clone())),
_ => Ok(PatternToken::Alts(tokens)),
}
} else {
Err(Error::InvalidPattern(format!(
"pattern element must be a string or list of strings (got {})",
value.get_type()
)))
}
}
fn parse_examples<'v>(examples: UnpackList<Value<'v>>) -> Result<Vec<Vec<String>>> {
examples.items.into_iter().map(parse_example).collect()
}
fn parse_example<'v>(value: Value<'v>) -> Result<Vec<String>> {
if let Some(raw) = value.unpack_str() {
parse_string_example(raw)
} else if let Some(list) = ListRef::from_value(value) {
parse_list_example(list)
} else {
Err(Error::InvalidExample(format!(
"example must be a string or list of strings (got {})",
value.get_type()
)))
}
}
fn parse_string_example(raw: &str) -> Result<Vec<String>> {
let tokens = shlex::split(raw).ok_or_else(|| {
Error::InvalidExample("example string has invalid shell syntax".to_string())
})?;
if tokens.is_empty() {
Err(Error::InvalidExample(
"example cannot be an empty string".to_string(),
))
} else {
Ok(tokens)
}
}
fn parse_list_example(list: &ListRef) -> Result<Vec<String>> {
let tokens: Vec<String> = list
.content()
.iter()
.map(|value| {
value
.unpack_str()
.ok_or_else(|| {
Error::InvalidExample(format!(
"example tokens must be strings (got {})",
value.get_type()
))
})
.map(str::to_string)
})
.collect::<Result<_>>()?;
if tokens.is_empty() {
Err(Error::InvalidExample(
"example cannot be an empty list".to_string(),
))
} else {
Ok(tokens)
}
}
fn policy_builder<'v, 'a>(eval: &Evaluator<'v, 'a, '_>) -> RefMut<'a, PolicyBuilder> {
#[expect(clippy::expect_used)]
eval.extra
.as_ref()
.expect("policy_builder requires Evaluator.extra to be populated")
.downcast_ref::<RefCell<PolicyBuilder>>()
.expect("Evaluator.extra must contain a PolicyBuilder")
.borrow_mut()
}
#[starlark_module]
fn policy_builtins(builder: &mut GlobalsBuilder) {
fn prefix_rule<'v>(
pattern: UnpackList<Value<'v>>,
decision: Option<&'v str>,
r#match: Option<UnpackList<Value<'v>>>,
not_match: Option<UnpackList<Value<'v>>>,
eval: &mut Evaluator<'v, '_, '_>,
) -> anyhow::Result<NoneType> {
let decision = match decision {
Some(raw) => Decision::parse(raw)?,
None => Decision::Allow,
};
let pattern_tokens = parse_pattern(pattern)?;
let matches: Vec<Vec<String>> =
r#match.map(parse_examples).transpose()?.unwrap_or_default();
let not_matches: Vec<Vec<String>> = not_match
.map(parse_examples)
.transpose()?
.unwrap_or_default();
let mut builder = policy_builder(eval);
let (first_token, remaining_tokens) = pattern_tokens
.split_first()
.ok_or_else(|| Error::InvalidPattern("pattern cannot be empty".to_string()))?;
let rest: Arc<[PatternToken]> = remaining_tokens.to_vec().into();
let rules: Vec<RuleRef> = first_token
.alternatives()
.iter()
.map(|head| {
Arc::new(PrefixRule {
pattern: PrefixPattern {
first: Arc::from(head.as_str()),
rest: rest.clone(),
},
decision,
}) as RuleRef
})
.collect();
validate_not_match_examples(&rules, &not_matches)?;
validate_match_examples(&rules, &matches)?;
rules.into_iter().for_each(|rule| builder.add_rule(rule));
Ok(NoneType)
}
}

View File

@@ -0,0 +1,58 @@
use crate::decision::Decision;
use crate::rule::RuleMatch;
use crate::rule::RuleRef;
use multimap::MultiMap;
use serde::Deserialize;
use serde::Serialize;
#[derive(Clone, Debug)]
pub struct Policy {
rules_by_program: MultiMap<String, RuleRef>,
}
impl Policy {
pub fn new(rules_by_program: MultiMap<String, RuleRef>) -> Self {
Self { rules_by_program }
}
pub fn rules(&self) -> &MultiMap<String, RuleRef> {
&self.rules_by_program
}
pub fn check(&self, cmd: &[String]) -> Evaluation {
let rules = match cmd.first() {
Some(first) => match self.rules_by_program.get_vec(first) {
Some(rules) => rules,
None => return Evaluation::NoMatch,
},
None => return Evaluation::NoMatch,
};
let matched_rules: Vec<RuleMatch> =
rules.iter().filter_map(|rule| rule.matches(cmd)).collect();
match matched_rules.iter().map(RuleMatch::decision).max() {
Some(decision) => Evaluation::Match {
decision,
matched_rules,
},
None => Evaluation::NoMatch,
}
}
}
#[derive(Clone, Debug, Eq, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum Evaluation {
NoMatch,
Match {
decision: Decision,
#[serde(rename = "matchedRules")]
matched_rules: Vec<RuleMatch>,
},
}
impl Evaluation {
pub fn is_match(&self) -> bool {
matches!(self, Self::Match { .. })
}
}

View File

@@ -0,0 +1,147 @@
use crate::decision::Decision;
use crate::error::Error;
use crate::error::Result;
use serde::Deserialize;
use serde::Serialize;
use shlex::try_join;
use std::any::Any;
use std::fmt::Debug;
use std::sync::Arc;
/// Matches a single command token, either a fixed string or one of several allowed alternatives.
#[derive(Clone, Debug, Eq, PartialEq)]
pub enum PatternToken {
Single(String),
Alts(Vec<String>),
}
impl PatternToken {
fn matches(&self, token: &str) -> bool {
match self {
Self::Single(expected) => expected == token,
Self::Alts(alternatives) => alternatives.iter().any(|alt| alt == token),
}
}
pub fn alternatives(&self) -> &[String] {
match self {
Self::Single(expected) => std::slice::from_ref(expected),
Self::Alts(alternatives) => alternatives,
}
}
}
/// Prefix matcher for commands with support for alternative match tokens.
/// First token is fixed since we key by the first token in policy.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct PrefixPattern {
pub first: Arc<str>,
pub rest: Arc<[PatternToken]>,
}
impl PrefixPattern {
pub fn matches_prefix(&self, cmd: &[String]) -> Option<Vec<String>> {
let pattern_length = self.rest.len() + 1;
if cmd.len() < pattern_length || cmd[0] != self.first.as_ref() {
return None;
}
for (pattern_token, cmd_token) in self.rest.iter().zip(&cmd[1..pattern_length]) {
if !pattern_token.matches(cmd_token) {
return None;
}
}
Some(cmd[..pattern_length].to_vec())
}
}
#[derive(Clone, Debug, Eq, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum RuleMatch {
PrefixRuleMatch {
#[serde(rename = "matchedPrefix")]
matched_prefix: Vec<String>,
decision: Decision,
},
}
impl RuleMatch {
pub fn decision(&self) -> Decision {
match self {
Self::PrefixRuleMatch { decision, .. } => *decision,
}
}
}
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct PrefixRule {
pub pattern: PrefixPattern,
pub decision: Decision,
}
pub trait Rule: Any + Debug + Send + Sync {
fn program(&self) -> &str;
fn matches(&self, cmd: &[String]) -> Option<RuleMatch>;
}
pub type RuleRef = Arc<dyn Rule>;
impl Rule for PrefixRule {
fn program(&self) -> &str {
self.pattern.first.as_ref()
}
fn matches(&self, cmd: &[String]) -> Option<RuleMatch> {
self.pattern
.matches_prefix(cmd)
.map(|matched_prefix| RuleMatch::PrefixRuleMatch {
matched_prefix,
decision: self.decision,
})
}
}
/// Count how many rules match each provided example and error if any example is unmatched.
pub(crate) fn validate_match_examples(rules: &[RuleRef], matches: &[Vec<String>]) -> Result<()> {
let mut unmatched_examples = Vec::new();
for example in matches {
if rules.iter().any(|rule| rule.matches(example).is_some()) {
continue;
}
unmatched_examples.push(
try_join(example.iter().map(String::as_str))
.unwrap_or_else(|_| "unable to render example".to_string()),
);
}
if unmatched_examples.is_empty() {
Ok(())
} else {
Err(Error::ExampleDidNotMatch {
rules: rules.iter().map(|rule| format!("{rule:?}")).collect(),
examples: unmatched_examples,
})
}
}
/// Ensure that no rule matches any provided negative example.
pub(crate) fn validate_not_match_examples(
rules: &[RuleRef],
not_matches: &[Vec<String>],
) -> Result<()> {
for example in not_matches {
if let Some(rule) = rules.iter().find(|rule| rule.matches(example).is_some()) {
return Err(Error::ExampleDidMatch {
rule: format!("{rule:?}"),
example: try_join(example.iter().map(String::as_str))
.unwrap_or_else(|_| "unable to render example".to_string()),
});
}
}
Ok(())
}

View File

@@ -0,0 +1,256 @@
use std::any::Any;
use std::sync::Arc;
use codex_execpolicy2::Decision;
use codex_execpolicy2::Evaluation;
use codex_execpolicy2::PolicyParser;
use codex_execpolicy2::RuleMatch;
use codex_execpolicy2::RuleRef;
use codex_execpolicy2::rule::PatternToken;
use codex_execpolicy2::rule::PrefixPattern;
use codex_execpolicy2::rule::PrefixRule;
use pretty_assertions::assert_eq;
fn tokens(cmd: &[&str]) -> Vec<String> {
cmd.iter().map(std::string::ToString::to_string).collect()
}
#[derive(Clone, Debug, Eq, PartialEq)]
enum RuleSnapshot {
Prefix(PrefixRule),
}
fn rule_snapshots(rules: &[RuleRef]) -> Vec<RuleSnapshot> {
rules
.iter()
.map(|rule| {
let rule_any = rule.as_ref() as &dyn Any;
if let Some(prefix_rule) = rule_any.downcast_ref::<PrefixRule>() {
RuleSnapshot::Prefix(prefix_rule.clone())
} else {
panic!("unexpected rule type in RuleRef: {rule:?}");
}
})
.collect()
}
#[test]
fn basic_match() {
let policy_src = r#"
prefix_rule(
pattern = ["git", "status"],
)
"#;
let policy = PolicyParser::parse("test.codexpolicy", policy_src).expect("parse policy");
let cmd = tokens(&["git", "status"]);
let evaluation = policy.check(&cmd);
assert_eq!(
Evaluation::Match {
decision: Decision::Allow,
matched_rules: vec![RuleMatch::PrefixRuleMatch {
matched_prefix: tokens(&["git", "status"]),
decision: Decision::Allow,
}],
},
evaluation
);
}
#[test]
fn only_first_token_alias_expands_to_multiple_rules() {
let policy_src = r#"
prefix_rule(
pattern = [["bash", "sh"], ["-c", "-l"]],
)
"#;
let policy = PolicyParser::parse("test.codexpolicy", policy_src).expect("parse policy");
let bash_rules = rule_snapshots(policy.rules().get_vec("bash").expect("bash rules"));
let sh_rules = rule_snapshots(policy.rules().get_vec("sh").expect("sh rules"));
assert_eq!(
vec![RuleSnapshot::Prefix(PrefixRule {
pattern: PrefixPattern {
first: Arc::from("bash"),
rest: vec![PatternToken::Alts(vec!["-c".to_string(), "-l".to_string()])].into(),
},
decision: Decision::Allow,
})],
bash_rules
);
assert_eq!(
vec![RuleSnapshot::Prefix(PrefixRule {
pattern: PrefixPattern {
first: Arc::from("sh"),
rest: vec![PatternToken::Alts(vec!["-c".to_string(), "-l".to_string()])].into(),
},
decision: Decision::Allow,
})],
sh_rules
);
let bash_eval = policy.check(&tokens(&["bash", "-c", "echo", "hi"]));
assert_eq!(
Evaluation::Match {
decision: Decision::Allow,
matched_rules: vec![RuleMatch::PrefixRuleMatch {
matched_prefix: tokens(&["bash", "-c"]),
decision: Decision::Allow,
}],
},
bash_eval
);
let sh_eval = policy.check(&tokens(&["sh", "-l", "echo", "hi"]));
assert_eq!(
Evaluation::Match {
decision: Decision::Allow,
matched_rules: vec![RuleMatch::PrefixRuleMatch {
matched_prefix: tokens(&["sh", "-l"]),
decision: Decision::Allow,
}],
},
sh_eval
);
}
#[test]
fn tail_aliases_are_not_cartesian_expanded() {
let policy_src = r#"
prefix_rule(
pattern = ["npm", ["i", "install"], ["--legacy-peer-deps", "--no-save"]],
)
"#;
let policy = PolicyParser::parse("test.codexpolicy", policy_src).expect("parse policy");
let rules = rule_snapshots(policy.rules().get_vec("npm").expect("npm rules"));
assert_eq!(
vec![RuleSnapshot::Prefix(PrefixRule {
pattern: PrefixPattern {
first: Arc::from("npm"),
rest: vec![
PatternToken::Alts(vec!["i".to_string(), "install".to_string()]),
PatternToken::Alts(vec![
"--legacy-peer-deps".to_string(),
"--no-save".to_string(),
]),
]
.into(),
},
decision: Decision::Allow,
})],
rules
);
let npm_i = policy.check(&tokens(&["npm", "i", "--legacy-peer-deps"]));
assert_eq!(
Evaluation::Match {
decision: Decision::Allow,
matched_rules: vec![RuleMatch::PrefixRuleMatch {
matched_prefix: tokens(&["npm", "i", "--legacy-peer-deps"]),
decision: Decision::Allow,
}],
},
npm_i
);
let npm_install = policy.check(&tokens(&["npm", "install", "--no-save", "leftpad"]));
assert_eq!(
Evaluation::Match {
decision: Decision::Allow,
matched_rules: vec![RuleMatch::PrefixRuleMatch {
matched_prefix: tokens(&["npm", "install", "--no-save"]),
decision: Decision::Allow,
}],
},
npm_install
);
}
#[test]
fn match_and_not_match_examples_are_enforced() {
let policy_src = r#"
prefix_rule(
pattern = ["git", "status"],
match = [["git", "status"], "git status"],
not_match = [
["git", "--config", "color.status=always", "status"],
"git --config color.status=always status",
],
)
"#;
let policy = PolicyParser::parse("test.codexpolicy", policy_src).expect("parse policy");
let match_eval = policy.check(&tokens(&["git", "status"]));
assert_eq!(
Evaluation::Match {
decision: Decision::Allow,
matched_rules: vec![RuleMatch::PrefixRuleMatch {
matched_prefix: tokens(&["git", "status"]),
decision: Decision::Allow,
}],
},
match_eval
);
let no_match_eval = policy.check(&tokens(&[
"git",
"--config",
"color.status=always",
"status",
]));
assert_eq!(Evaluation::NoMatch, no_match_eval);
}
#[test]
fn strictest_decision_wins_across_matches() {
let policy_src = r#"
prefix_rule(
pattern = ["git", "status"],
decision = "allow",
)
prefix_rule(
pattern = ["git"],
decision = "prompt",
)
prefix_rule(
pattern = ["git", "commit"],
decision = "forbidden",
)
"#;
let policy = PolicyParser::parse("test.codexpolicy", policy_src).expect("parse policy");
let status = policy.check(&tokens(&["git", "status"]));
assert_eq!(
Evaluation::Match {
decision: Decision::Prompt,
matched_rules: vec![
RuleMatch::PrefixRuleMatch {
matched_prefix: tokens(&["git", "status"]),
decision: Decision::Allow,
},
RuleMatch::PrefixRuleMatch {
matched_prefix: tokens(&["git"]),
decision: Decision::Prompt,
},
],
},
status
);
let commit = policy.check(&tokens(&["git", "commit", "-m", "hi"]));
assert_eq!(
Evaluation::Match {
decision: Decision::Forbidden,
matched_rules: vec![
RuleMatch::PrefixRuleMatch {
matched_prefix: tokens(&["git"]),
decision: Decision::Prompt,
},
RuleMatch::PrefixRuleMatch {
matched_prefix: tokens(&["git", "commit"]),
decision: Decision::Forbidden,
},
],
},
commit
);
}

View File

@@ -0,0 +1,24 @@
[package]
name = "codex-lmstudio"
version.workspace = true
edition.workspace = true
[lib]
name = "codex_lmstudio"
path = "src/lib.rs"
[dependencies]
codex-core = { path = "../core" }
reqwest = { version = "0.12", features = ["json", "stream"] }
serde_json = "1"
tokio = { version = "1", features = ["rt"] }
tracing = { version = "0.1.41", features = ["log"] }
which = "6.0"
[dev-dependencies]
wiremock = "0.6"
tokio = { version = "1", features = ["full"] }
[lints]
workspace = true

View File

@@ -0,0 +1,397 @@
use codex_core::LMSTUDIO_OSS_PROVIDER_ID;
use codex_core::config::Config;
use std::io;
use std::path::Path;
#[derive(Clone)]
pub struct LMStudioClient {
client: reqwest::Client,
base_url: String,
}
const LMSTUDIO_CONNECTION_ERROR: &str = "LM Studio is not responding. Install from https://lmstudio.ai/download and run 'lms server start'.";
impl LMStudioClient {
pub async fn try_from_provider(config: &Config) -> std::io::Result<Self> {
let provider = config
.model_providers
.get(LMSTUDIO_OSS_PROVIDER_ID)
.ok_or_else(|| {
io::Error::new(
io::ErrorKind::NotFound,
format!("Built-in provider {LMSTUDIO_OSS_PROVIDER_ID} not found",),
)
})?;
let base_url = provider.base_url.as_ref().ok_or_else(|| {
io::Error::new(
io::ErrorKind::InvalidData,
"oss provider must have a base_url",
)
})?;
let client = reqwest::Client::builder()
.connect_timeout(std::time::Duration::from_secs(5))
.build()
.unwrap_or_else(|_| reqwest::Client::new());
let client = LMStudioClient {
client,
base_url: base_url.to_string(),
};
client.check_server().await?;
Ok(client)
}
async fn check_server(&self) -> io::Result<()> {
let url = format!("{}/models", self.base_url.trim_end_matches('/'));
let response = self.client.get(&url).send().await;
if let Ok(resp) = response {
if resp.status().is_success() {
Ok(())
} else {
Err(io::Error::other(format!(
"Server returned error: {} {LMSTUDIO_CONNECTION_ERROR}",
resp.status()
)))
}
} else {
Err(io::Error::other(LMSTUDIO_CONNECTION_ERROR))
}
}
// Load a model by sending an empty request with max_tokens 1
pub async fn load_model(&self, model: &str) -> io::Result<()> {
let url = format!("{}/responses", self.base_url.trim_end_matches('/'));
let request_body = serde_json::json!({
"model": model,
"input": "",
"max_output_tokens": 1
});
let response = self
.client
.post(&url)
.header("Content-Type", "application/json")
.json(&request_body)
.send()
.await
.map_err(|e| io::Error::other(format!("Request failed: {e}")))?;
if response.status().is_success() {
tracing::info!("Successfully loaded model '{model}'");
Ok(())
} else {
Err(io::Error::other(format!(
"Failed to load model: {}",
response.status()
)))
}
}
// Return the list of models available on the LM Studio server.
pub async fn fetch_models(&self) -> io::Result<Vec<String>> {
let url = format!("{}/models", self.base_url.trim_end_matches('/'));
let response = self
.client
.get(&url)
.send()
.await
.map_err(|e| io::Error::other(format!("Request failed: {e}")))?;
if response.status().is_success() {
let json: serde_json::Value = response.json().await.map_err(|e| {
io::Error::new(io::ErrorKind::InvalidData, format!("JSON parse error: {e}"))
})?;
let models = json["data"]
.as_array()
.ok_or_else(|| {
io::Error::new(io::ErrorKind::InvalidData, "No 'data' array in response")
})?
.iter()
.filter_map(|model| model["id"].as_str())
.map(std::string::ToString::to_string)
.collect();
Ok(models)
} else {
Err(io::Error::other(format!(
"Failed to fetch models: {}",
response.status()
)))
}
}
// Find lms, checking fallback paths if not in PATH
fn find_lms() -> std::io::Result<String> {
Self::find_lms_with_home_dir(None)
}
fn find_lms_with_home_dir(home_dir: Option<&str>) -> std::io::Result<String> {
// First try 'lms' in PATH
if which::which("lms").is_ok() {
return Ok("lms".to_string());
}
// Platform-specific fallback paths
let home = match home_dir {
Some(dir) => dir.to_string(),
None => {
#[cfg(unix)]
{
std::env::var("HOME").unwrap_or_default()
}
#[cfg(windows)]
{
std::env::var("USERPROFILE").unwrap_or_default()
}
}
};
#[cfg(unix)]
let fallback_path = format!("{home}/.lmstudio/bin/lms");
#[cfg(windows)]
let fallback_path = format!("{home}/.lmstudio/bin/lms.exe");
if Path::new(&fallback_path).exists() {
Ok(fallback_path)
} else {
Err(std::io::Error::new(
std::io::ErrorKind::NotFound,
"LM Studio not found. Please install LM Studio from https://lmstudio.ai/",
))
}
}
pub async fn download_model(&self, model: &str) -> std::io::Result<()> {
let lms = Self::find_lms()?;
eprintln!("Downloading model: {model}");
let status = std::process::Command::new(&lms)
.args(["get", "--yes", model])
.stdout(std::process::Stdio::inherit())
.stderr(std::process::Stdio::null())
.status()
.map_err(|e| {
std::io::Error::other(format!("Failed to execute '{lms} get --yes {model}': {e}"))
})?;
if !status.success() {
return Err(std::io::Error::other(format!(
"Model download failed with exit code: {}",
status.code().unwrap_or(-1)
)));
}
tracing::info!("Successfully downloaded model '{model}'");
Ok(())
}
/// Low-level constructor given a raw host root, e.g. "http://localhost:1234".
#[cfg(test)]
fn from_host_root(host_root: impl Into<String>) -> Self {
let client = reqwest::Client::builder()
.connect_timeout(std::time::Duration::from_secs(5))
.build()
.unwrap_or_else(|_| reqwest::Client::new());
Self {
client,
base_url: host_root.into(),
}
}
}
#[cfg(test)]
mod tests {
#![allow(clippy::expect_used, clippy::unwrap_used)]
use super::*;
#[tokio::test]
async fn test_fetch_models_happy_path() {
if std::env::var(codex_core::spawn::CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR).is_ok() {
tracing::info!(
"{} is set; skipping test_fetch_models_happy_path",
codex_core::spawn::CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR
);
return;
}
let server = wiremock::MockServer::start().await;
wiremock::Mock::given(wiremock::matchers::method("GET"))
.and(wiremock::matchers::path("/models"))
.respond_with(
wiremock::ResponseTemplate::new(200).set_body_raw(
serde_json::json!({
"data": [
{"id": "openai/gpt-oss-20b"},
]
})
.to_string(),
"application/json",
),
)
.mount(&server)
.await;
let client = LMStudioClient::from_host_root(server.uri());
let models = client.fetch_models().await.expect("fetch models");
assert!(models.contains(&"openai/gpt-oss-20b".to_string()));
}
#[tokio::test]
async fn test_fetch_models_no_data_array() {
if std::env::var(codex_core::spawn::CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR).is_ok() {
tracing::info!(
"{} is set; skipping test_fetch_models_no_data_array",
codex_core::spawn::CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR
);
return;
}
let server = wiremock::MockServer::start().await;
wiremock::Mock::given(wiremock::matchers::method("GET"))
.and(wiremock::matchers::path("/models"))
.respond_with(
wiremock::ResponseTemplate::new(200)
.set_body_raw(serde_json::json!({}).to_string(), "application/json"),
)
.mount(&server)
.await;
let client = LMStudioClient::from_host_root(server.uri());
let result = client.fetch_models().await;
assert!(result.is_err());
assert!(
result
.unwrap_err()
.to_string()
.contains("No 'data' array in response")
);
}
#[tokio::test]
async fn test_fetch_models_server_error() {
if std::env::var(codex_core::spawn::CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR).is_ok() {
tracing::info!(
"{} is set; skipping test_fetch_models_server_error",
codex_core::spawn::CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR
);
return;
}
let server = wiremock::MockServer::start().await;
wiremock::Mock::given(wiremock::matchers::method("GET"))
.and(wiremock::matchers::path("/models"))
.respond_with(wiremock::ResponseTemplate::new(500))
.mount(&server)
.await;
let client = LMStudioClient::from_host_root(server.uri());
let result = client.fetch_models().await;
assert!(result.is_err());
assert!(
result
.unwrap_err()
.to_string()
.contains("Failed to fetch models: 500")
);
}
#[tokio::test]
async fn test_check_server_happy_path() {
if std::env::var(codex_core::spawn::CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR).is_ok() {
tracing::info!(
"{} is set; skipping test_check_server_happy_path",
codex_core::spawn::CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR
);
return;
}
let server = wiremock::MockServer::start().await;
wiremock::Mock::given(wiremock::matchers::method("GET"))
.and(wiremock::matchers::path("/models"))
.respond_with(wiremock::ResponseTemplate::new(200))
.mount(&server)
.await;
let client = LMStudioClient::from_host_root(server.uri());
client
.check_server()
.await
.expect("server check should pass");
}
#[tokio::test]
async fn test_check_server_error() {
if std::env::var(codex_core::spawn::CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR).is_ok() {
tracing::info!(
"{} is set; skipping test_check_server_error",
codex_core::spawn::CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR
);
return;
}
let server = wiremock::MockServer::start().await;
wiremock::Mock::given(wiremock::matchers::method("GET"))
.and(wiremock::matchers::path("/models"))
.respond_with(wiremock::ResponseTemplate::new(404))
.mount(&server)
.await;
let client = LMStudioClient::from_host_root(server.uri());
let result = client.check_server().await;
assert!(result.is_err());
assert!(
result
.unwrap_err()
.to_string()
.contains("Server returned error: 404")
);
}
#[test]
fn test_find_lms() {
let result = LMStudioClient::find_lms();
match result {
Ok(_) => {
// lms was found in PATH - that's fine
}
Err(e) => {
// Expected error when LM Studio not installed
assert!(e.to_string().contains("LM Studio not found"));
}
}
}
#[test]
fn test_find_lms_with_mock_home() {
// Test fallback path construction without touching env vars
#[cfg(unix)]
{
let result = LMStudioClient::find_lms_with_home_dir(Some("/test/home"));
if let Err(e) = result {
assert!(e.to_string().contains("LM Studio not found"));
}
}
#[cfg(windows)]
{
let result = LMStudioClient::find_lms_with_home_dir(Some("C:\\test\\home"));
if let Err(e) = result {
assert!(e.to_string().contains("LM Studio not found"));
}
}
}
#[test]
fn test_from_host_root() {
let client = LMStudioClient::from_host_root("http://localhost:1234");
assert_eq!(client.base_url, "http://localhost:1234");
let client = LMStudioClient::from_host_root("https://example.com:8080/api");
assert_eq!(client.base_url, "https://example.com:8080/api");
}
}

View File

@@ -0,0 +1,43 @@
mod client;
pub use client::LMStudioClient;
use codex_core::config::Config;
/// Default OSS model to use when `--oss` is passed without an explicit `-m`.
pub const DEFAULT_OSS_MODEL: &str = "openai/gpt-oss-20b";
/// Prepare the local OSS environment when `--oss` is selected.
///
/// - Ensures a local LM Studio server is reachable.
/// - Checks if the model exists locally and downloads it if missing.
pub async fn ensure_oss_ready(config: &Config) -> std::io::Result<()> {
let model: &str = config.model.as_ref();
// Verify local LM Studio is reachable.
let lmstudio_client = LMStudioClient::try_from_provider(config).await?;
match lmstudio_client.fetch_models().await {
Ok(models) => {
if !models.iter().any(|m| m == model) {
lmstudio_client.download_model(model).await?;
}
}
Err(err) => {
// Not fatal; higher layers may still proceed and surface errors later.
tracing::warn!("Failed to query local models from LM Studio: {}.", err);
}
}
// Load the model in the background
tokio::spawn({
let client = lmstudio_client.clone();
let model = model.to_string();
async move {
if let Err(e) = client.load_model(&model).await {
tracing::warn!("Failed to load model {}: {}", model, e);
}
}
});
Ok(())
}

View File

@@ -258,6 +258,9 @@ async fn run_codex_tool_session_inner(
EventMsg::AgentReasoningDelta(_) => {
// TODO: think how we want to support this in the MCP
}
EventMsg::McpStartupUpdate(_) | EventMsg::McpStartupComplete(_) => {
// Ignored in MCP tool runner.
}
EventMsg::AgentMessage(AgentMessageEvent { .. }) => {
// TODO: think how we want to support this in the MCP
}

View File

@@ -10,8 +10,8 @@ use crate::pull::PullEvent;
use crate::pull::PullProgressReporter;
use crate::url::base_url_to_host_root;
use crate::url::is_openai_compatible_base_url;
use codex_core::BUILT_IN_OSS_MODEL_PROVIDER_ID;
use codex_core::ModelProviderInfo;
use codex_core::OLLAMA_OSS_PROVIDER_ID;
use codex_core::WireApi;
use codex_core::config::Config;
@@ -34,11 +34,11 @@ impl OllamaClient {
// account.
let provider = config
.model_providers
.get(BUILT_IN_OSS_MODEL_PROVIDER_ID)
.get(OLLAMA_OSS_PROVIDER_ID)
.ok_or_else(|| {
io::Error::new(
io::ErrorKind::NotFound,
format!("Built-in provider {BUILT_IN_OSS_MODEL_PROVIDER_ID} not found",),
format!("Built-in provider {OLLAMA_OSS_PROVIDER_ID} not found",),
)
})?;
@@ -47,7 +47,8 @@ impl OllamaClient {
#[cfg(test)]
async fn try_from_provider_with_base_url(base_url: &str) -> io::Result<Self> {
let provider = codex_core::create_oss_provider_with_base_url(base_url);
let provider =
codex_core::create_oss_provider_with_base_url(base_url, codex_core::WireApi::Chat);
Self::try_from_provider(&provider).await
}

View File

@@ -478,6 +478,12 @@ pub enum EventMsg {
/// Ack the client's configure message.
SessionConfigured(SessionConfiguredEvent),
/// Incremental MCP startup progress updates.
McpStartupUpdate(McpStartupUpdateEvent),
/// Aggregate MCP startup completion summary.
McpStartupComplete(McpStartupCompleteEvent),
McpToolCallBegin(McpToolCallBeginEvent),
McpToolCallEnd(McpToolCallEndEvent),
@@ -1383,6 +1389,37 @@ pub struct McpListToolsResponseEvent {
pub auth_statuses: std::collections::HashMap<String, McpAuthStatus>,
}
#[derive(Debug, Clone, Deserialize, Serialize, JsonSchema, TS)]
pub struct McpStartupUpdateEvent {
/// Server name being started.
pub server: String,
/// Current startup status.
pub status: McpStartupStatus,
}
#[derive(Debug, Clone, Deserialize, Serialize, JsonSchema, TS)]
#[serde(rename_all = "snake_case", tag = "state")]
#[ts(rename_all = "snake_case", tag = "state")]
pub enum McpStartupStatus {
Starting,
Ready,
Failed { error: String },
Cancelled,
}
#[derive(Debug, Clone, Deserialize, Serialize, JsonSchema, TS, Default)]
pub struct McpStartupCompleteEvent {
pub ready: Vec<String>,
pub failed: Vec<McpStartupFailure>,
pub cancelled: Vec<String>,
}
#[derive(Debug, Clone, Deserialize, Serialize, JsonSchema, TS)]
pub struct McpStartupFailure {
pub server: String,
pub error: String,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, JsonSchema, TS)]
#[serde(rename_all = "snake_case")]
#[ts(rename_all = "snake_case")]
@@ -1589,4 +1626,47 @@ mod tests {
assert_eq!(deserialized, event);
Ok(())
}
#[test]
fn serialize_mcp_startup_update_event() -> Result<()> {
let event = Event {
id: "init".to_string(),
msg: EventMsg::McpStartupUpdate(McpStartupUpdateEvent {
server: "srv".to_string(),
status: McpStartupStatus::Failed {
error: "boom".to_string(),
},
}),
};
let value = serde_json::to_value(&event)?;
assert_eq!(value["msg"]["type"], "mcp_startup_update");
assert_eq!(value["msg"]["server"], "srv");
assert_eq!(value["msg"]["status"]["state"], "failed");
assert_eq!(value["msg"]["status"]["error"], "boom");
Ok(())
}
#[test]
fn serialize_mcp_startup_complete_event() -> Result<()> {
let event = Event {
id: "init".to_string(),
msg: EventMsg::McpStartupComplete(McpStartupCompleteEvent {
ready: vec!["a".to_string()],
failed: vec![McpStartupFailure {
server: "b".to_string(),
error: "bad".to_string(),
}],
cancelled: vec!["c".to_string()],
}),
};
let value = serde_json::to_value(&event)?;
assert_eq!(value["msg"]["type"], "mcp_startup_complete");
assert_eq!(value["msg"]["ready"][0], "a");
assert_eq!(value["msg"]["failed"][0]["server"], "b");
assert_eq!(value["msg"]["failed"][0]["error"], "bad");
assert_eq!(value["msg"]["cancelled"][0], "c");
Ok(())
}
}

View File

@@ -38,7 +38,6 @@ codex-core = { workspace = true }
codex-feedback = { workspace = true }
codex-file-search = { workspace = true }
codex-login = { workspace = true }
codex-ollama = { workspace = true }
codex-protocol = { workspace = true }
color-eyre = { workspace = true }
crossterm = { workspace = true, features = ["bracketed-paste", "event-stream"] }
@@ -62,6 +61,7 @@ ratatui = { workspace = true, features = [
] }
ratatui-macros = { workspace = true }
regex-lite = { workspace = true }
reqwest = { version = "0.12", features = ["json"] }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true, features = ["preserve_order"] }
shlex = { workspace = true }

View File

@@ -279,20 +279,23 @@ impl BottomPane {
// esc_backtrack_hint_visible removed; hints are controlled internally.
pub fn set_task_running(&mut self, running: bool) {
let was_running = self.is_task_running;
self.is_task_running = running;
self.composer.set_task_running(running);
if running {
if self.status.is_none() {
self.status = Some(StatusIndicatorWidget::new(
self.app_event_tx.clone(),
self.frame_requester.clone(),
));
if !was_running {
if self.status.is_none() {
self.status = Some(StatusIndicatorWidget::new(
self.app_event_tx.clone(),
self.frame_requester.clone(),
));
}
if let Some(status) = self.status.as_mut() {
status.set_interrupt_hint_visible(true);
}
self.request_redraw();
}
if let Some(status) = self.status.as_mut() {
status.set_interrupt_hint_visible(true);
}
self.request_redraw();
} else {
// Hide the status indicator when a task completes, but keep other modal views.
self.hide_status_indicator();

View File

@@ -27,6 +27,9 @@ use codex_core::protocol::ExecCommandSource;
use codex_core::protocol::ExitedReviewModeEvent;
use codex_core::protocol::ListCustomPromptsResponseEvent;
use codex_core::protocol::McpListToolsResponseEvent;
use codex_core::protocol::McpStartupCompleteEvent;
use codex_core::protocol::McpStartupStatus;
use codex_core::protocol::McpStartupUpdateEvent;
use codex_core::protocol::McpToolCallBeginEvent;
use codex_core::protocol::McpToolCallEndEvent;
use codex_core::protocol::Op;
@@ -259,6 +262,7 @@ pub(crate) struct ChatWidget {
stream_controller: Option<StreamController>,
running_commands: HashMap<String, RunningCommand>,
task_complete_pending: bool,
mcp_startup_status: Option<HashMap<String, McpStartupStatus>>,
// Queue of interruptive UI events deferred during an active write cycle
interrupts: InterruptManager,
// Accumulates the current reasoning block text to extract a header
@@ -567,8 +571,76 @@ impl ChatWidget {
self.maybe_send_next_queued_input();
}
fn on_warning(&mut self, message: String) {
self.add_to_history(history_cell::new_warning_event(message));
fn on_warning(&mut self, message: impl Into<String>) {
self.add_to_history(history_cell::new_warning_event(message.into()));
self.request_redraw();
}
fn on_mcp_startup_update(&mut self, ev: McpStartupUpdateEvent) {
let mut status = self.mcp_startup_status.take().unwrap_or_default();
if let McpStartupStatus::Failed { error } = &ev.status {
self.on_warning(error);
}
status.insert(ev.server, ev.status);
self.mcp_startup_status = Some(status);
self.bottom_pane.set_task_running(true);
if let Some(current) = &self.mcp_startup_status {
let total = current.len();
let mut starting: Vec<_> = current
.iter()
.filter_map(|(name, state)| {
if matches!(state, McpStartupStatus::Starting) {
Some(name)
} else {
None
}
})
.collect();
starting.sort();
if let Some(first) = starting.first() {
let completed = total.saturating_sub(starting.len());
let max_to_show = 3;
let mut to_show: Vec<String> = starting
.iter()
.take(max_to_show)
.map(ToString::to_string)
.collect();
if starting.len() > max_to_show {
to_show.push("".to_string());
}
let header = if total > 1 {
format!(
"Starting MCP servers ({completed}/{total}): {}",
to_show.join(", ")
)
} else {
format!("Booting MCP server: {first}")
};
self.set_status_header(header);
}
}
self.request_redraw();
}
fn on_mcp_startup_complete(&mut self, ev: McpStartupCompleteEvent) {
let mut parts = Vec::new();
if !ev.failed.is_empty() {
let failed_servers: Vec<_> = ev.failed.iter().map(|f| f.server.clone()).collect();
parts.push(format!("failed: {}", failed_servers.join(", ")));
}
if !ev.cancelled.is_empty() {
self.on_warning(format!(
"MCP startup interrupted. The following servers were not initialized: {}",
ev.cancelled.join(", ")
));
}
if !parts.is_empty() {
self.on_warning(format!("MCP startup incomplete ({})", parts.join("; ")));
}
self.mcp_startup_status = None;
self.bottom_pane.set_task_running(false);
self.maybe_send_next_queued_input();
self.request_redraw();
}
@@ -1061,6 +1133,7 @@ impl ChatWidget {
stream_controller: None,
running_commands: HashMap::new(),
task_complete_pending: false,
mcp_startup_status: None,
interrupts: InterruptManager::new(),
reasoning_buffer: String::new(),
full_reasoning_buffer: String::new(),
@@ -1128,6 +1201,7 @@ impl ChatWidget {
stream_controller: None,
running_commands: HashMap::new(),
task_complete_pending: false,
mcp_startup_status: None,
interrupts: InterruptManager::new(),
reasoning_buffer: String::new(),
full_reasoning_buffer: String::new(),
@@ -1540,6 +1614,8 @@ impl ChatWidget {
}
EventMsg::Warning(WarningEvent { message }) => self.on_warning(message),
EventMsg::Error(ErrorEvent { message }) => self.on_error(message),
EventMsg::McpStartupUpdate(ev) => self.on_mcp_startup_update(ev),
EventMsg::McpStartupComplete(ev) => self.on_mcp_startup_complete(ev),
EventMsg::TurnAborted(ev) => match ev.reason {
TurnAbortReason::Interrupted => {
self.on_interrupted_turn(ev.reason);

View File

@@ -12,7 +12,6 @@ use codex_core::CodexAuth;
use codex_core::config::Config;
use codex_core::config::ConfigOverrides;
use codex_core::config::ConfigToml;
use codex_core::config::OPENAI_DEFAULT_MODEL;
use codex_core::protocol::AgentMessageDeltaEvent;
use codex_core::protocol::AgentMessageEvent;
use codex_core::protocol::AgentReasoningDeltaEvent;
@@ -53,17 +52,12 @@ use crossterm::event::KeyEvent;
use crossterm::event::KeyModifiers;
use insta::assert_snapshot;
use pretty_assertions::assert_eq;
use std::fs::File;
use std::io::BufRead;
use std::io::BufReader;
use std::path::PathBuf;
use tempfile::NamedTempFile;
use tempfile::tempdir;
use tokio::sync::mpsc::error::TryRecvError;
use tokio::sync::mpsc::unbounded_channel;
const TEST_WARNING_MESSAGE: &str = "Heads up: Long conversations and multiple compactions can cause the model to be less accurate. Start a new conversation when possible to keep conversations small and targeted.";
fn test_config() -> Config {
// Use base defaults to avoid depending on host state.
Config::load_from_base_config_with_overrides(
@@ -74,70 +68,6 @@ fn test_config() -> Config {
.expect("config")
}
// Backward-compat shim for older session logs that predate the
// `formatted_output` field on ExecCommandEnd events.
fn upgrade_event_payload_for_tests(mut payload: serde_json::Value) -> serde_json::Value {
if let Some(obj) = payload.as_object_mut()
&& let Some(msg) = obj.get_mut("msg")
&& let Some(m) = msg.as_object_mut()
{
let ty = m.get("type").and_then(|v| v.as_str()).unwrap_or("");
if ty == "exec_command_end" {
let stdout = m.get("stdout").and_then(|v| v.as_str()).unwrap_or("");
let stderr = m.get("stderr").and_then(|v| v.as_str()).unwrap_or("");
let aggregated = if stderr.is_empty() {
stdout.to_string()
} else {
format!("{stdout}{stderr}")
};
if !m.contains_key("formatted_output") {
m.insert(
"formatted_output".to_string(),
serde_json::Value::String(aggregated.clone()),
);
}
if !m.contains_key("aggregated_output") {
m.insert(
"aggregated_output".to_string(),
serde_json::Value::String(aggregated),
);
}
} else if ty == "patch_apply_begin"
&& let Some(changes) = m.get_mut("changes").and_then(|v| v.as_object_mut())
{
for change in changes.values_mut() {
if change.get("type").is_some() {
continue;
}
if let Some(change_obj) = change.as_object_mut()
&& change_obj.len() == 1
&& let Some((legacy_kind, legacy_value)) = change_obj
.iter()
.next()
.map(|(k, v)| (k.clone(), v.clone()))
{
change_obj.clear();
change_obj.insert(
"type".to_string(),
serde_json::Value::String(legacy_kind.clone()),
);
match legacy_value {
serde_json::Value::Object(payload) => {
for (k, v) in payload {
change_obj.insert(k, v);
}
}
other => {
change_obj.insert("content".to_string(), other);
}
}
}
}
}
}
payload
}
fn snapshot(percent: f64) -> RateLimitSnapshot {
RateLimitSnapshot {
primary: Some(RateLimitWindow {
@@ -336,6 +266,7 @@ fn make_chatwidget_manual() -> (
stream_controller: None,
running_commands: HashMap::new(),
task_complete_pending: false,
mcp_startup_status: None,
interrupts: InterruptManager::new(),
reasoning_buffer: String::new(),
full_reasoning_buffer: String::new(),
@@ -718,30 +649,6 @@ fn active_blob(chat: &ChatWidget) -> String {
lines_to_single_string(&lines)
}
fn open_fixture(name: &str) -> File {
// 1) Prefer fixtures within this crate
{
let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
p.push("tests");
p.push("fixtures");
p.push(name);
if let Ok(f) = File::open(&p) {
return f;
}
}
// 2) Fallback to parent (workspace root)
{
let mut p = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
p.push("..");
p.push(name);
if let Ok(f) = File::open(&p) {
return f;
}
}
// 3) Last resort: CWD
File::open(name).expect("open fixture file")
}
#[test]
fn empty_enter_during_task_does_not_queue() {
let (mut chat, _rx, _op_rx) = make_chatwidget_manual();
@@ -1726,167 +1633,6 @@ fn disabled_slash_command_while_task_running_snapshot() {
assert_snapshot!(blob);
}
#[tokio::test]
async fn binary_size_transcript_snapshot() {
// the snapshot in this test depends on gpt-5-codex. Skip for now. We will consider
// creating snapshots for other models in the future.
if OPENAI_DEFAULT_MODEL != "gpt-5-codex" {
return;
}
let (mut chat, mut rx, _op_rx) = make_chatwidget_manual();
// Set up a VT100 test terminal to capture ANSI visual output
let width: u16 = 80;
let height: u16 = 2000;
let viewport = Rect::new(0, height - 1, width, 1);
let backend = VT100Backend::new(width, height);
let mut terminal = crate::custom_terminal::Terminal::with_options(backend)
.expect("failed to construct terminal");
terminal.set_viewport_area(viewport);
// Replay the recorded session into the widget and collect transcript
let file = open_fixture("binary-size-log.jsonl");
let reader = BufReader::new(file);
let mut transcript = String::new();
let mut has_emitted_history = false;
for line in reader.lines() {
let line = line.expect("read line");
if line.trim().is_empty() || line.starts_with('#') {
continue;
}
let Ok(v): Result<serde_json::Value, _> = serde_json::from_str(&line) else {
continue;
};
let Some(dir) = v.get("dir").and_then(|d| d.as_str()) else {
continue;
};
if dir != "to_tui" {
continue;
}
let Some(kind) = v.get("kind").and_then(|k| k.as_str()) else {
continue;
};
match kind {
"codex_event" => {
if let Some(payload) = v.get("payload") {
let ev: Event =
serde_json::from_value(upgrade_event_payload_for_tests(payload.clone()))
.expect("parse");
let ev = match ev {
Event {
msg: EventMsg::ExecCommandBegin(e),
..
} => {
// Re-parse the command
let parsed_cmd = codex_core::parse_command::parse_command(&e.command);
Event {
id: ev.id,
msg: EventMsg::ExecCommandBegin(ExecCommandBeginEvent {
call_id: e.call_id.clone(),
command: e.command,
cwd: e.cwd,
parsed_cmd,
source: ExecCommandSource::Agent,
interaction_input: e.interaction_input.clone(),
}),
}
}
_ => ev,
};
chat.handle_codex_event(ev);
while let Ok(app_ev) = rx.try_recv() {
if let AppEvent::InsertHistoryCell(cell) = app_ev {
let mut lines = cell.display_lines(width);
if has_emitted_history
&& !cell.is_stream_continuation()
&& !lines.is_empty()
{
lines.insert(0, "".into());
}
has_emitted_history = true;
transcript.push_str(&lines_to_single_string(&lines));
crate::insert_history::insert_history_lines(&mut terminal, lines)
.expect("Failed to insert history lines in test");
}
}
}
}
"app_event" => {
if let Some(variant) = v.get("variant").and_then(|s| s.as_str())
&& variant == "CommitTick"
{
chat.on_commit_tick();
while let Ok(app_ev) = rx.try_recv() {
if let AppEvent::InsertHistoryCell(cell) = app_ev {
let mut lines = cell.display_lines(width);
if has_emitted_history
&& !cell.is_stream_continuation()
&& !lines.is_empty()
{
lines.insert(0, "".into());
}
has_emitted_history = true;
transcript.push_str(&lines_to_single_string(&lines));
crate::insert_history::insert_history_lines(&mut terminal, lines)
.expect("Failed to insert history lines in test");
}
}
}
}
_ => {}
}
}
// Build the final VT100 visual by parsing the ANSI stream. Trim trailing spaces per line
// and drop trailing empty lines so the shape matches the ideal fixture exactly.
let screen = terminal.backend().vt100().screen();
let mut lines: Vec<String> = Vec::with_capacity(height as usize);
for row in 0..height {
let mut s = String::with_capacity(width as usize);
for col in 0..width {
if let Some(cell) = screen.cell(row, col) {
if let Some(ch) = cell.contents().chars().next() {
s.push(ch);
} else {
s.push(' ');
}
} else {
s.push(' ');
}
}
// Trim trailing spaces to match plain text fixture
lines.push(s.trim_end().to_string());
}
while lines.last().is_some_and(std::string::String::is_empty) {
lines.pop();
}
// Consider content only after the last session banner marker. Skip the transient
// 'thinking' header if present, and start from the first non-empty content line
// that follows. This keeps the snapshot stable across sessions.
const MARKER_PREFIX: &str = "To get started, describe a task or try one of these commands:";
let last_marker_line_idx = lines
.iter()
.rposition(|l| l.trim_start().starts_with(MARKER_PREFIX))
.expect("marker not found in visible output");
// Prefer the first assistant content line (blockquote '>' prefix) after the marker;
// fallback to the first non-empty, non-'thinking' line.
let start_idx = (last_marker_line_idx + 1..lines.len())
.find(|&idx| lines[idx].trim_start().starts_with('•'))
.unwrap_or_else(|| {
(last_marker_line_idx + 1..lines.len())
.find(|&idx| {
let t = lines[idx].trim_start();
!t.is_empty() && t != "thinking"
})
.expect("no content line found after marker")
});
// Snapshot the normalized visible transcript following the banner.
assert_snapshot!("binary_size_ideal_response", lines[start_idx..].join("\n"));
}
//
// Snapshot test: command approval modal
//
@@ -2692,7 +2438,7 @@ fn warning_event_adds_warning_history_cell() {
chat.handle_codex_event(Event {
id: "sub-1".into(),
msg: EventMsg::Warning(WarningEvent {
message: TEST_WARNING_MESSAGE.to_string(),
message: "test warning message".to_string(),
}),
});
@@ -2700,7 +2446,7 @@ fn warning_event_adds_warning_history_cell() {
assert_eq!(cells.len(), 1, "expected one warning history cell");
let rendered = lines_to_single_string(&cells[0]);
assert!(
rendered.contains(TEST_WARNING_MESSAGE),
rendered.contains("test warning message"),
"warning cell missing content: {rendered}"
);
}

View File

@@ -32,12 +32,16 @@ pub struct Cli {
#[arg(long, short = 'm')]
pub model: Option<String>,
/// Convenience flag to select the local open source model provider.
/// Equivalent to -c model_provider=oss; verifies a local Ollama server is
/// running.
/// Convenience flag to select the local open source model provider. Equivalent to -c
/// model_provider=oss; verifies a local LM Studio or Ollama server is running.
#[arg(long = "oss", default_value_t = false)]
pub oss: bool,
/// Specify which local provider to use (lmstudio or ollama).
/// If not specified with --oss, will use config default or show selection.
#[arg(long = "local-provider")]
pub oss_provider: Option<String>,
/// Configuration profile from config.toml to specify default options.
#[arg(long = "profile", short = 'p')]
pub config_profile: Option<String>,

View File

@@ -1018,10 +1018,8 @@ fn try_new_completed_mcp_tool_call_with_image_output(
}
#[allow(clippy::disallowed_methods)]
pub(crate) fn new_warning_event(message: String) -> PlainHistoryCell {
PlainHistoryCell {
lines: vec![vec![format!("{message}").yellow()].into()],
}
pub(crate) fn new_warning_event(message: String) -> PrefixedWrappedHistoryCell {
PrefixedWrappedHistoryCell::new(message.yellow(), "".yellow(), " ")
}
#[derive(Debug)]

View File

@@ -7,18 +7,21 @@ use additional_dirs::add_dir_warning_message;
use app::App;
pub use app::AppExitInfo;
use codex_app_server_protocol::AuthMode;
use codex_common::oss::ensure_oss_provider_ready;
use codex_common::oss::get_default_model_for_oss_provider;
use codex_core::AuthManager;
use codex_core::BUILT_IN_OSS_MODEL_PROVIDER_ID;
use codex_core::CodexAuth;
use codex_core::INTERACTIVE_SESSION_SOURCES;
use codex_core::RolloutRecorder;
use codex_core::auth::enforce_login_restrictions;
use codex_core::config::Config;
use codex_core::config::ConfigOverrides;
use codex_core::config::find_codex_home;
use codex_core::config::load_config_as_toml_with_cli_overrides;
use codex_core::config::resolve_oss_provider;
use codex_core::find_conversation_path_by_id_str;
use codex_core::get_platform_sandbox;
use codex_core::protocol::AskForApproval;
use codex_ollama::DEFAULT_OSS_MODEL;
use codex_protocol::config_types::SandboxMode;
use opentelemetry_appender_tracing::layer::OpenTelemetryTracingBridge;
use std::fs::OpenOptions;
@@ -56,6 +59,7 @@ mod markdown_render;
mod markdown_stream;
mod model_migration;
pub mod onboarding;
mod oss_selection;
mod pager_overlay;
pub mod public_widgets;
mod render;
@@ -124,21 +128,75 @@ pub async fn run_main(
// When using `--oss`, let the bootstrapper pick the model (defaulting to
// gpt-oss:20b) and ensure it is present locally. Also, force the builtin
let raw_overrides = cli.config_overrides.raw_overrides.clone();
// `oss` model provider.
let overrides_cli = codex_common::CliConfigOverrides { raw_overrides };
let cli_kv_overrides = match overrides_cli.parse_overrides() {
// Parse `-c` overrides from the CLI.
Ok(v) => v,
#[allow(clippy::print_stderr)]
Err(e) => {
eprintln!("Error parsing -c overrides: {e}");
std::process::exit(1);
}
};
// we load config.toml here to determine project state.
#[allow(clippy::print_stderr)]
let codex_home = match find_codex_home() {
Ok(codex_home) => codex_home.to_path_buf(),
Err(err) => {
eprintln!("Error finding codex home: {err}");
std::process::exit(1);
}
};
#[allow(clippy::print_stderr)]
let config_toml =
match load_config_as_toml_with_cli_overrides(&codex_home, cli_kv_overrides.clone()).await {
Ok(config_toml) => config_toml,
Err(err) => {
eprintln!("Error loading config.toml: {err}");
std::process::exit(1);
}
};
let model_provider_override = if cli.oss {
let resolved = resolve_oss_provider(
cli.oss_provider.as_deref(),
&config_toml,
cli.config_profile.clone(),
);
if let Some(provider) = resolved {
Some(provider)
} else {
// No provider configured, prompt the user
let provider = oss_selection::select_oss_provider(&codex_home).await?;
if provider == "__CANCELLED__" {
return Err(std::io::Error::other(
"OSS provider selection was cancelled by user",
));
}
Some(provider)
}
} else {
None
};
// When using `--oss`, let the bootstrapper pick the model based on selected provider
let model = if let Some(model) = &cli.model {
Some(model.clone())
} else if cli.oss {
Some(DEFAULT_OSS_MODEL.to_owned())
// Use the provider from model_provider_override
model_provider_override
.as_ref()
.and_then(|provider_id| get_default_model_for_oss_provider(provider_id))
.map(std::borrow::ToOwned::to_owned)
} else {
None // No model specified, will use the default.
};
let model_provider_override = if cli.oss {
Some(BUILT_IN_OSS_MODEL_PROVIDER_ID.to_owned())
} else {
None
};
// canonicalize the cwd
let cwd = cli.cwd.clone().map(|p| p.canonicalize().unwrap_or(p));
let additional_dirs = cli.add_dir.clone();
@@ -149,7 +207,7 @@ pub async fn run_main(
approval_policy,
sandbox_mode,
cwd,
model_provider: model_provider_override,
model_provider: model_provider_override.clone(),
config_profile: cli.config_profile.clone(),
codex_linux_sandbox_exe,
base_instructions: None,
@@ -161,16 +219,6 @@ pub async fn run_main(
experimental_sandbox_command_assessment: None,
additional_writable_roots: additional_dirs,
};
let raw_overrides = cli.config_overrides.raw_overrides.clone();
let overrides_cli = codex_common::CliConfigOverrides { raw_overrides };
let cli_kv_overrides = match overrides_cli.parse_overrides() {
Ok(v) => v,
#[allow(clippy::print_stderr)]
Err(e) => {
eprintln!("Error parsing -c overrides: {e}");
std::process::exit(1);
}
};
let config = load_config_or_exit(cli_kv_overrides.clone(), overrides.clone()).await;
@@ -232,10 +280,19 @@ pub async fn run_main(
.with_target(false)
.with_filter(targets);
if cli.oss {
codex_ollama::ensure_oss_ready(&config)
.await
.map_err(|e| std::io::Error::other(format!("OSS setup failed: {e}")))?;
if cli.oss && model_provider_override.is_some() {
// We're in the oss section, so provider_id should be Some
// Let's handle None case gracefully though just in case
let provider_id = match model_provider_override.as_ref() {
Some(id) => id,
None => {
error!("OSS provider unexpectedly not set when oss flag is used");
return Err(std::io::Error::other(
"OSS provider not set but oss flag was used",
));
}
};
ensure_oss_provider_ready(provider_id, &config).await?;
}
let otel = codex_core::otel_init::build_provider(&config, env!("CARGO_PKG_VERSION"));

View File

@@ -0,0 +1,369 @@
use std::io;
use std::sync::LazyLock;
use codex_core::DEFAULT_LMSTUDIO_PORT;
use codex_core::DEFAULT_OLLAMA_PORT;
use codex_core::LMSTUDIO_OSS_PROVIDER_ID;
use codex_core::OLLAMA_OSS_PROVIDER_ID;
use codex_core::config::set_default_oss_provider;
use crossterm::event::Event;
use crossterm::event::KeyCode;
use crossterm::event::KeyEvent;
use crossterm::event::KeyEventKind;
use crossterm::event::{self};
use crossterm::execute;
use crossterm::terminal::EnterAlternateScreen;
use crossterm::terminal::LeaveAlternateScreen;
use crossterm::terminal::disable_raw_mode;
use crossterm::terminal::enable_raw_mode;
use ratatui::Terminal;
use ratatui::backend::CrosstermBackend;
use ratatui::buffer::Buffer;
use ratatui::layout::Alignment;
use ratatui::layout::Constraint;
use ratatui::layout::Direction;
use ratatui::layout::Layout;
use ratatui::layout::Margin;
use ratatui::layout::Rect;
use ratatui::prelude::*;
use ratatui::style::Color;
use ratatui::style::Modifier;
use ratatui::style::Style;
use ratatui::text::Line;
use ratatui::text::Span;
use ratatui::widgets::Paragraph;
use ratatui::widgets::Widget;
use ratatui::widgets::WidgetRef;
use ratatui::widgets::Wrap;
use std::time::Duration;
#[derive(Clone)]
struct ProviderOption {
name: String,
status: ProviderStatus,
}
#[derive(Clone)]
enum ProviderStatus {
Running,
NotRunning,
Unknown,
}
/// Options displayed in the *select* mode.
///
/// The `key` is matched case-insensitively.
struct SelectOption {
label: Line<'static>,
description: &'static str,
key: KeyCode,
provider_id: &'static str,
}
static OSS_SELECT_OPTIONS: LazyLock<Vec<SelectOption>> = LazyLock::new(|| {
vec![
SelectOption {
label: Line::from(vec!["L".underlined(), "M Studio".into()]),
description: "Local LM Studio server (default port 1234)",
key: KeyCode::Char('l'),
provider_id: LMSTUDIO_OSS_PROVIDER_ID,
},
SelectOption {
label: Line::from(vec!["O".underlined(), "llama".into()]),
description: "Local Ollama server (default port 11434)",
key: KeyCode::Char('o'),
provider_id: OLLAMA_OSS_PROVIDER_ID,
},
]
});
pub struct OssSelectionWidget<'a> {
select_options: &'a Vec<SelectOption>,
confirmation_prompt: Paragraph<'a>,
/// Currently selected index in *select* mode.
selected_option: usize,
/// Set to `true` once a decision has been sent the parent view can then
/// remove this widget from its queue.
done: bool,
selection: Option<String>,
}
impl OssSelectionWidget<'_> {
fn new(lmstudio_status: ProviderStatus, ollama_status: ProviderStatus) -> io::Result<Self> {
let providers = vec![
ProviderOption {
name: "LM Studio".to_string(),
status: lmstudio_status,
},
ProviderOption {
name: "Ollama".to_string(),
status: ollama_status,
},
];
let mut contents: Vec<Line> = vec![
Line::from(vec![
"? ".fg(Color::Blue),
"Select an open-source provider".bold(),
]),
Line::from(""),
Line::from(" Choose which local AI server to use for your session."),
Line::from(""),
];
// Add status indicators for each provider
for provider in &providers {
let (status_symbol, status_color) = get_status_symbol_and_color(&provider.status);
contents.push(Line::from(vec![
Span::raw(" "),
Span::styled(status_symbol, Style::default().fg(status_color)),
Span::raw(format!(" {} ", provider.name)),
]));
}
contents.push(Line::from(""));
contents.push(Line::from(" ● Running ○ Not Running").add_modifier(Modifier::DIM));
contents.push(Line::from(""));
contents.push(
Line::from(" Press Enter to select • Ctrl+C to exit").add_modifier(Modifier::DIM),
);
let confirmation_prompt = Paragraph::new(contents).wrap(Wrap { trim: false });
Ok(Self {
select_options: &OSS_SELECT_OPTIONS,
confirmation_prompt,
selected_option: 0,
done: false,
selection: None,
})
}
fn get_confirmation_prompt_height(&self, width: u16) -> u16 {
// Should cache this for last value of width.
self.confirmation_prompt.line_count(width) as u16
}
/// Process a `KeyEvent` coming from crossterm. Always consumes the event
/// while the modal is visible.
/// Process a key event originating from crossterm. As the modal fully
/// captures input while visible, we don't need to report whether the event
/// was consumed—callers can assume it always is.
pub fn handle_key_event(&mut self, key: KeyEvent) -> Option<String> {
if key.kind == KeyEventKind::Press {
self.handle_select_key(key);
}
if self.done {
self.selection.clone()
} else {
None
}
}
/// Normalize a key for comparison.
/// - For `KeyCode::Char`, converts to lowercase for case-insensitive matching.
/// - Other key codes are returned unchanged.
fn normalize_keycode(code: KeyCode) -> KeyCode {
match code {
KeyCode::Char(c) => KeyCode::Char(c.to_ascii_lowercase()),
other => other,
}
}
fn handle_select_key(&mut self, key_event: KeyEvent) {
match key_event.code {
KeyCode::Char('c')
if key_event
.modifiers
.contains(crossterm::event::KeyModifiers::CONTROL) =>
{
self.send_decision("__CANCELLED__".to_string());
}
KeyCode::Left => {
self.selected_option = (self.selected_option + self.select_options.len() - 1)
% self.select_options.len();
}
KeyCode::Right => {
self.selected_option = (self.selected_option + 1) % self.select_options.len();
}
KeyCode::Enter => {
let opt = &self.select_options[self.selected_option];
self.send_decision(opt.provider_id.to_string());
}
KeyCode::Esc => {
self.send_decision(LMSTUDIO_OSS_PROVIDER_ID.to_string());
}
other => {
let normalized = Self::normalize_keycode(other);
if let Some(opt) = self
.select_options
.iter()
.find(|opt| Self::normalize_keycode(opt.key) == normalized)
{
self.send_decision(opt.provider_id.to_string());
}
}
}
}
fn send_decision(&mut self, selection: String) {
self.selection = Some(selection);
self.done = true;
}
/// Returns `true` once the user has made a decision and the widget no
/// longer needs to be displayed.
pub fn is_complete(&self) -> bool {
self.done
}
pub fn desired_height(&self, width: u16) -> u16 {
self.get_confirmation_prompt_height(width) + self.select_options.len() as u16
}
}
impl WidgetRef for &OssSelectionWidget<'_> {
fn render_ref(&self, area: Rect, buf: &mut Buffer) {
let prompt_height = self.get_confirmation_prompt_height(area.width);
let [prompt_chunk, response_chunk] = Layout::default()
.direction(Direction::Vertical)
.constraints([Constraint::Length(prompt_height), Constraint::Min(0)])
.areas(area);
let lines: Vec<Line> = self
.select_options
.iter()
.enumerate()
.map(|(idx, opt)| {
let style = if idx == self.selected_option {
Style::new().bg(Color::Cyan).fg(Color::Black)
} else {
Style::new().bg(Color::DarkGray)
};
opt.label.clone().alignment(Alignment::Center).style(style)
})
.collect();
let [title_area, button_area, description_area] = Layout::vertical([
Constraint::Length(1),
Constraint::Length(1),
Constraint::Min(0),
])
.areas(response_chunk.inner(Margin::new(1, 0)));
Line::from("Select provider?").render(title_area, buf);
self.confirmation_prompt.clone().render(prompt_chunk, buf);
let areas = Layout::horizontal(
lines
.iter()
.map(|l| Constraint::Length(l.width() as u16 + 2)),
)
.spacing(1)
.split(button_area);
for (idx, area) in areas.iter().enumerate() {
let line = &lines[idx];
line.render(*area, buf);
}
Line::from(self.select_options[self.selected_option].description)
.style(Style::new().italic().fg(Color::DarkGray))
.render(description_area.inner(Margin::new(1, 0)), buf);
}
}
fn get_status_symbol_and_color(status: &ProviderStatus) -> (&'static str, Color) {
match status {
ProviderStatus::Running => ("", Color::Green),
ProviderStatus::NotRunning => ("", Color::Red),
ProviderStatus::Unknown => ("?", Color::Yellow),
}
}
pub async fn select_oss_provider(codex_home: &std::path::Path) -> io::Result<String> {
// Check provider statuses first
let lmstudio_status = check_lmstudio_status().await;
let ollama_status = check_ollama_status().await;
// Autoselect if only one is running
match (&lmstudio_status, &ollama_status) {
(ProviderStatus::Running, ProviderStatus::NotRunning) => {
let provider = LMSTUDIO_OSS_PROVIDER_ID.to_string();
return Ok(provider);
}
(ProviderStatus::NotRunning, ProviderStatus::Running) => {
let provider = OLLAMA_OSS_PROVIDER_ID.to_string();
return Ok(provider);
}
_ => {
// Both running or both not running - show UI
}
}
let mut widget = OssSelectionWidget::new(lmstudio_status, ollama_status)?;
enable_raw_mode()?;
let mut stdout = io::stdout();
execute!(stdout, EnterAlternateScreen)?;
let backend = CrosstermBackend::new(stdout);
let mut terminal = Terminal::new(backend)?;
let result = loop {
terminal.draw(|f| {
(&widget).render_ref(f.area(), f.buffer_mut());
})?;
if let Event::Key(key_event) = event::read()?
&& let Some(selection) = widget.handle_key_event(key_event)
{
break Ok(selection);
}
};
disable_raw_mode()?;
execute!(terminal.backend_mut(), LeaveAlternateScreen)?;
// If the user manually selected an OSS provider, we save it as the
// default one to use later.
if let Ok(ref provider) = result
&& let Err(e) = set_default_oss_provider(codex_home, provider)
{
tracing::warn!("Failed to save OSS provider preference: {e}");
}
result
}
async fn check_lmstudio_status() -> ProviderStatus {
match check_port_status(DEFAULT_LMSTUDIO_PORT).await {
Ok(true) => ProviderStatus::Running,
Ok(false) => ProviderStatus::NotRunning,
Err(_) => ProviderStatus::Unknown,
}
}
async fn check_ollama_status() -> ProviderStatus {
match check_port_status(DEFAULT_OLLAMA_PORT).await {
Ok(true) => ProviderStatus::Running,
Ok(false) => ProviderStatus::NotRunning,
Err(_) => ProviderStatus::Unknown,
}
}
async fn check_port_status(port: u16) -> io::Result<bool> {
let client = reqwest::Client::builder()
.timeout(Duration::from_secs(2))
.build()
.map_err(io::Error::other)?;
let url = format!("http://localhost:{port}");
match client.get(&url).send().await {
Ok(response) => Ok(response.status().is_success()),
Err(_) => Ok(false), // Connection failed = not running
}
}

View File

@@ -145,7 +145,6 @@ impl Renderable for StatusIndicatorWidget {
let elapsed_duration = self.elapsed_duration_at(now);
let pretty_elapsed = fmt_elapsed_compact(elapsed_duration.as_secs());
// Plain rendering: no borders or padding so the live cell is visually indistinguishable from terminal scrollback.
let mut spans = Vec::with_capacity(5);
spans.push(spinner(Some(self.last_resume_at)));
spans.push(" ".into());

File diff suppressed because one or more lines are too long

View File

@@ -253,6 +253,20 @@ This is analogous to `model_context_window`, but for the maximum number of outpu
> See also [`codex exec`](./exec.md) to see how these model settings influence non-interactive runs.
### oss_provider
Specifies the default OSS provider to use when running Codex. This is used when the `--oss` flag is provided without a specific provider.
Valid values are:
- `"lmstudio"` - Use LM Studio as the local model provider
- `"ollama"` - Use Ollama as the local model provider
```toml
# Example: Set default OSS provider to LM Studio
oss_provider = "lmstudio"
```
## Execution environment
### approval_policy