clone/codex
1
0
Fork 0
mirror of https://github.com/openai/codex.git synced 2026-05-04 19:36:45 +00:00
Code Issues Packages Projects Releases Wiki Activity

[mcp] Expand tool search to custom MCPs. (#16944)

Browse Source 
- [x] Expand tool search to custom MCPs.
- [x] Rename several variables/fields to be more generic.

Updated tool & server name lifecycles:

**Raw Identity**

ToolInfo.server_name is raw MCP server name.
ToolInfo.tool.name is raw MCP tool name.
MCP calls route back to raw via parse_tool_name() returning
(tool.server_name, tool.tool.name).
mcpServerStatus/list now groups by raw server and keys tools by
Tool.name: mod.rs:599
App-server just forwards that grouped raw snapshot:
codex_message_processor.rs:5245

**Callable Names**

On list-tools, we create provisional callable_namespace / callable_name:
mcp_connection_manager.rs:1556
For non-app MCP, provisional callable name starts as raw tool name.
For codex-apps, provisional callable name is sanitized and strips
connector name/id prefix; namespace includes connector name.
Then qualify_tools() sanitizes callable namespace + name to ASCII alnum
/ _ only: mcp_tool_names.rs:128
Note: this is stricter than Responses API. Hyphen is currently replaced
with _ for code-mode compatibility.

**Collision Handling**

We do initially collapse example-server and example_server to the same
base.
Then qualify_tools() detects distinct raw namespace identities behind
the same sanitized namespace and appends a hash to the callable
namespace: mcp_tool_names.rs:137
Same idea for tool-name collisions: hash suffix goes on callable tool
name.
Final list_all_tools() map key is callable_namespace + callable_name:
mcp_connection_manager.rs:769

**Direct Model Tools**

Direct MCP tool declarations use the full qualified sanitized key as the
Responses function name.
The raw rmcp Tool is converted but renamed for model exposure.

**Tool Search / Deferred**

Tool search result namespace = final ToolInfo.callable_namespace:
tool_search.rs:85
Tool search result nested name = final ToolInfo.callable_name:
tool_search.rs:86
Deferred tool handler is registered as "{namespace}:{name}":
tool_registry_plan.rs:248
When a function call comes back, core recombines namespace + name, looks
up the full qualified key, and gets the raw server/tool for MCP
execution: codex.rs:4353

**Separate Legacy Snapshot**

collect_mcp_snapshot_from_manager_with_detail() still returns a map
keyed by qualified callable name.
mcpServerStatus/list no longer uses that; it uses
McpServerStatusSnapshot, which is raw-inventory shaped.
This commit is contained in:
Matthew Zeng
2026-04-09 13:34:52 -07:00
committed by GitHub
parent 545f3daba0
commit d7f99b0fa6
26 changed files with 1297 additions and 737 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
codex-rs/codex-mcp/src/lib.rs
View File

@@ -1,5 +1,6 @@
pub(crate) mod mcp;
pub(crate) mod mcp_connection_manager;
pub(crate) mod mcp_tool_names;
pub use mcp::CODEX_APPS_MCP_SERVER_NAME;
pub use mcp::McpAuthStatusEntry;
@@ -8,10 +9,13 @@ pub use mcp::McpManager;
pub use mcp::McpOAuthLoginConfig;
pub use mcp::McpOAuthLoginSupport;
pub use mcp::McpOAuthScopesSource;
pub use mcp::McpServerStatusSnapshot;
pub use mcp::McpSnapshotDetail;
pub use mcp::ResolvedMcpOAuthScopes;
pub use mcp::ToolPluginProvenance;
pub use mcp::canonical_mcp_server_key;
pub use mcp::collect_mcp_server_status_snapshot;
pub use mcp::collect_mcp_server_status_snapshot_with_detail;
pub use mcp::collect_mcp_snapshot;
pub use mcp::collect_mcp_snapshot_from_manager;
pub use mcp::collect_mcp_snapshot_from_manager_with_detail;

318
codex-rs/codex-mcp/src/mcp/mod.rs
View File

@@ -29,6 +29,7 @@ use codex_protocol::mcp::Resource;
use codex_protocol::mcp::ResourceTemplate;
use codex_protocol::mcp::Tool;
use codex_protocol::protocol::AskForApproval;
use codex_protocol::protocol::McpAuthStatus;
use codex_protocol::protocol::McpListToolsResponseEvent;
use codex_protocol::protocol::SandboxPolicy;
use serde_json::Value;
@@ -379,6 +380,79 @@ pub async fn collect_mcp_snapshot_with_detail(
snapshot
}
#[derive(Debug, Clone)]
pub struct McpServerStatusSnapshot {
pub tools_by_server: HashMap<String, HashMap<String, Tool>>,
pub resources: HashMap<String, Vec<Resource>>,
pub resource_templates: HashMap<String, Vec<ResourceTemplate>>,
pub auth_statuses: HashMap<String, McpAuthStatus>,
}
pub async fn collect_mcp_server_status_snapshot(
config: &McpConfig,
auth: Option<&CodexAuth>,
submit_id: String,
) -> McpServerStatusSnapshot {
collect_mcp_server_status_snapshot_with_detail(config, auth, submit_id, McpSnapshotDetail::Full)
.await
}
pub async fn collect_mcp_server_status_snapshot_with_detail(
config: &McpConfig,
auth: Option<&CodexAuth>,
submit_id: String,
detail: McpSnapshotDetail,
) -> McpServerStatusSnapshot {
let mcp_servers = effective_mcp_servers(config, auth);
let tool_plugin_provenance = tool_plugin_provenance(config);
if mcp_servers.is_empty() {
return McpServerStatusSnapshot {
tools_by_server: HashMap::new(),
resources: HashMap::new(),
resource_templates: HashMap::new(),
auth_statuses: HashMap::new(),
};
}
let auth_status_entries =
compute_auth_statuses(mcp_servers.iter(), config.mcp_oauth_credentials_store_mode).await;
let (tx_event, rx_event) = unbounded();
drop(rx_event);
let sandbox_state = SandboxState {
sandbox_policy: SandboxPolicy::new_read_only_policy(),
codex_linux_sandbox_exe: config.codex_linux_sandbox_exe.clone(),
sandbox_cwd: env::current_dir().unwrap_or_else(|_| PathBuf::from("/")),
use_legacy_landlock: config.use_legacy_landlock,
};
let (mcp_connection_manager, cancel_token) = McpConnectionManager::new(
&mcp_servers,
config.mcp_oauth_credentials_store_mode,
auth_status_entries.clone(),
&config.approval_policy,
submit_id,
tx_event,
sandbox_state,
config.codex_home.clone(),
codex_apps_tools_cache_key(auth),
tool_plugin_provenance,
)
.await;
let snapshot = collect_mcp_server_status_snapshot_from_manager(
&mcp_connection_manager,
auth_status_entries,
detail,
)
.await;
cancel_token.cancel();
snapshot
}
pub fn split_qualified_tool_name(qualified_name: &str) -> Option<(String, String)> {
let mut parts = qualified_name.split(MCP_TOOL_NAME_DELIMITER);
let prefix = parts.next()?;
@@ -408,6 +482,154 @@ pub fn group_tools_by_server(
grouped
}
fn protocol_tool_from_rmcp_tool(name: &str, tool: &rmcp::model::Tool) -> Option<Tool> {
match serde_json::to_value(tool) {
Ok(value) => match Tool::from_mcp_value(value) {
Ok(tool) => Some(tool),
Err(err) => {
tracing::warn!("Failed to convert MCP tool '{name}': {err}");
None
}
},
Err(err) => {
tracing::warn!("Failed to serialize MCP tool '{name}': {err}");
None
}
}
}
fn auth_statuses_from_entries(
auth_status_entries: &HashMap<String, crate::mcp::auth::McpAuthStatusEntry>,
) -> HashMap<String, McpAuthStatus> {
auth_status_entries
.iter()
.map(|(name, entry)| (name.clone(), entry.auth_status))
.collect::<HashMap<_, _>>()
}
fn convert_mcp_resources(
resources: HashMap<String, Vec<rmcp::model::Resource>>,
) -> HashMap<String, Vec<Resource>> {
resources
.into_iter()
.map(|(name, resources)| {
let resources = resources
.into_iter()
.filter_map(|resource| match serde_json::to_value(resource) {
Ok(value) => match Resource::from_mcp_value(value.clone()) {
Ok(resource) => Some(resource),
Err(err) => {
let (uri, resource_name) = match value {
Value::Object(obj) => (
obj.get("uri")
.and_then(|v| v.as_str().map(ToString::to_string)),
obj.get("name")
.and_then(|v| v.as_str().map(ToString::to_string)),
),
_ => (None, None),
};
tracing::warn!(
"Failed to convert MCP resource (uri={uri:?}, name={resource_name:?}): {err}"
);
None
}
},
Err(err) => {
tracing::warn!("Failed to serialize MCP resource: {err}");
None
}
})
.collect::<Vec<_>>();
(name, resources)
})
.collect::<HashMap<_, _>>()
}
fn convert_mcp_resource_templates(
resource_templates: HashMap<String, Vec<rmcp::model::ResourceTemplate>>,
) -> HashMap<String, Vec<ResourceTemplate>> {
resource_templates
.into_iter()
.map(|(name, templates)| {
let templates = templates
.into_iter()
.filter_map(|template| match serde_json::to_value(template) {
Ok(value) => match ResourceTemplate::from_mcp_value(value.clone()) {
Ok(template) => Some(template),
Err(err) => {
let (uri_template, template_name) = match value {
Value::Object(obj) => (
obj.get("uriTemplate")
.or_else(|| obj.get("uri_template"))
.and_then(|v| v.as_str().map(ToString::to_string)),
obj.get("name")
.and_then(|v| v.as_str().map(ToString::to_string)),
),
_ => (None, None),
};
tracing::warn!(
"Failed to convert MCP resource template (uri_template={uri_template:?}, name={template_name:?}): {err}"
);
None
}
},
Err(err) => {
tracing::warn!("Failed to serialize MCP resource template: {err}");
None
}
})
.collect::<Vec<_>>();
(name, templates)
})
.collect::<HashMap<_, _>>()
}
async fn collect_mcp_server_status_snapshot_from_manager(
mcp_connection_manager: &McpConnectionManager,
auth_status_entries: HashMap<String, crate::mcp::auth::McpAuthStatusEntry>,
detail: McpSnapshotDetail,
) -> McpServerStatusSnapshot {
let (tools, resources, resource_templates) = tokio::join!(
mcp_connection_manager.list_all_tools(),
async {
if detail.include_resources() {
mcp_connection_manager.list_all_resources().await
} else {
HashMap::new()
}
},
async {
if detail.include_resources() {
mcp_connection_manager.list_all_resource_templates().await
} else {
HashMap::new()
}
},
);
let mut tools_by_server = HashMap::<String, HashMap<String, Tool>>::new();
for (_qualified_name, tool_info) in tools {
let raw_tool_name = tool_info.tool.name.to_string();
let Some(tool) = protocol_tool_from_rmcp_tool(&raw_tool_name, &tool_info.tool) else {
continue;
};
let tool_name = tool.name.clone();
tools_by_server
.entry(tool_info.server_name)
.or_default()
.insert(tool_name, tool);
}
McpServerStatusSnapshot {
tools_by_server,
resources: convert_mcp_resources(resources),
resource_templates: convert_mcp_resource_templates(resource_templates),
auth_statuses: auth_statuses_from_entries(&auth_status_entries),
}
}
pub async fn collect_mcp_snapshot_from_manager(
mcp_connection_manager: &McpConnectionManager,
auth_status_entries: HashMap<String, McpAuthStatusEntry>,
@@ -443,104 +665,18 @@ pub async fn collect_mcp_snapshot_from_manager_with_detail(
},
);
let auth_statuses = auth_status_entries
.iter()
.map(|(name, entry)| (name.clone(), entry.auth_status))
.collect::<HashMap<_, _>>();
let tools = tools
.into_iter()
.filter_map(|(name, tool)| match serde_json::to_value(tool.tool) {
Ok(value) => match Tool::from_mcp_value(value) {
Ok(tool) => Some((name, tool)),
Err(err) => {
tracing::warn!("Failed to convert MCP tool '{name}': {err}");
None
}
},
Err(err) => {
tracing::warn!("Failed to serialize MCP tool '{name}': {err}");
None
}
})
.collect::<HashMap<_, _>>();
let resources = resources
.into_iter()
.map(|(name, resources)| {
let resources = resources
.into_iter()
.filter_map(|resource| match serde_json::to_value(resource) {
Ok(value) => match Resource::from_mcp_value(value.clone()) {
Ok(resource) => Some(resource),
Err(err) => {
let (uri, resource_name) = match value {
Value::Object(obj) => (
obj.get("uri")
.and_then(|v| v.as_str().map(ToString::to_string)),
obj.get("name")
.and_then(|v| v.as_str().map(ToString::to_string)),
),
_ => (None, None),
};
tracing::warn!(
"Failed to convert MCP resource (uri={uri:?}, name={resource_name:?}): {err}"
);
None
}
},
Err(err) => {
tracing::warn!("Failed to serialize MCP resource: {err}");
None
}
})
.collect::<Vec<_>>();
(name, resources)
})
.collect::<HashMap<_, _>>();
let resource_templates = resource_templates
.into_iter()
.map(|(name, templates)| {
let templates = templates
.into_iter()
.filter_map(|template| match serde_json::to_value(template) {
Ok(value) => match ResourceTemplate::from_mcp_value(value.clone()) {
Ok(template) => Some(template),
Err(err) => {
let (uri_template, template_name) = match value {
Value::Object(obj) => (
obj.get("uriTemplate")
.or_else(|| obj.get("uri_template"))
.and_then(|v| v.as_str().map(ToString::to_string)),
obj.get("name")
.and_then(|v| v.as_str().map(ToString::to_string)),
),
_ => (None, None),
};
tracing::warn!(
"Failed to convert MCP resource template (uri_template={uri_template:?}, name={template_name:?}): {err}"
);
None
}
},
Err(err) => {
tracing::warn!("Failed to serialize MCP resource template: {err}");
None
}
})
.collect::<Vec<_>>();
(name, templates)
.filter_map(|(name, tool)| {
protocol_tool_from_rmcp_tool(&name, &tool.tool).map(|tool| (name, tool))
})
.collect::<HashMap<_, _>>();
McpListToolsResponseEvent {
tools,
resources,
resource_templates,
auth_statuses,
resources: convert_mcp_resources(resources),
resource_templates: convert_mcp_resource_templates(resource_templates),
auth_statuses: auth_statuses_from_entries(&auth_status_entries),
}
}

105
codex-rs/codex-mcp/src/mcp_connection_manager.rs
View File

@@ -3,8 +3,8 @@
//! The [`McpConnectionManager`] owns one [`codex_rmcp_client::RmcpClient`] per
//! configured server (keyed by the *server name*). It offers convenience
//! helpers to query the available tools across *all* servers and returns them
//! in a single aggregated map using the fully-qualified tool name
//! `"<server><MCP_TOOL_NAME_DELIMITER><tool>"` as the key.
//! in a single aggregated map using the model-visible fully-qualified tool name
//! as the key.
use std::borrow::Cow;
use std::collections::HashMap;
@@ -26,8 +26,8 @@ use crate::mcp::ToolPluginProvenance;
use crate::mcp::configured_mcp_servers;
use crate::mcp::effective_mcp_servers;
use crate::mcp::mcp_permission_prompt_is_auto_approved;
use crate::mcp::sanitize_responses_api_tool_name;
use crate::mcp::tool_plugin_provenance;
pub(crate) use crate::mcp_tool_names::qualify_tools;
use anyhow::Context;
use anyhow::Result;
use anyhow::anyhow;
@@ -92,13 +92,8 @@ use codex_login::CodexAuth;
use codex_utils_plugins::mcp_connector::is_connector_id_allowed;
use codex_utils_plugins::mcp_connector::sanitize_name;
/// Delimiter used to separate the server name from the tool name in a fully
/// qualified tool name.
///
/// OpenAI requires tool names to conform to `^[a-zA-Z0-9_-]+$`, so we must
/// choose a delimiter from this character set.
/// Delimiter used to separate MCP tool-name parts.
const MCP_TOOL_NAME_DELIMITER: &str = "__";
const MAX_TOOL_NAME_LENGTH: usize = 64;
/// Default timeout for initializing MCP server & initially listing tools.
pub const DEFAULT_STARTUP_TIMEOUT: Duration = Duration::from_secs(30);
@@ -106,7 +101,7 @@ pub const DEFAULT_STARTUP_TIMEOUT: Duration = Duration::from_secs(30);
/// Default timeout for individual tool calls.
const DEFAULT_TOOL_TIMEOUT: Duration = Duration::from_secs(120);
const CODEX_APPS_TOOLS_CACHE_SCHEMA_VERSION: u8 = 1;
const CODEX_APPS_TOOLS_CACHE_SCHEMA_VERSION: u8 = 2;
const CODEX_APPS_TOOLS_CACHE_DIR: &str = "cache/codex_apps_tools";
const MCP_TOOLS_LIST_DURATION_METRIC: &str = "codex.mcp.tools.list.duration_ms";
const MCP_TOOLS_FETCH_UNCACHED_DURATION_METRIC: &str = "codex.mcp.tools.fetch_uncached.duration_ms";
@@ -138,59 +133,20 @@ pub fn codex_apps_tools_cache_key(auth: Option<&CodexAuth>) -> CodexAppsToolsCac
}
}
fn qualify_tools<I>(tools: I) -> HashMap<String, ToolInfo>
where
I: IntoIterator<Item = ToolInfo>,
{
let mut used_names = HashSet::new();
let mut seen_raw_names = HashSet::new();
let mut qualified_tools = HashMap::new();
for tool in tools {
let qualified_name_raw = if tool.server_name != CODEX_APPS_MCP_SERVER_NAME {
format!(
"mcp{}{}{}{}",
MCP_TOOL_NAME_DELIMITER, tool.server_name, MCP_TOOL_NAME_DELIMITER, tool.tool_name
)
} else {
format!("{}{}", tool.tool_namespace, tool.tool_name)
};
if !seen_raw_names.insert(qualified_name_raw.clone()) {
warn!("skipping duplicated tool {}", qualified_name_raw);
continue;
}
// Start from a "pretty" name (sanitized), then deterministically disambiguate on
// collisions by appending a hash of the *raw* (unsanitized) qualified name. This
// ensures tools like `foo.bar` and `foo_bar` don't collapse to the same key.
let mut qualified_name = sanitize_responses_api_tool_name(&qualified_name_raw);
// Enforce length constraints early; use the raw name for the hash input so the
// output remains stable even when sanitization changes.
if qualified_name.len() > MAX_TOOL_NAME_LENGTH {
let sha1_str = sha1_hex(&qualified_name_raw);
let prefix_len = MAX_TOOL_NAME_LENGTH - sha1_str.len();
qualified_name = format!("{}{}", &qualified_name[..prefix_len], sha1_str);
}
if used_names.contains(&qualified_name) {
warn!("skipping duplicated tool {}", qualified_name);
continue;
}
used_names.insert(qualified_name.clone());
qualified_tools.insert(qualified_name, tool);
}
qualified_tools
}
#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct ToolInfo {
/// Raw MCP server name used for routing the tool call.
pub server_name: String,
pub tool_name: String,
pub tool_namespace: String,
/// Model-visible tool name used in Responses API tool declarations.
#[serde(rename = "tool_name", alias = "callable_name")]
pub callable_name: String,
/// Model-visible namespace used for deferred tool loading.
#[serde(rename = "tool_namespace", alias = "callable_namespace")]
pub callable_namespace: String,
/// Instructions from the MCP server initialize result.
#[serde(default)]
pub server_instructions: Option<String>,
/// Raw MCP tool definition; `tool.name` is sent back to the MCP server.
pub tool: Tool,
pub connector_id: Option<String>,
pub connector_name: Option<String>,
@@ -951,14 +907,14 @@ impl McpConnectionManager {
/// fully-qualified name for the tool.
#[instrument(level = "trace", skip_all)]
pub async fn list_all_tools(&self) -> HashMap<String, ToolInfo> {
let mut tools = HashMap::new();
let mut tools = Vec::new();
for managed_client in self.clients.values() {
let Some(server_tools) = managed_client.listed_tools().await else {
continue;
};
tools.extend(qualify_tools(server_tools));
tools.extend(server_tools);
}
tools
qualify_tools(tools)
}
/// Force-refresh codex apps tools by bypassing the in-process cache.
@@ -1362,7 +1318,7 @@ fn normalize_codex_apps_tool_title(
value.to_string()
}
fn normalize_codex_apps_tool_name(
fn normalize_codex_apps_callable_name(
server_name: &str,
tool_name: &str,
connector_id: Option<&str>,
@@ -1397,10 +1353,13 @@ fn normalize_codex_apps_tool_name(
tool_name
}
fn normalize_codex_apps_namespace(server_name: &str, connector_name: Option<&str>) -> String {
if server_name != CODEX_APPS_MCP_SERVER_NAME {
server_name.to_string()
} else if let Some(connector_name) = connector_name {
fn normalize_codex_apps_callable_namespace(
server_name: &str,
connector_name: Option<&str>,
) -> String {
if server_name == CODEX_APPS_MCP_SERVER_NAME
&& let Some(connector_name) = connector_name
{
format!(
"mcp{}{}{}{}",
MCP_TOOL_NAME_DELIMITER,
@@ -1409,7 +1368,7 @@ fn normalize_codex_apps_namespace(server_name: &str, connector_name: Option<&str
sanitize_name(connector_name)
)
} else {
server_name.to_string()
format!("mcp{MCP_TOOL_NAME_DELIMITER}{server_name}{MCP_TOOL_NAME_DELIMITER}")
}
}
@@ -1741,14 +1700,16 @@ async fn list_tools_for_client_uncached(
.tools
.into_iter()
.map(|tool| {
let tool_name = normalize_codex_apps_tool_name(
let callable_name = normalize_codex_apps_callable_name(
server_name,
&tool.tool.name,
tool.connector_id.as_deref(),
tool.connector_name.as_deref(),
);
let tool_namespace =
normalize_codex_apps_namespace(server_name, tool.connector_name.as_deref());
let callable_namespace = normalize_codex_apps_callable_namespace(
server_name,
tool.connector_name.as_deref(),
);
let connector_name = tool.connector_name;
let connector_description = tool.connector_description;
let mut tool_def = tool.tool;
@@ -1761,8 +1722,8 @@ async fn list_tools_for_client_uncached(
}
ToolInfo {
server_name: server_name.to_owned(),
tool_name,
tool_namespace,
callable_name,
callable_namespace,
server_instructions: server_instructions.map(str::to_string),
tool: tool_def,
connector_id: tool.connector_id,

128
codex-rs/codex-mcp/src/mcp_connection_manager_tests.rs
View File

@@ -10,14 +10,11 @@ use std::sync::Arc;
use tempfile::tempdir;
fn create_test_tool(server_name: &str, tool_name: &str) -> ToolInfo {
let tool_namespace = format!("mcp__{server_name}__");
ToolInfo {
server_name: server_name.to_string(),
tool_name: tool_name.to_string(),
tool_namespace: if server_name == CODEX_APPS_MCP_SERVER_NAME {
format!("mcp__{server_name}__")
} else {
server_name.to_string()
},
callable_name: tool_name.to_string(),
callable_namespace: tool_namespace,
server_instructions: None,
tool: Tool {
name: tool_name.to_string().into(),
@@ -294,16 +291,12 @@ fn test_qualify_tools_long_names_same_server() {
let mut keys: Vec<_> = qualified_tools.keys().cloned().collect();
keys.sort();
assert_eq!(keys[0].len(), 64);
assert_eq!(
keys[0],
"mcp__my_server__extremel119a2b97664e41363932dc84de21e2ff1b93b3e9"
);
assert_eq!(keys[1].len(), 64);
assert_eq!(
keys[1],
"mcp__my_server__yet_anot419a82a89325c1b477274a41f8c65ea5f3a7f341"
assert!(keys.iter().all(|key| key.len() == 64));
assert!(keys.iter().all(|key| key.starts_with("mcp__my_server__")));
assert!(
keys.iter()
.all(|key| key.chars().all(|c| c.is_ascii_alphanumeric() || c == '_')),
"qualified names must be code-mode compatible: {keys:?}"
);
}
@@ -316,19 +309,96 @@ fn test_qualify_tools_sanitizes_invalid_characters() {
assert_eq!(qualified_tools.len(), 1);
let (qualified_name, tool) = qualified_tools.into_iter().next().expect("one tool");
assert_eq!(qualified_name, "mcp__server_one__tool_two_three");
assert_eq!(
format!("{}{}", tool.callable_namespace, tool.callable_name),
qualified_name
);
// The key is sanitized for OpenAI, but we keep original parts for the actual MCP call.
// The key and callable parts are sanitized for model-visible tool calls, but
// the raw MCP name is preserved for the actual MCP call.
assert_eq!(tool.server_name, "server.one");
assert_eq!(tool.tool_name, "tool.two-three");
assert_eq!(tool.callable_namespace, "mcp__server_one__");
assert_eq!(tool.callable_name, "tool_two_three");
assert_eq!(tool.tool.name, "tool.two-three");
assert!(
qualified_name
.chars()
.all(|c| c.is_ascii_alphanumeric() || c == '_' || c == '-'),
"qualified name must be Responses API compatible: {qualified_name:?}"
.all(|c| c.is_ascii_alphanumeric() || c == '_'),
"qualified name must be code-mode compatible: {qualified_name:?}"
);
}
#[test]
fn test_qualify_tools_keeps_hyphenated_mcp_tools_callable() {
let tools = vec![create_test_tool("music-studio", "get-strudel-guide")];
let qualified_tools = qualify_tools(tools);
assert_eq!(qualified_tools.len(), 1);
let (qualified_name, tool) = qualified_tools.into_iter().next().expect("one tool");
assert_eq!(qualified_name, "mcp__music_studio__get_strudel_guide");
assert_eq!(tool.callable_namespace, "mcp__music_studio__");
assert_eq!(tool.callable_name, "get_strudel_guide");
assert_eq!(tool.tool.name, "get-strudel-guide");
}
#[test]
fn test_qualify_tools_disambiguates_sanitized_namespace_collisions() {
let tools = vec![
create_test_tool("basic-server", "lookup"),
create_test_tool("basic_server", "query"),
];
let qualified_tools = qualify_tools(tools);
assert_eq!(qualified_tools.len(), 2);
let mut namespaces = qualified_tools
.values()
.map(|tool| tool.callable_namespace.as_str())
.collect::<Vec<_>>();
namespaces.sort();
namespaces.dedup();
assert_eq!(namespaces.len(), 2);
let raw_servers = qualified_tools
.values()
.map(|tool| tool.server_name.as_str())
.collect::<HashSet<_>>();
assert_eq!(raw_servers, HashSet::from(["basic-server", "basic_server"]));
assert!(
qualified_tools
.keys()
.all(|key| key.chars().all(|c| c.is_ascii_alphanumeric() || c == '_')),
"qualified names must be code-mode compatible: {qualified_tools:?}"
);
}
#[test]
fn test_qualify_tools_disambiguates_sanitized_tool_name_collisions() {
let tools = vec![
create_test_tool("server", "tool-name"),
create_test_tool("server", "tool_name"),
];
let qualified_tools = qualify_tools(tools);
assert_eq!(qualified_tools.len(), 2);
let raw_tool_names = qualified_tools
.values()
.map(|tool| tool.tool.name.to_string())
.collect::<HashSet<_>>();
assert_eq!(
raw_tool_names,
HashSet::from(["tool-name".to_string(), "tool_name".to_string()])
);
let callable_tool_names = qualified_tools
.values()
.map(|tool| tool.callable_name.as_str())
.collect::<HashSet<_>>();
assert_eq!(callable_tool_names.len(), 2);
}
#[test]
fn tool_filter_allows_by_default() {
let filter = ToolFilter::default();
@@ -393,7 +463,7 @@ fn filter_tools_applies_per_server_filters() {
assert_eq!(filtered.len(), 1);
assert_eq!(filtered[0].server_name, "server1");
assert_eq!(filtered[0].tool_name, "tool_a");
assert_eq!(filtered[0].callable_name, "tool_a");
}
#[test]
@@ -410,12 +480,12 @@ fn codex_apps_tools_cache_is_overwritten_by_last_write() {
write_cached_codex_apps_tools(&cache_context, &tools_gateway_1);
let cached_gateway_1 =
read_cached_codex_apps_tools(&cache_context).expect("cache entry exists for first write");
assert_eq!(cached_gateway_1[0].tool_name, "one");
assert_eq!(cached_gateway_1[0].callable_name, "one");
write_cached_codex_apps_tools(&cache_context, &tools_gateway_2);
let cached_gateway_2 =
read_cached_codex_apps_tools(&cache_context).expect("cache entry exists for second write");
assert_eq!(cached_gateway_2[0].tool_name, "two");
assert_eq!(cached_gateway_2[0].callable_name, "two");
}
#[test]
@@ -442,8 +512,8 @@ fn codex_apps_tools_cache_is_scoped_per_user() {
let read_user_2 =
read_cached_codex_apps_tools(&cache_context_user_2).expect("cache entry for user two");
assert_eq!(read_user_1[0].tool_name, "one");
assert_eq!(read_user_2[0].tool_name, "two");
assert_eq!(read_user_1[0].callable_name, "one");
assert_eq!(read_user_2[0].callable_name, "two");
assert_ne!(
cache_context_user_1.cache_path(),
cache_context_user_2.cache_path(),
@@ -478,7 +548,7 @@ fn codex_apps_tools_cache_filters_disallowed_connectors() {
let cached = read_cached_codex_apps_tools(&cache_context).expect("cache entry exists for user");
assert_eq!(cached.len(), 1);
assert_eq!(cached[0].tool_name, "allowed_tool");
assert_eq!(cached[0].callable_name, "allowed_tool");
assert_eq!(cached[0].connector_id.as_deref(), Some("calendar"));
}
@@ -543,7 +613,7 @@ fn startup_cached_codex_apps_tools_loads_from_disk_cache() {
assert_eq!(startup_tools.len(), 1);
assert_eq!(startup_tools[0].server_name, CODEX_APPS_MCP_SERVER_NAME);
assert_eq!(startup_tools[0].tool_name, "calendar_search");
assert_eq!(startup_tools[0].callable_name, "calendar_search");
}
#[tokio::test]
@@ -573,7 +643,7 @@ async fn list_all_tools_uses_startup_snapshot_while_client_is_pending() {
.get("mcp__codex_apps__calendar_create_event")
.expect("tool from startup cache");
assert_eq!(tool.server_name, CODEX_APPS_MCP_SERVER_NAME);
assert_eq!(tool.tool_name, "calendar_create_event");
assert_eq!(tool.callable_name, "calendar_create_event");
}
#[tokio::test]
@@ -655,7 +725,7 @@ async fn list_all_tools_uses_startup_snapshot_when_client_startup_fails() {
.get("mcp__codex_apps__calendar_create_event")
.expect("tool from startup cache");
assert_eq!(tool.server_name, CODEX_APPS_MCP_SERVER_NAME);
assert_eq!(tool.tool_name, "calendar_create_event");
assert_eq!(tool.callable_name, "calendar_create_event");
}
#[test]

202
codex-rs/codex-mcp/src/mcp_tool_names.rs Normal file
View File

@@ -0,0 +1,202 @@
//! Allocates model-visible MCP tool names while preserving raw MCP identities.
use std::collections::HashMap;
use std::collections::HashSet;
use sha1::Digest;
use sha1::Sha1;
use tracing::warn;
use crate::mcp::sanitize_responses_api_tool_name;
use crate::mcp_connection_manager::ToolInfo;
const MCP_TOOL_NAME_DELIMITER: &str = "__";
const MAX_TOOL_NAME_LENGTH: usize = 64;
const CALLABLE_NAME_HASH_LEN: usize = 12;
fn sha1_hex(s: &str) -> String {
let mut hasher = Sha1::new();
hasher.update(s.as_bytes());
let sha1 = hasher.finalize();
format!("{sha1:x}")
}
fn callable_name_hash_suffix(raw_identity: &str) -> String {
let hash = sha1_hex(raw_identity);
format!("_{}", &hash[..CALLABLE_NAME_HASH_LEN])
}
fn append_hash_suffix(value: &str, raw_identity: &str) -> String {
format!("{value}{}", callable_name_hash_suffix(raw_identity))
}
fn append_namespace_hash_suffix(namespace: &str, raw_identity: &str) -> String {
if let Some(namespace) = namespace.strip_suffix(MCP_TOOL_NAME_DELIMITER) {
format!(
"{}{}{}",
namespace,
callable_name_hash_suffix(raw_identity),
MCP_TOOL_NAME_DELIMITER
)
} else {
append_hash_suffix(namespace, raw_identity)
}
}
fn truncate_name(value: &str, max_len: usize) -> String {
value.chars().take(max_len).collect()
}
fn fit_callable_parts_with_hash(
namespace: &str,
tool_name: &str,
raw_identity: &str,
) -> (String, String) {
let suffix = callable_name_hash_suffix(raw_identity);
let max_tool_len = MAX_TOOL_NAME_LENGTH.saturating_sub(namespace.len());
if max_tool_len >= suffix.len() {
let prefix_len = max_tool_len - suffix.len();
return (
namespace.to_string(),
format!("{}{}", truncate_name(tool_name, prefix_len), suffix),
);
}
let max_namespace_len = MAX_TOOL_NAME_LENGTH - suffix.len();
(truncate_name(namespace, max_namespace_len), suffix)
}
fn unique_callable_parts(
namespace: &str,
tool_name: &str,
raw_identity: &str,
used_names: &mut HashSet<String>,
) -> (String, String, String) {
let qualified_name = format!("{namespace}{tool_name}");
if qualified_name.len() <= MAX_TOOL_NAME_LENGTH && used_names.insert(qualified_name.clone()) {
return (namespace.to_string(), tool_name.to_string(), qualified_name);
}
let mut attempt = 0_u32;
loop {
let hash_input = if attempt == 0 {
raw_identity.to_string()
} else {
format!("{raw_identity}\0{attempt}")
};
let (namespace, tool_name) =
fit_callable_parts_with_hash(namespace, tool_name, &hash_input);
let qualified_name = format!("{namespace}{tool_name}");
if used_names.insert(qualified_name.clone()) {
return (namespace, tool_name, qualified_name);
}
attempt = attempt.saturating_add(1);
}
}
#[derive(Debug)]
struct CallableToolCandidate {
tool: ToolInfo,
raw_namespace_identity: String,
raw_tool_identity: String,
callable_namespace: String,
callable_name: String,
}
/// Returns a qualified-name lookup for MCP tools.
///
/// Raw MCP server/tool names are kept on each [`ToolInfo`] for protocol calls, while
/// `callable_namespace` / `callable_name` are sanitized and, when necessary, hashed so
/// every model-visible `mcp__namespace__tool` name is unique and <= 64 bytes.
pub(crate) fn qualify_tools<I>(tools: I) -> HashMap<String, ToolInfo>
where
I: IntoIterator<Item = ToolInfo>,
{
let mut seen_raw_names = HashSet::new();
let mut candidates = Vec::new();
for tool in tools {
let raw_namespace_identity = format!(
"{}\0{}\0{}",
tool.server_name,
tool.callable_namespace,
tool.connector_id.as_deref().unwrap_or_default()
);
let raw_tool_identity = format!(
"{}\0{}\0{}",
raw_namespace_identity, tool.callable_name, tool.tool.name
);
if !seen_raw_names.insert(raw_tool_identity.clone()) {
warn!("skipping duplicated tool {}", tool.tool.name);
continue;
}
candidates.push(CallableToolCandidate {
callable_namespace: sanitize_responses_api_tool_name(&tool.callable_namespace),
callable_name: sanitize_responses_api_tool_name(&tool.callable_name),
raw_namespace_identity,
raw_tool_identity,
tool,
});
}
let mut namespace_identities_by_base = HashMap::<String, HashSet<String>>::new();
for candidate in &candidates {
namespace_identities_by_base
.entry(candidate.callable_namespace.clone())
.or_default()
.insert(candidate.raw_namespace_identity.clone());
}
let colliding_namespaces = namespace_identities_by_base
.into_iter()
.filter_map(|(namespace, identities)| (identities.len() > 1).then_some(namespace))
.collect::<HashSet<_>>();
for candidate in &mut candidates {
if colliding_namespaces.contains(&candidate.callable_namespace) {
candidate.callable_namespace = append_namespace_hash_suffix(
&candidate.callable_namespace,
&candidate.raw_namespace_identity,
);
}
}
let mut tool_identities_by_base = HashMap::<(String, String), HashSet<String>>::new();
for candidate in &candidates {
tool_identities_by_base
.entry((
candidate.callable_namespace.clone(),
candidate.callable_name.clone(),
))
.or_default()
.insert(candidate.raw_tool_identity.clone());
}
let colliding_tools = tool_identities_by_base
.into_iter()
.filter_map(|(key, identities)| (identities.len() > 1).then_some(key))
.collect::<HashSet<_>>();
for candidate in &mut candidates {
if colliding_tools.contains(&(
candidate.callable_namespace.clone(),
candidate.callable_name.clone(),
)) {
candidate.callable_name =
append_hash_suffix(&candidate.callable_name, &candidate.raw_tool_identity);
}
}
candidates.sort_by(|left, right| left.raw_tool_identity.cmp(&right.raw_tool_identity));
let mut used_names = HashSet::new();
let mut qualified_tools = HashMap::new();
for mut candidate in candidates {
let (callable_namespace, callable_name, qualified_name) = unique_callable_parts(
&candidate.callable_namespace,
&candidate.callable_name,
&candidate.raw_tool_identity,
&mut used_names,
);
candidate.tool.callable_namespace = callable_namespace;
candidate.tool.callable_name = callable_name;
qualified_tools.insert(qualified_name, candidate.tool);
}
qualified_tools
}