clone/codex
1
0
Fork 0
mirror of https://github.com/openai/codex.git synced 2026-06-01 19:02:59 +00:00
Code Issues Packages Projects Releases Wiki Activity

[tool_suggest] Improve tool_suggest triggering conditions. (#20091)

Browse Source 
## Summary
- Tighten `tool_suggest` guidance so it prefers explicit plugin install
requests, while still allowing a connector install when the relevant
plugin is already installed and a needed connector from that plugin is
missing.
- Tell the model not to call `tool_suggest` in parallel with other
tools.

## Testing
- `cargo test -p codex-tools tool_suggest`
- `cargo test -p codex-core tool_suggest`
This commit is contained in:
Matthew Zeng
2026-04-29 13:41:12 -07:00
committed by GitHub
parent 0690ab0842
commit 8ce48f9968
5 changed files with 83 additions and 48 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
codex-rs/tools/src/tool_discovery.rs
View File

@@ -272,11 +272,6 @@ pub fn collect_tool_search_source_infos<'a>(
}
pub fn create_tool_suggest_tool(discoverable_tools: &[ToolSuggestEntry]) -> ToolSpec {
let discoverable_tool_ids = discoverable_tools
.iter()
.map(|tool| tool.id.as_str())
.collect::<Vec<_>>()
.join(", ");
let properties = BTreeMap::from([
(
"tool_type".to_string(),
@@ -287,15 +282,11 @@ pub fn create_tool_suggest_tool(discoverable_tools: &[ToolSuggestEntry]) -> Tool
),
(
"action_type".to_string(),
JsonSchema::string(Some(
"Suggested action for the tool. Use \"install\" or \"enable\".".to_string(),
)),
JsonSchema::string(Some("Suggested action for the tool. Use \"install\".".to_string())),
),
(
"tool_id".to_string(),
JsonSchema::string(Some(format!(
"Connector or plugin id to suggest. Must be one of: {discoverable_tool_ids}."
))),
JsonSchema::string(Some("Connector or plugin id to suggest.".to_string())),
),
(
"suggest_reason".to_string(),
@@ -308,7 +299,7 @@ pub fn create_tool_suggest_tool(discoverable_tools: &[ToolSuggestEntry]) -> Tool
let discoverable_tools = format_discoverable_tools(discoverable_tools);
let description = format!(
"# Tool suggestion discovery\n\nSuggests a missing connector in an installed plugin, or in narrower cases a not installed but discoverable plugin, when the user clearly wants a capability that is not currently available in the active `tools` list.\n\nUse this ONLY when:\n- You've already tried to find a matching available tool for the user's request but couldn't find a good match. This includes `{TOOL_SEARCH_TOOL_NAME}` (if available) and other means.\n- For connectors/apps that are not installed but needed for an installed plugin, suggest to install them if the task requirements match precisely.\n- For plugins that are not installed but discoverable, only suggest discoverable and installable plugins when the user's intent very explicitly and unambiguously matches that plugin itself. Do not suggest a plugin just because one of its connectors or capabilities seems relevant.\n\nTool suggestions should only use the discoverable tools listed here. DO NOT explore or recommend tools that are not on this list.\n\nDiscoverable tools:\n{discoverable_tools}\n\nWorkflow:\n\n1. Ensure all possible means have been exhausted to find an existing available tool but none of them matches the request intent.\n2. Match the user's request against the discoverable tools list above. Apply the stricter explicit-and-unambiguous rule for *discoverable tools* like plugin install suggestions; *missing tools* like connector install suggestions continue to use the normal clear-fit standard.\n3. If one tool clearly fits, call `{TOOL_SUGGEST_TOOL_NAME}` with:\n - `tool_type`: `connector` or `plugin`\n - `action_type`: `install` or `enable`\n - `tool_id`: exact id from the discoverable tools list above\n - `suggest_reason`: concise one-line user-facing reason this tool can help with the current request\n4. After the suggestion flow completes:\n - if the user finished the install or enable flow, continue by searching again or using the newly available tool\n - if the user did not finish, continue without that tool, and don't suggest that tool again unless the user explicitly asks for it."
"# Tool suggestion discovery\n\nUse this tool only to ask the user to install one known plugin or connector from the list below. The list contains known candidates that are not currently installed.\n\nUse this ONLY when all of the following are true:\n- The user explicitly wants a specific plugin or connector that is not already available in the current context or active `tools` list.\n- `{TOOL_SEARCH_TOOL_NAME}` is not available, or it has already been called and did not find or make the requested tool callable.\n- The tool is one of the known installable plugins or connectors listed below. Only ask to install tools from this list.\n\nDo not use tool suggestion for adjacent capabilities, broad recommendations, or tools that merely seem useful. The user's intent must clearly match one listed tool.\n\nKnown plugins/connectors available to install:\n{discoverable_tools}\n\nWorkflow:\n\n1. Check the current context and active `tools` list first. If `{TOOL_SEARCH_TOOL_NAME}` is available, call `{TOOL_SEARCH_TOOL_NAME}` before calling `{TOOL_SUGGEST_TOOL_NAME}`. Do not use tool suggestion if the needed tool is already available, found through `{TOOL_SEARCH_TOOL_NAME}`, or callable after discovery.\n2. Match the user's explicit request against the known plugin/connector list above. Only proceed when one listed plugin or connector exactly fits.\n3. If we found both connectors and plugins to suggest, use plugins first, only use connectors if the corresponding plugin is installed but the connector is not.\n4. If one tool clearly fits, call `{TOOL_SUGGEST_TOOL_NAME}` with:\n - `tool_type`: `connector` or `plugin`\n - `action_type`: `install`\n - `tool_id`: exact id from the known plugin/connector list above\n - `suggest_reason`: concise one-line user-facing reason this tool can help with the current request\n5. After the suggestion flow completes:\n - if the user finished the install flow, continue by searching again or using the newly available tool\n - if the user did not finish, continue without that tool, and don't suggest that tool again unless the user explicitly asks for it.\n\nIMPORTANT: DO NOT call this tool in parallel with other tools."
);
ToolSpec::Function(ResponsesApiTool {

32
codex-rs/tools/src/tool_discovery_tests.rs
View File

@@ -50,6 +50,32 @@ fn create_tool_search_tool_deduplicates_and_renders_enabled_sources() {
#[test]
fn create_tool_suggest_tool_uses_plugin_summary_fallback() {
let expected_description = concat!(
"# Tool suggestion discovery\n\n",
"Use this tool only to ask the user to install one known plugin or connector from the list below. The list contains known candidates that are not currently installed.\n\n",
"Use this ONLY when all of the following are true:\n",
"- The user explicitly wants a specific plugin or connector that is not already available in the current context or active `tools` list.\n",
"- `tool_search` is not available, or it has already been called and did not find or make the requested tool callable.\n",
"- The tool is one of the known installable plugins or connectors listed below. Only ask to install tools from this list.\n\n",
"Do not use tool suggestion for adjacent capabilities, broad recommendations, or tools that merely seem useful. The user's intent must clearly match one listed tool.\n\n",
"Known plugins/connectors available to install:\n",
"- GitHub (id: `github`, type: plugin, action: install): skills; MCP servers: github-mcp; app connectors: github-app\n",
"- Slack (id: `slack@openai-curated`, type: connector, action: install): No description provided.\n\n",
"Workflow:\n\n",
"1. Check the current context and active `tools` list first. If `tool_search` is available, call `tool_search` before calling `tool_suggest`. Do not use tool suggestion if the needed tool is already available, found through `tool_search`, or callable after discovery.\n",
"2. Match the user's explicit request against the known plugin/connector list above. Only proceed when one listed plugin or connector exactly fits.\n",
"3. If we found both connectors and plugins to suggest, use plugins first, only use connectors if the corresponding plugin is installed but the connector is not.\n",
"4. If one tool clearly fits, call `tool_suggest` with:\n",
" - `tool_type`: `connector` or `plugin`\n",
" - `action_type`: `install`\n",
" - `tool_id`: exact id from the known plugin/connector list above\n",
" - `suggest_reason`: concise one-line user-facing reason this tool can help with the current request\n",
"5. After the suggestion flow completes:\n",
" - if the user finished the install flow, continue by searching again or using the newly available tool\n",
" - if the user did not finish, continue without that tool, and don't suggest that tool again unless the user explicitly asks for it.\n\n",
"IMPORTANT: DO NOT call this tool in parallel with other tools.",
);
assert_eq!(
create_tool_suggest_tool(&[
ToolSuggestEntry {
@@ -73,14 +99,14 @@ fn create_tool_suggest_tool_uses_plugin_summary_fallback() {
]),
ToolSpec::Function(ResponsesApiTool {
name: "tool_suggest".to_string(),
description: "# Tool suggestion discovery\n\nSuggests a missing connector in an installed plugin, or in narrower cases a not installed but discoverable plugin, when the user clearly wants a capability that is not currently available in the active `tools` list.\n\nUse this ONLY when:\n- You've already tried to find a matching available tool for the user's request but couldn't find a good match. This includes `tool_search` (if available) and other means.\n- For connectors/apps that are not installed but needed for an installed plugin, suggest to install them if the task requirements match precisely.\n- For plugins that are not installed but discoverable, only suggest discoverable and installable plugins when the user's intent very explicitly and unambiguously matches that plugin itself. Do not suggest a plugin just because one of its connectors or capabilities seems relevant.\n\nTool suggestions should only use the discoverable tools listed here. DO NOT explore or recommend tools that are not on this list.\n\nDiscoverable tools:\n- GitHub (id: `github`, type: plugin, action: install): skills; MCP servers: github-mcp; app connectors: github-app\n- Slack (id: `slack@openai-curated`, type: connector, action: install): No description provided.\n\nWorkflow:\n\n1. Ensure all possible means have been exhausted to find an existing available tool but none of them matches the request intent.\n2. Match the user's request against the discoverable tools list above. Apply the stricter explicit-and-unambiguous rule for *discoverable tools* like plugin install suggestions; *missing tools* like connector install suggestions continue to use the normal clear-fit standard.\n3. If one tool clearly fits, call `tool_suggest` with:\n - `tool_type`: `connector` or `plugin`\n - `action_type`: `install` or `enable`\n - `tool_id`: exact id from the discoverable tools list above\n - `suggest_reason`: concise one-line user-facing reason this tool can help with the current request\n4. After the suggestion flow completes:\n - if the user finished the install or enable flow, continue by searching again or using the newly available tool\n - if the user did not finish, continue without that tool, and don't suggest that tool again unless the user explicitly asks for it.".to_string(),
description: expected_description.to_string(),
strict: false,
defer_loading: None,
parameters: JsonSchema::object(BTreeMap::from([
(
"action_type".to_string(),
JsonSchema::string(Some(
"Suggested action for the tool. Use \"install\" or \"enable\"."
"Suggested action for the tool. Use \"install\"."
.to_string(),
),),
),
@@ -94,7 +120,7 @@ fn create_tool_suggest_tool_uses_plugin_summary_fallback() {
(
"tool_id".to_string(),
JsonSchema::string(Some(
"Connector or plugin id to suggest. Must be one of: slack@openai-curated, github."
"Connector or plugin id to suggest."
.to_string(),
),),
),

38
codex-rs/tools/src/tool_registry_plan_tests.rs
View File

@@ -1721,17 +1721,18 @@ fn tool_suggest_can_be_registered_without_search_tool() {
);
assert_contains_tool_names(&tools, &[TOOL_SUGGEST_TOOL_NAME]);
let tool_suggest = find_tool(&tools, TOOL_SUGGEST_TOOL_NAME);
assert!(tool_suggest.supports_parallel_tool_calls);
assert_lacks_tool_name(&tools, TOOL_SEARCH_TOOL_NAME);
let tool_suggest = find_tool(&tools, TOOL_SUGGEST_TOOL_NAME);
let ToolSpec::Function(ResponsesApiTool { description, .. }) = &tool_suggest.spec else {
panic!("expected function tool");
};
assert!(description.contains(
"Suggests a missing connector in an installed plugin, or in narrower cases a not installed but discoverable plugin"
"Use this tool only to ask the user to install one known plugin or connector from the list below. The list contains known candidates that are not currently installed."
));
assert!(description.contains(
"You've already tried to find a matching available tool for the user's request but couldn't find a good match. This includes `tool_search` (if available) and other means."
"`tool_search` is not available, or it has already been called and did not find or make the requested tool callable."
));
}
@@ -1776,13 +1777,17 @@ fn tool_suggest_description_lists_discoverable_tools() {
})),
];
let (tools, _) = build_specs_with_discoverable_tools(
let (tools, handlers) = build_specs_with_discoverable_tools(
&tools_config,
/*mcp_tools*/ None,
/*deferred_mcp_tools*/ None,
Some(discoverable_tools),
&[],
);
assert!(handlers.contains(&ToolHandlerSpec {
name: ToolName::plain(TOOL_SUGGEST_TOOL_NAME),
kind: ToolHandlerKind::ToolSuggest,
}));
let tool_suggest = find_tool(&tools, TOOL_SUGGEST_TOOL_NAME);
let ToolSpec::Function(ResponsesApiTool {
@@ -1794,7 +1799,7 @@ fn tool_suggest_description_lists_discoverable_tools() {
panic!("expected function tool");
};
assert!(description.contains(
"Suggests a missing connector in an installed plugin, or in narrower cases a not installed but discoverable plugin"
"Use this tool only to ask the user to install one known plugin or connector from the list below. The list contains known candidates that are not currently installed."
));
assert!(description.contains("Google Calendar"));
assert!(description.contains("Gmail"));
@@ -1802,28 +1807,37 @@ fn tool_suggest_description_lists_discoverable_tools() {
assert!(description.contains("Plan events and schedules."));
assert!(description.contains("Find and summarize email threads."));
assert!(description.contains("id: `sample@test`, type: plugin, action: install"));
assert!(description.contains("`action_type`: `install` or `enable`"));
assert!(description.contains("`action_type`: `install`"));
assert!(
description.contains("skills; MCP servers: sample-docs; app connectors: connector_sample")
);
assert!(
description.contains(
"You've already tried to find a matching available tool for the user's request but couldn't find a good match. This includes `tool_search` (if available) and other means."
"The user explicitly wants a specific plugin or connector that is not already available in the current context or active `tools` list."
)
);
assert!(description.contains(
"For connectors/apps that are not installed but needed for an installed plugin, suggest to install them if the task requirements match precisely."
"`tool_search` is not available, or it has already been called and did not find or make the requested tool callable."
));
assert!(description.contains(
"For plugins that are not installed but discoverable, only suggest discoverable and installable plugins when the user's intent very explicitly and unambiguously matches that plugin itself."
"The tool is one of the known installable plugins or connectors listed below. Only ask to install tools from this list."
));
assert!(description.contains(
"Do not suggest a plugin just because one of its connectors or capabilities seems relevant."
"Do not use tool suggestion for adjacent capabilities, broad recommendations, or tools that merely seem useful."
));
assert!(description.contains("IMPORTANT: DO NOT call this tool in parallel with other tools."));
assert!(description.contains(
"Do not use tool suggestion if the needed tool is already available, found through `tool_search`, or callable after discovery."
));
assert!(description.contains(
"Apply the stricter explicit-and-unambiguous rule for *discoverable tools* like plugin install suggestions; *missing tools* like connector install suggestions continue to use the normal clear-fit standard."
"If `tool_search` is available, call `tool_search` before calling `tool_suggest`."
));
assert!(!description.contains("targeted lookup"));
assert!(!description.contains("broad or speculative searches"));
assert!(description.contains("Only proceed when one listed plugin or connector exactly fits."));
assert!(description.contains(
"If we found both connectors and plugins to suggest, use plugins first, only use connectors if the corresponding plugin is installed but the connector is not."
));
assert!(description.contains("DO NOT explore or recommend tools that are not on this list."));
assert!(!description.contains("{{discoverable_tools}}"));
assert!(!description.contains("tool_search fails to find a good match"));
let (_, required) = expect_object_schema(parameters);