mirror of
https://github.com/openai/codex.git
synced 2026-05-16 09:12:54 +00:00
## Why Codex intentionally ignores unknown `config.toml` fields by default so older and newer config files keep working across versions. That leniency also makes typo detection hard because misspelled or misplaced keys disappear silently. This change adds an opt-in strict config mode so users and tooling can fail fast on unrecognized config fields without changing the default permissive behavior. This feature is possible because `serde_ignored` exposes the exact signal Codex needs: it lets Codex run ordinary Serde deserialization while recording fields Serde would otherwise ignore. That avoids requiring `#[serde(deny_unknown_fields)]` across every config type and keeps strict validation opt-in around the existing config model. ## What Changed ### Added strict config validation - Added `serde_ignored`-based validation for `ConfigToml` in `codex-rs/config/src/strict_config.rs`. - Combined `serde_ignored` with `serde_path_to_error` so strict mode preserves typed config error paths while also collecting fields Serde would otherwise ignore. - Added strict-mode validation for unknown `[features]` keys, including keys that would otherwise be accepted by `FeaturesToml`'s flattened boolean map. - Kept typed config errors ahead of ignored-field reporting, so malformed known fields are reported before unknown-field diagnostics. - Added source-range diagnostics for top-level and nested unknown config fields, including non-file managed preference source names. ### Kept parsing single-pass per source - Reworked file and managed-config loading so strict validation reuses the already parsed `TomlValue` for that source. - For actual config files and managed config strings, the loader now reads once, parses once, and validates that same parsed value instead of deserializing multiple times. - Validated `-c` / `--config` override layers with the same base-directory context used for normal relative-path resolution, so unknown override keys are still reported when another override contains a relative path. ### Scoped `--strict-config` to config-heavy entry points - Added support for `--strict-config` on the main config-loading entry points where it is most useful: - `codex` - `codex resume` - `codex fork` - `codex exec` - `codex review` - `codex mcp-server` - `codex app-server` when running the server itself - the standalone `codex-app-server` binary - the standalone `codex-exec` binary - Commands outside that set now reject `--strict-config` early with targeted errors instead of accepting it everywhere through shared CLI plumbing. - `codex app-server` subcommands such as `proxy`, `daemon`, and `generate-*` are intentionally excluded from the first rollout. - When app-server strict mode sees invalid config, app-server exits with the config error instead of logging a warning and continuing with defaults. - Introduced a dedicated `ReviewCommand` wrapper in `codex-rs/cli` instead of extending shared `ReviewArgs`, so `--strict-config` stays on the outer config-loading command surface and does not become part of the reusable review payload used by `codex exec review`. ### Coverage - Added tests for top-level and nested unknown config fields, unknown `[features]` keys, typed-error precedence, source-location reporting, and non-file managed preference source names. - Added CLI coverage showing invalid `--enable`, invalid `--disable`, and unknown `-c` overrides still error when `--strict-config` is present, including compound-looking feature names such as `multi_agent_v2.subagent_usage_hint_text`. - Added integration coverage showing both `codex app-server --strict-config` and standalone `codex-app-server --strict-config` exit with an error for unknown config fields instead of starting with fallback defaults. - Added coverage showing unsupported command surfaces reject `--strict-config` with explicit errors. ## Example Usage Run Codex with strict config validation enabled: ```shell codex --strict-config ``` Strict config mode is also available on the supported config-heavy subcommands: ```shell codex --strict-config exec "explain this repository" codex review --strict-config --uncommitted codex mcp-server --strict-config codex app-server --strict-config --listen off codex-app-server --strict-config --listen off ``` For example, if `~/.codex/config.toml` contains a typo in a key name: ```toml model = "gpt-5" approval_polic = "on-request" ``` then `codex --strict-config` reports the misspelled key instead of silently ignoring it. The path is shortened to `~` here for readability: ```text $ codex --strict-config Error loading config.toml: ~/.codex/config.toml:2:1: unknown configuration field `approval_polic` | 2 | approval_polic = "on-request" | ^^^^^^^^^^^^^^ ``` Without `--strict-config`, Codex keeps the existing permissive behavior and ignores the unknown key. Strict config mode also validates ad-hoc `-c` / `--config` overrides: ```text $ codex --strict-config -c foo=bar Error: unknown configuration field `foo` in -c/--config override $ codex --strict-config -c features.foo=true Error: unknown configuration field `features.foo` in -c/--config override ``` Invalid feature toggles are rejected too, including values that look like nested config paths: ```text $ codex --strict-config --enable does_not_exist Error: Unknown feature flag: does_not_exist $ codex --strict-config --disable does_not_exist Error: Unknown feature flag: does_not_exist $ codex --strict-config --enable multi_agent_v2.subagent_usage_hint_text Error: Unknown feature flag: multi_agent_v2.subagent_usage_hint_text ``` Unsupported commands reject the flag explicitly: ```text $ codex --strict-config cloud list Error: `--strict-config` is not supported for `codex cloud` ``` ## Verification The `codex-cli` `strict_config` tests cover invalid `--enable`, invalid `--disable`, the compound `multi_agent_v2.subagent_usage_hint_text` case, unknown `-c` overrides, app-server strict startup failure through `codex app-server`, and rejection for unsupported commands such as `codex cloud`, `codex mcp`, `codex remote-control`, and `codex app-server proxy`. The config and config-loader tests cover unknown top-level fields, unknown nested fields, unknown `[features]` keys, source-location reporting, non-file managed config sources, and `-c` validation for keys such as `features.foo`. The app-server test suite covers standalone `codex-app-server --strict-config` startup failure for an unknown config field. ## Documentation The Codex CLI docs on developers.openai.com/codex should mention `--strict-config` as an opt-in validation mode for supported config-heavy entry points once this ships.
478 lines
14 KiB
Rust
478 lines
14 KiB
Rust
//! Helpers for mapping config parse/validation failures to file locations and
|
|
//! rendering them in a user-friendly way.
|
|
|
|
use crate::ConfigLayerEntry;
|
|
use crate::ConfigLayerStack;
|
|
use crate::ConfigLayerStackOrdering;
|
|
use codex_app_server_protocol::ConfigLayerSource;
|
|
use codex_utils_absolute_path::AbsolutePathBufGuard;
|
|
use serde::de::DeserializeOwned;
|
|
use serde_path_to_error::Path as SerdePath;
|
|
use serde_path_to_error::Segment as SerdeSegment;
|
|
use std::fmt;
|
|
use std::fmt::Write;
|
|
use std::io;
|
|
use std::path::Path;
|
|
use std::path::PathBuf;
|
|
use toml_edit::Document;
|
|
use toml_edit::Item;
|
|
use toml_edit::Table;
|
|
use toml_edit::Value;
|
|
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
pub struct TextPosition {
|
|
pub line: usize,
|
|
pub column: usize,
|
|
}
|
|
|
|
/// Text range in 1-based line/column coordinates.
|
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
|
pub struct TextRange {
|
|
pub start: TextPosition,
|
|
pub end: TextPosition,
|
|
}
|
|
|
|
#[derive(Debug, Clone, PartialEq, Eq)]
|
|
pub struct ConfigError {
|
|
pub path: PathBuf,
|
|
pub range: TextRange,
|
|
pub message: String,
|
|
}
|
|
|
|
impl ConfigError {
|
|
pub fn new(path: PathBuf, range: TextRange, message: impl Into<String>) -> Self {
|
|
Self {
|
|
path,
|
|
range,
|
|
message: message.into(),
|
|
}
|
|
}
|
|
}
|
|
|
|
#[derive(Debug)]
|
|
pub struct ConfigLoadError {
|
|
error: ConfigError,
|
|
source: Option<toml::de::Error>,
|
|
}
|
|
|
|
impl ConfigLoadError {
|
|
pub fn new(error: ConfigError, source: Option<toml::de::Error>) -> Self {
|
|
Self { error, source }
|
|
}
|
|
|
|
pub fn config_error(&self) -> &ConfigError {
|
|
&self.error
|
|
}
|
|
}
|
|
|
|
impl fmt::Display for ConfigLoadError {
|
|
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
|
write!(
|
|
f,
|
|
"{}:{}:{}: {}",
|
|
self.error.path.display(),
|
|
self.error.range.start.line,
|
|
self.error.range.start.column,
|
|
self.error.message
|
|
)
|
|
}
|
|
}
|
|
|
|
impl std::error::Error for ConfigLoadError {
|
|
fn source(&self) -> Option<&(dyn std::error::Error + 'static)> {
|
|
self.source
|
|
.as_ref()
|
|
.map(|err| err as &dyn std::error::Error)
|
|
}
|
|
}
|
|
|
|
#[derive(Clone, Copy)]
|
|
pub(crate) enum ConfigDiagnosticSource<'a> {
|
|
Path(&'a Path),
|
|
#[cfg(any(target_os = "macos", test))]
|
|
DisplayName(&'a str),
|
|
}
|
|
|
|
impl ConfigDiagnosticSource<'_> {
|
|
pub(crate) fn to_path_buf(self) -> PathBuf {
|
|
match self {
|
|
ConfigDiagnosticSource::Path(path) => path.to_path_buf(),
|
|
#[cfg(any(target_os = "macos", test))]
|
|
ConfigDiagnosticSource::DisplayName(name) => PathBuf::from(name),
|
|
}
|
|
}
|
|
}
|
|
|
|
pub fn io_error_from_config_error(
|
|
kind: io::ErrorKind,
|
|
error: ConfigError,
|
|
source: Option<toml::de::Error>,
|
|
) -> io::Error {
|
|
io::Error::new(kind, ConfigLoadError::new(error, source))
|
|
}
|
|
|
|
pub fn config_error_from_toml(
|
|
path: impl AsRef<Path>,
|
|
contents: &str,
|
|
err: toml::de::Error,
|
|
) -> ConfigError {
|
|
config_error_from_toml_for_source(ConfigDiagnosticSource::Path(path.as_ref()), contents, err)
|
|
}
|
|
|
|
pub(crate) fn config_error_from_toml_for_source(
|
|
source: ConfigDiagnosticSource<'_>,
|
|
contents: &str,
|
|
err: toml::de::Error,
|
|
) -> ConfigError {
|
|
let range = err
|
|
.span()
|
|
.map(|span| text_range_from_span(contents, span))
|
|
.unwrap_or_else(default_range);
|
|
ConfigError::new(source.to_path_buf(), range, err.message())
|
|
}
|
|
|
|
pub fn config_error_from_typed_toml<T: DeserializeOwned>(
|
|
path: impl AsRef<Path>,
|
|
contents: &str,
|
|
) -> Option<ConfigError> {
|
|
config_error_from_typed_toml_for_source::<T>(
|
|
ConfigDiagnosticSource::Path(path.as_ref()),
|
|
contents,
|
|
)
|
|
}
|
|
|
|
fn config_error_from_typed_toml_for_source<T: DeserializeOwned>(
|
|
source: ConfigDiagnosticSource<'_>,
|
|
contents: &str,
|
|
) -> Option<ConfigError> {
|
|
let deserializer = match toml::de::Deserializer::parse(contents) {
|
|
Ok(deserializer) => deserializer,
|
|
Err(err) => return Some(config_error_from_toml_for_source(source, contents, err)),
|
|
};
|
|
|
|
let result: Result<T, _> = serde_path_to_error::deserialize(deserializer);
|
|
match result {
|
|
Ok(_) => None,
|
|
Err(err) => {
|
|
let path_hint = err.path().clone();
|
|
let toml_err: toml::de::Error = err.into_inner();
|
|
let range = span_for_config_path(contents, &path_hint)
|
|
.or_else(|| toml_err.span())
|
|
.map(|span| text_range_from_span(contents, span))
|
|
.unwrap_or_else(default_range);
|
|
Some(ConfigError::new(
|
|
source.to_path_buf(),
|
|
range,
|
|
toml_err.message(),
|
|
))
|
|
}
|
|
}
|
|
}
|
|
|
|
pub async fn first_layer_config_error<T: DeserializeOwned>(
|
|
layers: &ConfigLayerStack,
|
|
config_toml_file: &str,
|
|
) -> Option<ConfigError> {
|
|
// When the merged config fails schema validation, we surface the first concrete
|
|
// per-file error to point users at a specific file and range rather than an
|
|
// opaque merged-layer failure.
|
|
first_layer_config_error_for_entries::<T, _>(
|
|
layers.get_layers(
|
|
ConfigLayerStackOrdering::LowestPrecedenceFirst,
|
|
/*include_disabled*/ false,
|
|
),
|
|
config_toml_file,
|
|
)
|
|
.await
|
|
}
|
|
|
|
pub async fn first_layer_config_error_from_entries<T: DeserializeOwned>(
|
|
layers: &[ConfigLayerEntry],
|
|
config_toml_file: &str,
|
|
) -> Option<ConfigError> {
|
|
first_layer_config_error_for_entries::<T, _>(layers.iter(), config_toml_file).await
|
|
}
|
|
|
|
async fn first_layer_config_error_for_entries<'a, T: DeserializeOwned, I>(
|
|
layers: I,
|
|
config_toml_file: &str,
|
|
) -> Option<ConfigError>
|
|
where
|
|
I: IntoIterator<Item = &'a ConfigLayerEntry>,
|
|
{
|
|
for layer in layers {
|
|
let Some(path) = config_path_for_layer(layer, config_toml_file) else {
|
|
continue;
|
|
};
|
|
let contents = match tokio::fs::read_to_string(&path).await {
|
|
Ok(contents) => contents,
|
|
Err(err) if err.kind() == io::ErrorKind::NotFound => continue,
|
|
Err(err) => {
|
|
tracing::debug!("Failed to read config file {}: {err}", path.display());
|
|
continue;
|
|
}
|
|
};
|
|
|
|
let Some(parent) = path.parent() else {
|
|
tracing::debug!("Config file {} has no parent directory", path.display());
|
|
continue;
|
|
};
|
|
let _guard = AbsolutePathBufGuard::new(parent);
|
|
if let Some(error) = config_error_from_typed_toml::<T>(&path, &contents) {
|
|
return Some(error);
|
|
}
|
|
}
|
|
|
|
None
|
|
}
|
|
|
|
fn config_path_for_layer(layer: &ConfigLayerEntry, config_toml_file: &str) -> Option<PathBuf> {
|
|
match &layer.name {
|
|
ConfigLayerSource::System { file } => Some(file.to_path_buf()),
|
|
ConfigLayerSource::User { file } => Some(file.to_path_buf()),
|
|
ConfigLayerSource::Project { dot_codex_folder } => {
|
|
Some(dot_codex_folder.as_path().join(config_toml_file))
|
|
}
|
|
ConfigLayerSource::LegacyManagedConfigTomlFromFile { file } => Some(file.to_path_buf()),
|
|
ConfigLayerSource::Mdm { .. }
|
|
| ConfigLayerSource::SessionFlags
|
|
| ConfigLayerSource::LegacyManagedConfigTomlFromMdm => None,
|
|
}
|
|
}
|
|
|
|
pub(crate) fn text_range_from_span(contents: &str, span: std::ops::Range<usize>) -> TextRange {
|
|
let start = position_for_offset(contents, span.start);
|
|
let end_index = if span.end > span.start {
|
|
span.end - 1
|
|
} else {
|
|
span.end
|
|
};
|
|
let end = position_for_offset(contents, end_index);
|
|
TextRange { start, end }
|
|
}
|
|
|
|
pub fn format_config_error(error: &ConfigError, contents: &str) -> String {
|
|
let mut output = String::new();
|
|
let start = error.range.start;
|
|
let _ = writeln!(
|
|
output,
|
|
"{}:{}:{}: {}",
|
|
error.path.display(),
|
|
start.line,
|
|
start.column,
|
|
error.message
|
|
);
|
|
|
|
let line_index = start.line.saturating_sub(1);
|
|
let line = match contents.lines().nth(line_index) {
|
|
Some(line) => line.trim_end_matches('\r'),
|
|
None => return output.trim_end().to_string(),
|
|
};
|
|
|
|
let line_number = start.line;
|
|
let gutter = line_number.to_string().len();
|
|
let _ = writeln!(output, "{:width$} |", "", width = gutter);
|
|
let _ = writeln!(output, "{line_number:>gutter$} | {line}");
|
|
|
|
let highlight_len = if error.range.end.line == error.range.start.line
|
|
&& error.range.end.column >= error.range.start.column
|
|
{
|
|
error.range.end.column - error.range.start.column + 1
|
|
} else {
|
|
1
|
|
};
|
|
let spaces = " ".repeat(start.column.saturating_sub(1));
|
|
let carets = "^".repeat(highlight_len.max(1));
|
|
let _ = writeln!(output, "{:width$} | {spaces}{carets}", "", width = gutter);
|
|
output.trim_end().to_string()
|
|
}
|
|
|
|
pub fn format_config_error_with_source(error: &ConfigError) -> String {
|
|
match std::fs::read_to_string(&error.path) {
|
|
Ok(contents) => format_config_error(error, &contents),
|
|
Err(_) => format_config_error(error, ""),
|
|
}
|
|
}
|
|
|
|
fn position_for_offset(contents: &str, index: usize) -> TextPosition {
|
|
let bytes = contents.as_bytes();
|
|
if bytes.is_empty() {
|
|
return TextPosition { line: 1, column: 1 };
|
|
}
|
|
|
|
let safe_index = index.min(bytes.len().saturating_sub(1));
|
|
let column_offset = index.saturating_sub(safe_index);
|
|
let index = safe_index;
|
|
|
|
let line_start = bytes[..index]
|
|
.iter()
|
|
.rposition(|byte| *byte == b'\n')
|
|
.map(|pos| pos + 1)
|
|
.unwrap_or(0);
|
|
let line = bytes[..line_start]
|
|
.iter()
|
|
.filter(|byte| **byte == b'\n')
|
|
.count();
|
|
|
|
let column = std::str::from_utf8(&bytes[line_start..=index])
|
|
.map(|slice| slice.chars().count().saturating_sub(1))
|
|
.unwrap_or_else(|_| index - line_start);
|
|
let column = column + column_offset;
|
|
|
|
TextPosition {
|
|
line: line + 1,
|
|
column: column + 1,
|
|
}
|
|
}
|
|
|
|
pub(crate) fn default_range() -> TextRange {
|
|
let position = TextPosition { line: 1, column: 1 };
|
|
TextRange {
|
|
start: position,
|
|
end: position,
|
|
}
|
|
}
|
|
|
|
enum TomlNode<'a> {
|
|
Item(&'a Item),
|
|
Table(&'a Table),
|
|
Value(&'a Value),
|
|
}
|
|
|
|
fn span_for_path(contents: &str, path: &SerdePath) -> Option<std::ops::Range<usize>> {
|
|
let doc = contents.parse::<Document<String>>().ok()?;
|
|
let node = node_for_path(doc.as_item(), path)?;
|
|
match node {
|
|
TomlNode::Item(item) => item.span(),
|
|
TomlNode::Table(table) => table.span(),
|
|
TomlNode::Value(value) => value.span(),
|
|
}
|
|
}
|
|
|
|
pub(crate) fn span_for_config_path(
|
|
contents: &str,
|
|
path: &SerdePath,
|
|
) -> Option<std::ops::Range<usize>> {
|
|
if is_features_table_path(path)
|
|
&& let Some(span) = span_for_features_value(contents)
|
|
{
|
|
return Some(span);
|
|
}
|
|
span_for_path(contents, path)
|
|
}
|
|
|
|
pub(crate) fn span_for_toml_key_path(
|
|
contents: &str,
|
|
path: &[String],
|
|
) -> Option<std::ops::Range<usize>> {
|
|
let doc = contents.parse::<Document<String>>().ok()?;
|
|
let mut node = TomlNode::Item(doc.as_item());
|
|
for (index, segment) in path.iter().enumerate() {
|
|
if index + 1 == path.len() {
|
|
let key_span = match &node {
|
|
TomlNode::Item(item) => item
|
|
.as_table_like()
|
|
.and_then(|table| table.get_key_value(segment))
|
|
.and_then(|(key, _)| key.span()),
|
|
TomlNode::Table(table) => {
|
|
table.get_key_value(segment).and_then(|(key, _)| key.span())
|
|
}
|
|
TomlNode::Value(Value::InlineTable(table)) => {
|
|
table.get_key_value(segment).and_then(|(key, _)| key.span())
|
|
}
|
|
_ => None,
|
|
};
|
|
if key_span.is_some() {
|
|
return key_span;
|
|
}
|
|
}
|
|
|
|
if let Some(next) = map_child(&node, segment) {
|
|
node = next;
|
|
continue;
|
|
}
|
|
|
|
let index = segment.parse::<usize>().ok()?;
|
|
node = seq_child(&node, index)?;
|
|
}
|
|
|
|
match node {
|
|
TomlNode::Item(item) => item.span(),
|
|
TomlNode::Table(table) => table.span(),
|
|
TomlNode::Value(value) => value.span(),
|
|
}
|
|
}
|
|
|
|
fn is_features_table_path(path: &SerdePath) -> bool {
|
|
let mut segments = path.iter();
|
|
matches!(segments.next(), Some(SerdeSegment::Map { key }) if key == "features")
|
|
&& segments.next().is_none()
|
|
}
|
|
|
|
fn span_for_features_value(contents: &str) -> Option<std::ops::Range<usize>> {
|
|
let doc = contents.parse::<Document<String>>().ok()?;
|
|
let root = doc.as_item().as_table_like()?;
|
|
let features_item = root.get("features")?;
|
|
let features_table = features_item.as_table_like()?;
|
|
for (_, item) in features_table.iter() {
|
|
match item {
|
|
Item::Value(Value::Boolean(_)) => continue,
|
|
Item::Value(value) => return value.span(),
|
|
Item::Table(table) => return table.span(),
|
|
Item::ArrayOfTables(array) => return array.span(),
|
|
Item::None => continue,
|
|
}
|
|
}
|
|
None
|
|
}
|
|
|
|
fn node_for_path<'a>(item: &'a Item, path: &SerdePath) -> Option<TomlNode<'a>> {
|
|
let segments: Vec<_> = path.iter().cloned().collect();
|
|
let mut node = TomlNode::Item(item);
|
|
let mut index = 0;
|
|
while index < segments.len() {
|
|
match &segments[index] {
|
|
SerdeSegment::Map { key } | SerdeSegment::Enum { variant: key } => {
|
|
if let Some(next) = map_child(&node, key) {
|
|
node = next;
|
|
index += 1;
|
|
continue;
|
|
}
|
|
|
|
if index + 1 < segments.len() {
|
|
index += 1;
|
|
continue;
|
|
}
|
|
return None;
|
|
}
|
|
SerdeSegment::Seq { index: seq_index } => {
|
|
node = seq_child(&node, *seq_index)?;
|
|
index += 1;
|
|
}
|
|
SerdeSegment::Unknown => return None,
|
|
}
|
|
}
|
|
Some(node)
|
|
}
|
|
|
|
fn map_child<'a>(node: &TomlNode<'a>, key: &str) -> Option<TomlNode<'a>> {
|
|
match node {
|
|
TomlNode::Item(item) => {
|
|
let table = item.as_table_like()?;
|
|
table.get(key).map(TomlNode::Item)
|
|
}
|
|
TomlNode::Table(table) => table.get(key).map(TomlNode::Item),
|
|
TomlNode::Value(Value::InlineTable(table)) => table.get(key).map(TomlNode::Value),
|
|
_ => None,
|
|
}
|
|
}
|
|
|
|
fn seq_child<'a>(node: &TomlNode<'a>, index: usize) -> Option<TomlNode<'a>> {
|
|
match node {
|
|
TomlNode::Item(Item::Value(Value::Array(array))) => array.get(index).map(TomlNode::Value),
|
|
TomlNode::Item(Item::ArrayOfTables(array)) => array.get(index).map(TomlNode::Table),
|
|
TomlNode::Value(Value::Array(array)) => array.get(index).map(TomlNode::Value),
|
|
_ => None,
|
|
}
|
|
}
|