mirror of
https://github.com/openai/codex.git
synced 2026-04-27 08:05:51 +00:00
feat: migrate to new constraint-based loading strategy (#8251)
This is a significant change to how layers of configuration are applied. In particular, the `ConfigLayerStack` now has two important fields: - `layers: Vec<ConfigLayerEntry>` - `requirements: ConfigRequirements` We merge `TomlValue`s across the layers, but they are subject to `ConfigRequirements` before creating a `Config`. How I would review this PR: - start with `codex-rs/app-server-protocol/src/protocol/v2.rs` and note the new variants added to the `ConfigLayerSource` enum: `LegacyManagedConfigTomlFromFile` and `LegacyManagedConfigTomlFromMdm` - note that `ConfigLayerSource` now has a `precedence()` method and implements `PartialOrd` - `codex-rs/core/src/config_loader/layer_io.rs` is responsible for loading "admin" preferences from `/etc/codex/managed_config.toml` and MDM. Because `/etc/codex/managed_config.toml` is now deprecated in favor of `/etc/codex/requirements.toml` and `/etc/codex/config.toml`, we now include some extra information on the `LoadedConfigLayers` returned in `layer_io.rs`. - `codex-rs/core/src/config_loader/mod.rs` has major changes to `load_config_layers_state()`, which is what produces `ConfigLayerStack`. The docstring has the new specification and describes the various layers that will be loaded and the precedence order. - It uses the information from `LoaderOverrides` "twice," both in the spirit of legacy support: - We use one instances to derive an instance of `ConfigRequirements`. Currently, the only field in `managed_config.toml` that contributes to `ConfigRequirements` is `approval_policy`. This PR introduces `Constrained::allow_only()` to support this. - We use a clone of `LoaderOverrides` to derive `ConfigLayerSource::LegacyManagedConfigTomlFromFile` and `ConfigLayerSource::LegacyManagedConfigTomlFromMdm` layers, as appropriate. As before, this ends up being a "best effort" at enterprise controls, but is enforcement is not guaranteed like it is for `ConfigRequirements`. - Now we only create a "user" layer if `$CODEX_HOME/config.toml` exists. (Previously, a user layer was always created for `ConfigLayerStack`.) - Similarly, we only add a "session flags" layer if there are CLI overrides. - `config_loader/state.rs` contains the updated implementation for `ConfigLayerStack`. Note the public API is largely the same as before, but the implementation is quite different. We leverage the fact that `ConfigLayerSource` is now `PartialOrd` to ensure layers are in the correct order. - A `Config` constructed via `ConfigBuilder.build()` will use `load_config_layers_state()` to create the `ConfigLayerStack` and use the associated `ConfigRequirements` when constructing the `Config` object. - That said, a `Config` constructed via `Config::load_from_base_config_with_overrides()` does _not_ yet use `ConfigBuilder`, so it creates a `ConfigRequirements::default()` instead of loading a proper `ConfigRequirements`. I will fix this in a subsequent PR. Then the following files are mostly test changes: ``` codex-rs/app-server/tests/suite/v2/config_rpc.rs codex-rs/core/src/config/service.rs codex-rs/core/src/config_loader/tests.rs ``` Again, because we do not always include "user" and "session flags" layers when the contents are empty, `ConfigLayerStack` sometimes has fewer layers than before (and the precedence order changed slightly), which is the main reason integration tests changed.
This commit is contained in:
Michael Bolin
@@ -1,3 +1,4 @@
|
||||
mod config_requirements;
|
||||
mod fingerprint;
|
||||
mod layer_io;
|
||||
#[cfg(target_os = "macos")]
|
||||
@@ -10,74 +11,176 @@ mod state;
|
||||
mod tests;
|
||||
|
||||
use crate::config::CONFIG_TOML_FILE;
|
||||
use crate::config_loader::layer_io::LoadedConfigLayers;
|
||||
use codex_app_server_protocol::ConfigLayerSource;
|
||||
use codex_protocol::protocol::AskForApproval;
|
||||
use codex_utils_absolute_path::AbsolutePathBuf;
|
||||
use serde::Deserialize;
|
||||
use std::io;
|
||||
use std::path::Path;
|
||||
use toml::Value as TomlValue;
|
||||
|
||||
pub use config_requirements::ConfigRequirements;
|
||||
pub use merge::merge_toml_values;
|
||||
pub use state::ConfigLayerEntry;
|
||||
pub use state::ConfigLayerStack;
|
||||
pub use state::LoaderOverrides;
|
||||
|
||||
const MDM_PREFERENCES_DOMAIN: &str = "com.openai.codex";
|
||||
const MDM_PREFERENCES_KEY: &str = "config_toml_base64";
|
||||
|
||||
/// Configuration layering pipeline (top overrides bottom):
|
||||
/// To build up the set of admin-enforced constraints, we build up from multiple
|
||||
/// configuration layers in the following order, but a constraint defined in an
|
||||
/// earlier layer cannot be overridden by a later layer:
|
||||
///
|
||||
/// +-------------------------+
|
||||
/// | Managed preferences (*) |
|
||||
/// +-------------------------+
|
||||
/// ^
|
||||
/// |
|
||||
/// +-------------------------+
|
||||
/// | managed_config.toml |
|
||||
/// +-------------------------+
|
||||
/// ^
|
||||
/// |
|
||||
/// +-------------------------+
|
||||
/// | config.toml (base) |
|
||||
/// +-------------------------+
|
||||
/// - admin: managed preferences (*)
|
||||
/// - system `/etc/codex/requirements.toml`
|
||||
///
|
||||
/// For backwards compatibility, we also load from
|
||||
/// `/etc/codex/managed_config.toml` and map it to
|
||||
/// `/etc/codex/requirements.toml`.
|
||||
///
|
||||
/// Configuration is built up from multiple layers in the following order:
|
||||
///
|
||||
/// - admin: managed preferences (*)
|
||||
/// - system `/etc/codex/config.toml`
|
||||
/// - user `${CODEX_HOME}/config.toml`
|
||||
/// - cwd `${PWD}/config.toml`
|
||||
/// - tree parent directories up to root looking for `./.codex/config.toml`
|
||||
/// - repo `$(git rev-parse --show-toplevel)/.codex/config.toml`
|
||||
/// - runtime e.g., --config flags, model selector in UI
|
||||
///
|
||||
/// (*) Only available on macOS via managed device profiles.
|
||||
///
|
||||
/// See https://developers.openai.com/codex/security for details.
|
||||
pub async fn load_config_layers_state(
|
||||
codex_home: &Path,
|
||||
cli_overrides: &[(String, TomlValue)],
|
||||
overrides: LoaderOverrides,
|
||||
) -> io::Result<ConfigLayerStack> {
|
||||
let managed_config_path = overrides
|
||||
.managed_config_path
|
||||
.clone()
|
||||
.unwrap_or_else(|| layer_io::managed_config_default_path(codex_home));
|
||||
let loaded_config_layers = layer_io::load_config_layers_internal(codex_home, overrides).await?;
|
||||
let requirements = load_requirements_from_legacy_scheme(loaded_config_layers.clone()).await?;
|
||||
|
||||
let layers = layer_io::load_config_layers_internal(codex_home, overrides).await?;
|
||||
let cli_overrides_layer = overrides::build_cli_overrides_layer(cli_overrides);
|
||||
let user_file = AbsolutePathBuf::from_absolute_path(codex_home.join(CONFIG_TOML_FILE))?;
|
||||
// TODO(mbolin): Honor /etc/codex/requirements.toml.
|
||||
|
||||
let system = match layers.managed_config {
|
||||
Some(cfg) => {
|
||||
let system_file = AbsolutePathBuf::from_absolute_path(managed_config_path.clone())?;
|
||||
Some(ConfigLayerEntry::new(
|
||||
ConfigLayerSource::System { file: system_file },
|
||||
cfg,
|
||||
))
|
||||
let mut layers = Vec::<ConfigLayerEntry>::new();
|
||||
|
||||
// TODO(mbolin): Honor managed preferences (macOS only).
|
||||
// TODO(mbolin): Honor /etc/codex/config.toml.
|
||||
|
||||
// Add a layer for $CODEX_HOME/config.toml if it exists. Note if the file
|
||||
// exists, but is malformed, then this error should be propagated to the
|
||||
// user.
|
||||
let user_file = AbsolutePathBuf::resolve_path_against_base(CONFIG_TOML_FILE, codex_home)?;
|
||||
match tokio::fs::read_to_string(&user_file).await {
|
||||
Ok(contents) => {
|
||||
let user_config: TomlValue = toml::from_str(&contents).map_err(|e| {
|
||||
io::Error::new(
|
||||
io::ErrorKind::InvalidData,
|
||||
format!(
|
||||
"Error parsing user config file {}: {e}",
|
||||
user_file.as_path().display(),
|
||||
),
|
||||
)
|
||||
})?;
|
||||
layers.push(ConfigLayerEntry::new(
|
||||
ConfigLayerSource::User { file: user_file },
|
||||
user_config,
|
||||
));
|
||||
}
|
||||
None => None,
|
||||
};
|
||||
Err(e) => {
|
||||
if e.kind() != io::ErrorKind::NotFound {
|
||||
return Err(io::Error::new(
|
||||
e.kind(),
|
||||
format!(
|
||||
"Failed to read user config file {}: {e}",
|
||||
user_file.as_path().display(),
|
||||
),
|
||||
));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
Ok(ConfigLayerStack {
|
||||
user: ConfigLayerEntry::new(ConfigLayerSource::User { file: user_file }, layers.base),
|
||||
session_flags: ConfigLayerEntry::new(ConfigLayerSource::SessionFlags, cli_overrides_layer),
|
||||
system,
|
||||
mdm: layers.managed_preferences.map(|cfg| {
|
||||
ConfigLayerEntry::new(
|
||||
ConfigLayerSource::Mdm {
|
||||
domain: MDM_PREFERENCES_DOMAIN.to_string(),
|
||||
key: MDM_PREFERENCES_KEY.to_string(),
|
||||
},
|
||||
cfg,
|
||||
)
|
||||
}),
|
||||
})
|
||||
// TODO(mbolin): Add layers for cwd, tree, and repo config files.
|
||||
|
||||
// Add a layer for runtime overrides from the CLI or UI, if any exist.
|
||||
if !cli_overrides.is_empty() {
|
||||
let cli_overrides_layer = overrides::build_cli_overrides_layer(cli_overrides);
|
||||
layers.push(ConfigLayerEntry::new(
|
||||
ConfigLayerSource::SessionFlags,
|
||||
cli_overrides_layer,
|
||||
));
|
||||
}
|
||||
|
||||
// Make a best-effort to support the legacy `managed_config.toml` as a
|
||||
// config layer on top of everything else. For fields in
|
||||
// `managed_config.toml` that do not have an equivalent in
|
||||
// `ConfigRequirements`, note users can still override these values on a
|
||||
// per-turn basis in the TUI and VS Code.
|
||||
let LoadedConfigLayers {
|
||||
managed_config,
|
||||
managed_config_from_mdm,
|
||||
} = loaded_config_layers;
|
||||
if let Some(config) = managed_config {
|
||||
layers.push(ConfigLayerEntry::new(
|
||||
ConfigLayerSource::LegacyManagedConfigTomlFromFile {
|
||||
file: config.file.clone(),
|
||||
},
|
||||
config.managed_config,
|
||||
));
|
||||
}
|
||||
if let Some(config) = managed_config_from_mdm {
|
||||
layers.push(ConfigLayerEntry::new(
|
||||
ConfigLayerSource::LegacyManagedConfigTomlFromMdm,
|
||||
config,
|
||||
));
|
||||
}
|
||||
|
||||
ConfigLayerStack::new(layers, requirements)
|
||||
}
|
||||
|
||||
async fn load_requirements_from_legacy_scheme(
|
||||
loaded_config_layers: LoadedConfigLayers,
|
||||
) -> io::Result<ConfigRequirements> {
|
||||
let mut config_requirements = ConfigRequirements::default();
|
||||
|
||||
// In this implementation, later layers override earlier layers, so list
|
||||
// managed_config_from_mdm last because it has the highest precedence.
|
||||
let LoadedConfigLayers {
|
||||
managed_config,
|
||||
managed_config_from_mdm,
|
||||
} = loaded_config_layers;
|
||||
for config in [
|
||||
managed_config.map(|c| c.managed_config),
|
||||
managed_config_from_mdm,
|
||||
]
|
||||
.into_iter()
|
||||
.flatten()
|
||||
{
|
||||
let legacy_config: LegacyManagedConfigToml =
|
||||
config.try_into().map_err(|err: toml::de::Error| {
|
||||
io::Error::new(
|
||||
io::ErrorKind::InvalidData,
|
||||
format!("Failed to parse config requirements as TOML: {err}"),
|
||||
)
|
||||
})?;
|
||||
|
||||
let LegacyManagedConfigToml { approval_policy } = legacy_config;
|
||||
if let Some(approval_policy) = approval_policy {
|
||||
config_requirements.approval_policy =
|
||||
crate::config::Constrained::allow_only(approval_policy);
|
||||
}
|
||||
}
|
||||
|
||||
Ok(config_requirements)
|
||||
}
|
||||
|
||||
/// The legacy mechanism for specifying admin-enforced configuration is to read
|
||||
/// from a file like `/etc/codex/managed_config.toml` that has the same
|
||||
/// structure as `config.toml` where fields like `approval_policy` can specify
|
||||
/// exactly one value rather than a list of allowed values.
|
||||
///
|
||||
/// If present, re-interpret `managed_config.toml` as a `requirements.toml`
|
||||
/// where each specified field is treated as a constraint allowing only that
|
||||
/// value.
|
||||
#[derive(Deserialize, Debug, Clone, Default, PartialEq)]
|
||||
struct LegacyManagedConfigToml {
|
||||
approval_policy: Option<AskForApproval>,
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user