Load managed requirements from MDM key `requirements_toml_base64`.
Tested on my Mac (using `defaults` to set the preference, though this
would be set by MDM in production):
```
➜ codex git:(gt/mdm-requirements) defaults read com.openai.codex requirements_toml_base64 | base64 -d
allowed_approval_policies = ["on-request"]
➜ codex git:(gt/mdm-requirements) just c --yolo
cargo run --bin codex -- "$@"
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.26s
Running `target/debug/codex --yolo`
Error loading configuration: value `Never` is not in the allowed set [OnRequest]
error: Recipe `codex` failed on line 11 with exit code 1
➜ codex git:(gt/mdm-requirements) defaults delete com.openai.codex requirements_toml_base64
➜ codex git:(gt/mdm-requirements) just c --yolo
cargo run --bin codex -- "$@"
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.24s
Running `target/debug/codex --yolo`
╭──────────────────────────────────────────────────────────╮
│ >_ OpenAI Codex (v0.0.0) │
│ │
│ model: codex-auto-balanced medium /model to change │
│ directory: ~/code/codex/codex-rs │
╰──────────────────────────────────────────────────────────╯
Tip: Start a fresh idea with /new; the previous session stays in history.
```
This isn't very useful parameter.
logic:
```
if model puts `**` in their reasoning, trim it and visualize the header.
if couldn't trim: don't render
if model doesn't support: don't render
```
We can simplify to:
```
if could trim, visualize header.
if not, don't render
```
This adds logic to load `/etc/codex/config.toml` and associate it with
`ConfigLayerSource::System` on UNIX. I refactored the code so it shares
logic with the creation of the `ConfigLayerSource::User` layer.
- allow configuring `project_root_markers` in `config.toml`
(user/system/MDM) to control project discovery beyond `.git`
- honor the markers after merging pre-project layers; default to
`[".git"]` when unset and skip ancestor walk when set to an empty array
- document the option and add coverage for alternate markers in config
loader tests
- We now support `.codex/config.toml` in repo (from `cwd` up to the
first `.git` found, if any) as layers in `ConfigLayerStack`. A new
`ConfigLayerSource::Project` variant was added to support this.
- In doing this work, I realized that we were resolving relative paths
in `config.toml` after merging everything into one `toml::Value`, which
is wrong: paths should be relativized with respect to the folder
containing the `config.toml` that was deserialized. This PR introduces a
deserialize/re-serialize strategy to account for this in
`resolve_config_paths()`. (This is why `Serialize` is added to so many
types as part of this PR.)
- Added tests to verify this new behavior.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/8354).
* #8359
* __->__ #8354
### Summary
With codesigning on Mac, Windows and Linux, we should be able to safely
remove `features.rmcp_client` and `use_experimental_use_rmcp_client`
check from the codebase now.
## TUI2: Normalize Mouse Scroll Input Across Terminals (Wheel +
Trackpad)
This changes TUI2 scrolling to a stream-based model that normalizes
terminal scroll event density into consistent wheel behavior (default:
~3 transcript lines per physical wheel notch) while keeping trackpad
input higher fidelity via fractional accumulation.
Primary code: `codex-rs/tui2/src/tui/scrolling/mouse.rs`
Doc of record (model + probe-derived data):
`codex-rs/tui2/docs/scroll_input_model.md`
### Why
Terminals encode both mouse wheels and trackpads as discrete scroll
up/down events with direction but no magnitude, and they vary widely in
how many raw events they emit per physical wheel notch (commonly 1, 3,
or 9+). Timing alone doesn’t reliably distinguish wheel vs trackpad, so
cadence-based heuristics are unstable across terminals/hardware.
This PR treats scroll input as short *streams* separated by silence or
direction flips, normalizes raw event density into tick-equivalents,
coalesces redraws for dense streams, and exposes explicit config
overrides.
### What Changed
#### Scroll Model (TUI2)
- Stream detection
- Start a stream on the first scroll event.
- End a stream on an idle gap (`STREAM_GAP_MS`) or a direction flip.
- Normalization
- Convert raw events into tick-equivalents using per-terminal
`tui.scroll_events_per_tick`.
- Wheel-like vs trackpad-like behavior
- Wheel-like: fixed “classic” lines per wheel notch; flush immediately
for responsiveness.
- Trackpad-like: fractional accumulation + carry across stream
boundaries; coalesce flushes to ~60Hz to avoid floods and reduce “stop
lag / overshoot”.
- Trackpad divisor is intentionally capped: `min(scroll_events_per_tick,
3)` so terminals with dense wheel ticks (e.g. 9 events per notch) don’t
make trackpads feel artificially slow.
- Auto mode (default)
- Start conservatively as trackpad-like (avoid overshoot).
- Promote to wheel-like if the first tick-worth of events arrives
quickly.
- Fallback for 1-event-per-tick terminals (no tick-completion timing
signal).
#### Trackpad Acceleration
Some terminals produce relatively low vertical event density for
trackpad gestures, which makes large/faster swipes feel sluggish even
when small motions feel correct. To address that, trackpad-like streams
apply a bounded multiplier based on event count:
- `multiplier = clamp(1 + abs(events) / scroll_trackpad_accel_events,
1..scroll_trackpad_accel_max)`
The multiplier is applied to the trackpad stream’s computed line delta
(including carried fractional remainder). Defaults are conservative and
bounded.
#### Config Knobs (TUI2)
All keys live under `[tui]`:
- `scroll_wheel_lines`: lines per physical wheel notch (default: 3).
- `scroll_events_per_tick`: raw vertical scroll events per physical
wheel notch (terminal-specific default; fallback: 3).
- Wheel-like per-event contribution: `scroll_wheel_lines /
scroll_events_per_tick`.
- `scroll_trackpad_lines`: baseline trackpad sensitivity (default: 1).
- Trackpad-like per-event contribution: `scroll_trackpad_lines /
min(scroll_events_per_tick, 3)`.
- `scroll_trackpad_accel_events` / `scroll_trackpad_accel_max`: bounded
trackpad acceleration (defaults: 30 / 3).
- `scroll_mode = auto|wheel|trackpad`: force behavior or use the
heuristic (default: `auto`).
- `scroll_wheel_tick_detect_max_ms`: auto-mode promotion threshold (ms).
- `scroll_wheel_like_max_duration_ms`: auto-mode fallback for
1-event-per-tick terminals (ms).
- `scroll_invert`: invert scroll direction (applies to wheel +
trackpad).
Config docs: `docs/config.md` and field docs in
`codex-rs/core/src/config/types.rs`.
#### App Integration
- The app schedules follow-up ticks to close idle streams (via
`ScrollUpdate::next_tick_in` and `schedule_frame_in`) and finalizes
streams on draw ticks.
- `codex-rs/tui2/src/app.rs`
#### Docs
- Single doc of record describing the model + preserved probe
findings/spec:
- `codex-rs/tui2/docs/scroll_input_model.md`
#### Other (jj-only friendliness)
- `codex-rs/tui2/src/diff_render.rs`: prefer stable cwd-relative paths
when the file is under the cwd even if there’s no `.git`.
### Terminal Defaults
Per-terminal defaults are derived from scroll-probe logs (see doc).
Notable:
- Ghostty currently defaults to `scroll_events_per_tick = 3` even though
logs measured ~9 in one setup. This is a deliberate stopgap; if your
Ghostty build emits ~9 events per wheel notch, set:
```toml
[tui]
scroll_events_per_tick = 9
```
### Testing
- `just fmt`
- `just fix -p codex-core --allow-no-vcs`
- `cargo test -p codex-core --lib` (pass)
- `cargo test -p codex-tui2` (scroll tests pass; remaining failures are
known flaky VT100 color tests in `insert_history`)
### Review Focus
- Stream finalization + frame scheduling in `codex-rs/tui2/src/app.rs`.
- Auto-mode promotion thresholds and the 1-event-per-tick fallback
behavior.
- Trackpad divisor cap (`min(events_per_tick, 3)`) and acceleration
defaults.
- Ghostty default tradeoff (3 vs ~9) and whether we should change it.
`load_config_layers_state()` should load config from a
`.codex/config.toml` in any folder between the `cwd` for a thread and
the project root. Though in order to do that,
`load_config_layers_state()` needs to know what the `cwd` is, so this PR
does the work to thread the `cwd` through for existing callsites.
A notable exception is the `/config` endpoint in app server for which a
`cwd` is not guaranteed to be associated with the query, so the `cwd`
param is `Option<AbsolutePathBuf>` to account for this case.
The logic to make use of the `cwd` will be done in a follow-up PR.
This adds support for `allowed_sandbox_modes` in `requirements.toml` and
provides legacy support for constraining sandbox modes in
`managed_config.toml`. This is converted to `Constrained<SandboxPolicy>`
in `ConfigRequirements` and applied to `Config` such that constraints
are enforced throughout the harness.
Note that, because `managed_config.toml` is deprecated, we do not add
support for the new `external-sandbox` variant recently introduced in
https://github.com/openai/codex/pull/8290. As noted, that variant is not
supported in `config.toml` today, but can be configured programmatically
via app server.
This will make it easier to test for expected errors in unit tests since
we can compare based on the field values rather than the message (which
might change over time). See https://github.com/openai/codex/pull/8298
for an example.
It also ensures more consistency in the way a `ConstraintError` is
constructed.
https://github.com/openai/codex/pull/8235 introduced `ConfigBuilder` and
this PR updates all call non-test call sites to use it instead of
`Config::load_from_base_config_with_overrides()`.
This is important because `load_from_base_config_with_overrides()` uses
an empty `ConfigRequirements`, which is a reasonable default for testing
so the tests are not influenced by the settings on the host. This method
is now guarded by `#[cfg(test)]` so it cannot be used by business logic.
Because `ConfigBuilder::build()` is `async`, many of the test methods
had to be migrated to be `async`, as well. On the bright side, this made
it possible to eliminate a bunch of `block_on_future()` stuff.
This is a significant change to how layers of configuration are applied.
In particular, the `ConfigLayerStack` now has two important fields:
- `layers: Vec<ConfigLayerEntry>`
- `requirements: ConfigRequirements`
We merge `TomlValue`s across the layers, but they are subject to
`ConfigRequirements` before creating a `Config`.
How I would review this PR:
- start with `codex-rs/app-server-protocol/src/protocol/v2.rs` and note
the new variants added to the `ConfigLayerSource` enum:
`LegacyManagedConfigTomlFromFile` and `LegacyManagedConfigTomlFromMdm`
- note that `ConfigLayerSource` now has a `precedence()` method and
implements `PartialOrd`
- `codex-rs/core/src/config_loader/layer_io.rs` is responsible for
loading "admin" preferences from `/etc/codex/managed_config.toml` and
MDM. Because `/etc/codex/managed_config.toml` is now deprecated in favor
of `/etc/codex/requirements.toml` and `/etc/codex/config.toml`, we now
include some extra information on the `LoadedConfigLayers` returned in
`layer_io.rs`.
- `codex-rs/core/src/config_loader/mod.rs` has major changes to
`load_config_layers_state()`, which is what produces `ConfigLayerStack`.
The docstring has the new specification and describes the various layers
that will be loaded and the precedence order.
- It uses the information from `LoaderOverrides` "twice," both in the
spirit of legacy support:
- We use one instances to derive an instance of `ConfigRequirements`.
Currently, the only field in `managed_config.toml` that contributes to
`ConfigRequirements` is `approval_policy`. This PR introduces
`Constrained::allow_only()` to support this.
- We use a clone of `LoaderOverrides` to derive
`ConfigLayerSource::LegacyManagedConfigTomlFromFile` and
`ConfigLayerSource::LegacyManagedConfigTomlFromMdm` layers, as
appropriate. As before, this ends up being a "best effort" at enterprise
controls, but is enforcement is not guaranteed like it is for
`ConfigRequirements`.
- Now we only create a "user" layer if `$CODEX_HOME/config.toml` exists.
(Previously, a user layer was always created for `ConfigLayerStack`.)
- Similarly, we only add a "session flags" layer if there are CLI
overrides.
- `config_loader/state.rs` contains the updated implementation for
`ConfigLayerStack`. Note the public API is largely the same as before,
but the implementation is quite different. We leverage the fact that
`ConfigLayerSource` is now `PartialOrd` to ensure layers are in the
correct order.
- A `Config` constructed via `ConfigBuilder.build()` will use
`load_config_layers_state()` to create the `ConfigLayerStack` and use
the associated `ConfigRequirements` when constructing the `Config`
object.
- That said, a `Config` constructed via
`Config::load_from_base_config_with_overrides()` does _not_ yet use
`ConfigBuilder`, so it creates a `ConfigRequirements::default()` instead
of loading a proper `ConfigRequirements`. I will fix this in a
subsequent PR.
Then the following files are mostly test changes:
```
codex-rs/app-server/tests/suite/v2/config_rpc.rs
codex-rs/core/src/config/service.rs
codex-rs/core/src/config_loader/tests.rs
```
Again, because we do not always include "user" and "session flags"
layers when the contents are empty, `ConfigLayerStack` sometimes has
fewer layers than before (and the precedence order changed slightly),
which is the main reason integration tests changed.
Introduce `ConfigBuilder` as an alternative to our existing `Config`
constructors.
I noticed that the existing constructors,
`Config::load_with_cli_overrides()` and
`Config::load_with_cli_overrides_and_harness_overrides()`, did not take
`codex_home` as a parameter, which can be a problem.
Historically, when Codex was purely a CLI, we wanted to be extra sure
that the creation of `codex_home` was always done via
`find_codex_home()`, so we did not expose `codex_home` as a parameter
when creating `Config` in business logic. But in integration tests,
`codex_home` nearly always needs to be configured (as a temp directory),
which is why callers would have to go through
`Config::load_from_base_config_with_overrides()` instead.
Now that the Codex harness also functions as an app server, which could
conceivably load multiple threads where `codex_home` is parameterized
differently in each one, I think it makes sense to make this
configurable. Going to a builder pattern makes it more flexible to
ensure an arbitrary permutation of options can be set when constructing
a `Config` while using the appropriate defaults for the options that
aren't set explicitly.
Ultimately, I think this should make it possible for us to make
`Config::load_from_base_config_with_overrides()` private because all
integration tests should be able to leverage `ConfigBuilder` instead.
Though there could be edge cases, so I'll pursue that migration after we
get through the current config overhaul.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/8235).
* #8237
* __->__ #8235
This PR does various types of cleanup before I can proceed with more
ambitious changes to config loading.
First, I noticed duplicated code across these two methods:
774bd9e432/codex-rs/core/src/config/mod.rs (L314-L324)774bd9e432/codex-rs/core/src/config/mod.rs (L334-L344)
This has now been consolidated in
`load_config_as_toml_with_cli_overrides()`.
Further, I noticed that `Config::load_with_cli_overrides()` took two
similar arguments:
774bd9e432/codex-rs/core/src/config/mod.rs (L308-L311)
The difference between `cli_overrides` and `overrides` was not
immediately obvious to me. At first glance, it appears that one should
be able to be expressed in terms of the other, but it turns out that
some fields of `ConfigOverrides` (such as `cwd` and
`codex_linux_sandbox_exe`) are, by design, not configurable via a
`.toml` file or a command-line `--config` flag.
That said, I discovered that many callers of
`Config::load_with_cli_overrides()` were passing
`ConfigOverrides::default()` for `overrides`, so I created two separate
methods:
- `Config::load_with_cli_overrides(cli_overrides: Vec<(String,
TomlValue)>)`
- `Config::load_with_cli_overrides_and_harness_overrides(cli_overrides:
Vec<(String, TomlValue)>, harness_overrides: ConfigOverrides)`
The latter has a long name, as it is _not_ what should be used in the
common case, so the extra typing is designed to draw attention to this
fact. I tried to update the existing callsites to use the shorter name,
where possible.
Further, in the cases where `ConfigOverrides` is used, usually only a
limited subset of fields are actually set, so I updated the declarations
to leverage `..Default::default()` where possible.
We should not have any `PathBuf` fields in `ConfigToml` or any of the
transitive structs we include, as we should use `AbsolutePathBuf`
instead so that we do not have to keep track of the file from which
`ConfigToml` was loaded such that we need it to resolve relative paths
later when the values of `ConfigToml` are used.
I only found two instances of this: `experimental_instructions_file` and
`experimental_compact_prompt_file`. Incidentally, when these were
specified as relative paths, they were resolved against `cwd` rather
than `config.toml`'s parent, which seems wrong to me. I changed the
behavior so they are resolved against the parent folder of the
`config.toml` being parsed, which we get "for free" due to the
introduction of `AbsolutePathBufGuard ` in
https://github.com/openai/codex/pull/7796.
While it is not great to change the behavior of a released feature,
these fields are prefixed with `experimental_`, which I interpret to
mean we have the liberty to change the contract.
For reference:
- `experimental_instructions_file` was introduced in
https://github.com/openai/codex/pull/1803
- `experimental_compact_prompt_file` was introduced in
https://github.com/openai/codex/pull/5959
Constrain `approval_policy` through new `admin_policy` config.
This PR will:
1. Add a `admin_policy` section to config, with a single field (for now)
`allowed_approval_policies`. This list constrains the set of
user-settable `approval_policy`s.
2. Introduce a new `Constrained<T>` type, which combines a current value
and a validator function. The validator function ensures disallowed
values are not set.
3. Change the type of `approval_policy` on `Config` and
`SessionConfiguration` from `AskForApproval` to
`Constrained<AskForApproval>`. The validator function is set by the
values passed into `allowed_approval_policies`.
4. `GenericDisplayRow`: add a `disabled_reason: Option<String>`. When
set, it disables selection of the value and indicates as such in the
menu. This also makes it unselectable with arrow keys or numbers. This
is used in the `/approvals` menu.
Follow ups are:
1. Do the same thing to `sandbox_policy`.
2. Propagate the allowed set of values through app-server for the
extension (though already this should prevent app-server from setting
this values, it's just that we want to disable UI elements that are
unsettable).
Happy to split this PR up if you prefer, into the logical numbered areas
above. Especially if there are parts we want to gavel on separately
(e.g. admin_policy).
Disabled full access:
<img width="1680" height="380" alt="image"
src="https://github.com/user-attachments/assets/1fb61c8c-1fcb-4dc4-8355-2293edb52ba0"
/>
Disabled `--yolo` on startup:
<img width="749" height="76" alt="image"
src="https://github.com/user-attachments/assets/0a1211a0-6eb1-40d6-a1d7-439c41e94ddb"
/>
CODEX-4087
This attempts to tighten up the types related to "config layers."
Currently, `ConfigLayerEntry` is defined as follows:
bef36f4ae7/codex-rs/core/src/config_loader/state.rs (L19-L25)
but the `source` field is a bit of a lie, as:
- for `ConfigLayerName::Mdm`, it is
`"com.openai.codex/config_toml_base64"`
- for `ConfigLayerName::SessionFlags`, it is `"--config"`
- for `ConfigLayerName::User`, it is `"config.toml"` (just the file
name, not the path to the `config.toml` on disk that was read)
- for `ConfigLayerName::System`, it seems like it is usually
`/etc/codex/managed_config.toml` in practice, though on Windows, it is
`%CODEX_HOME%/managed_config.toml`:
bef36f4ae7/codex-rs/core/src/config_loader/layer_io.rs (L84-L101)
All that is to say, in three out of the four `ConfigLayerName`, `source`
is a `PathBuf` that is not an absolute path (or even a true path).
This PR tries to uplevel things by eliminating `source` from
`ConfigLayerEntry` and turning `ConfigLayerName` into a disjoint union
named `ConfigLayerSource` that has the appropriate metadata for each
variant, favoring the use of `AbsolutePathBuf` where appropriate:
```rust
pub enum ConfigLayerSource {
/// Managed preferences layer delivered by MDM (macOS only).
#[serde(rename_all = "camelCase")]
#[ts(rename_all = "camelCase")]
Mdm { domain: String, key: String },
/// Managed config layer from a file (usually `managed_config.toml`).
#[serde(rename_all = "camelCase")]
#[ts(rename_all = "camelCase")]
System { file: AbsolutePathBuf },
/// Session-layer overrides supplied via `-c`/`--config`.
SessionFlags,
/// User config layer from a file (usually `config.toml`).
#[serde(rename_all = "camelCase")]
#[ts(rename_all = "camelCase")]
User { file: AbsolutePathBuf },
}
```
This PR makes sure that inline comment is preserved for mcp server
config and arbitrary key/value setPath config.
---------
Co-authored-by: celia-oai <celia@openai.com>
Changes the `writable_roots` field of the `WorkspaceWrite` variant of
the `SandboxPolicy` enum from `Vec<PathBuf>` to `Vec<AbsolutePathBuf>`.
This is helpful because now callers can be sure the value is an absolute
path rather than a relative one. (Though when using an absolute path in
a Seatbelt config policy, we still have to _canonicalize_ it first.)
Because `writable_roots` can be read from a config file, it is important
that we are able to resolve relative paths properly using the parent
folder of the config file as the base path.
# External (non-OpenAI) Pull Request Requirements
Before opening this Pull Request, please read the dedicated
"Contributing" markdown file or your PR may be closed:
https://github.com/openai/codex/blob/main/docs/contributing.md
If your PR conforms to our contribution guidelines, replace this text
with a detailed and high quality description of your changes.
Include a link to a bug report or enhancement request.
- Make Config.model optional and centralize default-selection logic in
ModelsManager, including a default_model helper (with
codex-auto-balanced when available) so sessions now carry an explicit
chosen model separate from the base config.
- Resolve `model` once in `core` and `tui` from config. Then store the
state of it on other structs.
- Move refreshing models to be before resolving the default model
Make sure that config writes preserve comments and order of configs by
utilizing the ConfigEditsBuilder in core.
Tested by running a real example and made sure that nothing in the
config file changes other than the configs to edit.
This PR attempts to solve two problems by introducing a
`AbsolutePathBuf` type with a special deserializer:
- `AbsolutePathBuf` attempts to be a generally useful abstraction, as it
ensures, by constructing, that it represents a value that is an
absolute, normalized path, which is a stronger guarantee than an
arbitrary `PathBuf`.
- Values in `config.toml` that can be either an absolute or relative
path should be resolved against the folder containing the `config.toml`
in the relative path case. This PR makes this easy to support: the main
cost is ensuring `AbsolutePathBufGuard` is used inside
`deserialize_config_toml_with_base()`.
While `AbsolutePathBufGuard` may seem slightly distasteful because it
relies on thread-local storage, this seems much cleaner to me than using
than my various experiments with
https://docs.rs/serde/latest/serde/de/trait.DeserializeSeed.html.
Further, since the `deserialize()` method from the `Deserialize` trait
is not async, we do not really have to worry about the deserialization
work being spread across multiple threads in a way that would interfere
with `AbsolutePathBufGuard`.
To start, this PR introduces the use of `AbsolutePathBuf` in
`OtelTlsConfig`. Note how this simplifies `otel_provider.rs` because it
no longer requires `settings.codex_home` to be threaded through.
Furthermore, this sets us up better for a world where multiple
`config.toml` files from different folders could be loaded and then
merged together, as the absolutifying of the paths must be done against
the correct parent folder.
This is a step towards removing the need to know `model` when
constructing config. We firstly don't need to know `model_info` and just
respect if the user has already set it. Next step, we don't need to know
`model` unless the user explicitly set it in `config.toml`
This PR moves `ModelsFamily` to `openai_models`. It also propagates
`ModelsManager` to session services and use it to drive model family. We
also make `derive_default_model_family` private because it's a step
towards what we want: one place that gives model configuration.
This is a second step at having one source of truth for models
information and config: `ModelsManager`.
Next steps would be to remove `ModelsFamily` from config. That's massive
because it's being used in 41 occasions mostly pre launching `codex`.
Also, we need to make `find_family_for_model` private. It's also big
because it's being used in 21 occasions ~ all tests.
- Introduce `openai_models` in `/core`
- Move `PRESETS` under it
- Move `ModelPreset`, `ModelUpgrade`, `ReasoningEffortPreset`,
`ReasoningEffortPreset`, and `ReasoningEffortPreset` to `protocol`
- Introduce `Op::ListModels` and `EventMsg::AvailableModels`
Next steps:
- migrate `app-server` and `tui` to use the introduced Operation
This PR honors the `history.max_bytes` configuration parameter by
trimming `history.jsonl` whenever it grows past the configured limit.
While appending new entries we retain the newest record, drop the oldest
lines to stay within the byte budget, and serialize the compacted file
back to disk under the same lock to keep writers safe.
