mirror of
https://github.com/openai/codex.git
synced 2026-05-16 01:02:48 +00:00
xli-codex/generate-python-sdk
24 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Channing Conger
|
36460387ec |
Enable V8 sandboxing for source-built builds (#21146)
## Summary
This is the first PR in the V8 in-process sandboxing rollout.
It adds the build-system and Rust feature plumbing needed to support
sandboxed V8 builds, then enables sandboxing by default for the
source-built Bazel V8 path that we control directly. It deliberately
keeps the published `rusty_v8` artifact workflows on their current
non-sandboxed contract so this PR can land and ship independently before
we change any released artifacts.
## Rollout plan
- [x] **PR 1: land sandbox plumbing and default source-built Bazel V8 to
sandboxed mode**
- [ ] **PR 2: publish sandbox-enabled release artifacts and add
compatibility validation**
- Produce sandboxed artifact pairs for every released Cargo target that
does not already use the source-built Bazel path.
- Add CI coverage that consumes those sandboxed artifacts and verifies:
- `codex-v8-poc` reports sandbox enabled
- `codex-code-mode` builds/tests against the sandboxed path
- [ ] **PR 3: switch release consumers to sandboxed artifacts by
default**
- Update released artifact selectors/checksums.
- Enable the Rust `v8_enable_sandbox` feature in the default release
path.
- Make the sandboxed artifact family the normal path for published
builds.
- [ ] **PR 4: remove rollout-only compatibility paths**
- Remove the temporary non-sandbox release compatibility config once the
new default has shipped and baked.
- Keep the invariant tests permanently.
|
||
|
Michael Bolin
|
30de54da36 |
bazel: run sharded rust integration tests (#21057)
## Why Bazel CI was not actually exercising some sharded Rust integration-test targets on macOS. The `rules_rust` sharding wrapper expects a symlink runfiles tree, but this repo runs Bazel with `--noenable_runfiles`. In that configuration the wrapper could fail to find the generated test binary, produce an empty test list, and exit successfully. That made targets such as `//codex-rs/core:core-all-test` look green even when Cargo CI could still catch failures in the same Rust tests. The coverage gap appears to have been introduced by [#18082](https://github.com/openai/codex/pull/18082), which enabled rules_rust native sharding on `//codex-rs/core:core-all-test` and the other large Rust test labels. The manifest-runfiles setup itself predates that change in [#10098](https://github.com/openai/codex/pull/10098), but #18082 is where the affected integration tests started running through the incompatible rules_rust sharding wrapper. [#18913](https://github.com/openai/codex/pull/18913) fixed the same class of issue for wrapped unit-test shards, but integration-test shards were still going through the rules_rust wrapper until this PR. We still do not have the V8/code-mode pieces stable under the Bazel CI cross-compile setup, so this keeps those tests out of Bazel while restoring coverage for the rest of the sharded Rust integration suites. Cargo CI remains responsible for V8/code-mode coverage for now. This change did uncover a real failing core test on `main`: `approved_folder_write_request_permissions_unblocks_later_apply_patch`. That fix is split into [#21060](https://github.com/openai/codex/pull/21060), which enables the `apply_patch` tool in the test, teaches the aggregate core test binary to dispatch the sandboxed filesystem helper, canonicalizes the macOS temp patch target, and isolates the core test harness from managed local/enterprise config. Keeping that fix separate lets this PR stay focused on restoring Bazel coverage while documenting the first failure it exposed. ## What changed - Build sharded Rust integration tests as manual `*-bin` binaries and run them through the existing manifest-aware `workspace_root_test` launcher. - Keep Bazel sharding on the launcher target so Rust test cases are still distributed by stable test-name hashing. - Configure Bazel CI to skip Rust tests whose names contain `suite::code_mode::`. - Exclude the standalone `codex-rs/code-mode` and `codex-rs/v8-poc` unit-test targets from `bazel.yml`. ## Verification - `bazel query --output=build //codex-rs/core:core-all-test` now shows `workspace_root_test` wrapping `//codex-rs/core:core-all-test-bin`. - `bazel test --test_output=all --nocache_test_results --test_sharding_strategy=disabled //codex-rs/core:core-all-test --test_filter=suite::request_permissions_tool::approved_folder_write_request_permissions_unblocks_later_apply_patch` runs the actual Rust test body and passes. - `bazel test --test_output=errors --nocache_test_results --test_env=CODEX_BAZEL_TEST_SKIP_FILTERS=suite::code_mode:: //codex-rs/core:core-all-test` runs the sharded target with code-mode skipped and passes overall locally, with one flaky attempt retried by the existing `flaky = True` setting. |
||
|
Michael Bolin
|
466798aa83 |
ci: cross-compile Windows Bazel tests (#20585)
## Status This is the Bazel PR-CI cross-compilation follow-up to #20485. It is intentionally split from the Cargo/cargo-xwin release-build PoC so #20485 can stay as the historical release-build exploration. The unrelated async-utils test cleanup has been moved to #20686, so this PR is focused on the Windows Bazel CI path. The intended tradeoff is now explicit in `.github/workflows/bazel.yml`: pull requests get the fast Windows cross-compiled Bazel test leg, while post-merge pushes to `main` run both that fast cross leg and a fully native Windows Bazel test leg. The native main-only job keeps full V8/code-mode coverage and gets a 40-minute timeout because it is less latency-sensitive than PR CI. All other Bazel jobs remain at 30 minutes. ## Why Windows Bazel PR CI currently does the expensive part of the build on Windows. A native Windows Bazel test job on `main` completed in about 28m12s, leaving very little headroom under the 30-minute job timeout and making Windows the slowest PR signal. #20485 showed that Windows cross-compilation can be materially faster for Cargo release builds, but PR CI needs Bazel because Bazel owns our test sharding, flaky-test retries, and integration-test layout. This PR applies the same high-level shape we already use for macOS Bazel CI: compile with remote Linux execution, then run platform-specific tests on the platform runner. The compromise is deliberately signal-aware: code-mode/V8 changes are rare enough that PR CI can accept losing the direct V8/code-mode smoke-test signal temporarily, while `main` still runs the native Windows job post-merge to catch that class of regression. A follow-up PR should investigate making the cross-built Windows gnullvm V8 archive pass the direct V8/code-mode tests so this tradeoff can eventually go away. ## What Changed - Adds a `ci-windows-cross` Bazel config that targets `x86_64-pc-windows-gnullvm`, uses Linux RBE for build actions, and keeps `TestRunner` actions local on the Windows runner. - Adds explicit Windows platform definitions for `windows_x86_64_gnullvm`, `windows_x86_64_msvc`, and a bridge toolchain that lets gnullvm test targets execute under the Windows MSVC host platform. - Updates the Windows Bazel PR test leg to opt into the cross-compile path via `--windows-cross-compile` and `--remote-download-toplevel`. - Adds a `test-windows-native-main` job that runs only for `push` events on `refs/heads/main`, uses the native Windows Bazel path, includes V8/code-mode smoke tests, and has `timeout-minutes: 40`. - Keeps fork/community PRs without `BUILDBUDDY_API_KEY` on the previous local Windows MSVC-host fallback, including `--host_platform=//:local_windows_msvc` and `--jobs=8`. - Preserves the existing integration-test shape on non-gnullvm platforms, while generating Windows-cross wrapper targets only for `windows_gnullvm`. - Resolves `CARGO_BIN_EXE_*` values from runfiles at test runtime, avoiding hard-coded Cargo paths and duplicate test runfiles. - Extends the V8 Bazel patches enough for the `x86_64-pc-windows-gnullvm` target and Linux remote execution path. - Makes the Windows sandbox test cwd derive from `INSTA_WORKSPACE_ROOT` at runtime when Bazel provides it, because cross-compiled binaries may contain Linux compile-time paths. - Keeps the direct V8/code-mode unit smoke tests out of the Windows cross PR path for now while native Windows CI continues to cover them post-merge. ## Command Shape The fast Windows PR test leg invokes the normal Bazel CI wrapper like this: ```shell ./.github/scripts/run-bazel-ci.sh \ --print-failed-action-summary \ --print-failed-test-logs \ --windows-cross-compile \ --remote-download-toplevel \ -- \ test \ --test_tag_filters=-argument-comment-lint \ --test_verbose_timeout_warnings \ --build_metadata=COMMIT_SHA=${GITHUB_SHA} \ -- \ //... \ -//third_party/v8:all \ -//codex-rs/code-mode:code-mode-unit-tests \ -//codex-rs/v8-poc:v8-poc-unit-tests ``` With the BuildBuddy secret available on Windows, the wrapper selects `--config=ci-windows-cross` and appends the important Windows-cross overrides after rc expansion: ```shell --host_platform=//:rbe --shell_executable=/bin/bash --action_env=PATH=/usr/bin:/bin --host_action_env=PATH=/usr/bin:/bin --test_env=PATH=${CODEX_BAZEL_WINDOWS_PATH} ``` The native post-merge Windows job intentionally omits `--windows-cross-compile` and does not exclude the V8/code-mode unit targets: ```shell ./.github/scripts/run-bazel-ci.sh \ --print-failed-action-summary \ --print-failed-test-logs \ -- \ test \ --test_tag_filters=-argument-comment-lint \ --test_verbose_timeout_warnings \ --build_metadata=COMMIT_SHA=${GITHUB_SHA} \ --build_metadata=TAG_windows_native_main=true \ -- \ //... \ -//third_party/v8:all ``` ## Research Notes The existing macOS Bazel CI config already uses the model we want here: build actions run remotely with `--strategy=remote`, but `TestRunner` actions execute on the macOS runner. This PR mirrors that pattern for Windows with `--strategy=TestRunner=local`. The important Bazel detail is that `rules_rs` is already targeting `x86_64-pc-windows-gnullvm` for Windows Bazel PR tests. This PR changes where the build actions execute; it does not switch the Bazel PR test target to Cargo, `cargo-nextest`, or the MSVC release target. Cargo release builds differ from this Bazel path for V8: the normal Windows Cargo release target is MSVC, and `rusty_v8` publishes prebuilt Windows MSVC `.lib.gz` archives. The Bazel PR path targets `windows-gnullvm`; `rusty_v8` does not publish a prebuilt Windows GNU/gnullvm archive, so this PR builds that archive in-tree. That Linux-RBE-built gnullvm archive currently crashes in direct V8/code-mode smoke tests, which is why the workflow keeps native Windows coverage on `main`. The less obvious Bazel detail is test wrapper selection. Bazel chooses the Windows test wrapper (`tw.exe`) from the test action execution platform, not merely from the Rust target triple. The outer `workspace_root_test` therefore declares the default test toolchain and uses the bridge toolchain above so the test action executes on Windows while its inner Rust binary is built for gnullvm. The V8 investigation exposed a Windows-client gotcha: even when an action execution platform is Linux RBE, Bazel can still derive the genrule shell path from the Windows client. That produced remote commands trying to run `C:\Program Files\Git\usr\bin\bash.exe` on Linux workers. The wrapper now passes `--shell_executable=/bin/bash` with `--host_platform=//:rbe` for the Windows cross path. The same Windows-client/Linux-RBE boundary also affected `third_party/v8:binding_cc`: a multiline genrule command can carry CRLF line endings into Linux remote bash, which failed as `$'\r'`. That genrule now keeps the `sed` command on one physical shell line while using an explicit Starlark join so the shell arguments stay readable. ## Verification Local checks included: ```shell bash -n .github/scripts/run-bazel-ci.sh bash -n workspace_root_test_launcher.sh.tpl ruby -e "require %q{yaml}; YAML.load_file(%q{.github/workflows/bazel.yml}); puts %q{ok}" RUNNER_OS=Linux ./scripts/list-bazel-clippy-targets.sh RUNNER_OS=Windows ./scripts/list-bazel-clippy-targets.sh RUNNER_OS=Linux ./tools/argument-comment-lint/list-bazel-targets.sh RUNNER_OS=Windows ./tools/argument-comment-lint/list-bazel-targets.sh ``` The Linux clippy and argument-comment target lists contain zero `*-windows-cross-bin` labels, while the Windows lists still include 47 Windows-cross internal test binaries. CI evidence: - Baseline native Windows Bazel test on `main`: success in about 28m12s, https://github.com/openai/codex/actions/runs/25206257208/job/73907325959 - Green Windows-cross Bazel run on the split PR before adding the main-only native leg: Windows test 9m16s, Windows release verify 5m10s, Windows clippy 4m43s, https://github.com/openai/codex/actions/runs/25231890068 - The latest SHA adds the explicit PR-vs-main tradeoff in `bazel.yml`; CI is rerunning on that focused diff. ## Follow-Up A subsequent PR should investigate making a cross-built Windows binary work with V8/code-mode enabled. Likely options are either making the Linux-RBE-built `windows-gnullvm` V8 archive correct at runtime, or evaluating whether a Bazel MSVC target/toolchain can reuse the same prebuilt MSVC `rusty_v8` archive shape that Cargo release builds already use. |
||
|
Michael Bolin
|
e8ba912fcc |
test: set Rust test thread stack size (#19067)
## Summary Set `RUST_MIN_STACK=8388608` for Rust test entry points so libtest-spawned test threads get an 8 MiB stack. The Windows BuildBuddy failure on #18893 showed `//codex-rs/tui:tui-unit-tests` exiting with a stack overflow in a `#[tokio::test]` even though later test binaries in the shard printed successful summaries. Default `#[tokio::test]` uses a current-thread Tokio runtime, which means the async test body is driven on libtest's std-spawned test thread. Increasing the test thread stack addresses that failure mode directly. To date, we have been fixing these stack-pressure problems with localized future-size reductions, such as #13429, and by adding `Box::pin()` in specific async wrapper chains. This gives us a baseline test-runner stack size instead of continuing to patch individual tests only after CI finds another large async future. ## What changed - Added `common --test_env=RUST_MIN_STACK=8388608` in `.bazelrc` so Bazel test actions receive the env var through Bazel's cache-keyed test environment path. - Set the same `RUST_MIN_STACK` value for Cargo/nextest CI entry points and `just test`. - Annotated the existing Windows Bazel linker stack reserve as 8 MiB so it stays aligned with the libtest thread stack size. ## Testing - `just --list` - parsed `.github/workflows/rust-ci.yml` and `.github/workflows/rust-ci-full.yml` with Ruby's YAML loader - compared `bazel aquery` `TestRunner` action keys before/after explicit `--test_env=RUST_MIN_STACK=...` and after moving the Bazel env to `.bazelrc` - `bazel test //codex-rs/tui:tui-unit-tests --test_output=errors` - failed locally on the existing sandbox-specific status snapshot permission mismatch, but loaded the Starlark changes and ran the TUI test shards |
||
|
Michael Bolin
|
536952eeee |
bazel: run wrapped Rust unit test shards (#18913)
## Why The `codex-tui` Cargo test suite was catching stale snapshot expectations, but the matching Bazel unit-test target was still green. The TUI unit target is wrapped by `workspace_root_test` so tests run from the repository root and Insta can resolve Cargo-like snapshot paths. After native Bazel sharding was enabled for that wrapped target, rules_rust also inserted its own sharding wrapper around the Rust test binary. Those two wrappers did not compose: rules_rust's sharding wrapper expects to run from its own runfiles cwd, while `workspace_root_test` deliberately changes cwd to the repo root before invoking the test. In that configuration, the inner wrapper could fail to enumerate the Rust tests and exit successfully with empty shards, so snapshot regressions were not being exercised by Bazel. ## What Changed - Stop enabling rules_rust's inner `experimental_enable_sharding` for unit-test binaries created by `codex_rust_crate`. - Keep the configured `shard_count` on the outer `workspace_root_test` target. - Add libtest sharding directly to `workspace_root_test_launcher.sh.tpl` and `workspace_root_test_launcher.bat.tpl` after the launcher has resolved the actual test binary and established the intended repository-root cwd. - Partition tests by a stable FNV-1a hash of each libtest test name, matching the stable-shard behavior we wanted without depending on the inner rules_rust wrapper. - Preserve ad-hoc local test filters by running the resolved test binary directly when explicit test args are supplied. - On Windows, run selected libtest names from the shard list in bounded PowerShell batches instead of concatenating every selected test into one `cmd.exe` command line. This PR is stacked on top of #18912, which contains only the snapshot expectation updates exposed once the Bazel target actually runs the TUI unit tests. It is also the reason #18916 becomes visible: once this wrapper fix makes Bazel execute the affected `codex-core` test, that test needs its own executable-path setup fixed. ## Verification - `cargo test -p codex-tui` - `bazel test //codex-rs/tui:tui-unit-tests --test_output=errors` - `bazel test //codex-rs/tui:all --test_output=errors` - `bash -n workspace_root_test_launcher.sh.tpl` - Exercised the Windows PowerShell batching fragment locally with a fake test binary and shard-list file. |
||
|
Michael Bolin
|
96d35dd640 |
bazel: use native rust test sharding (#18082)
## Why The large Rust test suites are slow and include some of our flakiest tests, so we want to run them with Bazel native sharding while keeping shard membership stable between runs. This is the simpler follow-up to the explicit-label experiment in #17998. Since #18397 upgraded Codex to `rules_rs` `0.0.58`, which includes the stable test-name hashing support from hermeticbuild/rules_rust#14, this PR only needs to wire Codex's Bazel macros into that support. Using native sharding preserves BuildBuddy's sharded-test UI and Bazel's per-shard test action caching. Using stable name hashing avoids reshuffling every test when one test is added or removed. ## What Changed `codex_rust_crate` now accepts `test_shard_counts` and applies the right Bazel/rules_rust attributes to generated unit and integration test rules. Matched tests are also marked `flaky = True`, giving them Bazel's default three attempts. This PR shards these labels 8 ways: ```text //codex-rs/core:core-all-test //codex-rs/core:core-unit-tests //codex-rs/app-server:app-server-all-test //codex-rs/app-server:app-server-unit-tests //codex-rs/tui:tui-unit-tests ``` ## Verification `bazel query --output=build` over the selected public labels and their inner unit-test binaries confirmed the expected `shard_count = 8`, `flaky = True`, and `experimental_enable_sharding = True` attributes. Also verified that we see the shards as expected in BuildBuddy so they can be analyzed independently. Co-authored-by: Codex <noreply@openai.com> |
||
|
evawong-oai
|
17d94bd1e3 |
[docs] Revert extra changes from PR 17848 (#18003)
## Summary 1. Revert https://github.com/openai/codex/pull/17848 so the Bazel and `BUILD` file changes leave `main`. 2. Prepare for a narrower follow up that restores only `SECURITY.md`. ## Validation 1. Reviewed the revert diff against `main`. 2. Ran a clean diff check before push. |
||
|
evawong-oai
|
0bb438bca6 |
[docs] Add security boundaries reference in SECURITY.md (#17848)
## Summary 1. Add a Security Boundaries section to `SECURITY.md`. 2. Point readers to the Codex Agent approvals and security documentation for sandboxing, approvals, and network controls. ## Validation 1. Reviewed the `SECURITY.md` diff in a clean worktree. 2. No tests run. Docs only change. |
||
|
Ahmed Ibrahim
|
19bd018300 |
Wire realtime WebRTC native media into Bazel (#17145)
- Builds codex-realtime-webrtc through the normal Bazel Rust macro so native macOS WebRTC sources are included.\n- Shares the macOS -ObjC link flag with Bazel targets that can link libwebrtc. --------- Co-authored-by: Codex <noreply@openai.com> |
||
|
Michael Bolin
|
eaf12beacf |
Codex/windows bazel rust test coverage no rs (#16528)
# Why this PR exists This PR is trying to fix a coverage gap in the Windows Bazel Rust test lane. Before this change, the Windows `bazel test //...` job was nominally part of PR CI, but a non-trivial set of `//codex-rs/...` Rust test targets did not actually contribute test signal on Windows. In particular, targets such as `//codex-rs/core:core-unit-tests`, `//codex-rs/core:core-all-test`, and `//codex-rs/login:login-unit-tests` were incompatible during Bazel analysis on the Windows gnullvm platform, so they never reached test execution there. That is why the Cargo-powered Windows CI job could surface Windows-only failures that the Bazel-powered job did not report: Cargo was executing those tests, while Bazel was silently dropping them from the runnable target set. The main goal of this PR is to make the Windows Bazel test lane execute those Rust test targets instead of skipping them during analysis, while still preserving `windows-gnullvm` as the target configuration for the code under test. In other words: use an MSVC host/exec toolchain where Bazel helper binaries and build scripts need it, but continue compiling the actual crate targets with the Windows gnullvm cfgs that our current Bazel matrix is supposed to exercise. # Important scope note This branch intentionally removes the non-resource-loading `.rs` test and production-code changes from the earlier `codex/windows-bazel-rust-test-coverage` branch. The only Rust source changes kept here are runfiles/resource-loading fixes in TUI tests: - `codex-rs/tui/src/chatwidget/tests.rs` - `codex-rs/tui/tests/manager_dependency_regression.rs` That is deliberate. Since the corresponding tests already pass under Cargo, this PR is meant to test whether Bazel infrastructure/toolchain fixes alone are enough to get a healthy Windows Bazel test signal, without changing test behavior for Windows timing, shell output, or SQLite file-locking. # How this PR changes the Windows Bazel setup ## 1. Split Windows host/exec and target concerns in the Bazel test lane The core change is that the Windows Bazel test job now opts into an MSVC host platform for Bazel execution-time tools, but only for `bazel test`, not for the Bazel clippy build. Files: - `.github/workflows/bazel.yml` - `.github/scripts/run-bazel-ci.sh` - `MODULE.bazel` What changed: - `run-bazel-ci.sh` now accepts `--windows-msvc-host-platform`. - When that flag is present on Windows, the wrapper appends `--host_platform=//:local_windows_msvc` unless the caller already provided an explicit `--host_platform`. - `bazel.yml` passes that wrapper flag only for the Windows `bazel test //...` job. - The Bazel clippy job intentionally does **not** pass that flag, so clippy stays on the default Windows gnullvm host/exec path and continues linting against the target cfgs we care about. - `run-bazel-ci.sh` also now forwards `CODEX_JS_REPL_NODE_PATH` on Windows and normalizes the `node` executable path with `cygpath -w`, so tests that need Node resolve the runner's Node installation correctly under the Windows Bazel test environment. Why this helps: - The original incompatibility chain was mostly on the **exec/tool** side of the graph, not in the Rust test code itself. Moving host tools to MSVC lets Bazel resolve helper binaries and generators that were not viable on the gnullvm exec platform. - Keeping the target platform on gnullvm preserves cfg coverage for the crates under test, which is important because some Windows behavior differs between `msvc` and `gnullvm`. ## 2. Teach the repo's Bazel Rust macro about Windows link flags and integration-test knobs Files: - `defs.bzl` - `codex-rs/core/BUILD.bazel` - `codex-rs/otel/BUILD.bazel` - `codex-rs/tui/BUILD.bazel` What changed: - Replaced the old gnullvm-only linker flag block with `WINDOWS_RUSTC_LINK_FLAGS`, which now handles both Windows ABIs: - gnullvm gets `-C link-arg=-Wl,--stack,8388608` - MSVC gets `-C link-arg=/STACK:8388608`, `-C link-arg=/NODEFAULTLIB:libucrt.lib`, and `-C link-arg=ucrt.lib` - Threaded those Windows link flags into generated `rust_binary`, unit-test binaries, and integration-test binaries. - Extended `codex_rust_crate(...)` with: - `integration_test_args` - `integration_test_timeout` - Used those new knobs to: - mark `//codex-rs/core:core-all-test` as a long-running integration test - serialize `//codex-rs/otel:otel-all-test` with `--test-threads=1` - Added `src/**/*.rs` to `codex-rs/tui` test runfiles, because one regression test scans source files at runtime and Bazel does not expose source-tree directories unless they are declared as data. Why this helps: - Once host-side MSVC tools are available, we still need the generated Rust test binaries to link correctly on Windows. The MSVC-side stack/UCRT flags make those binaries behave more like their Cargo-built equivalents. - The integration-test macro knobs avoid hardcoding one-off test behavior in ad hoc BUILD rules and make the generated test targets more expressive where Bazel and Cargo have different runtime defaults. ## 3. Patch `rules_rs` / `rules_rust` so Windows MSVC exec-side Rust and build scripts are actually usable Files: - `MODULE.bazel` - `patches/rules_rs_windows_exec_linker.patch` - `patches/rules_rust_windows_bootstrap_process_wrapper_linker.patch` - `patches/rules_rust_windows_build_script_runner_paths.patch` - `patches/rules_rust_windows_exec_msvc_build_script_env.patch` - `patches/rules_rust_windows_msvc_direct_link_args.patch` - `patches/rules_rust_windows_process_wrapper_skip_temp_outputs.patch` - `patches/BUILD.bazel` What these patches do: - `rules_rs_windows_exec_linker.patch` - Adds a `rust-lld` filegroup for Windows Rust toolchain repos, symlinked to `lld-link.exe` from `PATH`. - Marks Windows toolchains as using a direct linker driver. - Supplies Windows stdlib link flags for both gnullvm and MSVC. - `rules_rust_windows_bootstrap_process_wrapper_linker.patch` - For Windows MSVC Rust targets, prefers the Rust toolchain linker over an inherited C++ linker path like `clang++`. - This specifically avoids the broken mixed-mode command line where rustc emits MSVC-style `/NOLOGO` / `/LIBPATH:` / `/OUT:` arguments but Bazel still invokes `clang++.exe`. - `rules_rust_windows_build_script_runner_paths.patch` - Normalizes forward-slash execroot-relative paths into Windows path separators before joining them on Windows. - Uses short Windows paths for `RUSTC`, `OUT_DIR`, and the build-script working directory to avoid path-length and quoting issues in third-party build scripts. - Exposes `RULES_RUST_BAZEL_BUILD_SCRIPT_RUNNER=1` to build scripts so crate-local patches can detect "this is running under Bazel's build-script runner". - Fixes the Windows runfiles cleanup filter so generated files with retained suffixes are actually retained. - `rules_rust_windows_exec_msvc_build_script_env.patch` - For exec-side Windows MSVC build scripts, stops force-injecting Bazel's `CC`, `CXX`, `LD`, `CFLAGS`, and `CXXFLAGS` when that would send GNU-flavored tool paths/flags into MSVC-oriented Cargo build scripts. - Rewrites or strips GNU-only `--sysroot`, MinGW include/library paths, stack-protector, and `_FORTIFY_SOURCE` flags on the MSVC exec path. - The practical effect is that build scripts can fall back to the Visual Studio toolchain environment already exported by CI instead of crashing inside Bazel's hermetic `clang.exe` setup. - `rules_rust_windows_msvc_direct_link_args.patch` - When using a direct linker on Windows, stops forwarding GNU driver flags such as `-L...` and `--sysroot=...` that `lld-link.exe` does not understand. - Passes non-`.lib` native artifacts as explicit `-Clink-arg=<path>` entries when needed. - Filters C++ runtime libraries to `.lib` artifacts on the Windows direct-driver path. - `rules_rust_windows_process_wrapper_skip_temp_outputs.patch` - Excludes transient `*.tmp*` and `*.rcgu.o` files from process-wrapper dependency search-path consolidation, so unstable compiler outputs do not get treated as real link search-path inputs. Why this helps: - The host-platform split alone was not enough. Once Bazel started analyzing/running previously incompatible Rust tests on Windows, the next failures were in toolchain plumbing: - MSVC-targeted Rust tests were being linked through `clang++` with MSVC-style arguments. - Cargo build scripts running under Bazel's Windows MSVC exec platform were handed Unix/GNU-flavored path and flag shapes. - Some generated paths were too long or had path-separator forms that third-party Windows build scripts did not tolerate. - These patches make that mixed Bazel/Cargo/Rust/MSVC path workable enough for the test lane to actually build and run the affected crates. ## 4. Patch third-party crate build scripts that were not robust under Bazel's Windows MSVC build-script path Files: - `MODULE.bazel` - `patches/aws-lc-sys_windows_msvc_prebuilt_nasm.patch` - `patches/ring_windows_msvc_include_dirs.patch` - `patches/zstd-sys_windows_msvc_include_dirs.patch` What changed: - `aws-lc-sys` - Detects Bazel's Windows MSVC build-script runner via `RULES_RUST_BAZEL_BUILD_SCRIPT_RUNNER` or a `bazel-out` manifest-dir path. - Uses `clang-cl` for Bazel Windows MSVC builds when no explicit `CC`/`CXX` is set. - Allows prebuilt NASM on the Bazel Windows MSVC path even when `nasm` is not available directly in the runner environment. - Avoids canonicalizing `CARGO_MANIFEST_DIR` in the Bazel Windows MSVC case, because that path may point into Bazel output/runfiles state where preserving the given path is more reliable than forcing a local filesystem canonicalization. - `ring` - Under the Bazel Windows MSVC build-script runner, copies the pregenerated source tree into `OUT_DIR` and uses that as the generated-source root. - Adds include paths needed by MSVC compilation for Fiat/curve25519/P-256 generated headers. - Rewrites a few relative includes in C sources so the added include directories are sufficient. - `zstd-sys` - Adds MSVC-only include directories for `compress`, `decompress`, and feature-gated dictionary/legacy/seekable sources. - Skips `-fvisibility=hidden` on MSVC targets, where that GCC/Clang-style flag is not the right mechanism. Why this helps: - After the `rules_rust` plumbing started running build scripts on the Windows MSVC exec path, some third-party crates still failed for crate-local reasons: wrong compiler choice, missing include directories, build-script assumptions about manifest paths, or Unix-only C compiler flags. - These crate patches address those crate-local assumptions so the larger toolchain change can actually reach first-party Rust test execution. ## 5. Keep the only `.rs` test changes to Bazel/Cargo runfiles parity Files: - `codex-rs/tui/src/chatwidget/tests.rs` - `codex-rs/tui/tests/manager_dependency_regression.rs` What changed: - Instead of asking `find_resource!` for a directory runfile like `src/chatwidget/snapshots` or `src`, these tests now resolve one known file runfile first and then walk to its parent directory. Why this helps: - Bazel runfiles are more reliable for explicitly declared files than for source-tree directories that happen to exist in a Cargo checkout. - This keeps the tests working under both Cargo and Bazel without changing their actual assertions. # What we tried before landing on this shape, and why those attempts did not work ## Attempt 1: Force `--host_platform=//:local_windows_msvc` for all Windows Bazel jobs This did make the previously incompatible test targets show up during analysis, but it also pushed the Bazel clippy job and some unrelated build actions onto the MSVC exec path. Why that was bad: - Windows clippy started running third-party Cargo build scripts with Bazel's MSVC exec settings and crashed in crates such as `tree-sitter` and `libsqlite3-sys`. - That was a regression in a job that was previously giving useful gnullvm-targeted lint signal. What this PR does instead: - The wrapper flag is opt-in, and `bazel.yml` uses it only for the Windows `bazel test` lane. - The clippy lane stays on the default Windows gnullvm host/exec configuration. ## Attempt 2: Broaden the `rules_rust` linker override to all Windows Rust actions This fixed the MSVC test-lane failure where normal `rust_test` targets were linked through `clang++` with MSVC-style arguments, but it broke the default gnullvm path. Why that was bad: - `@@rules_rs++rules_rust+rules_rust//util/process_wrapper:process_wrapper` on the gnullvm exec platform started linking with `lld-link.exe` and then failed to resolve MinGW-style libraries such as `-lkernel32`, `-luser32`, and `-lmingw32`. What this PR does instead: - The linker override is restricted to Windows MSVC targets only. - The gnullvm path keeps its original linker behavior, while MSVC uses the direct Windows linker. ## Attempt 3: Keep everything on pure Windows gnullvm and patch the V8 / Python incompatibility chain instead This would have preserved a single Windows ABI everywhere, but it is a much larger project than this PR. Why that was not the practical first step: - The original incompatibility chain ran through exec-side generators and helper tools, not only through crate code. - `third_party/v8` is already special-cased on Windows gnullvm because `rusty_v8` only publishes Windows prebuilts under MSVC names. - Fixing that path likely means deeper changes in V8/rules_python/rules_rust toolchain resolution and generator execution, not just one local CI flag. What this PR does instead: - Keep gnullvm for the target cfgs we want to exercise. - Move only the Windows test lane's host/exec platform to MSVC, then patch the build-script/linker boundary enough for that split configuration to work. ## Attempt 4: Validate compatibility with `bazel test --nobuild ...` This turned out to be a misleading local validation command. Why: - `bazel test --nobuild ...` can successfully analyze targets and then still exit 1 with "Couldn't start the build. Unable to run tests" because there are no runnable test actions after `--nobuild`. Better local check: ```powershell bazel build --nobuild --keep_going --host_platform=//:local_windows_msvc //codex-rs/login:login-unit-tests //codex-rs/core:core-unit-tests //codex-rs/core:core-all-test ``` # Which patches probably deserve upstream follow-up My rough take is that the `rules_rs` / `rules_rust` patches are the highest-value upstream candidates, because they are fixing generic Windows host/exec + MSVC direct-linker behavior rather than Codex-specific test logic. Strong upstream candidates: - `patches/rules_rs_windows_exec_linker.patch` - `patches/rules_rust_windows_bootstrap_process_wrapper_linker.patch` - `patches/rules_rust_windows_build_script_runner_paths.patch` - `patches/rules_rust_windows_exec_msvc_build_script_env.patch` - `patches/rules_rust_windows_msvc_direct_link_args.patch` - `patches/rules_rust_windows_process_wrapper_skip_temp_outputs.patch` Why these seem upstreamable: - They address general-purpose problems in the Windows MSVC exec path: - missing direct-linker exposure for Rust toolchains - wrong linker selection when rustc emits MSVC-style args - Windows path normalization/short-path issues in the build-script runner - forwarding GNU-flavored CC/link flags into MSVC Cargo build scripts - unstable temp outputs polluting process-wrapper search-path state Potentially upstreamable crate patches, but likely with more care: - `patches/zstd-sys_windows_msvc_include_dirs.patch` - `patches/ring_windows_msvc_include_dirs.patch` - `patches/aws-lc-sys_windows_msvc_prebuilt_nasm.patch` Notes on those: - The `zstd-sys` and `ring` include-path fixes look fairly generic for MSVC/Bazel build-script environments and may be straightforward to propose upstream after we confirm CI stability. - The `aws-lc-sys` patch is useful, but it includes a Bazel-specific environment probe and CI-specific compiler fallback behavior. That probably needs a cleaner upstream-facing shape before sending it out, so upstream maintainers are not forced to adopt Codex's exact CI assumptions. Probably not worth upstreaming as-is: - The repo-local Starlark/test target changes in `defs.bzl`, `codex-rs/*/BUILD.bazel`, and `.github/scripts/run-bazel-ci.sh` are mostly Codex-specific policy and CI wiring, not generic rules changes. # Validation notes for reviewers On this branch, I ran the following local checks after dropping the non-resource-loading Rust edits: ```powershell cargo test -p codex-tui just --shell 'C:\Program Files\Git\bin\bash.exe' --shell-arg -lc -- fix -p codex-tui python .\tools\argument-comment-lint\run-prebuilt-linter.py -p codex-tui just --shell 'C:\Program Files\Git\bin\bash.exe' --shell-arg -lc fmt ``` One local caveat: - `just argument-comment-lint` still fails on this Windows machine for an unrelated Bazel toolchain-resolution issue in `//codex-rs/exec:exec-all-test`, so I used the direct prebuilt linter for `codex-tui` as the local fallback. # Expected reviewer takeaway If this PR goes green, the important conclusion is that the Windows Bazel test coverage gap was primarily a Bazel host/exec toolchain problem, not a need to make the Rust tests themselves Windows-specific. That would be a strong signal that the deleted non-resource-loading Rust test edits from the earlier branch should stay out, and that future work should focus on upstreaming the generic `rules_rs` / `rules_rust` Windows fixes and reducing the crate-local patch surface. |
||
|
Michael Bolin
|
fce0f76d57 |
build: migrate argument-comment-lint to a native Bazel aspect (#16106)
## Why `argument-comment-lint` had become a PR bottleneck because the repo-wide lane was still effectively running a `cargo dylint`-style flow across the workspace instead of reusing Bazel's Rust dependency graph. That kept the lint enforced, but it threw away the main benefit of moving this job under Bazel in the first place: metadata reuse and cacheable per-target analysis in the same shape as Clippy. This change moves the repo-wide lint onto a native Bazel Rust aspect so Linux and macOS can lint `codex-rs` without rebuilding the world crate-by-crate through the wrapper path. ## What Changed - add a nightly Rust toolchain with `rustc-dev` for Bazel and a dedicated crate-universe repo for `tools/argument-comment-lint` - add `tools/argument-comment-lint/driver.rs` and `tools/argument-comment-lint/lint_aspect.bzl` so Bazel can run the lint as a custom `rustc_driver` - switch repo-wide `just argument-comment-lint` and the Linux/macOS `rust-ci` lanes to `bazel build --config=argument-comment-lint //codex-rs/...` - keep the Python/DotSlash wrappers as the package-scoped fallback path and as the current Windows CI path - gate the Dylint entrypoint behind a `bazel_native` feature so the Bazel-native library avoids the `dylint_*` packaging stack - update the aspect runtime environment so the driver can locate `rustc_driver` correctly under remote execution - keep the dedicated `tools/argument-comment-lint` package tests and wrapper unit tests in CI so the source and packaged entrypoints remain covered ## Verification - `python3 -m unittest discover -s tools/argument-comment-lint -p 'test_*.py'` - `cargo test` in `tools/argument-comment-lint` - `bazel build //tools/argument-comment-lint:argument-comment-lint-driver --@rules_rust//rust/toolchain/channel=nightly` - `bazel build --config=argument-comment-lint //codex-rs/utils/path-utils:all` - `bazel build --config=argument-comment-lint //codex-rs/rollout:rollout` --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/16106). * #16120 * __->__ #16106 |
||
|
Michael Bolin
|
e39ddc61b1 |
bazel: add Windows gnullvm stack flags to unit test binaries (#16074)
## Summary Add the Windows gnullvm stack-reserve flags to the `*-unit-tests-bin` path in `codex_rust_crate()`. ## Why This is the narrow code fix behind the earlier review comment on [#16067](https://github.com/openai/codex/pull/16067). That comment was stale relative to the workflow-only PR it landed on, but it pointed at a real gap in `defs.bzl`. Today, `codex_rust_crate()` already appends `WINDOWS_GNULLVM_RUSTC_STACK_FLAGS` for: - `rust_binary()` targets - integration-test `rust_test()` targets But the unit-test binary path still omitted those flags. That meant the generated `*-unit-tests-bin` executables were not built the same way as the rest of the Windows gnullvm executables in the macro. ## What Changed - Added `WINDOWS_GNULLVM_RUSTC_STACK_FLAGS` to the `unit_test_binary` `rust_test()` rule in `defs.bzl` - Added a short comment explaining why unit-test binaries need the same stack-reserve treatment as binaries and integration tests on Windows gnullvm ## Testing - `bazel query '//codex-rs/core:*'` - `bazel query '//codex-rs/shell-command:*'` Those queries load packages that exercise `codex_rust_crate()`, including `*-unit-tests-bin` targets. The actual runtime effect is Windows-specific, so the real end-to-end confirmation still comes from Windows CI. |
||
|
Michael Bolin
|
343d1af3da |
bazel: enable the full Windows gnullvm CI path (#15952)
## Why This PR is the current, consolidated follow-up to the earlier Windows Bazel attempt in #11229. The goal is no longer just to get a tiny Windows smoke job limping along: it is to make the ordinary Bazel CI path usable on `windows-latest` for `x86_64-pc-windows-gnullvm`, with the same broad `//...` test shape that macOS and Linux already use. The earlier smoke-list version of this work was useful as a foothold, but it was not a good long-term landing point. Windows Bazel kept surfacing real issues outside that allowlist: - GitHub's Windows runner exposed runfiles-manifest bugs such as `FINDSTR: Cannot open D:MANIFEST`, which broke Bazel test launchers even when the manifest file existed. - `rules_rs`, `rules_rust`, LLVM extraction, and Abseil still needed `windows-gnullvm`-specific fixes for our hermetic toolchain. - the V8 path needed more work than just turning the Windows matrix entry back on: `rusty_v8` does not ship Windows GNU artifacts in the same shape we need, and Bazel's in-tree V8 build needed a set of Windows GNU portability fixes. Windows performance pressure also pushed this toward a full solution instead of a permanent smoke suite. During this investigation we hit targets such as `//codex-rs/shell-command:shell-command-unit-tests` that were much more expensive on Windows because they repeatedly spawn real PowerShell parsers (see #16057 for one concrete example of that pressure). That made it much more valuable to get the real Windows Bazel path working than to keep iterating on a narrowly curated subset. The net result is that this PR now aims for the same CI contract on Windows that we already expect elsewhere: keep standalone `//third_party/v8:all` out of the ordinary Bazel lane, but allow V8 consumers under `//codex-rs/...` to build and test transitively through `//...`. ## What Changed ### CI and workflow wiring - re-enable the `windows-latest` / `x86_64-pc-windows-gnullvm` Bazel matrix entry in `.github/workflows/bazel.yml` - move the Windows Bazel output root to `D:\b` and enable `git config --global core.longpaths true` in `.github/actions/setup-bazel-ci/action.yml` - keep the ordinary Bazel target set on Windows aligned with macOS and Linux by running `//...` while excluding only standalone `//third_party/v8:all` targets from the normal lane ### Toolchain and module support for `windows-gnullvm` - patch `rules_rs` so `windows-gnullvm` is modeled as a distinct Windows exec/toolchain platform instead of collapsing into the generic Windows shape - patch `rules_rust` build-script environment handling so llvm-mingw build-script probes do not inherit unsupported `-fstack-protector*` flags - patch the LLVM module archive so it extracts cleanly on Windows and provides the MinGW libraries this toolchain needs - patch Abseil so its thread-local identity path matches the hermetic `windows-gnullvm` toolchain instead of taking an incompatible MinGW pthread path - keep both MSVC and GNU Windows targets in the generated Cargo metadata because the current V8 release-asset story still uses MSVC-shaped names in some places while the Bazel build targets the GNU ABI ### Windows test-launch and binary-behavior fixes - update `workspace_root_test_launcher.bat.tpl` to read the runfiles manifest directly instead of shelling out to `findstr`, which was the source of the `D:MANIFEST` failures on the GitHub Windows runner - thread a larger Windows GNU stack reserve through `defs.bzl` so Bazel-built binaries that pull in V8 behave correctly both under normal builds and under `bazel test` - remove the no-longer-needed Windows bootstrap sh-toolchain override from `.bazelrc` ### V8 / `rusty_v8` Windows GNU support - export and apply the new Windows GNU patch set from `patches/BUILD.bazel` / `MODULE.bazel` - patch the V8 module/rules/source layers so the in-tree V8 build can produce Windows GNU archives under Bazel - teach `third_party/v8/BUILD.bazel` to build Windows GNU static archives in-tree instead of aliasing them to the MSVC prebuilts - reuse the Linux release binding for the experimental Windows GNU path where `rusty_v8` does not currently publish a Windows GNU binding artifact ## Testing - the primary end-to-end validation for this work is the `Bazel` workflow plus `v8-canary`, since the hard parts are Windows-specific and depend on real GitHub runner behavior - before consolidation back onto this PR, the same net change passed the full Bazel matrix in [run 23675590471](https://github.com/openai/codex/actions/runs/23675590471) and passed `v8-canary` in [run 23675590453](https://github.com/openai/codex/actions/runs/23675590453) - those successful runs included the `windows-latest` / `x86_64-pc-windows-gnullvm` Bazel job with the ordinary `//...` path, not the earlier Windows smoke allowlist --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/15952). * #16067 * __->__ #15952 |
||
|
Charley Cunningham
|
7ba1fccfc1 |
fix(ci): restore guardian coverage and bazel unit tests (#13912)
## Summary - restore the guardian review request snapshot test and its tracked snapshot after it was dropped from `main` - make Bazel Rust unit-test wrappers resolve runfiles correctly on manifest-only platforms like macOS and point Insta at the real workspace root - harden the shell-escalation socket-closure assertion so the musl Bazel test no longer depends on fd reuse behavior ## Verification - cargo test -p codex-core guardian_review_request_layout_matches_model_visible_request_snapshot - cargo test -p codex-shell-escalation - bazel test //codex-rs/exec:exec-unit-tests //codex-rs/shell-escalation:shell-escalation-unit-tests Supersedes #13894. --------- Co-authored-by: Ahmed Ibrahim <aibrahim@openai.com> Co-authored-by: viyatb-oai <viyatb@openai.com> Co-authored-by: Codex <noreply@openai.com> |
||
|
zbarsky-openai
|
2d8c1575b8 |
[bazel] Bump rules_rs and llvm (#13366)
# External (non-OpenAI) Pull Request Requirements Before opening this Pull Request, please read the dedicated "Contributing" markdown file or your PR may be closed: https://github.com/openai/codex/blob/main/docs/contributing.md If your PR conforms to our contribution guidelines, replace this text with a detailed and high quality description of your changes. Include a link to a bug report or enhancement request. |
||
|
zbarsky-openai
|
2668789560 |
[bazel] Fix proc_macro_dep libs (#12274)
If a first-party proc_macro crate has tests/binaries that would get autogenerated by the macro, it was being handled incorrectly. Found by an external OS contributor! |
||
|
Anton Panasenko
|
1d95656149 |
bazel: fix snapshot parity for tests/*.rs rust_test targets (#11893)
## Summary - make `rust_test` targets generated from `tests/*.rs` use Cargo-style crate names (file stem) so snapshot names match Cargo (`all__...` instead of Bazel-derived names) - split lib vs `tests/*.rs` test env wiring in `codex_rust_crate` to keep existing lib snapshot behavior while applying Bazel runfiles-compatible workspace root for `tests/*.rs` - compute the `tests/*.rs` snapshot workspace root from package depth so `insta` resolves committed snapshots under Bazel `--noenable_runfiles` ## Validation - `bazelisk test //codex-rs/core:core-all-test --test_arg=suite::compact:: --cache_test_results=no` - `bazelisk test //codex-rs/core:core-all-test --test_arg=suite::compact_remote:: --cache_test_results=no` |
||
|
viyatb-oai
|
923f931121 |
build(linux-sandbox): always compile vendored bubblewrap on Linux; remove CODEX_BWRAP_ENABLE_FFI (#11498)
## Summary This PR removes the temporary `CODEX_BWRAP_ENABLE_FFI` flag and makes Linux builds always compile vendored bubblewrap support for `codex-linux-sandbox`. ## Changes - Removed `CODEX_BWRAP_ENABLE_FFI` gating from `codex-rs/linux-sandbox/build.rs`. - Linux builds now fail fast if vendored bubblewrap compilation fails (instead of warning and continuing). - Updated fallback/help text in `codex-rs/linux-sandbox/src/vendored_bwrap.rs` to remove references to `CODEX_BWRAP_ENABLE_FFI`. - Removed `CODEX_BWRAP_ENABLE_FFI` env wiring from: - `.github/workflows/rust-ci.yml` - `.github/workflows/bazel.yml` - `.github/workflows/rust-release.yml` --------- Co-authored-by: David Zbarsky <zbarsky@openai.com> |
||
|
zbarsky-openai
|
86183847fd |
[bazel] Upgrade some rulesets in preparation for enabling windows, part 2 (#11197)
https://github.com/openai/codex/pull/11109 had automerge set, so I didn't get to address feedback before merging, oops! |
||
|
zbarsky-openai
|
44a1355133 | [bazel] Upgrade some rulesets in preparation for enabling windows (#11109) | ||
|
jif-oai
|
3cc9122ee2 |
feat: experimental flags (#10231)
## Problem being solved
- We need a single, reliable way to mark app-server API surface as
experimental so that:
1. the runtime can reject experimental usage unless the client opts in
2. generated TS/JSON schemas can exclude experimental methods/fields for
stable clients.
Right now that’s easy to drift or miss when done ad-hoc.
## How to declare experimental methods and fields
- **Experimental method**: add `#[experimental("method/name")]` to the
`ClientRequest` variant in `client_request_definitions!`.
- **Experimental field**: on the params struct, derive `ExperimentalApi`
and annotate the field with `#[experimental("method/name.field")]` + set
`inspect_params: true` for the method variant so
`ClientRequest::experimental_reason()` inspects params for experimental
fields.
## How the macro solves it
- The new derive macro lives in
`codex-rs/codex-experimental-api-macros/src/lib.rs` and is used via
`#[derive(ExperimentalApi)]` plus `#[experimental("reason")]`
attributes.
- **Structs**:
- Generates `ExperimentalApi::experimental_reason(&self)` that checks
only annotated fields.
- The “presence” check is type-aware:
- `Option<T>`: `is_some_and(...)` recursively checks inner.
- `Vec`/`HashMap`/`BTreeMap`: must be non-empty.
- `bool`: must be `true`.
- Other types: considered present (returns `true`).
- Registers each experimental field in an `inventory` with `(type_name,
serialized field name, reason)` and exposes `EXPERIMENTAL_FIELDS` for
that type. Field names are converted from `snake_case` to `camelCase`
for schema/TS filtering.
- **Enums**:
- Generates an exhaustive `match` returning `Some(reason)` for annotated
variants and `None` otherwise (no wildcard arm).
- **Wiring**:
- Runtime gating uses `ExperimentalApi::experimental_reason()` in
`codex-rs/app-server/src/message_processor.rs` to reject requests unless
`InitializeParams.capabilities.experimental_api == true`.
- Schema/TS export filters use the inventory list and
`EXPERIMENTAL_CLIENT_METHODS` from `client_request_definitions!` to
strip experimental methods/fields when `experimental_api` is false.
|
||
|
zbarsky-openai
|
8497163363 |
[bazel] Improve runfiles handling (#10098)
we can't use runfiles directory on Windows due to path lengths, so swap to manifest strategy. Parsing the manifest is a bit complex and the format is changing in Bazel upstream, so pull in the official Rust library (via a small hack to make it importable...) and cleanup all the associated logic to work cleanly in both bazel and cargo without extra confusion |
||
|
jcoens-openai
|
e04851816d |
Remove stale TODO comment from defs.bzl (#9787)
### Motivation - Remove an outdated comment in `defs.bzl` referencing `cargo_build_script` that is no longer relevant. ### Description - Delete the stale `# TODO(zbarsky): cargo_build_script support?` line so the logic flows directly from `binaries` to `lib_srcs` in `defs.bzl`. ### Testing - Ran `git diff --check` which produced no errors. ------ [Codex Task](https://chatgpt.com/codex/tasks/task_i_6973d9ac757c8331be475a8fb0f90a88) |
||
|
zbarsky-openai
|
2a06d64bc9 |
feat: add support for building with Bazel (#8875)
This PR configures Codex CLI so it can be built with [Bazel](https://bazel.build) in addition to Cargo. The `.bazelrc` includes configuration so that remote builds can be done using [BuildBuddy](https://www.buildbuddy.io). If you are familiar with Bazel, things should work as you expect, e.g., run `bazel test //... --keep-going` to run all the tests in the repo, but we have also added some new aliases in the `justfile` for convenience: - `just bazel-test` to run tests locally - `just bazel-remote-test` to run tests remotely (currently, the remote build is for x86_64 Linux regardless of your host platform). Note we are currently seeing the following test failures in the remote build, so we still need to figure out what is happening here: ``` failures: suite::compact::manual_compact_twice_preserves_latest_user_messages suite::compact_resume_fork::compact_resume_after_second_compaction_preserves_history suite::compact_resume_fork::compact_resume_and_fork_preserve_model_history_view ``` - `just build-for-release` to build release binaries for all platforms/architectures remotely To setup remote execution: - [Create a buildbuddy account](https://app.buildbuddy.io/) (OpenAI employees should also request org access at https://openai.buildbuddy.io/join/ with their `@openai.com` email address.) - [Copy your API key](https://app.buildbuddy.io/docs/setup/) to `~/.bazelrc` (add the line `build --remote_header=x-buildbuddy-api-key=YOUR_KEY`) - Use `--config=remote` in your `bazel` invocations (or add `common --config=remote` to your `~/.bazelrc`, or use the `just` commands) ## CI In terms of CI, this PR introduces `.github/workflows/bazel.yml`, which uses Bazel to run the tests _locally_ on Mac and Linux GitHub runners (we are working on supporting Windows, but that is not ready yet). Note that the failures we are seeing in `just bazel-remote-test` do not occur on these GitHub CI jobs, so everything in `.github/workflows/bazel.yml` is green right now. The `bazel.yml` uses extra config in `.github/workflows/ci.bazelrc` so that macOS CI jobs build _remotely_ on Linux hosts (using the `docker://docker.io/mbolin491/codex-bazel` Docker image declared in the root `BUILD.bazel`) using cross-compilation to build the macOS artifacts. Then these artifacts are downloaded locally to GitHub's macOS runner so the tests can be executed natively. This is the relevant config that enables this: ``` common:macos --config=remote common:macos --strategy=remote common:macos --strategy=TestRunner=darwin-sandbox,local ``` Because of the remote caching benefits we get from BuildBuddy, these new CI jobs can be extremely fast! For example, consider these two jobs that ran all the tests on Linux x86_64: - Bazel 1m37s https://github.com/openai/codex/actions/runs/20861063212/job/59940545209?pr=8875 - Cargo 9m20s https://github.com/openai/codex/actions/runs/20861063192/job/59940559592?pr=8875 For now, we will continue to run both the Bazel and Cargo jobs for PRs, but once we add support for Windows and running Clippy, we should be able to cutover to using Bazel exclusively for PRs, which should still speed things up considerably. We will probably continue to run the Cargo jobs post-merge for commits that land on `main` as a sanity check. Release builds will also continue to be done by Cargo for now. Earlier attempt at this PR: https://github.com/openai/codex/pull/8832 Earlier attempt to add support for Buck2, now abandoned: https://github.com/openai/codex/pull/8504 --------- Co-authored-by: David Zbarsky <dzbarsky@gmail.com> Co-authored-by: Michael Bolin <mbolin@openai.com> |