mirror of
https://github.com/openai/codex.git
synced 2026-04-25 15:15:15 +00:00
19f0d196d1
## Why Follow-up to #16106. `argument-comment-lint` already runs as a native Bazel aspect on Linux and macOS, but Windows is still the long pole in `rust-ci`. To move Windows onto the same native Bazel lane, the toolchain split has to let exec-side helper binaries build in an MSVC environment while still linting repo crates as `windows-gnullvm`. Pushing the Windows lane onto the native Bazel path exposed a second round of Windows-only issues in the mixed exec-toolchain plumbing after the initial wrapper/target fixes landed. ## What Changed - keep the Windows lint lanes on the native Bazel/aspect path in `rust-ci.yml` and `rust-ci-full.yml` - add a dedicated `local_windows_msvc` platform for exec-side helper binaries while keeping `local_windows` as the `windows-gnullvm` target platform - patch `rules_rust` so `repository_set(...)` preserves explicit exec-platform constraints for the generated toolchains, keep the Windows-specific bootstrap/direct-link fixes needed for the nightly lint driver, and expose exec-side `rustc-dev` `.rlib`s to the MSVC sysroot - register the custom Windows nightly toolchain set with MSVC exec constraints while still exposing both `x86_64-pc-windows-msvc` and `x86_64-pc-windows-gnullvm` targets - enable `dev_components` on the custom Windows nightly repository set so the MSVC exec helper toolchain actually downloads the compiler-internal crates that `clippy_utils` needs - teach `run-argument-comment-lint-bazel.sh` to enumerate concrete Windows Rust rules, normalize the resulting labels, and skip explicitly requested incompatible targets instead of failing before the lint run starts - patch `rules_rust` build-script env propagation so exec-side `windows-msvc` helper crates drop forwarded MinGW include and linker search paths as whole flag/path pairs instead of emitting malformed `CFLAGS`, `CXXFLAGS`, and `LDFLAGS` - export the Windows VS/MSVC SDK environment in `setup-bazel-ci` and pass the relevant variables through `run-bazel-ci.sh` via `--action_env` / `--host_action_env` so Bazel build scripts can see the MSVC and UCRT headers on native Windows runs - add inline comments to the Windows `setup-bazel-ci` MSVC environment export step so it is easier to audit how `vswhere`, `VsDevCmd.bat`, and the filtered `GITHUB_ENV` export fit together - patch `aws-lc-sys` to skip its standalone `memcmp` probe under Bazel `windows-msvc` build-script environments, which avoids a Windows-native toolchain mismatch that blocked the lint lane before it reached the aspect execution - patch `aws-lc-sys` to prefer its bundled `prebuilt-nasm` objects for Bazel `windows-msvc` build-script runs, which avoids missing `generated-src/win-x86_64/*.asm` runfiles in the exec-side helper toolchain - annotate the Linux test-only callsites in `codex-rs/linux-sandbox` and `codex-rs/core` that the wider native lint coverage surfaced ## Patches This PR introduces a large patch stack because the Windows Bazel lint lane currently depends on behavior that upstream dependencies do not provide out of the box in the mixed `windows-gnullvm` target / `windows-msvc` exec-toolchain setup. - Most of the `rules_rust` patches look like upstream candidates rather than OpenAI-only policy. Preserving explicit exec-platform constraints, forwarding the right MSVC/UCRT environment into exec-side build scripts, exposing exec-side `rustc-dev` artifacts, and keeping the Windows bootstrap/linker behavior coherent all look like fixes to the Bazel/Rust integration layer itself. - The two `aws-lc-sys` patches are more tactical. They special-case Bazel `windows-msvc` build-script environments to avoid a `memcmp` probe mismatch and missing NASM runfiles. Those may be harder to upstream as-is because they rely on Bazel-specific detection instead of a general Cargo/build-script contract. - Short term, carrying these patches in-tree is reasonable because they unblock a real CI lane and are still narrow enough to audit. Long term, the goal should not be to keep growing a permanent local fork of either dependency. - My current expectation is that the `rules_rust` patches are less controversial and should be broken out into focused upstream proposals, while the `aws-lc-sys` patches are more likely to be temporary escape hatches unless that crate wants a more general hook for hermetic build systems. Suggested follow-up plan: 1. Split the `rules_rust` deltas into upstream-sized PRs or issues with minimized repros. 2. Revisit the `aws-lc-sys` patches during the next dependency bump and see whether they can be replaced by an upstream fix, a crate upgrade, or a cleaner opt-in mechanism. 3. Treat each dependency update as a chance to delete patches one by one so the local patch set only contains still-needed deltas. ## Verification - `./.github/scripts/run-argument-comment-lint-bazel.sh --config=argument-comment-lint --keep_going` - `RUNNER_OS=Windows ./.github/scripts/run-argument-comment-lint-bazel.sh --nobuild --config=argument-comment-lint --platforms=//:local_windows --keep_going` - `cargo test -p codex-linux-sandbox` - `cargo test -p codex-core shell_snapshot_tests` - `just argument-comment-lint` ## References - #16106
72 lines
3.4 KiB
Diff
72 lines
3.4 KiB
Diff
# What: compile exec-side Rust binaries against the exec Windows triple instead
|
|
# of the lint target triple.
|
|
# Why: Windows native argument-comment-lint keeps the repo target platform on
|
|
# `windows-gnullvm` to preserve cfg coverage, but exec-side helper binaries
|
|
# (build.rs, runners, bootstrap tools) must link as host tools. With
|
|
# `toolchain_linker_preference=rust`, rules_rust was still feeding those exec
|
|
# binaries the `windows-gnullvm` target/std path, which broke linking under the
|
|
# native Bazel lint lane.
|
|
|
|
diff --git a/rust/private/rustc.bzl b/rust/private/rustc.bzl
|
|
--- a/rust/private/rustc.bzl
|
|
+++ b/rust/private/rustc.bzl
|
|
@@ -129,6 +129,20 @@
|
|
build_setting = config.bool(flag = True),
|
|
)
|
|
|
|
-def _get_rustc_env(attr, toolchain, crate_name):
|
|
+def _effective_target_arch(toolchain, use_exec_target):
|
|
+ return toolchain.exec_triple.arch if use_exec_target else toolchain.target_arch
|
|
+
|
|
+def _effective_target_os(toolchain, use_exec_target):
|
|
+ return toolchain.exec_triple.system if use_exec_target else toolchain.target_os
|
|
+
|
|
+def _effective_target_flag_value(toolchain, use_exec_target):
|
|
+ return toolchain.exec_triple.str if use_exec_target else toolchain.target_flag_value
|
|
+
|
|
+def _effective_rust_std_paths(toolchain, use_exec_target):
|
|
+ if use_exec_target:
|
|
+ return ["{}/lib/rustlib/{}/lib".format(toolchain.sysroot, toolchain.exec_triple.str)]
|
|
+ return toolchain.rust_std_paths
|
|
+
|
|
+def _get_rustc_env(attr, toolchain, crate_name, use_exec_target = False):
|
|
"""Gathers rustc environment variables
|
|
|
|
@@ -147,6 +161,6 @@
|
|
|
|
result = {
|
|
- "CARGO_CFG_TARGET_ARCH": "" if toolchain.target_arch == None else toolchain.target_arch,
|
|
- "CARGO_CFG_TARGET_OS": "" if toolchain.target_os == None else toolchain.target_os,
|
|
+ "CARGO_CFG_TARGET_ARCH": "" if _effective_target_arch(toolchain, use_exec_target) == None else _effective_target_arch(toolchain, use_exec_target),
|
|
+ "CARGO_CFG_TARGET_OS": "" if _effective_target_os(toolchain, use_exec_target) == None else _effective_target_os(toolchain, use_exec_target),
|
|
"CARGO_CRATE_NAME": crate_name,
|
|
"CARGO_PKG_AUTHORS": "",
|
|
@@ -997,9 +1011,11 @@
|
|
if build_metadata and not use_json_output:
|
|
fail("build_metadata requires parse_json_output")
|
|
|
|
+ use_exec_target = is_exec_configuration(ctx) and crate_info.type == "bin"
|
|
+
|
|
output_dir = getattr(crate_info.output, "dirname", None)
|
|
linker_script = getattr(file, "linker_script", None)
|
|
|
|
- env = _get_rustc_env(attr, toolchain, crate_info.name)
|
|
+ env = _get_rustc_env(attr, toolchain, crate_info.name, use_exec_target)
|
|
|
|
# Wrapper args first
|
|
@@ -1138,5 +1154,5 @@
|
|
if error_format != "json":
|
|
# Color is not compatible with json output.
|
|
rustc_flags.add("--color=always")
|
|
- rustc_flags.add(toolchain.target_flag_value, format = "--target=%s")
|
|
+ rustc_flags.add(_effective_target_flag_value(toolchain, use_exec_target), format = "--target=%s")
|
|
if hasattr(attr, "crate_features"):
|
|
@@ -1144,6 +1160,6 @@
|
|
if linker_script:
|
|
rustc_flags.add(linker_script, format = "--codegen=link-arg=-T%s")
|
|
|
|
# Tell Rustc where to find the standard library (or libcore)
|
|
- rustc_flags.add_all(toolchain.rust_std_paths, before_each = "-L", format_each = "%s")
|
|
+ rustc_flags.add_all(_effective_rust_std_paths(toolchain, use_exec_target), before_each = "-L", format_each = "%s")
|
|
rustc_flags.add_all(rust_flags, map_each = map_flag)
|