mirror of
https://github.com/openai/codex.git
synced 2026-04-26 23:55:25 +00:00
343d1af3da
## Why This PR is the current, consolidated follow-up to the earlier Windows Bazel attempt in #11229. The goal is no longer just to get a tiny Windows smoke job limping along: it is to make the ordinary Bazel CI path usable on `windows-latest` for `x86_64-pc-windows-gnullvm`, with the same broad `//...` test shape that macOS and Linux already use. The earlier smoke-list version of this work was useful as a foothold, but it was not a good long-term landing point. Windows Bazel kept surfacing real issues outside that allowlist: - GitHub's Windows runner exposed runfiles-manifest bugs such as `FINDSTR: Cannot open D:MANIFEST`, which broke Bazel test launchers even when the manifest file existed. - `rules_rs`, `rules_rust`, LLVM extraction, and Abseil still needed `windows-gnullvm`-specific fixes for our hermetic toolchain. - the V8 path needed more work than just turning the Windows matrix entry back on: `rusty_v8` does not ship Windows GNU artifacts in the same shape we need, and Bazel's in-tree V8 build needed a set of Windows GNU portability fixes. Windows performance pressure also pushed this toward a full solution instead of a permanent smoke suite. During this investigation we hit targets such as `//codex-rs/shell-command:shell-command-unit-tests` that were much more expensive on Windows because they repeatedly spawn real PowerShell parsers (see #16057 for one concrete example of that pressure). That made it much more valuable to get the real Windows Bazel path working than to keep iterating on a narrowly curated subset. The net result is that this PR now aims for the same CI contract on Windows that we already expect elsewhere: keep standalone `//third_party/v8:all` out of the ordinary Bazel lane, but allow V8 consumers under `//codex-rs/...` to build and test transitively through `//...`. ## What Changed ### CI and workflow wiring - re-enable the `windows-latest` / `x86_64-pc-windows-gnullvm` Bazel matrix entry in `.github/workflows/bazel.yml` - move the Windows Bazel output root to `D:\b` and enable `git config --global core.longpaths true` in `.github/actions/setup-bazel-ci/action.yml` - keep the ordinary Bazel target set on Windows aligned with macOS and Linux by running `//...` while excluding only standalone `//third_party/v8:all` targets from the normal lane ### Toolchain and module support for `windows-gnullvm` - patch `rules_rs` so `windows-gnullvm` is modeled as a distinct Windows exec/toolchain platform instead of collapsing into the generic Windows shape - patch `rules_rust` build-script environment handling so llvm-mingw build-script probes do not inherit unsupported `-fstack-protector*` flags - patch the LLVM module archive so it extracts cleanly on Windows and provides the MinGW libraries this toolchain needs - patch Abseil so its thread-local identity path matches the hermetic `windows-gnullvm` toolchain instead of taking an incompatible MinGW pthread path - keep both MSVC and GNU Windows targets in the generated Cargo metadata because the current V8 release-asset story still uses MSVC-shaped names in some places while the Bazel build targets the GNU ABI ### Windows test-launch and binary-behavior fixes - update `workspace_root_test_launcher.bat.tpl` to read the runfiles manifest directly instead of shelling out to `findstr`, which was the source of the `D:MANIFEST` failures on the GitHub Windows runner - thread a larger Windows GNU stack reserve through `defs.bzl` so Bazel-built binaries that pull in V8 behave correctly both under normal builds and under `bazel test` - remove the no-longer-needed Windows bootstrap sh-toolchain override from `.bazelrc` ### V8 / `rusty_v8` Windows GNU support - export and apply the new Windows GNU patch set from `patches/BUILD.bazel` / `MODULE.bazel` - patch the V8 module/rules/source layers so the in-tree V8 build can produce Windows GNU archives under Bazel - teach `third_party/v8/BUILD.bazel` to build Windows GNU static archives in-tree instead of aliasing them to the MSVC prebuilts - reuse the Linux release binding for the experimental Windows GNU path where `rusty_v8` does not currently publish a Windows GNU binding artifact ## Testing - the primary end-to-end validation for this work is the `Bazel` workflow plus `v8-canary`, since the hard parts are Windows-specific and depend on real GitHub runner behavior - before consolidation back onto this PR, the same net change passed the full Bazel matrix in [run 23675590471](https://github.com/openai/codex/actions/runs/23675590471) and passed `v8-canary` in [run 23675590453](https://github.com/openai/codex/actions/runs/23675590453) - those successful runs included the `windows-latest` / `x86_64-pc-windows-gnullvm` Bazel job with the ordinary `//...` path, not the earlier Windows smoke allowlist --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/15952). * #16067 * __->__ #15952
152 lines
6.5 KiB
Diff
152 lines
6.5 KiB
Diff
# What: teach `rules_rs` that `windows-gnullvm` is a distinct Windows exec ABI.
|
|
# Scope: experimental platform/toolchain naming only; no Cargo target changes.
|
|
|
|
diff --git a/rs/experimental/platforms/triples.bzl b/rs/experimental/platforms/triples.bzl
|
|
index 3ca3bb1..dd15656 100644
|
|
--- a/rs/experimental/platforms/triples.bzl
|
|
+++ b/rs/experimental/platforms/triples.bzl
|
|
@@ -30,7 +30,9 @@ SUPPORTED_EXEC_TRIPLES = [
|
|
"x86_64-unknown-linux-gnu",
|
|
"aarch64-unknown-linux-gnu",
|
|
"x86_64-pc-windows-msvc",
|
|
+ "x86_64-pc-windows-gnullvm",
|
|
"aarch64-pc-windows-msvc",
|
|
+ "aarch64-pc-windows-gnullvm",
|
|
"x86_64-apple-darwin",
|
|
"aarch64-apple-darwin",
|
|
]
|
|
diff --git a/rs/experimental/toolchains/declare_rustc_toolchains.bzl b/rs/experimental/toolchains/declare_rustc_toolchains.bzl
|
|
index b9a0ce1..67e491c 100644
|
|
--- a/rs/experimental/toolchains/declare_rustc_toolchains.bzl
|
|
+++ b/rs/experimental/toolchains/declare_rustc_toolchains.bzl
|
|
@@ -10,6 +10,11 @@ def _channel(version):
|
|
return "beta"
|
|
return "stable"
|
|
|
|
+def _exec_triple_suffix(exec_triple):
|
|
+ if exec_triple.system == "windows":
|
|
+ return "{}_{}_{}".format(exec_triple.system, exec_triple.arch, exec_triple.abi)
|
|
+ return "{}_{}".format(exec_triple.system, exec_triple.arch)
|
|
+
|
|
def declare_rustc_toolchains(
|
|
*,
|
|
version,
|
|
@@ -23,15 +28,14 @@ def declare_rustc_toolchains(
|
|
|
|
for triple in execs:
|
|
exec_triple = _parse_triple(triple)
|
|
- triple_suffix = exec_triple.system + "_" + exec_triple.arch
|
|
+ triple_suffix = _exec_triple_suffix(exec_triple)
|
|
|
|
rustc_repo_label = "@rustc_{}_{}//:".format(triple_suffix, version_key)
|
|
cargo_repo_label = "@cargo_{}_{}//:".format(triple_suffix, version_key)
|
|
clippy_repo_label = "@clippy_{}_{}//:".format(triple_suffix, version_key)
|
|
|
|
- rust_toolchain_name = "{}_{}_{}_rust_toolchain".format(
|
|
- exec_triple.system,
|
|
- exec_triple.arch,
|
|
+ rust_toolchain_name = "{}_{}_rust_toolchain".format(
|
|
+ triple_suffix,
|
|
version_key,
|
|
)
|
|
|
|
@@ -90,11 +94,8 @@ def declare_rustc_toolchains(
|
|
target_key = sanitize_triple(target_triple)
|
|
|
|
native.toolchain(
|
|
- name = "{}_{}_to_{}_{}".format(exec_triple.system, exec_triple.arch, target_key, version_key),
|
|
- exec_compatible_with = [
|
|
- "@platforms//os:" + exec_triple.system,
|
|
- "@platforms//cpu:" + exec_triple.arch,
|
|
- ],
|
|
+ name = "{}_to_{}_{}".format(triple_suffix, target_key, version_key),
|
|
+ exec_compatible_with = triple_to_constraint_set(triple),
|
|
target_compatible_with = triple_to_constraint_set(target_triple),
|
|
target_settings = [
|
|
"@rules_rust//rust/toolchain/channel:" + channel,
|
|
diff --git a/rs/experimental/toolchains/declare_rustfmt_toolchains.bzl b/rs/experimental/toolchains/declare_rustfmt_toolchains.bzl
|
|
index a219209..ecb6b05 100644
|
|
--- a/rs/experimental/toolchains/declare_rustfmt_toolchains.bzl
|
|
+++ b/rs/experimental/toolchains/declare_rustfmt_toolchains.bzl
|
|
@@ -1,8 +1,13 @@
|
|
load("@rules_rust//rust:toolchain.bzl", "rustfmt_toolchain")
|
|
load("@rules_rust//rust/platform:triple.bzl", _parse_triple = "triple")
|
|
-load("//rs/experimental/platforms:triples.bzl", "SUPPORTED_EXEC_TRIPLES")
|
|
+load("//rs/experimental/platforms:triples.bzl", "SUPPORTED_EXEC_TRIPLES", "triple_to_constraint_set")
|
|
load("//rs/experimental/toolchains:toolchain_utils.bzl", "sanitize_version")
|
|
|
|
+def _exec_triple_suffix(exec_triple):
|
|
+ if exec_triple.system == "windows":
|
|
+ return "{}_{}_{}".format(exec_triple.system, exec_triple.arch, exec_triple.abi)
|
|
+ return "{}_{}".format(exec_triple.system, exec_triple.arch)
|
|
+
|
|
def _channel(version):
|
|
if version.startswith("nightly"):
|
|
return "nightly"
|
|
@@ -22,14 +27,13 @@ def declare_rustfmt_toolchains(
|
|
|
|
for triple in execs:
|
|
exec_triple = _parse_triple(triple)
|
|
- triple_suffix = exec_triple.system + "_" + exec_triple.arch
|
|
+ triple_suffix = _exec_triple_suffix(exec_triple)
|
|
|
|
rustc_repo_label = "@rustc_{}_{}//:".format(triple_suffix, version_key)
|
|
rustfmt_repo_label = "@rustfmt_{}_{}//:".format(triple_suffix, rustfmt_version_key)
|
|
|
|
- rustfmt_toolchain_name = "{}_{}_{}_rustfmt_toolchain".format(
|
|
- exec_triple.system,
|
|
- exec_triple.arch,
|
|
+ rustfmt_toolchain_name = "{}_{}_rustfmt_toolchain".format(
|
|
+ triple_suffix,
|
|
version_key,
|
|
)
|
|
|
|
@@ -43,11 +47,8 @@ def declare_rustfmt_toolchains(
|
|
)
|
|
|
|
native.toolchain(
|
|
- name = "{}_{}_rustfmt_{}".format(exec_triple.system, exec_triple.arch, version_key),
|
|
- exec_compatible_with = [
|
|
- "@platforms//os:" + exec_triple.system,
|
|
- "@platforms//cpu:" + exec_triple.arch,
|
|
- ],
|
|
+ name = "{}_rustfmt_{}".format(triple_suffix, version_key),
|
|
+ exec_compatible_with = triple_to_constraint_set(triple),
|
|
target_compatible_with = [],
|
|
target_settings = [
|
|
"@rules_rust//rust/toolchain/channel:" + channel,
|
|
diff --git a/rs/experimental/toolchains/module_extension.bzl b/rs/experimental/toolchains/module_extension.bzl
|
|
index 7bb0205..ace556b 100644
|
|
--- a/rs/experimental/toolchains/module_extension.bzl
|
|
+++ b/rs/experimental/toolchains/module_extension.bzl
|
|
@@ -37,6 +37,11 @@ def _normalize_arch_name(arch):
|
|
return "aarch64"
|
|
return arch
|
|
|
|
+def _exec_triple_suffix(exec_triple):
|
|
+ if exec_triple.system == "windows":
|
|
+ return "{}_{}_{}".format(exec_triple.system, exec_triple.arch, exec_triple.abi)
|
|
+ return "{}_{}".format(exec_triple.system, exec_triple.arch)
|
|
+
|
|
def _sanitize_path_fragment(path):
|
|
return path.replace("/", "_").replace(":", "_")
|
|
|
|
@@ -181,7 +186,7 @@ def _toolchains_impl(mctx):
|
|
for triple in SUPPORTED_EXEC_TRIPLES:
|
|
exec_triple = _parse_triple(triple)
|
|
|
|
- triple_suffix = exec_triple.system + "_" + exec_triple.arch
|
|
+ triple_suffix = _exec_triple_suffix(exec_triple)
|
|
rustc_name = "rustc_{}_{}".format(triple_suffix, version_key)
|
|
|
|
rustc_repository(
|
|
@@ -230,7 +235,7 @@ def _toolchains_impl(mctx):
|
|
|
|
for triple in SUPPORTED_EXEC_TRIPLES:
|
|
exec_triple = _parse_triple(triple)
|
|
- triple_suffix = exec_triple.system + "_" + exec_triple.arch
|
|
+ triple_suffix = _exec_triple_suffix(exec_triple)
|
|
|
|
rustfmt_repository(
|
|
name = "rustfmt_{}_{}".format(triple_suffix, version_key),
|