mirror of
https://github.com/openai/codex.git
synced 2026-05-13 15:52:40 +00:00
30de54da36
## Why Bazel CI was not actually exercising some sharded Rust integration-test targets on macOS. The `rules_rust` sharding wrapper expects a symlink runfiles tree, but this repo runs Bazel with `--noenable_runfiles`. In that configuration the wrapper could fail to find the generated test binary, produce an empty test list, and exit successfully. That made targets such as `//codex-rs/core:core-all-test` look green even when Cargo CI could still catch failures in the same Rust tests. The coverage gap appears to have been introduced by [#18082](https://github.com/openai/codex/pull/18082), which enabled rules_rust native sharding on `//codex-rs/core:core-all-test` and the other large Rust test labels. The manifest-runfiles setup itself predates that change in [#10098](https://github.com/openai/codex/pull/10098), but #18082 is where the affected integration tests started running through the incompatible rules_rust sharding wrapper. [#18913](https://github.com/openai/codex/pull/18913) fixed the same class of issue for wrapped unit-test shards, but integration-test shards were still going through the rules_rust wrapper until this PR. We still do not have the V8/code-mode pieces stable under the Bazel CI cross-compile setup, so this keeps those tests out of Bazel while restoring coverage for the rest of the sharded Rust integration suites. Cargo CI remains responsible for V8/code-mode coverage for now. This change did uncover a real failing core test on `main`: `approved_folder_write_request_permissions_unblocks_later_apply_patch`. That fix is split into [#21060](https://github.com/openai/codex/pull/21060), which enables the `apply_patch` tool in the test, teaches the aggregate core test binary to dispatch the sandboxed filesystem helper, canonicalizes the macOS temp patch target, and isolates the core test harness from managed local/enterprise config. Keeping that fix separate lets this PR stay focused on restoring Bazel coverage while documenting the first failure it exposed. ## What changed - Build sharded Rust integration tests as manual `*-bin` binaries and run them through the existing manifest-aware `workspace_root_test` launcher. - Keep Bazel sharding on the launcher target so Rust test cases are still distributed by stable test-name hashing. - Configure Bazel CI to skip Rust tests whose names contain `suite::code_mode::`. - Exclude the standalone `codex-rs/code-mode` and `codex-rs/v8-poc` unit-test targets from `bazel.yml`. ## Verification - `bazel query --output=build //codex-rs/core:core-all-test` now shows `workspace_root_test` wrapping `//codex-rs/core:core-all-test-bin`. - `bazel test --test_output=all --nocache_test_results --test_sharding_strategy=disabled //codex-rs/core:core-all-test --test_filter=suite::request_permissions_tool::approved_folder_write_request_permissions_unblocks_later_apply_patch` runs the actual Rust test body and passes. - `bazel test --test_output=errors --nocache_test_results --test_env=CODEX_BAZEL_TEST_SKIP_FILTERS=suite::code_mode:: //codex-rs/core:core-all-test` runs the sharded target with code-mode skipped and passes overall locally, with one flaky attempt retried by the existing `flaky = True` setting.
392 lines
16 KiB
YAML
392 lines
16 KiB
YAML
name: Bazel
|
|
|
|
# Note this workflow was originally derived from:
|
|
# https://github.com/cerisier/toolchains_llvm_bootstrapped/blob/main/.github/workflows/ci.yaml
|
|
|
|
on:
|
|
pull_request: {}
|
|
push:
|
|
branches:
|
|
- main
|
|
workflow_dispatch:
|
|
|
|
concurrency:
|
|
# Cancel previous actions from the same PR or branch except 'main' branch.
|
|
# See https://docs.github.com/en/actions/using-jobs/using-concurrency and https://docs.github.com/en/actions/learn-github-actions/contexts for more info.
|
|
group: concurrency-group::${{ github.workflow }}::${{ github.event.pull_request.number > 0 && format('pr-{0}', github.event.pull_request.number) || github.ref_name }}${{ github.ref_name == 'main' && format('::{0}', github.run_id) || ''}}
|
|
cancel-in-progress: ${{ github.ref_name != 'main' }}
|
|
jobs:
|
|
test:
|
|
# PRs use a fast Windows cross-compiled test leg for pre-merge signal.
|
|
# Post-merge pushes to main also run the native Windows test job below for
|
|
# broader Windows signal without putting PR latency back on the critical
|
|
# path. Cargo CI owns V8/code-mode test coverage for now.
|
|
timeout-minutes: 30
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
# macOS
|
|
- os: macos-15-xlarge
|
|
target: aarch64-apple-darwin
|
|
- os: macos-15-xlarge
|
|
target: x86_64-apple-darwin
|
|
|
|
# Linux
|
|
- os: ubuntu-24.04
|
|
target: x86_64-unknown-linux-gnu
|
|
- os: ubuntu-24.04
|
|
target: x86_64-unknown-linux-musl
|
|
# 2026-02-27 Bazel tests have been flaky on arm in CI.
|
|
# Disable until we can investigate and stabilize them.
|
|
# - os: ubuntu-24.04-arm
|
|
# target: aarch64-unknown-linux-musl
|
|
# - os: ubuntu-24.04-arm
|
|
# target: aarch64-unknown-linux-gnu
|
|
|
|
# Windows fast path: build the windows-gnullvm binaries with Linux
|
|
# RBE, then run the resulting Windows tests on the Windows runner.
|
|
# Cargo CI preserves V8/code-mode coverage while Bazel CI keeps broad
|
|
# non-code-mode signal.
|
|
- os: windows-latest
|
|
target: x86_64-pc-windows-gnullvm
|
|
runs-on: ${{ matrix.os }}
|
|
|
|
# Configure a human readable name for each job
|
|
name: Bazel test on ${{ matrix.os }} for ${{ matrix.target }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
|
|
|
|
- name: Check rusty_v8 MODULE.bazel checksums
|
|
if: matrix.os == 'ubuntu-24.04' && matrix.target == 'x86_64-unknown-linux-gnu'
|
|
shell: bash
|
|
run: |
|
|
python3 .github/scripts/rusty_v8_bazel.py check-module-bazel
|
|
python3 -m unittest discover -s .github/scripts -p test_rusty_v8_bazel.py
|
|
|
|
- name: Prepare Bazel CI
|
|
id: prepare_bazel
|
|
uses: ./.github/actions/prepare-bazel-ci
|
|
with:
|
|
target: ${{ matrix.target }}
|
|
cache-scope: bazel-${{ github.job }}
|
|
install-test-prereqs: "true"
|
|
- name: Check MODULE.bazel.lock is up to date
|
|
if: matrix.os == 'ubuntu-24.04' && matrix.target == 'x86_64-unknown-linux-gnu'
|
|
shell: bash
|
|
run: ./scripts/check-module-bazel-lock.sh
|
|
|
|
- name: bazel test //...
|
|
env:
|
|
BUILDBUDDY_API_KEY: ${{ secrets.BUILDBUDDY_API_KEY }}
|
|
shell: bash
|
|
run: |
|
|
bazel_targets=(
|
|
//...
|
|
# Keep standalone V8 library targets out of the ordinary Bazel CI
|
|
# path. V8 consumers under `//codex-rs/...` still participate
|
|
# transitively through `//...`.
|
|
-//third_party/v8:all
|
|
# V8-backed code-mode tests are covered by Cargo CI. Bazel CI
|
|
# cross-compiles in several legs, and those tests are not stable in
|
|
# that setup yet.
|
|
-//codex-rs/code-mode:code-mode-unit-tests
|
|
-//codex-rs/v8-poc:v8-poc-unit-tests
|
|
)
|
|
|
|
bazel_wrapper_args=(
|
|
--print-failed-action-summary
|
|
--print-failed-test-logs
|
|
)
|
|
bazel_test_args=(
|
|
test
|
|
--test_tag_filters=-argument-comment-lint
|
|
--test_verbose_timeout_warnings
|
|
--build_metadata=COMMIT_SHA=${GITHUB_SHA}
|
|
)
|
|
if [[ "${RUNNER_OS}" == "Windows" ]]; then
|
|
bazel_wrapper_args+=(
|
|
--windows-cross-compile
|
|
--remote-download-toplevel
|
|
)
|
|
fi
|
|
|
|
./.github/scripts/run-bazel-ci.sh \
|
|
"${bazel_wrapper_args[@]}" \
|
|
-- \
|
|
"${bazel_test_args[@]}" \
|
|
-- \
|
|
"${bazel_targets[@]}"
|
|
|
|
- name: Upload Bazel execution logs
|
|
if: always() && !cancelled()
|
|
continue-on-error: true
|
|
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7
|
|
with:
|
|
name: bazel-execution-logs-test-${{ matrix.target }}
|
|
path: ${{ runner.temp }}/bazel-execution-logs
|
|
if-no-files-found: ignore
|
|
|
|
# Save the job-scoped Bazel repository cache after cache misses. Keep the
|
|
# upload non-fatal so cache service issues never fail the job itself.
|
|
- name: Save bazel repository cache
|
|
if: always() && !cancelled() && steps.prepare_bazel.outputs.repository-cache-hit != 'true'
|
|
continue-on-error: true
|
|
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
|
|
with:
|
|
path: ${{ steps.prepare_bazel.outputs.repository-cache-path }}
|
|
key: ${{ steps.prepare_bazel.outputs.repository-cache-key }}
|
|
|
|
test-windows-native-main:
|
|
# Native Windows Bazel tests are slower and frequently approach the
|
|
# 30-minute PR budget. Run this only for post-merge commits to main and give
|
|
# it a larger timeout.
|
|
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
|
timeout-minutes: 40
|
|
runs-on: windows-latest
|
|
name: Bazel test on windows-latest for x86_64-pc-windows-gnullvm (native main)
|
|
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
|
|
|
|
- name: Prepare Bazel CI
|
|
id: prepare_bazel
|
|
uses: ./.github/actions/prepare-bazel-ci
|
|
with:
|
|
target: x86_64-pc-windows-gnullvm
|
|
cache-scope: bazel-${{ github.job }}
|
|
install-test-prereqs: "true"
|
|
|
|
- name: bazel test //...
|
|
env:
|
|
BUILDBUDDY_API_KEY: ${{ secrets.BUILDBUDDY_API_KEY }}
|
|
shell: bash
|
|
run: |
|
|
bazel_targets=(
|
|
//...
|
|
# Keep standalone V8 library targets out of the ordinary Bazel CI
|
|
# path. V8 consumers under `//codex-rs/...` still participate
|
|
# transitively through `//...`.
|
|
-//third_party/v8:all
|
|
# Keep this aligned with the main Bazel job. The native Windows
|
|
# job preserves broad post-merge coverage, but code-mode/V8 tests
|
|
# are covered by Cargo CI rather than Bazel for now.
|
|
-//codex-rs/code-mode:code-mode-unit-tests
|
|
-//codex-rs/v8-poc:v8-poc-unit-tests
|
|
)
|
|
|
|
bazel_test_args=(
|
|
test
|
|
--test_tag_filters=-argument-comment-lint
|
|
--test_verbose_timeout_warnings
|
|
--build_metadata=COMMIT_SHA=${GITHUB_SHA}
|
|
--build_metadata=TAG_windows_native_main=true
|
|
)
|
|
|
|
./.github/scripts/run-bazel-ci.sh \
|
|
--print-failed-action-summary \
|
|
--print-failed-test-logs \
|
|
-- \
|
|
"${bazel_test_args[@]}" \
|
|
-- \
|
|
"${bazel_targets[@]}"
|
|
|
|
- name: Upload Bazel execution logs
|
|
if: always() && !cancelled()
|
|
continue-on-error: true
|
|
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7
|
|
with:
|
|
name: bazel-execution-logs-test-windows-native-x86_64-pc-windows-gnullvm
|
|
path: ${{ runner.temp }}/bazel-execution-logs
|
|
if-no-files-found: ignore
|
|
|
|
# Save the job-scoped Bazel repository cache after cache misses. Keep the
|
|
# upload non-fatal so cache service issues never fail the job itself.
|
|
- name: Save bazel repository cache
|
|
if: always() && !cancelled() && steps.prepare_bazel.outputs.repository-cache-hit != 'true'
|
|
continue-on-error: true
|
|
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
|
|
with:
|
|
path: ${{ steps.prepare_bazel.outputs.repository-cache-path }}
|
|
key: ${{ steps.prepare_bazel.outputs.repository-cache-key }}
|
|
|
|
clippy:
|
|
timeout-minutes: 30
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
# Keep Linux lint coverage on x64 and add the arm64 macOS path that
|
|
# the Bazel test job already exercises. Add Windows gnullvm as well
|
|
# so PRs get Bazel-native lint signal on the same Windows toolchain
|
|
# that the Bazel test job uses.
|
|
- os: ubuntu-24.04
|
|
target: x86_64-unknown-linux-gnu
|
|
- os: macos-15-xlarge
|
|
target: aarch64-apple-darwin
|
|
- os: windows-latest
|
|
target: x86_64-pc-windows-gnullvm
|
|
runs-on: ${{ matrix.os }}
|
|
name: Bazel clippy on ${{ matrix.os }} for ${{ matrix.target }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
|
|
|
|
- name: Prepare Bazel CI
|
|
id: prepare_bazel
|
|
uses: ./.github/actions/prepare-bazel-ci
|
|
with:
|
|
target: ${{ matrix.target }}
|
|
cache-scope: bazel-${{ github.job }}
|
|
|
|
- name: bazel build --config=clippy lint targets
|
|
env:
|
|
BUILDBUDDY_API_KEY: ${{ secrets.BUILDBUDDY_API_KEY }}
|
|
shell: bash
|
|
run: |
|
|
bazel_clippy_args=(
|
|
--config=clippy
|
|
--build_metadata=COMMIT_SHA=${GITHUB_SHA}
|
|
--build_metadata=TAG_job=clippy
|
|
)
|
|
bazel_wrapper_args=()
|
|
bazel_target_list_args=()
|
|
if [[ "${RUNNER_OS}" == "Windows" ]]; then
|
|
# Keep this aligned with the fast Windows Bazel test job: use
|
|
# Linux RBE for clippy build actions while targeting Windows
|
|
# gnullvm. Fork/community PRs without the BuildBuddy secret fall
|
|
# back inside `run-bazel-ci.sh` to the previous local Windows MSVC
|
|
# host-platform shape.
|
|
bazel_wrapper_args+=(--windows-cross-compile)
|
|
bazel_target_list_args+=(--windows-cross-compile)
|
|
if [[ -z "${BUILDBUDDY_API_KEY:-}" ]]; then
|
|
# The fork fallback can see incompatible explicit Windows-cross
|
|
# internal test binaries in the generated target list. Preserve
|
|
# the old local-fallback behavior there.
|
|
bazel_clippy_args+=(--skip_incompatible_explicit_targets)
|
|
fi
|
|
fi
|
|
|
|
bazel_target_lines="$(./scripts/list-bazel-clippy-targets.sh "${bazel_target_list_args[@]}")"
|
|
bazel_targets=()
|
|
while IFS= read -r target; do
|
|
bazel_targets+=("${target}")
|
|
done <<< "${bazel_target_lines}"
|
|
|
|
./.github/scripts/run-bazel-ci.sh \
|
|
--print-failed-action-summary \
|
|
"${bazel_wrapper_args[@]}" \
|
|
-- \
|
|
build \
|
|
"${bazel_clippy_args[@]}" \
|
|
-- \
|
|
"${bazel_targets[@]}"
|
|
|
|
- name: Upload Bazel execution logs
|
|
if: always() && !cancelled()
|
|
continue-on-error: true
|
|
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7
|
|
with:
|
|
name: bazel-execution-logs-clippy-${{ matrix.target }}
|
|
path: ${{ runner.temp }}/bazel-execution-logs
|
|
if-no-files-found: ignore
|
|
|
|
# Save the job-scoped Bazel repository cache after cache misses. Keep the
|
|
# upload non-fatal so cache service issues never fail the job itself.
|
|
- name: Save bazel repository cache
|
|
if: always() && !cancelled() && steps.prepare_bazel.outputs.repository-cache-hit != 'true'
|
|
continue-on-error: true
|
|
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
|
|
with:
|
|
path: ${{ steps.prepare_bazel.outputs.repository-cache-path }}
|
|
key: ${{ steps.prepare_bazel.outputs.repository-cache-key }}
|
|
|
|
verify-release-build:
|
|
timeout-minutes: 30
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- os: ubuntu-24.04
|
|
target: x86_64-unknown-linux-gnu
|
|
- os: macos-15-xlarge
|
|
target: aarch64-apple-darwin
|
|
- os: windows-latest
|
|
target: x86_64-pc-windows-gnullvm
|
|
runs-on: ${{ matrix.os }}
|
|
name: Verify release build on ${{ matrix.os }} for ${{ matrix.target }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
|
|
|
|
- name: Prepare Bazel CI
|
|
id: prepare_bazel
|
|
uses: ./.github/actions/prepare-bazel-ci
|
|
with:
|
|
target: ${{ matrix.target }}
|
|
cache-scope: bazel-${{ github.job }}
|
|
|
|
- name: bazel build verify-release-build targets
|
|
env:
|
|
BUILDBUDDY_API_KEY: ${{ secrets.BUILDBUDDY_API_KEY }}
|
|
shell: bash
|
|
run: |
|
|
# This job exists to compile Rust code behind
|
|
# `cfg(not(debug_assertions))` so PR CI catches failures that would
|
|
# otherwise show up only in a release build. We do not need the full
|
|
# optimizer and debug-info work that normally comes with a release
|
|
# build to get that signal, so keep Bazel in `fastbuild` and disable
|
|
# Rust debug assertions explicitly.
|
|
bazel_wrapper_args=()
|
|
if [[ "${RUNNER_OS}" == "Windows" ]]; then
|
|
# This is build-only signal, so use the same Linux-RBE
|
|
# cross-compile path as the fast Windows test and clippy jobs.
|
|
# Fork/community PRs without the BuildBuddy secret fall back
|
|
# inside `run-bazel-ci.sh` to the previous local Windows MSVC
|
|
# host-platform shape.
|
|
bazel_wrapper_args+=(--windows-cross-compile)
|
|
fi
|
|
|
|
bazel_build_args=(
|
|
--compilation_mode=fastbuild
|
|
--@rules_rust//rust/settings:extra_rustc_flag=-Cdebug-assertions=no
|
|
--@rules_rust//rust/settings:extra_exec_rustc_flag=-Cdebug-assertions=no
|
|
--build_metadata=COMMIT_SHA=${GITHUB_SHA}
|
|
--build_metadata=TAG_job=verify-release-build
|
|
--build_metadata=TAG_rust_debug_assertions=off
|
|
)
|
|
|
|
bazel_target_lines="$(bash ./scripts/list-bazel-release-targets.sh)"
|
|
bazel_targets=()
|
|
while IFS= read -r target; do
|
|
bazel_targets+=("${target}")
|
|
done <<< "${bazel_target_lines}"
|
|
|
|
./.github/scripts/run-bazel-ci.sh \
|
|
"${bazel_wrapper_args[@]}" \
|
|
-- \
|
|
build \
|
|
"${bazel_build_args[@]}" \
|
|
-- \
|
|
"${bazel_targets[@]}"
|
|
|
|
- name: Upload Bazel execution logs
|
|
if: always() && !cancelled()
|
|
continue-on-error: true
|
|
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7
|
|
with:
|
|
name: bazel-execution-logs-verify-release-build-${{ matrix.target }}
|
|
path: ${{ runner.temp }}/bazel-execution-logs
|
|
if-no-files-found: ignore
|
|
|
|
# Save the job-scoped Bazel repository cache after cache misses. Keep the
|
|
# upload non-fatal so cache service issues never fail the job itself.
|
|
- name: Save bazel repository cache
|
|
if: always() && !cancelled() && steps.prepare_bazel.outputs.repository-cache-hit != 'true'
|
|
continue-on-error: true
|
|
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
|
|
with:
|
|
path: ${{ steps.prepare_bazel.outputs.repository-cache-path }}
|
|
key: ${{ steps.prepare_bazel.outputs.repository-cache-key }}
|