mirror of
https://github.com/openai/codex.git
synced 2026-05-15 16:53:05 +00:00
7c79a18688
**Summary** - Add `codex-bwrap`, a standalone `bwrap` binary built from the existing vendored bubblewrap sources. - Remove the linked vendored bwrap path from `codex-linux-sandbox`; runtime now prefers system `bwrap` and falls back to bundled `codex-resources/bwrap`. - Add bundled SHA-256 verification with missing/all-zero digest as the dev-mode skip value, then exec the verified file through `/proc/self/fd`. - Keep `launcher.rs` focused on choosing and dispatching the preferred launcher. Bundled lookup, digest verification, and bundled exec now live in `linux-sandbox/src/bundled_bwrap.rs`; Bazel runfiles lookup lives in `linux-sandbox/src/bazel_bwrap.rs`; shared argv/fd exec helpers live in `linux-sandbox/src/exec_util.rs`. - Teach Bazel tests to surface the Bazel-built `//codex-rs/bwrap:bwrap` through `CARGO_BIN_EXE_bwrap`; `codex-linux-sandbox` only honors that fallback in debug Bazel runfiles environments so release/user runtime lookup stays tied to `codex-resources/bwrap`. - Allow `codex-exec-server` filesystem helpers to preserve just the Bazel bwrap/runfiles variables they need in debug Bazel builds, since those helpers intentionally rebuild a small environment before spawning `codex-linux-sandbox`. - Verify the Bazel bwrap target in Linux release CI with a build-only check. Running `bwrap --version` is too strong for GitHub runners because bubblewrap still attempts namespace setup there. **Verification** - Latest update: `cargo test -p codex-linux-sandbox` - Latest update: `just fix -p codex-linux-sandbox` - `cargo check --target x86_64-unknown-linux-gnu -p codex-linux-sandbox` could not run locally because this macOS machine does not have `x86_64-linux-gnu-gcc`; GitHub Linux Bazel CI is expected to cover the Linux-only modules. - Earlier in this PR: `cargo test -p codex-bwrap` - Earlier in this PR: `cargo test -p codex-exec-server` - Earlier in this PR: `cargo check --release -p codex-exec-server` - Earlier in this PR: `just fix -p codex-linux-sandbox -p codex-exec-server` - Earlier in this PR: `bazel test --nobuild //codex-rs/linux-sandbox:linux-sandbox-all-test //codex-rs/core:core-all-test //codex-rs/exec-server:exec-server-file_system-test //codex-rs/app-server:app-server-all-test` (analysis completed; Bazel then refuses to run tests under `--nobuild`) - Earlier in this PR: `bazel build --nobuild //codex-rs/bwrap:bwrap` - Prior to this update: `just bazel-lock-update`, `just bazel-lock-check`, and YAML parse check for `.github/workflows/bazel.yml` --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/21255). * #21257 * #21256 * __->__ #21255
408 lines
16 KiB
YAML
408 lines
16 KiB
YAML
name: Bazel
|
|
|
|
# Note this workflow was originally derived from:
|
|
# https://github.com/cerisier/toolchains_llvm_bootstrapped/blob/main/.github/workflows/ci.yaml
|
|
|
|
on:
|
|
pull_request: {}
|
|
push:
|
|
branches:
|
|
- main
|
|
workflow_dispatch:
|
|
|
|
concurrency:
|
|
# Cancel previous actions from the same PR or branch except 'main' branch.
|
|
# See https://docs.github.com/en/actions/using-jobs/using-concurrency and https://docs.github.com/en/actions/learn-github-actions/contexts for more info.
|
|
group: concurrency-group::${{ github.workflow }}::${{ github.event.pull_request.number > 0 && format('pr-{0}', github.event.pull_request.number) || github.ref_name }}${{ github.ref_name == 'main' && format('::{0}', github.run_id) || ''}}
|
|
cancel-in-progress: ${{ github.ref_name != 'main' }}
|
|
jobs:
|
|
test:
|
|
# PRs use a fast Windows cross-compiled test leg for pre-merge signal.
|
|
# Post-merge pushes to main also run the native Windows test job below for
|
|
# broader Windows signal without putting PR latency back on the critical
|
|
# path. Cargo CI owns V8/code-mode test coverage for now.
|
|
timeout-minutes: 30
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
# macOS
|
|
- os: macos-15-xlarge
|
|
target: aarch64-apple-darwin
|
|
- os: macos-15-xlarge
|
|
target: x86_64-apple-darwin
|
|
|
|
# Linux
|
|
- os: ubuntu-24.04
|
|
target: x86_64-unknown-linux-gnu
|
|
- os: ubuntu-24.04
|
|
target: x86_64-unknown-linux-musl
|
|
# 2026-02-27 Bazel tests have been flaky on arm in CI.
|
|
# Disable until we can investigate and stabilize them.
|
|
# - os: ubuntu-24.04-arm
|
|
# target: aarch64-unknown-linux-musl
|
|
# - os: ubuntu-24.04-arm
|
|
# target: aarch64-unknown-linux-gnu
|
|
|
|
# Windows fast path: build the windows-gnullvm binaries with Linux
|
|
# RBE, then run the resulting Windows tests on the Windows runner.
|
|
# Cargo CI preserves V8/code-mode coverage while Bazel CI keeps broad
|
|
# non-code-mode signal.
|
|
- os: windows-latest
|
|
target: x86_64-pc-windows-gnullvm
|
|
runs-on: ${{ matrix.os }}
|
|
|
|
# Configure a human readable name for each job
|
|
name: Bazel test on ${{ matrix.os }} for ${{ matrix.target }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
|
|
|
|
- name: Check rusty_v8 MODULE.bazel checksums
|
|
if: matrix.os == 'ubuntu-24.04' && matrix.target == 'x86_64-unknown-linux-gnu'
|
|
shell: bash
|
|
run: |
|
|
python3 .github/scripts/rusty_v8_bazel.py check-module-bazel
|
|
python3 -m unittest discover -s .github/scripts -p test_rusty_v8_bazel.py
|
|
|
|
- name: Prepare Bazel CI
|
|
id: prepare_bazel
|
|
uses: ./.github/actions/prepare-bazel-ci
|
|
with:
|
|
target: ${{ matrix.target }}
|
|
cache-scope: bazel-${{ github.job }}
|
|
install-test-prereqs: "true"
|
|
- name: Check MODULE.bazel.lock is up to date
|
|
if: matrix.os == 'ubuntu-24.04' && matrix.target == 'x86_64-unknown-linux-gnu'
|
|
shell: bash
|
|
run: ./scripts/check-module-bazel-lock.sh
|
|
|
|
- name: bazel test //...
|
|
env:
|
|
BUILDBUDDY_API_KEY: ${{ secrets.BUILDBUDDY_API_KEY }}
|
|
shell: bash
|
|
run: |
|
|
bazel_targets=(
|
|
//...
|
|
# Keep standalone V8 library targets out of the ordinary Bazel CI
|
|
# path. V8 consumers under `//codex-rs/...` still participate
|
|
# transitively through `//...`.
|
|
-//third_party/v8:all
|
|
# V8-backed code-mode tests are covered by Cargo CI. Bazel CI
|
|
# cross-compiles in several legs, and those tests are not stable in
|
|
# that setup yet.
|
|
-//codex-rs/code-mode:code-mode-unit-tests
|
|
-//codex-rs/v8-poc:v8-poc-unit-tests
|
|
)
|
|
|
|
bazel_wrapper_args=(
|
|
--print-failed-action-summary
|
|
--print-failed-test-logs
|
|
)
|
|
bazel_test_args=(
|
|
test
|
|
--test_tag_filters=-argument-comment-lint
|
|
--test_verbose_timeout_warnings
|
|
--build_metadata=COMMIT_SHA=${GITHUB_SHA}
|
|
)
|
|
if [[ "${RUNNER_OS}" == "Windows" ]]; then
|
|
bazel_wrapper_args+=(
|
|
--windows-cross-compile
|
|
--remote-download-toplevel
|
|
)
|
|
fi
|
|
|
|
./.github/scripts/run-bazel-ci.sh \
|
|
"${bazel_wrapper_args[@]}" \
|
|
-- \
|
|
"${bazel_test_args[@]}" \
|
|
-- \
|
|
"${bazel_targets[@]}"
|
|
|
|
- name: Upload Bazel execution logs
|
|
if: always() && !cancelled()
|
|
continue-on-error: true
|
|
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7
|
|
with:
|
|
name: bazel-execution-logs-test-${{ matrix.target }}
|
|
path: ${{ runner.temp }}/bazel-execution-logs
|
|
if-no-files-found: ignore
|
|
|
|
# Save the job-scoped Bazel repository cache after cache misses. Keep the
|
|
# upload non-fatal so cache service issues never fail the job itself.
|
|
- name: Save bazel repository cache
|
|
if: always() && !cancelled() && steps.prepare_bazel.outputs.repository-cache-hit != 'true'
|
|
continue-on-error: true
|
|
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
|
|
with:
|
|
path: ${{ steps.prepare_bazel.outputs.repository-cache-path }}
|
|
key: ${{ steps.prepare_bazel.outputs.repository-cache-key }}
|
|
|
|
test-windows-native-main:
|
|
# Native Windows Bazel tests are slower and frequently approach the
|
|
# 30-minute PR budget. Run this only for post-merge commits to main and give
|
|
# it a larger timeout.
|
|
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
|
timeout-minutes: 40
|
|
runs-on: windows-latest
|
|
name: Bazel test on windows-latest for x86_64-pc-windows-gnullvm (native main)
|
|
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
|
|
|
|
- name: Prepare Bazel CI
|
|
id: prepare_bazel
|
|
uses: ./.github/actions/prepare-bazel-ci
|
|
with:
|
|
target: x86_64-pc-windows-gnullvm
|
|
cache-scope: bazel-${{ github.job }}
|
|
install-test-prereqs: "true"
|
|
|
|
- name: bazel test //...
|
|
env:
|
|
BUILDBUDDY_API_KEY: ${{ secrets.BUILDBUDDY_API_KEY }}
|
|
shell: bash
|
|
run: |
|
|
bazel_targets=(
|
|
//...
|
|
# Keep standalone V8 library targets out of the ordinary Bazel CI
|
|
# path. V8 consumers under `//codex-rs/...` still participate
|
|
# transitively through `//...`.
|
|
-//third_party/v8:all
|
|
# Keep this aligned with the main Bazel job. The native Windows
|
|
# job preserves broad post-merge coverage, but code-mode/V8 tests
|
|
# are covered by Cargo CI rather than Bazel for now.
|
|
-//codex-rs/code-mode:code-mode-unit-tests
|
|
-//codex-rs/v8-poc:v8-poc-unit-tests
|
|
)
|
|
|
|
bazel_test_args=(
|
|
test
|
|
--test_tag_filters=-argument-comment-lint
|
|
--test_verbose_timeout_warnings
|
|
--build_metadata=COMMIT_SHA=${GITHUB_SHA}
|
|
--build_metadata=TAG_windows_native_main=true
|
|
)
|
|
|
|
./.github/scripts/run-bazel-ci.sh \
|
|
--print-failed-action-summary \
|
|
--print-failed-test-logs \
|
|
-- \
|
|
"${bazel_test_args[@]}" \
|
|
-- \
|
|
"${bazel_targets[@]}"
|
|
|
|
- name: Upload Bazel execution logs
|
|
if: always() && !cancelled()
|
|
continue-on-error: true
|
|
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7
|
|
with:
|
|
name: bazel-execution-logs-test-windows-native-x86_64-pc-windows-gnullvm
|
|
path: ${{ runner.temp }}/bazel-execution-logs
|
|
if-no-files-found: ignore
|
|
|
|
# Save the job-scoped Bazel repository cache after cache misses. Keep the
|
|
# upload non-fatal so cache service issues never fail the job itself.
|
|
- name: Save bazel repository cache
|
|
if: always() && !cancelled() && steps.prepare_bazel.outputs.repository-cache-hit != 'true'
|
|
continue-on-error: true
|
|
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
|
|
with:
|
|
path: ${{ steps.prepare_bazel.outputs.repository-cache-path }}
|
|
key: ${{ steps.prepare_bazel.outputs.repository-cache-key }}
|
|
|
|
clippy:
|
|
timeout-minutes: 30
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
# Keep Linux lint coverage on x64 and add the arm64 macOS path that
|
|
# the Bazel test job already exercises. Add Windows gnullvm as well
|
|
# so PRs get Bazel-native lint signal on the same Windows toolchain
|
|
# that the Bazel test job uses.
|
|
- os: ubuntu-24.04
|
|
target: x86_64-unknown-linux-gnu
|
|
- os: macos-15-xlarge
|
|
target: aarch64-apple-darwin
|
|
- os: windows-latest
|
|
target: x86_64-pc-windows-gnullvm
|
|
runs-on: ${{ matrix.os }}
|
|
name: Bazel clippy on ${{ matrix.os }} for ${{ matrix.target }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
|
|
|
|
- name: Prepare Bazel CI
|
|
id: prepare_bazel
|
|
uses: ./.github/actions/prepare-bazel-ci
|
|
with:
|
|
target: ${{ matrix.target }}
|
|
cache-scope: bazel-${{ github.job }}
|
|
|
|
- name: bazel build --config=clippy lint targets
|
|
env:
|
|
BUILDBUDDY_API_KEY: ${{ secrets.BUILDBUDDY_API_KEY }}
|
|
shell: bash
|
|
run: |
|
|
bazel_clippy_args=(
|
|
--config=clippy
|
|
--build_metadata=COMMIT_SHA=${GITHUB_SHA}
|
|
--build_metadata=TAG_job=clippy
|
|
)
|
|
bazel_wrapper_args=()
|
|
bazel_target_list_args=()
|
|
if [[ "${RUNNER_OS}" == "Windows" ]]; then
|
|
# Keep this aligned with the fast Windows Bazel test job: use
|
|
# Linux RBE for clippy build actions while targeting Windows
|
|
# gnullvm. Fork/community PRs without the BuildBuddy secret fall
|
|
# back inside `run-bazel-ci.sh` to the previous local Windows MSVC
|
|
# host-platform shape.
|
|
bazel_wrapper_args+=(--windows-cross-compile)
|
|
bazel_target_list_args+=(--windows-cross-compile)
|
|
if [[ -z "${BUILDBUDDY_API_KEY:-}" ]]; then
|
|
# The fork fallback can see incompatible explicit Windows-cross
|
|
# internal test binaries in the generated target list. Preserve
|
|
# the old local-fallback behavior there.
|
|
bazel_clippy_args+=(--skip_incompatible_explicit_targets)
|
|
fi
|
|
fi
|
|
|
|
bazel_target_lines="$(./scripts/list-bazel-clippy-targets.sh "${bazel_target_list_args[@]}")"
|
|
bazel_targets=()
|
|
while IFS= read -r target; do
|
|
bazel_targets+=("${target}")
|
|
done <<< "${bazel_target_lines}"
|
|
|
|
./.github/scripts/run-bazel-ci.sh \
|
|
--print-failed-action-summary \
|
|
"${bazel_wrapper_args[@]}" \
|
|
-- \
|
|
build \
|
|
"${bazel_clippy_args[@]}" \
|
|
-- \
|
|
"${bazel_targets[@]}"
|
|
|
|
- name: Upload Bazel execution logs
|
|
if: always() && !cancelled()
|
|
continue-on-error: true
|
|
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7
|
|
with:
|
|
name: bazel-execution-logs-clippy-${{ matrix.target }}
|
|
path: ${{ runner.temp }}/bazel-execution-logs
|
|
if-no-files-found: ignore
|
|
|
|
# Save the job-scoped Bazel repository cache after cache misses. Keep the
|
|
# upload non-fatal so cache service issues never fail the job itself.
|
|
- name: Save bazel repository cache
|
|
if: always() && !cancelled() && steps.prepare_bazel.outputs.repository-cache-hit != 'true'
|
|
continue-on-error: true
|
|
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
|
|
with:
|
|
path: ${{ steps.prepare_bazel.outputs.repository-cache-path }}
|
|
key: ${{ steps.prepare_bazel.outputs.repository-cache-key }}
|
|
|
|
verify-release-build:
|
|
timeout-minutes: 30
|
|
strategy:
|
|
fail-fast: false
|
|
matrix:
|
|
include:
|
|
- os: ubuntu-24.04
|
|
target: x86_64-unknown-linux-gnu
|
|
- os: macos-15-xlarge
|
|
target: aarch64-apple-darwin
|
|
- os: windows-latest
|
|
target: x86_64-pc-windows-gnullvm
|
|
runs-on: ${{ matrix.os }}
|
|
name: Verify release build on ${{ matrix.os }} for ${{ matrix.target }}
|
|
|
|
steps:
|
|
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6
|
|
|
|
- name: Prepare Bazel CI
|
|
id: prepare_bazel
|
|
uses: ./.github/actions/prepare-bazel-ci
|
|
with:
|
|
target: ${{ matrix.target }}
|
|
cache-scope: bazel-${{ github.job }}
|
|
|
|
- name: bazel build verify-release-build targets
|
|
env:
|
|
BUILDBUDDY_API_KEY: ${{ secrets.BUILDBUDDY_API_KEY }}
|
|
shell: bash
|
|
run: |
|
|
# This job exists to compile Rust code behind
|
|
# `cfg(not(debug_assertions))` so PR CI catches failures that would
|
|
# otherwise show up only in a release build. We do not need the full
|
|
# optimizer and debug-info work that normally comes with a release
|
|
# build to get that signal, so keep Bazel in `fastbuild` and disable
|
|
# Rust debug assertions explicitly.
|
|
bazel_wrapper_args=()
|
|
if [[ "${RUNNER_OS}" == "Windows" ]]; then
|
|
# This is build-only signal, so use the same Linux-RBE
|
|
# cross-compile path as the fast Windows test and clippy jobs.
|
|
# Fork/community PRs without the BuildBuddy secret fall back
|
|
# inside `run-bazel-ci.sh` to the previous local Windows MSVC
|
|
# host-platform shape.
|
|
bazel_wrapper_args+=(--windows-cross-compile)
|
|
fi
|
|
|
|
bazel_build_args=(
|
|
--compilation_mode=fastbuild
|
|
--@rules_rust//rust/settings:extra_rustc_flag=-Cdebug-assertions=no
|
|
--@rules_rust//rust/settings:extra_exec_rustc_flag=-Cdebug-assertions=no
|
|
--build_metadata=COMMIT_SHA=${GITHUB_SHA}
|
|
--build_metadata=TAG_job=verify-release-build
|
|
--build_metadata=TAG_rust_debug_assertions=off
|
|
)
|
|
|
|
bazel_target_lines="$(bash ./scripts/list-bazel-release-targets.sh)"
|
|
bazel_targets=()
|
|
while IFS= read -r target; do
|
|
bazel_targets+=("${target}")
|
|
done <<< "${bazel_target_lines}"
|
|
|
|
./.github/scripts/run-bazel-ci.sh \
|
|
"${bazel_wrapper_args[@]}" \
|
|
-- \
|
|
build \
|
|
"${bazel_build_args[@]}" \
|
|
-- \
|
|
"${bazel_targets[@]}"
|
|
|
|
- name: Verify Bazel builds bwrap
|
|
if: runner.os == 'Linux'
|
|
env:
|
|
BUILDBUDDY_API_KEY: ${{ secrets.BUILDBUDDY_API_KEY }}
|
|
shell: bash
|
|
run: |
|
|
./.github/scripts/run-bazel-ci.sh \
|
|
--remote-download-toplevel \
|
|
--print-failed-action-summary \
|
|
-- \
|
|
build \
|
|
--build_metadata=COMMIT_SHA=${GITHUB_SHA} \
|
|
--build_metadata=TAG_job=verify-bwrap \
|
|
-- \
|
|
//codex-rs/bwrap:bwrap
|
|
|
|
- name: Upload Bazel execution logs
|
|
if: always() && !cancelled()
|
|
continue-on-error: true
|
|
uses: actions/upload-artifact@bbbca2ddaa5d8feaa63e36b76fdaad77386f024f # v7
|
|
with:
|
|
name: bazel-execution-logs-verify-release-build-${{ matrix.target }}
|
|
path: ${{ runner.temp }}/bazel-execution-logs
|
|
if-no-files-found: ignore
|
|
|
|
# Save the job-scoped Bazel repository cache after cache misses. Keep the
|
|
# upload non-fatal so cache service issues never fail the job itself.
|
|
- name: Save bazel repository cache
|
|
if: always() && !cancelled() && steps.prepare_bazel.outputs.repository-cache-hit != 'true'
|
|
continue-on-error: true
|
|
uses: actions/cache/save@668228422ae6a00e4ad889ee87cd7109ec5666a7 # v5
|
|
with:
|
|
path: ${{ steps.prepare_bazel.outputs.repository-cache-path }}
|
|
key: ${{ steps.prepare_bazel.outputs.repository-cache-key }}
|