Files
codex/workspace_root_test_launcher.sh.tpl
Michael Bolin 536952eeee bazel: run wrapped Rust unit test shards (#18913)
## Why

The `codex-tui` Cargo test suite was catching stale snapshot
expectations, but the matching Bazel unit-test target was still green.
The TUI unit target is wrapped by `workspace_root_test` so tests run
from the repository root and Insta can resolve Cargo-like snapshot
paths. After native Bazel sharding was enabled for that wrapped target,
rules_rust also inserted its own sharding wrapper around the Rust test
binary.

Those two wrappers did not compose: rules_rust's sharding wrapper
expects to run from its own runfiles cwd, while `workspace_root_test`
deliberately changes cwd to the repo root before invoking the test. In
that configuration, the inner wrapper could fail to enumerate the Rust
tests and exit successfully with empty shards, so snapshot regressions
were not being exercised by Bazel.

## What Changed

- Stop enabling rules_rust's inner `experimental_enable_sharding` for
unit-test binaries created by `codex_rust_crate`.
- Keep the configured `shard_count` on the outer `workspace_root_test`
target.
- Add libtest sharding directly to `workspace_root_test_launcher.sh.tpl`
and `workspace_root_test_launcher.bat.tpl` after the launcher has
resolved the actual test binary and established the intended
repository-root cwd.
- Partition tests by a stable FNV-1a hash of each libtest test name,
matching the stable-shard behavior we wanted without depending on the
inner rules_rust wrapper.
- Preserve ad-hoc local test filters by running the resolved test binary
directly when explicit test args are supplied.
- On Windows, run selected libtest names from the shard list in bounded
PowerShell batches instead of concatenating every selected test into one
`cmd.exe` command line.

This PR is stacked on top of #18912, which contains only the snapshot
expectation updates exposed once the Bazel target actually runs the TUI
unit tests. It is also the reason #18916 becomes visible: once this
wrapper fix makes Bazel execute the affected `codex-core` test, that
test needs its own executable-path setup fixed.

## Verification

- `cargo test -p codex-tui`
- `bazel test //codex-rs/tui:tui-unit-tests --test_output=errors`
- `bazel test //codex-rs/tui:all --test_output=errors`
- `bash -n workspace_root_test_launcher.sh.tpl`
- Exercised the Windows PowerShell batching fragment locally with a fake
test binary and shard-list file.
2026-04-21 18:35:47 -07:00

120 lines
3.4 KiB
Smarty

#!/usr/bin/env bash
set -euo pipefail
resolve_runfile() {
local logical_path="$1"
local workspace_logical_path="${logical_path}"
if [[ -n "${TEST_WORKSPACE:-}" ]]; then
workspace_logical_path="${TEST_WORKSPACE}/${logical_path}"
fi
for runfiles_root in "${RUNFILES_DIR:-}" "${TEST_SRCDIR:-}"; do
if [[ -n "${runfiles_root}" && -e "${runfiles_root}/${logical_path}" ]]; then
printf '%s\n' "${runfiles_root}/${logical_path}"
return 0
fi
if [[ -n "${runfiles_root}" && -e "${runfiles_root}/${workspace_logical_path}" ]]; then
printf '%s\n' "${runfiles_root}/${workspace_logical_path}"
return 0
fi
done
local manifest="${RUNFILES_MANIFEST_FILE:-}"
if [[ -z "${manifest}" ]]; then
if [[ -f "$0.runfiles_manifest" ]]; then
manifest="$0.runfiles_manifest"
elif [[ -f "$0.exe.runfiles_manifest" ]]; then
manifest="$0.exe.runfiles_manifest"
fi
fi
if [[ -n "${manifest}" && -f "${manifest}" ]]; then
local resolved=""
resolved="$(awk -v key="${logical_path}" '$1 == key { $1 = ""; sub(/^ /, ""); print; exit }' "${manifest}")"
if [[ -z "${resolved}" ]]; then
resolved="$(awk -v key="${workspace_logical_path}" '$1 == key { $1 = ""; sub(/^ /, ""); print; exit }' "${manifest}")"
fi
if [[ -n "${resolved}" ]]; then
printf '%s\n' "${resolved}"
return 0
fi
fi
echo "failed to resolve runfile: $logical_path" >&2
return 1
}
workspace_root_marker="$(resolve_runfile "__WORKSPACE_ROOT_MARKER__")"
workspace_root="$(dirname "$(dirname "$(dirname "${workspace_root_marker}")")")"
test_bin="$(resolve_runfile "__TEST_BIN__")"
test_shard_index() {
local test_name="$1"
# FNV-1a 32-bit hash. Keep this stable so adding one test does not reshuffle
# unrelated tests between shards.
local hash=2166136261
local byte
local char
local i
local LC_ALL=C
for ((i = 0; i < ${#test_name}; i++)); do
char="${test_name:i:1}"
printf -v byte "%d" "'$char"
hash=$(( ((hash ^ byte) * 16777619) & 0xffffffff ))
done
echo $(( hash % TOTAL_SHARDS ))
}
run_sharded_libtest() {
if [[ -n "${TEST_SHARD_STATUS_FILE:-}" && "${TEST_TOTAL_SHARDS:-0}" != "0" ]]; then
touch "${TEST_SHARD_STATUS_FILE}"
fi
# Extra libtest args are usually ad-hoc local filters. Preserve those exactly
# rather than combining them with generated exact filters.
if [[ $# -gt 0 ]]; then
exec "${test_bin}" "$@"
fi
if [[ -z "${SHARD_INDEX}" ]]; then
echo "TEST_SHARD_INDEX or RULES_RUST_TEST_SHARD_INDEX must be set when sharding is enabled" >&2
exit 1
fi
local list_output
local test_list
list_output="$("${test_bin}" --list --format terse)"
test_list="$(printf '%s\n' "${list_output}" | grep ': test$' | sed 's/: test$//' | LC_ALL=C sort || true)"
if [[ -z "${test_list}" ]]; then
exit 0
fi
local shard_tests=()
local test_name
while IFS= read -r test_name; do
if (( $(test_shard_index "${test_name}") == SHARD_INDEX )); then
shard_tests+=("${test_name}")
fi
done <<< "${test_list}"
if [[ ${#shard_tests[@]} -eq 0 ]]; then
exit 0
fi
exec "${test_bin}" "${shard_tests[@]}" --exact
}
export INSTA_WORKSPACE_ROOT="${workspace_root}"
cd "${workspace_root}"
TOTAL_SHARDS="${RULES_RUST_TEST_TOTAL_SHARDS:-${TEST_TOTAL_SHARDS:-}}"
SHARD_INDEX="${RULES_RUST_TEST_SHARD_INDEX:-${TEST_SHARD_INDEX:-}}"
if [[ -n "${TOTAL_SHARDS}" && "${TOTAL_SHARDS}" != "0" ]]; then
run_sharded_libtest "$@"
fi
exec "${test_bin}" "$@"