build: fetch rg for Codex packages (#23526)

## Why

The Codex package builder should produce a complete package without
requiring callers to pre-populate `rg` under `codex-cli/vendor` or have
`dotslash` installed on `PATH`. The repo already tracks the
authoritative DotSlash manifest in `codex-cli/bin/rg`, so the builder
can read that metadata directly and fetch the correct ripgrep archive
for the target it is packaging.

## What changed

- Added `scripts/codex_package/ripgrep.py` to parse `codex-cli/bin/rg`
after stripping the shebang, select the target platform entry, download
the configured artifact, and verify the recorded size and SHA-256
digest.
- Added a cache under `$TMPDIR/codex-package/<target>-rg` so verified
archives can be reused without fetching again.
- Extracted `rg`/`rg.exe` from `tar.gz` and `zip` artifacts into the
package-builder cache, then copied that into `codex-path` through the
existing package layout flow.
- Kept `--rg-bin` as an explicit local override for offline tests and
unusual local workflows.
- Documented the default `rg` fetch/cache behavior in
`scripts/codex_package/README.md`.

## Verification

- Ran wrapper/module syntax compilation.
- Ran `scripts/build_codex_package.py --help` from `/private/tmp`.
- Ran a local manifest fetch test covering shebang-stripped manifest
parsing, `tar.gz` extraction, `zip` extraction, size/SHA-256
verification, and cache reuse after deleting the original source
archives.
- Ran fake-cargo package/archive builds for macOS, Linux, and Windows
target layouts with `--rg-bin`, including an assertion that generated
tar archives contain no duplicate member names.




---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23526).
* #23541
* __->__ #23526
This commit is contained in:
Michael Bolin
2026-05-19 15:52:17 -07:00
committed by GitHub
parent 27c4c67b15
commit 59f262a2b4
4 changed files with 213 additions and 29 deletions

View File

@@ -14,6 +14,7 @@ class TargetSpec:
target: str
is_windows: bool
is_linux: bool
dotslash_platform: str
@property
def exe_suffix(self) -> str:
@@ -42,53 +43,43 @@ TARGET_SPECS: dict[str, TargetSpec] = {
target="x86_64-unknown-linux-musl",
is_windows=False,
is_linux=True,
dotslash_platform="linux-x86_64",
),
"aarch64-unknown-linux-musl": TargetSpec(
target="aarch64-unknown-linux-musl",
is_windows=False,
is_linux=True,
dotslash_platform="linux-aarch64",
),
"x86_64-apple-darwin": TargetSpec(
target="x86_64-apple-darwin",
is_windows=False,
is_linux=False,
dotslash_platform="macos-x86_64",
),
"aarch64-apple-darwin": TargetSpec(
target="aarch64-apple-darwin",
is_windows=False,
is_linux=False,
dotslash_platform="macos-aarch64",
),
"x86_64-pc-windows-msvc": TargetSpec(
target="x86_64-pc-windows-msvc",
is_windows=True,
is_linux=False,
dotslash_platform="windows-x86_64",
),
"aarch64-pc-windows-msvc": TargetSpec(
target="aarch64-pc-windows-msvc",
is_windows=True,
is_linux=False,
dotslash_platform="windows-aarch64",
),
}
def resolve_rg_bin(spec: TargetSpec, rg_bin: Path | None) -> Path:
return resolve_input_path(
rg_bin,
default_rg_candidates(spec),
"ripgrep executable",
"--rg-bin",
)
def default_rg_candidates(spec: TargetSpec) -> list[Path]:
return [
REPO_ROOT / "codex-cli" / "vendor" / spec.target / "path" / spec.rg_name,
]
def resolve_input_path(
explicit_path: Path | None,
default_candidates: list[Path],
description: str,
flag_name: str,
) -> Path:
@@ -100,14 +91,7 @@ def resolve_input_path(
raise RuntimeError(f"{description} is not executable: {path}")
return path
for candidate in default_candidates:
if candidate.is_file():
return candidate.resolve()
candidates = "\n".join(f" - {candidate}" for candidate in default_candidates)
raise RuntimeError(
f"Could not find {description}. Pass {flag_name}, or create one of:\n{candidates}"
)
raise RuntimeError(f"Must specify {flag_name} for {description}.")
def is_executable(path: Path) -> bool: