Commit Graph

7 Commits

Author SHA1 Message Date
Michael Bolin
80c4a978f8 release: package prebuilt resource binaries (#23759)
## Why

Release packaging should be a staging step once release binaries have
already been built and signed. The Windows release job was downloading
and signing `codex-command-runner.exe` and
`codex-windows-sandbox-setup.exe`, but `scripts/build_codex_package.py`
still rebuilt those helpers while creating the package archives.

That makes the package step slower and, more importantly, risks putting
helper binaries in the archive that were produced after the signing
step. Linux had the same shape for package resources: `bwrap` could be
rebuilt by the package builder instead of being passed in as a prebuilt
release artifact.

This builds on #23752, which fixes `.tar.zst` creation when Windows
runners rely on the repository DotSlash `zstd` wrapper.

## What changed

- Add explicit prebuilt resource inputs to the Codex package builder:
  - `--bwrap-bin`
  - `--codex-command-runner-bin`
  - `--codex-windows-sandbox-setup-bin`
- Make `.github/scripts/build-codex-package-archive.sh` pass resource
binaries from the release output directory when they are already
present.
- Build Linux `bwrap` for app-server release jobs too, so app-server
package creation does not invoke Cargo just to supply the package
resource.
- Keep macOS package creation as a no-Cargo path when `--entrypoint-bin`
is provided, since macOS packages have no resource binaries.
- Add unit coverage showing prebuilt macOS, Linux, and Windows package
inputs result in no source-built binaries.

## Verification

- `python3 -m unittest discover -s scripts/codex_package -p 'test_*.py'`
- `python3 -m py_compile scripts/codex_package/*.py`
- `bash -n .github/scripts/build-codex-package-archive.sh`
- Dry-ran Linux and Windows package builds with fake prebuilt resources
and a nonexistent Cargo path to verify the package builder did not
invoke Cargo.


---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23759).
* #23760
* __->__ #23759
2026-05-20 14:51:46 -07:00
Michael Bolin
f48be015d6 release: use DotSlash zstd for package archives (#23752)
## Why

The Windows release job installed DotSlash successfully, but package
archive creation still failed while writing `codex-package-*.tar.zst`.
The Python archiver used `shutil.which("zstd")`, which does not reliably
find the extensionless DotSlash manifest at `.github/workflows/zstd`
from native Windows Python.

That left release packaging dependent on a command named exactly `zstd`
being discoverable on `PATH`, even though the repository already carries
a DotSlash wrapper for Windows runners.

## What changed

- Add `resolve_zstd_command()` to prefer a real `zstd` binary when
present.
- Fall back to invoking `dotslash .github/workflows/zstd` when `zstd` is
not on `PATH`.
- Keep the error explicit when neither `zstd` nor the DotSlash fallback
is available.
- Add unit coverage for direct `zstd`, DotSlash fallback, and
missing-tool error paths.

## Verification

- `python3 -m unittest discover -s scripts/codex_package -p 'test_*.py'`
- `python3 -m py_compile scripts/codex_package/*.py`
2026-05-20 14:28:11 -07:00
Michael Bolin
cb05de6724 dotslash: publish Codex entrypoints from package archives (#23638)
## Summary

DotSlash should resolve the same canonical package archives used by
standalone installers and npm platform packages, rather than continuing
to point at single-binary zstd artifacts or the older Linux bundle
archive.

This updates the Codex CLI and `codex-app-server` DotSlash release
config entries to match `codex-package-<target>.tar.gz` and
`codex-app-server-package-<target>.tar.gz`, with paths that select
`bin/codex` or `bin/codex-app-server` inside the extracted package. The
other helper outputs stay on their existing per-binary artifacts for
now.

## Test plan

- `python3 -m json.tool .github/dotslash-config.json > /dev/null`
- Ran a Python regex smoke test that checked every updated `codex` and
`codex-app-server` platform entry against the archive names emitted by
`.github/scripts/build-codex-package-archive.sh`.
2026-05-20 12:18:10 -07:00
Michael Bolin
343a74076f build: package prebuilt Codex entrypoints (#23586)
## Why

The package builder should describe the binaries it is actually
packaging, not require callers to restate release metadata out of band.
A caller-provided `--version` flag can drift from the workspace version,
but running the target entrypoint to discover its version breaks
cross-target packages when the produced binary cannot execute on the
build host.

This PR keeps package metadata tied to the repository source of truth by
reading `[workspace.package].version` from `codex-rs/Cargo.toml`. It
also prepares the package layout for `codex-app-server` packages: the
same package structure can now represent either the CLI entrypoint or
the app-server entrypoint while keeping shared sidecars such as `rg`,
`bwrap`, and Windows sandbox helpers in the existing package
directories.

## What changed

- Removes the `--version` CLI flag from
`scripts/build_codex_package.py`.
- Adds Cargo.toml version discovery for `codex-package.json.version` via
`codex-rs/Cargo.toml`.
- Adds `--entrypoint-bin` so callers can package a prebuilt entrypoint
instead of rebuilding it with Cargo.
- Makes `--variant` an explicit choice between `codex` and
`codex-app-server`, and uses it to select the cargo binary and packaged
`bin/` entrypoint name.
- Updates `scripts/codex_package/README.md` to document variants,
prebuilt entrypoints, and Cargo.toml version detection.

## Verification

- Compiled `scripts/build_codex_package.py` and
`scripts/codex_package/*.py` with `PYTHONDONTWRITEBYTECODE=1`.
- Ran `scripts/build_codex_package.py --help` and verified `--version`
is gone while `--variant` and `--entrypoint-bin` are present.
- Verified the package builder reads version `0.0.0` from
`codex-rs/Cargo.toml`.
- Built a fake cross-target `codex-app-server` package using a
non-executable `--entrypoint-bin`; verified metadata records version
`0.0.0`, variant `codex-app-server`, and `bin/codex-app-server` as the
entrypoint.
2026-05-19 22:10:03 -07:00
Michael Bolin
79f044ed34 build: default Codex package target and output (#23541)
## Why

The package builder should be easy to run during local iteration.
Requiring callers to provide both a target triple and an output
directory every time makes the common host-package case more awkward
than necessary.

This PR keeps explicit overrides available, but makes the default
invocation useful: build for the current host platform and place the
package in a fresh temporary directory. Because a temp output path is
otherwise easy to lose, the builder continues to print the final package
directory path when it completes.

## What changed

- Makes `--target` optional and maps the host OS/architecture to
supported Codex package target triples.
- Uses GNU Linux target triples for Linux host defaults, while keeping
the musl targets available for release jobs that pass `--target`
explicitly.
- Makes `--package-dir` optional and creates a new `codex-package-*`
temp directory when omitted.
- Documents the new defaults in `scripts/codex_package/README.md`.

## Verification

- Compiled `scripts/build_codex_package.py` and
`scripts/codex_package/*.py` with `PYTHONDONTWRITEBYTECODE=1`.
- Ran `scripts/build_codex_package.py --help` from outside the repo.
- Verified Linux host detection maps `x86_64` and `aarch64` to GNU
target triples.
- Ran a fake-Cargo package build while omitting both `--target` and
`--package-dir`; verified the generated metadata target, expected
package files, and printed temp package path.
- Ran a fake-Cargo package build for `x86_64-unknown-linux-gnu` and
verified `codex`, `bwrap`, and `rg` are assembled into the package.
2026-05-20 00:05:43 +00:00
Michael Bolin
59f262a2b4 build: fetch rg for Codex packages (#23526)
## Why

The Codex package builder should produce a complete package without
requiring callers to pre-populate `rg` under `codex-cli/vendor` or have
`dotslash` installed on `PATH`. The repo already tracks the
authoritative DotSlash manifest in `codex-cli/bin/rg`, so the builder
can read that metadata directly and fetch the correct ripgrep archive
for the target it is packaging.

## What changed

- Added `scripts/codex_package/ripgrep.py` to parse `codex-cli/bin/rg`
after stripping the shebang, select the target platform entry, download
the configured artifact, and verify the recorded size and SHA-256
digest.
- Added a cache under `$TMPDIR/codex-package/<target>-rg` so verified
archives can be reused without fetching again.
- Extracted `rg`/`rg.exe` from `tar.gz` and `zip` artifacts into the
package-builder cache, then copied that into `codex-path` through the
existing package layout flow.
- Kept `--rg-bin` as an explicit local override for offline tests and
unusual local workflows.
- Documented the default `rg` fetch/cache behavior in
`scripts/codex_package/README.md`.

## Verification

- Ran wrapper/module syntax compilation.
- Ran `scripts/build_codex_package.py --help` from `/private/tmp`.
- Ran a local manifest fetch test covering shebang-stripped manifest
parsing, `tar.gz` extraction, `zip` extraction, size/SHA-256
verification, and cache reuse after deleting the original source
archives.
- Ran fake-cargo package/archive builds for macOS, Linux, and Windows
target layouts with `--rg-bin`, including an assertion that generated
tar archives contain no duplicate member names.




---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23526).
* #23541
* __->__ #23526
2026-05-19 15:52:17 -07:00
Michael Bolin
7f4d7ae3a4 build: add Codex package builder (#23513)
## Why

Codex CLI packaging is currently split across npm staging, standalone
installers, and release bundle creation, which makes it hard to define
and validate a single valid package directory. This adds the first
standalone package builder so later release paths can converge on the
same canonical layout.

## What changed

- Added `scripts/build_codex_package.py` as the stable executable
wrapper around `scripts/codex_package`.
- Added modules for CLI parsing, target metadata, grouped cargo builds,
package layout validation, and archive writing.
- The builder creates a package directory with `codex-package.json`,
`bin/`, `codex-resources/`, and `codex-path`, and can serialize it as
`.tar.gz`, `.tar.zst`, or `.zip`.
- Source-built artifacts are built by one grouped `cargo build`: `codex`
for all targets, `bwrap` for Linux, and the Windows sandbox helpers for
Windows. `rg` remains an input because it is vendored from upstream
rather than built from this repo.
- Added `scripts/codex_package/README.md` to document the package
layout, source-built artifacts, and cargo profile behavior.

## Verification

- Ran wrapper/module syntax compilation.
- Ran `scripts/build_codex_package.py --help` from `/private/tmp`.
- Ran fake-cargo package/archive builds for macOS, Linux, and Windows
target layouts, including an assertion that generated tar archives
contain no duplicate member names.


---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23513).
* #23526
* __->__ #23513
2026-05-19 19:54:03 +00:00