2 Commits

Author	SHA1	Message	Date
jgershen-oai	9e905528bb	Fix custom CA login behind TLS-inspecting proxies (#20676) ... Refs: https://linear.app/openai/issue/SE-6311/login-fails-for-experian-users-behind-tls-inspecting-proxy ## Summary - When a custom CA bundle is configured, force the shared `codex-client` reqwest builder onto rustls before registering custom roots. - Add the `rustls-tls-native-roots` reqwest feature so the rustls client preserves native roots plus the enterprise CA bundle. - Add subprocess TLS coverage for both a direct local TLS 1.3 server and a hermetic local CONNECT TLS-intercepting proxy that forwards a token-exchange-shaped POST to a local origin. ## Plain-language explanation Experian users are behind a TLS-inspecting proxy, so the login token exchange needs to trust the enterprise CA bundle from `CODEX_CA_CERTIFICATE` or `SSL_CERT_FILE`. Before this change, that custom-CA branch still used reqwest default TLS selection, which could fail in the proxy environment. Now, only when a custom CA is configured, Codex selects rustls first and then adds the custom CA roots, matching the validated behavior from the Experian test build while leaving normal system-root clients unchanged. The new regression test recreates the enterprise-proxy shape locally: the probe client sends an HTTPS `POST /oauth/token` through an explicit HTTP CONNECT proxy, the proxy presents a leaf certificate signed by a runtime-generated test CA, decrypts the request, forwards it to a local origin, and relays the `ok` response back. ## Scope note - The actual production fix is the first commit: `8368119282 Fix custom CA reqwest clients to use rustls`. - The second commit is integration-test coverage only. It generates all test CA and localhost certificate material at runtime. ## Validation - `cd codex-rs && cargo test -p codex-client --test ca_env posts_to_token_origin_through_tls_intercepting_proxy_with_custom_ca_bundle -- --nocapture` - `cd codex-rs && cargo test -p codex-client` - `cd codex-rs && cargo test -p codex-login` - `cd codex-rs && just fmt` - `cd codex-rs && just bazel-lock-update` - `cd codex-rs && just bazel-lock-check` - `cd codex-rs && just fix -p codex-client`	2026-05-01 17:51:49 -07:00
Josh McKinney	6912da84a8	client: extend custom CA handling across HTTPS and websocket clients (#14239) ... ## Stacked PRs This work is now effectively split across two steps: - #14178: add custom CA support for browser and device-code login flows, docs, and hermetic subprocess tests - #14239: extend that shared custom CA handling across Codex HTTPS clients and secure websocket TLS Note: #14240 was merged into this branch while it was stacked on top of this PR. This PR now subsumes that websocket follow-up and should be treated as the combined change. Builds on top of #14178. ## Problem Custom CA support landed first in the login path, but the real requirement is broader. Codex constructs outbound TLS clients in multiple places, and both HTTPS and secure websocket paths can fail behind enterprise TLS interception if they do not honor `CODEX_CA_CERTIFICATE` or `SSL_CERT_FILE` consistently. This PR broadens the shared custom-CA logic beyond login and applies the same policy to websocket TLS, so the enterprise-proxy story is no longer split between “HTTPS works” and “websockets still fail”. ## What This Delivers Custom CA support is no longer limited to login. Codex outbound HTTPS clients and secure websocket connections can now honor the same `CODEX_CA_CERTIFICATE` / `SSL_CERT_FILE` configuration, so enterprise proxy/intercept setups work more consistently end-to-end. For users and operators, nothing new needs to be configured beyond the same CA env vars introduced in #14178. The change is that more of Codex now respects them, including websocket-backed flows that were previously still using default trust roots. I also manually validated the proxy path locally with mitmproxy using: `CODEX_CA_CERTIFICATE=~/.mitmproxy/mitmproxy-ca-cert.pem HTTPS_PROXY=http://127.0.0.1:8080 just codex` with mitmproxy installed via `brew install mitmproxy` and configured as the macOS system proxy. ## Mental model `codex-client` is now the owner of shared custom-CA policy for outbound TLS client construction. Reqwest callers start from the builder configuration they already need, then pass that builder through `build_reqwest_client_with_custom_ca(...)`. Websocket callers ask the same module for a rustls client config when a custom CA bundle is configured. The env precedence is the same everywhere: - `CODEX_CA_CERTIFICATE` wins - otherwise fall back to `SSL_CERT_FILE` - otherwise use system roots The helper is intentionally narrow. It loads every usable certificate from the configured PEM bundle into the appropriate root store and returns either a configured transport or a typed error that explains what went wrong. ## Non-goals This does not add handshake-level integration tests against a live TLS endpoint. It does not validate that the configured bundle forms a meaningful certificate chain. It also does not try to force every transport in the repo through one abstraction; it extends the shared CA policy across the reqwest and websocket paths that actually needed it. ## Tradeoffs The main tradeoff is centralizing CA behavior in `codex-client` while still leaving adoption up to call sites. That keeps the implementation additive and reviewable, but it means the rule "outbound Codex TLS that should honor enterprise roots must use the shared helper" is still partly enforced socially rather than by types. For websockets, the shared helper only builds an explicit rustls config when a custom CA bundle is configured. When no override env var is set, websocket callers still use their ordinary default connector path. ## Architecture `codex-client::custom_ca` now owns CA bundle selection, PEM normalization, mixed-section parsing, certificate extraction, typed CA-loading errors, and optional rustls client-config construction for websocket TLS. The affected consumers now call into that shared helper directly rather than carrying login-local CA behavior: - backend-client - cloud-tasks - RMCP client paths that use `reqwest` - TUI voice HTTP paths - `codex-core` default reqwest client construction - `codex-api` websocket clients for both responses and realtime websocket connections The subprocess CA probe, env-sensitive integration tests, and shared PEM fixtures also live in `codex-client`, which is now the actual owner of the behavior they exercise. ## Observability The shared CA path logs: - which environment variable selected the bundle - which path was loaded - how many certificates were accepted - when `TRUSTED CERTIFICATE` labels were normalized - when CRLs were ignored - where client construction failed Returned errors remain user-facing and include the relevant env var, path, and remediation hint. That same error model now applies whether the failure surfaced while building a reqwest client or websocket TLS configuration. ## Tests Pure unit tests in `codex-client` cover env precedence and PEM normalization behavior. Real client construction remains in subprocess tests so the suite can control process env and avoid the macOS seatbelt panic path that motivated the hermetic test split. The subprocess coverage verifies: - `CODEX_CA_CERTIFICATE` precedence over `SSL_CERT_FILE` - fallback to `SSL_CERT_FILE` - single-cert and multi-cert bundles - malformed and empty-file errors - OpenSSL `TRUSTED CERTIFICATE` handling - CRL tolerance for well-formed CRL sections The websocket side is covered by the existing `codex-api` / `codex-core` websocket test suites plus the manual mitmproxy validation above. --------- Co-authored-by: Ivan Zakharchanka <3axap4eHko@gmail.com> Co-authored-by: Codex <noreply@openai.com>	2026-03-13 00:59:26 +00:00