mirror of
https://github.com/openai/codex.git
synced 2026-05-17 17:53:06 +00:00
e7e6267ab3
## Summary Moves the WebRTC realtime sideband websocket join out of the voice start critical path. Call creation still posts the SDP offer and session config synchronously so the client gets the SDP answer, but the sideband websocket now connects in the input task async and doesn't block conversation state installation. This lets the normal realtime input channels buffer text, handoff output, and audio while the WebRTC sideband websocket is connecting. If the sideband join fails while the conversation is still active, the task sends a RealtimeEvent::Error through the existing events_tx / fanout path. To rephrase this: * No longer blocked on sideband: the client can receive the SDP answer earlier, set up the WebRTC peer connection, and let the media leg progress while the sideband websocket joins. * Still blocked on sideband: queued text, handoff output, and sideband server events cannot flow until connect_webrtc_sideband(...).await finishes and then run_realtime_input_task(...) starts ## Validation - `env CODEX_SKIP_VENDORED_BWRAP=1 cargo test --manifest-path codex-rs/Cargo.toml -p codex-core --test all conversation_webrtc_start_posts_generated_session` `CODEX_SKIP_VENDORED_BWRAP=1` is needed in this local environment because `libcap.pc` is not installed for the vendored bubblewrap build. ## Testing I tested this locally by running `cargo run -p codex-cli --bin codex -- --enable realtime_conversation` and invoking `/realtime`. Then, we get logs emitted in `~/.codex/log/codex-tui.log`. ### Before the Change Logging commit (c0299e6edf) ``` 2026-05-04T16:06:09.251956Z INFO session_loop{thread_id=019df3b9-e3d8-7271-b13a-b880119aa4c2}:submission_dispatch{otel.name="op.dispatch.realtime_conversation_start" submission.id="019df3bd-65df-7ee2-8125-1d6701fe39d2" codex.op="realtime_conversation_start"}: codex_core::realtime_conversation: starting realtime conversation 2026-05-04T16:06:09.251980Z INFO session_loop{thread_id=019df3b9-e3d8-7271-b13a-b880119aa4c2}:submission_dispatch{otel.name="op.dispatch.realtime_conversation_start" submission.id="019df3bd-65df-7ee2-8125-1d6701fe39d2" codex.op="realtime_conversation_start"}: codex_core::realtime_conversation: creating realtime call transport="webrtc" 2026-05-04T16:06:10.365722Z INFO session_loop{thread_id=019df3b9-e3d8-7271-b13a-b880119aa4c2}:submission_dispatch{otel.name="op.dispatch.realtime_conversation_start" submission.id="019df3bd-65df-7ee2-8125-1d6701fe39d2" codex.op="realtime_conversation_start"}: codex_core::realtime_conversation: realtime call created; sdp answer ready transport="webrtc" call_id=rtc_u0_Dbq65nhak5eLjQZ73yhAy elapsed_ms=1113 total_elapsed_ms=1113 2026-05-04T16:06:10.365843Z INFO session_loop{thread_id=019df3b9-e3d8-7271-b13a-b880119aa4c2}:submission_dispatch{otel.name="op.dispatch.realtime_conversation_start" submission.id="019df3bd-65df-7ee2-8125-1d6701fe39d2" codex.op="realtime_conversation_start"}: codex_core::realtime_conversation: connecting realtime sideband websocket call_id=rtc_u0_Dbq65nhak5eLjQZ73yhAy 2026-05-04T16:06:10.784528Z INFO session_loop{thread_id=019df3b9-e3d8-7271-b13a-b880119aa4c2}:submission_dispatch{otel.name="op.dispatch.realtime_conversation_start" submission.id="019df3bd-65df-7ee2-8125-1d6701fe39d2" codex.op="realtime_conversation_start"}: codex_core::realtime_conversation: connected realtime sideband websocket call_id=rtc_u0_Dbq65nhak5eLjQZ73yhAy elapsed_ms=418 total_elapsed_ms=1532 2026-05-04T16:06:10.784665Z INFO session_loop{thread_id=019df3b9-e3d8-7271-b13a-b880119aa4c2}:submission_dispatch{otel.name="op.dispatch.realtime_conversation_start" submission.id="019df3bd-65df-7ee2-8125-1d6701fe39d2" codex.op="realtime_conversation_start"}: codex_core::realtime_conversation: realtime conversation started ``` ### After the Change Logging commit (c8b00ac21a) ``` 2026-05-04T15:41:24.080363Z INFO ... codex_core::realtime_conversation: starting realtime conversation 2026-05-04T15:41:24.080434Z INFO ... codex_core::realtime_conversation: creating realtime call transport="webrtc" 2026-05-04T15:41:25.106906Z INFO ... codex_core::realtime_conversation: realtime call created; sdp answer ready transport="webrtc" call_id=rtc_u0_Dbpi8nhak5eLjQZ73yhAy elapsed_ms=1026 total_elapsed_ms=1026 2026-05-04T15:41:25.107067Z INFO ... codex_core::realtime_conversation: spawned realtime sideband connection task transport="webrtc" total_elapsed_ms=1026 2026-05-04T15:41:25.107160Z INFO ... codex_core::realtime_conversation: realtime conversation started 2026-05-04T15:41:25.107185Z INFO codex_core::realtime_conversation: connecting realtime sideband websocket call_id=rtc_u0_Dbpi8nhak5eLjQZ73yhAy 2026-05-04T15:41:25.107352Z INFO ... codex_core::realtime_conversation: sent realtime sdp answer to client 2026-05-04T15:41:26.076685Z INFO codex_core::realtime_conversation: connected realtime sideband websocket call_id=rtc_u0_Dbpi8nhak5eLjQZ73yhAy elapsed_ms=969 total_elapsed_ms=1996 2026-05-04T15:41:26.573893Z INFO codex_core::realtime_conversation: realtime session updated realtime_session_id=sess_u0_Dbpi8nhak5eLjQZ73yhAy 2026-05-04T15:41:26.573970Z INFO codex_core::realtime_conversation: received realtime conversation event event=SessionUpdated { ... } ``` ### Conclusion Here we see that we saved about a half a second in conversation startup (1532ms -> 969ms). This also checks out with my sanity tests; I was seeing at most a second of saving. --------- Co-authored-by: Codex <noreply@openai.com>