mirror of
https://github.com/openai/codex.git
synced 2026-05-03 19:06:58 +00:00
## Summary - Stream proposed plans in Plan Mode using `<proposed_plan>` tags parsed in core, emitting plan deltas plus a plan `ThreadItem`, while stripping tags from normal assistant output. - Persist plan items and rebuild them on resume so proposed plans show in thread history. - Wire plan items/deltas through app-server protocol v2 and render a dedicated proposed-plan view in the TUI, including the “Implement this plan?” prompt only when a plan item is present. ## Changes ### Core (`codex-rs/core`) - Added a generic, line-based tag parser that buffers each line until it can disprove a tag prefix; implements auto-close on `finish()` for unterminated tags. `codex-rs/core/src/tagged_block_parser.rs` - Refactored proposed plan parsing to wrap the generic parser. `codex-rs/core/src/proposed_plan_parser.rs` - In plan mode, stream assistant deltas as: - **Normal text** → `AgentMessageContentDelta` - **Plan text** → `PlanDelta` + `TurnItem::Plan` start/completion (`codex-rs/core/src/codex.rs`) - Final plan item content is derived from the completed assistant message (authoritative), not necessarily the concatenated deltas. - Strips `<proposed_plan>` blocks from assistant text in plan mode so tags don’t appear in normal messages. (`codex-rs/core/src/stream_events_utils.rs`) - Persist `ItemCompleted` events only for plan items for rollout replay. (`codex-rs/core/src/rollout/policy.rs`) - Guard `update_plan` tool in Plan Mode with a clear error message. (`codex-rs/core/src/tools/handlers/plan.rs`) - Updated Plan Mode prompt to: - keep `<proposed_plan>` out of non-final reasoning/preambles - require exact tag formatting - allow only one `<proposed_plan>` block per turn (`codex-rs/core/templates/collaboration_mode/plan.md`) ### Protocol / App-server protocol - Added `TurnItem::Plan` and `PlanDeltaEvent` to core protocol items. (`codex-rs/protocol/src/items.rs`, `codex-rs/protocol/src/protocol.rs`) - Added v2 `ThreadItem::Plan` and `PlanDeltaNotification` with EXPERIMENTAL markers and note that deltas may not match the final plan item. (`codex-rs/app-server-protocol/src/protocol/v2.rs`) - Added plan delta route in app-server protocol common mapping. (`codex-rs/app-server-protocol/src/protocol/common.rs`) - Rebuild plan items from persisted `ItemCompleted` events on resume. (`codex-rs/app-server-protocol/src/protocol/thread_history.rs`) ### App-server - Forward plan deltas to v2 clients and map core plan items to v2 plan items. (`codex-rs/app-server/src/bespoke_event_handling.rs`, `codex-rs/app-server/src/codex_message_processor.rs`) - Added v2 plan item tests. (`codex-rs/app-server/tests/suite/v2/plan_item.rs`) ### TUI - Added a dedicated proposed plan history cell with special background and padding, and moved “• Proposed Plan” outside the highlighted block. (`codex-rs/tui/src/history_cell.rs`, `codex-rs/tui/src/style.rs`) - Only show “Implement this plan?” when a plan item exists. (`codex-rs/tui/src/chatwidget.rs`, `codex-rs/tui/src/chatwidget/tests.rs`) <img width="831" height="847" alt="Screenshot 2026-01-29 at 7 06 24 PM" src="https://github.com/user-attachments/assets/69794c8c-f96b-4d36-92ef-c1f5c3a8f286" /> ### Docs / Misc - Updated protocol docs to mention plan deltas. (`codex-rs/docs/protocol_v1.md`) - Minor plumbing updates in exec/debug clients to tolerate plan deltas. (`codex-rs/debug-client/src/reader.rs`, `codex-rs/exec/...`) ## Tests - Added core integration tests: - Plan mode strips plan from agent messages. - Missing `</proposed_plan>` closes at end-of-message. (`codex-rs/core/tests/suite/items.rs`) - Added unit tests for generic tag parser (prefix buffering, non-tag lines, auto-close). (`codex-rs/core/src/tagged_block_parser.rs`) - Existing app-server plan item tests in v2. (`codex-rs/app-server/tests/suite/v2/plan_item.rs`) ## Notes / Behavior - Plan output no longer appears in standard assistant text in Plan Mode; it streams via `PlanDelta` and completes as a `TurnItem::Plan`. - The final plan item content is authoritative and may diverge from streamed deltas (documented as experimental). - Reasoning summaries are not filtered; prompt instructs the model not to include `<proposed_plan>` outside the final plan message. ## Codex Author `codex fork 019bec2d-b09d-7450-b292-d7bcdddcdbfb`
327 lines
9.6 KiB
Rust
327 lines
9.6 KiB
Rust
use crate::history_cell::HistoryCell;
|
|
use crate::history_cell::{self};
|
|
use crate::render::line_utils::prefix_lines;
|
|
use crate::style::proposed_plan_style;
|
|
use ratatui::prelude::Stylize;
|
|
use ratatui::text::Line;
|
|
|
|
use super::StreamState;
|
|
|
|
/// Controller that manages newline-gated streaming, header emission, and
|
|
/// commit animation across streams.
|
|
pub(crate) struct StreamController {
|
|
state: StreamState,
|
|
finishing_after_drain: bool,
|
|
header_emitted: bool,
|
|
}
|
|
|
|
impl StreamController {
|
|
pub(crate) fn new(width: Option<usize>) -> Self {
|
|
Self {
|
|
state: StreamState::new(width),
|
|
finishing_after_drain: false,
|
|
header_emitted: false,
|
|
}
|
|
}
|
|
|
|
/// Push a delta; if it contains a newline, commit completed lines and start animation.
|
|
pub(crate) fn push(&mut self, delta: &str) -> bool {
|
|
let state = &mut self.state;
|
|
if !delta.is_empty() {
|
|
state.has_seen_delta = true;
|
|
}
|
|
state.collector.push_delta(delta);
|
|
if delta.contains('\n') {
|
|
let newly_completed = state.collector.commit_complete_lines();
|
|
if !newly_completed.is_empty() {
|
|
state.enqueue(newly_completed);
|
|
return true;
|
|
}
|
|
}
|
|
false
|
|
}
|
|
|
|
/// Finalize the active stream. Drain and emit now.
|
|
pub(crate) fn finalize(&mut self) -> Option<Box<dyn HistoryCell>> {
|
|
// Finalize collector first.
|
|
let remaining = {
|
|
let state = &mut self.state;
|
|
state.collector.finalize_and_drain()
|
|
};
|
|
// Collect all output first to avoid emitting headers when there is no content.
|
|
let mut out_lines = Vec::new();
|
|
{
|
|
let state = &mut self.state;
|
|
if !remaining.is_empty() {
|
|
state.enqueue(remaining);
|
|
}
|
|
let step = state.drain_all();
|
|
out_lines.extend(step);
|
|
}
|
|
|
|
// Cleanup
|
|
self.state.clear();
|
|
self.finishing_after_drain = false;
|
|
self.emit(out_lines)
|
|
}
|
|
|
|
/// Step animation: commit at most one queued line and handle end-of-drain cleanup.
|
|
pub(crate) fn on_commit_tick(&mut self) -> (Option<Box<dyn HistoryCell>>, bool) {
|
|
let step = self.state.step();
|
|
(self.emit(step), self.state.is_idle())
|
|
}
|
|
|
|
fn emit(&mut self, lines: Vec<Line<'static>>) -> Option<Box<dyn HistoryCell>> {
|
|
if lines.is_empty() {
|
|
return None;
|
|
}
|
|
Some(Box::new(history_cell::AgentMessageCell::new(lines, {
|
|
let header_emitted = self.header_emitted;
|
|
self.header_emitted = true;
|
|
!header_emitted
|
|
})))
|
|
}
|
|
}
|
|
|
|
/// Controller that streams proposed plan markdown into a styled plan block.
|
|
pub(crate) struct PlanStreamController {
|
|
state: StreamState,
|
|
header_emitted: bool,
|
|
top_padding_emitted: bool,
|
|
}
|
|
|
|
impl PlanStreamController {
|
|
pub(crate) fn new(width: Option<usize>) -> Self {
|
|
Self {
|
|
state: StreamState::new(width),
|
|
header_emitted: false,
|
|
top_padding_emitted: false,
|
|
}
|
|
}
|
|
|
|
/// Push a delta; if it contains a newline, commit completed lines and start animation.
|
|
pub(crate) fn push(&mut self, delta: &str) -> bool {
|
|
let state = &mut self.state;
|
|
if !delta.is_empty() {
|
|
state.has_seen_delta = true;
|
|
}
|
|
state.collector.push_delta(delta);
|
|
if delta.contains('\n') {
|
|
let newly_completed = state.collector.commit_complete_lines();
|
|
if !newly_completed.is_empty() {
|
|
state.enqueue(newly_completed);
|
|
return true;
|
|
}
|
|
}
|
|
false
|
|
}
|
|
|
|
/// Finalize the active stream. Drain and emit now.
|
|
pub(crate) fn finalize(&mut self) -> Option<Box<dyn HistoryCell>> {
|
|
let remaining = {
|
|
let state = &mut self.state;
|
|
state.collector.finalize_and_drain()
|
|
};
|
|
let mut out_lines = Vec::new();
|
|
{
|
|
let state = &mut self.state;
|
|
if !remaining.is_empty() {
|
|
state.enqueue(remaining);
|
|
}
|
|
let step = state.drain_all();
|
|
out_lines.extend(step);
|
|
}
|
|
|
|
self.state.clear();
|
|
self.emit(out_lines, true)
|
|
}
|
|
|
|
/// Step animation: commit at most one queued line and handle end-of-drain cleanup.
|
|
pub(crate) fn on_commit_tick(&mut self) -> (Option<Box<dyn HistoryCell>>, bool) {
|
|
let step = self.state.step();
|
|
(self.emit(step, false), self.state.is_idle())
|
|
}
|
|
|
|
fn emit(
|
|
&mut self,
|
|
lines: Vec<Line<'static>>,
|
|
include_bottom_padding: bool,
|
|
) -> Option<Box<dyn HistoryCell>> {
|
|
if lines.is_empty() && !include_bottom_padding {
|
|
return None;
|
|
}
|
|
|
|
let mut out_lines: Vec<Line<'static>> = Vec::new();
|
|
let is_stream_continuation = self.header_emitted;
|
|
if !self.header_emitted {
|
|
out_lines.push(vec!["• ".dim(), "Proposed Plan".bold()].into());
|
|
out_lines.push(Line::from(" "));
|
|
self.header_emitted = true;
|
|
}
|
|
|
|
let mut plan_lines: Vec<Line<'static>> = Vec::new();
|
|
if !self.top_padding_emitted {
|
|
plan_lines.push(Line::from(" "));
|
|
self.top_padding_emitted = true;
|
|
}
|
|
plan_lines.extend(lines);
|
|
if include_bottom_padding {
|
|
plan_lines.push(Line::from(" "));
|
|
}
|
|
|
|
let plan_style = proposed_plan_style();
|
|
let plan_lines = prefix_lines(plan_lines, " ".into(), " ".into())
|
|
.into_iter()
|
|
.map(|line| line.style(plan_style))
|
|
.collect::<Vec<_>>();
|
|
out_lines.extend(plan_lines);
|
|
|
|
Some(Box::new(history_cell::new_proposed_plan_stream(
|
|
out_lines,
|
|
is_stream_continuation,
|
|
)))
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
fn lines_to_plain_strings(lines: &[ratatui::text::Line<'_>]) -> Vec<String> {
|
|
lines
|
|
.iter()
|
|
.map(|l| {
|
|
l.spans
|
|
.iter()
|
|
.map(|s| s.content.clone())
|
|
.collect::<Vec<_>>()
|
|
.join("")
|
|
})
|
|
.collect()
|
|
}
|
|
|
|
#[tokio::test]
|
|
async fn controller_loose_vs_tight_with_commit_ticks_matches_full() {
|
|
let mut ctrl = StreamController::new(None);
|
|
let mut lines = Vec::new();
|
|
|
|
// Exact deltas from the session log (section: Loose vs. tight list items)
|
|
let deltas = vec![
|
|
"\n\n",
|
|
"Loose",
|
|
" vs",
|
|
".",
|
|
" tight",
|
|
" list",
|
|
" items",
|
|
":\n",
|
|
"1",
|
|
".",
|
|
" Tight",
|
|
" item",
|
|
"\n",
|
|
"2",
|
|
".",
|
|
" Another",
|
|
" tight",
|
|
" item",
|
|
"\n\n",
|
|
"1",
|
|
".",
|
|
" Loose",
|
|
" item",
|
|
" with",
|
|
" its",
|
|
" own",
|
|
" paragraph",
|
|
".\n\n",
|
|
" ",
|
|
" This",
|
|
" paragraph",
|
|
" belongs",
|
|
" to",
|
|
" the",
|
|
" same",
|
|
" list",
|
|
" item",
|
|
".\n\n",
|
|
"2",
|
|
".",
|
|
" Second",
|
|
" loose",
|
|
" item",
|
|
" with",
|
|
" a",
|
|
" nested",
|
|
" list",
|
|
" after",
|
|
" a",
|
|
" blank",
|
|
" line",
|
|
".\n\n",
|
|
" ",
|
|
" -",
|
|
" Nested",
|
|
" bullet",
|
|
" under",
|
|
" a",
|
|
" loose",
|
|
" item",
|
|
"\n",
|
|
" ",
|
|
" -",
|
|
" Another",
|
|
" nested",
|
|
" bullet",
|
|
"\n\n",
|
|
];
|
|
|
|
// Simulate streaming with a commit tick attempt after each delta.
|
|
for d in deltas.iter() {
|
|
ctrl.push(d);
|
|
while let (Some(cell), idle) = ctrl.on_commit_tick() {
|
|
lines.extend(cell.transcript_lines(u16::MAX));
|
|
if idle {
|
|
break;
|
|
}
|
|
}
|
|
}
|
|
// Finalize and flush remaining lines now.
|
|
if let Some(cell) = ctrl.finalize() {
|
|
lines.extend(cell.transcript_lines(u16::MAX));
|
|
}
|
|
|
|
let streamed: Vec<_> = lines_to_plain_strings(&lines)
|
|
.into_iter()
|
|
// skip • and 2-space indentation
|
|
.map(|s| s.chars().skip(2).collect::<String>())
|
|
.collect();
|
|
|
|
// Full render of the same source
|
|
let source: String = deltas.iter().copied().collect();
|
|
let mut rendered: Vec<ratatui::text::Line<'static>> = Vec::new();
|
|
crate::markdown::append_markdown(&source, None, &mut rendered);
|
|
let rendered_strs = lines_to_plain_strings(&rendered);
|
|
|
|
assert_eq!(streamed, rendered_strs);
|
|
|
|
// Also assert exact expected plain strings for clarity.
|
|
let expected = vec![
|
|
"Loose vs. tight list items:".to_string(),
|
|
"".to_string(),
|
|
"1. Tight item".to_string(),
|
|
"2. Another tight item".to_string(),
|
|
"3. Loose item with its own paragraph.".to_string(),
|
|
"".to_string(),
|
|
" This paragraph belongs to the same list item.".to_string(),
|
|
"4. Second loose item with a nested list after a blank line.".to_string(),
|
|
" - Nested bullet under a loose item".to_string(),
|
|
" - Another nested bullet".to_string(),
|
|
];
|
|
assert_eq!(
|
|
streamed, expected,
|
|
"expected exact rendered lines for loose/tight section"
|
|
);
|
|
}
|
|
}
|