mirror of
https://github.com/openai/codex.git
synced 2026-05-25 13:34:51 +00:00
dcab40123f
## Summary - Add agent job support: spawn a batch of sub-agents from CSV, auto-run, auto-export, and store results in SQLite. - Simplify workflow: remove run/resume/get-status/export tools; spawn is deterministic and completes in one call. - Improve exec UX: stable, single-line progress bar with ETA; suppress sub-agent chatter in exec. ## Why Enables map-reduce style workflows over arbitrarily large repos using the existing Codex orchestrator. This addresses review feedback about overly complex job controls and non-deterministic monitoring. ## Demo (progress bar) ``` ./codex-rs/target/debug/codex exec \ --enable collab \ --enable sqlite \ --full-auto \ --progress-cursor \ -c agents.max_threads=16 \ -C /Users/daveaitel/code/codex \ - <<'PROMPT' Create /tmp/agent_job_progress_demo.csv with columns: path,area and 30 rows: path = item-01..item-30, area = test. Then call spawn_agents_on_csv with: - csv_path: /tmp/agent_job_progress_demo.csv - instruction: "Run `python - <<'PY'` to sleep a random 0.3–1.2s, then output JSON with keys: path, score (int). Set score = 1." - output_csv_path: /tmp/agent_job_progress_demo_out.csv PROMPT ``` ## Review feedback addressed - Auto-start jobs on spawn; removed run/resume/status/export tools. - Auto-export on success. - More descriptive tool spec + clearer prompts. - Avoid deadlocks on spawn failure; pending/running handled safely. - Progress bar no longer scrolls; stable single-line redraw. ## Tests - `cd codex-rs && cargo test -p codex-exec` - `cd codex-rs && cargo build -p codex-cli`
63 lines
2.1 KiB
Rust
63 lines
2.1 KiB
Rust
//! SQLite-backed state for rollout metadata.
|
|
//!
|
|
//! This crate is intentionally small and focused: it extracts rollout metadata
|
|
//! from JSONL rollouts and mirrors it into a local SQLite database. Backfill
|
|
//! orchestration and rollout scanning live in `codex-core`.
|
|
|
|
mod extract;
|
|
pub mod log_db;
|
|
mod migrations;
|
|
mod model;
|
|
mod paths;
|
|
mod runtime;
|
|
|
|
pub use model::LogEntry;
|
|
pub use model::LogQuery;
|
|
pub use model::LogRow;
|
|
pub use model::Phase2JobClaimOutcome;
|
|
/// Preferred entrypoint: owns configuration and metrics.
|
|
pub use runtime::StateRuntime;
|
|
|
|
/// Low-level storage engine: useful for focused tests.
|
|
///
|
|
/// Most consumers should prefer [`StateRuntime`].
|
|
pub use extract::apply_rollout_item;
|
|
pub use model::AgentJob;
|
|
pub use model::AgentJobCreateParams;
|
|
pub use model::AgentJobItem;
|
|
pub use model::AgentJobItemCreateParams;
|
|
pub use model::AgentJobItemStatus;
|
|
pub use model::AgentJobProgress;
|
|
pub use model::AgentJobStatus;
|
|
pub use model::Anchor;
|
|
pub use model::BackfillState;
|
|
pub use model::BackfillStats;
|
|
pub use model::BackfillStatus;
|
|
pub use model::ExtractionOutcome;
|
|
pub use model::SortKey;
|
|
pub use model::Stage1JobClaim;
|
|
pub use model::Stage1JobClaimOutcome;
|
|
pub use model::Stage1Output;
|
|
pub use model::Stage1StartupClaimParams;
|
|
pub use model::ThreadMetadata;
|
|
pub use model::ThreadMetadataBuilder;
|
|
pub use model::ThreadsPage;
|
|
pub use runtime::state_db_filename;
|
|
pub use runtime::state_db_path;
|
|
|
|
/// Environment variable for overriding the SQLite state database home directory.
|
|
pub const SQLITE_HOME_ENV: &str = "CODEX_SQLITE_HOME";
|
|
|
|
pub const STATE_DB_FILENAME: &str = "state";
|
|
pub const STATE_DB_VERSION: u32 = 5;
|
|
|
|
const METRIC_DB_INIT: &str = "codex.db.init";
|
|
/// Errors encountered during DB operations. Tags: [stage]
|
|
pub const DB_ERROR_METRIC: &str = "codex.db.error";
|
|
/// Metrics on backfill process. Tags: [status]
|
|
pub const DB_METRIC_BACKFILL: &str = "codex.db.backfill";
|
|
/// Metrics on backfill duration. Tags: [status]
|
|
pub const DB_METRIC_BACKFILL_DURATION_MS: &str = "codex.db.backfill.duration_ms";
|
|
/// Metrics on errors during comparison between DB and rollout file. Tags: [stage]
|
|
pub const DB_METRIC_COMPARE_ERROR: &str = "codex.db.compare_error";
|