mirror of
https://github.com/openai/codex.git
synced 2026-05-24 21:14:51 +00:00
ba57aab13a
## Why Thread goals are moving toward extension-owned runtime behavior, but their persisted state was still stored in the shared state database. This makes the goal store harder to isolate and keeps future storage splits tied to ad hoc runtime plumbing. This PR gives goals their own SQLite database while keeping the existing `StateRuntime` entry point. The goal is to make this the pattern for adding more dedicated runtime databases later. This also reduce load on existing DB and reduce contention ## Limitation Thread preview from goal is not supported anymore. I'm looking into this [EDIT]: solved ## What changed - Added a dedicated `goals_1.sqlite` database with its own `goals_migrations` directory. - Moved `thread_goals` creation into the goals DB migration set. - Dropped the old `thread_goals` table from the main state DB with a normal state migration. There is intentionally no backfill for existing goal rows. - Changed `GoalStore` to be backed only by the goals DB pool. - Removed the old goal-write side effect that filled empty `threads.preview` values from the goal objective. - Added shared runtime DB path metadata so startup, telemetry, `codex doctor`, and repair handling can include future DBs without bespoke path lists. - Updated Bazel compile data so the new goals migration directory is available to `sqlx::migrate!`. ## Verification - `cargo check --tests -p codex-state -p codex-cli -p codex-core -p codex-app-server` - `just fix -p codex-state` - `just fix -p codex-cli` - `just fix -p codex-app-server`
91 lines
3.3 KiB
Rust
91 lines
3.3 KiB
Rust
//! SQLite-backed state for rollout metadata.
|
|
//!
|
|
//! This crate is intentionally small and focused: it extracts rollout metadata
|
|
//! from JSONL rollouts and mirrors it into a local SQLite database. Backfill
|
|
//! orchestration and rollout scanning live in `codex-core`.
|
|
|
|
mod extract;
|
|
pub mod log_db;
|
|
mod migrations;
|
|
mod model;
|
|
mod paths;
|
|
mod runtime;
|
|
mod telemetry;
|
|
|
|
pub use model::LogEntry;
|
|
pub use model::LogQuery;
|
|
pub use model::LogRow;
|
|
pub use model::Phase2JobClaimOutcome;
|
|
/// Preferred entrypoint: owns configuration and metrics.
|
|
pub use runtime::StateRuntime;
|
|
|
|
/// Low-level storage engine: useful for focused tests.
|
|
///
|
|
/// Most consumers should prefer [`StateRuntime`].
|
|
pub use extract::apply_rollout_item;
|
|
pub use extract::rollout_item_affects_thread_metadata;
|
|
pub use model::AgentJob;
|
|
pub use model::AgentJobCreateParams;
|
|
pub use model::AgentJobItem;
|
|
pub use model::AgentJobItemCreateParams;
|
|
pub use model::AgentJobItemStatus;
|
|
pub use model::AgentJobProgress;
|
|
pub use model::AgentJobStatus;
|
|
pub use model::Anchor;
|
|
pub use model::BackfillState;
|
|
pub use model::BackfillStats;
|
|
pub use model::BackfillStatus;
|
|
pub use model::DirectionalThreadSpawnEdgeStatus;
|
|
pub use model::ExtractionOutcome;
|
|
pub use model::SortDirection;
|
|
pub use model::SortKey;
|
|
pub use model::Stage1JobClaim;
|
|
pub use model::Stage1JobClaimOutcome;
|
|
pub use model::Stage1Output;
|
|
pub use model::Stage1StartupClaimParams;
|
|
pub use model::ThreadGoal;
|
|
pub use model::ThreadGoalStatus;
|
|
pub use model::ThreadMetadata;
|
|
pub use model::ThreadMetadataBuilder;
|
|
pub use model::ThreadsPage;
|
|
pub use runtime::GoalStore;
|
|
pub use runtime::RemoteControlEnrollmentRecord;
|
|
pub use runtime::RuntimeDbPath;
|
|
pub use runtime::ThreadFilterOptions;
|
|
pub use runtime::ThreadGoalAccountingMode;
|
|
pub use runtime::ThreadGoalAccountingOutcome;
|
|
pub use runtime::ThreadGoalUpdate;
|
|
pub use runtime::goals_db_filename;
|
|
pub use runtime::goals_db_path;
|
|
pub use runtime::logs_db_filename;
|
|
pub use runtime::logs_db_path;
|
|
pub use runtime::runtime_db_paths;
|
|
pub use runtime::sqlite_integrity_check;
|
|
pub use runtime::state_db_filename;
|
|
pub use runtime::state_db_path;
|
|
pub use telemetry::DbTelemetry;
|
|
pub use telemetry::DbTelemetryHandle;
|
|
pub use telemetry::install_process_db_telemetry;
|
|
pub use telemetry::record_backfill_gate;
|
|
pub use telemetry::record_fallback;
|
|
|
|
/// Environment variable for overriding the SQLite state database home directory.
|
|
pub const SQLITE_HOME_ENV: &str = "CODEX_SQLITE_HOME";
|
|
|
|
pub const LOGS_DB_FILENAME: &str = "logs_2.sqlite";
|
|
pub const GOALS_DB_FILENAME: &str = "goals_1.sqlite";
|
|
pub const STATE_DB_FILENAME: &str = "state_5.sqlite";
|
|
|
|
/// Errors encountered during DB operations. Tags: [stage]
|
|
pub const DB_ERROR_METRIC: &str = "codex.db.error";
|
|
/// Metrics on backfill process. Tags: [status]
|
|
pub const DB_METRIC_BACKFILL: &str = "codex.db.backfill";
|
|
/// Metrics on backfill duration. Tags: [status]
|
|
pub const DB_METRIC_BACKFILL_DURATION_MS: &str = "codex.db.backfill.duration_ms";
|
|
/// SQLite initialization attempts. Tags: [status, phase, db, error]
|
|
pub const DB_INIT_METRIC: &str = "codex.sqlite.init.count";
|
|
/// SQLite initialization latency. Tags: [status, phase, db, error]
|
|
pub const DB_INIT_DURATION_METRIC: &str = "codex.sqlite.init.duration_ms";
|
|
/// Rollout fallback attempts. Tags: [caller, reason]
|
|
pub const DB_FALLBACK_METRIC: &str = "codex.sqlite.fallback.count";
|