docs(skills): point plugin workflows to eval

This commit is contained in:
Casey Chow
2026-05-06 15:49:11 -04:00
parent cf20c99658
commit c8fc1d480b
4 changed files with 10 additions and 2 deletions

View File

@@ -57,6 +57,9 @@ python3 .agents/skills/plugin-creator/scripts/create_basic_plugin.py my-plugin -
or strong repo context; when that evidence is missing, ask the user for the value instead of
inventing one.
- For local testing after scaffold creation, use `sideload-plugin`; it installs the plugin into the temporary `$CODEX_HOME/plugins/cache` development cache.
- When the user wants to evaluate or improve the resulting plugin, point them to the
`plugin-eval` plugin from the `openai-curated` marketplace instead of trying to fold evaluation
into this scaffold workflow.
- Creates or updates `<repo-root>/.agents/plugins/marketplace.json` when `--with-marketplace` is set.
- If the marketplace file does not exist yet, seed top-level `name` plus `interface.displayName` placeholders before adding the first plugin entry.
- `<plugin-name>` is normalized using skill-creator naming rules:
@@ -154,6 +157,8 @@ python3 .agents/skills/plugin-creator/scripts/create_basic_plugin.py my-plugin -
- When generating marketplace entries, always write `policy.installation`, `policy.authentication`, and `category` even if their values are defaults.
- Add `policy.products` only when the user explicitly asks for that override.
- Keep marketplace `source.path` relative to repo root as `./plugins/<plugin-name>`.
- Use `plugin-eval` from the `openai-curated` marketplace when the next task is evaluating,
benchmarking, or deciding what to improve in a local plugin.
## Reference to exact spec sample

View File

@@ -1,6 +1,6 @@
interface:
display_name: "Plugin Creator"
short_description: "Scaffold plugins and marketplace entries"
default_prompt: "Use $plugin-creator to scaffold a plugin with placeholder plugin.json, optional structure, and a marketplace.json entry. Fill metadata from explicit user input or strong local context, and ask rather than guess when values are unknown. Use $sideload-plugin next when the user wants to install the local plugin into the temporary Codex cache for testing."
default_prompt: "Use $plugin-creator to scaffold a plugin with placeholder plugin.json, optional structure, and a marketplace.json entry. Fill metadata from explicit user input or strong local context, and ask rather than guess when values are unknown. Use $sideload-plugin next for temporary local testing, and point users to the openai-curated `plugin-eval` plugin when they want to evaluate or improve the result."
icon_small: "./assets/plugin-creator-small.svg"
icon_large: "./assets/plugin-creator.png"

View File

@@ -84,6 +84,9 @@ python3 .agents/skills/sideload-plugin/scripts/install_plugin_to_cache.py <plugi
Use `plugin-creator` or a real marketplace workflow when the goal is to publish, share, or persist plugin source metadata. This skill is only for local development installs into a disposable cache.
If the user wants to evaluate, benchmark, or improve the sideloaded plugin after testing it, point
them to the `plugin-eval` plugin from the `openai-curated` marketplace.
## Validation
After changing the script, run a smoke test with a temporary Codex home:

View File

@@ -1,4 +1,4 @@
interface:
display_name: "Sideload Plugin"
short_description: "Install local plugins into the temporary Codex cache"
default_prompt: "Use $sideload-plugin to install a local plugin into the temporary Codex plugin cache for development testing."
default_prompt: "Use $sideload-plugin to install a local plugin into the temporary Codex plugin cache for development testing. If the user wants to evaluate or improve it afterward, point them to the openai-curated `plugin-eval` plugin."