retro Skill
基于 ax 代理体验图谱的引导式实验循环回顾。引导用户处理未决提案(带脚手架接受或拒绝)、待定裁定(确认建议裁定或覆盖)以及近期的 harness-hook 有效性信号。当用户说"let's do an ax retro"、"ax retrospective"、"review my ax proposals"、"triage proposals"、"experiment loop status"、"lock pending verdicts"、"hook effectiveness review"、"intervention review"、"self-improvement session"或调用 /ax:retro 时触发。通过本地 `ax improve` 和 `ax hooks` CLI 读写。不要自动触发。
安装方式:把技能目录放入 ~/.claude/skills/(Claude Code)或在 claude.ai 设置中启用;也可复制右侧安装命令一键添加。
技能指令原文(SKILL.md)
ax:retro - guided experiment-loop session
Closes the self-improvement loop. Claude orchestrates ax improve …
commands; the user decides each row.
Assumes ax (axctl) is on PATH. If ax improve list fails, tell the user
to check docs/development.md#setup (DuckDB dylib setup - no daemon
required) and stop.
When to fire
ONLY fire on explicit triggers:
- "let's do an ax retro" / "ax retrospective" / "retro time"
- "review my ax proposals" / "triage proposals"
- "what's my experiment loop status" / "lock pending verdicts"
- "hook effectiveness review" / "intervention review"
- "self-improvement session"
/ax:retroslash command (if the plugin marketplace publishes one)
Do NOT fire on a generic "look at my recent work" - that risks dragging
unrelated context into the loop.
Defaults
- Window for hook signals: last 7 days. Widen to 30 if evidence is sparse.
- Don't apply changes silently. Every accept/reject/verdict gets the
user's explicit yes per row.
- The retro is read-mostly. Its side-effects are the Step 1 checkpoint
measurements, the task briefs accept emits, and verdict locks.
Workflow
Step 0 - Drain pending session retros
Before the proposal queue, check whether prior sessions still owe a
retro. This is the "quota arbitrage" path - idle Opus budget chews
through the backlog so the experiment loop has signal next time.
- Run:
ax retro pending --since=7 --idle-min=30 --json
Returns sessions in the last 7 days that have no reviewed graph
edge yet AND look finished (explicit ended_at, or last turn is
>30min idle). If the list is empty, skip to Step 1.
- Show the list to the user as 1 line per session (project · turns ·
model · reason). Ask:
N session(s) pending retro. Want me to dispatch the
retro-reviewer subagent for all of them in parallel, or pick a
subset?
- On
allor: for each chosen session, write a brief:
ax retro brief --session=<session_id>
This writes .ax/tasks/retro/.md with frontmatter (transcript
path, suggested model, turn count, etc.) and a body that tells the
reviewer what to do.
- Dispatch one
retro-reviewersubagent per brief, in parallel. Pass
each brief path in the prompt; let the subagent's frontmatter pin
model: opus (override per session if suggested_model differs and
the user asked you to economize).
If the retro-reviewer subagent type doesn't resolve (not installed,
or the active harness is not Claude Code), read and review the brief INLINE
using its required-output instructions instead of abandoning the backlog.
- Wait for all subagents. Aggregate results: counts of retros emitted,
proposals recommended, model-fit suggestions. Render as a short
summary. The user does not approve retro emissions per row - the
subagent already wrote them. The user DOES decide on resulting
proposals in Step 2.
- The
reviewededge now exists for each drained session, so a
re-run of ax retro pending should show fewer rows.
If the user declines Step 0, move on. The backlog stays - next retro
picks it up.
Step 1 - Snapshot
Measure first, then read. The checkpoint pass is what turns due windows
into current ones; a verdict read taken before it reports whatever the
last run happened to leave behind.
- Run the prerequisite ALONE and wait for it:
AX_NO_AUTO_INGEST=1 ax improve checkpoint --json
It reads the published snapshot and writes checkpoint judgments only -
no transcript parsing, no guidance edits, no verdict locks. Leave
--force out of an ordinary retro; it rewrites unreviewed windows that
are already current.
- On success, run the reads (parallel is fine). Prefix EVERY command -
a prefix on the first one leaves the freshness drive on for the rest:
AX_NO_AUTO_INGEST=1 ax improve list --status=open --json
AX_NO_AUTO_INGEST=1 ax improve list --status=accepted --json
AX_NO_AUTO_INGEST=1 ax improve verdict --json
AX_NO_AUTO_INGEST=1 ax retro list --since=7 --json # cluster-derived friction summary
AX_NO_AUTO_INGEST=1 ax hooks summary --since=7 --tail=20 # optional; tolerate failure
- If the checkpoint run fails, tell the user measurement is unavailable
and continue with proposal review only. Any suggestion already stored
belongs to an earlier run - report it as that, and skip the verdict
step.
- If the checkpoint result reports
cacheRefreshRequiredabove zero, the
opportunity evidence needs deriving; say so, and treat those
experiments as unmeasured. When the user asks for current evidence, run
three operations in order, each awaited on its own: ax ingest (wait
for successful publication), the checkpoint prerequisite, then the
reads.
ax retro list reflects three pattern types now:
- tool failures (skill form) ->
Pre- guardproposals - correction pressure (guidance form) -> "Reduce recurring user
corrections" proposals targeting CLAUDE.md
- friction kinds (skill form, one per kind) -> `Address recurring
friction` proposals
If any of those surfaced, mention them so the user knows to triage in
Step 2.
Compute counts: open proposals (by form), accepted experiments with
locked_verdict IS NONE, and - separately - experiments whose current
view carries a current_reason (insufficient data or a lifecycle state).
Then render to the user as 2-4 lines, e.g.:
7 open proposals (3 skill, 4 guidance). 2 accepted experiments are
waiting on a verdict. Hook activity last 7d: 142 invocations, 3
blocking errors. Want to triage proposals first, lock the pending
verdicts, or skim hook signals?
If both proposal/verdict queues are empty: tell the user nothing's due
and offer ax ingest --derive-only to refresh evidence.
Step 2 - Triage open proposals
Order open proposals by frequency desc. For each, in turn:
- Run
ax improve show --json(or reuse the row from
step 1).
- Render as 3-5 lines. Example for a skill proposal:
Schema change guardrail (skill · freq=9 · confidence=high)
Hypothesis: schema edits often surface in fix-chains within ~14d.
Trigger: fix commits overlap schema files.
Behavior: run schema lint + one read/write smoke before edit.
- Ask the user: accept, reject, or skip.
- Branch:
- accept → run
ax improve accept.
By default this emits a TASK BRIEF and returns its task_path; it
does not install the artifact. Report that path.
Offer: "Want me to implement the brief now?"
If yes: implement it, then run ax improve lint so ax reconciles the
marker it finds on disk and records the installed artifact. Until
lint records one, the experiment has no measurable installation.
- reject → ask for a short reason (≤80 chars).
Run ax improve reject --reason "".
- skip → no command. Move on; the proposal stays open for the
next retro.
After the loop, summarize: "Accepted 3, rejected 1, skipped 2."
Step 3 - Verdict review
For each experiment whose latest checkpoint is unlocked
(locked_verdict IS NONE), in age order:
- Run
AX_NO_AUTO_INGEST=1 ax improve verdictto fetch the
experiment + checkpoint history.
- When the current view carries a
current_reason, there is no
suggestion to confirm. Report the reason as it is - `no opportunities
in the window, no detector for this form, retired` - and move to
the next experiment. A suggestion in the checkpoints history is
history, not a recommendation.
- Otherwise render the current checkpoint as 2-3 lines:
Schema change guardrail - +30s checkpoint
12 opportunities in window, 8 addressed (66%) - observed use.
Suggested: adopted.
- Ask the user to confirm the suggested verdict OR override:
adopted(artifact is doing real work)ignored(user wrote it but never invoked it)regressed(it made things worse)partial(mixed signal)no_longer_needed(pattern self-resolved; trigger stopped firing)
- Run
ax improve verdict --setto lock it. All
five values stay available to the user by hand, including
no_longer_needed, which the algorithm never suggests on its own.
Step 4 - Hook effectiveness pass (optional)
Only run if the user asked for hook review OR if step-1 found ≥3
blocking errors. Light touch - this section is read-only.
- Show top hooks from
ax hooks summary --since=7 --tail=20if not
already shown.
- If a hook keeps blocking, ask: *"Want to inspect a recent
invocation?"* Then run
ax hooks invocations --command="" --tail=5 and render.
- Backtest known feedback cases:
ax hooks cases enforce-worktree --tail=50 --window=3
Treat each backtest result as one case type. Report pass/fail/
inconclusive counts.
- Interpretation:
- A blocking hook error is not automatically bad. If the next few
agent actions show corrected behavior, it's a useful corrective
signal.
- A successful hook is not automatically useful. Look for downstream
behavior change.
hook_progresswithout a terminal success/blocking event is a
telemetry gap unless correlated with visible behavior.
- Prefer deterministic backtests over model judgment.
- To author a NEW guard from a recurring failure:
ax hooks init, write adefineHookhook in~/.ax/hooks/,ax hooks backtestit against history, thenax hooks install --providers=claude,codex.
Step 5 - Close out
Output a one-paragraph summary:
- Counts: accepted / rejected / skipped / verdicts locked.
- Any scaffolded SKILL.md files that still need refinement.
- When the next retro is recommended. Compute: earliest
experiment.created_at + 7d among accepted-but-unlocked
experiments, formatted as "next retro suggested around YYYY-MM-DD".
Then ask whether the user wants to commit the scaffolded skill files +
proposal-status changes (DB is local, but SKILL.md files are on disk
and may belong in version control).
How to track feedback
The retro itself produces durable signal that the experiment loop
already captures:
- Acceptance rate by form - after the session, derive from
proposal.status. If skill-form gets accepted 80% but guidance gets
rejected 80%, the derive-proposals stage is over-eager on the wrong
form. Surface as an observation.
- Reject reasons -
proposal.reject_reasonis a free-text corpus.
After the session run:
ax improve list --status=rejected --json | jq '.[].reject_reason'
Look for repeated phrases ("duplicate of existing hook"). When a
pattern emerges, the derive-proposals stage should dedupe against it
- tell the user.
- Verdict surprises - when the user overrides a suggested verdict,
note it. Repeated overrides mean the verdict math is biased.
These are observations, not actions. Report in the close-out; don't
write to insight tables.
CLI reference Claude calls
ax improve list [--form=skill|subagent|hook|guidance|automation] \
[--status=open|accepted|rejected|superseded|all] [--json]
ax improve show <dedupe_sig> [--json]
ax improve accept <dedupe_sig> [--force]
ax improve reject <dedupe_sig> --reason "<text>"
ax improve verdict [<dedupe_sig>] [--set <verdict>] [--json]
ax improve checkpoint [--force] # Step 1 prerequisite; --force only on request
ax improve reset --yes # destructive; only when user requests
ax retro pending [--since=N] [--idle-min=N] [--json] # Step 0 backlog
ax retro brief --session=<id> [--out-dir=<path>] [--json]
ax retro emit --session=<id> [--source=<src>] [--from-file=<json>]
ax retro list [--since=N] [--limit=N] [--json]
ax hooks summary [--since=N] [--tail=N]
ax hooks invocations [--command="<name>"] [--tail=N]
ax hooks cases <case-name> [--tail=N] [--window=N]
--force on accept overwrites an existing SKILL.md scaffold. Only use
when the user explicitly says so.
reset --yes wipes ALL proposal/experiment/checkpoint state. NEVER run
without explicit user confirmation in this session.
Failure modes
ax improve listreturns empty → runax ingest --derive-onlyonce,
retry. If still empty, evidence is genuinely thin; tell the user.
ax improve acceptreportsscaffold_exists→ ask the user if they
want --force or to abandon.
ax improve verdict --setreportsverdict_locked→ that experiment
is already finalized; show the locked value and move on.
ax improve checkpointfails → measurement is unavailable for this
retro. Say so, skip Step 3, and keep Step 2 going.
ax hooks summaryreturns nothing → retry with--since=30; if
still empty, the hook telemetry pipeline is idle, surface as a TODO.
- Read/query error → tell the user to check
docs/development.md#setup
(AX_DUCKDB_DYLIB).
Anti-patterns
- Don't dump raw JSON. Render summaries.
- Don't run
ax improve acceptfor every open proposal in a batch; the
user must say yes per row.
- Don't write to
~/.claude/skills/directly. The CLI handles that. - Don't propose deleting a scaffolded SKILL.md mid-retro; that's a
separate cleanup task.
- Don't auto-implement experiments from the hook pass. Recommendations
only; the user decides + commits.