agent-qa Authoring

Workflow

1. Discover the local surface with agent_qa_discover.
2. Inspect active config with agent_qa_get_config, especially targets, devices, providers, and services.mcp.
3. Load references/agent-qa-contracts.json when you need exact schema fields or ID contracts.
4. Generate every new ID with agent-qa tooling:

• MCP: agent_qa_generate_id
• CLI fallback: agent-qa ids generate
• npx fallback: npx --yes agent-qa ids generate

1. Never hand-write IDs. Validate existing IDs with agent_qa_validate_id or agent-qa ids validate --json.
2. Validate definitions before saving:

• Tests: agent_qa_validate_test or agent_qa_validate_definition with kind: "test"
• Suites: agent_qa_validate_suite or agent_qa_validate_definition with kind: "suite"
• Hooks: agent_qa_validate_definition with kind: "hooks"

1. Prefer MCP authoring mutations:

• Tests: agent_qa_create_test, agent_qa_update_test, agent_qa_delete_test
• Suites: agent_qa_create_suite, agent_qa_update_suite, agent_qa_delete_suite
• Hooks: agent_qa_create_hook, agent_qa_update_hook, agent_qa_delete_hook

1. Use CLI/YAML fallback only when MCP is unavailable. Keep file paths matched by workspace.testMatch or workspace.suiteMatch.

Required ID Contracts

• Test IDs: t_ + 10 id-agent words.
• Suite IDs: s_ + 10 id-agent words.
• Hook IDs: h_ + 10 id-agent words.
• Run IDs: r_ + 10 id-agent words.
• Observation IDs: obs_ + 10 id-agent words.

Before Running

• Validate YAML first.
• Prefer agent_qa_enqueue_test_run and agent_qa_enqueue_suite_run over shelling out.
• If using CLI fallback, run only after validation succeeds.

Do Not

• Do not invent config keys.
• Do not use legacy root config buckets.
• Do not hand-write IDs.
• Do not mutate files outside the configured workspace patterns.