$ cat ~/map/pi-arena.md

pi-arena

Fair, replayable tournaments for pi coding agents: same task, isolated workspaces, objective scoring, and a browser replay console.

tools✓ Nov 1

Clone the repository, then run npm run demo for the deterministic three-contestant local tournament (no API key) and open arena-demo/index.html for the replay console; per SKILL_USAGE.md, pi install <path to your local clone> adds the package to pi, and /skill:arena loads the tournament protocol from skills/arena/SKILL.md inside a session.

## TL;DR

A tournament harness for pi coding agents: contestants run the same task in isolated workspaces, required tests and measurable signals score the results, and a shareable browser report replays every run — built for fair comparisons between models, providers, prompts, and agent strategies, with a deterministic local demo that needs no API key.

## Key points

• $Fairness is enforced by code, not trusted to the model: the tournament contract in skills/arena/SKILL.md fixes equal starting state, isolated workspaces, one shared prompt, and model-independent tool policy.
• $Not an LLM judge disguised as a benchmark: required tests are objective, model-based judging is optional, and a judge score never overrides a failing required test.
• $The demo scoring contract: 70% required evaluation checks, 20% efficiency from recorded actions, 10% scope discipline including avoiding test edits; real projects define their own weights in a versioned evaluation file.
• $The replay console turns each run into a small race with action timelines and evaluation checks, and reports separate objective results from optional judging.
• $Scores are only comparable when the task, starting revision, evaluator, and budget are identical; a score is evidence for the declared evaluator, not proof of overall software quality.
• $Safety boundaries: network access, credentials, and destructive commands require explicit user policy, credentials never enter a worktree or report, and the runner fails closed when the requested isolation cannot be established.
• $The first release targets local Node.js fixtures and Linux/macOS development environments; the demo runs three deterministic local contestants without API spending.
• $Roadmap: a pi JSONL RPC contestant adapter, git worktree isolation, pluggable evaluators for npm test, pytest, and type checking, blind human review with signed report manifests, and a GitHub Action publishing static reports.

## FAQ

## References

• README (scoring contract, design boundaries, roadmap) ↗
• Tournament protocol (skills/arena/SKILL.md) ↗

Last updated: Oct 30, 2026