> ## Documentation Index
> Fetch the complete documentation index at: https://docs.monolex.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Monobench Reference
> Complete command reference for the monobench CLI.
*Published contract: `monobench 0.2.7`, OpenCLIs build
`2026-07-12-042014`. This page covers both the native instance harness and the
complete public bench-set grammar embedded in that payload.*
This is the **released** command reference. `monobench docs`, `monobench
verify`, and the newer process-adapter experiment verbs exist in the current
development checkout but not in build `2026-07-12-042014`; the installed
payload returns `unknown command` for `docs` and `verify`. They will enter this
reference only after a newer OpenCLIs artifact is published.
Inventory proof: 35 canonical top-level commands, nine global `bench`
estate/data actions, and 48 canonical `bench ` actions. `memo` aliases
`note`; `contamination` aliases `integrity`. Counts exclude aliases but include
help/version as top-level commands.
## Commands
### version
Print the installed binary version.
```bash theme={null}
monobench version
```
### help
Print the full command reference. Running bare `monobench` with no arguments
prints the same text.
```bash theme={null}
monobench help
```
### root
Print the active benchmark root plus candidate roots — use when
installed-vs-source or working-directory inference is confusing.
```bash theme={null}
monobench root
```
### list
List the instance corpus.
```bash theme={null}
monobench list [options]
```
| Option | Description |
| ----------------------- | ------------------------------------------------------ |
| `--catalog` | Show source/cohort/tool-profile/language columns |
| `--source S` | Filter by problem source |
| `--cohort C` | Filter by curated cohort |
| `--tool-profile T` | Filter by intended target tool profile |
| `--semantics-supported` | Only languages currently covered by monogram-semantics |
### tools
List available tool adapters (`baseline`, `monogram`, `monogram-mcp`,
`codegraph`, or a custom one).
```bash theme={null}
monobench tools
```
### show
Print the task given to the agent — the symptom, never the answer.
```bash theme={null}
monobench show [--catalog] [--spoil]
```
`--spoil` reveals the gated ground truth. Never paste its output into
anything that will also solve the instance.
### run
Run ONE arm of ONE instance: clone the repo at the pinned tag, index per the
tool adapter, invoke the model, grade.
```bash theme={null}
monobench run [note...]
```
### prepare
Pre-index a tool once and snapshot the result, so `matrix --prepared` skips
per-run indexing.
```bash theme={null}
monobench prepare [--tools a,b]
```
### matrix
Run a multi-run comparison for one instance — one CLI+model combination per
command; repeat the command for the next model.
```bash theme={null}
monobench matrix --tools a,b --cli --model [options]
```
| Option | Description |
| ------------------------ | ------------------------------------------------------------ |
| `--tools a,b` | Arms to compare |
| `--cli` | `claude`, `codex`, `agy`, `grok`, or `opencode` |
| `--model` | Full model label for that CLI |
| `--via` | `direct` (default) or `niia` |
| `--effort` | Effort label, mapped per CLI |
| `--prepared` | Reuse a `prepare`d index |
| `--isolate` | `worktree` (default) or single-lane `shared` |
| `--runs N` | Repeats per arm |
| `--jobs N` | Parallel workers |
| `--tag T` / `--note TXT` | Human-readable experiment metadata |
| `--semantics-supported` | Refuse languages outside current monogram-semantics coverage |
### sweep
Run matrix-style jobs across multiple instances at once.
```bash theme={null}
monobench sweep [--source S] [--cohort C] [--tool-profile T]
```
Add `--semantics-supported` to filter/refuse instances outside the current
semantic extractor language set.
### absorb
Project external Harbor tasks into the native default+ catalog without copying
their tests or solution.
```bash theme={null}
monobench absorb \
(--task | --all) [--cohort C] [--tool-profile T] [--dry-run]
```
`--overwrite` replaces an existing projection; `--allow-duplicate` disables
the default same-bench-task duplicate guard. Preview first.
### stop
Gracefully stop an active matrix/sweep after in-flight runs finish.
```bash theme={null}
monobench stop
```
### clean
Delete recorded runs for an instance — all of them, or one arm's.
```bash theme={null}
monobench clean [arm-prefix]
```
Deletes result data. Use before a fresh re-bench, not casually.
### tui
A keyboard-driven wrapper over the same read-only views: browse instances,
arms, and runs. Introduces no new judgment logic.
```bash theme={null}
monobench tui
monobench tui --probe 0,1,2 # non-TTY deterministic path probe
```
### grade
Automatic keyword grade for all runs of an instance, or one run.
```bash theme={null}
monobench grade [run]
```
### judge
Build an answer-key-aware review prompt for a final grader. It does not call
a model itself — the solver process stays separate from the reviewer that
sees the key.
```bash theme={null}
monobench judge [--model m] [--write]
```
### review
Record the checked final grade.
```bash theme={null}
monobench review --final [--reason TEXT] [--judge-model m]
```
`GRADE` is one of `FULL`, `NAME_ONLY`, `DECOY`, `MISS`, `NO_RESULT`,
`INVALID`, `FORFEIT`.
### status
Per-instance run state, chronologically, with an optional live view.
```bash theme={null}
monobench status [--live] [--every N] [--detail]
```
### watch
Global status across every instance.
```bash theme={null}
monobench watch [--live] [--every N]
```
### report
Per-CLI/model comparison for one instance: FULL hit-rate, review status,
median cost/tokens/time, tool-adoption share.
```bash theme={null}
monobench report [--since DUR]
```
`--since 9h` isolates one session — all-time totals conflate old arms.
### summary
Cross-instance leaderboard: FULL hit-rate and median wall time per arm.
```bash theme={null}
monobench summary [--since DUR]
```
### gate
Evaluate a native-result signal arm against its baseline. Promotion requires a
minimum sample count, pooled margin, one-sided two-proportion significance,
multi-instance coverage, and no per-instance regression beyond tolerance.
This is a read-only report over recorded results.
```bash theme={null}
monobench gate [baseline-arm] \
[--min-n N] [--margin PCT] [--z Z] \
[--min-insts K] [--strat-tol PCT]
```
The output is `PROMOTE` or `HOLD` with pooled and stratified reasons. It does
not create runs or change result weight.
### ladder
Run three consecutive read-only gates over the four native layer-isolation
rungs: baseline → `monogram-own` → `monogram-nologos` → `monogram`. The view
attributes own-index, semantic, and Logos net lift without rewriting grades.
```bash theme={null}
monobench ladder [--cli C] [--model M] [--version V] \
[--instances id,id] [--min-n N] [--margin PCT] [--z Z] \
[--min-insts K] [--strat-tol PCT]
```
### column
One arm's per-instance grade breakdown, plus judged-vs-unreviewed coverage.
```bash theme={null}
monobench column [--since DUR]
```
### adoption
Per-run tool-call breakdown: mono% share, first-use point, and git attempts
(each must read as denied).
```bash theme={null}
monobench adoption
```
### monogram-audit
Diagnose Monogram command/result failure patterns in solver telemetry and
surface maker-state recommendations.
```bash theme={null}
monobench monogram-audit [--tag T | --run RUN] [--json]
```
### inspect
Quick peek at one run: grade/review status, artifact sizes and age, event
counts, and a next action.
```bash theme={null}
monobench inspect
```
### integrity
Heuristic contamination-risk scan. A high score means "rerun before counting
this," not an automatic verdict.
```bash theme={null}
monobench integrity [run]
```
`monobench contamination ...` is an accepted alias.
### evidence
Search one run's transcript for a pattern; without a run, rank every run by
whether it concluded on that pattern.
```bash theme={null}
monobench evidence [run] [--pattern P]
```
### trace
Ordered tool-call timeline for one run.
```bash theme={null}
monobench trace
```
### export
Render one run's full transcript to markdown for later search and comparison.
```bash theme={null}
monobench export
```
### note
Attach human metadata to an existing run (also available as `memo`).
```bash theme={null}
monobench note [--tag T] [--note TXT]
```
### meter
Summarize tokens, cache breakdown, and cost for a raw model-session file.
```bash theme={null}
monobench meter
```
### add
Scaffold a native benchmark instance from `instances/_TEMPLATE/`.
```bash theme={null}
monobench add
```
This writes instance files. Keep `symptom.md` spoiler-free and the answer key in
the gated `ground_truth.md` and grading fields.
***
## Bench-Set Commands
`monobench bench ...` plugs into DeepSWE and SWE-Bench Verified/Pro,
each graded through its own external verifier (see the
[overview](/ai-clis/monobench/overview)). This is a public CLI surface, so its
complete command inventory is documented below.
### Estate Setup and Data
| Command | What it does | Effect or proof boundary |
| ------------------------------------------ | ------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `bench setup` | Prints the ordered new-machine runbook and completion checks for native and external harnesses. | Read-only guide. |
| `bench doctor` | Checks Pier, Docker/Colima, architecture support, auth, tool host, and served-versus-local versions. | Required preflight; it does not create benchmark truth. |
| `bench health` | Reports disk, containers, VM, and active benchmark capacity. | Read-only resource gate. |
| `bench stage-tools` | Stages portable Monogram/NIIA and optional Monograph payloads, writes `VERSIONS`, and starts the `:8899` tool host. | Executable/cache/process mutation; run after reboot or a stale doctor row. |
| `bench list` | Lists registered bench sets and tool arms. | Read-only discovery. |
| `bench watch` | Shows the state of every bench-set estate. | Read-only overview; `bench watch` drills into one set. |
| `bench fetch-baselines swe-bench-verified` | Imports published per-instance solved sets and computes empirical solve rates. | Network/data write; use the field's headroom rather than inventing a local baseline survey. |
| `bench import ` | Converts SWE-Bench Verified/Pro data into Pier tasks. | Data write; supports ids/limits/rate/difficulty and Pro language filters. Keep `--include-spec` off for reportable Pro runs. |
| `bench prune-cache` | Reclaims Docker build cache and optionally stopped containers or Colima sparse-disk blocks. | Destructive maintenance; dry-run first and confirm no live sweep. `--force` overrides the live-work refusal. |
`doctor` proves the served Linux artifacts rather than only the macOS/host PATH.
Every versioned tool arm records that served version, and cache reuse is valid
only when the cached tool version matches it.
### Per-Set Measurement and State
Here `` is `deepswe`, `swe-bench-verified`, or `swe-bench-pro`.
| Command | What it does | Effect or truth boundary |
| ------------------------------------------ | ---------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `bench tasks` | Lists imported tasks and current filtering metadata. | Read-only. |
| `bench run --tool ` | Solves one task through Pier and invokes the set's verifier. | Expensive Docker/model mutation; supports `--cli`, `--model`, and `--via`. |
| `bench sweep` | Runs a task × arm × repeat experiment. | Writes jobs, cache, inflight claims, and archive; use `--sample`, `--tools`, `--repeats`, `--batch-size`, `--jobs`, and `--no-cache` deliberately. |
| `bench status [--json]` | Shows canonical live markers, cache/archive axes, pause state, and current jobs. | Read-only; JSON is the full automation state. |
| `bench agent-status [--json]` | Emits a compact AI-first subset with liveness, latest version-honest axes, active jobs, and next commands. | Read-only schema `monobench.bench-agent-status.v1`. |
| `bench agent-tail` | Reads a bounded active or selected job agent-log tail. | Read-only; `--lines` default 80/max 500, optional `--job`, structured JSON. |
| `bench report ` | Counts every physical job attempt for a task, including retries hidden by cache deduplication. | Read-only physical-run truth. |
| `bench lift --tool --model ` | Measures tool-solved rate only where baseline never solved and the tool ran. | Reportable admission-gated lift; `--version` or `--all-versions` controls version scope. |
| `bench schemes [--model M] [--json]` | Compares binary, `g95`, `g90`, `g85`, and guarded-mean scoring per task/arm/tool version. | Read-only, P2P-guarded what-if view computed from cache rows; it does not rewrite reward or select policy. |
| `bench checkpoint` | Writes canonical state to `latest-checkpoint.json` plus history. | Explicit handoff write; optional `--note` and `--json`. |
| `bench pause` | Requests safe stop before a new task claim. | Writes pause state; already-running old binaries may not honor it. |
| `bench resume` | Removes the pause request. | State mutation; does not resurrect a dead solver. |
| `bench snapshot` | Writes a dated cache subset/full copy plus manifest for one tool/model axis. | Audit artifact write. |
| `bench archive` | Verifies/resynchronizes durable aggregate axes. | Archive write/repair; `--versions` exposes every version rather than only latest clean rate. |
| `bench watch` | Explores one set by axis → task → run → normalized commands. | Read-only; filters by tool/model/run. |
Never launch a sweep through `| tail`: `tail` waits for EOF and makes a healthy
long-running sweep look frozen. Launch it directly or redirect logs, then use
`status`, `agent-status`, and `agent-tail`.
### Recent-Session Cohort Commands
| Command | What it does | Main filters |
| -------------------------------------------- | ------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------- |
| `jobs` | Lists recent physical Pier sessions with state, reward, provider/model, steps, task, and exact path. | `--max`, `--scan`, task/provider/text/reward/running filters, `--json` |
| `session-scan` | Batch-runs the single-session audit logic and summarizes status, flags, command mix, and Monogram use. | Same cohort filters plus `--flag` and `--audit-contains` |
| `session-queue` | Ranks concrete sessions by status and risk/warn/info gaps, then prints the next drill-down command. | Same cohort filters |
| `session-latest` | Selects the newest sessions per provider/model/schema/status/task cohort. | `--group-by`, `--top`, and cohort filters |
| `session-trends` | Aggregates cross-session status, command, query, path, and repeated-pattern motifs. | `--top` and cohort filters |
| `session-compare --left --right ` | Compares normalized audits, flags, command mixes, Monogram subcommands, and first differing command. | `--max`, `--json` |
All six are read-only routing surfaces. They select the job path to pass to the
single-session commands below; they do not rerun a solver or verifier.
### Single-Session Analysis Commands
Every command in this table is read-only and requires
`--job ` unless noted.
| Command | What it explains | Important controls |
| --------------------- | --------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- |
| `session-brief` | One-page continuation dashboard: verdict, audit flags, artifacts, signals, checkpoints, anchors, next commands. | `--max`, `--full`, `--json` |
| `session-gaps` | Missing cache/reward/verifier/patch/test/routing evidence and risky rows. | `--max`, `--full`, `--json` |
| `session-plan` | Deduplicated ordered runbook assembled from the other review views. | `--max`, `--full`, `--json` |
| `session-outline` | Chunked turn ranges before opening raw text. | `--chunk`, `--max`, `--full`, `--json` |
| `session-evidence` | Effective verdict, cache/reward, bounded verifier, patch, result, and checkpoint evidence. | `--artifact-lines`, `--contains`, `--max`, `--full` |
| `session-actions` | Ranked next analysis commands from gaps, signals, checkpoints, and evidence. | `--max`, `--full`, `--json` |
| `session-scorecard` | Compact status/count/signal hub with original turn/step coordinates. | `--max`, `--full`, `--json` |
| `session-windows` | Bounded turn contexts around deduplicated scorecard centers. | `--radius`, `--max`, `--full` |
| `session-handoff` | Transfer summary for another AI or human reviewer. | `--max`, `--full`, `--json` |
| `session-checkpoints` | Planning, tool-routing, search, test, verifier, patch, error, and final milestones. | Turn/range/around/radius/text/kind/flag filters |
| `session-artifacts` | Bounded, line-numbered raw cache/verifier/reward/stdout/stderr/patch/agent/result/log files. | `--artifact`, `--contains`, `--max-lines`, `--head` |
| `session-digest` | First message/tool/Monogram/sequence/reindex/test/error anchors. | `--max`, `--json` |
| `session-map` | Dense normalized command table retaining original turn and step ids. | Turn/range/around/text/kind/subcommand/flag filters |
| `session-queries` | Extracted search phrases with tool, scope path, and flags. | Map filters plus `--query` and `--path` |
| `session-patterns` | Repeated command, query, scope, path, and warning motifs. | Turn/range/around/text/kind/flag filters |
| `session-paths` | Touched path counts, first/last turns, sources, and artifact/repo/verifier flags. | Turn/range/around/text/path/flag filters |
| `inspect-job` | Provider-neutral Pier artifact dossier. | `--json` |
| `session` | Normalized ATIF turns with raw-agent fallback for active jobs. | `--full`, `--max`, `--tail`, turn/range/around/source/text, tools/messages/Monogram/error filters |
| `session-audit` | Command mix, Monogram routing, stale-index/reindex, raw fallback, reward, and running state. | `--json` |
| `timeline` | Merges normalized turns with cache/reward/verifier/patch/result checkpoints. | Same range/content filters as `session` |
| `verdict` | Classifies effective outcome as pass, partial, fail, running, or unknown. | Reads cache first for externally graded SWE-Bench; otherwise `reward.txt`. |
| `tests` | Reads raw per-test verifier rows from `verifier/ctrf.json`, including F2P/P2P suite tags. | `--status`, `--suite`, `--contains`, `--max`, `--json`; CTRF-era jobs only, read-only. |
| `commands` | Lists normalized CLI commands from archived job telemetry. | `--tool`, `--model`, `--task`, `--job`, `--all`, `--command`, `--contains`, `--max`, `--json` |
### Recovery and Pinpoint Cleanup
| Command | What it does | Safety contract |
| ---------------------------------------------------------- | --------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
| `recover-job --tool --model --job ` | Records one explicit completed DeepSWE `reward.txt` into cache, archive, and checkpoint. | DeepSWE only; never scan arbitrary job dirs or use in-job reward as SWE-Bench external truth. |
| `drop-job --job ` | Previews removal of one contaminated/interrupted physical job. | Dry-run by default; `--apply` needs exactly one cache match. `--delete-raw` removes only that exact path; broad `jobs/` targets are refused. |
| `scrub-harness-errors` | Finds cached reward-zero rows whose logs prove setup failed before the solver started. | Dry-run first; `--apply` removes matched rows and re-archives only affected axes. |
| `rescore` | Backfills F2P/P2P score-detail fields from each existing cache row's own persisted artifacts. | Plan-only by default. `--apply` changes only six score-detail fields, never reward/confounded/run\_quality, and cannot resurrect removed rows. |
```bash theme={null}
monobench bench rescore [--apply] [--from-stdout] \
[--tool ] [--task ] [--model ] [--json]
```
DeepSWE's in-job `reward.txt` is its grade. SWE-Bench Verified and Pro report
the external cache/archive outcome. A raw pass rate pools easy tasks and is not
the same claim as admission-gated `lift`.
***
## Examples
```bash theme={null}
# Compare a tool against baseline
monobench run bun-1.3.10-toThreadSafe baseline
monobench run bun-1.3.10-toThreadSafe monogram
monobench report bun-1.3.10-toThreadSafe
# Multi-run comparison across a tool and a model
monobench matrix bun-1.3.10-toThreadSafe \
--tools baseline,monogram --cli claude --model haiku --runs 3 --jobs 2
# Investigate a MISS
monobench evidence bun-1.3.10-toThreadSafe --pattern ROOTCAUSE
monobench trace bun-1.3.10-toThreadSafe
monobench export bun-1.3.10-toThreadSafe
```