Reproduce a benchmark

Verify tracked evidence and publish a new dated result.

The application repository owns benchmark procedures and artifacts. Use this sequence for a new result.

1. Read the procedure

Open the canonical benchmark guide. Select the benchmark that matches your question.

2. Freeze the source

source_sha="$(git rev-parse HEAD)"
test -z "$(git status --porcelain)"

Record the full source SHA. Branch names move and cannot bind evidence.

3. Preregister the run

Record:

• Arms and interleaving schedule
• Exact topology, placement, resources, Image, ingress, HTTP version, and backends
• Warmup and measured sample counts
• Timer boundaries and required attribution
• Failure, cleanup, and promotion rules
• Cost ceiling and authorization

4. Run once and retain failures

Use the command listed on Run benchmarks. Keep retries and replacement samples at zero unless the preregistration explicitly studies them. Preserve partial and rejected artifacts.

5. Validate the artifact

Run the benchmark-specific validator and the repository checks:

uv run pytest tests/benchmarks
uv run ruff check .
uv run mypy src
uv run pytest
uv run python scripts/export_openapi.py --check

6. Publish a dated report

Pin links to full commit SHAs. Include configuration, sample counts, failures, cleanup, statistics, and limits. Keep raw secrets and resource identifiers in a private archive.

The hosted site explains the evidence. The application repository remains the source for benchmark code, policy, reports, and sanitized artifacts.