> ## Documentation Index
> Fetch the complete documentation index at: https://modal-computer-use.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Reproduce a benchmark

> Verify tracked evidence and publish a new dated result.

The application repository owns benchmark procedures and artifacts. Use this sequence for a new result.

## 1. Read the procedure

Open the [canonical benchmark guide](https://github.com/ashtonchew/modal-computer-use/blob/b60c1cb7495200e36a738c0f6e07961b1d2db93c/docs/benchmarking.md). Select the benchmark that matches your question.

## 2. Freeze the source

```bash theme={"system"}
source_sha="$(git rev-parse HEAD)"
test -z "$(git status --porcelain)"
```

Record the full source SHA. Branch names move and cannot bind evidence.

## 3. Preregister the run

Record:

* Arms and interleaving schedule
* Exact topology, placement, resources, Image, ingress, HTTP version, and backends
* Warmup and measured sample counts
* Timer boundaries and required attribution
* Failure, cleanup, and promotion rules
* Cost ceiling and authorization

## 4. Run once and retain failures

Use the command listed on [Run benchmarks](/benchmarks/run). Keep retries and replacement samples at zero unless the preregistration explicitly studies them. Preserve partial and rejected artifacts.

## 5. Validate the artifact

Run the benchmark-specific validator and the repository checks:

```bash theme={"system"}
uv run pytest tests/benchmarks
uv run ruff check .
uv run mypy src
uv run pytest
uv run python scripts/export_openapi.py --check
```

## 6. Publish a dated report

Pin links to full commit SHAs. Include configuration, sample counts, failures, cleanup, statistics, and limits. Keep raw secrets and resource identifiers in a private archive.

<Note>
  The hosted site explains the evidence. The application repository remains the source for benchmark code, policy, reports, and sanitized artifacts.
</Note>


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.