Faithfully transcribe audio into text (speech-to-text via API)

What we did not test, and why

Not specified in this bundle.

Legal preflight, verdict per tool

Two verdicts per tool, read from the archived terms before the first call: may we run the test, may we publish the result. A tool that was not run keeps its recorded reason. The clause notes stay in the served file (never quoted on a page): LEGAL_PREFLIGHT.json.

Right of reply

Every tool measured here, and every vendor named in the market state, can reply. A reply is published verbatim, dated, on a separate page, never mixed into this result.

• gemini-2-5-flash: notified 3 Sep 2026, window closes 17 Sep 2026, no reply received
• gemini-2-5-pro: notified 3 Sep 2026, window closes 17 Sep 2026, no reply received
• gemini-3-5-flash: notified 3 Sep 2026, window closes 17 Sep 2026, no reply received
• gemini-3-5-transcribe: notified 3 Sep 2026, window closes 17 Sep 2026, no reply received

How to reproduce

gladiator reproduce runs/TRANSCRIPTION-001

Canonical JSON: latest result of the category, this run. Evidence path in the public repository: runs/TRANSCRIPTION-001.

Reproduction files served byte-identical to the repository: events.ndjson (hash-chained journal), PROTOCOL.lock, contest.yaml, LEGAL_PREFLIGHT.json; schema result.v1.json. Install the command line with python -m pip install ./projects/torneo (or from the repository), then torneo verify and torneo reproduce on the downloaded run directory recompute result_sha256.

Audit in five minutes: the record, the draft and the verdict, side by side, for a sample of this run.

Machine surfaces (MCP server, canonical JSON, llms.txt): For agents.

What this does not prove

Errata

Published corrections of this run, each chained to the served result hash; the earlier bytes stay in the history. Full record: ERRATA.md.

• ERR-001, 3 Sep 2026: erratum ERR-001: limits LIM-2 nommait OpenAI comme vendeur des 4 solutions alors que les 4 participants sont vendor Google; amendement A2 (contest.yaml gelé). Estimations, intervalles, rangs et statuts inchangés. (served result ac21d556 replaced by 1658e291)
• ERR-002, 4 Sep 2026: ERR-002: conflicts served as [] while the vendor of all four measured arms is a tool devlo pays per call (served result 1658e291 replaced by 082730f5)
• ERR-003, 5 Sep 2026: ERR-003: promise, task, context, dataset description and the five limits were served in French on an English-only surface; amendment A4 translates the nine frozen strings (contest.yaml), French originals preserved verbatim in PROTOCOL_AMENDMENTS.md. Estimates, intervals, ranks, status and directive unchanged. (served result 082730f5 replaced by e6377e76)

• Swiss accents and B2B register are not covered by this run (no open corpus available; phase 2).
• 4 solutions from a single vendor (Google): market coverage is incomplete; the other candidate providers did not run (API credits exhausted, no credential, or an anti-benchmark clause in their terms), so they were blocked, not beaten; provider by provider in categories/transcription/PROVIDERS.json and legal/.
• tnorm.v1 does not normalize numbers.
• Wall-clock latency depends on the runner's network.
• Canonical metric = share of clips transcribed perfectly (tnorm.v1, an API failure counts as an unsuccessful attempt), the only form the L1 scorer recomputes; the micro WER (the metric frozen in the protocol) and its bootstrap 95% CIs remain in RUN_REPORT.json, reproducible from events.l2stub.ndjson (amendment A1).

What no TORNEO result can tell you, for every category: Limits.

Conflicts and funding

Run by devlo lane L2 (agent), funding: fonds propres devlo. Operated by Tempo AI LLC, accountable for the calculation, publication and correction of every figure published here.

Conflicts declared in this bundle:

• gemini-3-5-transcribe: devlo is a paying customer of Google, the vendor of this arm, billed per call; devlo is a user of the tool, not a vendor; same conditions and same protocol as any customer, journaled; what this run cost is in FUNDING.json (CONFLICT_REGISTER.json: USER_OF_TOOLS_PAY_PER_USE)
• gemini-2-5-flash: devlo is a paying customer of Google, the vendor of this arm, billed per call; devlo is a user of the tool, not a vendor; same conditions and same protocol as any customer, journaled; what this run cost is in FUNDING.json (CONFLICT_REGISTER.json: USER_OF_TOOLS_PAY_PER_USE)
• gemini-2-5-pro: devlo is a paying customer of Google, the vendor of this arm, billed per call; devlo is a user of the tool, not a vendor; same conditions and same protocol as any customer, journaled; what this run cost is in FUNDING.json (CONFLICT_REGISTER.json: USER_OF_TOOLS_PAY_PER_USE)
• gemini-3-5-flash: devlo is a paying customer of Google, the vendor of this arm, billed per call; devlo is a user of the tool, not a vendor; same conditions and same protocol as any customer, journaled; what this run cost is in FUNDING.json (CONFLICT_REGISTER.json: USER_OF_TOOLS_PAY_PER_USE)