Check our work
Every claim has a trail
You should not have to take our word for any of this. Here is what is in the evidence archive and how to check it in a few minutes.
Download the evidence archive (ZIP)
Check it in one minute
- Download the evidence archive and open a terminal in its folder.
- Run
./VERIFY.sh(orshasum -a 256 -c SHA256SUMS.txt). Every line should say OK. - Pick any claim below. Open the files named beside it and compare them with the paper.
- If you can show a claim is wrong, tell us. Corrections are logged.
The headline claims and their traces
| ID | Claim | Where to look |
|---|---|---|
| C1 | Ministral: 93 claims scored; 45 (48.4%) unsupported or contradicted | research/ministral-3-3b/adjudication-v1.0/summary-metrics.txt |
| C2 | Ministral: 18 of 37 cited claims cite a source that does not support them (48.6%) | research/ministral-3-3b/adjudication-v1.0/claim-adjudication.csv |
| C3 | Qwen identity sentence in 32 of 32 no-system-message runs of Qwen 2.5 Instruct and Coder; in none of 96 other logs | 06_Claim-Trace/CLAIM-TRACE-C3-runtime-logs.csv |
| C4 | The same sentence is in the official upstream Qwen2.5-1.5B-Instruct chat template | 02_Evidence-Register/Source-Records.json (S3) |
| C5 | Ministral gave no citations in the China grounded trials and four in each USA stock-grounded trial | results.csv in each locked-v1.0 run folder |
| C6 | Tokens per second: Ministral about 2.3 to 3.6 in grounded conditions (elapsed-time means); Gemma about 6.7; Hailo-hosted models about 6 to 7, except Llama 3.2 3B at about 2.5 | results.csv in the locked-v1.0 and v2 run folders |
These six are the headline claims. They are not every claim in the paper; the paper’s Appendix C says so.
What the archive contains
| Folder | Contents | Files | Size |
|---|---|---|---|
(top level) | 2 | 3 KB | |
02_Evidence-Register | Every outside fact the documents rely on: claim, primary source, date read, the exact excerpt and its hash | 3 | 42 KB |
03_Research-Data | Prompts, frozen evidence packets, raw requests, responses, answers and runtime logs, scoring records, the claim-by-claim adjudication, scripts, model hashes and the system inventory. No model weights | 1,024 | 2.4 MB |
04_Vendor-Outreach | The six requests for technical comment sent on 12 September 2026, with personal addresses and Bcc lines removed; the notice sent to Mistral AI on 6 October 2026 (text as sent and its attachment); and a timeline | 12 | 528 KB |
05_Hardware-Evidence | Photographs of the Raspberry Pi AI HAT+ 2 retail box, cropped, not retouched | 4 | 1023 KB |
06_Claim-Trace | The traceability matrix and the per-log table behind the Qwen finding | 4 | 84 KB |
What is not included, and why
- Model weights. Not redistributed. The hashes and acquisition information identify the exact files tested.
- Third-party documents. Not copied. The register gives the address, the date read and a short excerpt.
- Personal data. Removed from the emails. The original emails and uncropped photographs are held by the author; their hashes are listed so they can be produced for verification.
Limits
Findings apply to the exact configurations tested, on one hardware platform and two topic packets. All 128 runtime-log hashes behind the Qwen finding are listed in a manifest made on October 12, 2026, and each copied log matches its archived log. 80 of the 128 were also sealed when the runs were archived; 48 were not, so for those 48 the evidence of integrity is the match between those two later computations.