The TOPS Trap: Executive Summary

In brief

1. Evidence does not guarantee grounding. Giving a model sources and telling it to answer only from them did not reliably keep it inside those sources.
2. The prompt you type is not the prompt the model receives. Chat templates and runtimes add instructions the user never sees.
3. Fluency and citations are not proof. Well-written answers and citation markers still carried unsupported claims.
4. Behavior changed with the topic. Matched China and United States tests produced different behavior in some configurations. The study measures this and does not assume a cause.
5. Speed did not predict trust. The configuration with the strongest source discipline ran at about the same speed as accelerator-hosted models that failed more often.
6. The box says Taiwan. The model says Alibaba. Nothing on the box mentions Alibaba or China, yet the Qwen models running on it were built by Alibaba, in China. Where something was made, or where its maker is headquartered, is not its supply chain.

The setup was a Raspberry Pi 5 with the Raspberry Pi AI HAT+ 2, whose Hailo-10H accelerator is listed at 40 TOPS (INT4). Six configurations were tested under controlled conditions. A seventh, DeepSeek R1 Distill Qwen 1.5B, failed a basic three-sentence instruction check during early exploration and was excluded before controlled testing. It is not scored and not ranked.

Need the method, every result and the context? Read the full paper.