What was found
Six findings
Each finding states what the tests showed, what they did not show and where the files are. Findings apply to the exact configurations tested.
Evidence did not guarantee grounding
Models added facts that were not in the packet, ignored the citation requirement, or claimed the packet lacked information it contained. Qwen 2.5 Coder showed the last error: in the China grounded condition it said the evidence was absent when it was present.
The sharpest case is Ministral 3 3B. Its locked record was adjudicated claim by claim: of 93 claims, 15 (16.1%) were supported by the packet and 45 (48.4%) were unsupported or contradicted.
Evidence boundary · Ministral 3 3B
What the evidence supported, claim by claim
Each of the 93 claims in the locked test record, checked against the packet the model was allowed to use.
- Supported 15 (16.1%)
- Partly supported 33 (35.5%)
- Unsupported 40 (43.0%)
- Contradicted 5 (5.4%)
45 of 93 (48.4%) were unsupported or contradicted by the evidence the model was given.
Key takeaway Fewer than one claim in six was fully supported. Nearly half crossed the evidence boundary.
Hidden instructions shaped the effective prompt
In all 32 runs of Qwen 2.5 Instruct and Qwen 2.5 Coder with no system message, the runtime log shows the sentence “You are Qwen, created by Alibaba Cloud. You are a helpful assistant.” added to the prompt. It was absent when the study’s system message was supplied, and absent from all 64 Qwen 2 and Llama 3.2 logs.
The same sentence is in the official upstream Qwen2.5-1.5B-Instruct chat template. That shows the sentence exists upstream. It does not show which template the accelerator-hosted build used, or who put it there. In a separate test, a request for an exact response was overridden by a phrase from the system layer.
Control provenance · four accelerator-hosted models
The prompt you type is not the prompt the model receives
128 runtime logs. In every run where the study supplied no system message, two of the models had a sentence added that nobody typed.
Key takeaway The sentence appeared in 32 of 32 such runs of the two Qwen 2.5 models and in none of the other 96 logs.
Fluency and citations were not proof
Ministral’s answers were fluent and organized, which made the provenance failures harder to see. Of the 37 claims that carried a citation, 5 (13.5%) cited correctly and 18 (48.6%) cited a source that did not support the claim. In the three China grounded trials shown, it gave no citations at all.
In one answer the model credited a rationale to Andrew Jackson and cited source S1 for it. S1 does not say that. A citation shows only that the model produced a citation marker. Source adherence requires a verifiable link between the claim and the evidence.
Citation integrity · Ministral 3 3B
A citation is a marker, not proof
Among claims that carried a citation, fewer than one in seven cited correctly.
- Cited correctly 5 (5.4%)
- Cited, partly supports 14 (15.1%)
- Cited, does not support 18 (19.4%)
- No citation 56 (60.2%)
18 of the 37 cited claims (48.6%) pointed to a source that did not support them.
| Topic · condition | Claims | Supported | Unsupported or contradicted | Claims with a citation |
|---|---|---|---|---|
| China · stock-grounded | 32 | 7 | 14 | 0 |
| China · neutral-grounded | 24 | 3 | 15 | 0 |
| USA · stock-grounded | 21 | 3 | 8 | 21 |
| USA · neutral-grounded | 16 | 2 | 8 | 16 |
Key takeaway A citation shows only that the model produced a marker. Source adherence needs a verifiable link between claim and evidence.
Behavior changed with the topic
In matched tests, some configurations answered one topic substantively and gave a non-answer on the other. Qwen 2.5 Instruct reversed its pattern when the system message changed. That does not fit a simple model in which one topic is permanently suppressed.
The results show that topic-conditioned behavior has to be measured. They do not show why it occurs. The study reports the observation first and attributes a cause only where the evidence supports one; here it does not.
Speed and hardware did not predict trust
Gemma ran on the CPU alone at roughly 6 to 7 tokens per second (mean 6.7) and showed the strongest source discipline. Several accelerator-hosted 1.5B models ran at similar speeds with repeated grounding failures. Ministral ran at roughly 2.3 to 3.6 tokens per second and produced fluent answers with weak source separation.
Computational efficiency and reliability behaved as separate dimensions. One result points the other way from the failures above: the tested Gemma configuration repeatedly said when the evidence did not establish a requested fact. That shows source discipline is achievable for a compact local model. It does not show that Gemma is trustworthy in general.
Speed is not trust · six controlled configurations
The fastest chip did not buy the most reliable answers
Position along the bar is speed. The note beside it is what the study observed.
Tokens per second (approximate, from elapsed-time means)
Key takeaway Computational efficiency and reliability behaved as separate dimensions.
The box says Taiwan. The model says Alibaba.
Check the box. Raspberry Pi. Empowered by Hailo. Made in Taiwan. Not one mention of Alibaba or China: not on the box, Raspberry Pi’s product page or Hailo’s launch announcement, and Raspberry Pi’s launch article lists Qwen as a model name without saying whose it is. Yet the Qwen models running on that board were built by Alibaba, in China, and the instruction the study found in the prompt names Alibaba. The label says where the hardware was made. It does not say who trained the model, who wrote its templates, or who can change its instructions.
Where something was made, or where its maker is headquartered, is not its supply chain. The paper tracks three. Physical: the hardware. Software and model: the weights, runtime and chat template. Control: the instructions and policies that steer behavior, and who can change them. A trusted device is not a trusted model, and local does not mean every rule was written locally.
This is a supply-chain question, not a nationality test. The study does not show who inserted the sentence, or why the models behaved as they did.
Sources. Author’s photographs of the retail box, taken October 2, 2026 (evidence archive, the 05_Hardware-Evidence folder; paper Appendix A, S9). Raspberry Pi: AI HAT+ 2 product page and launch article. Hailo: press release of July 22, 2025. Alibaba: QwenLM/Qwen3 and the corporate FAQ. Runtime logs, claim C3, and the upstream template (Qwen/Qwen2.5-1.5B-Instruct, revision 989aa7980e4c…). Full paper, Section 9, pages 187 to 193. Pages were read October 16, 2026.
Finding 6 · Supply chain
The box says Taiwan. The model says Alibaba.
Where something was made, or where its maker is headquartered, is not its supply chain.
Physical
The hardware
Raspberry Pi platform · Hailo-10H accelerator · board labelled “Made in Taiwan”
On the box and the product pages.
Software and model
The weights, runtime and chat template
The Qwen model family, built by Alibaba in China · the runtime that loads it
Not on the box. Not on Raspberry Pi’s product page or Hailo’s launch announcement.
Control
The instructions that steer behavior, and who can change them
System messages · templates · policies. Found in the prompt: “You are Qwen, created by Alibaba Cloud.”
Found only by reading the runtime logs.
Key takeaway A trusted device is not a trusted model, and local does not mean every rule was written locally.
What the results do and do not establish
Supported by the tests
- Source grounding was not reliable across the tested configurations.
- Retrieval is not provenance.
- The user-visible prompt was not always the effective prompt.
- Model provenance alone was insufficient; execution and control provenance also mattered.
- Some configurations showed topic-conditioned behavior.
- Confidence did not reliably track evidence or correctness.
- Gemma showed the strongest source discipline of the configurations tested (a limited claim).
Not established by the tests
- CCP direction of the observed outputs.
- Malicious intent by any vendor.
- That Alibaba authored every observed template element.
- That Hailo caused the model failures.
- That Hailo has no engineering value.
- That local models are inherently unreliable.
- That CPU inference is inherently more trustworthy.
Every headline number links to its trace on the evidence page. The method is on the method page.