AIToolworxThe TOPS Trap

What to do about it

Engineer trust around the model

The investigation began as a search for a capable local model and ended as a requirement: assume every model can fail, and build the system so that failure does not become authoritative.

Four principles

  1. The model can propose; policy and evidence constrain it.
  2. Verification happens outside the model.
  3. Authorization is separate from generation.
  4. Logging does not depend on the model.

Retrieval is not provenance, citations are not proof, local control is not complete control, and TOPS are not trust.

Figure 13 from the paper. AI has more than one supply chain: physical, model and software, execution, and control. All four converge on the enterprise trust boundary. Key insight: a single label such as Made in Taiwan does not reveal the full supply chain. Trust follows the dependency chain, not the shipping label.
From the paper, Figure 13. Trust follows the dependency chain, not the shipping label.
The engineering lesson: the model may reason about the evidence, but the evidence remains authoritative.

Eight recommendations

  1. Stop selling compute as trust. Publish trust characteristics alongside performance characteristics.
  2. Identify the exact artifact: model, revision, quantization, conversion path, tokenizer and chat template.
  3. Treat chat templates as versioned software, and make default system instructions visible to operators.
  4. Document the full model-conversion chain and keep a model bill of materials.
  5. Do not make the model responsible for enforcing its own evidence boundary.
  6. Verify citations independently; a citation is not proof of grounding.
  7. Test for false absence as well as hallucination.
  8. Separate observation from attribution when reporting results.

A proposed trust benchmark

The full paper proposes an Edge-AI Trust Benchmark: 16 trust domains in five families (evidence integrity, epistemic discipline, control and provenance, operational reliability, and system governance). It reports a trust profile beside a separate performance profile, and critical failures cannot be averaged away.

Proposed · Edge-AI Trust Benchmark

Sixteen domains, reported as a profile

Trust sits beside performance as a separate axis. Critical failures cannot be averaged away.

  1. 1Source grounding
  2. 2Evidence recognition
  3. 3Evidence conflict
  4. 4Citation integrity
  5. 5Uncertainty and epistemic discipline
  6. 6Instruction transparency and control provenance
  7. 7Behavioral symmetry
  8. 8Provenance and artifact integrity
  9. 9Supply-chain visibility
  10. 10Technical reliability
  11. 11Repeatability
  12. 12Reproducibility
  13. 13Independent replication
  14. 14Auditability
  15. 15Architectural containment
  16. 16Consequence-aware action control

Key takeaway Performance answers “Can it run?” Trust answers “Can it be trusted?”

Source: Benchmark Domains 1 to 16 in the full paper. This is an AIToolworx framework and has not been adopted as an independent standard.
Disclosure. The benchmark is an AIToolworx-developed framework and has not been adopted as an independent industry standard or accredited certification program. AIToolworx has a commercial interest in AI architecture, security consulting and application of the trust-assessment methodology it proposes.