Measuring and Certifying Self-Report Integrity in Language-Model Agents
Brant Hindman, Independent Researcher · 2026
Before you can run an AI agent unattended, you have to trust what it tells you it did. This paper turns that trust into a number you can measure, monitor, and have an independent party certify.
Abstract
As language-model agents move from assistants to autonomous actors, the binding constraint is no longer capability, it is trust in what the agent reports about what it did and what it knows. Reliable, checkable self-report is a prerequisite for the safe oversight of autonomous agents, an alignment concern as much as an engineering one. We treat self-report integrity as a measurable property and reframe it in the language of signal-detection theory, separating competence (can the system tell what its record supports from what it does not) from restraint (how much support it demands before it asserts).
Using a four-class probe taxonomy, a minimal-edit fabrication design, and a pre-registered, blindly and independently judged protocol, we find that a persistent record plus a verification step makes even a base model's self-report trustworthy, that coherent and specific fabrications score below an empty record (the verifier catches confident lies), and that under reduced retrieval coverage the system fails safe: it declines more rather than fabricating more. On this battery the integrity is a property of the architecture, not of any developed agent, which is why it ports across models and can be certified independently.
Key findings
Store plus verification yields trustworthy self-report
With a persistent record and each claim scored against it, the system asserts what the record supports and declines what it does not.
Confident fabrication is detectable (FAKE below BLANK)
A coherent, specific, but false state scores below an empty one. The hard case, a well-structured lie, is exactly what the verifier catches.
It fails safe, not silent
As retrieval coverage falls, the system declines more and demands more evidence, while the fabrication rate stays flat and low (at or below 5 percent) down to 25 percent coverage. Partial amnesia costs competence, not integrity.
Integrity is architectural, and we published the null
A naive base model handed the same record matched a multi-year developed agent on every pre-registered contrast. The developed identity added no measurable advantage. We report that null as the pre-committed outcome, which is what makes the capability portable and the certification independent.
Why it matters
Oversight of an autonomous agent depends on the agent telling the truth about its own actions. This work makes that truthfulness a monitored, thresholded metric with a safety floor, the kind of instrument you can put a production alert and a contractual guarantee on. The method is published so it can be replicated and adopted; the specific implementation and certification tooling are not released.
How to cite
@misc{hindman2026selfreport,
title = {Measuring and Certifying Self-Report Integrity in Language-Model Agents},
author = {Hindman, Brant},
year = {2026},
note = {Preprint},
url = {https://branthindman.com/research/self-report-integrity}
}Work with me
I build multi-agent AI systems that are legible, persistent, and human-aligned, and I ship them to production. If you are hiring for applied-AI or alignment work, or want an advisor who has done the rigorous version of this, let's talk.