Agent readiness, and what your last release cost you
Two engagements against ARS v1.0: an assessment of one production agent deployment, and the change-impact analysis that keeps its findings from expiring.
Two questions, and they decay at different rates
The first is is this agent ready — whose identity it acts under, what it can destroy, who approved that, and how you would know afterwards. Enterprise review boards do not ask whether an agent is impressive; they ask those four, in that order, and a programme that cannot answer them re-litigates its own existence at every review cycle.
The second is is it still ready. An assessment describes a system on a particular day. The agent then ships twice a week, and by the second month nobody can say which of the findings still hold. That is the part of an assurance programme that decays fastest, and the usual answer — re-run the whole review — costs more than most teams will pay more than once.
Who is doing the assessment
I have built and run agentic AI in production inside a regulated utility, where the deployment had to clear legal, cybersecurity and enterprise architecture review before it could ship and again before it could grow. The controls in this standard are the ones those reviews asked about, in the order they asked about them, and the failure notes attached to each check are drawn from what actually goes wrong rather than from a threat taxonomy.
The standard
ARS v1.0 is published in full, with a content hash, so an assessment cites a specific text rather than a version number. 41 checks across 8 tiers, each with a pass criterion, the way it fails in production, and one flag naming the instrument that can read it.
It is a draft, and it says so in its own posture field: no second party has applied it without me. Until that happens it is a rubric with a versioned changelog rather than a standard, and the document carries that statement rather than burying it.
What has changed is what the instruments have been pointed at. They have now read 200 public agent repositories that other people wrote, at pinned commits, and the result is measured rather than asserted: of 4,445 tools observed, 103 state anything at all about whether they change state. Where a tool does say so, it is almost always accurate. The gap is not that declarations are wrong — it is that there are nearly none, which is the first half of an assurance argument going missing at ecosystem scale. The findings, with every figure's three buckets →
That is evidence about the problem, not about me. It is still true that no client has applied this standard, and that no repository in that corpus has ever authored an Agent Contract. A scan of somebody else's code is not a second party applying a rubric to their own system, and this page is not going to describe it as one.
Read ARS v1.0 → · how a control is assessed →
A worked example
20 of the 41 checks can be read from a repository, and a harness does that part. Here is its output against a deliberately unsafe toy agent: the verdict, every finding with its file and line, and a remediation for each one.
It is also an honest demonstration of the limits. The harness cannot award better than Conditional to anything, at any score, because 21 of the checks are not readable from code at all. 13 of the 20 it does read are capped at a score of 1, because static analysis cannot tell a working mechanism from the appearance of one. Those ceilings are published in the standard rather than held in the tool.
Read the sample assessment report →
The two engagements
Agent Readiness Review
2 weeksA full assessment of one agent deployment against all 41 checks in ARS v1.0: the 20 that can be read from code and configuration, the 11 that need a running system, and the 10 that need interviews and document review.
- A scored assessment against every check, with the evidence each score rests on and its location.
- A remediation plan, ordered by blast radius rather than by effort.
- The control-to-mechanism mapping ARS-41 asks for, in the form legal, cybersecurity and enterprise architecture review actually want to receive.
- The harness output, so your team can re-run the static subset on every change without me.
- An Agent Contract for the deployment — a provider-neutral declaration of what it is permitted and expected to do. It is the input the second engagement runs on, and it is yours whether or not there is a second engagement.
- Not a certification. Nothing here certifies anything, and the standard says so in its own scoring model.
- Not a penetration test, and not a code audit. It assesses the controls around the agent, not the whole application.
- Not a judgement about whether the agent is good at its job. That is a capability question; this is a readiness one, and conflating them is how a security review gets answered with a demo.
Change-Impact Analysis
per release, or on a cadenceThe agent shipped again. This answers, for that specific change: which evidence it invalidated, which evidence survived, which tests have to be re-run, which approvals have to be re-taken, and whether it can ship. The comparison is semantic rather than textual — it reports that the agent gained the ability to transmit to arbitrary external recipients, not that a line was added to an array.
- A named list of what changed, drawn from a closed vocabulary of 35 change categories — each one carrying the controls it touches, a severity floor it cannot fall below, and the nearby safe change it must not fire on.
- The evidence arithmetic: what is invalidated, and what is retained by identity. An item is retained exactly when its id is in the prior ledger and in no invalidation set. Nothing is re-derived and nothing "still looks fine".
- A verdict — PROMOTE, REVIEW or BLOCK — from 11 named rules, each printed in the artifact, with no score anywhere in the path and the exact conditions that would clear it.
- Where source is available, a reconciliation: what the implementation carries that the declaration does not. A tool defined only in code is still a tool.
- The machine-readable artifact, canonically hashed with timestamps excluded, so re-running it over the same inputs produces the same bytes and your team can dispute any line of it.
- It compares declarations. Where observations are supplied it also reports what the implementation carries and the declaration does not — but a contract that is simply wrong about a fact no adapter can see is not caught by this or by anything else here.
- It is not a re-assessment. It says which parts of the last one stopped being true.
- One agent, two versions — a new send capability, a widened scope, a new external destination, a model version change and one approval requirement weakened. Verdict: BLOCK, with what would have to change to reach review.
- The contract said read-only; the TypeScript could send — the declaration did not change and the agent did. The tool is declared nowhere as data; it was found by parsing the source. Verdict: BLOCK.
- Evidence produced rather than asserted — a reference agent in two configurations, seven controls probed on a running system, and the difference between evidence that a control holds and evidence that it does not. It also states plainly what a run against a local fixture cannot establish.
Each page ships every artifact it was computed from, so the result can be recomputed rather than taken on trust.
How the first two weeks run
| Week | What happens |
|---|---|
| Before | A short call on what the agent does and what it can reach. Some deployments do not need this assessment, and it is cheaper for both of us to find that out first. |
| 1 | Static subset against the repository, then system access for the 11 runtime checks: identity propagation, server-side authorization, the audit plane, replay, tenant isolation, injection testing. |
| 2 | Interviews and document review for the 10 governance checks: approval volume, data-flow policy, exfiltration channels, retention, incident process. Then findings, scoring and the remediation plan. |
| After | You keep the harness and the Agent Contract. The static subset re-runs on every change without me, which is the part that would otherwise expire the day the assessment ends. |
Price, scope and how to start
Engagement pricing on inquiry. Scope is one agent deployment. A second deployment in the same organisation is usually faster, because most of Tier 3 and Tier 7 is shared.
Email Contact@thirdocular.com with what the agent does and what it can reach. That is enough to say whether this is the right assessment for it, which is sometimes no.