Something has shifted in how software gets built. Agents now write large portions of the implementation. People steer architecture, quality and outcome. Delivery cycles that used to run for weeks compress into days.
The instinctive objection is that this must be less safe. That isn’t the interesting problem — and in regulated environments, it isn’t the real one either.
What we used to get for free
In a regulated environment, when someone asks why a system behaves a certain way, the answer is usually recoverable. There’s a named author on the commit. There’s a ticket describing the requirement. There may be a design note, and there’s almost always a pull-request conversation where two people argued about an edge case. In the last resort, there’s a person you can ask.
None of that was designed as an evidence chain. It accumulated as a by-product of humans working together, and change-management controls have been quietly leaning on it for years.
Compress the cycle and let agents do most of the writing, and the by-product stops appearing. You don’t lose provenance because agents are careless. You lose it because you used to get it for free, and now you have to capture it deliberately.
Four things that break
Authorship stops matching accountability. The commit is signed by a person. Neither the signature nor the blame view tells you who actually wrote the code. Regulated change control needs a named responsible person, not a named typist — but responsibility now has to be asserted explicitly, because it can no longer be inferred.
The reasoning is ephemeral. Which task specification produced this? Which files were in context? Which model, at which version? All of it exists for a few minutes and then evaporates. Six months later, “why does this do that?” has no answer — and it’s the question auditors, incident reviews and new team members all eventually ask.
Review becomes a control that silently fails. If human review is the compensating control for machine authorship, it has to actually be one. Approving two thousand lines in an afternoon isn’t review; it’s a signature. This is the failure we’d expect to see first, because it looks like compliance right up until someone examines it.
The agent is an undeclared system. The model that wrote a module in January isn’t the model that wrote another one in June. Behaviour changed between them. The agent has credentials, repository access and often environment access — and in most organisations it appears in no inventory, has no documented owner, and has never been risk-assessed.
What to capture instead
None of this argues against agent-based development. It argues for capturing deliberately what used to accumulate on its own:
- A generation record attached to the change — model and tool version, task specification, files in context, timestamp
- Explicit human accountability — a named person who reviewed and accepted, recorded as an assertion
- Deliberately scoped agent access — repositories, secrets, environments, external tools
- Risk-tiered review depth — not everything needs line-by-line scrutiny; deciding which does is the control
- The agent in your inventory — owner, permissions, model version, review cadence
- Retention that matches the code — provenance records are only useful if they outlive the sprint
Why this matters sooner than it looks
Most organisations adopting agent-based development are doing it for speed, and the speed is real. The questions come later — usually from an auditor, an incident review, or a client’s security questionnaire.
The awkward part is that provenance can’t be reconstructed retroactively. If you didn’t capture the generation record at the time, it’s gone. The organisations that will be able to answer these questions in two years are the ones capturing the record now, while it costs almost nothing.
There’s a symmetry here. Most of our work involves asking organisations two questions about their AI: what are you actually running, and what data feeds it? Agent-based development turns the same questions inward.