Agents that recommend dispositions are models in the sense of model risk management guidance, and they should be governed like one.

SR 11-7, the supervisory guidance most US banks use for model risk, defines a model broadly: a quantitative method that turns inputs into estimates used for decisions. An agent that reads an alert and recommends clearing or escalating it fits that definition. So does a rule that scores a sanctions match. The fact that the agent writes in sentences does not change what it is.

Why compliance teams resist the label

Calling an agent a model brings it into the model inventory, the validation calendar and the annual review. That feels like overhead for a tool that is meant to save time. It is not. The model risk team already has the vocabulary examiners use, and borrowing it is faster than inventing a new one.

It also protects the program. An agent that was never validated is an agent whose recommendations cannot be defended. When an examiner asks why 3,077 of 3,118 alerts were closed before an analyst saw them, the answer has to be a document, not a demo.

What examiners ask

What is the model for, how was it validated before use, how is it monitored, and how are changes controlled. Each answer should exist in writing before the first alert is worked.

• Purpose: which queues the agent works, and which decisions it may recommend.

• Data: which systems it reads, and which it never touches.

• Validation: how its recommendations were tested before go-live.

• Monitoring: what is measured each month, and who reads it.

• Change control: how a new version is approved, and how it can be rolled back.

• Human oversight: who signs, and when a second signature is required.

Validation against closed cases

The cleanest test set is one the bank already has: alerts and cases that analysts closed, with their dispositions. The agent works them again without seeing the outcome, and its recommendations are compared with what the analysts decided.

Agreement alone is not enough. A model that agrees 99 percent of the time and misses the cases that became SARs has failed. Validation should report agreement separately for alerts that were cleared and for alerts that were escalated, and every disagreement on an escalated case should be read by a person.

Below-the-line samples belong in the same exercise. If the agent is working alerts that rules would never have fired on, its results there need the same scrutiny.

Monitoring in production

A validated model can drift. Customers change, products change, the rules feeding the queue change. The monitoring that matters most is the simplest: how often analysts overturn the agent, by queue, by month.

A rising overturn rate on one queue is a signal to look, not a verdict. It may mean the agent is wrong. It may mean a rule upstream changed and the alerts look different. Either way, someone should know within weeks, not at the annual review.

Who signs, and when twice

In Tipline, the agents recommend and an analyst decides every case that leaves the queue as a case. No agent files a SAR on its own. Above a risk score the team sets, a second analyst signs as well. The threshold is a setting, written into the governance pack, and changing it is itself a logged change.

This is the part examiners tend to test in person. They pick a case and ask who made the decision. The answer should be a name, a timestamp and the evidence that person saw.

What we ship with each release

A validation score against labelled closed cases, the agreement rate between agents and analysts by queue, and a change log that ties every behavior change to a version.

Each release note states what changed in the agent's behavior, which queues are affected and what the validation showed before and after. A bank can hold a release, run it on its own closed cases first, and approve it through its usual model change process. At Northbank Retail, one of the stories on this site, median time to disposition went from 2d 4h to 6m 41s. That figure is only worth something because the record behind each disposition can be opened.

What to do on Monday

Ask your model risk team whether any agent or scoring tool in the AML program is in the model inventory. If not, write the one-page purpose statement first: what it does, what it reads, who signs. Everything else in the governance pack builds on that page.