Every monitoring rule has a threshold, and every threshold is a trade-off: lower it and the queue floods, raise it and real cases slip under the line.

Most tuning starts from the queue. The team is behind, a rule produces thousands of alerts and very few cases, and someone proposes raising the threshold. The proposal is reasonable. The question is how to prove that the alerts removed were not the ones that mattered.

Two ways to get it wrong

The first is tuning by volume: picking a threshold because it brings the queue back to a size the team can handle. That says nothing about what is lost. The second is never tuning at all, and letting a rule that once made sense keep firing on activity that is no longer unusual for the customer base.

Both leave the program unable to answer the examiner's question: why is the threshold where it is?

Above the line

Above-the-line testing looks at alerts the rule already produces and asks how many became cases or SARs. It shows which rules are noisy. It does not show what the rule misses, because activity under the threshold never reaches anyone.

Test below the line

Below-the-line testing samples the activity just under the current threshold and works it as if it had alerted. If those samples produce cases, the threshold is too high.

The same test works in the other direction. To raise a threshold, sample the band between the current value and the proposed one. If that band produces no cases, the change is supported. If it does, the team knows what it would give up.

• Define the band: the range just under the threshold, or between old and new values.

• Draw a sample large enough to be read with confidence, and record how it was drawn.

• Work each sampled item to the same standard as a live alert.

• Record every case or SAR the sample produces, with the reason.

• Keep the sample, the results and the decision together in one file.

Sample size is a judgment the team should write down. A sample too small to produce a single case proves nothing in either direction. The test plan should state how many items were drawn from each band, why that number was chosen, and what result would change the decision, before anyone works the sample.

Let the agents carry the sample

Working a sample used to take weeks of analyst time. Tipline's agents work the sampled alerts the same way they work the live queue, and the results are reviewed by the team.

That means the same grouping by beneficial owner, the same checks against 24 months of history, the same written reason for each recommendation. Sampled items that the agents recommend escalating go to an analyst, exactly as live alerts would. The analysts' decisions, not the agents' recommendations, are what count in the test results.

Because the work is cheaper, the sample can be larger and the test can run more often. A rule can be checked every quarter instead of once every few years.

Segments before thresholds

A single threshold across the whole customer base is often the real problem. A cash deposit rule set at the same value for a payroll account, a small grocery store and a newly opened trading company will be too low for one and too high for another. Splitting the population into segments, by customer type, product or expected activity, lets each threshold match the activity it watches.

Segmentation needs the same evidence as tuning. Each segment should have a written definition, a count of customers and its own below-the-line results. A segment created only to make a noisy rule quieter, with no test behind it, will be hard to defend. The agents work samples per segment, so the test results arrive already split the way the rules are.

Document the decision

The tuning record keeps the sample, the findings and the new threshold side by side, so the change can be explained to an examiner a year later.

The record should also say who approved the change and when it took effect, and it should be linked to the rule version. If a case appears later in the band that was tuned out, the team can go back to the record and see what the sample showed at the time.

What to do on Monday

Pick the rule with the most alerts and the fewest SARs over the last six months. Find the record that explains its current threshold. If there is none, that rule is the first candidate for a below-the-line test, before anyone proposes a new value.