Prioritise a proportionate review

You have ten analyst-days to review a monitoring programme and six legitimate questions from a committee. The questions cannot all receive equal depth. Good selection is not taking the first ten convenient case files. Begin by asking which information could change a decision the institution must make soon. If a new rule is about to launch, feed completeness and shadow testing matter before a lengthy redesign of dashboard appearance. If transactions may have dropped out of monitoring, test that possibility before optimising alert-closure speed. This is the value-of-information idea: a small test may be valuable because it distinguishes two actions, while a large report adds little certainty. Its value need not be expressed as a precise monetary number. A reasoned qualitative ordering can balance the consequence of being wrong against the time needed to obtain evidence.

List decisions before listing tests. Here they might be: keep the current rule temporarily, add a targeted manual control for a feed gap, or commission a broader redesign test. For each, name the decisive evidence, the date it is needed and the authorised decision owner. Protecting a channel requires the count of eligible transactions absent from monitoring, not the total alert count alone. Changing a threshold needs a sample showing the effect on meaningful cases and investigation capacity, not just a closure percentage. Classify tests as essential before the decision, useful if time remains, or deferred with the temporarily accepted risk stated. Deferred does not mean forgotten: give it a reopening date and an event that brings it forward immediately if the exposure worsens.

Estimate the real cost of each test: obtaining permission, extracting data, cleaning it, analysing it, checking results with staff who understand the rule, and recording the decision consequence. If one extract takes eight days and another test can expose a broken feed in one day, begin with the second. Speed does not justify a biased sample, however. Specify the test population, sampling method, period and fields, failure criterion and response to failure. When reviewing closed alerts, do not let the team that closed them hand-pick only cases it considers exemplary. Combine an appropriate random sample with higher-exposure cases and document why each stratum was selected. Do not claim statistical generalisation if the sample design does not support it. State the result's reach and limit.

Present two or three work plans instead of one as if it were inevitable. A rapid coverage-preserving option tests the feed and replays known transactions, then adds interim protection if it fails. A broader option adds a closure sample and measures threshold effects by product segment, but needs more time and access. A delay option retains the current rule with temporary surveillance while evidence is gathered; it has operational cost and delay risk. Compare loss-of-detection exposure, investigation capacity, feasibility and evidence quality. The advice may be to start with the rapid option and move to the broader one if interim protection fails or uncertainty persists. Record a stop rule for ordinary review: confirmed loss of a whole transaction category goes immediately to the control owner for an urgent protection decision, without waiting for the advisory report.