Advisory scope and stakeholder power · Lesson 4

A conflicting evidence register and a decision focused collection plan

4 min read

Lesson objectives

  • Frame an open advisory brief, map affected parties, test competing mechanisms, and prioritize decision-relevant evidence.
Reading tools

Advisory work can be troubled by conflicting evidence rather than a simple lack of evidence. A budget record puts subsidy expenditure at 100 million units, a partial survey estimates 30 million reaching low income households, an operator predicts that higher prices will reduce service, and timetable records suggest that some routes were cut before any policy announcement. Do not pick the sentence that best matches the sponsor's preference. Ask what each record measures: appropriated or paid expenditure, a benefit value or beneficiary count, a forecasted view or observed action. Two statements may look inconsistent because they answer different questions. Two reports may appear to corroborate each other while copying the same underlying data. Record each figure's origin and calculation instead of counting repeated quotations as independent support.

Build an evidence register with seven fields: claim, original source, period, population or denominator, estimation method, access limit, and consequence of error for the decision. The claim that 30 million reaches low income households may come from only 60% of recorded transactions. If absent beneficiaries use more subsidy, the estimate may be low; if they use less, it may be high. Do not assign the direction without testing. A claim that every fuel cost increase will pass through to fares could be useful testimony about an operator's intent, yet cannot establish every contract's outcome. Compare intent with price clauses and postchange invoices. Explain confidence in words such as 'low because the sample frame excludes people outside the system', rather than unexplained stars.

Conflicting figures do not imply that their average is true. If finance reports 100 million in savings while a transfer unit reports 60 million, check whether one is gross before compensation and the other net after it. Correct a definition mismatch first; if the definition agrees and the gap remains, inspect administration costs, baseline year, and consumption response. With qualitative evidence, speak to groups with different exposure rather than repeatedly interviewing the best connected party. Preserve an unresolved conflict in the briefing alongside the evidence needed to resolve it, so deleting a line does not create false certainty. 'The evidence is inconclusive' is actionable when connected to a decision threshold and a feasible test before the deadline.

Select further evidence by its value to the decision, not its ease of collection. Ask whether a plausible error range for each gap could reverse the preferred option. If transfer costs may range from 20 to 40 million, a sensitivity calculation is straightforward. If eligibility records miss half of the group the policy promises to protect, the option may become unacceptable however large the savings. Start with information that separates alternatives within six weeks: an audit of beneficiary sampling, actual transfer cost from a small operation, cost pass-through clauses, and transport service dates. Assign an owner and due date to each request and say what decision follows if it does not arrive. Do not promise a comprehensive survey that cannot be completed, and do not prioritize an easy study solely because it is available.

Check independence at the end. If two reports quote the same 30 million survey estimate, they are not two separate validations. If an operator denying a price effect also holds a contract allowing pass-through, inspect actual behavior rather than either dismissing or trusting the account alone. In a decision memo, classify evidence as supporting, conflicting, or inconclusive, and distinguish what is known now, learnable before the deadline, and testable only after a trial. Set a rule in advance: pause expansion if compensation costs cross a stated threshold or transfers fail to reach households; revisit phasing if reach is verified and service remains stable. This is not a ritual equal weighting of opinions. It connects evidential strength to the risk the decision maker is prepared to accept.

Lesson summary

Advisory work can be troubled by conflicting evidence rather than a simple lack of evidence.