Advisory mandate and evidence · Lesson 3

Evidence inventory and competing hypotheses

4 min read

Lesson objectives

  • Build an evidence matrix that separates observations from assumptions and tests at least two explanations
Reading tools

Diagnosis should not start by exporting everything from an HR information system. Put the mandate's decision at the top of a page, then write hypotheses that would imply different interventions. Rising departures might reflect relative pay, changed shifts, or weak supervision. Give each hypothesis a testable prediction. If pay is the stronger explanation, are exits concentrated in roles with a documented gap against a relevant market? If shifts matter, did the increase follow a roster change among groups exposed to it? If supervision matters, are there persistent differences between teams doing comparable work under similar conditions? A failed prediction does not necessarily eliminate a hypothesis, but it changes the weight placed on it.

Make an evidence table with columns for claim, source, period, population and denominator, collection method, source quality, alternative explanation, and the decision that might change. An exit register counts leavers but does not explain motives. An exit interview conveys a respondent's experience, yet respondents may differ from people who refuse. A survey records answers from participants at a point in time; poor response or missing language access can bias interpretation. Professional expertise, stakeholder views, published research, and organizational data contribute different forms of evidence. Weigh them for the actual question instead of treating any one as a final proof. CIPD's public people-analytics guidance describes this broader evidence practice; it does not say that one stream makes all others unnecessary.

Test definitions before numbers. A departure rate requires a specified numerator and a denominator, such as average workforce size over the same period. Do not compare 18 leavers among 240 workers with ten among 300 without displaying the different exposure. Do not combine voluntary resignations, project-contract endings, and redundancies under one label when the decision is about retention. In seasonal work, a higher count may reflect the volume of work rather than a worse employee experience. A change in exit-recording software can create an administrative break in the series. Record each metric's definition, version, and change date before plotting a trend. Privacy constrains permissible aggregation: the wish for a stronger analysis does not authorize access to individual records without purpose and permission.

End the inventory with a qualitative confidence judgement and its reasons: high for the departure count if records are complete, lower for causal interpretation when interviews are few and two conditions changed together. Do not turn that judgement into an invented probability. Choose a verification step with high decision value; classifying known contract endings may be faster and cheaper than a large new survey. If evidence still cannot distinguish two costly options, a reversible pilot or a request for specifically missing evidence is a defensible recommendation. Confident prose cannot repair an identification problem.

More interviews will not distinguish transport from supervision if every question leads toward the same preferred cause. Write predicted observations first. The transport account predicts access trouble among people relying on the new route, including days with similar supervision. The supervision account predicts different feedback or task allocation among those working with that manager, including people who travel easily. Seek contrary cases: someone who never uses transport yet left, a route user who stayed, and another shift with the same manager. These are not a controlled experiment, but they constrain a hasty story. Consider interaction too: late arrival may intensify a harsh management response. If safe comparison is unavailable, repair a demonstrated access barrier while continuing a fair review of supervisory practice, not an accusation based on timing alone.

Lesson summary

Build an evidence matrix that separates observations from assumptions and tests at least two explanations