Build a testable theory of change
6 min read
Lesson objectives
- Apply build a testable theory of change to a development case, explaining inputs and decision limits.
Please sign in to use all the tools
Once you sign in, the reading tools are yours: larger or smaller text, typeface, line spacing, focus mode, highlighting and bold, and notes saved to your account and gathered in My study.
A theory of change is not a decorative diagram at the end of a proposal. It is a testable account of why specified resources and activities are expected to produce an outcome for a defined group under defined conditions. Start with the baseline problem, then step backward: which constraint can the initiative change? If a shortage of interviews drives poor employment, training cannot automatically be the only remedy. Write a causal proposition: if eligible jobseekers gain access to documented interviews after their skills are aligned, and if firms have genuine vacancies, sustained employment may rise relative to usual service. The sentence identifies a mechanism, an external condition, an outcome, and an alternative. Ask what could disconfirm it. If interviews rise while offers do not, quality or demand may be a different constraint. A good theory makes failure informative rather than hiding it behind a broad activity description.
Draw a short chain with measurement at each consequential transition: eligible people, offered people, starters, practical-task passers, interviewees, people receiving offers, and people sustaining paid work. Do not place every measure in an ‘impact’ box. Offer, start, and completion measure delivery. Passing a job-relevant task is an intermediate skill measure. Sustained work is a final individual outcome, but its rate among participants alone does not show what would have happened without service. Initiative impact estimates the difference from a comparison alternative after addressing population and circumstance differences. Specify the measurement date, person as unit, and treatment of missing follow-up. Add a protective measure such as wrongful exclusion or data disclosure if the service pathway creates that risk. If a stage cannot be measured because records are missing, do not connect it with a solid arrow implying proof. Mark it as a pending hypothesis and test it.
Review context and distribution before fixing the theory. Matching may favor people with internet access and complete documents while leaving another eligible group behind. Add an alternate access route and an early check of offer gaps among groups. Firms' vacancies may also shift with the season; treat this as an external factor rather than automatic proof that trainers failed. Yet ‘the context changed’ must not excuse every weak result. If vacancies are known to be falling, reducing the cohort or choosing another sector may be the right design decision. Distinguish an assumption the team can influence, one it must monitor, and a risk it must not ignore. Give each an early signal and a predefined event that prompts adaptation or pause. Do not replace these rules with a later subjective judgment that results were ‘satisfactory’ without a prior definition.
Make the theory revisable when new data arrive. Record what each assumption predicted before delivery and what was observed afterward rather than silently editing the diagram until it appears perpetually correct. Suppose the team expected half of skill-test passers to reach an interview within two months but only a quarter do. Examine where the gap sits: fewer partner vacancies, candidate profiles not sent, or rejection of the practical task? The rate alone does not prescribe a remedy. Set a review date after the first delivery cycle and request a transition table of unique people and exit reasons, distinguishing voluntary withdrawal from a missed invitation. If identifiers or event dates are not linked, defer interpretation until the record is repaired. This is the minimum needed to learn from the trial, not an extra administrative ritual.
In the version shown to the resource committee, tie each decision to the limits of inference. Operating records can show whether service reached people and appointments occurred. Outcome follow-up can show what happened to participants, subject to missing data. A suitable comparison design can attempt to estimate the difference attributable to intervention under its assumptions. Economic value then requires resource costs, incremental benefits, and the people who receive them; it is not the number of jobs multiplied by an assumed wage. Confidence cannot simply be borrowed from one stage for the next. If the initiative passes eligibility gates but impact remains unknown, ‘continue a bounded trial and learn’ may be more honest than either ‘scale’ or ‘cancel.’ Give the next decision an explicit date and evidence requirement.
Lesson summary
Build a testable theory of change