Problem framing and theory of change · Lesson 2

Draw an intervention logic with assumptions and rival explanations

8 min read

Lesson objectives

  • Draw an intervention logic with assumptions and rival explanations
Reading tools

The mentoring team in the previous lesson named employment as its aim but measured confidence among respondents after sessions. This is a substantive gap. Confidence may help people apply for jobs, yet it may rise without changing opportunities when suitable vacancies are scarce. A theory of change is a conditional account of how activities are expected to lead to outcomes. It is not a promise that the links have occurred. It gives the team assumptions that can fail and shows where implementation evidence is needed and where an effect question demands a comparison.

Start with activities the initiative controls: inviting eligible people, assigning mentors and providing four sessions. Next identify an observable output, such as how many received the planned dose. Then specify a plausible near change, such as the ability to prepare a suitable résumé, a later behavior such as submitting documented applications, and a more distant outcome such as gaining appropriate work. Do not jump from “we delivered 720 sessions” to “we created 90 jobs.” The first counts service delivery. The second attributes labor-market outcomes that depend on vacancies and employer decisions beyond the team’s control.

For every arrow, write the mechanism and a condition. Sessions may improve résumé skill if the content fits participants’ needs, participants attend, and feedback is understood. Skill may lead to applications if people have access to a device, time and suitable vacancies. Applications may lead to jobs if employers respond; hiring season may matter. If applications do not change, do not immediately declare the mentoring worthless. Check whether the planned dose was delivered, whether skill changed, and whether opportunity access was the obstacle. This explanation work still does not establish impact; skill may change for other reasons.

A useful diagram has an evidence column alongside every node. A session register needs a rule: did someone complete four full sessions or merely appear on four sign-in sheets? A résumé standard needs a rubric and reviewer guidance so ratings do not drift across mentors. For applications, specify the follow-up month and how they are verified; participant reports are useful but differ from submission receipts. For the distant outcome, define the type of job and measurement time instead of using the unqualified word “employed.” The theory thereby becomes a staged measurement plan, not a decorative chain of arrows.

Record at least one rival explanation for each consequential outcome. Applications might rise because another organization opened a job fair during the same period. They might fall despite better skill because of a seasonal slump. If centers differ in internet access, an overall average could hide a decision-relevant disparity. The team cannot enumerate every possibility. Select rivals that could change the scale decision and can be tested or at least monitored. Mark each node and arrow as observed implementation, plausible mechanism, or impact requiring a comparison. These labels prevent observations from being quietly promoted into causal conclusions.

Rank assumptions by importance and testability. “Every graduate finds a job” is not a useful assumption; it is the outcome under investigation. “A participant can access at least one suitable vacancy within six weeks” can be examined with a vacancy record and a definition of suitability, though access may vary across neighborhoods and occupations. Label assumptions as critical or supporting, and note whether the needed evidence already exists. If a critical link fails, revise the interpretation and recommendation even if other indicators look strong. The diagram becomes a working decision tool rather than an illustration shown only at the launch meeting.

Keep planned dose separate from benefit. Four sessions may be the intended dose, yet an attendee who never uses feedback may not receive the mechanism in the intended way. Another person may benefit from two focused sessions. Do not redefine success after seeing who responded. Specify exposure levels in advance and retain separate measures of attendance, delivery quality and application of feedback. When a later analysis distinguishes the effect of an offer from the effect of actual participation, the path from offer to uptake is visible. Restricting the sample to completers to claim an effect recreates the selection problem from the previous lesson.

At a weekly review, ask for three kinds of evidence. Implementation evidence asks whether the activity reached people as planned. Mechanism evidence asks whether the targeted skill improved or the obstacle eased. Outcome evidence asks whether the specified application or employment result occurred. Strong attendance without a skill signal suggests revisiting the mentoring content. Improved skill without applications suggests examining access to vacancies. Applications rising across groups during a public job fair warn against crediting the initiative without a comparison. The appropriate decision follows the location of the broken link rather than the most favorable indicator.

A theory of change can be revised when new information appears, but log the date, rationale and version. If the team adds a digital application session after discovering a platform barrier, that session is a new intervention component; it does not prove the original version worked. Keep outcomes specified before the revision separate from exploratory outcomes noticed afterward. Otherwise almost any later change can be portrayed as fulfilling an earlier goal. Version history will matter for comparison design, analysis and truthful communication. A clear chain with assumptions, evidence and limits is more useful than dozens of boxes that cannot be tested.

Lesson summary

Apply draw an intervention logic with assumptions and rival explanations to a hypothetical case and justify a bounded decision.