Turning a policy option into a testable pathway of change
5 min read
Lesson objectives
- Link follow-up messages and appointment capacity to completed visits and suitable indicators.
Please sign in to use all the tools
Once you sign in, the reading tools are yours: larger or smaller text, typeface, line spacing, focus mode, highlighting and bold, and notes saved to your account and gathered in My study.
‘Launch a digital follow-up service’ sounds like a policy option but names a tool without showing how it changes an outcome patients value. A pathway or theory of change links resources, activity, output and outcome, with assumptions at each step. Its job is not to make success look inevitable. It reveals the link that needs testing, other options that might achieve the same end, and unintended effects. For a visit after referral, a phone can remind a patient but cannot create an available appointment. If clinic capacity is the bottleneck, more messages may increase frustration rather than access.
Define the population and event first. A fictional network has 2,000 referrals for follow-up within 30 days; 1,200 people completed a visit, or 60% if each referral represents one person in that period. The body wants a higher rate but has not established the cause of the gap. Map two possible routes: a confirmation message before the appointment or a coordinator calling people without a confirmed booking and finding another slot. Identify resources: a system and messaging fees for the first, staff time and scheduling capacity for the second. A booked appointment is not a completed visit; someone may book and not attend, or attend after the specified horizon.
For messages, inputs are a referral list, valid contact numbers and privacy oversight. The activity is sending a message and receiving a reply. Outputs are messages sent, delivered and confirmed. A near outcome is a completed visit within 30 days. A more distant outcome might be fewer avoidable complications. Each arrow needs a condition: delivery requires a correct number, confirmation requires comprehension, and a visit requires a slot, transport and perhaps clinical approval. Do not jump from message count to fewer complications. The latter needs appropriate measurement and comparison, possibly over a period longer than one follow-up cycle.
In training data, 1,600 messages were sent, 1,100 recorded as delivered, and 800 recipients of delivered messages visited within 30 days. Delivery was 1,100/1,600=68.75%; visits among delivered-message recipients were 800/1,100≈72.7%. Do not call 72.7% the effect of messages. People without a delivered message may differ in digital access, location or need, and we lack their visit rates and a comparable control group. Sending to only 1,600 of 2,000 referrals also raises a selection question: were 400 excluded for lack of a phone or for an administrative reason? The targeted group's rate could rise while the overall access gap remains.
The counterfactual comparator is what would have happened to the same people in the same period without the new option. We cannot observe both states directly, so design a defensible comparison. Authorized groups might be assigned to alternatives in a trial, or comparable areas might be studied with prior trends checked, depending on ethical and operational limits. A simple before-and-after result may capture seasonality or clinic expansion. If coordination is offered only to the highest-need group, comparing its raw outcome with everyone else is unfair because groups differed beforehand. Explain the comparison choice and its limits before giving an effect estimate.
Specify a deliverable option rather than a slogan: ‘For three months, coordinate booking for referrals without a confirmed slot after seven days, at no more than 100 cases weekly, escalating clinical-risk cases to the appropriate professional.’ Measure eligible cases, contacts, 30-day visits, waiting time for those outside the programme, and staff hours. Compare with current practice and messages using the same population and horizon. If capacity is fixed and coordination merely moves one group ahead of another, expanding capacity might be a better option to test. Continuation should depend on results, not the message-delivery count alone.
Keep process and outcome indicators separate. Message deliveries divided by messages sent measure implementation; visits within 30 days divided by eligible referrals measure an outcome. Add a harm indicator such as waiting-time changes for those not included. Name the data source, extraction date and reviewer of missing records. A 70% visit target means 1,400 of 2,000, a 200-visit gap from the 1,200 baseline. It does not imply that messages will create 200 additional visits. Estimate each option's effect through comparison, then consider cost and distribution before recommending scale. Deferring expansion may be responsible if safety evidence or authority remains unresolved.
A pathway can reveal data missing from the headline indicator. Available appointment slots, average call length and time to respond to a message constrain delivery. If the clinic has only 100 open slots monthly, predicting 200 extra visits from reminders alone is inconsistent unless the plan redistributes or expands capacity. Put that constraint beside the target. If coordination requires personal health information, specify the minimum necessary data and the responsible handler under the institution’s actual rules; a worthwhile objective does not by itself authorize unrestricted sharing.
Set a learning rule before the pilot so every result is not explained after the fact. For example, rising overall visits without longer waits for excluded patients, at an affordable staffing cost, might justify assessing expansion; worsening waits could trigger a change to allocation. These are training examples, not universal legal or clinical thresholds. Name a possible safety signal that goes directly to a professional review, and distinguish a pause for bad data from a pause for possible harm. An advance rule makes an indicator serve a decision rather than decorate a dashboard.
Lesson summary
Reference solution: valid-phone coverage is 600/900≈66.