Defining the policy problem and evidence · Lesson 1

From a service gap to a decision-ready policy question

5 min read

Lesson objectives

  • Frame a referral decision card specifying population, baseline, comparator, gap and approval authority.
Reading tools

Many policy briefs begin with an appealing phrase: ‘We need to improve access to care.’ It does not tell an analyst which decision is pending, who is affected, or how improvement will be recognized. Here we turn it into a question with a defined comparator. A policy question is not a predetermined answer. It describes a choice that a specified body can make within its resources and authority. WHO guidance on synthesizing health policy and systems evidence emphasizes framing the question with attention to context, equity and the nature of evidence. We use that as a method, not as a local rule or a licence to decide funding.

Separate observation, explanation and goal. An observation has a definition, period and source, such as the share of referrals resulting in a visit within fourteen days. An explanation is a hypothesis about cause: perhaps clinic capacity is scarce, booking procedures are confusing, or patients cannot travel. A goal is a desired value selected through deliberation; it does not follow automatically from the observation. If the three become ‘appointments are scarce, so expand the clinic’, you have selected both cause and remedy before checking either. Overall capacity might be adequate while daily distribution or confirmation is the actual problem.

Write a seven-field decision question card. Population: who is in the denominator, and who could be excluded unfairly? Baseline: what happens now to that same group? Option: what specific feasible change is proposed? Outcome: identify at least one measure that matters to service users and define numerator and denominator. Horizon: when will costs and outcomes be counted? Perspective: who pays or bears costs, such as the purchaser and households, since a cost can move between them. Decision owner: who can authorize a pilot or funding change, and who only analyzes it? If a field is missing, record a data question rather than guessing before economic analysis.

Define the baseline before selecting an intervention. Suppose one month produced 1,000 referrals: 620 led to visits within 14 days, 250 to later visits, and 130 had no recorded visit by day 30. The mutually exclusive groups sum to 1,000. The early-visit proportion is 620/1,000=62%; the no-visit proportion at day 30 is 13%. Do not add these measures and call the result ‘delay’: one measures speed, the other completion at a different horizon. Check that the unit is a unique referral, not contacts or bookings; a referral can lead to several bookings. If visit records are uploaded late, the apparent no-visit group grows. That is a measurement problem before it is evidence of service failure.

Equity makes the aggregate less reassuring. Among 100 referrals classified as high need, 50 resulted in a visit within 14 days. Among 900 other referrals, 570 did. The early-visit rates are 50% and about 63.3%, while their numerators still sum to 620. Do not immediately call the difference provider bias. Case severity, specialty capacity, distance and coding quality may differ. It is a signal for separate investigation, and an improving overall average would not suffice if access for the high-need group deteriorated. Check that need was classified before the result and by a consistent rule, then request wait-time and specialty distributions rather than claiming causality from an aggregate table.

The comparator is continuation of the current pathway over the same period, described operationally: how referrals are sent, who books, and when a case closes. Two candidate options are centralized booking for high-need cases and follow-up confirmation messages for everyone. The first may change priority and capacity available to other groups; the second may improve attendance without creating more slots. Specify the mechanism, resources and possible unintended effect for each. Do not assume either option achieves the target simply because it is implemented. If a hypothetical target is 75% early visits, the arithmetic gap is 750−620=130 additional early visits per 1,000 referrals. That is a target gap, not an established intervention effect.

The decision card can ask: In a fictional network with 1,000 monthly referrals, would coordinated booking for high-need cases, compared with current booking, increase visits within 14 days over six months without reducing access for other groups, and what would the increase cost from purchaser and household perspectives? The assumed decision owner is the locally authorized service committee; the analyst presents options and assumptions. Before estimating economic value, seek referral and visit timestamps, clinic capacity, coordination costs, travel burden and data-completeness checks. If referrals cannot be linked to visits, do not report a falsely precise rate. Clinical evidence and legal obligations go to the appropriate professional and official authority, not to a decision made from this training exercise.

Before choosing the monitoring indicator, define its observation rule: a completed visit, starting from the timestamp of a unique referral, within 14 days. Referrals without an appointment remain in the denominator because they may be the very problem under study. Referrals sent near the data-extraction date need a full 14-day observation window; otherwise the rate falls merely because follow-up is incomplete. Compare months using the same start, end, specialties and linkage rule. Report counts alongside rates. A rise from 62% to 70% could result from excluding difficult referrals rather than improving access, so inspect exclusions and the high-need distribution.

Lesson summary

Reference solution: 500/800=62.