Frame the problem and decision criteria from conflicting evidence
7 min read
Lesson objectives
- Frame a testable problem hypothesis and distinguish non-tradable constraints from criteria that can be compared.
Please sign in to use all the tools
Once you sign in, the reading tools are yours: larger or smaller text, typeface, line spacing, focus mode, highlighting and bold, and notes saved to your account and gathered in My study.
A common error in sustainable transport planning is to define the problem as 'we need more buses'. Service is a possible means. The underlying problem may be that residents cannot reach suitable jobs at their starting times, or that a transfer journey is unreliable. Frame the problem with the affected people, the unmet outcome, place and time, evidence and a boundary that must not be crossed. For example: in the morning work-start period, residents of neighbourhood S can reach fewer jobs within 45 minutes than comparable districts despite two nominal routes, partly because of transfer waits, with a larger gap for people who need a step-free path. This is a testable statement, not a final causal finding. It needs opportunity, waiting and accessible-path measurements and a comparable sample. 'Congestion' alone is too broad: it may be a symptom whose appropriate treatment depends on journey purposes and who bears the harm.
Build a conflicting-evidence register instead of selecting the number that favours a preferred project. Traffic counters may report slower vehicles while a journey matrix shows improved bus access, and residents on the edge may report lost evening service. For each item record unit, area, period, method, sample size and possible instrument changes. All three observations can be true because they measure different things. Ask about alternative causes: roadworks, a new school term, a ticketing change or a relocated activity. Do not settle the issue with a citywide mean that hides peripheral areas, nor with one interview presented as representative. Assign a justified confidence description: stronger where definitions and periods match and a series exists, limited where a clear gap remains. Specify a short test for the gap before an expensive decision, such as repeated door-to-door travel-time measurements in both morning and evening. The point is to make disagreement investigable, not to make every item carry an invented precision score.
Separate eligibility gates from ranking. A gate covers constraints that cannot be compensated by a higher score elsewhere: a crossing that requires an engineering safety review, a binding budget ceiling, declared minimum access for a group that may lose service, or lawful use of the data. After ineligible alternatives are set aside, compare eligible ones on criteria declared before seeing their scores: reachable opportunities, complete journey time, emissions activity, lifecycle cost, deliverability and distribution of gains and losses. Do not present every criterion as precise if it is a range estimate. State whether higher or lower is better, the unit, estimate owner, source and update date. A weight is a governance choice to justify and test for sensitivity. Changing weights after the outcome merely to legitimate one option is not a sound appraisal. A lower-scored option may become preferable when new evidence reveals a hard constraint, but the changed decision rule must be recorded.
Write a short causal chain for each package: resource, implementation, service output, changed behaviour or access, then impact. A bus lane makes travel time more reliable only if service can operate and lane rules work. Ridership may rise if frequency and fare are usable. Car activity may decline if new riders come from cars rather than walking alone. Each arrow is a hypothesis with a distinct indicator. Metres of bus lane are an implementation output; boardings are service activity; reachable jobs are an outcome; vehicle emissions are an estimated impact. Confusing these levels lets a plan claim transition before it occurs. Include unintended effects such as delivery delay, traffic diversion to a residential street, loss of a usable footway or higher travel cost for one group. Do not dismiss professional objections as resistance to change: they may reveal data absent from the appraisal.
A useful diagnostic decision states what will be done now and what remains open. Make a one-page decision sheet with the bounded problem, supporting and conflicting evidence, confidence limits, affected groups, options to test, eligibility gates, three to five outcome indicators and a review date. Say 'we will test' rather than 'proved' where no valid before-and-after or comparison exists. Share hypotheses with stakeholders using language that distinguishes an experiment from a final commitment. Where data are sensitive, publish aggregated indicators and limitations rather than individual journey records. Predeclare a change rule before the pilot, such as no loss of access for the least-served district or improved reliability within the operating-cost ceiling. This stops a committee from treating every later result as success or failure according to the pressure of the moment. Diagnosis can legitimately end in a small test with a measurement plan rather than immediate selection of a major capital scheme.
Lesson summary
Conflicting evidence becomes a testable hypothesis; eligibility gates precede ranking, and the pilot has predefined stop and adjust indicators.