Baselines, denominators and comparisons
4 min read
Lesson objectives
- Compute an urban-service indicator from a baseline, denominator and defined segment, explaining when two districts cannot be compared.
Please sign in to use all the tools
Once you sign in, the reading tools are yours: larger or smaller text, typeface, line spacing, focus mode, highlighting and bold, and notes saved to your account and gathered in My study.
One indicator answers one question
If a city says “late reports fell,” ask whether their count, rate or delay duration fell. These are different questions. An indicator is an observable or computable definition of one aspect of an outcome. A baseline is its value before an intervention, measured using a stated method. A useful indicator specifies the numerator, denominator, unit, period, area and exclusions. U4SSC's selection principles include comparability, data availability and simplicity of calculation. Merely giving two figures the same label does not make them comparable. If the neighbourhood boundary or reporting channel changes, explain it before interpreting the difference.
Consider waste collection. “Number of complaints” can rise because reporting became easier, even when collections did not worsen. “Share of observed bins above the fullness threshold at scheduled pickup” is closer to the physical service, but it requires a consistent field sample. No single indicator captures the whole service. Match the measure to the decision. To improve report handling, examine verified closure time and the share closed on time. To reduce overflow, observe bin condition on the ground. Preserve each data source's definition: app records describe app users, not every person who encountered a problem.
Worked case: which district performs better?
Segments show what an average hides
A citywide completion rate might be 80%, but it could be 95% in well-connected areas and 50% in poorly connected areas. The average hides an access gap. Segment results by a feature relevant to the decision, such as district, service channel or trip type. Avoid tiny groups that risk revealing identities or produce unstable rates. Do not infer the cause of a gap from the rate alone. Test whether connectivity, office hours or the definition of an application differs. Calculate each group's rate using that group's denominator; do not divide the citywide 80% by the number of districts. When combining groups of unequal size, use the original counts: total numerators divided by total denominators. A simple average of group percentages is valid only for a different question or equal denominators.
For example, Area A has 90 completions among 100 requests and Area B has 10 among 20. Their rates are 90% and 50%. Their simple average is 70%, but the combined request-level rate is (90 + 10) ÷ (100 + 20) = 100 ÷ 120 ≈ 83.3%. The gap is not a rounding mistake: a simple average gives each area equal weight, while the combined rate gives each request equal weight. Choose the unit that answers the question before aggregating. If you want the experience of a typical request, use the combined rate. If you want to compare two districts as separate units, show both rates. Do not conceal the group sizes behind a citywide percentage.
Practice and answer
Lesson summary
If a city says “late reports fell,” ask whether their count, rate or delay duration fell. These are different questions. An indicator is an observable or computable definition of one aspect of an outcome. A baseline is its value before an intervention, measured using a stated method. A useful indicator specifies the numerator, denominator, unit, peri