Frame the workforce decision and its evidence · Lesson 6

Segmentation and aggregate-rate reversals

12 min read

Lesson objectives

  • Reweight unit-level outcomes and diagnose a Simpson-type reversal before comparing workforce performance.
Reading tools

A service dashboard reports a lower two-day resolution rate after training. The team manager recalls improvement on both kinds of request covered by the training. Must one of them be wrong? Not necessarily. An aggregate rate is a weighted average of subgroup rates. It can move in the opposite direction when subgroup weights change. This is often called a Simpson-type reversal. It does not make every aggregate useless. It means the decision must distinguish the overall customer experience from staff performance on comparable requests. A product launch may increase the share of difficult cases; then separating performance from case mix is essential before rewarding or blaming a team. People analytics uses segments to explain business outcomes, but selecting segments only after seeing results can create a convenient story. Define task-relevant segments before drawing a conclusion.

To isolate the mix effect descriptively, hold quarter-one weights fixed: two-thirds easy and one-third difficult. The standardized quarter-two rate is (2/3 × 90%) + (1/3 × 45.7%), about 75.2%. That is about 8.6 points above the original 66.7% on the original case mix, while the actual customer experience in quarter two remained 51.25%. A professional report presents both figures instead of selecting the comforting one: conditional performance improved, but overall service deteriorated because the cases became much harder. The decision might call for specialized capacity or better handling of complex work, neither canceling training nor merely congratulating the team. If the definition of 'difficult' changed between quarters, even the standardized comparison is suspect. Inspect the classification rule and a sample of requests before relying on it.

Segmentation is not permission to split without limit. More segments leave fewer observations per cell, making rates volatile and potentially identifying individuals. Choose a pre-outcome variable with operational meaning, such as request complexity or service channel. Do not segment on a variable that was produced by the outcome and then claim a causal explanation. If a ticket is labeled 'difficult' only after it has been delayed, controlling for difficulty can remove part of the delay under investigation. In comparisons across teams, ask whether the teams perform the same work. An escalation team should not be judged on the denominator of a first-contact team. Show numerators and denominators alongside percentages so readers can see that a shift from one of two to two of two is not robust evidence of a fifty-point improvement.

Choose comparison weights before choosing a winner

A service manager reports falling monthly completion and requests training for everyone. The team lead explains that complex cases doubled while processing improved within every category. Two questions now matter: what service did customers actually experience, and how did like-for-like processing change? The aggregate answers the first; category rates and fixed weights help answer the second. Retain both in the report. Customers need actual service performance, while management needs diagnosis to choose a remedy. Disagreement between measures is not a reason to delete one. Label their questions and do not use a diagnostic standardized rate to hide a real service decline.

Start fixed-mix comparison with a classification applicable in both periods. Suppose cases are simple or complex. Define complexity from features recorded before processing, so delayed cases cannot be reassigned to the hard category after outcomes are known. Relevant features might include multiple systems or required specialist review. Test a common sample with supervisors from both periods. Resolve disagreements about the same cases before calculation, then keep the classification version and approval date. Major changes to category meaning make a rate difference harder to interpret as genuine performance change.

A different purpose may justify different reference weights. Next-month planning could use forecast demand mix instead of last month's. Comparing branches might use a common customer population they jointly serve. Record the reason before displaying branch names. Reference weights can change gap size, and sometimes ranking when branches lead in different categories. Include original rates and counts so a reviewer can recalculate under another mix. An agreed reference prevents changing weights repeatedly until a preferred branch comes first.

Inspect small cells before relying on a standardized result. A branch completing both of only two complex cases has a one-hundred-percent rate but little evidence for performance under larger demand. Common weights may give that small cell considerable influence. Show its count and obtain a longer observation period or more cases when needed. If a category is absent entirely, its rate is unknown for that branch. Combining categories may be defensible with a coherent operational definition, but inventing a missing rate adds no evidence. State which part is comparable and which requires further investigation.

This reading connects resource planning to work type. More complex demand might justify specialist cover or different referral, even while current staff become more proficient. Link each category to effort, required skill, and service obligation. That gives a sharper capacity question than training everyone. Review may also show that a simple-looking case waits on an external system rather than needing more staff hours. Fixing the wait then becomes a hiring alternative. Standardized rates are diagnostic starting points; delay mechanisms and option costs need their own investigation. Move from weights to a specific operational choice rather than debating team rank alone.

Consider a ranking that changes with purpose. Branch one leads on simple cases and branch two on complex cases. A new programme expected to generate mostly complex work may need branch two's expertise. General service leadership may use a different standard reflecting current organization-wide demand. Naming the purpose prevents applying one league table to every decision. Combining the branches' strengths may be more useful than declaring an absolute winner. Ask leaders to document criteria before moving people and preserve the expected-mix rationale so the decision can be revisited if actual demand differs.

Check that categories cover all demand without overlap. A case can be classified both by channel and complexity, but those are different dimensions. A cross-tabulation is useful; adding phone and email rates to simple and complex rates would count cases twice. Define each table's row unit and verify that mutually exclusive category counts sum to distinct requests. Show an unclassified category rather than guessing its allocation. Unknown cases may affect reference weights, so inspect their size before publishing comparisons. Also state whether classification describes the case on arrival or after processing, since that choice can change the performance story.

Tomorrow, take an aggregate indicator that moved in the latest report. Split it into two operationally meaningful categories and calculate their rates and weights in both periods. Show actual performance first, then a standardized comparison with an explicitly justified reference. If decline disappears within categories, focus discussion on mix and the capacity it requires. If it persists, inspect methods or effort within category. These steps turn a chart surprise into a testable operating question while preserving visibility of the service customers actually received.

Lesson summary

A service dashboard reports a lower two-day resolution rate after training.