Metric contract and evidence brief
12 min read
Lesson objectives
- Specify metric ownership, refresh, threshold, and evidence limitations in a decision-ready brief.
Please sign in to use all the tools
Once you sign in, the reading tools are yours: larger or smaller text, typeface, line spacing, focus mode, highlighting and bold, and notes saved to your account and gathered in My study.
Once a team understands denominators, cohorts, and segments, a different challenge arises: how do definitions survive a change of manager or reporting tool? Use a compact metric contract linking each number to a decision, owner, calculation, refresh interval, and limits of use. This is not a ceremonial sign-off. It lets the analyst, operations, HR, and finance discuss the same phenomenon. Operations may say 'vacancy' and mean an uncovered shift; HR may mean an authorized seat without an occupant. Both numbers can be correct while their shared label creates a false dispute. Name the first a shift-coverage gap and the second a position vacancy rate, then show how one may influence the other. CIPD's people analytics practitioner guide begins with focus and investigation and connects analysis to decisions. The contract makes that progression repeatable in daily work.
A card for 'resolved within two days' states its purpose: decide whether additional service capacity is needed. Its numerator is complex enterprise requests closed with an acceptable resolution within 48 hours of opening. Its denominator is all complex enterprise requests that have had at least 48 hours of follow-up during the period; very recent requests sit in a pending count until eligible. The reporting period is a defined operating week and time zone. The source is the ticket system linked to reopening records. Requests waiting for customer input appear in a separate segment instead of silently disappearing. The operations director owns the service definition, the data steward owns extraction controls, and HR interprets capacity without owning every ticket. An illustrative action threshold might start a cause review after two weeks below 80% with at least 200 eligible requests each week. That threshold is not a universal fact; owners must test it against delay cost and historical volatility before adopting it.
An evidence brief condenses the inquiry to a decision on one page: question and alternatives, observations, assumptions, unknowns, implications for each choice, recommended move, and review date. Return to the first lesson's case: 760 of 1,000 requests met the two-day target and demand may reach 1,300. Do not write, 'We need 384 analysts.' The number 384 is additional timely resolutions needed for an 88% goal, not people. Write instead: 'Conditional on 1,300 requests, the gap against baseline throughput is 384 timely resolutions. We will test the capacity of triage, training, and hiring after estimating effort by request type. If volume reaches 1,600 or quality falls, we will revisit the options.' This separates arithmetic, assumption, and later decision.
Add a short decision table in prose. Shift redistribution is quick and reversible but may weaken another coverage window. Training requires time and a test that skills transfer to actual requests. Hiring might add capacity later and creates continuing cost. Triage might reduce waiting without increasing headcount but requires error monitoring. Do not convert these descriptions into invented numeric scores. Ask each owner for estimates, ranges, and acceptance conditions. If a service manager's account conflicts with reopening data, do not hide the conflict; jointly inspect a sample under a shared definition of acceptable resolution. A brief is useful when a decision maker can see exactly which assumption would change the recommendation, rather than merely seeing more charts.
When the metric definition changes during operations
You enter a hiring meeting with a time-to-fill chart. A colleague presents another on the same subject with the opposite trend. Before checking arithmetic, compare definition cards. One clock may start at position approval and another at advertisement; one may end at starting work and the other at offer acceptance. Each duration describes a different part of the journey. Explain which stage the manager can influence now and tie the metric to it. This makes disagreement diagnostically useful. If the decision concerns the full journey, show its stages with dated events rather than average two unlike definitions.
A metric card includes the business question, record unit, clock endpoints, and inclusion and exclusion rules. Add each date's source and correction owner. Specify treatment of holidays, cancelled vacancies, and declined offers. Calendar days and working days may both be appropriate for different purposes, provided the full unit appears in the result label. Someone rerunning the calculation should obtain the same record count and denominator. Include an ordinary and an exceptional case as a practical definition test. If readers apply the card differently, the definition needs further precision.
Treat exclusions as choices that change the question. Excluding cancelled roles describes time among completed journeys. Assessing recruiter workload also requires the effort spent on cancelled requisitions. Report their counts and cancellation reasons separately rather than force incomplete durations into an unsuitable mean. For sourcing-channel comparison, ensure channel attribution does not change mid-record without dates. Assigning the final outcome to the last visible channel might credit it for work started elsewhere. A sound metric contract addresses attribution rules as carefully as division or date arithmetic.
Assign responsibility for definition changes. The analyst proposes a change and its likely impact; the steward confirms field availability; the metric owner approves meaning. The organization then communicates with users. Even a small start-clock adjustment may affect a dashboard, operating target, and budget report together. List dependent outputs before release and trace one sample through them to check consistent meaning. This prevents a corrected source feeding a dashboard still using the old definition. The contract can stay concise when focused on what a decision maker must know to interpret the number.
Begin an evidence memo with the reader's decision. State the result, confidence in its denominator, and the trade-off it changes. Put detailed definitions in a supporting note instead of loading the first sentence. For example, post-acceptance waiting may rise while selection time remains similar. A suitable next step is checking onboarding processes rather than blaming recruitment for the whole journey. If the interpretation is a hypothesis, label it and name its test. This connects a number to a reasonable action while preserving traceability to source calculations.
Resist reducing the card to a familiar name. Everyone may say time to hire while using different start events. Ask two users to apply the definition to the same record without coaching. If they choose different dates, record the disagreement, add a clarifying example, and repeat the check. This tests user understanding rather than analyst ability. It may also reveal missing source fields. A definition requiring an unrecorded date needs improved collection or an explicit statement that measurement is not yet complete.
At week's end, choose three metrics serving one decision and classify each as outcome, capacity input, or quality constraint. Resolution speed might be the outcome, specialist hours the input, and reopening the guardrail. Keep a separate contract for each with a common version date for the memo combining them. When one unit changes, check the arithmetic relationship before extending the chart. This makes the report transferable to another analyst while keeping its decision intelligible after the original author leaves.
Lesson summary
Once a team understands denominators, cohorts, and segments, a different challenge arises: how do definitions survive a change of manager or reporting tool? Use a compact metric contract linking each number to a decision, owner, calculation, refresh interval, and limits of use.