Audit data quality and missingness

Water and food records may appear compatible because they carry the same date, while their units and collection processes differ. A utility log measures volume delivered at a particular point; a market log may count incoming shipments rather than food reaching households. Before linking them, write a small data dictionary for each field: variable, unit, measurement point, period, meaning of zero, missing-value code, entry source and correction owner. If zero means 'no shipment arrived' and a blank means 'the vendor did not submit a record,' filling blanks with zero invents a food shortfall. Conversely, dropping every incomplete row without examination may remove the traders most disrupted by the event.

Inventory coverage before computing a rate. How many records were expected, how many arrived complete, how many had a verified zero and how many remain unknown? Record the answer by subgroup, source and time. Missingness associated with lost connectivity or business interruption makes the received records potentially unrepresentative; do not assume their mean applies to the absent group. Align the market and water observation windows. Utility water metered from midnight to midnight cannot be compared directly with boxes invoiced when a market closes at 18:00 if the outage began that evening. Either regroup both logs into a shared window or report the timing difference as an inference limit. Retain timestamps so the alignment can be audited.

Check duplication as well. The same box may appear in a washing-site dispatch log and again in a market receipt log; summing both would double-count food. Use batch identifiers or transfer time, with personal data protected, to identify where the flow is counted once. If identifiers are unavailable, keep 'left washing' and 'entered market' as distinct rows rather than adding them. Check water units too: one cubic meter is one thousand liters, but unit conversion does not resolve different meter locations or network loss. Market access cannot be inferred from utility volume alone. Preserve each transformation and assumption in a reproducible record and reconcile raw totals before calculating rates.

Choose a response to missingness based on the decision, not the wish for a neat table. For an early warning, 'four of ten vendor records missing' plus urgent phone checks may be enough. Comparing two months may require similar collection coverage in each month or sensitivity analysis; copying last month's average into blank fields is not a neutral repair. Any statistical imputation needs a justification and a label as an estimate, and may be unsuitable for an urgent allocation decision. Keep raw and corrected datasets separate, with who authorized each correction and why, so the issue remains visible to later reviewers.

Set a clear acceptance rule for each source. A point-of-sale report should carry the case date, box count, product definition and a verifiable submission trace. A report without a date does not automatically belong to the outage day. A warehouse entry of '24' may count parcels rather than boxes: adding it would be a unit error before even considering duplication. Document any verified parcel-to-box conversion and its reviewer. If a value is unusual, do not delete it merely because it disturbs the average. Check with the record owner whether it was a bulk delivery, duplicate entry or real change. Keep the raw version available for audit.

Connect data quality to the action priority. If missing reports concentrate in one neighborhood, ranking neighborhoods by mean sales may rank reporting intensity rather than food availability. Calculate coverage for each neighborhood first and judge whether comparison is defensible. If a committee must act today, it may publish a temporary allocation based on confirmed stock and observed risks, assign someone to recover missing records within hours and schedule a review. 'We do not know the precise market total' is different from 'we know nothing.' Confirmed count, coverage and reason for missingness are useful evidence if their limits stay visible. A decision log should state which conclusion would change when the absent reports arrive.

Use bounds for missing status rather than a single falsely precise number. Eighty verified receipts out of one hundred eligible orders, ten unknown and ten failed imply an eighty-percent confirmed lower bound and ninety-percent possible upper bound. If unknowns cluster in a remote area, do not spread them evenly for convenience. Check whether missingness comes from a phone-entry fault or from drivers who skip logging failed deliveries; the latter biases a complete-case rate upward. Preserve raw and corrected records with reasons, distinguish a true zero from absence, and never call a scenario range a statistical margin of error without a sampling design.