Growth economics and measurement architecture · Lesson 4

Consent and data quality

8 min read

Lesson objectives

  • Interpret observed and paid outcomes under consent changes and missing data.
Reading tools

A growth dashboard can change without any change in customer behavior. A new storage-consent choice, tag firing time or definition of purchase is enough. Data quality is therefore a decision problem, not merely a clean-table problem. When someone declines a form of measurement, the organization must respect that choice and applicable obligations. Google Consent Mode is a way to adjust the behavior of Google tags and SDKs according to user choices. Its documentation distinguishes a basic implementation that blocks tags until consent from an advanced implementation that adjusts tag behavior under configured defaults. That describes a Google product, not a general license to collect data or a legal conclusion across countries. The team should define purpose, permission basis and implementation with the appropriate specialists before relying on the resulting indicators.

Separate three layers in a report: transactions recorded in the commerce system, analytics events observed under a measurement configuration, and values estimated by a model to fill part of a gap. A finance transaction confirms a sale under the organization’s definition but does not alone identify which ad caused it. An analytics event helps describe the path, yet events can be missed or duplicated. A modeled estimate is not an observed row for a particular person; do not add it to observed events a second time. Google documentation says denying certain consent types limits collection and modeling may fill measurement gaps in its products. If a reported figure blends observed and modeled components, disclose their roles and comparability limits rather than treating it as a detailed customer ledger.

Create a quality card for each consequential metric: field completeness, value validity, key uniqueness, timing consistency and fitness of the definition for the decision. For purchase, transaction_id completeness is the share of purchase events carrying it. Validity means status and value reconcile with the payment record. Uniqueness means one intended order per key. Timing means delayed events are not compared with a financial period already closed. Fitness means new customer represents a first real purchase rather than a first visit on a new device. Set monitoring boundaries and a response for each dimension, but do not choose a threshold simply to pass an existing report. Base it on how an error could alter the decision and the cost of delaying action.

Do not respond to lower consent by circumventing the individual’s choice. Distinguish measurement that a particular permission allows from transactions an organization may use under its own lawful policy, seeking specialist advice in its jurisdiction. Analytically, examine whether willingness to consent differs by device, country or acquisition channel. If it does, the consenting sample may not represent the whole population; a simple global adjustment factor has untested assumptions. The team could show a scenario range for unobserved purchases or use a small compatible controlled test. State the assumption beside the number: “purchase rate among people consenting to this measurement,” not “purchase rate of all customers.”

Monitor the event chain after every site release: eligible visits, tag arrival rate, checkout starts, paid orders in commerce records and refunds. Look for an abrupt break on the day an interface or consent platform changes. If missing observations grow in one browser, test that browser’s permitted journey on a test device before declaring that market demand fell. Record the release time, investigator and fix. Stop using a channel-allocation indicator when its possible bias exceeds the performance difference on which the budget decision depends. This does not require stopping every dashboard: paid transaction monitoring may continue while an unreliable attribution conclusion is withheld.

The team can test the sensitivity of a channel decision without pretending to know what is missing. Suppose A has 100 observed orders, B has 110, and the commerce system has 30 unassigned paid orders. If their contribution difference is small, examine two logical bounds. Assigning all 30 to A produces 130 versus 110; assigning all to B produces 100 versus 140. This is an intentionally extreme range, not a prediction. It reveals that the ordering is not stable under missingness. Do not call the upper bound the likely answer. Investigate payment-flow characteristics that may narrow the unknown distribution without overriding user choices. When plausible allocations reverse the ranking, the report alone cannot support a hard-to-reverse spending decision. A bounded spending test or a delay until reconciliation improves may be appropriate.

Specify who owns the metric definition, who repairs the event flow and who authorizes use of the report for a decision. At a weekly meeting, an analyst can show paid-order coverage and the latest tag change; a commercial lead can show the margin gap between channels; the budget holder can judge whether possible measurement error is small enough relative to the performance gap to act. If the quality condition fails, record “decision deferred because coverage differs by channel” with a recheck date and a concrete repair step. This prevents an unknown value from becoming a silent zero in the dashboard. It does not automatically justify collecting more personal information. The repair may be a clearer event definition or reconciliation of order records already permitted for use.

Completeness differs from fitness for use. All purchase events might have valid identifiers while bank-transfer payments reach finance two days later and never enter the event stream. The key check passes, but the channel-funding question remains biased. Test coverage across payment methods and segments, not just empty fields. If an analytics provider supplies a modeled value for missing activity, examine what it estimates and at what granularity before splitting it into tiny groups. The same dataset may support a broad trend yet fail a narrow channel decision.

Lesson summary

Separate consent-driven observation changes from demand changes.