Growth economics and measurement architecture

Review questions

Growth Economics and Measurement Architecture — Integrated Summary

A growth decision begins by defining the unit of value to increase: a completed positive-contribution order or a customer cohort that repays acquisition cost within a stated horizon. Visits and registrations may drive the result but are not value by themselves. Give each rate a numerator, denominator and time window, then test whether a rise accompanies worsening returns or service quality. Cohort economics show how low acquisition cost can hide weak retention. Accumulate contribution over time and report “not yet paid back” when observation ends before break-even. Do not extend historical performance to more spending or unobserved months without an explicit assumption. For an analytics dashboard to support this decision, define journey events, stable order keys and firing times, and reconcile purchases with the payment ledger; equal totals do not establish matching orders. When consent or tags change, separate financial transactions, observed analytics events and model estimates. Missing observations may differ by channel, so one automatic multiplier does not repair them. A sound decision memo states the calculated lead, a guardrail, the evidence that could reverse the ranking and a bounded test when measurement uncertainty exceeds the performance gap.

Review questions

Test your understanding with these questions. Marking and saved results are available with a free account.

  1. Question 1Two cohorts each have 1,000 unique visitors. A records 100 orders from 80 buyers; B records 90 orders from 90 buyers. To compare the share of visitors who purchased, which numerator is appropriate?
  2. Question 2Campaign A produces 300 orders at contribution 20 each and acquisition cost 5,000. Campaign B produces 200 orders at contribution 35 each and cost 5,000. With equal horizons and no later orders, what does net contribution support?
  3. Question 3Acquiring a cohort costs 6,000. Its contribution in months one to three is 2,000, then 2,500, then 1,000. What is its first observed payback month?
  4. Question 4Channel A retention stays at 70% and B at 90% across two months. A grows from 20% to 80% of new customers and overall retention falls. Which interpretation best fits these data?
  5. Question 5The payment ledger has 400 unique paid orders. Analytics has 420 purchase events: 20 repeat existing order IDs and 10 lack an ID. What follows before claiming complete reconciliation?
  6. Question 6A product offers a free trial followed by first payment. It fires purchase when the trial-success page opens, even without payment. Which contract repair is most suitable?
  7. Question 7Before a consent change analytics observes 200 purchases among 5,000 observed users; afterward 180 among 3,000. Payments across all eligible visitors are 260 then 270 out of 6,000 each period. Which conclusion is supported?
  8. Question 8Channel A has 100 observed orders and B 110. Payment records contain 30 orders unattributed to either channel. What decision handles uncertainty in their ranking?
  9. Question 9Segment A wins 10 contracts contributing 300 each at campaign cost 1,000. B wins 8 contracts contributing 500 each at cost 1,500. On the same horizon with no other cost, which has higher net contribution?
  10. Question 10Segment B shows higher margin in a small test, but the product cannot activate it without an unavailable integration. What follows from an eligibility-gated segment decision?
  11. Question 11Page A has 10,000 impressions, 200 clicks and 10 orders contributing 20 each. B has 5,000 impressions, 250 clicks and 10 orders contributing 50 each. Which reading is correct?
  12. Question 12After a site move, a service page loses search impressions but remains reachable through direct links. What should precede rewriting its offer?
  13. Question 13Campaign A has attributed revenue 20,000, variable cost 16,000 and ad spend 3,000. B has revenue 18,000, variable cost 12,000 and spend 3,000. Which has higher net contribution?
  14. Question 14Three ad-spend tranches of 1,000 each produce pre-ad contribution of 1,700, 1,200 and 700 in order. What is the marginal result of the third tranche alone?
  15. Question 15An ad report attributes 100 orders. In a balanced study with 1,000 people per arm, 130 exposed people buy and 110 held-back people buy. What is estimated incremental purchasing among exposed people?
  16. Question 16In a test, 180 of 1,500 exposed people buy and 100 of 1,000 held-back people buy. Using the holdout rate as baseline, what lift is estimated at treatment size?
  17. Question 17A closed funnel records 2,000 checkout starts and 1,400 purchase events. The order system confirms 1,600 distinct paid orders for the same cohort, 200 of which lack an event. What is the first professional conclusion?
  18. Question 18After a mobile release, mobile purchase events fall, while confirmed mobile paid orders remain roughly stable. Which check best separates the likely cause?
  19. Question 19In an enterprise service, ten colleagues share one contract decision. Which assignment unit best limits pricing-page spillover?
  20. Question 20Assignment logged 500 customers per arm, but the purchase-outcome table shows 500 controls and 350 treatments. What must precede interpretation of the page effect?
  21. Question 21An experiment estimates a +2 percentage-point purchase effect with an approximate interval from -0.7 to +4.7. Rollout needs at least +0.5 point to pay for itself. What is the best reading?
  22. Question 22A team checked p every day and stopped the test the first time it fell below 0.05, though the plan had no early-stop rule. What is wrong?
  23. Question 23Version A has 40 payers ×120 contribution and five refunds ×40 additional cost. B has 30 payers and ten refunds at the same values. Which has higher net contribution?
  24. Question 24Week A has 60 purchases per 1,000 visits and week B has 76. Purchase rates within both high-intent search and content segments stayed constant, but B had more high-intent search traffic. What is the safest interpretation?
  25. Question 25A cohort starts with 100 subscribers. Sixty complete a core job in month one and forty-five in month two. What is month-two cohort job retention?
  26. Question 26August’s cohort has not reached day 60, while July’s has. How should August day-60 retention appear?
  27. Question 27A journey sends a setup reminder after the customer completed setup that morning. What decision check should be fixed before sending?
  28. Question 28In a message test, 54 of 110 renew in treatment and 45 of 110 in holdout. Renewal contribution is 50 and journey cost is 120. What is sample net effect?
  29. Question 29A cohort of 120 has a first purchase contributing 30 each, thirty-six second purchases contributing 40, and cohort service cost of 360. What is observed value per acquired customer before acquisition cost?
  30. Question 30Projected customer value is 44–49 against acquisition cost of 40, but expansion may change audience quality. What is a disciplined decision?
  31. Question 31In a win-back test, 36 of 100 return with a discount versus 26 of 100 in holdout. Contribution is 55 per discounted returner and 70 otherwise; campaign cost is 120. What is net effect?
  32. Question 32A customer seeks cancellation because of a confirmed billing error. Which test design is acceptable?
  33. Question 33Baseline: 10,000 visitors, 20% offer reach and 10% payment among viewers. Expansion: 12,000 visitors, 18% reach and 10% payment. How many extra payers?
  34. Question 34Plan: 200 payers from 10,000 visits at 2%. Actual: 8,000 visits at 2.25%=180. How do the drivers bridge the gap?
  35. Question 35Budget 6,000 in 2,000 tranches. Search nets: 1,400, 600, minus 100; email nets: 800, 100. Which feasible set is best?
  36. Question 36The highest-net tranche set forecasts forty-five new customers but onboarding capacity is thirty-five. What should happen?
  37. Question 37Ads show 500 conversions, analytics 450 purchase events and billing 420 unique paid orders. Which source anchors contribution?
  38. Question 38This month: 200 orders at contribution 100 before refund effect, forty refunds at extra cost 50. Previous month: 192 orders, twelve refunds at the same values. How did net change before ads?
  39. Question 39In the capstone, 180 first contracts contribute 100 each, ninety renewals contribute 80 each, service costs 2,000 and acquisition 12,000. What is cohort net?
  40. Question 40An extra ad spend of 2,000 forecasts twenty new contracts at first-contract contribution 100, with unknown renewal probability. What does the calculation support?