Growth economics and measurement architecture Review questions Growth Economics and Measurement Architecture — Integrated Summary A growth decision begins by defining the unit of value to increase: a completed positive-contribution order or a customer cohort that repays acquisition cost within a stated horizon. Visits and registrations may drive the result but are not value by themselves. Give each rate a numerator, denominator and time window, then test whether a rise accompanies worsening returns or service quality. Cohort economics show how low acquisition cost can hide weak retention. Accumulate contribution over time and report “not yet paid back” when observation ends before break-even. Do not extend historical performance to more spending or unobserved months without an explicit assumption. For an analytics dashboard to support this decision, define journey events, stable order keys and firing times, and reconcile purchases with the payment ledger; equal totals do not establish matching orders. When consent or tags change, separate financial transactions, observed analytics events and model estimates. Missing observations may differ by channel, so one automatic multiplier does not repair them. A sound decision memo states the calculated lead, a guardrail, the evidence that could reverse the ranking and a bounded test when measurement uncertainty exceeds the performance gap.
Review questions Test your understanding with these questions. Marking and saved results are available with a free account.
Question 1 Two cohorts each have 1,000 unique visitors. A records 100 orders from 80 buyers; B records 90 orders from 90 buyers. To compare the share of visitors who purchased, which numerator is appropriate? Order counts: 100 and 90 Buyer counts: 80 and 90 Completed session counts Collected order revenue Question 2 Campaign A produces 300 orders at contribution 20 each and acquisition cost 5,000. Campaign B produces 200 orders at contribution 35 each and cost 5,000. With equal horizons and no later orders, what does net contribution support? A leads because it has more orders B leads: 2,000 net versus 1,000 They tie because spend is equal A leads because cost per order is 16.67 versus 25 Question 3 Acquiring a cohort costs 6,000. Its contribution in months one to three is 2,000, then 2,500, then 1,000. What is its first observed payback month? Month one Month two Month three No observed payback yet Question 4 Channel A retention stays at 70% and B at 90% across two months. A grows from 20% to 80% of new customers and overall retention falls. Which interpretation best fits these data? Product quality deteriorated in both channels The changed mix can lower the overall rate CAC necessarily rose in both channels Use only blended retention to judge product quality Question 5 The payment ledger has 400 unique paid orders. Analytics has 420 purchase events: 20 repeat existing order IDs and 10 lack an ID. What follows before claiming complete reconciliation? Removing repeats leaves 400, proving a match Ten events lack keys, so order matching is unproved There are actually 420 paid orders All analytics events must be deleted Question 6 A product offers a free trial followed by first payment. It fires purchase when the trial-success page opens, even without payment. Which contract repair is most suitable? Record trial_start and confirmed subscription_payment separately Keep purchase because it raises conversion rate Send purchase from the server when the trial starts Rename purchase as conversion without separating the stages Question 7 Before a consent change analytics observes 200 purchases among 5,000 observed users; afterward 180 among 3,000. Payments across all eligible visitors are 260 then 270 out of 6,000 each period. Which conclusion is supported? All-visitor purchasing rose from 4% to 6% Observed rate rose while paid all-visitor rate moved slightly The campaign caused ten extra purchases Fewer observed purchases prove overall demand fell Question 8 Channel A has 100 observed orders and B 110. Payment records contain 30 orders unattributed to either channel. What decision handles uncertainty in their ranking? B certainly leads because 110 exceeds 100 A certainly leads because all unknown orders belong to it Investigate missingness or run a bounded test before a large transfer Multiply both counts by 1.15 to prove their ranking Question 9 Segment A wins 10 contracts contributing 300 each at campaign cost 1,000. B wins 8 contracts contributing 500 each at cost 1,500. On the same horizon with no other cost, which has higher net contribution? A at 2,000 net The segments tie B at 2,500 net A because it wins 10 contracts versus 8 Question 10 Segment B shows higher margin in a small test, but the product cannot activate it without an unavailable integration. What follows from an eligibility-gated segment decision? Defer B or validate integration before scaling it Increase margin weight until B ranks first Buy more traffic to prove intent Treat favorable interviews as activation evidence Question 11 Page A has 10,000 impressions, 200 clicks and 10 orders contributing 20 each. B has 5,000 impressions, 250 clicks and 10 orders contributing 50 each. Which reading is correct? A has higher CTR and contribution A contributes more because it has more impressions Their click counts become equal after impression adjustment B has higher CTR and contribution Question 12 After a site move, a service page loses search impressions but remains reachable through direct links. What should precede rewriting its offer? Change the price to gain clicks Double copy length without technical checks Check URL, redirects, discovery and indexing Treat one day of recent data as final proof Question 13 Campaign A has attributed revenue 20,000, variable cost 16,000 and ad spend 3,000. B has revenue 18,000, variable cost 12,000 and spend 3,000. Which has higher net contribution? B at 3,000 net A at 1,000 net A because revenue is higher A tie because ad spend is equal Question 14 Three ad-spend tranches of 1,000 each produce pre-ad contribution of 1,700, 1,200 and 700 in order. What is the marginal result of the third tranche alone? Gain 600 because cumulative result is positive Gain 700 because it produced contribution Break-even because tranche sizes are equal Loss 300 on this tranche Question 15 An ad report attributes 100 orders. In a balanced study with 1,000 people per arm, 130 exposed people buy and 110 held-back people buy. What is estimated incremental purchasing among exposed people? 100, the attributed count 20, the equal-arm difference 130, treatment purchases 240, total purchases across arms Question 16 In a test, 180 of 1,500 exposed people buy and 100 of 1,000 held-back people buy. Using the holdout rate as baseline, what lift is estimated at treatment size? 80 orders by subtracting raw counts 180 orders because all are in treatment 30 orders after applying the 10% control rate to 1,500 50 orders because the arms differ in size Question 17 A closed funnel records 2,000 checkout starts and 1,400 purchase events. The order system confirms 1,600 distinct paid orders for the same cohort, 200 of which lack an event. What is the first professional conclusion? Actual conversion is definitely 70%. Order-based conversion is 80%; investigate the event gap. Discard all funnel data. The page lost 200 customers. Question 18 After a mobile release, mobile purchase events fall, while confirmed mobile paid orders remain roughly stable. Which check best separates the likely cause? Compare ad prices across the two days. Increase the mobile budget. Check event dispatch and order identifiers in the updated mobile flow. Call the decline a causal interface effect. Question 19 In an enterprise service, ten colleagues share one contract decision. Which assignment unit best limits pricing-page spillover? The organization, with its version held through the window. Each click. Each employee independently. Each weekday. Question 20 Assignment logged 500 customers per arm, but the purchase-outcome table shows 500 controls and 350 treatments. What must precede interpretation of the page effect? Immediately calculate the rate over 850. Declare control the winner. Reassign the missing customers after seeing outcomes. Trace missing treatment records and inspect sample ratios. Question 21 An experiment estimates a +2 percentage-point purchase effect with an approximate interval from -0.7 to +4.7. Rollout needs at least +0.5 point to pay for itself. What is the best reading? Profit is certain because the point estimate is 2. Failure is certain because the lower end is negative. Value remains uncertain; collect evidence or use a guarded limited launch. A p-value alone decides. Question 22 A team checked p every day and stopped the test the first time it fell below 0.05, though the plan had no early-stop rule. What is wrong? Nothing; stopping strengthens the result. Unplanned repeated looks change the threshold’s interpretation. Only the button size is wrong. All earlier experiments must be deleted. Question 23 Version A has 40 payers ×120 contribution and five refunds ×40 additional cost. B has 30 payers and ten refunds at the same values. Which has higher net contribution? B because trial starts might be higher. A at 4,600 versus B at 3,200. B at 4,600. They tie before advertising cost. Question 24 Week A has 60 purchases per 1,000 visits and week B has 76. Purchase rates within both high-intent search and content segments stayed constant, but B had more high-intent search traffic. What is the safest interpretation? Page B caused a 1.6-point lift. Page A outperformed within every segment. The result cannot be analyzed. Audience mix may explain the total; test pages concurrently. Question 25 A cohort starts with 100 subscribers. Sixty complete a core job in month one and forty-five in month two. What is month-two cohort job retention? 45%. 75%. 60%. It cannot be calculated. Question 26 August’s cohort has not reached day 60, while July’s has. How should August day-60 retention appear? Zero, because none appears in the day-60 report. Copy its day-30 rate. Mark it immature until the horizon closes. Use July’s average without a label. Question 27 A journey sends a setup reminder after the customer completed setup that morning. What decision check should be fixed before sending? Send more messages so the right one arrives. Recheck success and suppression state at send time. Delete success logs. Measure email opens only. Question 28 In a message test, 54 of 110 renew in treatment and 45 of 110 in holdout. Renewal contribution is 50 and journey cost is 120. What is sample net effect? 450. 2,580. 330. Zero because holdout also renewed. Question 29 A cohort of 120 has a first purchase contributing 30 each, thirty-six second purchases contributing 40, and cohort service cost of 360. What is observed value per acquired customer before acquisition cost? 39. 40. 130. 4,680. Question 30 Projected customer value is 44–49 against acquisition cost of 40, but expansion may change audience quality. What is a disciplined decision? Double spending immediately because the interval midpoint exceeds 40. Stop all acquisition forever. Call 49 an observed value. Test a bounded tranche and inspect marginal customer value. Question 31 In a win-back test, 36 of 100 return with a discount versus 26 of 100 in holdout. Contribution is 55 per discounted returner and 70 otherwise; campaign cost is 120. What is net effect? 700. 40. 1,860. It cannot be estimated because controls returned. Question 32 A customer seeks cancellation because of a confirmed billing error. Which test design is acceptable? Withhold correction from holdout to estimate its effect. Send everyone a discount before fixing the issue. Correct the error for all, then test acceptable follow-up wording. Delete the cancellation request from the data. Question 33 Baseline: 10,000 visitors, 20% offer reach and 10% payment among viewers. Expansion: 12,000 visitors, 18% reach and 10% payment. How many extra payers? 40. 16. 216. None. Question 34 Plan: 200 payers from 10,000 visits at 2%. Actual: 8,000 visits at 2.25%=180. How do the drivers bridge the gap? Volume −40, rate +20. The page definitely lost twenty. There is no gap. Volume +20, rate −40. Question 35 Budget 6,000 in 2,000 tranches. Search nets: 1,400, 600, minus 100; email nets: 800, 100. Which feasible set is best? All search. Email one and two, search one. Search one, email one, search two. Search three, email two, search two. Question 36 The highest-net tranche set forecasts forty-five new customers but onboarding capacity is thirty-five. What should happen? Run it because net is positive. Increase ads to offset delays. Cancel every channel. Rerank under thirty-five or price more capacity. Question 37 Ads show 500 conversions, analytics 450 purchase events and billing 420 unique paid orders. Which source anchors contribution? 500 because attribution is broad. 450 because it is automated. 420, while reconciling gaps. The average of all three. Question 38 This month: 200 orders at contribution 100 before refund effect, forty refunds at extra cost 50. Previous month: 192 orders, twelve refunds at the same values. How did net change before ads? Up 800. Up 600. No change. Down 600. Question 39 In the capstone, 180 first contracts contribute 100 each, ninety renewals contribute 80 each, service costs 2,000 and acquisition 12,000. What is cohort net? 11,200. 18,000. 7,200. 25,200. Question 40 An extra ad spend of 2,000 forecasts twenty new contracts at first-contract contribution 100, with unknown renewal probability. What does the calculation support? Automatically add 1,600 renewal contribution. First contribution merely equals spend before other costs; seek lift and capacity evidence. Raise spend because events are below contracts. Nothing can be calculated. Create a free account to mark your answers and save your progress.