Opportunity and AI fit Review questions Opportunity and AI fit — Integrated summary Begin with a user job, baseline, and non-AI alternative. Value error costs and adoption, then allocate discovery funds using falsifiable evidence. Frame a user job, baseline and non-ai alternative before proposing a model. Value the opportunity with error costs, adoption and counterfactual baseline. Design a value hypothesis and proof threshold. Allocate discovery investment across an ai opportunity portfolio.
Review questions Test your understanding with these questions. Marking and saved results are available with a free account.
Question 1 What defensible product step comes before buying a routing classifier? Value all 270 other requests as certain savings Train on whichever team closed each request Map the journey and measure each step's wait and transfer cause Raise a routing-accuracy target without defining success Question 2 What is the conditional weekly operating value? 70 units 160 units 250 units 40 units Question 3 Which judgment separates adoption from product value? Release widely because opening is high Stop expansion and diagnose corrections and missing resolution gain Redefine the correction ceiling at 12% Exclude requests where the draft was not opened Question 4 Which plan respects money, capacity and dependency? Run all three because shared data value offsets overspend Run B and C concurrently because they cost exactly 45 Fund B and C and omit shared-data cost Test A, then choose B or C after source and capacity evidence Question 5 What reliable count of unique labeled examples is available now? 8,000 by direct subtraction Unknown until identity reconciliation, deduplication, then label counting 8,500 because duplication cannot affect labels 10,000 because every row is technically available Question 6 What first capability test best fits this job? Send vendor answers to everyone because fluency is high Train a custom model solely on old policies Test dated retrieval with passage and abstention before adding generation Hide sources to reduce agent distraction Question 7 What are recall for required cases and total review requests? 80% and 56 requests 57.1% and 40 requests 80% and 24 requests 88% and 200 requests Question 8 Which control should reject this case while accepting a matching current version? Count link length and reject short URLs Reject every answer containing a link Check bilingual text presence Compare request time with indexed document identity and effective date Question 9 Which design assigns decision authority precisely? Let the model approve all high-confidence cases Let the model suggest; staff decide exceptions under a documented cap and authority Let staff approve the first ten cases, then transfer authority to the model Let the vendor decide because it hosts the model Question 10 What interface correction is most appropriate? Enlarge the same statement Hide the rule source to avoid confusing users Show a confidence number without the contract date State that the result is provisional and request the contract date before deciding Question 11 What follows before declaring the incident over? Identify affected recipients, correct replies, and record impact Restart because shutdown succeeded Delete logs to save storage Wait for the first complaint before scoping Question 12 What is the most useful initial diagnosis? Remove the slow-connection slice from reporting Increase model size without examining latency Separate connection and timeout failure from prediction error by slice Accept the overall result because it exceeds target Question 13 What is the average variable cost per case? 1.2 currency units 2.4 currency units 7.2 currency units 0.24 currency units Question 14 What is the disciplined conclusion? Product value is proven because engagement rose The model necessarily failed on accuracy A proxy improved; the decision needs customer-outcome and harm-guardrail evidence The control group should be removed Question 15 What is the release-gate verdict? Do not pass until the language limit and correction owner are fixed or an explicit exception is documented Pass because the average succeeds Pass because correction can be assigned after launch Delete the second language from scope Question 16 What risk should be priced and constrained in the contract? Higher call speed Less need for quality monitoring Guaranteeing equal quality across vendors Lock-in cost and loss of auditability and migration Question 17 What should the launch lead do? Expand to 20% for a larger sample Pause expansion, return to the safe path, and investigate Remove the first hour from reporting Wait for month-end reporting Question 18 Which hypothesis should be tested first? The model became one day older All change is necessarily random The new input form changed feature distribution or completeness Reviewers' judgment degraded without evidence Question 19 What is the first disciplined decision? Contain under delegated authority, record time and scope, and alert incident command Wait for the technical owner until tomorrow Erase the recommendations before preserving them Announce that every customer was harmed Question 20 What is a sound interim decision? Automatically increase funding because it operates Fully fund the alternative because it is new Hide cost and show benefit alone Reassess causes and a stop option; fund a limited alternative test with clear criteria Create a free account to mark your answers and save your progress.
