Framing Business Questions, Data and Metrics

Review questions

Integrated Summary: From a Business Question to Trustworthy Data and Metrics

Business data analysis starts before any tool is opened. The sound starting point is the business need and the decision that could change because of the result, then an analytical question that available data can answer. “Performance is poor” is a broad symptom. “How did order completion proportion and average order value change from Q1 to Q2 by region and customer type?” defines measures, a period and segments. If the task is to estimate a campaign effect but there is no exposure variable, do not invent one or call a descriptive change an “effect.” Answer what the data support and record the measurement gap.

Once the question is framed, understand what the data represent. A data source tells where values came from, the unit of observation tells what a row represents, and grain states the finest stored level. This matters when joining tables. In a synthetic example, three orders have values 100, 60 and 140, so the correct total is 300. After a one-to-many join from orders to order lines, order values can repeat on each line and produce the incorrect total 680. The formula did not fail; row grain changed. Therefore define keys and relationships, use a data dictionary that explains meaning, type, unit, allowed values and derivation rules, and keep simple lineage from source through transformations.

A metric is a definition before it is a number. A good metric card states numerator, denominator, period, segment, inclusion/exclusion rules and source when relevant. 930 on-time completions out of 1,200 eligible orders gives 77.5%. If the previous period was 74.0%, the direct change is 3.5 percentage points, while relative change is about 4.73%. Confusing the two changes the message. A metric becomes a KPI when it is tied to an objective, decision, monitoring and responsibility; a fictional teaching target is not a universal benchmark. Denominator stability matters as much as the metric label.

Next, assess data quality in relation to intended use. The six dimensions used in this subject are completeness, uniqueness, consistency, timeliness, validity and accuracy. Keep them distinct. In a synthetic dataset, 97 unique records remain after three confirmed reload duplicates are removed, and channel is missing in 8 records. Completeness is therefore 89/97, about 91.8%. Negative quantity values are a separate validity issue, not a completeness issue. Missing does not mean zero, an apparent duplicate should not be deleted before understanding grain, and an outlier is not an error merely because it is far from other values.

Good cleaning is traceable: preserve a path back to source and record the rule, affected count, action, rationale and analytical effect. Quality dimensions can also trade off. A rapid report may be more timely but less fully verified. The right choice depends on the decision purpose and cost of error, and provisional status and completeness should be disclosed. The four lessons therefore form one chain: a bounded question, data understood at the correct grain, a precisely defined metric, and documented quality processing. If any link fails, the calculations can look polished while answering the wrong business decision.

Review questions

Test your understanding with these questions. Marking and saved results are available with a free account.

Question 1A manager says, “Performance is poor and we want to know why.” Which is the best initial analytical question when the order log contains only period, region, customer type, order value and completion status? What is the true cause of the performance decline? How did completion proportion and average order value change from Q1 to Q2 by region and customer type? Which employee is responsible for the decline? Did the marketing campaign cause the drop in orders?
Question 2Which element is a business decision rather than an analytical question or operation? Choosing which region to prioritize for service improvement. Calculating completion proportion in each region. Comparing average order value across two periods. Segmenting results by customer type.
Question 3You are asked to measure a campaign’s effect, but there is no field showing who was exposed to the campaign. What is the most disciplined action? Assume every new customer was exposed. Use region as a substitute for exposure and call the result the campaign effect. Answer the descriptive changes that are possible and record missing exposure data as a measurement gap. Drop the question without documenting it.
Question 4Which two essential boundaries are still missing from the question “Did performance improve?” Software name and chart color. Manager name and staff count. Time period and the measure that defines improvement. Choice of forecasting algorithm.
Question 5There is one row for each customer in each month. What is the unit of observation? Customer only. Month only. Customer-month. Order.
Question 6Three orders have values of 100, 60 and 140. What is the correct total order value before any join to order lines? 300 360 440 680
Question 7Which data-dictionary definition makes customer_type most reproducible? Customer type as shown in the report. New or old customer according to team experience. A category used in analysis when needed. new if no completed order exists before the order date, returning otherwise, derived from the order log.
Question 8A daily operational report is based on a data extract that is three days old. Which property should be recorded first? Dashboard color. Extract timestamp/source freshness because the decision may require newer data. Name of the person who opened the file. Number of charts in the report.
Question 9930 of 1,200 eligible orders were completed on time. What is the proportion? 74.0% 77.5% 93.0% 129.0%
Question 10A metric rises from 74.0% to 77.5%. What is the direct difference? 3.5 percentage points 3.5% of orders 4.73 percentage points 77.5 percentage points
Question 11Which definition makes “on-time completion proportion” most reproducible? good orders ÷ all orders. fast orders ÷ important orders. eligible orders completed on/before due time ÷ eligible orders that became due in the period, under fixed inclusion rules. completed orders ÷ customers.
Question 12What makes a metric suitable to act as a KPI in this lesson? It is displayed in a large font on a dashboard. It is always a percentage. It is tied to an objective, decision, monitoring and clear responsibility, with a target/threshold when useful. It comes from paid software.
Question 13A required field is present in 95% of records, but some present values are wrong. What is the most precise conclusion? Overall data quality must be 95%. Completeness is about 95%, while accuracy requires a separate check. Accuracy is 95% because values are present. Validity, completeness and accuracy are the same thing.
Question 14You find a repeated order_id. What is the first step before deletion? Delete every duplicate immediately. Replace the identifier with a random value. Check unit of observation, key and duplication source to determine whether repetition is legitimate or a load error. Convert duplicate rows to missing values.
Question 15Which cleaning-log entry is the most traceable? We cleaned the data and it is now good. We removed some rows because they looked wrong. Rule: closed_at ≥ opened_at; affected 2/500; flagged for review and temporarily excluded from duration metric; no automatic correction without evidence. We applied cleaning best practice.
Question 16A daily report needs speed, but full verification delays data by two days. What is the best response? Always ignore timeliness because accuracy is absolutely more important. Publish any data immediately without labeling status. Balance purpose and error risk; if provisional data are published, label them, state completeness and update after verification. Combine all dimensions into one undefined quality score.