Diagnosing Business Problems with Data

🔄 Quick Recall: In the previous lesson, you learned to use AI for analytics — context-rich prompts for data exploration, anomaly detection that flags the unusual (but not necessarily the important), and predictive analysis that requires diagnosis and intervention design to be useful. Now you’ll learn the diagnostic techniques that answer the hardest analytics question: why is this happening?

Cohort Analysis: Finding What Averages Hide

Averages lie. They blend new customers with old ones, strong months with weak ones, and improving trends with declining ones. Cohort analysis separates your data into groups that share a common starting point, then tracks each group over time.

How cohort analysis works:

Reading this table: the January cohort retained 92% of customers after one month. The March cohort only retained 85% after one month. That declining trend is invisible in the overall retention metric — but it’s the early warning signal that something changed.

When to use cohort analysis:

• Retention is “stable” but something feels off
• You’ve made a change and need to measure its impact on new users vs. existing ones
• Customer quality seems to be shifting but aggregate metrics don’t show it
• You want to compare the behavior of different customer groups over time

Help me build a cohort analysis.

Data: [describe your data — customer sign-up dates,
purchase history, usage logs, etc.]
Question: [what are you trying to understand?]
Cohort definition: [how should customers be grouped?
By sign-up month, acquisition channel, plan type, etc.]

Analyze:
1. How does each cohort's retention/engagement compare?
2. Is there a trend — are newer cohorts better or worse?
3. At which time period do you see the biggest drop-off?
4. What changed between the best-performing and
   worst-performing cohorts?
5. What does this suggest about root cause?

✅ Quick Check: Why does cohort analysis reveal problems that aggregate metrics miss? Because aggregates blend all customers together — a mix of loyal old customers and churning new ones can average out to “stable.” Cohort analysis separates by starting point, so you can see that your January cohort retained at 92% but your April cohort only retained at 82% — revealing a declining trend that the aggregate hides.

Funnel Analysis: Finding Where Value Leaks

Every business has a funnel — a sequence of steps from first contact to desired outcome. Funnel analysis measures the conversion rate at each step to find where the biggest drop-offs are.

Funnel analysis for different business types:

The leverage point principle: The funnel stage with the lowest conversion rate is usually your highest-leverage improvement opportunity. Improving a 5% conversion step to 10% doubles output from that point down. Improving a 50% step to 55% adds only 10%.

Funnel diagnosis questions:

1. Which step has the biggest absolute drop-off?
2. Which step has the lowest conversion rate relative to benchmarks?
3. Does the drop-off vary by segment (channel, device, customer type)?
4. Has the drop-off changed over time? (Getting better or worse?)

Comparative Analysis: Finding What’s Different

When performance varies — between teams, products, customer segments, or time periods — comparative analysis reveals why.

The comparative profiling technique:

1. Define two groups: high performers vs. average (or growing segment vs. declining segment)
2. Compare every available metric between the two groups
3. Identify the biggest gaps — these are your diagnostic clues
4. Validate with qualitative research (interviews, observations)

This comparison reveals that retained customers log in frequently and reach first value quickly, while churned customers barely engage in week one. The fix isn’t “reduce churn” — it’s “accelerate first-week engagement and time-to-value.”

✅ Quick Check: Why is comparative analysis more useful than analyzing a single group? Because analyzing one group in isolation tells you what is. Comparing two groups tells you what’s different — and differences are where insights live. Knowing that churned customers average 2.1 first-week logins is just a number. Knowing that retained customers average 8.2 first-week logins makes 2.1 an obvious problem to fix.

Key Takeaways

• Cohort analysis exposes trends that aggregate metrics hide — declining retention in newer cohorts, improving conversion after a product change, or shifting customer quality over time — by tracking groups with shared starting points separately
• Funnel analysis identifies your highest-leverage improvement opportunity: the step with the lowest conversion rate, where a small improvement multiplies results for every step below it
• The “more traffic” instinct is usually wrong — fixing leaks in the middle and bottom of the funnel is almost always cheaper and faster than pouring more into the top
• Comparative profiling (top vs. average, retained vs. churned, growing vs. declining) reveals what’s different between groups — and differences are where actionable insights live
• Single-variable explanations for complex outcomes (“they just make more calls”) are almost always oversimplified — compare every available metric to find the real drivers, then validate with qualitative research

Up Next: You’ll learn to communicate your findings effectively — turning data analysis into compelling stories that executives understand, believe, and act on.