Rapid Data Exploration

The Exploration Phase

In the previous lesson, we explored asking better questions. Now let’s build on that foundation. Before you can answer questions about data, you need to understand the data itself.

What columns exist? What do they mean? What are the value ranges? Where are the gaps?

This is data exploration—and it’s where AI dramatically speeds up your workflow.

The Data Profiling Checklist

For any new dataset, understand:

1. Structure

• How many rows (records)?
• How many columns (fields)?
• What are the column names?

2. Data Types

• Which columns are numeric?
• Which are text (categorical)?
• Which are dates?
• Which are identifiers?

3. Value Ranges

• What’s the min/max for numeric columns?
• What unique values exist in categorical columns?
• What date range is covered?

4. Quality Issues

• Missing values (which columns, how many)?
• Duplicates?
• Obvious errors or outliers?
• Inconsistent formats?

5. Relationships

• How do columns relate to each other?
• What could be used to join this with other data?

AI-Powered Data Profiling

Use AI to profile a dataset quickly:

Here's my dataset (first 100 rows):

[Paste data]

Please provide a data profile:

1. STRUCTURE
   - Number of columns and their names
   - Apparent purpose of each column

2. DATA TYPES
   - Classify each column (numeric, categorical, date, ID)
   - Flag any columns with mixed types

3. VALUE ANALYSIS
   - For numeric columns: min, max, apparent average
   - For categorical columns: unique values (up to 10)
   - For date columns: range covered

4. QUALITY ISSUES
   - Columns with missing values and approximate %
   - Obvious outliers or suspicious values
   - Inconsistencies (formatting, naming)

5. INITIAL OBSERVATIONS
   - Anything unusual or noteworthy
   - Potential relationships between columns

In one prompt, you get what might take an hour manually.

Common Data Quality Issues

Learn to spot these quickly:

Missing Values

What to look for: Blanks, “N/A”, “NULL”, “#N/A”, “0” used as placeholder

Questions to ask:

• Is the missing data random or systematic?
• Should we exclude these rows, fill them, or investigate why they’re missing?

Outliers

What to look for: Values far outside normal range

Questions to ask:

• Are these data errors or legitimate extreme cases?
• Will they skew averages and totals?
• Should they be treated separately?

Duplicates

What to look for: Identical or near-identical rows

Questions to ask:

• Are these true duplicates or valid repeat entries?
• What makes a row unique?

Inconsistent Formats

What to look for:

• Dates in different formats (01/15/2024 vs. 2024-01-15)
• Text variations (USA, US, United States)
• Numeric inconsistencies ($1,000 vs 1000)

Quick check: Before moving on, can you recall the key concept we just covered? Try to explain it in your own words before continuing.

Suspicious Patterns

What to look for:

• Too many round numbers (suggests estimates, not actual data)
• Default values used excessively
• Negative values where unexpected

The 5-Minute Exploration Routine

When you get a new dataset, run through this quickly:

Minute 1-2: Get the basics

AI: "Summarize this dataset. What are the columns, how many rows, and what time period does it cover?"

Minute 3-4: Check quality

AI: "Identify any data quality issues: missing values, outliers, duplicates, or inconsistencies."

Minute 5: Initial patterns

AI: "What patterns or relationships do you notice in this data? What questions could this data answer?"

Five minutes of exploration saves hours of working with bad data.

Understanding Data Relationships

Data rarely lives in isolation. Understanding relationships matters:

Within the Dataset

Ask AI:

Looking at these columns, what relationships might exist?
- Which columns might be correlated?
- Which columns might be derived from others?
- What groupings make sense?

With Other Data

Think about:

• What could this data be joined with?
• What ID fields could link to other datasets?
• What context is missing that other data could provide?

Practical Exploration Workflow

For Spreadsheet Data

1. Open and scan — Get visual sense of structure
2. Filter columns — Check unique values in key columns
3. Sort columns — Find min/max, spot outliers
4. Use AI — Profile for issues you might miss

For Large Datasets

1. Sample first — Work with representative sample
2. Profile the sample — Understand structure and issues
3. Validate patterns — Confirm findings on full dataset
4. Document issues — Note what needs cleaning

Exercise: Profile This Data

Here’s a sample dataset. Profile it using the checklist:

Date,Customer,Region,Product,Revenue,Units
2024-01-15,ACME Corp,North,Widget A,15000,100
2024-01-16,Beta Inc,South,Widget B,22500,150
2024-01-17,ACME Corp,North,Widget A,-500,
01/18/2024,Gamma LLC,East,Widget C,18000,120
2024-01-19,ACME Corp,north,widget a,16000,105
2024-01-20,Delta Co,West,Widget B,0,0
2024-01-21,ACME Corp,North,Widget A,15500,NULL

What issues do you spot?

Key Takeaways

• Always profile data before analysis—understand structure, types, ranges, and issues
• Use AI to speed up profiling: one prompt can reveal what takes an hour manually
• Watch for common issues: missing values, outliers, duplicates, format inconsistencies
• The 5-minute exploration routine catches problems early
• Understand relationships within the dataset and with other data sources
• Document issues found—you’ll need this for data cleaning

Next: creating visualizations that communicate findings clearly.

Up next: In the next lesson, we’ll dive into Visualizations That Communicate.