Exploratory Data Analysis (EDA) with AI Prompts

EDA is where data science projects live or die — miss a key distribution or an unexpected correlation and your model will underperform in ways that are hard to diagnose later. AI can be your most productive EDA companion if you give it a structured brief. This topic shows you how.

1. Introduction

Exploratory data analysis is the process of getting to know a dataset before you model it — understanding its shape, distributions, relationships, and anomalies. Traditionally this means writing dozens of small scripts and plots. With well-structured prompts, you can ask AI to write an entire EDA notebook in one go, covering univariate statistics, bivariate correlations, time-series patterns, and anomaly flags — all tailored to your specific schema and business question.

2. The Concept Explained

Good EDA moves through layers of understanding, from broad to specific. First you understand the overall shape of the data (how many rows, what types). Then you look at individual column distributions. Then you explore relationships between columns. Finally, you look for anomalies — values or patterns that don't fit the expected story. Think of it like a pyramid: wide at the base (shape), narrowing toward specific hypotheses at the top.

An AI EDA prompt should mirror this pyramid. Start with a prompt that generates a data profiling report, then follow up with targeted prompts for correlations, time trends, and outlier investigation. Chaining prompts through the pyramid is far more effective than asking for "a complete EDA" in a single shot.

Prompt chain structure for EDA

Prompt 1 — Profile: Generate df.describe(), null counts, unique value counts, and dtype summary.
Prompt 2 — Distributions: Plot histograms for numeric columns, bar charts for categoricals.
Prompt 3 — Relationships: Correlation heatmap, scatter plots for target vs top features.
Prompt 4 — Anomalies: Flag rows outside 3 standard deviations; check for impossible values per column.

3. The Problem Without This Technique

Do an EDA on my data and find insights.

Act as a data analyst running EDA in a Jupyter notebook.

Dataset: subscription SaaS metrics, ~50,000 rows.
Columns:
  customer_id (int), signup_date (datetime),
  plan_type (str: basic/pro/enterprise),
  monthly_revenue (float), churn_date (nullable datetime),
  country (str, 40 unique values), support_tickets (int).

Business question: What customer characteristics
and behaviours predict churn within 90 days?

Generate a Pandas + Matplotlib EDA script that:
1. Prints shape, dtypes, and null counts.
2. Plots distributions of monthly_revenue and
   support_tickets (histogram + box plot each).
3. Plots churn rate by plan_type and country (bar charts).
4. Shows a correlation matrix for numeric columns.
5. Flags any customer_id values that appear more than once.

Use plt.tight_layout() and label all axes clearly.
Return only the code, structured as commented sections.

4. The Solution

Chain your EDA prompts through the pyramid. The first prompt profiles the data; subsequent prompts drill into specific layers. Always state the business question — "what predicts churn?" shapes which analyses are actually useful and prevents the AI from spending effort on irrelevant columns.

5. Step-by-Step Breakdown

1. Share the schema and row count. The AI needs to know what it is working with before it can suggest sensible visualisations.
2. State the business question. This filters the EDA to analyses that matter rather than analyses that are generically possible.
3. Request profiling first. Shape, dtypes, nulls, unique counts — this is fast to generate and immediately surfaced issues.
4. Ask for visualisations by data type. Numeric → histogram + box plot. Categorical → bar chart. Date → line chart over time. Pairwise → scatter / heatmap.
5. Follow up with anomaly detection. After you see the distributions, ask a second prompt: "Flag rows where monthly_revenue > 3 standard deviations above the mean."
6. Iterate with follow-up questions. EDA is a conversation. Paste the output back in: "The histogram shows a bimodal distribution in monthly_revenue. What would cause that and how should I investigate further?"

6. Practice Exercises

7. Key Takeaways