EDA (Exploratory Data Analysis) ka matlab hai data se baat karna — model banane se pehle samajhna ki data me kya hai, kaisa hai aur kaunse patterns hain. Accha EDA aapko galat assumptions se bachata hai aur aage ke features ka idea deta hai.
Is lesson me ek company ke 200 employees ka dataset lenge (practice ke liye NumPy se generate kiya hai):
Setup
import numpy as np
import pandas as pd
rng = np.random.default_rng(7)
n = 200
department = rng.choice(["IT", "Sales", "HR", "Finance"], size=n, p=[0.4, 0.3, 0.1, 0.2])
experience = rng.integers(0, 16, size=n)
education = rng.choice(["Graduate", "Post Graduate"], size=n, p=[0.65, 0.35])
base = {"IT": 45000, "Sales": 35000, "HR": 32000, "Finance": 42000}
salary = (np.array([base[d] for d in department])
+ experience * 4000
+ np.where(education == "Post Graduate", 8000, 0)
+ rng.normal(0, 6000, size=n)).round(-2)
emp = pd.DataFrame({"department": department, "experience": experience,
"education": education, "salary": salary})
print(emp.head())
print(emp.shape)
Output
department experience education salary
0 Sales 10 Graduate 79500.0
1 Finance 12 Graduate 94900.0
2 HR 6 Post Graduate 58200.0
3 IT 10 Post Graduate 84700.0
4 IT 3 Post Graduate 62900.0
(200, 4)
Step 1: Overview
Example
emp.info()
print()
print(emp.describe().round(1))
Output
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 200 entries, 0 to 199
Data columns (total 4 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 department 200 non-null object
1 experience 200 non-null int64
2 education 200 non-null object
3 salary 200 non-null float64
dtypes: float64(1), int64(1), object(2)
memory usage: 6.4+ KB
experience salary
count 200.0 200.0
mean 7.7 73323.0
std 4.5 19619.0
min 0.0 23600.0
25% 4.0 57275.0
50% 8.0 73400.0
75% 12.0 88850.0
max 15.0 119100.0
Koi missing value nahi hai, experience 0 se 15 saal tak hai, aur salary ka mean aur median (50%) kaafi paas hain — yaani salary me bahut bade outliers nahi hain.
Step 2: Univariate analysis (ek column ek baar me)
Category columns ke liye value_counts, number columns ke liye distribution:
Example
print(emp["department"].value_counts())
print()
print(emp["education"].value_counts(normalize=True).round(2))
print()
print("Salary skewness:", round(emp["salary"].skew(), 2))
Output
department
IT 77
Sales 66
Finance 44
HR 13
Name: count, dtype: int64
education
Graduate 0.63
Post Graduate 0.37
Name: proportion, dtype: float64
Salary skewness: -0.06
Skewness 0 ke paas ho to distribution lagbhag symmetric hai. Chart se dekhte hain:
Example
import matplotlib.pyplot as plt
plt.hist(emp["salary"], bins=20, color="#f97316", edgecolor="white")
plt.title("Salary distribution")
plt.xlabel("Salary (Rs)")
plt.ylabel("Employees")
plt.show()
Step 3: Bivariate analysis (do columns ka rishta)
Category vs number — groupby se:
Example
print(emp.groupby("department")["salary"].agg(["count", "mean", "median"]).round(0)
.sort_values("mean", ascending=False))
Output
count mean median
department
IT 77 76704.0 80500.0
Finance 44 75450.0 78250.0
Sales 66 69011.0 68350.0
HR 13 67992.0 70600.0
IT department ki average salary sabse zyada hai aur HR ki sabse kam. Boxplot se har department ka poora spread dikhta hai:
Example
emp.boxplot(column="salary", by="department", grid=False)
plt.title("Department ke hisaab se salary")
plt.suptitle("")
plt.ylabel("Salary (Rs)")
plt.show()
Number vs number — correlation se:
Example
print("Experience vs Salary correlation:",
round(emp["experience"].corr(emp["salary"]), 3))
print()
print(emp.groupby("education")["salary"].mean().round(0))
Output
Experience vs Salary correlation: 0.915
education
Graduate 69872.0
Post Graduate 79199.0
Name: salary, dtype: float64
Correlation 0.91 hai — ye strong positive rishta hai: experience badhta hai to salary bhi badhti hai. Post Graduates ki average salary bhi zyada hai.
Step 4: Category vs category — crosstab
Example
print(pd.crosstab(emp["department"], emp["education"], margins=True))
Output
education Graduate Post Graduate All
department
Finance 29 15 44
HR 7 6 13
IT 51 26 77
Sales 39 27 66
All 126 74 200
Step 5: Insights likhna
EDA ka aakhri aur sabse zaroori step hai — jo mila usse simple bhasha me likhna:
- Company me 200 employees hain, sabse zyada IT department me.
- Experience aur salary ka correlation 0.91 hai — salary ka sabse bada driver experience hai.
- Post Graduate employees graduates se average Rs 9,326 zyada kamaate hain.
- Data me missing values ya bade outliers nahi hain, to ye seedha modeling ke liye taiyaar hai.
EDA checklist
shape,info(),describe()— overview- Missing values aur duplicates
- Har column ka distribution (value_counts, histogram)
- Target ke saath har feature ka rishta (groupby, boxplot, correlation)
- Outliers aur ajeeb values
- Insights likhna