Missing Values Handle Karna: dropna, fillna aur Imputation

Lesson 11 of 26

Real data me values gayab hona bahut aam hai — form me kisi ne field khali chhod di, sensor band ho gaya, ya system me error aa gaya. Pandas me missing value NaN (Not a Number) ya None dikhti hai. Zyadatar ML models missing values ke saath chal hi nahi paate, isliye inhe handle karna zaroori hai.

Setup

Python 
import numpy as np
import pandas as pd

students = pd.DataFrame({
    "name": ["Aman", "Riya", "Kabir", "Sara", "Dev", "Isha"],
    "age": [21, np.nan, 23, 22, np.nan, 24],
    "city": ["Delhi", "Mumbai", None, "Delhi", "Pune", None],
    "marks": [78, 85, np.nan, 91, 66, 72],
})
print(students)

Output

Plain Text 
    name   age    city  marks
0   Aman  21.0   Delhi   78.0
1   Riya   NaN  Mumbai   85.0
2  Kabir  23.0    None    NaN
3   Sara  22.0   Delhi   91.0
4    Dev   NaN    Pune   66.0
5   Isha  24.0    None   72.0

Missing values dhoondhna

Example

Python 
print(students.isnull().sum())
print()
print("Missing %:")
print((students.isnull().mean() * 100).round(1))

Output

Plain Text 
name     0
age      2
city     2
marks    1
dtype: int64

Missing %:
name      0.0
age      33.3
city     33.3
marks    16.7
dtype: float64

Option 1: dropna() — rows hata dena

Example

Python 
print(students.dropna())                   # jis row me koi bhi NaN, wo gayi
print()
print(students.dropna(subset=["marks"]))   # sirf marks missing ho to hatao
print()
print(students.dropna(thresh=3))           # kam se kam 3 values honi chahiye

Output

Plain Text 
   name   age   city  marks
0  Aman  21.0  Delhi   78.0
3  Sara  22.0  Delhi   91.0

   name   age    city  marks
0  Aman  21.0   Delhi   78.0
1  Riya   NaN  Mumbai   85.0
3  Sara  22.0   Delhi   91.0
4   Dev   NaN    Pune   66.0
5  Isha  24.0    None   72.0

   name   age    city  marks
0  Aman  21.0   Delhi   78.0
1  Riya   NaN  Mumbai   85.0
3  Sara  22.0   Delhi   91.0
4   Dev   NaN    Pune   66.0
5  Isha  24.0    None   72.0

Dhyan dijiye — pehle command ne 6 me se sirf 2 rows chhodi! Rows hataane se kaafi data kho sakta hai, isliye ye tabhi kariye jab missing rows bahut kam hon.

Option 2: fillna() — value bhar dena

Numbers ke liye mean ya median, aur text ke liye mode (sabse common value) ya "Unknown":

Example

Python 
filled = students.copy()
filled["age"] = filled["age"].fillna(filled["age"].median())
filled["marks"] = filled["marks"].fillna(filled["marks"].mean())
filled["city"] = filled["city"].fillna(filled["city"].mode()[0])
print(filled)

Output

Plain Text 
    name   age    city  marks
0   Aman  21.0   Delhi   78.0
1   Riya  22.5  Mumbai   85.0
2  Kabir  23.0   Delhi   78.4
3   Sara  22.0   Delhi   91.0
4    Dev  22.5    Pune   66.0
5   Isha  24.0   Delhi   72.0

Ek saath kai columns ke liye dictionary de sakte hain:

Example

Python 
print(students.fillna({"age": 22, "city": "Unknown", "marks": 0}))

Output

Plain Text 
    name   age     city  marks
0   Aman  21.0    Delhi   78.0
1   Riya  22.0   Mumbai   85.0
2  Kabir  23.0  Unknown    0.0
3   Sara  22.0    Delhi   91.0
4    Dev  22.0     Pune   66.0
5   Isha  24.0  Unknown   72.0

Time series: ffill aur interpolate

Roz ke temperature jaise data me pichhli value aage le jaana (ffill) ya beech ki value andaaza lagana (interpolate) zyada sahi hota hai:

Example

Python 
temps = pd.Series([30, np.nan, np.nan, 33, 34],
                  index=pd.date_range("2026-06-01", periods=5))

print(pd.DataFrame({
    "original": temps,
    "ffill": temps.ffill(),
    "interpolate": temps.interpolate(),
}))

Output

Plain Text 
            original  ffill  interpolate
2026-06-01      30.0   30.0         30.0
2026-06-02       NaN   30.0         31.0
2026-06-03       NaN   30.0         32.0
2026-06-04      33.0   33.0         33.0
2026-06-05      34.0   34.0         34.0

Group ke hisaab se fill karna

Poore data ka average hamesha sahi nahi. IT wale ki missing salary ko HR ke average se bharna galat hoga. groupby().transform() se har group ka apna median lagaiye:

Example

Python 
emp = pd.DataFrame({
    "dept": ["IT", "IT", "IT", "HR", "HR", "HR"],
    "salary": [80000, np.nan, 90000, 40000, 42000, np.nan],
})
emp["salary_filled"] = emp["salary"].fillna(emp.groupby("dept")["salary"].transform("median"))
print(emp)

Output

Plain Text 
  dept   salary  salary_filled
0   IT  80000.0        80000.0
1   IT      NaN        85000.0
2   IT  90000.0        90000.0
3   HR  40000.0        40000.0
4   HR  42000.0        42000.0
5   HR      NaN        41000.0

scikit-learn ka SimpleImputer

ML pipelines me SimpleImputer use hota hai — ye training data se value seekhta hai aur wahi value naye data pe lagata hai:

Example

Python 
from sklearn.impute import SimpleImputer

imputer = SimpleImputer(strategy="median")
result = imputer.fit_transform(students[["age", "marks"]])
print(result)
print("Seekhi hui medians:", imputer.statistics_)

Output

Plain Text 
[[21.  78. ]
 [22.5 85. ]
 [23.  78. ]
 [22.  91. ]
 [22.5 66. ]
 [24.  72. ]]
Seekhi hui medians: [22.5 78. ]

Kab kaunsa tareeka?

Situation

Tareeka

Bahut kam rows missing (< 5%)

dropna()

Number column, outliers nahi

Mean se fill

Number column, outliers hain

Median se fill (outliers ka asar nahi)

Category / text column

Mode ya "Unknown"

Time series

ffill() ya interpolate()

Groups me alag pattern

groupby().transform()

Column me 60-70%+ missing

Column hi drop karne pe vichaar kariye

Kabhi-kabhi missing hona khud ek information hai — jaise loan form me income na bharna. Aise me fill karne se pehle ek naya column income_missing = df["income"].isnull() bana lena accha idea hai.

On This Page