Decision Tree bilkul insaan ki tarah sochta hai — sawaal poochta jaata hai: petal length 2.45 se kam hai? Haan → setosa. Nahi → agla sawaal... Isko samajhna aur samjhaana dono aasaan hai.
Decision Tree: rules dekhiye
Example
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier, export_text
iris = load_iris(as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(
iris.data, iris.target, test_size=0.2, random_state=42, stratify=iris.target
)
tree = DecisionTreeClassifier(max_depth=2, random_state=42)
tree.fit(X_train, y_train)
print(export_text(tree, feature_names=list(iris.feature_names)))
print("Test accuracy:", round(tree.score(X_test, y_test), 3))
Output
|--- petal length (cm) <= 2.45
| |--- class: 0
|--- petal length (cm) > 2.45
| |--- petal width (cm) <= 1.65
| | |--- class: 1
| |--- petal width (cm) > 1.65
| | |--- class: 2
Test accuracy: 0.933
class: 0, 1, 2 ka matlab setosa, versicolor, virginica. Sirf 2 level ke sawaalon se tree ne flowers ko kaafi achhe se alag kar diya.
Tree kaise decide karta hai ki kya poochna hai?
Har step pe tree wo sawaal chunta hai jo data ko sabse 'saaf' groups me baante — yaani har group me zyadatar ek hi class ho. Is saafi ko Gini impurity ya entropy se naapte hain.
Overfitting: tree ko khula chhod diya to?
Bina limit ka tree tab tak sawaal poochta rehta hai jab tak har training example sahi na ho jaaye — yaani ratta. Breast cancer data pe dekhte hain:
Example
import numpy as np
import pandas as pd
from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
cancer = load_breast_cancer(as_frame=True)
X, y = cancer.data, cancer.target # 0 = malignant, 1 = benign
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
for depth in [1, 2, 3, 5, None]:
t = DecisionTreeClassifier(max_depth=depth, random_state=42).fit(X_train, y_train)
print(f"max_depth={str(depth):>4}: train={t.score(X_train, y_train):.3f}, "
f"test={t.score(X_test, y_test):.3f}")
Output
max_depth= 1: train=0.923, test=0.921
max_depth= 2: train=0.958, test=0.895
max_depth= 3: train=0.976, test=0.939
max_depth= 5: train=0.993, test=0.921
max_depth=None: train=1.000, test=0.912
Depth badhne pe train score 1.0 tak pahunch jaata hai, par test score ek point ke baad girne lagta hai — ye overfitting ki nishaani hai. max_depth, min_samples_leaf jaise settings se tree ko control karte hain.
Random Forest: kai trees ki voting
Ek tree galti kar sakta hai, par sau trees ki raay zyada bharosemand hai. Random Forest bahut saare trees banata hai — har ek ko data ka thoda alag random hissa aur features ka random subset deta hai — aur aakhir me voting hoti hai. Isko ensemble learning kehte hain.
Example
from sklearn.ensemble import RandomForestClassifier
forest = RandomForestClassifier(n_estimators=200, random_state=42, n_jobs=-1)
forest.fit(X_train, y_train)
print("Train accuracy:", round(forest.score(X_train, y_train), 3))
print("Test accuracy :", round(forest.score(X_test, y_test), 3))
Output
Train accuracy: 1.0
Test accuracy : 0.956
Feature importance
Random Forest batata hai ki prediction me kaunsa feature kitna kaam aaya:
Example
importance = pd.Series(forest.feature_importances_, index=X.columns)
print(importance.sort_values(ascending=False).head(5).round(3))
Output
worst perimeter 0.133
worst area 0.128
worst concave points 0.108
mean concave points 0.094
worst radius 0.091
dtype: float64
Example
import matplotlib.pyplot as plt
importance.sort_values().tail(8).plot(kind="barh", color="#10b981")
plt.title("Top 8 features (Random Forest)")
plt.xlabel("Importance")
plt.show()
Decision Tree vs Random Forest
Decision Tree | Random Forest | |
|---|---|---|
Samajhna | Bahut aasaan — rules dikhte hain | Mushkil (sau trees) |
Overfitting | Jaldi hoti hai | Kaafi kam |
Accuracy | Theek-thaak | Aam taur pe behtar |
Speed | Bahut fast | Thoda slow |
Scaling chahiye? | Nahi | Nahi |
Tree-based models ka agla level Gradient Boosting hai (XGBoost, LightGBM) — tabular data pe competitions me yahi sabse zyada jeet-te hain. Ek practical example: California Housing Price Prediction with Random Forest.