Credit Risk Scorecard

Explore real applicants

Twenty real, held-out test-set applicants the model never trained on, spanning every risk decile. Each card shows the actual raw application data, the model's real predicted probability of default, and what actually happened. Deliberately includes cases the model got wrong, a scorecard that only shows its wins isn't a scorecard, it's marketing.

Why WoE/IV and a from-scratch LR, not just "call XGBoost"

Credit scorecards face regulatory and model-risk scrutiny that most ML use cases don't: a rejected applicant can ask why, and "the gradient boosted trees said so" isn't an acceptable answer in most jurisdictions. Weight of Evidence encoding and logistic regression are the industry-standard combination precisely because every coefficient is interpretable and every score decomposes into named reasons. Building both from scratch, validated line-for-line against scikit-learn, was the only way to be sure this project understood the method rather than just calling a library that already does.

How it was built

Load and join

8 Home Credit tables joined on SK_ID_CURR: application, bureau, prior applications, POS/cash, credit cards, installments. (src/io_utils.py)

Feature engineering

418 columns from anomaly flags, ratios, and per-applicant aggregation of every relational table. (src/feature_engineering_*.py)

WoE / IV selection

Quantile binning and Information Value from first principles, ranked on the train split only to avoid leakage. Top 80 of 400 numeric features selected. (src/woe_iv.py)

From-scratch logistic regression

Sigmoid, cross-entropy loss, batch gradient descent, L2 regularization, in NumPy, validated against sklearn on the same data. (src/from_scratch_lr.py)

LightGBM benchmark

A library call, honestly labeled as such, to see how much a tree ensemble adds over the linear model.

Hand-coded metrics

AUC (rank-sum), GINI, KS, PSI, Brier, calibration, every one validated against scikit-learn to floating-point precision. (src/metrics.py)

Results: verified pipeline output, not notebook estimates

These numbers are the real output of run_pipeline.py run end-to-end against the real Kaggle dataset, re-run and reconfirmed to match to the digit. An earlier version of this project's docs claimed a from-scratch gradient boosting implementation and a 0.7747 AUC that were never actually built or run, see README.md and BUILD_STATUS.md for that history.

0.7774

LightGBM AUC

0.7509

From-scratch LR AUC

0.9985

LR / sklearn correlation

0.004

Max coef diff (10-feature set)

0.5548

LightGBM GINI

0.4251

LightGBM KS

Top 20 features by Information Value, dominated by external bureau scores

Calibration curve — Calibration: from-scratch LR tracks the diagonal, LightGBM does not

Lift chart by score decile — Lift by score decile: 3.6x the average default rate in the riskiest decile

Predicted probability distribution by outcome — Predicted probability distribution: real separation, real overlap

What this doesn't claim

Honest limitation #1: LightGBM's raw probabilities are badly calibrated

LightGBM wins on discrimination (AUC, GINI, KS all higher than the LR) but its raw output systematically understates default probability, at a true default rate of ~25% in the highest-risk decile, its mean predicted probability sits under 10%. It would need isotonic or Platt calibration before feeding an expected-loss calculation (EL = PD x LGD x EAD) directly.

Honest limitation #2: coefficient-level gap on the full feature set

The from-scratch LR's coefficients differ from sklearn's by up to 0.298 on the full 80-feature set, even though predictions correlate at 0.9985 and AUC matches to 4 decimals. This is multicollinearity among the IV-selected features (several EXT_SOURCE derivatives are near-duplicates by construction), not a bug, on a smaller low-collinearity 10-feature set the max coefficient diff drops to 0.004.

Honest limitation #3: random split, not out-of-time

This run uses a random stratified train/val/test split. Home Credit's DAYS_DECISION field would support a real temporal holdout, which is the harder and more realistic test for a model meant to predict future defaults. Also unaddressed: the ~18 categorical columns (occupation type, income type, etc.) are currently dropped rather than encoded.

A credit scorecard built the way
a model-risk team would audit it