# Leakage Taxonomy

## 1. Outcome-side fields
Columns written because the outcome happened.
- Churn: `cancellation_reason`, `retention_offer_accepted`, `account_status = 'closed'`.
- Credit: `collections_flag`, `charge_off_date`, `restructured_loan`.
- Fraud: `chargeback_amount`, `case_status`, `analyst_decision`.
- Healthcare: discharge disposition, billing codes finalised after the stay, "treatment for X" when predicting X.
Detection: single-feature AUC, is-missing AUC, outcome-like names (LD001/LD002/LD004).

## 2. Overwritten current-state fields
A field that is correct today but was different at prediction time: `last_login_date`, `plan`, `address`,
`credit_limit`, `segment` recomputed monthly. Joining today's value to a snapshot from six months ago leaks
anything that changed because of the outcome (customers who churned stop logging in; their `last_login`
is old, but customers who are still active have recent ones even if they were inactive at the snapshot).
Detection: dates later than the snapshot (LD003), signal stronger on old rows (LD008).

## 3. Aggregates over the future
`lifetime_orders`, `total_tickets`, `avg_rating` computed at extraction time over all history, including
events after the prediction time. Fix: aggregate only events with `ts < prediction_time`.

## 4. Proxies of the label
Variables generated by the same process as the label: "number of collection calls" when predicting default;
"refund issued" when predicting complaints; a downstream model's score that was trained on the label.

## 5. Identifiers and ordering
- Autoincrement IDs assigned when a case is opened (cases opened later are more likely to be fraud
  investigations).
- Files sorted by label; patient IDs encoding the hospital that treats the sickest patients.
- Image/record file names containing the class.
Detection: AUC of row index or ID (LD005).

## 6. Duplicates and near-duplicates across splits
The same customer, device, patient or text appears in train and test (often under multiple snapshots).
The model memorises entities. Fix: group-aware splits (GroupKFold), deduplication. See
train-test-split-auditor.

## 7. Target encoding and target statistics in-sample
Category means of the target computed on all rows, including the row being encoded (LK006). With rare levels
the encoding *is* the label.

## 8. Temporal leakage in validation
Random splits on data where the model predicts the future: training contains later periods than some test
rows, so the model learns period effects and uses "future" patterns. Fix: time-ordered splits, gaps equal to
the label window when windows overlap (LK007).

## 9. Preprocessing and selection leakage
Scalers, imputers, PCA, encoders, feature selection or resampling fitted before the split (LK001-LK003).
Effect size ranges from negligible (standard scaling, big data) to dramatic (feature selection among thousands
of features; SMOTE before split).

## 10. Leakage through hyperparameter search
Tuning on the test set, or reporting the best of many configurations on the same test data. Fix: nested CV
or a final untouched holdout; report the score of the selected configuration on fresh data.

## 11. Label leakage via joins
Joining a table that is filtered by the label (only churned customers have rows in `exit_survey`), so the
existence of a joined row encodes the label. Missingness AUC detects it.

## 12. Competition-style leaks
Leaky metadata (file sizes, timestamps), test-set structure, row order. Useful to recognise; never to
exploit in production models.

## Real cases worth knowing

| Case | Leak |
|------|------|
| Pneumonia mortality model ranked asthma patients as low risk | asthma patients went straight to ICU (treatment encoded in data) |
| Prostate cancer model used `PROSSY` (surgery) | treatment after diagnosis |
| KDD Cup 2008 breast cancer | patient ID encoded the screening site with different base rates |
| Kaggle competitions where file IDs or timestamps predicted the label | data collection artefacts |
| Credit models using "number of collection calls" | consequence of default |