anomaly-detection-model-builder

Anomaly detection is ranking rows by how unusual they are and deciding how many to look at. The hard parts are not the algorithm: defining unusual relative to what (a feature's normal range, the usual combination of features, the entity's own history, the time of day), choosing the alert budget, and evaluating with almost no labels. scripts/anomaly_scores.py scores rows with four complementary detectors (robust z, Mahalanobis, k-NN distance, Isolation Forest) plus a rank ensemble, and explains each alert; scripts/anomaly_eval.py measures precision at the alert budget when labels exist and detector agreement and thresholds when they do not.

#Trigger Conditions

Activate this skill when the user:

• Wants to find outliers, unusual transactions, faulty sensors, abnormal user behaviour or data-quality errors
• Has few or no labels for the bad events (fraud before labels, new failure modes)
• Asks Isolation Forest vs LOF vs one-class SVM vs autoencoder, or how to set contamination
• Needs to explain why a row was flagged, or to choose a threshold for an alert queue
• Wants to evaluate an anomaly detector or compare detectors

Supervised fraud models (enough labels): fraud-detection-model-designer; PCA-based process monitoring (T², Q): pca-dimensionality-reducer; rare-positive classification: imbalanced-class-handler; time-series anomalies via forecasts: time-series-forecasting-model-builder.

#Workflow

1. DEFINE     -> what is anomalous: point / contextual (given time, entity, segment) / collective (a sequence)
2. NORMAL     -> a reference period / population that is mostly normal; remove known incidents
3. FEATURES   -> deviations from the right context (vs own history, vs peer group, vs time of day)
4. SCORE      -> python scripts/anomaly_scores.py --train ref.csv --score new.csv --drop id --out scores.csv
5. BUDGET     -> how many alerts per day can be reviewed? threshold = score of the k-th row
6. EVALUATE   -> python scripts/anomaly_eval.py scores.csv --scores ... --label known --k 10 20 50
7. EXPLAIN    -> deviating features per alert; route to the owner; collect review outcomes as labels
8. ITERATE    -> labels accumulate -> semi-supervised -> supervised model (fraud-detection skill)

#Anomaly Types and Features

The best detector cannot see a contextual anomaly if the context is not in the features. Build "relative to normal for this entity / time" features first (time-based-feature-builder).

#Detector Guide

On the sensor fixture (6 planted anomalies, one contextual) every detector reaches ROC-AUC >= 0.99; the contextual one (temperature 76 and pressure 26.5, each plausible alone) is ranked high mainly by Mahalanobis. The pure-Python Isolation Forest's PR-AUC (0.976) is in line with scikit-learn's (0.944) on the same data.

Code for each detector in scikit-learn / PyOD: detector-recipes.md.

#Thresholds and Alert Budgets

anomaly_eval.py prints each of these per score column.

#Evaluation With and Without Labels

Few labels mean wide intervals: report counts ("caught 5 of 6 known incidents at 10 alerts/day"). Evaluation design and label collection loops: evaluation-and-operations.md.

#Explaining Alerts

For each alert show the features that deviate most (robust z per feature), the detector(s) that flagged it, and the context (entity history, peer group). anomaly_scores.py lists the top three deviating features per row; for Isolation Forest models in production, SHAP's TreeExplainer also works.

#Output Format

When building an anomaly detector, deliver:

1. Definition - anomaly type(s), unit (row / entity / window), reference period.
2. Features - context-relative features and scaling.
3. Detectors - chosen detectors and why; scores from anomaly_scores.py.
4. Threshold - alert budget and the matching cut-off.
5. Evaluation - precision@k / recall on known incidents (or agreement and synthetic recall).
6. Explanations and workflow - per-alert reasons, review queue, how labels feed back.

#Scripts

python scripts/anomaly_scores.py --train sensors.csv --drop reading_id is_anomaly --top 10
python scripts/anomaly_scores.py --train reference.csv --score new.csv --drop id --out scores.csv
python scripts/anomaly_scores.py --train data.csv --columns temperature pressure --k 10 --trees 200 --json

Fits on --train (the normal reference) and scores --score (default: the training rows): robust z (max |x − median| / 1.4826 MAD), Mahalanobis with covariance from the central 90% of rows, distance to the k-th nearest neighbour on standardised features, Isolation Forest (--trees, subsamples of 256), and the mean of rank-normalised scores; top rows with their three most deviating features; --out writes every score. Exit codes: 0 scored, 2 usage error.

python scripts/anomaly_eval.py scores.csv --scores score_iforest score_knn score_ensemble --label is_anomaly --k 5 10 20
python scripts/anomaly_eval.py scores.csv --scores score_robust_z score_mahalanobis --k 10

With --label: ROC-AUC, PR-AUC, precision@k and recall@k per score and the best detector per budget (ties listed). Without labels: Spearman agreement and top-k Jaccard between detectors. Always: thresholds at each budget, 99th/99.5th percentiles and median + 3 MAD with the number of rows each flags. Exit codes: 0 evaluated, 2 usage error.

#Additional Resources