A response model predicts who will convert. An uplift model predicts who will convert because they were treated. The two lists differ:
- persuadables respond only when treated; target them
- sure things respond anyway; the treatment is wasted on them
- lost causes never respond
- sleeping dogs respond less when treated; the treatment does harm
Targeting by response probability spends budget on sure things and can wake the sleeping dogs. Uplift modelling needs randomised data (or known propensities) and its own evaluation:
scripts/rct_check.pyvalidates the experimentscripts/gen_uplift_script.pytrains the meta-learnersscripts/uplift_eval.pymeasures Qini, AUUC, deciles and the value of targeting
Trigger Conditions
Activate this skill when the user:
- Wants to target a campaign, discount, retention offer, notification or treatment by incremental effect
- Asks about uplift models, CATE, heterogeneous treatment effects, meta-learners or Qini / AUUC
- Has an A/B test and wants to know who benefited, or who was hurt
- Targets "high propensity" customers and sees little incremental lift
A/B test design and analysis of the average effect: ml-success-metric-selector for metric choice. Churn timing: survival-analysis-model-builder.
Workflow
1. DATA -> a randomised test (or logged propensities); treatment, outcome, pre-treatment covariates only
2. CHECK -> python scripts/rct_check.py data.csv --treatment-col treatment --outcome-col converted --covariates age segment --segments segment
3. MODEL -> python scripts/gen_uplift_script.py --data data.csv --covariates age segment -o uplift.py; python uplift.py
4. EVALUATE -> python scripts/uplift_eval.py holdout_scores.csv --scores s_learner t_learner x_learner class_transform --cost-per-treatment 0.5 --value-per-outcome 20
5. DECIDE -> treat the fraction with the best profit; exclude sleeping dogs; confirm with a new randomised rolloutBefore Modelling: Is There Anything to Model?
| Check | rct_check.py | If it fails |
|---|---|---|
| Randomisation worked | sample-ratio mismatch test (p < 0.001 = broken) | stop: fix assignment or logging |
| Groups comparable | SMD per covariate (abs value > 0.1 = imbalance) | investigate; adjust or re-randomise |
| Average effect | ATE with CI, relative lift, MDE | a null ATE can still hide opposite segment effects |
| Effects differ | per-segment effects and Cochran's Q | no heterogeneity: treat everyone or no-one, no uplift model needed |
Fixture (randomised, 2,000 people): ATE +3.2 points [+0.3, +6.1]. The segments diverge: A +13.2, B +2.1 and C −4.4 (Q = 21.5, p = 2e-5). These recover the planted effects (A +10, B +2, C −3). The numbers match scipy (SRM p, SMD). Covariates must be measured before treatment. Post-treatment variables (clicks on the offer) are not covariates. Design and data rules: uplift-data-and-design.md.
Learners
| Learner | Idea | Strength | Weakness |
|---|---|---|---|
| S-learner | one model with t as a feature | simple, data-efficient | can shrink effects to zero; a linear model without interactions gives one effect for everyone |
| T-learner | separate treated and control models | flexible | noisy when one arm is small; differences of two errors |
| X-learner | impute individual effects, regress them, weight by propensity | good with unbalanced arms | more stages |
| Class transformation | z = 1 for (treated, converted) or (control, not converted) | one classifier | binary outcomes; needs 50/50 or reweighting |
| Causal forest / DR-learner | honest splitting / doubly robust pseudo-outcomes | valid inference, observational data | econml / grf; more tuning |
Fixture holdout (800 people), Qini coefficient: class transformation 0.214, T-learner 0.179, X-learner 0.174 and S-learner 0.070 with logistic bases. With LightGBM bases: S 0.170, X 0.143, T 0.115, class transform 0.104. No learner wins everywhere, so compare them on the same holdout. Formulas and libraries: uplift-learners.md.
Evaluation
| Measure | Meaning |
|---|---|
| Qini coefficient | area between the Qini curve and random targeting, normalised by the perfect curve (1 perfect, 0 random) |
| AUUC | the same for the uplift curve |
| Uplift by decile | treated minus control rate per score decile with CI; should fall from top to bottom |
| Sleeping dogs | deciles with a significantly negative uplift: exclude them |
| Targeting value | IPW estimate of the outcome when treating the top q%, against treat-all and treat-none |
| Profit | value x outcomes - cost x treated, per person |
uplift_eval.py matches scikit-uplift's qini_auc_score and uplift_auc_score exactly (fixture: 0.1768 and
0.0785). On the fixture's provided score, the bottom decile is a significant sleeping-dog group (−10.1
points). Treating the top 30% earns +2.86 per person, against +2.31 for treating everyone.
Individual effects are never observed, so uplift can only be evaluated on randomised holdout data through these group comparisons. Evaluation details: uplift-evaluation.md.
Output Format
When building an uplift model, deliver:
- Experiment check - SRM, balance, ATE with CI, segment effects and heterogeneity.
- Models - learners, base models, covariates (pre-treatment only).
- Holdout evaluation - Qini/AUUC per learner, the decile table with CIs, sleeping dogs.
- Targeting decision - fraction to treat, expected incremental outcomes and profit against treat-all.
- Rollout plan - a randomised validation of the targeting policy with a holdout control.
Scripts
python scripts/rct_check.py uplift.csv --treatment-col treatment --outcome-col converted --covariates age segment --segments segment
python scripts/rct_check.py exp.csv --treatment-col variant --outcome-col revenue --expected-share 0.2 --jsonExit codes: 0 checks passed, 1 sample-ratio mismatch or imbalance, 2 usage error.
python scripts/gen_uplift_script.py --data uplift.csv --covariates age segment -o uplift.py
python scripts/gen_uplift_script.py --data exp.csv --covariates age plan region --learners t_learner x_learner --base logregWrites holdout_scores.csv and uplift_models.joblib. Exit codes: 0 written, 2 usage error.
python scripts/uplift_eval.py holdout_scores.csv --scores t_learner x_learner --cost-per-treatment 0.5 --value-per-outcome 20
python scripts/uplift_eval.py preds.csv --scores model --propensity-col p_treat --target-fracs 0.1 0.2 0.5 --jsonExit codes: 0 evaluated, 2 usage or read error. scripts/_uplift_io.py holds the shared parsing and
chi-square functions.
Additional Resources
| File | Content |
|---|---|
| uplift-data-and-design.md | the four customer types, experiment design for uplift, covariates, observational data, sample size |
| uplift-learners.md | S/T/X/R/DR learners, class transformation, uplift trees and causal forests, libraries |
| uplift-evaluation.md | Qini and uplift curves, normalisation, deciles, policy value with IPW, profit, rollout |
anomaly-detection-model-builder
Build and evaluate anomaly detection for tabular, sensor and transaction data with few or no labels: define the anomaly type (point, contextual, collective), choose detectors (robust z-score/MAD and IQR rules, Mahalanobis with robust covariance, k-NN distance, LOF, Isolation Forest, one-class SVM, PCA reconstruction / autoencoders, forecasting residuals for time series), combine them by rank, explain each alert by its deviating features, set thresholds from an alert budget or the score distribution, and evaluate with precision@k, PR-AUC and recall on known incidents, or detector agreement when no labels exist. Includes pure-Python detectors (robust z, Mahalanobis, kNN, Isolation Forest, ensemble) and an evaluation script.
catboost-model-builder
Build CatBoost models that use its strengths correctly: raw categorical columns as cat_features (ordered target statistics, no manual encoding), text_features, Pool construction with NaN-safe categories, ordered boosting and has_time for temporal data, symmetric (oblivious) trees, overfitting detector with eval_set and use_best_model, losses for classification, regression (Quantile, Poisson, Tweedie) and ranking (YetiRank), class weights, monotone constraints, feature importance (PredictionValuesChange, LossFunctionChange) and SHAP values, GPU training and model export. Plans cat/text/numeric columns from a CSV and generates a runnable training script, and checks parameter sets for the conflicts CatBoost rejects (iterations vs n_estimators, Bayesian bootstrap with subsample, multiple class-weight options, depth > 16, use_best_model without eval_set).
categorical-encoding-selector
Choose and implement the right encoding for every categorical column given the model family and the data: one-hot (with infrequent-level grouping and handle_unknown), ordinal with an explicit order, cross-fitted target encoding (scikit-learn TargetEncoder, CatBoost ordered statistics, category_encoders inside CV), frequency/count encoding, feature hashing, native categorical support in LightGBM, HistGradientBoosting, XGBoost and CatBoost, prefix levels for codes (zip3, NAICS-2), and embeddings. Measures cardinality, rare-level share, unseen-level rate on a time-ordered holdout and target signal per column, emits the ColumnTransformer, and quantifies target-encoding leakage (in-sample vs out-of-fold vs holdout AUC).
classification-model-selector
Choose the right classification algorithm for tabular, text or mixed data and prove the choice with a fair bake-off: logistic regression (L2, L1, elastic net), linear SVM, naive Bayes, k-NN, RBF SVM, decision trees, random forest, extra trees, HistGradientBoosting, LightGBM, XGBoost, CatBoost, explainable boosting machines (EBM/GAM), points scorecards and MLPs, matched to data size, width, categorical and high-cardinality share, missingness, class imbalance, text, and constraints such as interpretability, monotonic constraints, calibrated probabilities, latency budgets and tiny deployable models. Ranks candidates with starting hyperparameters and cautions, then generates a bake-off script running every candidate on identical CV folds with per-fold metrics, fit time and prediction latency.
clustering-algorithm-selector
Choose the right clustering algorithm and check that clustering makes sense at all: cluster tendency (Hopkins statistic with a correlated-features caveat, nearest-neighbour isolation), then k-means / MiniBatchKMeans, Gaussian mixtures with BIC, agglomerative (Ward, average, complete, single) with dendrograms, DBSCAN, HDBSCAN, OPTICS, spectral clustering, BIRCH, mean shift, k-prototypes and k-modes for mixed or categorical data, and Gower distance, matched to data size, expected shapes, noise, varying density, known k, soft memberships, hierarchy and streaming. Covers preprocessing (scaling, encoding, dimensionality, distance metrics), validation (internal indices, bootstrap stability, external labels, business profiling) and assigning new points.
collaborative-filtering-implementer
Implement collaborative filtering recommenders that beat popularity: item-based kNN with cosine, Jaccard or asymmetric similarity, shrinkage, neighbourhood size, positive-only or mean-centred rating signals and 'because you liked X' explanations; implicit ALS with confidence weighting, BPR pairwise ranking and the closed-form EASE model in plain NumPy, tuned on a temporal validation split by recall@k and NDCG@k and refit on all data; plus user-kNN, explicit matrix factorisation, the implicit/LightFM libraries, hyperparameter ranges, cold-start fallbacks, scaling with sparse matrices and approximate nearest neighbours. Produces top-k lists in the format of the recommendation evaluator.