Classical Machine Learning skills
Frame ML problems, engineer and select features, train and tune scikit-learn, XGBoost, LightGBM and CatBoost models, cluster and reduce dimensions, validate without leakage, explain models with SHAP, PDP, LIME and counterfactuals, and build forecasting, survival, recommendation, ranking, fraud and uplift models.
Install every Classical Machine Learning skill at once in Claude Code:
/plugin marketplace add MuhammadHusnainAli/vector-agentic-skills /plugin install classical-ml-skills@vector-agentic-skills
With npx skills, for any agent:
npx skills add https://agenticskills.vector-ai-lab.com --skill anomaly-detection-model-builder --skill catboost-model-builder --skill categorical-encoding-selector --skill classification-model-selector --skill clustering-algorithm-selector --skill collaborative-filtering-implementer --skill counterfactual-explanation-generator --skill cross-validation-strategy-designer --skill dbscan-parameter-tuner --skill feature-engineering-idea-generator --skill feature-importance-reporter --skill feature-scaling-advisor --skill feature-selection-analyzer --skill feature-store-schema-designer --skill fraud-detection-model-designer --skill hyperparameter-search-planner --skill imbalanced-class-handler --skill kmeans-cluster-count-finder --skill learning-curve-diagnoser --skill learning-to-rank-model-builder --skill lime-explanation-generator --skill ml-baseline-model-builder --skill ml-feasibility-assessor --skill ml-model-comparison-reporter --skill ml-problem-framing-advisor --skill ml-project-scoping-writer --skill ml-success-metric-selector --skill ml-vs-rules-decision-advisor --skill model-card-writer --skill optuna-study-writer --skill overfitting-underfitting-diagnoser --skill partial-dependence-plot-builder --skill pca-dimensionality-reducer --skill recommendation-system-designer --skill regression-model-trainer --skill scikit-learn-pipeline-builder --skill shap-value-explainer --skill survival-analysis-model-builder --skill target-leakage-detector --skill time-based-feature-builder --skill time-series-forecasting-model-builder --skill train-test-split-auditor --skill umap-tsne-embedding-visualizer --skill uplift-model-builder --skill xgboost-lightgbm-tuner
Or download them all:
mkdir -p ~/.claude/skills && curl -fsSL https://agenticskills.vector-ai-lab.com/dl/classical-ml-skills.tar.gz | tar -xz -C ~/.claude/skills
anomaly-detection-model-builder
Build and evaluate anomaly detection for tabular, sensor and transaction data with few or no labels: define the anomaly type (point, contextual, collective), choose detectors (robust z-score/MAD and IQR rules, Mahalanobis with robust covariance, k-NN distance, LOF, Isolation Forest, one-class SVM, PCA reconstruction / autoencoders, forecasting residuals for time series), combine them by rank, explain each alert by its deviating features, set thresholds from an alert budget or the score distribution, and evaluate with precision@k, PR-AUC and recall on known incidents, or detector agreement when no labels exist. Includes pure-Python detectors (robust z, Mahalanobis, kNN, Isolation Forest, ensemble) and an evaluation script.
catboost-model-builder
Build CatBoost models that use its strengths correctly: raw categorical columns as cat_features (ordered target statistics, no manual encoding), text_features, Pool construction with NaN-safe categories, ordered boosting and has_time for temporal data, symmetric (oblivious) trees, overfitting detector with eval_set and use_best_model, losses for classification, regression (Quantile, Poisson, Tweedie) and ranking (YetiRank), class weights, monotone constraints, feature importance (PredictionValuesChange, LossFunctionChange) and SHAP values, GPU training and model export. Plans cat/text/numeric columns from a CSV and generates a runnable training script, and checks parameter sets for the conflicts CatBoost rejects (iterations vs n_estimators, Bayesian bootstrap with subsample, multiple class-weight options, depth > 16, use_best_model without eval_set).
categorical-encoding-selector
Choose and implement the right encoding for every categorical column given the model family and the data: one-hot (with infrequent-level grouping and handle_unknown), ordinal with an explicit order, cross-fitted target encoding (scikit-learn TargetEncoder, CatBoost ordered statistics, category_encoders inside CV), frequency/count encoding, feature hashing, native categorical support in LightGBM, HistGradientBoosting, XGBoost and CatBoost, prefix levels for codes (zip3, NAICS-2), and embeddings. Measures cardinality, rare-level share, unseen-level rate on a time-ordered holdout and target signal per column, emits the ColumnTransformer, and quantifies target-encoding leakage (in-sample vs out-of-fold vs holdout AUC).
classification-model-selector
Choose the right classification algorithm for tabular, text or mixed data and prove the choice with a fair bake-off: logistic regression (L2, L1, elastic net), linear SVM, naive Bayes, k-NN, RBF SVM, decision trees, random forest, extra trees, HistGradientBoosting, LightGBM, XGBoost, CatBoost, explainable boosting machines (EBM/GAM), points scorecards and MLPs, matched to data size, width, categorical and high-cardinality share, missingness, class imbalance, text, and constraints such as interpretability, monotonic constraints, calibrated probabilities, latency budgets and tiny deployable models. Ranks candidates with starting hyperparameters and cautions, then generates a bake-off script running every candidate on identical CV folds with per-fold metrics, fit time and prediction latency.
clustering-algorithm-selector
Choose the right clustering algorithm and check that clustering makes sense at all: cluster tendency (Hopkins statistic with a correlated-features caveat, nearest-neighbour isolation), then k-means / MiniBatchKMeans, Gaussian mixtures with BIC, agglomerative (Ward, average, complete, single) with dendrograms, DBSCAN, HDBSCAN, OPTICS, spectral clustering, BIRCH, mean shift, k-prototypes and k-modes for mixed or categorical data, and Gower distance, matched to data size, expected shapes, noise, varying density, known k, soft memberships, hierarchy and streaming. Covers preprocessing (scaling, encoding, dimensionality, distance metrics), validation (internal indices, bootstrap stability, external labels, business profiling) and assigning new points.
collaborative-filtering-implementer
Implement collaborative filtering recommenders that beat popularity: item-based kNN with cosine, Jaccard or asymmetric similarity, shrinkage, neighbourhood size, positive-only or mean-centred rating signals and 'because you liked X' explanations; implicit ALS with confidence weighting, BPR pairwise ranking and the closed-form EASE model in plain NumPy, tuned on a temporal validation split by recall@k and NDCG@k and refit on all data; plus user-kNN, explicit matrix factorisation, the implicit/LightFM libraries, hyperparameter ranges, cold-start fallbacks, scaling with sparse matrices and approximate nearest neighbours. Produces top-k lists in the format of the recommendation evaluator.
counterfactual-explanation-generator
Generate counterfactual explanations and algorithmic recourse: the smallest realistic change to a row's features that changes the model's decision. Solves exact minimal-cost recourse for linear and logistic models (MAD-scaled L1 cost, immutable features, increase/decrease directions, bounds, integer and binary features, diverse alternatives, features every counterfactual must change, and a no-recourse finding when immutable features decide), generates DiCE counterfactuals for any scikit-learn model with constraints turned into permitted ranges and every result re-predicted, and validates counterfactuals from any method for constraint violations, implausible values, off-manifold rows, broken derived-feature relations and failure to flip the prediction.
cross-validation-strategy-designer
Design the cross-validation scheme that mirrors how a model will be used, and check the folds before trusting any score: StratifiedKFold and repeated CV for i.i.d. data, GroupKFold / StratifiedGroupKFold / LeaveOneGroupOut when entities repeat (customers, patients, devices, sessions), TimeSeriesSplit with gaps and expanding or sliding windows for forward prediction, purging and embargo for overlapping label windows, nested CV for tuning, spatial and hierarchical grouping, multilabel stratification, small-data choices (repeats, LOO) and the final untouched holdout. Detects repeated-entity and time columns from the data, recommends a splitter with scikit-learn code, and previews or audits folds for group overlap, time-order violations, label-window gaps and missing classes.
dbscan-parameter-tuner
Tune DBSCAN (and know when to switch to HDBSCAN or OPTICS): choose min_samples from dimensionality and the smallest meaningful group, find eps from the knee of the k-distance curve with a noise-share bracket, grid-search eps x min_samples for stable regions (cluster count that persists across neighbouring settings, noise share, largest-cluster share, silhouette on non-noise points, ARI against reference labels), and handle scaling, distance metrics (Euclidean, cosine, haversine for coordinates), high dimensions, varying densities, border points, noise interpretation, and assigning new points. Pure-Python k-distance and DBSCAN grid scripts validated against scikit-learn.
feature-engineering-idea-generator
Generate concrete, leakage-aware feature ideas for tabular ML and the pandas code to build them: infer column roles (IDs, entity keys, datetimes incl. unix epochs, years, amounts, counts, proportions, low- and high-cardinality categoricals, codes like zip, lat/lon, free text), then propose date parts and cyclical encodings, elapsed time to the prediction date, age from year, log1p and clipping for skewed values, missing indicators, rare-level grouping, frequency encoding, code prefixes, text length and TF-IDF, per-unit rates (total / tenure), date differences, geo distances, and point-in-time per-entity aggregates (prior counts, means, time since previous event, velocity). Ranks ideas P1-P3, flags columns that look written after the outcome, and includes domain playbooks (churn, fraud, credit, demand, marketing, real estate, IoT).
feature-importance-reporter
Produce honest feature importance reports by computing several methods on held-out data and explaining where they disagree: impurity/gain importance (LightGBM, XGBoost, CatBoost, sklearn MDI), standardised coefficients, permutation importance with repeats and standard deviations, grouped permutation for clusters of correlated features, drop-column importance and mean |SHAP|, all mapped back from encoded columns to input features. Flags features that look important only to impurity measures (split bias, redundancy), features with no measurable held-out effect, identifier-like columns, single features that dominate (leakage), and groups that share information.
feature-scaling-advisor
Decide whether and how to scale or transform numeric features for a given model: no scaling for trees and gradient boosting, StandardScaler, RobustScaler for outliers, MinMaxScaler and MaxAbsScaler (sparse data), log1p, Box-Cox and Yeo-Johnson PowerTransformer for skew, QuantileTransformer for heavy tails, clipping/winsorising, and target transforms (TransformedTargetRegressor) for skewed regression targets. Measures skew, robust outliers, zeros, negatives and bounds per column, shows how one wide column dominates raw distances, emits a leak-free ColumnTransformer, and measures the effect of each scaling on a k-NN model with cross-validation.
feature-selection-analyzer
Select features for tabular ML without fooling yourself: filter methods (missingness, near-zero variance, ID-like columns, univariate out-of-fold AUC / Spearman / mutual information, correlation-based redundancy groups keeping the strongest member), embedded methods (L1/elastic net, tree and GBM importance, SelectFromModel), wrapper methods (RFE/RFECV, sequential feature selection, Boruta), permutation importance on held-out data, stability selection across folds and bootstraps, and the cardinal rule that selection happens inside cross-validation. Flags near-perfect single features as probable leaks and measures importance stability across runs.
feature-store-schema-designer
Design feature store schemas that serve the same features to training and production: entities and join keys, sources with event and created timestamps, feature views grouped by entity and refresh cadence, dtypes, TTLs matched to source cadence, online vs offline stores, freshness SLAs, point-in-time correct historical retrieval, materialisation, feature services per model version, naming conventions with window suffixes, ownership, PII tagging and governance. Lints a JSON/YAML feature spec for 12 design errors and generates a working Feast repository (Entity, FileSource/BigQuerySource/SnowflakeSource/RedshiftSource, FeatureView, FeatureService, feature_store.yaml) verified with feast apply and get_historical_features; maps the design to Tecton, Databricks, SageMaker, Vertex AI and Hopsworks.
fraud-detection-model-designer
Design fraud detection models that survive production: point-in-time velocity and behaviour features per card, account or device (counts and amounts over sliding windows that exclude the current event, time since previous, amount relative to history, first-time merchant category or country), out-of-time splits with a label-maturity gap for chargeback delay, LightGBM with an Isolation Forest anomaly score, and evaluation the way fraud operations work: average precision against the fraud-rate baseline, precision/recall and value detection rate at alert budgets, recall at fixed false-positive rates, cost-optimal thresholds from fraud losses, review cost and customer friction, alert-capacity constraints and week-by-week stability. Covers rules versus models, label delay and noise, adversarial drift, graph and device signals, real-time scoring and feedback loops.
hyperparameter-search-planner
Plan hyperparameter searches that fit the compute budget and read their results correctly: choose grid, random, successive halving (HalvingRandomSearchCV), Bayesian/TPE (Optuna) or Hyperband/ASHA from fit cost, trial budget, parallel workers and the number of continuous parameters; define search spaces with log scales, conditional parameters and sensible ranges for logistic regression, elastic net, SVM, k-NN, random forest, extra trees, HistGradientBoosting, LightGBM, XGBoost, CatBoost and MLPs; fix what early stopping handles; then analyse cv_results_ or Optuna trials for best configurations, parameters at the edge of their range, sensitivity (including peaked effects), configurations within one standard error, and whether more trials would help.
imbalanced-class-handler
Handle imbalanced classification (fraud, churn, defaults, failures, rare diseases) without fooling yourself: measure with PR-AUC, recall at capacity and expected cost instead of accuracy, keep test sets at natural prevalence, stratified and repeated CV for few positives, choose thresholds from capacity, costs or precision/recall targets, apply class weights (class_weight, scale_pos_weight, auto_class_weights), resample only inside an imblearn Pipeline (random undersampling, SMOTE, ADASYN, SMOTE-NC, Tomek, ENN), correct probabilities after resampling with the prior-shift formula, calibrate, and know when to switch to anomaly detection or collect more labels. Profiles a dataset's imbalance severity with a cheapest-first strategy ladder and imblearn code, and tunes thresholds under six policies with prior correction.
kmeans-cluster-count-finder
Choose the number of clusters k for k-means (and k-means-like methods) with evidence instead of a single elbow plot: inertia elbow via the kneedle method, silhouette, Calinski-Harabasz, Davies-Bouldin, the gap statistic against uniform reference data (which also detects that there is no structure at all), bootstrap stability (adjusted Rand index on overlapping subsamples), BIC for Gaussian mixtures, cluster-size floors and business usefulness, then profile the chosen clusters (sizes, standardised feature differences, over-represented categories, outcome by cluster) so they can be named and acted on. Pure-Python k-means++ sweep validated against scikit-learn.
learning-curve-diagnoser
Build and read learning curves (training and validation score versus training-set size) to decide whether a model is data-limited, variance-limited or bias-limited, and therefore whether to collect more data, regularise, add capacity or engineer features. Generates scikit-learn learning_curve scripts with geometric size spacing, seeded shuffling, stratified/grouped/time-ordered splitters, fit times and a plot with std bands, then diagnoses the result: the gap at the largest size, validation gain per doubling, an inverse power-law fit capped at the training score with 2x/4x/10x predictions and the rows needed for a target score, plus warnings for noisy curves, memorisation, and validation that drops as data grows.
learning-to-rank-model-builder
Build learning-to-rank models for search, recommendations re-ranking and marketplace sorting: validate query-document data (contiguous query groups, single-label and no-relevant queries, duplicates, query-level features, single-feature baselines and leakage-like features), train LightGBM lambdarank and XGBoost rank:ndcg rankers with grouped cross-validation, early stopping on NDCG@k, label gains and truncation, and evaluate per query with NDCG (exponential or linear gain), MAP, MRR, ERR and precision@k, with tie handling made explicit and paired bootstrap / randomisation tests that tell whether a ranker really beats BM25 or the current system. Covers pointwise, pairwise and listwise losses, judgement and click labels, position bias, feature design and online interleaving.
lime-explanation-generator
Generate LIME (Local Interpretable Model-agnostic Explanations) for tabular predictions that are reproducible and honest: run LimeTabularExplainer with training statistics, categorical encoding decoded inside the prediction function (so Pipelines with one-hot encoders work), quartile rules or continuous slopes, kernel width and sample size chosen deliberately, and repeated runs with different seeds. Then check stability (sign consistency, coefficient of variation, top-k overlap), surrogate fidelity (weighted R^2) and the gap between surrogate and model, and compare signs with SHAP for the same row. Explains when LIME is the right tool, how kernel width and discretisation change the answer, and how to report it.
ml-baseline-model-builder
Build the baselines every ML model must beat before anyone trusts it: dummy majority/prior/mean/median predictors, the current business rule, a one-feature model (OneR), seasonal-naive and last-value forecasts, popularity recommenders, and a first honest linear model and HistGradientBoosting inside a leak-free scikit-learn Pipeline with a time-ordered, grouped or stratified holdout. Reports ROC-AUC, PR-AUC, log loss, Brier, balanced accuracy, MAE, RMSE, R2 and WAPE side by side, and warns when a first baseline looks too good (leakage).
ml-feasibility-assessor
Decide GO, GO WITH RISKS or NO-GO for an ML project before building it: enough labelled rows and minority examples, events per feature, label quality and coverage, history length and drift, single-feature signal (out-of-fold AUC / Spearman) and suspicious near-perfect predictors, data availability at prediction time, latency and serving constraints, the value ceiling given action capacity, and the cost of being wrong. Includes sample-size arithmetic for metric confidence intervals, AUC standard errors (Hanley-McNeil), McNemar test sizes and data collection time.
ml-model-comparison-reporter
Compare machine learning models honestly and write the comparison report: per-fold CV scores analysed with the corrected resampled t-test (Nadeau-Bengio), Wilcoxon signed-rank, Bayesian correlated t-test with a region of practical equivalence, and the Friedman test with average ranks for many models; single-test-set comparisons with the DeLong test for ROC-AUC, McNemar's test on decisions (threshold or top-K), and paired stratified bootstrap CIs for PR-AUC, log loss, Brier, MAE and RMSE; plus practical significance, multiple-comparison control, cost/latency trade-offs and a decision rule. Pure-Python implementations validated against scipy and a brute-force DeLong.
ml-problem-framing-advisor
Turn a vague business ask ("use AI to reduce churn", "predict demand", "find fraud") into a precise, buildable ML problem: the decision and action the prediction drives, who acts and with what capacity, the unit and moment of prediction, the label with its observation and outcome windows, label delay, features available at prediction time, task type (binary, multiclass, ordinal, regression, count, forecasting, ranking, anomaly, clustering, uplift), the baseline to beat and the metric at the operating point. Flags leakage-by-framing, proxy labels, feedback loops, selection bias and unactionable predictions.
ml-project-scoping-writer
Write the scoping document (one-pager, PRD section or project charter) for a machine learning project: summary, the decision and its users, success criteria with a primary metric at the operating point, target, baseline, guardrails and online confirmation, data sources with availability at prediction time and the label definition, a phased simplest-first approach, explicit in-scope and out-of-scope lists, milestones with dates and exit criteria, risks with mitigations (leakage, labels, feedback loops, adoption, drift), team and cost, go/no-go kill criteria and open questions. Generates the Markdown from a JSON spec with visible TODOs and lints existing scope docs for missing sections and vague commitments ("leverage AI", "real-time", "state-of-the-art", metrics with no target).
ml-success-metric-selector
Choose the metric that defines success for an ML model, and the guardrails around it, from the decision it serves: precision/recall@K for capacity-bound actions, expected cost at the cost-optimal threshold for asymmetric errors, PR-AUC vs ROC-AUC for rare events, log loss/Brier/ECE when probabilities are consumed, macro- vs weighted-F1 for multiclass, MAE vs RMSE vs WAPE vs MAPE vs Poisson deviance for regression, MASE/RMSSE and pinball loss for forecasts, NDCG/MRR/recall@k for ranking and recommendation, concordance for survival, Qini/AUUC for uplift. Computes the full metric panel from a predictions CSV (scikit-learn-identical numbers) and finds the cost-minimising threshold.
ml-vs-rules-decision-advisor
Decide whether a problem should be solved with hand-written rules, machine learning, a hybrid of both, or not yet: score twelve factors (labelled data volume, label availability, signal complexity, change rate, rule maintenance pain, scale, error cost, explainability and regulation, latency, team ML maturity, personalisation, feedback speed) with hard gates, learn a shallow readable rule set from a CSV with pure-Python CART and compare its holdout AUC with the ML model, emit the rule as a deployable SQL CASE, and design hybrids (rules as filters, guardrails, model score inside rules, ML-suggested rules).
model-card-writer
Write model cards that hold up to review: model details, intended and out-of-scope uses, training and evaluation data with periods and label definitions, overall metrics with confidence intervals, the baseline and decision threshold, performance disaggregated by group or segment with bootstrap intervals, fairness gaps (TPR/equal opportunity, FPR, positive rate and the four-fifths ratio), explainability, limitations, ethical considerations, risks with mitigations, and monitoring. Computes slice metrics from a predictions file, generates the card from a JSON spec with visible TODOs for missing sections, and lints existing cards for missing sections, placeholders, metrics without numbers or uncertainty, missing subgroup results and overclaiming language. Follows Mitchell et al. (2019) Model Cards and the Hugging Face card layout.
optuna-study-writer
Write correct, resumable Optuna hyperparameter studies for scikit-learn, LightGBM, XGBoost and CatBoost: an objective with a search space sized to the data, fixed CV folds built once (stratified, grouped or time-ordered), early stopping inside each fold, trial.report and should_prune for MedianPruner / HyperbandPruner, seeded multivariate TPESampler, SQLite storage with study_name and load_if_exists for resuming, stale-study stopping callbacks, user attributes (best iteration, fold std), trials export, parameter importances, conditional spaces, multi-objective studies, and refitting the best configuration with a lower learning rate. Generates verified study scripts for six model families and lints Optuna code: unprunable objectives, inconsistent parameters, per-trial folds, wrong direction, constant-returning handlers, missing seeds or storage, deprecated APIs.
overfitting-underfitting-diagnoser
Diagnose overfitting and underfitting from the evidence a model leaves behind: training histories (metric per boosting round or epoch from LightGBM, XGBoost, CatBoost, Keras or PyTorch) and validation curves (train and validation score across one hyperparameter such as max_depth, C, min_samples_leaf or alpha). Finds the best iteration on a smoothed curve, the overfitting onset, degradation and noise, and suggests early-stopping patience; flags under-training, plateaus, divergence and models that are not learning. For validation curves it labels underfit, sweet-spot and overfit zones, applies the one-standard-error rule to pick the simplest adequate value, infers complexity direction, and warns when the best value sits at the edge of the sweep. Includes the fix for each diagnosis per model family.
partial-dependence-plot-builder
Build and read partial dependence plots (PDP), individual conditional expectation (ICE) curves and accumulated local effects (ALE) for tabular models: generate scikit-learn partial_dependence scripts with percentile grids, the right response method and class, integer-safe columns, centred ICE with the PDP and a rug of observed values, two-way PDP heatmaps, and ALE that stays correct when features are correlated; then analyse the exported curves for direction, monotonicity, threshold-like jumps, heterogeneity, opposite effects or subgroups of different strength hidden by the average, the feature that separates them (interaction discovery), and grid regions without data support.
pca-dimensionality-reducer
Apply PCA correctly for dimensionality reduction, visualisation, decorrelation, compression and monitoring: standardise first (or justify covariance PCA), choose the number of components with cumulative explained variance, scree/elbow, Kaiser's rule and parallel analysis, interpret components through signed loadings, use PCA inside scikit-learn pipelines without leakage (PCA, IncrementalPCA, TruncatedSVD for sparse/text data, KernelPCA, whitening, randomized solver), and detect multivariate outliers and process faults with Hotelling's T² and Q (SPE) residuals and per-feature contributions. Pure-Python PCA report and T²/Q outlier scripts validated against scikit-learn.
recommendation-system-designer
Design recommendation systems from the interaction data up: profile users, items, density, cold start, popularity concentration (Gini, long-tail shares), explicit versus implicit feedback and repeats; choose between popularity, item-item kNN, matrix factorisation (ALS, BPR, EASE), content-based, hybrid, two-tower retrieval and learning-to-rank re-ranking; split by time (leave-last-n or global cut-off) instead of randomly; and evaluate top-k lists with precision, recall, hit rate, MRR, MAP and NDCG plus coverage, novelty, popularity bias, personalisation and diversity against a popularity baseline that is always computed. Covers candidate-generation and ranking architecture, cold-start strategies, business rules, online A/B metrics and feedback loops.
regression-model-trainer
Train regression models whose loss matches the target: squared error, absolute/Huber for heavy tails, log1p targets with retransformation (smearing) correction, Gamma for positive skewed amounts, Poisson for counts with exposure offsets, Tweedie for zero-inflated losses, two-part hurdle models, logit/cross-entropy for proportions, and quantile (pinball) loss and conformal prediction intervals for percentiles and uncertainty. Covers ridge/lasso/GLMs (PoissonRegressor, GammaRegressor, TweedieRegressor), HistGradientBoosting, LightGBM, XGBoost and CatBoost objectives, monotone constraints, sample weights and exposure, and residual diagnostics: calibration by prediction decile, heteroscedasticity, segment bias, interval coverage and worst errors with CI gates.
scikit-learn-pipeline-builder
Build correct, production-shaped scikit-learn pipelines: Pipeline and ColumnTransformer per column role (impute + scale, one-hot with infrequent grouping, cross-fitted TargetEncoder, native categoricals for HistGradientBoosting, TF-IDF + SVD for text, date transformers), the right CV splitter (StratifiedKFold, StratifiedGroupKFold, TimeSeriesSplit with gap), cross_validate with several metrics, RandomizedSearchCV / HalvingRandomSearchCV with step__param names, nested CV, custom transformers with get_feature_names_out, set_output, caching, joblib/skops persistence and a predict helper. Generates a runnable training script from a CSV and lints code for API misuse (hard labels into roc_auc_score, removed parameters, LabelEncoder on features, get_dummies, unpicklable lambdas, scoring on training data, missing random_state, nested n_jobs).
shap-value-explainer
Compute and explain SHAP values for tabular models correctly: choose the explainer (TreeExplainer interventional or tree_path_dependent, CatBoost native ShapValues, LinearExplainer, model-agnostic PermutationExplainer), pick the background data and the output space (log-odds versus probability), explain the final estimator of a scikit-learn Pipeline and sum one-hot columns back to their input feature, and check additivity. Summarise global importance (mean |SHAP| and share), direction (Spearman of value versus SHAP), how much of each effect depends on other features and the likely interaction partner, and the effect of missing values; turn one prediction into a text waterfall and a plain-language summary with probabilities. Covers pitfalls: correlated features sharing credit, causal misreadings, background choice, multiclass outputs.
survival-analysis-model-builder
Build time-to-event (survival) models that handle censoring correctly: Kaplan-Meier curves with Greenwood log-log confidence intervals, median survival with CIs, survival at chosen times, restricted mean survival time and the log-rank test; Cox proportional hazards with Efron ties, hazard ratios, concordance and the Grambsch-Therneau proportional-hazards test; and a generated comparison of penalised Cox, Weibull AFT, random survival forest and gradient-boosted survival models with Harrell's and Uno's C, time-dependent AUC, integrated Brier score and calibration. Covers churn, customer lifetime, equipment failure, credit default timing and clinical endpoints, censoring pitfalls, crossing curves, competing risks and time-varying covariates.
target-leakage-detector
Find and fix target leakage, the reason models score 0.99 offline and fail in production: features written after the prediction time (status, cancellation reason, refunds, chargebacks, closed dates), missingness patterns filled by the outcome, future timestamps, aggregates over all history, identifiers or row order that encode the label, target-derived columns, in-sample target encoding, and pipeline leaks in code (scaler/imputer/encoder/selector fitted before the split, SMOTE before the split, fitting on test data, early stopping on the test set, groupby target means, shuffled splits on temporal data). Scans a training table with out-of-fold single-feature AUC, is-missing AUC, future-date checks and period-stability checks, and lints Python training code with an AST linter.
time-based-feature-builder
Build leakage-safe time-based features from event logs: point-in-time window aggregates (counts, sums, means, max, distinct counts over 1h/24h/7d/30d) strictly before each cutoff, recency (time since last event), frequency and monetary (RFM), velocity and burst ratios, trends (short vs long window), expanding per-entity statistics, lags and rolling statistics for forecasting, EWMA, time-since-milestone, calendar and holiday features, and data-latency offsets. Supports snapshot spines (one row per entity per scoring date) and event-level scoring (each event scored on its own history), and generates matching SQL for PostgreSQL, DuckDB, BigQuery and Snowflake.
time-series-forecasting-model-builder
Build forecasting models that beat honest baselines: profile the series (frequency, gaps, duplicate timestamps, seasonality by autocorrelation, trend and seasonal strength, weekday and month effects, outliers, smooth/erratic/intermittent/lumpy demand by ADI and CV^2), backtest seasonal naive, drift, SES, Holt-Winters, Croston, SBA and TSB on rolling origins with MASE, RMSSE, WAPE and bias, score your own models' forecasts on the same origins, and generate a LightGBM forecaster with leak-free direct-strategy lags (every lag at least the horizon), rolling and calendar features, optional linear detrending (trees cannot extrapolate trend), multi-series support and split-conformal prediction intervals. Covers statsforecast/statsmodels ETS and ARIMA, global models, hierarchies and intermittent demand.
train-test-split-auditor
Audit a train/test (or train/validation, train/production) split before trusting test scores: exact duplicate rows across sets, entity leakage (the same customer, patient, device or session on both sides), time overlap in forward splits, near-duplicates (rounded numerics, case/whitespace variants, edited text copies via MinHash and LSH), feature drift (PSI, KS), single features that separate train from test (univariate adversarial AUC), target-distribution shift, unseen categories and missing-value differences; plus how to build a correct split (random, stratified, grouped, time-based, out-of-distribution) and how to run full adversarial validation.
umap-tsne-embedding-visualizer
Make and read 2-D maps of high-dimensional data (tabular features, embeddings from text/image models, single-cell style data) with UMAP, t-SNE and PCA without being misled: scaling or L2 normalisation, PCA pre-reduction to ~50 dimensions, UMAP n_neighbors/min_dist/metric and t-SNE perplexity/early exaggeration/learning rate/init choices, multiple seeds and parameter grids, colouring by labels or numeric values, transforming new points, and quantitative faithfulness checks: trustworthiness, continuity, k-NN preservation, pairwise-distance correlation and label k-NN accuracy in the original space vs the map. Explains what maps can and cannot show (cluster sizes, inter-cluster distances, density and outliers are unreliable).
uplift-model-builder
Build uplift (incremental response, heterogeneous treatment effect) models that target the people a campaign, offer or intervention actually changes: validate the randomised experiment (sample-ratio mismatch, covariate balance by standardised mean difference, average treatment effect with confidence interval and minimum detectable effect, segment effects with a heterogeneity test), train S-, T- and X-learners and the class-transformation approach, and evaluate on a holdout with Qini coefficient and AUUC (matching scikit-uplift), uplift by decile with confidence intervals, sleeping-dog detection and the inverse-propensity-weighted value and profit of treating the top fraction. Covers persuadables, sure things and sleeping dogs, observational data, causal forests and rollout.
xgboost-lightgbm-tuner
Tune XGBoost and LightGBM models efficiently and read their training curves: validation design and early stopping inside CV, fixed learning rate for search then lower rate with more trees for the final fit, capacity knobs (num_leaves, max_depth, min_child_samples, min_child_weight) sized to the data, subsampling, L1/L2 and split-gain regularisation, categorical handling, imbalance (scale_pos_weight vs is_unbalance vs no weighting), monotone and interaction constraints, Optuna/TPE search with fold-level pruning and parameter importances, and the LightGBM <-> XGBoost <-> CatBoost parameter map. Parses LightGBM, XGBoost and CatBoost logs or CSV histories to find the best iteration, overfitting onset, train/validation gap and whether more rounds or a lower learning rate are needed.