LIME explains one prediction by sampling perturbations around the row, querying the model, and fitting a
weighted linear surrogate. It works with any model, but the explanation depends on the random sample, the
kernel width and how continuous features are discretised. A single LIME run is an anecdote.
scripts/gen_lime_script.py writes a script that runs LIME correctly and repeatedly, and
scripts/lime_stability.py decides whether the result is stable and faithful enough to report.
Trigger Conditions
Activate this skill when the user:
- Asks for LIME explanations of individual predictions
- Gets a different LIME explanation each time they run it
- Needs a local explanation for a model SHAP cannot explain efficiently (external API, ensemble of mixed models)
- Wants rule-like explanations ("support_calls > 3 raises churn probability")
For tree and linear models, SHAP is usually the better default: exact, additive and deterministic (shap-value-explainer). Counterfactuals ("what would change the decision"): counterfactual-explanation-generator.
Workflow
1. GENERATE -> python scripts/gen_lime_script.py --model m.joblib --train train.csv --rows test.csv --target y --row 0 5 -o lime_run.py
2. RUN -> python lime_run.py (lime_runs_row<N>.csv, lime_row<N>.html)
3. CHECK -> python scripts/lime_stability.py lime_runs_row0.csv --compare shap_row0.csv
4. TUNE -> unstable: more samples / fixed kernel width; low fidelity: --no-discretize, smaller kernel width
5. REPORT -> stable features only, mean weights, fidelity, the comparison with SHAPSettings That Change the Answer
| Setting | Effect | Guidance |
|---|---|---|
num_samples (--samples) | more samples, lower variance | 5000 by default; raise to 10-20k when unstable |
kernel_width | the size of the "local" neighbourhood | default 0.75 x sqrt(n_features); smaller = more local, noisier |
discretize_continuous | quartile rules vs continuous slopes | rules are readable; slopes fit smooth models better (--no-discretize) |
| training data | perturbation statistics and category frequencies | use the training set, not the test rows |
| seed | which perturbations are drawn | repeat with several seeds; never report one run |
num_features (--top) | how many features the surrogate keeps | 5-10 |
Measured on the churn fixture (gen_lime_script.py + lime_stability.py): mean surrogate R^2 for the
same row went from 0.05 to 0.78 (logistic regression) and from 0.05 to 0.42 (LightGBM) when switching to
--no-discretize. The default quartile rules can be reproducible yet describe the model poorly. Details
and the algorithm: lime-method.md.
Stability and Fidelity
| Check | Rule in lime_stability.py | If it fails |
|---|---|---|
| sign consistency | >= 90% of runs agree on the sign | do not report that feature's direction |
| size noise | coefficient of variation <= 0.5 | report the direction, not the size |
| top-k membership | in the top-k in >= 80% of runs | the feature is borderline |
| top-k Jaccard | mean pairwise overlap >= 0.7 (negligible features ignored) | raise samples, fix the kernel width |
| fidelity | weighted R^2 of the surrogate >= 0.5 | try --no-discretize, a smaller kernel width, or SHAP |
| local gap | surrogate vs model prediction at the row | a large gap means the surrogate misses the model here |
| agreement | sign agreement with SHAP for the same row | investigate disagreeing features |
Verdicts: STABLE, STABLE BUT LOW FIDELITY and UNSTABLE. Interpretation and troubleshooting: stability-and-fidelity.md.
Reporting LIME
- Report only features marked stable, with their mean weight over runs and the number of runs.
- State the settings: samples, kernel width, discretisation, training data, and fidelity.
- LIME weights are slopes of a local surrogate, in output units per rule (or per unit for continuous features). They do not sum to the prediction the way SHAP values do.
- Rules like "0 < support_calls <= 1" describe the bin the row falls into relative to the others, not a threshold the model uses.
- A local explanation describes one row. Do not generalise it to the model.
Wording templates and when to prefer other methods: reporting-lime.md.
Output Format
When explaining a prediction with LIME, deliver:
- Setup - model, row, training data, samples, kernel width, discretisation, repeats.
- Explanation - the stable features with mean weights (and std), as rules or slopes.
- Quality - stability verdict, top-k Jaccard, fidelity (R^2), local gap, agreement with SHAP.
- Caveats - unstable features left out, local scope, non-causal reading.
Scripts
python scripts/gen_lime_script.py --model model.joblib --train train.csv --rows test.csv --target churned --row 0 5 -o lime_run.py
python scripts/gen_lime_script.py --model pipe.joblib --train train.csv --rows test.csv --target price --row 3 --samples 10000
python scripts/gen_lime_script.py --model m.joblib --train train.csv --rows test.csv --target y --no-discretize --kernel-width 2.0 --repeats 20Encodes categoricals for LIME and decodes them inside predict_fn, keeps integer dtypes, fills missing
numerics with training medians for LIME only, and explains the probability of --class (default: the last
class) or the regression output. Each repeat uses a new seed, and each row's runs go to one CSV.
Verified with the lime package on logistic-regression and LightGBM Pipelines (classification) and a
random-forest Pipeline (regression). Exit codes: 0 written, 2 usage error.
python scripts/lime_stability.py lime_runs_row0.csv
python scripts/lime_stability.py lime_runs_row0.csv --top-k 3 --compare shap_row0.csv --jsonExit codes: 0 stable, 1 unstable, 2 usage or read error. Low fidelity is reported in the verdict.
Additional Resources
| File | Content |
|---|---|
| lime-method.md | the algorithm, perturbation of tabular data, kernel, discretisation, settings, library calls |
| stability-and-fidelity.md | why LIME varies, the checks, troubleshooting, comparing with SHAP |
| reporting-lime.md | wording, what LIME weights mean, when to choose LIME, SHAP or counterfactuals |
anomaly-detection-model-builder
Build and evaluate anomaly detection for tabular, sensor and transaction data with few or no labels: define the anomaly type (point, contextual, collective), choose detectors (robust z-score/MAD and IQR rules, Mahalanobis with robust covariance, k-NN distance, LOF, Isolation Forest, one-class SVM, PCA reconstruction / autoencoders, forecasting residuals for time series), combine them by rank, explain each alert by its deviating features, set thresholds from an alert budget or the score distribution, and evaluate with precision@k, PR-AUC and recall on known incidents, or detector agreement when no labels exist. Includes pure-Python detectors (robust z, Mahalanobis, kNN, Isolation Forest, ensemble) and an evaluation script.
catboost-model-builder
Build CatBoost models that use its strengths correctly: raw categorical columns as cat_features (ordered target statistics, no manual encoding), text_features, Pool construction with NaN-safe categories, ordered boosting and has_time for temporal data, symmetric (oblivious) trees, overfitting detector with eval_set and use_best_model, losses for classification, regression (Quantile, Poisson, Tweedie) and ranking (YetiRank), class weights, monotone constraints, feature importance (PredictionValuesChange, LossFunctionChange) and SHAP values, GPU training and model export. Plans cat/text/numeric columns from a CSV and generates a runnable training script, and checks parameter sets for the conflicts CatBoost rejects (iterations vs n_estimators, Bayesian bootstrap with subsample, multiple class-weight options, depth > 16, use_best_model without eval_set).
categorical-encoding-selector
Choose and implement the right encoding for every categorical column given the model family and the data: one-hot (with infrequent-level grouping and handle_unknown), ordinal with an explicit order, cross-fitted target encoding (scikit-learn TargetEncoder, CatBoost ordered statistics, category_encoders inside CV), frequency/count encoding, feature hashing, native categorical support in LightGBM, HistGradientBoosting, XGBoost and CatBoost, prefix levels for codes (zip3, NAICS-2), and embeddings. Measures cardinality, rare-level share, unseen-level rate on a time-ordered holdout and target signal per column, emits the ColumnTransformer, and quantifies target-encoding leakage (in-sample vs out-of-fold vs holdout AUC).
classification-model-selector
Choose the right classification algorithm for tabular, text or mixed data and prove the choice with a fair bake-off: logistic regression (L2, L1, elastic net), linear SVM, naive Bayes, k-NN, RBF SVM, decision trees, random forest, extra trees, HistGradientBoosting, LightGBM, XGBoost, CatBoost, explainable boosting machines (EBM/GAM), points scorecards and MLPs, matched to data size, width, categorical and high-cardinality share, missingness, class imbalance, text, and constraints such as interpretability, monotonic constraints, calibrated probabilities, latency budgets and tiny deployable models. Ranks candidates with starting hyperparameters and cautions, then generates a bake-off script running every candidate on identical CV folds with per-fold metrics, fit time and prediction latency.
clustering-algorithm-selector
Choose the right clustering algorithm and check that clustering makes sense at all: cluster tendency (Hopkins statistic with a correlated-features caveat, nearest-neighbour isolation), then k-means / MiniBatchKMeans, Gaussian mixtures with BIC, agglomerative (Ward, average, complete, single) with dendrograms, DBSCAN, HDBSCAN, OPTICS, spectral clustering, BIRCH, mean shift, k-prototypes and k-modes for mixed or categorical data, and Gower distance, matched to data size, expected shapes, noise, varying density, known k, soft memberships, hierarchy and streaming. Covers preprocessing (scaling, encoding, dimensionality, distance metrics), validation (internal indices, bootstrap stability, external labels, business profiling) and assigning new points.
collaborative-filtering-implementer
Implement collaborative filtering recommenders that beat popularity: item-based kNN with cosine, Jaccard or asymmetric similarity, shrinkage, neighbourhood size, positive-only or mean-centred rating signals and 'because you liked X' explanations; implicit ALS with confidence weighting, BPR pairwise ranking and the closed-form EASE model in plain NumPy, tuned on a temporal validation split by recall@k and NDCG@k and refit on all data; plus user-kNN, explicit matrix factorisation, the implicit/LightFM libraries, hyperparameter ranges, cold-start fallbacks, scaling with sparse matrices and approximate nearest neighbours. Produces top-k lists in the format of the recommendation evaluator.