Ordered target statistics (how categoricals are encoded)
Naive target encoding uses each row's own label (leakage). CatBoost instead:
- draws a random permutation of the training rows (several, for different trees);
- encodes row i's category using only rows before i in the permutation:
ctr = (count_positive_before + prior) / (count_before + 1); - at prediction time uses statistics from all training rows.
This is the "CTR" (click-through-rate-style) feature. Several CTR types exist (Borders, Buckets,
Counter for frequency, BinarizedTargetMeanValue for regression); defaults are sensible.
simple_ctr, combinations_ctr and per_feature_ctr configure them.
Feature combinations
When a tree splits on a categorical CTR, CatBoost greedily adds combinations of categoricals (and binarised
numerics) for later splits, up to max_ctr_complexity features per combination (default 4). This captures
interactions like plan x region without manual crosses. Lower max_ctr_complexity (1-2) speeds training
on many categoricals.
Ordered boosting
Classic boosting computes gradients for a row with a model that was trained on that row (prediction shift).
CatBoost's ordered boosting (boosting_type="Ordered") uses models trained on preceding rows only. It costs
time and is the default only for small datasets; "Plain" is used otherwise. The ordered target statistics
apply in both modes.
Symmetric (oblivious) trees
Default grow_policy="SymmetricTree": every node at a depth uses the same split. Benefits: strong
regularisation, very fast inference (a depth-d tree is a lookup with d comparisons), good with
depth 4-8. Depthwise and Lossguide grow ordinary trees (more flexible, slower inference).
Text features
text_features=[...]: CatBoost tokenises (by default lowercase, split on space), builds dictionaries, and
computes estimators such as BoW (bag of words), NaiveBayes and BM25 per class. Configure with
text_processing (tokenizers, dictionaries, feature calcers). Strong enough for short text fields
(titles, descriptions, ticket bodies); for semantic tasks precompute sentence embeddings.
Embedding features
embedding_features=[...] for columns that hold vectors (numpy arrays). Estimators: LDA and KNN
(class-conditional). Useful for precomputed text/image embeddings combined with tabular data.
Pool
from catboost import Pool
pool = Pool(data=X, label=y, cat_features=CAT, text_features=TEXT, embedding_features=EMB,
weight=w, group_id=query_ids, baseline=offset, timestamp=ts)group_idfor ranking (rows of a group must be contiguous);baselinefor offsets (exposure in Poisson/Tweedie);weightfor sample weights;timestampwithhas_time=Truefor time-ordered processing.
Missing values
Numeric NaN: nan_mode="Min" (treated as smaller than all values) or "Max" / "Forbidden".
Categorical NaN: not allowed; convert to a string token.
anomaly-detection-model-builder
Build and evaluate anomaly detection for tabular, sensor and transaction data with few or no labels: define the anomaly type (point, contextual, collective), choose detectors (robust z-score/MAD and IQR rules, Mahalanobis with robust covariance, k-NN distance, LOF, Isolation Forest, one-class SVM, PCA reconstruction / autoencoders, forecasting residuals for time series), combine them by rank, explain each alert by its deviating features, set thresholds from an alert budget or the score distribution, and evaluate with precision@k, PR-AUC and recall on known incidents, or detector agreement when no labels exist. Includes pure-Python detectors (robust z, Mahalanobis, kNN, Isolation Forest, ensemble) and an evaluation script.
categorical-encoding-selector
Choose and implement the right encoding for every categorical column given the model family and the data: one-hot (with infrequent-level grouping and handle_unknown), ordinal with an explicit order, cross-fitted target encoding (scikit-learn TargetEncoder, CatBoost ordered statistics, category_encoders inside CV), frequency/count encoding, feature hashing, native categorical support in LightGBM, HistGradientBoosting, XGBoost and CatBoost, prefix levels for codes (zip3, NAICS-2), and embeddings. Measures cardinality, rare-level share, unseen-level rate on a time-ordered holdout and target signal per column, emits the ColumnTransformer, and quantifies target-encoding leakage (in-sample vs out-of-fold vs holdout AUC).
classification-model-selector
Choose the right classification algorithm for tabular, text or mixed data and prove the choice with a fair bake-off: logistic regression (L2, L1, elastic net), linear SVM, naive Bayes, k-NN, RBF SVM, decision trees, random forest, extra trees, HistGradientBoosting, LightGBM, XGBoost, CatBoost, explainable boosting machines (EBM/GAM), points scorecards and MLPs, matched to data size, width, categorical and high-cardinality share, missingness, class imbalance, text, and constraints such as interpretability, monotonic constraints, calibrated probabilities, latency budgets and tiny deployable models. Ranks candidates with starting hyperparameters and cautions, then generates a bake-off script running every candidate on identical CV folds with per-fold metrics, fit time and prediction latency.
clustering-algorithm-selector
Choose the right clustering algorithm and check that clustering makes sense at all: cluster tendency (Hopkins statistic with a correlated-features caveat, nearest-neighbour isolation), then k-means / MiniBatchKMeans, Gaussian mixtures with BIC, agglomerative (Ward, average, complete, single) with dendrograms, DBSCAN, HDBSCAN, OPTICS, spectral clustering, BIRCH, mean shift, k-prototypes and k-modes for mixed or categorical data, and Gower distance, matched to data size, expected shapes, noise, varying density, known k, soft memberships, hierarchy and streaming. Covers preprocessing (scaling, encoding, dimensionality, distance metrics), validation (internal indices, bootstrap stability, external labels, business profiling) and assigning new points.
collaborative-filtering-implementer
Implement collaborative filtering recommenders that beat popularity: item-based kNN with cosine, Jaccard or asymmetric similarity, shrinkage, neighbourhood size, positive-only or mean-centred rating signals and 'because you liked X' explanations; implicit ALS with confidence weighting, BPR pairwise ranking and the closed-form EASE model in plain NumPy, tuned on a temporal validation split by recall@k and NDCG@k and refit on all data; plus user-kNN, explicit matrix factorisation, the implicit/LightFM libraries, hyperparameter ranges, cold-start fallbacks, scaling with sparse matrices and approximate nearest neighbours. Produces top-k lists in the format of the recommendation evaluator.
counterfactual-explanation-generator
Generate counterfactual explanations and algorithmic recourse: the smallest realistic change to a row's features that changes the model's decision. Solves exact minimal-cost recourse for linear and logistic models (MAD-scaled L1 cost, immutable features, increase/decrease directions, bounds, integer and binary features, diverse alternatives, features every counterfactual must change, and a no-recourse finding when immutable features decide), generates DiCE counterfactuals for any scikit-learn model with constraints turned into permitted ranges and every result re-predicted, and validates counterfactuals from any method for constraint violations, implausible values, off-manifold rows, broken derived-feature relations and failure to flip the prediction.