Skip to content
v0.1.3latest release121 skills4 categories6 sub-categories247 tested scripts397 reference docsContact us
Vector Agentic Skills
Submit a skill

Classifier Profiles

Reference for classification-model-selector · 94 lines

AI & Machine Learning / Classical Machine Learningv1.0.0Updated OCT 25 20264 references2 scripts

Logistic regression

Linear log-odds model fitted by maximum likelihood with a penalty.

  • Key params: C (inverse regularisation, tune on a log grid 1e-3..1e2), penalty ("l2", "l1", "elasticnet" with l1_ratio), solver (lbfgs default; saga for L1/elastic net and large sparse data; liblinear for small L1), class_weight, max_iter (raise to 2000+).
  • Needs: scaled numerics, one-hot categoricals, explicit non-linearity (splines, bins, interactions).
  • Fails when: effects are strongly non-linear or interacting and nobody engineered features for it.
  • Bonus: good probabilities without class weights; coefficients map to odds ratios.

Linear SVM

Maximum-margin linear classifier (hinge loss).

  • Key params: C, loss (hinge / squared hinge), dual="auto".
  • Use: text and very wide sparse data. Wrap in CalibratedClassifierCV for probabilities.

Naive Bayes

Per-feature likelihoods combined with Bayes' rule.

  • MultinomialNB / ComplementNB for counts and TF-IDF; GaussianNB for continuous features; alpha smoothing. Instant, decent text baseline; probabilities are overconfident.

k-nearest neighbours

Vote among the k closest training rows.

  • Key params: n_neighbors (15-50 for probabilities), weights="distance", metric.
  • Needs: scaling, few features, small/medium data (prediction cost grows with n).

SVM with RBF kernel

  • Key params: C, gamma ("scale"), tune jointly on log grids.
  • Use: small dense data (< ~20k rows). Training is O(n²)-O(n³). probability=True runs internal CV (slow); prefer CalibratedClassifierCV.

Decision tree

Recursive axis-aligned splits.

  • Key params: max_depth, min_samples_leaf, ccp_alpha (cost-complexity pruning).
  • Use: readable rules, surrogates, SQL deployment. High variance alone.

Random forest

Bagged deep trees with random feature subsets.

  • Key params: n_estimators (300-1000; more never hurts accuracy, only cost), max_features ("sqrt", or 0.2-0.5), min_samples_leaf (1-10), max_samples (bootstrap fraction), class_weight="balanced_subsample".
  • Strengths: robust defaults, out-of-bag estimate (oob_score=True).
  • Weaknesses: probabilities squashed toward 0.5 (calibrate), large models, impurity importances biased.

Extra trees

Like RF but with random split thresholds: faster, often similar accuracy, smoother probability surface.

HistGradientBoosting (scikit-learn)

Histogram-based gradient boosting (LightGBM-like) in scikit-learn.

  • Key params: learning_rate (0.03-0.1), max_leaf_nodes (15-63), min_samples_leaf (20-200), l2_regularization, max_iter with early_stopping=True, categorical_features, monotonic_cst, class_weight.
  • Strengths: native NaN and categoricals (< 256 levels), no extra dependency.

LightGBM

Leaf-wise histogram GBM.

  • Key params: learning_rate, num_leaves (main capacity knob), min_child_samples (raise for small or noisy data), feature_fraction/colsample_bytree, bagging_fraction/subsample + bagging_freq, lambda_l1/l2, n_estimators with early stopping, categorical_feature, monotone_constraints.
  • Fails when: small data with defaults (leaf-wise growth overfits): increase min_child_samples, lower num_leaves.

XGBoost

Level-wise (default) or loss-guided histogram GBM.

  • Key params: max_depth (4-8), min_child_weight, eta/learning_rate, subsample, colsample_bytree, gamma, reg_lambda, reg_alpha, tree_method="hist", enable_categorical, monotone_constraints, scale_pos_weight.

CatBoost

Oblivious (symmetric) trees with ordered boosting and ordered target statistics for categoricals.

  • Key params: iterations, learning_rate, depth (4-8), l2_leaf_reg, cat_features, one_hot_max_size, auto_class_weights, od_type/od_wait (early stopping).
  • Strengths: excellent defaults, categoricals without preprocessing, fast inference (symmetric trees).

Explainable boosting machine (EBM)

A GAM learned by cyclic boosting of one feature at a time, plus a few pairwise interactions.

  • Key params: interactions (number of pairs), max_bins, outer_bags, learning_rate.
  • Strengths: exact per-feature contribution curves; editable; competitive accuracy on many tabular tasks.

MLP

Feed-forward neural network.

  • Key params: layer sizes, alpha (L2), learning_rate_init, early_stopping=True.
  • Use: large data, combined with embeddings; otherwise rarely the best tabular choice.

Rough expectations (tabular, medium data)

ModelAccuracy rankTraining speedInference speedTuning effort
Tuned GBM (LightGBM / XGBoost / CatBoost)1fastfastmedium
Default CatBoost / HistGBM2medium / fastfastlow
EBM2-3slowfastlow
Random forest3mediumslow-mediumlow
Logistic regression with good features3-4very fastvery fastlow
k-NN / SVM4-5variesslowmedium
More in Classical Machine Learning

anomaly-detection-model-builder

Build and evaluate anomaly detection for tabular, sensor and transaction data with few or no labels: define the anomaly type (point, contextual, collective), choose detectors (robust z-score/MAD and IQR rules, Mahalanobis with robust covariance, k-NN distance, LOF, Isolation Forest, one-class SVM, PCA reconstruction / autoencoders, forecasting residuals for time series), combine them by rank, explain each alert by its deviating features, set thresholds from an alert budget or the score distribution, and evaluate with precision@k, PR-AUC and recall on known incidents, or detector agreement when no labels exist. Includes pure-Python detectors (robust z, Mahalanobis, kNN, Isolation Forest, ensemble) and an evaluation script.

catboost-model-builder

Build CatBoost models that use its strengths correctly: raw categorical columns as cat_features (ordered target statistics, no manual encoding), text_features, Pool construction with NaN-safe categories, ordered boosting and has_time for temporal data, symmetric (oblivious) trees, overfitting detector with eval_set and use_best_model, losses for classification, regression (Quantile, Poisson, Tweedie) and ranking (YetiRank), class weights, monotone constraints, feature importance (PredictionValuesChange, LossFunctionChange) and SHAP values, GPU training and model export. Plans cat/text/numeric columns from a CSV and generates a runnable training script, and checks parameter sets for the conflicts CatBoost rejects (iterations vs n_estimators, Bayesian bootstrap with subsample, multiple class-weight options, depth > 16, use_best_model without eval_set).

categorical-encoding-selector

Choose and implement the right encoding for every categorical column given the model family and the data: one-hot (with infrequent-level grouping and handle_unknown), ordinal with an explicit order, cross-fitted target encoding (scikit-learn TargetEncoder, CatBoost ordered statistics, category_encoders inside CV), frequency/count encoding, feature hashing, native categorical support in LightGBM, HistGradientBoosting, XGBoost and CatBoost, prefix levels for codes (zip3, NAICS-2), and embeddings. Measures cardinality, rare-level share, unseen-level rate on a time-ordered holdout and target signal per column, emits the ColumnTransformer, and quantifies target-encoding leakage (in-sample vs out-of-fold vs holdout AUC).

clustering-algorithm-selector

Choose the right clustering algorithm and check that clustering makes sense at all: cluster tendency (Hopkins statistic with a correlated-features caveat, nearest-neighbour isolation), then k-means / MiniBatchKMeans, Gaussian mixtures with BIC, agglomerative (Ward, average, complete, single) with dendrograms, DBSCAN, HDBSCAN, OPTICS, spectral clustering, BIRCH, mean shift, k-prototypes and k-modes for mixed or categorical data, and Gower distance, matched to data size, expected shapes, noise, varying density, known k, soft memberships, hierarchy and streaming. Covers preprocessing (scaling, encoding, dimensionality, distance metrics), validation (internal indices, bootstrap stability, external labels, business profiling) and assigning new points.

collaborative-filtering-implementer

Implement collaborative filtering recommenders that beat popularity: item-based kNN with cosine, Jaccard or asymmetric similarity, shrinkage, neighbourhood size, positive-only or mean-centred rating signals and 'because you liked X' explanations; implicit ALS with confidence weighting, BPR pairwise ranking and the closed-form EASE model in plain NumPy, tuned on a temporal validation split by recall@k and NDCG@k and refit on all data; plus user-kNN, explicit matrix factorisation, the implicit/LightFM libraries, hyperparameter ranges, cold-start fallbacks, scaling with sparse matrices and approximate nearest neighbours. Produces top-k lists in the format of the recommendation evaluator.

counterfactual-explanation-generator

Generate counterfactual explanations and algorithmic recourse: the smallest realistic change to a row's features that changes the model's decision. Solves exact minimal-cost recourse for linear and logistic models (MAD-scaled L1 cost, immutable features, increase/decrease directions, bounds, integer and binary features, diverse alternatives, features every counterfactual must change, and a no-recourse finding when immutable features decide), generates DiCE counterfactuals for any scikit-learn model with constraints turned into permitted ranges and every result re-predicted, and validates counterfactuals from any method for constraint violations, implausible values, off-manifold rows, broken derived-feature relations and failure to flip the prediction.