Skip to content
v0.1.3latest release121 skills4 categories6 sub-categories247 tested scripts397 reference docsContact us
Vector Agentic Skills
Submit a skill

pca-dimensionality-reducer

Apply PCA correctly for dimensionality reduction, visualisation, decorrelation, compression and monitoring: standardise first (or justify covariance PCA), choose the number of components with cumulative explained variance, scree/elbow, Kaiser's rule and parallel analysis, interpret components through signed loadings, use PCA inside scikit-learn pipelines without leakage (PCA, IncrementalPCA, TruncatedSVD for sparse/text data, KernelPCA, whitening, randomized solver), and detect multivariate outliers and process faults with Hotelling's T² and Q (SPE) residuals and per-feature contributions. Pure-Python PCA report and T²/Q outlier scripts validated against scikit-learn. Use when someone asks how many principal components to keep, whether to scale before PCA, what a component means, wants to reduce features before modelling or clustering, or monitor sensors/processes with PCA.

AI & Machine Learning / Classical Machine Learningv1.0.0Updated OCT 25 20263 references2 scripts

PCA rotates the data onto orthogonal directions of maximum variance. It is the right tool for correlated numeric features: compressing them, decorrelating them for linear models, plotting high-dimensional data, and spotting rows that break the usual correlation pattern. It is the wrong tool when variance is not what matters (rare but important signals), for non-linear structure (UMAP/t-SNE for visualisation), or when features are on different scales and nobody standardised them. scripts/pca_report.py reports explained variance, four component-count rules and interpretable loadings, and warns about scaling; scripts/pca_outliers.py scores rows with Hotelling's T² and Q residuals.

Trigger Conditions

Activate this skill when the user:

  • Asks how many principal components to keep, or reads a scree plot
  • Asks whether to standardise before PCA, or why PC1 is "just one feature"
  • Wants to reduce many correlated features before regression, clustering or k-NN
  • Needs to interpret components (what does PC2 mean?)
  • Wants PCA on text/TF-IDF or other sparse data (TruncatedSVD) or on data that does not fit in memory
  • Monitors sensors, processes or KPIs for multivariate anomalies (T², SPE/Q charts)

Non-linear 2-D visualisation: umap-tsne-embedding-visualizer; choosing features by relevance to a target: feature-selection-analyzer; general anomaly detection: anomaly-detection-model-builder.

Workflow

text
1. SCALE     -> standardise (correlation PCA) unless all features share units and their variance matters
2. REPORT    -> python scripts/pca_report.py data.csv --drop id label
3. CHOOSE k  -> parallel analysis for "real" structure; cumulative variance (90-95%) for model inputs
4. INTERPRET -> top signed loadings per component; name components only when loadings are clear
5. USE       -> Pipeline([("scale", StandardScaler()), ("pca", PCA(k)), ("model", ...)]) fitted on train only
6. MONITOR   -> python scripts/pca_outliers.py --train reference.csv --score new.csv --k K   (T², Q)

Scale First (Usually)

DataPCA on
Features in different units (money, counts, days)correlation matrix: standardise first
Same units and variance is meaningful (pixel intensities, spectra, returns of similar assets)covariance matrix (no scaling)
Sparse counts / TF-IDFTruncatedSVD (no centring)

pca_report.py --no-scale on the wide fixture: revenue_usd (std ~1000) holds 100% of the variance, so PC1 is revenue. Standardised, the same data shows two real factors (PC1 46%, PC2 28%).

Choosing the Number of Components

RuleKeep componentsUse for
Cumulative varianceuntil 80/90/95%compression, model inputs (pick by downstream CV score if possible)
Scree / elbowbefore the curve flattensquick visual
Kaisereigenvalue > 1 (correlation PCA)rough; tends to over-keep with many features
Parallel analysis (Horn)eigenvalue above that of column-shuffled data"how many real dimensions?"; the most reliable rule
Downstream CVthe k that maximises the model's CV scorewhen PCA feeds a model

Wide fixture: 2 by parallel analysis (the two planted factors), 3 by Kaiser, 4 for 90%, 5 for 95%: different questions, different answers. Details and formulas: choosing-components.md.

Interpreting Components

text
PC1 (46.0%): f4 +0.45, f2 +0.45, f1 +0.45     -> a common "factor A" across f1-f5
PC2 (27.8%): f8 +0.58, f6 +0.58, f7 +0.57     -> "factor B" across f6-f8
PC3 (10.2%): f9 +0.83, revenue_usd -0.56      -> noise columns, no shared meaning
  • Loadings are the weights of original (standardised) features; signs are arbitrary per component (the script makes the largest loading positive).
  • Name a component only when a few features dominate with a coherent story; later components are often noise.
  • Varimax rotation (factor analysis) gives more interpretable axes when interpretation is the goal.

PCA in Pipelines

python
from sklearn.decomposition import PCA
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

model = make_pipeline(StandardScaler(), PCA(n_components=0.95, svd_solver="full"), LogisticRegression(max_iter=2000))
  • Fit inside the pipeline (CV refits PCA per fold): PCA on all data before splitting leaks test structure.
  • n_components=0.95 keeps enough components for 95% variance; tune k with CV when PCA feeds a model.
  • PCA rarely helps tree ensembles (they handle correlated features) and hurts interpretability; it helps linear models with collinearity, k-NN/k-means (distance concentration), and speed.
  • Sparse / text: TruncatedSVD(n_components=100) (LSA); out-of-core: IncrementalPCA(batch_size=...); non-linear: KernelPCA(kernel="rbf"); large dense data: svd_solver="randomized".

Variants, whitening and inverse transforms: pca-in-practice.md.

Multivariate Outliers and Monitoring

text
PCA with 2 components (70.7% variance) fitted on 300 rows; limits at the 0.99 training quantile: T2 14.2, Q 4.44
  line 42: Q    T2 14.2  Q 39.6  top Q contributors: temperature +4.04, vibration +3.73, pressure -3.06
  line 43: T2   T2 68.3  Q 1.31
against is_anomaly: precision 0.83, recall 0.83
  • T²: unusually far along normal directions (a big but consistent excursion).
  • Q / SPE: the row breaks the correlation structure (sensor fault, new regime); contributions show which features.
  • Fit on a clean reference period; set limits from it; score new data. Theory and control-limit options: pca-monitoring.md.

Output Format

When applying PCA, deliver:

  1. Scaling decision and why.
  2. Variance table - explained and cumulative per component (pca_report.py).
  3. k - the rules' answers and the chosen k for the purpose (structure vs compression vs model input).
  4. Interpretation - top loadings for the kept components, names where justified.
  5. Pipeline code - scaler + PCA + model, fitted on training folds.
  6. Monitoring (if relevant) - T²/Q limits, flagged rows, contributing features (pca_outliers.py).

Scripts

bash
python scripts/pca_report.py data.csv --drop id label
python scripts/pca_report.py data.csv --columns f1 f2 f3 f4 --top 4 --targets 0.9 0.99
python scripts/pca_report.py data.csv --no-scale --json

Pure-Python PCA (Jacobi eigendecomposition) on the correlation or covariance matrix: explained and cumulative variance, components needed for each target share, Kaiser count, parallel analysis with column-shuffled data (--pa-runs, --pa-quantile), top signed loadings per component, and an unscaled-dominance warning. Matches scikit-learn's explained variance ratios and loadings. Exit codes: 0 printed, 2 usage error.

bash
python scripts/pca_outliers.py --train reference.csv --score new.csv --drop id --k 2
python scripts/pca_outliers.py --train sensors.csv --drop reading_id --label is_anomaly --var 0.9 --alpha 0.995

Fits standardised PCA on --train (k from --k or --var), computes Hotelling's T² and Q residuals for the scored rows, empirical control limits at --alpha, flagged rows with the top Q-contributing features, and precision/recall against a --label column. Imports pca_report.py from the same folder. Exit codes: 0 scored, 2 usage error.

Additional Resources

FileContent
choosing-components.mdExplained variance, scree, Kaiser, parallel analysis, Velicer MAP, CV-based choice, broken-stick
pca-in-practice.mdscikit-learn PCA options, whitening, inverse_transform, IncrementalPCA, TruncatedSVD, KernelPCA, sparse PCA, factor analysis, pitfalls
pca-monitoring.mdHotelling T², Q/SPE, control limits (empirical, F, Jackson-Mudholkar), contribution plots, reference-period design
More in Classical Machine Learning

anomaly-detection-model-builder

Build and evaluate anomaly detection for tabular, sensor and transaction data with few or no labels: define the anomaly type (point, contextual, collective), choose detectors (robust z-score/MAD and IQR rules, Mahalanobis with robust covariance, k-NN distance, LOF, Isolation Forest, one-class SVM, PCA reconstruction / autoencoders, forecasting residuals for time series), combine them by rank, explain each alert by its deviating features, set thresholds from an alert budget or the score distribution, and evaluate with precision@k, PR-AUC and recall on known incidents, or detector agreement when no labels exist. Includes pure-Python detectors (robust z, Mahalanobis, kNN, Isolation Forest, ensemble) and an evaluation script.

catboost-model-builder

Build CatBoost models that use its strengths correctly: raw categorical columns as cat_features (ordered target statistics, no manual encoding), text_features, Pool construction with NaN-safe categories, ordered boosting and has_time for temporal data, symmetric (oblivious) trees, overfitting detector with eval_set and use_best_model, losses for classification, regression (Quantile, Poisson, Tweedie) and ranking (YetiRank), class weights, monotone constraints, feature importance (PredictionValuesChange, LossFunctionChange) and SHAP values, GPU training and model export. Plans cat/text/numeric columns from a CSV and generates a runnable training script, and checks parameter sets for the conflicts CatBoost rejects (iterations vs n_estimators, Bayesian bootstrap with subsample, multiple class-weight options, depth > 16, use_best_model without eval_set).

categorical-encoding-selector

Choose and implement the right encoding for every categorical column given the model family and the data: one-hot (with infrequent-level grouping and handle_unknown), ordinal with an explicit order, cross-fitted target encoding (scikit-learn TargetEncoder, CatBoost ordered statistics, category_encoders inside CV), frequency/count encoding, feature hashing, native categorical support in LightGBM, HistGradientBoosting, XGBoost and CatBoost, prefix levels for codes (zip3, NAICS-2), and embeddings. Measures cardinality, rare-level share, unseen-level rate on a time-ordered holdout and target signal per column, emits the ColumnTransformer, and quantifies target-encoding leakage (in-sample vs out-of-fold vs holdout AUC).

classification-model-selector

Choose the right classification algorithm for tabular, text or mixed data and prove the choice with a fair bake-off: logistic regression (L2, L1, elastic net), linear SVM, naive Bayes, k-NN, RBF SVM, decision trees, random forest, extra trees, HistGradientBoosting, LightGBM, XGBoost, CatBoost, explainable boosting machines (EBM/GAM), points scorecards and MLPs, matched to data size, width, categorical and high-cardinality share, missingness, class imbalance, text, and constraints such as interpretability, monotonic constraints, calibrated probabilities, latency budgets and tiny deployable models. Ranks candidates with starting hyperparameters and cautions, then generates a bake-off script running every candidate on identical CV folds with per-fold metrics, fit time and prediction latency.

clustering-algorithm-selector

Choose the right clustering algorithm and check that clustering makes sense at all: cluster tendency (Hopkins statistic with a correlated-features caveat, nearest-neighbour isolation), then k-means / MiniBatchKMeans, Gaussian mixtures with BIC, agglomerative (Ward, average, complete, single) with dendrograms, DBSCAN, HDBSCAN, OPTICS, spectral clustering, BIRCH, mean shift, k-prototypes and k-modes for mixed or categorical data, and Gower distance, matched to data size, expected shapes, noise, varying density, known k, soft memberships, hierarchy and streaming. Covers preprocessing (scaling, encoding, dimensionality, distance metrics), validation (internal indices, bootstrap stability, external labels, business profiling) and assigning new points.

collaborative-filtering-implementer

Implement collaborative filtering recommenders that beat popularity: item-based kNN with cosine, Jaccard or asymmetric similarity, shrinkage, neighbourhood size, positive-only or mean-centred rating signals and 'because you liked X' explanations; implicit ALS with confidence weighting, BPR pairwise ranking and the closed-form EASE model in plain NumPy, tuned on a temporal validation split by recall@k and NDCG@k and refit on all data; plus user-kNN, explicit matrix factorisation, the implicit/LightFM libraries, hyperparameter ranges, cold-start fallbacks, scaling with sparse matrices and approximate nearest neighbours. Produces top-k lists in the format of the recommendation evaluator.