Skip to content
v0.1.3latest release121 skills4 categories6 sub-categories247 tested scripts397 reference docsContact us
Vector Agentic Skills
Submit a skill

time-series-forecasting-model-builder

Build forecasting models that beat honest baselines: profile the series (frequency, gaps, duplicate timestamps, seasonality by autocorrelation, trend and seasonal strength, weekday and month effects, outliers, smooth/erratic/intermittent/lumpy demand by ADI and CV^2), backtest seasonal naive, drift, SES, Holt-Winters, Croston, SBA and TSB on rolling origins with MASE, RMSSE, WAPE and bias, score your own models' forecasts on the same origins, and generate a LightGBM forecaster with leak-free direct-strategy lags (every lag at least the horizon), rolling and calendar features, optional linear detrending (trees cannot extrapolate trend), multi-series support and split-conformal prediction intervals. Covers statsforecast/statsmodels ETS and ARIMA, global models, hierarchies and intermittent demand. Use when someone needs a demand, sales, traffic or load forecast, asks which forecasting model to use, how to backtest one, or why a model fails to beat naive.

AI & Machine Learning / Classical Machine Learningv1.0.0Updated OCT 25 20263 references4 scripts

Most forecasting failures are evaluation failures. Common ones:

  • comparing with no baseline, or a weak one
  • one train/test split instead of rolling origins
  • features that peek past the forecast origin
  • MAPE on series with zeros
  • tree models asked to extrapolate a trend

This skill makes the baseline and the protocol explicit first, then builds the model:

  • scripts/ts_profile.py describes the series and recommends the model families
  • scripts/ts_backtest.py scores the baselines (and your models) on identical rolling origins
  • scripts/gen_forecast_script.py writes a leak-free LightGBM forecaster whose backtest output plugs straight back into ts_backtest.py

Trigger Conditions

Activate this skill when the user:

  • Needs a forecast (demand, sales, traffic, calls, energy load, inventory, KPIs)
  • Asks which model to use (ETS, ARIMA, Prophet, LightGBM, Croston) or how to backtest
  • Has a model that does not beat naive, or degrades in production
  • Forecasts intermittent demand (spare parts, slow movers) or many related series

Time-based features for non-forecasting models: time-based-feature-builder. Splitting time data: cross-validation-strategy-designer.

Workflow

text
1. PROFILE   -> python scripts/ts_profile.py sales.csv           (frequency, gaps, season, strength, demand class)
2. BASELINE  -> python scripts/ts_backtest.py sales.csv --horizon 28 --folds 6
3. BUILD     -> python scripts/gen_forecast_script.py --data sales.csv --value-col units --freq D --horizon 28 --detrend -o forecast.py
4. COMPARE   -> python forecast.py; python scripts/ts_backtest.py sales.csv --horizon 28 --folds 6 --forecasts backtest_forecasts.csv
5. DECIDE    -> deploy only what beats the best baseline on the same origins (MASE/RMSSE skill, bias)
6. DELIVER   -> forecast with intervals, the backtest table, and a monitoring plan

Choosing a Model From the Profile

ProfileStart withThen try
Strong single seasonality, short historyseasonal naive, ETS / Holt-WintersTheta, ARIMA
Multiple seasonalities (hourly: daily + weekly)MSTL, TBATSLightGBM with calendar features
Strong trend + seasonality, enough historyHolt-Winters / ETSLightGBM on detrended target (--detrend)
Many related series (stores x products)per-series ETSone global LightGBM across series
Intermittent / lumpy (ADI >= 1.32)Croston / SBA / TSBprobabilistic models (negative binomial), aggregated forecasts
Strong external drivers (price, promotions, weather)regression with ARIMA errorsLightGBM with known-future covariates
Very short series (< 2 seasons)naive / drift / meanpool with similar series

Sales fixture (daily, 2 years, weekly cycle, trend, December peak):

ModelMASESkill vs snaiveWAPE
LightGBM, detrended (generated)0.889+46.5%4.0%
LightGBM, not detrended1.088+34.5%4.9%
Holt-Winters (stdlib)1.155+30.5%5.2%
statsforecast AutoETS1.367+17.8%6.2%
seasonal naive1.66207.5%

Model families and libraries: statistical-models.md and ml-forecasting.md.

Evaluation Protocol

  • Rolling origin: several forecast origins (5-10), each refit on data before it. One split is an anecdote.
  • Horizon = the business horizon: forecasting 28 days ahead is a different problem from 1 day ahead. Report error by horizon step.
  • Scale-free metrics: MASE (MAE / in-sample seasonal-naive MAE) to compare across series. Use RMSSE for intermittent demand, where MAE rewards forecasting zero. WAPE for business reporting. Always report bias.
  • Never MAPE with zeros or tiny values: it is undefined or explodes. sMAPE is asymmetric.
  • Same origins for every model: ts_backtest.py --forecasts scores external models on the baselines' origins (--write-origins exports them).
  • Multi-step MASE above 1 is normal: the denominator is a one-step error. Compare models through the skill column.

Metric definitions, pitfalls and reporting: forecast-evaluation.md.

Leakage Rules for ML Forecasters

LeakPrevention
lag 1..h-1 used to forecast h steps aheaddirect strategy with minimum lag = horizon (the generated script) or recursive forecasting
rolling windows that include the target periodwindows end H periods back (shift(H).rolling(w))
scaling / target encoding fitted on all datafit inside each training window
future covariates not known at forecast time (actual weather, actual price)use forecasts, or values known in advance (planned prices, calendars)
random K-foldrolling origin only

Prediction Intervals

The generated script builds split-conformal intervals from backtest residuals. Each step pools residuals over ±3 neighbouring steps and scales them by the series' recent level. In checks on the fixture, holdout coverage of the nominal 80% intervals was 0.71-0.86 in normal months but 0.46 in the December peak, which the backtest folds did not contain. Intervals are calibrated only for conditions the backtest covers. For seasonal peaks, include last year's peak in the folds (more folds, or origins chosen in that season).

Output Format

When delivering a forecast, provide:

  1. Profile - frequency, history, gaps, seasonality, trend, demand class, outliers.
  2. Backtest table - all baselines and candidates on the same origins: MASE/RMSSE, skill, WAPE, bias, error by horizon.
  3. Chosen model - why it wins, its features, and what it cannot capture.
  4. Forecast - point forecast with intervals and coverage evidence.
  5. Monitoring - the error metric tracked, the retraining cadence, and drift and bias alerts.

Scripts

bash
python scripts/ts_profile.py sales.csv
python scripts/ts_profile.py demand.csv --time-col week --value-col qty --horizon 8 --json

Strength measures come from a moving-average decomposition (0.89/0.94 on the fixture against statsmodels STL 0.94/0.97; ACF identical). Exit codes: 0 profiled, 2 usage or read error.

bash
python scripts/ts_backtest.py sales.csv --horizon 28 --folds 6
python scripts/ts_backtest.py demand.csv --horizon 8 --season 52 --forecasts my_forecasts.csv
python scripts/ts_backtest.py sales.csv --horizon 28 --folds 4 --write-origins origins.csv

Baselines are refit at each origin and clipped at zero for non-negative series. They are ranked by MASE, or by RMSSE when more than 30% of values are zero, with a warning for near-zero forecasts. The seasonal-naive MAE/MASE matches an independent pandas computation. Exit codes: 0 backtested, 2 usage or read error.

bash
python scripts/gen_forecast_script.py --data sales.csv --time-col date --value-col units --freq D --horizon 28 -o forecast.py
python scripts/gen_forecast_script.py --data stores.csv --id-col store --value-col sales --freq W --horizon 8 --fill zero --detrend

Writes backtest_forecasts.csv (for ts_backtest.py --forecasts) and forecast.csv (date, forecast, lo, hi). Exit codes: 0 written, 2 usage error. scripts/_ts_io.py holds the shared loading, frequency inference and gap filling.

Additional Resources

FileContent
forecast-evaluation.mdrolling origins, metrics (MASE, RMSSE, WAPE, bias, pinball, coverage), horizon analysis, pitfalls
statistical-models.mdETS, ARIMA, Theta, MSTL/TBATS, intermittent methods, statsforecast and statsmodels code
ml-forecasting.mdfeatures, direct vs recursive, global models, covariates, hierarchies, intervals, production
More in Classical Machine Learning

anomaly-detection-model-builder

Build and evaluate anomaly detection for tabular, sensor and transaction data with few or no labels: define the anomaly type (point, contextual, collective), choose detectors (robust z-score/MAD and IQR rules, Mahalanobis with robust covariance, k-NN distance, LOF, Isolation Forest, one-class SVM, PCA reconstruction / autoencoders, forecasting residuals for time series), combine them by rank, explain each alert by its deviating features, set thresholds from an alert budget or the score distribution, and evaluate with precision@k, PR-AUC and recall on known incidents, or detector agreement when no labels exist. Includes pure-Python detectors (robust z, Mahalanobis, kNN, Isolation Forest, ensemble) and an evaluation script.

catboost-model-builder

Build CatBoost models that use its strengths correctly: raw categorical columns as cat_features (ordered target statistics, no manual encoding), text_features, Pool construction with NaN-safe categories, ordered boosting and has_time for temporal data, symmetric (oblivious) trees, overfitting detector with eval_set and use_best_model, losses for classification, regression (Quantile, Poisson, Tweedie) and ranking (YetiRank), class weights, monotone constraints, feature importance (PredictionValuesChange, LossFunctionChange) and SHAP values, GPU training and model export. Plans cat/text/numeric columns from a CSV and generates a runnable training script, and checks parameter sets for the conflicts CatBoost rejects (iterations vs n_estimators, Bayesian bootstrap with subsample, multiple class-weight options, depth > 16, use_best_model without eval_set).

categorical-encoding-selector

Choose and implement the right encoding for every categorical column given the model family and the data: one-hot (with infrequent-level grouping and handle_unknown), ordinal with an explicit order, cross-fitted target encoding (scikit-learn TargetEncoder, CatBoost ordered statistics, category_encoders inside CV), frequency/count encoding, feature hashing, native categorical support in LightGBM, HistGradientBoosting, XGBoost and CatBoost, prefix levels for codes (zip3, NAICS-2), and embeddings. Measures cardinality, rare-level share, unseen-level rate on a time-ordered holdout and target signal per column, emits the ColumnTransformer, and quantifies target-encoding leakage (in-sample vs out-of-fold vs holdout AUC).

classification-model-selector

Choose the right classification algorithm for tabular, text or mixed data and prove the choice with a fair bake-off: logistic regression (L2, L1, elastic net), linear SVM, naive Bayes, k-NN, RBF SVM, decision trees, random forest, extra trees, HistGradientBoosting, LightGBM, XGBoost, CatBoost, explainable boosting machines (EBM/GAM), points scorecards and MLPs, matched to data size, width, categorical and high-cardinality share, missingness, class imbalance, text, and constraints such as interpretability, monotonic constraints, calibrated probabilities, latency budgets and tiny deployable models. Ranks candidates with starting hyperparameters and cautions, then generates a bake-off script running every candidate on identical CV folds with per-fold metrics, fit time and prediction latency.

clustering-algorithm-selector

Choose the right clustering algorithm and check that clustering makes sense at all: cluster tendency (Hopkins statistic with a correlated-features caveat, nearest-neighbour isolation), then k-means / MiniBatchKMeans, Gaussian mixtures with BIC, agglomerative (Ward, average, complete, single) with dendrograms, DBSCAN, HDBSCAN, OPTICS, spectral clustering, BIRCH, mean shift, k-prototypes and k-modes for mixed or categorical data, and Gower distance, matched to data size, expected shapes, noise, varying density, known k, soft memberships, hierarchy and streaming. Covers preprocessing (scaling, encoding, dimensionality, distance metrics), validation (internal indices, bootstrap stability, external labels, business profiling) and assigning new points.

collaborative-filtering-implementer

Implement collaborative filtering recommenders that beat popularity: item-based kNN with cosine, Jaccard or asymmetric similarity, shrinkage, neighbourhood size, positive-only or mean-centred rating signals and 'because you liked X' explanations; implicit ALS with confidence weighting, BPR pairwise ranking and the closed-form EASE model in plain NumPy, tuned on a temporal validation split by recall@k and NDCG@k and refit on all data; plus user-kNN, explicit matrix factorisation, the implicit/LightFM libraries, hyperparameter ranges, cold-start fallbacks, scaling with sparse matrices and approximate nearest neighbours. Produces top-k lists in the format of the recommendation evaluator.