# Losses and Targets

## What each loss estimates

| Loss | Minimiser of E[L(y, ŷ)] | Consequence |
|------|--------------------------|-------------|
| Squared error | conditional mean | sensitive to outliers; right for sums/totals |
| Absolute error | conditional median | robust; biased low for skewed targets when sums matter |
| Huber (δ) | between mean and median | quadratic near 0, linear beyond δ; tune δ (~1.35 x residual SD) |
| Pinball at q | conditional q-quantile | intervals, service levels |
| Poisson deviance | mean, variance ∝ mean | counts, rates with exposure |
| Gamma deviance | mean, variance ∝ mean² | positive amounts with multiplicative noise |
| Tweedie deviance, power p | mean, variance ∝ mean^p | p in (1,2): compound Poisson-Gamma, exact zeros + positive tail |

Train with the loss whose minimiser is the quantity the business reports. If you report MAE but train with
squared error on a skewed target, you optimise something else.

## Log targets and retransformation bias

Fitting `log1p(y)` with squared error estimates E[log(1 + y) | x]. Back-transforming gives roughly the
conditional **median** (exactly, for log-normal errors), which is below the mean:

```
E[y | x] = exp(μ + σ²/2)   while   exp(E[log y | x]) = exp(μ)
```
Corrections:
- **Smearing (Duan 1983):** `ŷ_mean = exp(prediction) * mean(exp(residuals_log))` with residuals on the log
  scale from validation data. Non-parametric and simple.
- **Normal theory:** multiply by `exp(σ̂²/2)` (assumes log-normal residuals).
- **Use a deviance loss instead** (Gamma, Tweedie) with a log link: estimates the mean directly.

When the decision uses typical values (median house price), the log model without correction is fine.

## Tweedie power

- p = 1: Poisson; p = 2: Gamma; 1 < p < 2: point mass at zero plus continuous positive part.
- Typical insurance pure premium: p ≈ 1.5-1.7.
- Choose by CV: fit p in {1.2, 1.4, 1.5, 1.6, 1.8} and compare out-of-fold **deviance at a fixed p** (or a
  business metric such as Gini/Lorenz for ranking); deviances at different p are not comparable to each other.
- `tweedie` packages and `statsmodels` can estimate p by profile likelihood.

## Two-part (hurdle) models

```
E[y | x] = P(y > 0 | x) * E[y | y > 0, x]
```
Fit a classifier on all rows and a positive regressor (Gamma / log target) on positive rows. Use when zeros
are generated by a different mechanism (no claim vs claim severity; no purchase vs basket size), and when you
need the two parts separately for decisions or explanations.

## Proportions

Targets in [0, 1] (utilisation, conversion rate per segment):
- logit transform: `y' = log((y + ε) / (1 − y + ε))`, fit, invert with `expit`;
- cross-entropy on soft labels (LightGBM `cross_entropy`, XGBoost `reg:logistic`) keeps predictions in (0, 1);
- beta regression (statsmodels) for a GLM view;
- for ratios of counts (clicks / impressions), model the counts with Poisson/binomial and the denominator as
  exposure or weight.

## Counts with exposure

Rate = count / exposure. Two equivalent fits:
- target = count, offset = log(exposure) (GLMs, LightGBM `init_score`, XGBoost `base_margin`);
- target = count / exposure, sample_weight = exposure (scikit-learn GLMs and HGB).

Forgetting exposure makes long-exposure entities look risky.

## Censored and truncated targets

Durations still running (censored) need survival models (survival-analysis-model-builder). Targets truncated
by a cap (sales capped by stock) underestimate demand: flag censored rows and model demand accordingly.