Time Series Forecasting Clinic in Python: ARIMA, Prophet, XGBoost

Section 1.1: What makes time series different from i.i.d. ML

In i.i.d. machine learning, we often assume each row is independent and identically distributed. In time series, that assumption is almost always false: observations are ordered, correlated, and frequently influenced by calendar effects, delayed reactions, and regime shifts. The ordering is not cosmetic—it defines what information was available at prediction time. If your model “peeks” at future values (directly or indirectly), it will look great offline and fail immediately in production.

Time series problems also come with explicit deployment semantics: you predict at time t using data available up to t, for one or more future times t+h. This immediately forces you to define: what is the forecast target (demand, revenue, visits), what is the forecast horizon (next day? next 12 weeks?), and what cadence will the forecast be produced (daily run producing 14-day forecast). These choices affect everything: feature construction, model class, metric interpretation, and even how missing values are treated.

Common mistakes include treating time as just another feature, shuffling data during cross-validation, or building features from “global” aggregates that include future periods (for example, normalizing by a mean computed over the entire dataset). Practical judgment means constantly asking: “Would I have known this at the moment the forecast is made?” If the answer is no, it’s leakage.

Another key difference is that time series often have multiple valid “truths” depending on measurement processes. Backfilled data, late-arriving transactions, and revised numbers are normal in business settings. You may need to decide whether to model the final revised value or the as-known-at-the-time value. This choice belongs to problem setup, not model tuning, because it changes what your training labels represent.

Section 1.2: Horizon, frequency, and aggregation decisions

A forecasting problem is not fully specified until you define three things precisely: target, horizon, and frequency (granularity). Start by writing a single sentence: “Every [frequency], we predict [target] for the next [horizon].” Example: “Every Monday, we forecast weekly unit demand for the next 12 weeks.” This sentence prevents a large class of misalignment bugs and stakeholder misunderstandings.

Horizon is not just a number—it encodes decisions. A 1-step-ahead forecast (tomorrow) can leverage short-term autocorrelation; a 52-step-ahead forecast (one year weekly) must model seasonality and long-term trend robustly. It also determines whether you need point forecasts only (single number) or prediction intervals (risk-aware planning). If your business decision is ordering inventory with lead time, the relevant horizon is often “lead time + review period,” not “next day because we have data daily.”

Frequency (daily, weekly, hourly) should match how decisions are made and how noisy your data is. Aggregation can improve signal-to-noise ratio but can also hide important patterns. If demand is intermittent daily but stable weekly, weekly forecasting may be more actionable. Be explicit about aggregation rules (sum vs mean vs last) and align them with the business meaning of the target.

• Sum is typical for sales/volume; mean for rates like conversion; last for end-of-period balances.
• Ensure consistent time boundaries: ISO weeks vs calendar weeks, timezone handling, and daylight savings for intraday data.

In pandas, create a clean time index early and treat it as a product artifact. Use pd.to_datetime, set the index, sort, and enforce a regular frequency with asfreq or resample. Then explicitly address missing timestamps. A missing day can mean “zero” (no sales) or “unknown” (data outage). The wrong assumption can bias both training and evaluation. Document your choice in code comments and in the project README so future you (or your teammate) doesn’t “fix” it later and invalidate results.

Section 1.3: Baselines (naive, seasonal naive, moving average)

Before building ARIMA, Prophet, or XGBoost, you need baselines. Baselines are not “toy models”; they are quality gates and debugging tools. If a sophisticated model cannot beat a naive baseline on a leakage-safe evaluation, either the model is misconfigured, the features leak, the split is wrong, or the problem is fundamentally hard at the chosen horizon/frequency.

The naive baseline predicts the last observed value: ŷ(t+h) = y(t). It is surprisingly strong for highly persistent series (e.g., slowly moving KPIs). The seasonal naive baseline repeats the value from the same season: daily data with weekly seasonality uses ŷ(t+h) = y(t-7); monthly data with yearly seasonality uses y(t-12). This baseline often outperforms many poorly tuned models because it respects seasonality without overfitting.

A moving average baseline predicts the mean of the recent window (e.g., last 7 or 28 points). This can reduce noise and can be a reasonable benchmark when the series is erratic. However, it can lag badly when there is trend or abrupt changes—knowing this behavior helps you interpret whether your “real” model is actually learning trend or just smoothing.

• Implement baselines in a few lines and keep them in your repository permanently.
• Plot baseline forecasts against actuals for a few windows; visuals catch index alignment bugs faster than metrics.
• Use baselines to sanity-check resampling decisions: if seasonal naive collapses after aggregation, you may have removed the seasonal signal.

Baselines also force clarity on forecasting mechanics. Are you producing a 14-day forecast all at once (multi-step), or predicting one day ahead repeatedly (recursive)? For baselines, both are easy to simulate and reveal how error compounds across the horizon. Later, the same thinking will apply to ARIMA (direct multi-step) and XGBoost (direct vs recursive strategies).

Section 1.4: Metrics: MAE, RMSE, MAPE/SMAPE, WAPE, pinball loss

Metrics translate forecast errors into decisions. Picking a metric is an engineering and business judgment: it encodes which errors hurt more and how you compare across products or time periods. Use at least one scale-dependent metric (e.g., MAE or RMSE) and one scale-independent metric (e.g., WAPE) when comparing across series.

MAE (mean absolute error) is robust and easy to interpret: “average absolute miss.” It penalizes all errors linearly, which often matches operational pain. RMSE penalizes large errors more (squared), making it sensitive to outliers. RMSE is useful when big misses are disproportionately costly, but it can also overemphasize one-off anomalies or data quality issues—so pair it with outlier handling and diagnostics.

MAPE (mean absolute percentage error) seems intuitive but breaks when actuals are near zero and can bias toward under-forecasting. SMAPE reduces some issues but still behaves oddly around zero. For many business settings, WAPE (weighted absolute percentage error) is a better default: sum(|y-ŷ|) / sum(|y|). WAPE is stable across scale and does not explode as easily as MAPE when there are small denominators.

If you care about uncertainty, service levels, or risk, evaluate quantile forecasts with pinball loss. Pinball loss at quantile q penalizes under-forecasting more when q is high (e.g., q=0.9 for high-service inventory planning). This connects naturally to “business cost functions”: stockouts vs overstock typically have asymmetric costs, and pinball loss lets you encode that asymmetry without inventing a complicated custom metric.

• Report metrics per horizon step (1-step, 2-step, …) to see how error grows with time.
• Complement global metrics with slices: weekends vs weekdays, promotions vs non-promotions, holiday periods.
• Decide whether to weight errors by revenue/volume; otherwise, low-impact series can dominate model selection.

Finally, define a “minimum acceptable improvement” over baseline. A 1% WAPE improvement may be meaningless for one business and huge for another. Make this explicit early so you don’t optimize endlessly for negligible gains.

Section 1.5: Data splitting: holdout, rolling origin, gaps and leakage

A leakage-safe split is the backbone of credible forecasting. Random splits are almost always invalid because they allow training on future patterns. Instead, use time-based splits that mimic deployment: train on the past, validate on a later period, and test on the most recent unseen period.

A simple holdout split (train up to time T, test after T) is a good start, but it can be fragile if your test window is unrepresentative (e.g., only holiday weeks). For more reliable model comparisons, use rolling origin (walk-forward) backtesting: create multiple folds where the training window moves forward and you forecast the next horizon each time. This reveals how models behave across regimes and reduces the chance you “win” by luck on a single window.

Many teams miss the need for gaps. If your features use rolling windows (e.g., last 28 days average) or labels have reporting delays, you may need to leave a gap between the end of training data and the start of validation/test. Without a gap, the model can indirectly access information too close to the forecast origin. Example: if you compute a rolling mean including the current day, and you predict tomorrow, that’s fine; but if your pipeline accidentally uses centered windows or forward-filled values, you can leak tomorrow into today.

• Always compute features within each fold using only data available up to the forecast origin.
• Beware of resampling: aggregating to weekly after splitting can mix train/test weeks depending on week boundaries.
• If you scale/normalize, fit scalers on the training portion only, then apply to validation/test.

Operationally, write split functions that accept a forecast horizon and produce explicit date ranges. Store the split boundaries (start/end timestamps) alongside results. This makes your experiments reproducible and audit-friendly—especially important when stakeholders ask why “the model was better last month.”

Section 1.6: Forecast communication: point vs interval and stakeholder needs

A forecast is only useful if it can be acted on. That means communicating it in the form stakeholders need, not in the form the model naturally produces. Start by clarifying the decision: staffing, inventory, budgeting, anomaly detection, or capacity planning each requires different outputs and tolerances.

Point forecasts (a single number) are simplest and often sufficient for dashboards. But many operational decisions need interval forecasts (e.g., 80% and 95% prediction intervals) to quantify risk. Intervals change behavior: instead of arguing about whose point forecast is “right,” teams can plan for best/worst cases and choose policies (like safety stock) explicitly. Even if you begin with point models, you should design your evaluation to accommodate intervals later (e.g., by tracking pinball loss or empirical coverage).

Communication also includes explanations at the right level. Stakeholders often ask: “Is demand going up because of trend, seasonality, or a holiday?” Prophet’s decomposed components will later provide a natural narrative, while ARIMA diagnostics help explain when a series is not stationary or has autocorrelation issues. For machine learning models like XGBoost, feature importance and partial dependence can help, but be careful: explanations should be consistent with time—don’t attribute effects to future-known variables.

• Report forecasts with timestamps, units, and aggregation definitions (e.g., “weekly sum, Mon–Sun”).
• Provide a baseline comparison chart; stakeholders trust improvements when they see them against naive methods.
• Include a short “known limitations” note: data gaps, regime changes, and expected drift.

To keep work reproducible, set up a minimal project template now: a data folder (raw/processed), a src/ package with data.py (loading and index cleaning), features.py (leakage-safe feature creation), models/ (arima.py, prophet.py, xgb.py), and evaluation.py (splits, metrics, backtests). Add a single configuration file for frequency, horizon, and metric choices. This structure turns forecasting from a notebook experiment into an engineering process—one you can iterate on confidently in the chapters ahead.

Section 2.1: Parsing datetimes, time zones, and consistent indexing

Start by making time explicit and unambiguous. In pandas, parse timestamps with pd.to_datetime, then decide whether the series should be timezone-aware. A common production failure is mixing local time (with daylight saving shifts) and UTC across sources. Choose a standard—often UTC—and convert everything to it. If your business reporting is in a local zone, convert to UTC for modeling and convert back only for presentation.

Next, enforce a single, consistent index. Set df = df.sort_values('ds').set_index('ds'), then remove duplicates carefully. Duplicate timestamps are not “noise”; they mean multiple measurements were recorded for the same interval. Decide whether to sum, mean, take last, or treat as an anomaly, and document the rule. For example, retail transactions may need aggregation, while sensor readings might need last() to reflect the final state.

Finally, audit the implied frequency and the continuity of the time axis. Use pd.infer_freq(df.index) as a hint, but don’t trust it blindly—gaps can cause inference to fail. Compute expected timestamps for the chosen frequency (e.g., hourly, daily, weekly) and compare against actual timestamps to locate missing intervals. Missing timestamps are different from missing values: a missing timestamp means the entire period is absent, which affects seasonality alignment and any lag/rolling features. Create an explicit calendar index with pd.date_range and reindex to make gaps visible as NaN. This single step turns hidden irregularity into something you can handle systematically.

• Practical check: plot the time deltas between consecutive timestamps; spikes flag gaps or clock shifts.
• Common mistake: resetting the index after resampling and losing the time alignment used for walk-forward splits.

With a stable DatetimeIndex, every later transformation—imputation, decomposition, feature windows—becomes reproducible and testable.

Section 2.2: Imputation strategies and resampling (up/down)

After reindexing to a complete calendar, you will see missing values. Treat them based on the data-generating process, not convenience. First classify missingness: (1) the measurement exists but was not recorded, (2) the period truly has no activity (e.g., store closed), or (3) the source system has an outage. Each implies a different fill strategy and different downstream interpretation.

Imputation should respect causality. Forward-fill (ffill) is reasonable for inventory levels or account balances (stateful series), but dangerous for demand because it creates artificial persistence. Interpolation (time or linear) can work for physical sensors but can distort business series with sharp promotions. For count-like targets, consider leaving gaps as missing and using models that can handle them, or imputing with seasonal medians computed from past data only (e.g., median of the same weekday over previous weeks). When you compute such statistics, ensure they are based solely on historical windows to avoid leakage.

Resampling is the other major decision. Downsampling (e.g., minute to hour, hour to day) uses resample with appropriate aggregation: sums for volume, means for rates, last for end-of-period states. Upsampling (e.g., day to hour) is riskier because you are inventing intra-period structure; if you must upsample for a model, separate “target frequency” from “feature frequency,” and avoid training the model to predict synthetic targets. Often, it’s better to keep the target at the natural frequency and engineer additional features from higher-frequency covariates aggregated to that target frequency.

• Practical workflow: df = df.asfreq('D') to enforce daily frequency, then choose a fill rule per column.
• Common mistake: using bfill (backfill) on the target; it explicitly uses future information and will inflate backtest results.

The outcome of this section is a regular, gap-aware series with imputation that you can justify to a stakeholder and defend during evaluation.

Section 2.3: Outliers: detection, winsorizing, and event annotation

Outliers in time series are rarely random; they often correspond to promotions, outages, holidays, stockouts, or sensor faults. Your first task is to distinguish “bad data” from “rare but real.” Removing real spikes may make a model look smoother, but it also teaches the model to under-forecast during important events.

Detect outliers with methods that respect time structure. A simple z-score on the entire series can fail when the series has trend or seasonality. Prefer rolling robust statistics: compute a rolling median and median absolute deviation (MAD), then flag points that deviate beyond a threshold. Alternatively, decompose first (STL in Section 2.4) and flag outliers on the residual component. Also look for level shifts (regime changes) using rolling means or change-point methods; a one-time step change is not an “outlier” but a new operating regime.

Treatment options should preserve labels and avoid leakage. Winsorizing caps extreme values to a percentile (e.g., 1st/99th) and is useful when you believe measurements saturate due to errors. But if an outlier is an event you care about, do not cap it away—annotate it. Create an event_flag feature, or better, create structured event metadata (promotion type, outage duration). For regime changes, consider splitting the modeling period, adding a post-change indicator, or retraining with more weight on recent data. Importantly, do not compute capping thresholds using the full dataset when backtesting; fit thresholds on the training window and apply to the test window to avoid peeking.

• Practical rule: fix obvious data errors (negative sales, impossible sensor values), annotate real events, and only then consider winsorizing.
• Common mistake: deleting outlier rows; this breaks the time axis and silently changes lag alignment.

Handled correctly, outliers become signal (events) or controlled noise (measurement error), rather than a source of brittle forecasts.

Section 2.4: Decomposition: trend/seasonal/residual and STL

Decomposition is your diagnostic microscope: it separates the series into trend, seasonal pattern(s), and residual variation. This does two things in practice: (1) it helps you choose model families and feature types, and (2) it reveals data issues like shifting seasonality, calendar effects, or outliers concentrated in the residuals.

Classical decomposition assumes additive (y = trend + seasonal + resid) or multiplicative structure. Additive is appropriate when seasonal amplitude is roughly constant; multiplicative fits when seasonal swings grow with level (common in revenue series). A quick visual check is to plot the series and ask: do peaks get taller as the mean rises? If yes, consider a log transform before modeling, which often turns multiplicative seasonality into additive.

STL (Seasonal-Trend decomposition using LOESS) is usually the most practical tool because it’s robust and handles complex seasonality better than classical methods. In Python, statsmodels.tsa.seasonal.STL lets you set the seasonal period (e.g., 7 for daily data with weekly seasonality, 24 for hourly daily seasonality). Choose the period from your domain knowledge and confirmed frequency, not from guesswork. After fitting, plot components and inspect: a drifting seasonal component may indicate changing behavior; large residual spikes are candidates for event annotation (Section 2.3).

• Practical workflow: apply STL on the training window only during backtests to avoid using future data for smoothing.
• Common mistake: decomposing an irregular index; STL expects a consistent frequency and can mislead if gaps were not made explicit.

Decomposition doesn’t replace modeling, but it gives you a map: whether you need seasonal terms, whether a log transform stabilizes variance, and whether “noise” is actually unmodeled structure.

Section 2.5: ACF/PACF intuition for autocorrelation structure

The autocorrelation function (ACF) and partial autocorrelation function (PACF) provide a compact summary of temporal dependence—critical for ARIMA/SARIMA and still useful for feature engineering in XGBoost. The ACF answers: “How correlated is the series with itself shifted by k steps?” The PACF answers: “How much correlation remains at lag k after accounting for shorter lags?”

In practice, you use ACF/PACF plots as heuristics, not rigid rules. ACF that decays slowly often indicates non-stationarity (trend), suggesting differencing or transformation. Seasonal spikes at regular intervals (e.g., lag 7 for daily data) confirm weekly seasonality and motivate seasonal differencing (SARIMA) or explicit seasonal features (Fourier terms, weekday dummies). For ARIMA identification, a “cutoff” in PACF with a decaying ACF can suggest an AR(p) structure, while a cutoff in ACF with a decaying PACF can suggest MA(q). Real data is messy, so focus on the strongest, most stable spikes.

Be careful with preprocessing: compute ACF/PACF on a stationary-ish series (often after log and/or differencing) to avoid plots dominated by trend. Also, if you imputed large gaps aggressively, you may create artificial autocorrelation. Use this section as a feedback loop: if the ACF shows an unrealistically strong lag-1 persistence after filling, reconsider your imputation strategy.

• Practical outcome: choose candidate seasonal period(s) and a short list of meaningful lags for both ARIMA orders and ML features.
• Common mistake: reading ACF/PACF on the full dataset and then tuning orders to it; always confirm choices via walk-forward backtesting.

When used with decomposition, ACF/PACF gives you a grounded sense of “memory” in the data—how far back the past meaningfully influences the future.

Section 2.6: Feature sets: lags, rolling windows, Fourier terms, holidays

To use XGBoost (or any supervised learner) for forecasting, you convert the time series into a tabular dataset where each row represents a forecast origin time t and features summarize the history up to t. The golden rule is alignment: features must be computable using only data available at prediction time, and the label must represent the target at t+h for your horizon h. For one-step-ahead daily forecasting, label is y.shift(-1); for 7-day ahead, label is y.shift(-7).

Core features include lag values (e.g., lag_1, lag_7, lag_14), rolling statistics (mean, median, std, min/max) over windows that match business cycles (7, 28, 56 days), and exponentially weighted moving averages for smoother memory. Always compute rolling features with closed='left' or by shifting first so the window excludes the current target time. Calendar features (day of week, month, week of year) often provide strong lift with little cost; encode them as integers or one-hot depending on the model and cardinality. For complex seasonality, Fourier terms offer a compact representation: generate sine/cosine pairs for the seasonal period (weekly, yearly) with a chosen order K; this is particularly helpful when you want Prophet-like seasonality in an ML model.

Holidays and events deserve explicit handling. Add binary indicators for known holidays and custom business events (promotions, paydays), and consider lead/lag windows (e.g., is_holiday, is_pre_holiday_1, is_post_holiday_1) because effects often spill over. If you annotated anomalies in Section 2.3, those flags can become features rather than removed points.

Finally, build a reusable pipeline. Write functions that: (1) enforce frequency and sorting, (2) fit imputation/outlier rules on training data, (3) generate features deterministically, and (4) return X, y aligned for any horizon. In backtesting, fit the pipeline inside each training fold and apply to the validation fold to avoid leakage. This discipline is what makes model comparisons fair and productionization straightforward.

• Practical starter set: lags [1, 7, 14, 28], rolling means over [7, 28], day-of-week, month, holiday flags, and optional Fourier weekly/yearly.
• Common mistake: computing rolling means on the full series once, then slicing folds; this leaks future values into early folds via the rolling window.

With these features and a leakage-safe pipeline, you can train XGBoost models that compete strongly with classical methods while staying interpretable and testable.

ARIMA-style models assume the underlying process is (approximately) stationary after differencing: its mean and autocovariance structure don’t drift over time. Many real series are not stationary in levels—think revenue that grows, or sensor readings with drift. The core question is: how many differences (d) do you need to remove unit-root behavior without destroying signal?

In practice, begin with plots before tests. Plot the series, and if relevant, its seasonal subseries (e.g., by month). If you see clear trend, consider a first difference: y.diff(). If you see strong seasonality (weekly or yearly), consider seasonal differencing later (Section 3.2). Then apply unit root tests as confirmation, not as a sole decision rule.

• ADF (Augmented Dickey-Fuller): null hypothesis is “unit root present” (non-stationary). A small p-value suggests stationarity.
• KPSS: null hypothesis is “stationary.” A small p-value suggests non-stationarity.

Using both is a pragmatic cross-check: if ADF fails to reject and KPSS rejects, you likely need differencing. If both suggest stationarity, keep d=0. If they disagree, rely on plots and downstream diagnostics (residual autocorrelation after fitting). Over-differencing is a common mistake: it can induce moving-average behavior, inflate forecast uncertainty, and produce negative autocorrelation at lag 1. A telltale sign is a differenced series that looks like alternating up/down noise with little persistence.

Engineering judgement: start with d in {0,1} for most business series; d=2 is rarely needed and often indicates you should revisit transformations (e.g., log) or structural breaks. If variance grows with level, apply a log or Box-Cox transform before differencing so “stationary” means stable variance as well as stable mean.

An ARIMA(p,d,q) model combines three ideas: autoregression (AR, order p), integration via differencing (I, order d), and moving average (MA, order q). SARIMA extends this with seasonal counterparts: (P,D,Q,s) where s is the seasonal period (e.g., 12 for monthly data with yearly seasonality, 7 for daily data with weekly seasonality).

Conceptually: AR terms say “today looks like a linear combination of recent past values.” MA terms say “today absorbs recent shocks (errors).” Differencing makes the series “about changes” rather than levels. Seasonal terms repeat the same logic but at lag s, 2s, etc.

Practical identification often starts with the ACF/PACF on the stationary version of the series (after any differencing you believe is required). Rules of thumb help, but don’t over-trust them:

• If PACF cuts off after lag p and ACF decays, try AR(p).
• If ACF cuts off after lag q and PACF decays, try MA(q).
• Seasonal spikes at lags s, 2s, ... suggest seasonal AR/MA.

Seasonal differencing (D=1) is powerful but blunt: it removes repeating yearly/weekly level shifts. Use it when the seasonality is persistent in amplitude and phase. If seasonal effects change over time, SARIMA may struggle; later chapters will show Prophet and XGBoost alternatives. Also note that s must match your data frequency: if you resample daily data to weekly, then a “yearly” seasonality might become s=52, not 365.

Interpretation matters: a SARIMA model with both d=1 and D=1 is modeling changes plus seasonal changes. This can be appropriate (e.g., year-over-year growth dynamics), but it also increases the risk of over-differencing and wide forecast intervals.

In Python, the practical workhorse is statsmodels.tsa.statespace.SARIMAX, which covers ARIMA and SARIMA (and allows exogenous regressors, though we’ll focus on univariate first). A clean baseline looks like this:

from statsmodels.tsa.statespace.sarimax import SARIMAX model = SARIMAX(y, order=(p,d,q), seasonal_order=(P,D,Q,s), trend='c', enforce_stationarity=False, enforce_invertibility=False) res = model.fit(disp=False)

Several pitfalls show up repeatedly in real projects:

• Wrong or missing frequency: if your index lacks a consistent freq, forecasting dates can be misaligned. Set it explicitly (e.g., y = y.asfreq('MS') for month-start) after handling missing timestamps.
• Missing values: SARIMAX can handle some missingness, but large gaps often degrade estimation and diagnostics. Impute thoughtfully or resample to a frequency where gaps are meaningful.
• Unstable variance: if the scale increases over time, fit on log1p transformed data and invert forecasts later. Otherwise, residual diagnostics may scream “heteroskedastic” even when the model is conceptually correct.
• Convergence warnings: don’t ignore them. Try simpler orders, provide better starting values, or consider limiting parameters. Often, convergence issues mean the model is over-parameterized for the data length.
• Trend specification: trend='c' adds an intercept (mean). If you difference (d>0), the intercept behaves differently; a drift term may be appropriate depending on the series. Be explicit and compare.

After fitting, read the summary for both parameters and sanity: are AR/MA coefficients extremely close to ±1? Are standard errors huge? Are parameters insignificant but numerous? Those are hints the model is too complex or the data isn’t supporting the chosen structure.

Finally, keep leakage out of the process: choose differencing, transformations, and order-selection rules using only training windows within a walk-forward setup. It’s easy to “peek” by tuning orders on the full dataset and then reporting test performance; ARIMA will look better than it really is.

A fitted ARIMA/SARIMA model is only credible if its residuals behave like white noise: no remaining autocorrelation, roughly constant variance, and no systematic structure. Diagnostics tell you what to fix: differencing, seasonal terms, outliers, transformations, or simply a simpler model.

Start with residual autocorrelation. Use res.plot_diagnostics() as a fast overview, then be specific:

• ACF of residuals: significant spikes indicate missed AR/MA structure or missing seasonal components.
• Ljung–Box test (often included in diagnostics): tests whether groups of autocorrelations are jointly zero. If it rejects, your model has leftover temporal structure.

If you see residual seasonality (spikes at s), add or adjust seasonal AR/MA (P/Q) or seasonal differencing (D). If you see short-lag autocorrelation, adjust p and q modestly—don’t jump to large orders immediately.

Next, consider normality. Forecast intervals in SARIMAX are commonly based on approximate Gaussian assumptions. Real residuals may be skewed or heavy-tailed, especially with intermittent demand or promotional spikes. A Q–Q plot that bends at the tails warns that “95% intervals” may undercover or overcover. You can still use the model, but communicate that intervals are approximate and consider transformations (log), outlier handling, or alternative interval methods in later chapters.

Heteroskedasticity (changing variance) often appears as “fan-shaped” residuals over time. In business series, it’s frequently solved by modeling on a log scale. If variance changes abruptly due to a regime shift, consider splitting the history or adding intervention variables (with SARIMAX exogenous regressors) rather than forcing one stationary error process to explain everything.

Common mistake: tuning orders purely to make ACF look perfect while ignoring that the model becomes unstable or uninterpretable. Diagnostics are a tool for actionable fixes, not for chasing a cosmetically flat residual plot.

Order selection is where many teams lose discipline. Information criteria such as AIC and BIC are useful because they reward goodness of fit while penalizing complexity. However, they are still in-sample criteria and can prefer models that fit quirks that don’t persist. Your goal is a model that forecasts well under the evaluation design that matches your horizon.

A practical workflow is layered:

• Define a candidate grid with small orders (e.g., p,q in 0–3; P,Q in 0–2; d and D decided from stationarity checks).
• Fit and filter by convergence and sanity (exclude models with extreme parameters or failed optimization).
• Rank by AIC/BIC to shortlist a few parsimonious models.
• Validate with time-aware backtesting (walk-forward) using your business metric (MAE, RMSE, MAPE/sMAPE, pinball loss for quantiles, etc.).

BIC penalizes complexity more strongly than AIC; when data is limited, BIC often chooses simpler models that generalize better. Parsimony is not aesthetic—it reduces parameter uncertainty and typically tightens forecast stability. If two models validate similarly, pick the simpler one (fewer parameters, clearer interpretation, faster refits).

Time-aware validation is essential: do not random-split time series. Use rolling-origin evaluation where each fold trains on a prefix and forecasts the next h steps. For multi-step horizons, evaluate the full path, not only one-step-ahead. Many ARIMA configurations look great at 1-step but degrade quickly at 8- or 12-step horizons.

Common mistake: selecting p,d,q on the full dataset with auto_arima-like logic, then claiming unbiased performance on a held-out window. If you automate order search, embed it inside each backtest fold to reflect how you’d operate in production.

Once a SARIMA model is fitted, forecasting is straightforward in API terms, but you still need to choose a strategy that matches your horizon and operational constraints. SARIMAX naturally supports recursive multi-step forecasting: it uses the model to forecast step 1, then treats that forecast as input for step 2, and so on. This is efficient and consistent with the model’s structure, but errors can accumulate over long horizons.

The alternative is direct forecasting: fit separate models for each horizon (e.g., one model for t+1, another for t+2). Direct strategies can reduce error accumulation and sometimes improve long-horizon accuracy, but they are heavier operationally and less “classic ARIMA.” In this chapter’s ARIMA/SARIMA context, you’ll typically use recursive forecasts and rely on good model selection and diagnostics to keep propagation in check.

In statsmodels, you typically forecast with:

fc = res.get_forecast(steps=h) mean = fc.predicted_mean ci = fc.conf_int(alpha=0.05)

Intervals deserve careful interpretation. The returned confidence intervals reflect uncertainty from the estimated error variance and the state evolution under the model assumptions. They do not automatically include uncertainty from model selection, regime changes, future outliers, or future covariates (if any). For business communication, it’s often better to describe them as “model-based uncertainty intervals under historical dynamics.”

Two practical checks before shipping a forecast: (1) confirm the forecast index aligns with your expected future timestamps (a frequent bug when frequency is not set), and (2) if you transformed the data (log/Box-Cox), invert both the mean forecast and intervals appropriately. For log transforms, naive exponentiation can bias the mean; consider whether you need bias correction depending on your use case.

Finally, match forecasting to evaluation: if the business cares about a 12-week plan, validate 12-step forecasts with walk-forward folds and compare both point accuracy and interval coverage. This closes the loop: stationarity and differencing choices affect stability; orders affect residual structure; diagnostics affect trust; and the forecast strategy determines how those choices play out across the horizon.

Prophet models a time series as an additive (or multiplicative) combination of components: a piecewise trend, one or more seasonalities, and event effects. This structure is a strong assumption: it implies your series can be decomposed into stable patterns that repeat over time plus a trend that changes at a limited number of points. Prophet tends to work well for metrics like daily signups, orders, web traffic, demand, and revenue—especially when the business has clear weekly cycles, yearly seasonality, and known holiday spikes.

Start by checking whether your data matches Prophet’s expectations. Prophet expects a regular timestamp column ds and a numeric target y. Missing dates are allowed, but you should think about whether missing days mean “zero” (no activity) or “unknown” (data not collected). A common mistake is to drop missing dates for a daily series, which can distort weekly seasonality because the model sees an irregular calendar. In pandas, resample to the intended frequency first (e.g., daily), then fill gaps appropriately (zeros for count metrics; forward-fill only when it is logically valid).

When you fit a first model, immediately read the component plots: trend, weekly seasonality, yearly seasonality, and holidays (if present). Component plots are not decoration—they are a diagnostic. If the weekly component shows an implausible shape (e.g., huge Saturday lift for a B2B product), you likely have data issues, wrong timezone alignment, or leakage (e.g., target includes weekend batch postings). If the trend shows sharp kinks every few weeks, your changepoint settings may be too flexible and are fitting noise.

• Use Prophet when the series has interpretable calendar structure and you can name key events.
• Be cautious for series dominated by autocorrelation without clear seasonality (often better for ARIMA/XGBoost).
• Be cautious when your forecasting horizon is long but the business is undergoing regime shifts you cannot encode.

Finally, choose additive vs multiplicative seasonality. If seasonal swings grow with the level of the series (e.g., traffic doubles and weekend swings double), multiplicative often matches reality better. Otherwise, stick with additive for simplicity and interpretability.

Prophet’s trend is a piecewise linear (or logistic) function with potential changepoints—times where the slope can change. The model does not “discover” arbitrary change at any time; it places candidate changepoints in an initial portion of the history (by default) and then uses a prior to decide how much to use them. Your job is to control this flexibility so the forecast is stable out-of-sample.

The most important knob is changepoint_prior_scale. Larger values allow more slope changes (risking overfit); smaller values enforce a smoother trend (risking underfit). In practice, you tune this with backtesting, not by eyeballing the fit. A common workflow is to try a small grid (e.g., 0.01, 0.05, 0.1, 0.5) and compare cross-validated error metrics across the same cutoffs. Watch for “trend chasing”: models with very low in-sample error but unstable future projections.

You can also control changepoint_range, which limits the portion of history where changepoints are considered (e.g., 0.8 means the last 20% has no new changepoints). This is useful when you plan to forecast beyond the training window and want the recent tail to represent a stable regime, or when the last part contains partial data (late-arriving transactions) that you do not want the trend to bend around.

• If your metric has saturation (e.g., user base approaching a ceiling), use logistic growth with cap (and optionally floor).
• If you see sharp trend kinks near the end of training, reduce changepoint_prior_scale or shorten changepoint_range.
• If the model lags behind a known structural change (pricing, product launch), consider manually specifying changepoints or adding event regressors rather than increasing flexibility blindly.

Engineering judgment: treat changepoints as “explanations” you must defend. If you cannot tie trend breaks to business changes or long-term shifts, you are likely fitting noise. The goal is not a perfect historical fit; it is a forecast that holds up across walk-forward cutoffs.

Seasonality is where Prophet shines for many business series. Built-in weekly and yearly seasonalities can be enabled automatically, but you should still validate them. The weekly component should reflect operational reality (store hours, B2B weekdays, paydays). The yearly component should reflect true annual cycles (holidays, weather-driven demand, budgeting cycles). After fitting, component plots help you confirm the shape and magnitude are sensible.

Seasonality in Prophet is represented via Fourier series. The fourier_order controls how wiggly the seasonal curve can be: higher order captures sharper peaks and troughs but increases overfitting risk. For daily data, weekly seasonality often works well with modest order; yearly seasonality may need more order for complex annual patterns, but only if you have enough history (ideally multiple years). A common mistake is to increase Fourier order to “fix” holiday spikes—those should be modeled as holidays/events, not forced into smooth seasonality.

You can add custom seasonalities when the calendar has meaningful cycles that are not weekly or yearly. Examples include hourly seasonality for intraday data, monthly seasonality for billing cycles, or a 14-day cycle for biweekly behavior. Use add_seasonality(name=..., period=..., fourier_order=...) and validate with backtesting. If you add many seasonalities, be disciplined: each added component increases variance and can create misleading patterns in the component plot.

• Rule of thumb: increase complexity only when the model underfits across multiple CV cutoffs.
• Check for frequency mismatches (timezone shifts, daylight savings) that can distort daily/weekly patterns.
• Consider seasonality_mode='multiplicative' when seasonal amplitude scales with the level.

Practical outcome: with well-chosen seasonalities, you can explain forecast behavior to stakeholders (“Mondays are consistently lower; December lifts due to annual seasonality”) and reduce the temptation to hand-tune adjustments outside the model.

Holidays and events should be treated as first-class features, not left for the trend or seasonality to absorb. Prophet supports holiday effects via a holiday dataframe with columns like holiday, ds, and optional lower_window/upper_window to extend effects before and after the date. This matters because many events are not single-day impulses: promotions ramp up, shipping deadlines pull demand forward, and holidays have lead/lag behavior.

Build holiday tables programmatically and keep them versioned with your pipeline. A common mistake is to hardcode dates in notebooks, then forget to update them for the next year. Another common mistake is to include company-specific events without deciding whether they will occur in the future. If you add an “Annual Conference” holiday but you are not sure it will happen next year, you must treat it as a scenario input, not a fixed calendar truth.

Prophet also includes built-in country holidays (depending on the package version) via methods like add_country_holidays. This is convenient, but be cautious: not all holidays affect your metric, and some may matter only for certain regions. If your business spans geographies, you may need multiple holiday calendars or separate models per region.

• Use windows for multi-day events (e.g., Black Friday weekend: lower_window=-1, upper_window=2).
• Prefer explicit events for one-off shocks (site outage, major campaign) rather than letting trend bend.
• Validate holiday usefulness with CV: keep events that consistently improve holdout performance.

Component plots for holidays are particularly useful: if a “holiday effect” is estimated as negative when you expect positive, it can indicate overlapping events, misaligned dates (UTC vs local), or a target series that already includes compensating behavior (pull-forward vs cannibalization). Interpret these plots as hypotheses to investigate, not as unquestionable truth.

External regressors let Prophet incorporate covariates such as price, marketing spend, inventory, temperature, or product availability. Done correctly, regressors can improve accuracy and make forecasts more actionable (“if we increase spend, expected demand rises”). Done incorrectly, they introduce leakage and create forecasts that cannot be produced in production.

The key discipline is availability: at forecast time, do you know the regressor values for the entire horizon? Some regressors are known (calendar flags, planned price, scheduled campaign). Others are not (future organic traffic, realized spend, competitor actions). If a regressor is unknown, you must either (a) forecast it separately, (b) replace it with a planned scenario, or (c) exclude it. A classic leakage mistake is using same-day web traffic to forecast same-day orders when the goal is to forecast orders days ahead; that regressor contains information from the target period.

Implementation details matter. You add a regressor with add_regressor, then include the column in both training and the future dataframe used for prediction. You should also standardize or let Prophet standardize depending on scale; large magnitude regressors can dominate if not handled properly. For binary regressors (on/off), be explicit about how they extend into the future (planned promotions) and avoid training on “realized” flags that were only known after the fact.

• Use planned, controllable regressors for scenario forecasts (e.g., promo_on=1 for next month).
• Audit regressors for “future information”: anything computed using future windows (rolling means centered, full-month totals) is leakage.
• When regressors correlate with seasonality (e.g., spend increases on weekends), interpret effects cautiously due to confounding.

Practical outcome: regressors turn Prophet from a purely extrapolative model into a tool for decision support, but only if you design regressor pipelines that can be reproduced reliably at prediction time.

Prophet’s default fit can look convincing even when it fails in real forecasting. The antidote is time series cross-validation (CV) with multiple cutoffs. Prophet provides utilities to generate rolling-origin forecasts: you choose an initial training window, a step size (period), and a forecast horizon. This design matches real operations: train on past data, predict the next H days, then roll forward and repeat.

When you run CV, compare models fairly: same cutoffs, same horizon, and the same preprocessing (including how missing dates and outliers are handled). Examine metrics like MAE/MAPE/SMAPE, but also look at error by horizon (day 1 vs day 14), because some models are good short-term but drift long-term. If your business cares about weekly totals, also evaluate aggregated errors; daily point accuracy may be less important than weekly bias.

Diagnostics in Prophet go beyond a single metric. Use residual analysis: plot errors over time to detect regime shifts, and check whether errors correlate with holidays or promotions (suggesting missing event features). Inspect component stability: if the seasonal shape changes drastically when you vary changepoint priors, that is a sign the model is using seasonality to compensate for trend misfit (or vice versa). Prefer the simplest model that performs consistently across cutoffs.

Intervals are another critical diagnostic. Prophet produces uncertainty intervals for forecasts; treat them as model-based uncertainty, not guarantees. If intervals are unrealistically narrow, you may have underrepresented noise (e.g., heavy outliers removed too aggressively) or chosen overly rigid settings. If they are extremely wide, the model may be too flexible or the data may be nonstationary in ways Prophet cannot capture.

• Design CV with the real decision cadence (daily reforecasting vs weekly planning).
• Report both point metrics and coverage: how often actuals fall inside the predicted interval.
• Use scenario-style futures for intervals + plans: baseline spend vs high spend, holiday on/off, etc.

By the end of this chapter, you should be able to fit Prophet, read and critique component plots, tune trend/seasonality for stability, add holidays and regressors without leakage, and validate with rigorous time series CV—producing forecasts that hold up under the same conditions you will face in production.

XGBoost forecasting begins by converting a time series into a tabular dataset. Choose a forecast horizon h (e.g., 1 day ahead, 7 days ahead). For each timestamp t, you will build features using information available up through t, and set the target to the future value at t+h. In pandas this is typically a shift: y = series.shift(-h), aligned so the row at time t has label y(t+h). The most common mistake is shifting the wrong direction or dropping rows in a way that misaligns features and targets.

Equally important is the feature timestamp. Suppose you compute a 7-day rolling mean. If you use rolling(7).mean() at time t, you must ensure it only uses values up to t, not including the future. In pandas, rolling windows are trailing by default, but leakage can occur if you accidentally center the window (e.g., center=True) or if you compute rolling stats on a series after filling missing values using information from the future. Keep a strict pipeline: sort by time, compute features with trailing windows, then shift the target, then drop rows with missing labels.

When you resample (e.g., hourly to daily), define the aggregation boundary carefully. A “daily” value can mean midnight-to-midnight or business-day; choose what matches your operational decision point. If you forecast tomorrow’s total sales at end-of-day today, your training rows should be indexed at end-of-day. Misplaced cutoffs produce models that look accurate but are not deployable because the features are not available when you need them.

• Rule of thumb: label time = “what you want to know,” feature time = “what you know now.”
• Sanity check: pick a random row and manually verify every feature can be computed without looking past its timestamp.

This framing step is where leakage prevention begins; everything else depends on getting these alignments correct.

Once the dataset is framed, your model power comes from features that summarize recent history. Start with lags: y(t-1), y(t-7), y(t-14), etc. Lags encode autocorrelation and seasonality directly. For daily data, include weekly lags; for hourly data, include daily and weekly lags (24, 48, 168). Avoid adding dozens of redundant lags blindly; pick lags that reflect known cycles and operational memory.

Next add rolling statistics to capture local level and variability: rolling mean, rolling median (often more robust), rolling standard deviation, min/max, and quantiles. Example: a 7-day rolling median can stabilize noisy demand. Be careful to compute these on the raw series prior to any target shift, and ensure windows are trailing. If outliers are common, consider winsorizing or using robust statistics (median, IQR) rather than only mean/std.

EWMA (exponentially weighted moving average) is a strong baseline feature because it responds faster to level changes than a long rolling mean while still smoothing noise. In pandas: series.ewm(span=14, adjust=False).mean(). Use multiple spans (short/medium/long) to give the model options. With tree models, you can also include differences like y(t-1) - y(t-7) or percent change; these help detect momentum or regime shifts.

Finally, consider interactions that reflect domain logic. For example, if you have price or promotion flags, interact them with recent demand features (e.g., promo × rolling_mean_7) to let the model learn “promo lift depends on baseline.” XGBoost can learn many interactions implicitly, but providing a few meaningful engineered interactions can improve sample efficiency and stability.

• Common mistake: computing rolling features on a train+test concatenation and then splitting. Always compute features on the full chronological series but evaluate with time-based splits; leakage occurs only if a feature at time t used values after t. Confirm this explicitly.
• Practical outcome: a feature set that captures level, seasonality, variability, and momentum in a way trees can exploit.

Lag and rolling features summarize the past; calendar features explain predictable structure that repeats even when the recent past is uninformative (e.g., after outages or sparse data). At minimum, add day-of-week and month (and often week-of-year). For hourly data, add hour-of-day. These can be encoded as integers or one-hot encoded; with XGBoost, one-hot often works well for small cardinalities, while integer encoding can be sufficient if splits can isolate categories cleanly.

Include is_weekend, is_month_start, is_month_end, and business-specific markers like fiscal periods or paydays if they affect behavior. For holiday effects, you can use a holiday calendar (country or region) and create flags such as is_holiday, days_to_holiday, and days_after_holiday. This helps the model learn lead/lag effects (shopping ramps up before a holiday; returns spike after).

Be explicit about time zones and daylight saving transitions. If your timestamps are localized, an “hour-of-day” feature may jump or repeat; decide whether to normalize to UTC or handle DST explicitly. For daily aggregates, ensure the day boundary matches the business definition (e.g., store day vs calendar day). If you get this wrong, the model may appear to learn a weekly pattern that is really an artifact of misaligned days.

Categorical exogenous variables (store_id, region, product category) can be included if you are forecasting multiple series. For pure single-series forecasting, focus on calendar and event features that are known in advance at prediction time. A strong practice is to separate features into known-future (calendar/holidays) and observed-to-date (lags/rollings). That separation will matter in multi-step forecasting strategies where you may not have future observations for recursive predictions.

With a supervised dataset in hand, train XGBoost as you would for tabular regression—but with time-aware validation. Use objective='reg:squarederror' for standard regression, or consider reg:pseudohubererror for robustness to outliers. If your target is strictly positive and highly skewed, a log transform (train on log1p(y), invert with expm1) often stabilizes training and improves relative error metrics.

Regularization is essential because lag/rolling features are highly correlated. Key controls include max_depth (or max_leaves), min_child_weight, subsample, colsample_bytree, and reg_lambda/reg_alpha. In forecasting, prefer slightly smaller trees with more boosting rounds rather than very deep trees that memorize local fluctuations. A typical starting point is depth 3–6 with subsampling (0.7–0.9) and column sampling (0.7–1.0).

Validation must respect time order. Do not use random splits. Use a holdout at the end or, better, walk-forward backtesting where you repeatedly train on an expanding (or rolling) window and validate on the next block. Early stopping should monitor an error metric on the validation block and stop when improvement stalls (e.g., 50 rounds). This prevents overfitting and gives a principled way to choose n_estimators without peeking at the test set.

• Common mistake: performing standard k-fold CV with shuffling, which leaks future patterns into training folds.
• Engineering judgement: choose a validation window that matches operational risk—if the business cares about next 28 days, validate on contiguous 28-day blocks, not scattered points.

Finally, keep a simple baseline (seasonal naive, EWMA, or Prophet) in your backtests. XGBoost should earn its complexity by consistently beating baselines under the same evaluation design.

Real forecasting rarely stops at one-step-ahead. If you need predictions for horizons 1…H, you must choose a multi-step strategy that matches both your feature availability and evaluation. The three common patterns are direct, recursive, and multioutput.

Direct forecasting trains one model per horizon: a model for h=1, another for h=7, etc. Each model uses the same feature timestamp t but a different shifted target y(t+h). This avoids error accumulation and is usually strongest when you can afford multiple models. It also aligns naturally with business metrics that weight certain horizons more (e.g., next-week staffing vs next-quarter planning).

Recursive forecasting trains a one-step model and then feeds its own predictions back as future lags to forecast further out. It is computationally cheap but can drift when errors compound. Recursive approaches require careful separation of features: once you move beyond t+1, any lag that depends on unknown future actuals must be replaced by predicted values. This often reduces accuracy unless the series is smooth and the model is stable.

Multioutput trains a single model to predict a vector of horizons at once (commonly via wrappers or by reformulating as multiple targets). It can learn shared structure across horizons but may be harder to tune and interpret. In practice, many teams prefer the direct approach for clarity and control.

Whichever pattern you choose, align evaluation. If you deploy weekly forecasts for the next 8 weeks, your backtest should simulate that: at each origin date, generate an 8-step forecast using only data available at origin, score each horizon separately, and then aggregate with a business-weighted function (e.g., higher weight on weeks 1–2). A frequent error is to report a single metric computed on a pooled set of horizons without checking that performance degrades gracefully as horizon increases.

XGBoost models are often criticized as “black boxes,” but you can extract practical explanations that help debugging and stakeholder trust. Start with built-in feature importance. Gain measures how much a feature improves splits across the trees; it is fast but can overemphasize features with many split opportunities or correlated groups (lags and rollings often share credit unpredictably). Weight (split count) is even less reliable for forecasting decisions.

Permutation importance is a more honest test: shuffle one feature in a validation set (time-respecting, not training), re-score the model, and measure performance drop. This reflects how dependent the model is on that feature for that period. It can be slow, and correlated features can mask each other (shuffling one lag may not hurt if another lag substitutes), but it is a strong sanity check for leakage: if a suspicious “future-looking” feature dominates permutation importance, investigate immediately.

SHAP (Shapley Additive Explanations) provides local explanations per prediction, attributing each forecast’s deviation from a baseline to individual features. In time series forecasting, SHAP is especially useful for answering: “Did the model predict higher because the last 7 days trended up, or because it’s a holiday week?” Use summary plots to see global drivers, and dependence plots to diagnose nonlinear thresholds (e.g., demand spikes when rolling_std exceeds a level).

• Workflow tip: interpret on the same walk-forward validation blocks used for scoring; explanations on the training period can hide instability.
• Practical outcome: you can justify feature choices, detect leakage, and refine features when explanations show spurious drivers (like ID columns or post-event signals).

Interpretability is not just for presentations. In forecasting, it is a debugging tool: if the model relies heavily on a feature that should not be predictive at decision time, you have found a pipeline flaw before it reaches production.

Walk-forward backtesting creates multiple “mini live runs.” For each cutoff time t, you train on data up to t and forecast the next H steps. Repeat across many cutoffs to measure performance stability. Two windowing choices matter: expanding and sliding windows.

Expanding window training grows over time (train on all history up to each cutoff). This often matches production when you keep all data and retrain periodically. It is robust for stable series and helps ARIMA/Prophet learn longer seasonal structure. The downside is regime changes: very old data can dilute recent patterns.

Sliding window (fixed-length) training uses only the most recent L observations. This can improve responsiveness when demand shifts or pricing changes. It is common for XGBoost feature models because lags/rolls already summarize recent behavior; older history may add noise. The trade-off is losing long seasonal cycles unless you include explicit calendar features.

• ARIMA/SARIMA: refit at each cutoff (or update if supported). Use consistent differencing and seasonal orders; avoid looking at future residual diagnostics.
• Prophet: refit per cutoff with the same holiday table and seasonality settings; keep the “future dataframe” strictly after the cutoff.
• XGBoost: rebuild features at each cutoff (lags, rolling means) using only past data; enforce this by generating features inside the backtest loop.

Common mistakes: (1) creating lag/rolling features once on the full dataset (leakage), (2) tuning hyperparameters on the same folds you report as final (optimistic bias), and (3) mixing different retraining cadences across models (unfair comparison). A practical default is: monthly cutoffs for daily data, weekly cutoffs for hourly data, and always include at least one full seasonal cycle in the backtest span.

Model comparison fails most often because forecasts are not aligned. “One-step ahead” error is not the same problem as “next 30 days” error. Before computing metrics, reconcile frequency, cutoff timestamps, and horizons so every model is evaluated on identical targets.

Frequency alignment: Decide the business reporting grain (e.g., daily). If the raw data is irregular or higher frequency, resample first and make missingness rules explicit (sum vs mean, zero-fill vs NA). Prophet and ARIMA assume a regular time index; XGBoost features also depend on consistent spacing. If you aggregate weekly, do not evaluate against daily truths.

Cutoff alignment: Define when the forecast is “issued.” If operations freeze orders at 5pm, your cutoff should be 5pm data, not end-of-day totals. In pandas, treat this as a strict slice: training data ends at cutoff inclusive, targets start at cutoff+1 step.

Horizon alignment: Evaluate per horizon step (h=1…H), not only the average. Many models look good at short horizons and degrade differently at long horizons. For multi-step forecasts: ARIMA/Prophet naturally produce a path; XGBoost may be direct (one model per horizon) or recursive (feeding predictions back). Compare them at the same horizon definition and document the strategy.

• Metric granularity: compute metrics per cutoff and per horizon, then aggregate (mean/median) to avoid one bad month hiding in the average.
• Time zone and calendar: ensure all series share time zone handling and holiday definitions; mismatches can look like “model error.”
• Target transformations: if you log-transform for XGBoost, invert before metric calculation; do not compare on different scales.

Practical outcome: once alignment is enforced, your leaderboard becomes meaningful. Without reconciliation, the “best” model is often just the one evaluated under easier conditions.

Metrics tell you who won; error analysis tells you why. After backtesting, build a small diagnostic pack: residual plots over time, residuals vs fitted values, and seasonal breakdowns (by day-of-week, month, holiday vs non-holiday). You are looking for structure—because structure implies fixable model bias.

Residual over time: Plot backtest residuals (y − ŷ) by timestamp and by cutoff. Drift from near-zero to consistently positive/negative indicates regime change or missing drivers. For ARIMA/SARIMA, also review ACF/PACF of residuals on training windows; autocorrelation suggests under-modeled seasonality or differencing issues. For Prophet, inspect component plots: trend saturation, weekly seasonality amplitude, and holiday spikes. For XGBoost, analyze errors against key features (e.g., promo flag, rolling mean) to find interactions you are not capturing.

Seasonal breakdown: Group errors by seasonal buckets. If Monday errors are consistently negative, you may need weekday seasonality (Prophet) or weekday one-hot features (XGBoost) or seasonal AR terms (SARIMA). If errors spike at month-end, add calendar features like “is_month_end” or include multiplicative seasonality.

• Stability checks: compare metric distributions across folds; a model that occasionally collapses may be riskier than a slightly worse but stable challenger.
• Outlier sensitivity: identify whether a few extreme events dominate MAE/RMSE; consider robust metrics or capped losses for selection.
• Bias vs variance: ARIMA may be biased under structural breaks; XGBoost may overfit if leakage or overly rich features exist.

Common mistake: only inspecting aggregate metrics and concluding “XGBoost wins.” A practical workflow is to produce three plots per model: (1) actual vs forecast for representative folds, (2) residuals by time, (3) error by season bucket. These usually reveal the next engineering move faster than another round of hyperparameter tuning.

Point forecasts are incomplete in operational settings. Safety stock, staffing buffers, and SLA planning require uncertainty. Your goal is not just to output intervals, but to ensure they are calibrated: a 90% interval should contain the true value about 90% of the time (under backtest conditions).

Evaluate coverage and width: For each horizon, compute empirical coverage (fraction of truths within [lower, upper]) and average width. Good intervals balance the two: high coverage with unnecessarily wide intervals is not useful; narrow intervals with poor coverage are misleading. Always report coverage per horizon because uncertainty typically grows with lead time.

Model-specific interval sources: SARIMA can produce analytical prediction intervals under assumptions; Prophet provides intervals via uncertainty sampling; XGBoost needs an explicit strategy (quantile regression, distributional assumptions on residuals, or conformal prediction). Regardless of method, evaluate the resulting intervals on the same backtest folds used for point accuracy.

• Calibration strategies:
  • Scale residuals: if coverage is low, inflate interval width using backtest residual quantiles per horizon.
  • Quantile models: train XGBoost with quantile loss for P10/P50/P90 and validate coverage.
  • Conformal prediction: compute nonconformity scores on recent folds and wrap any model’s point forecast with data-driven bands.
• Asymmetry: if under-forecasting is costlier, consider asymmetric intervals or optimize pinball loss at a higher quantile.

Common mistakes: calibrating on the test period (leakage), reporting one global coverage number (hides horizon failure), and ignoring conditional miscalibration (intervals fail specifically on holidays or promotions). A practical outcome is an interval report that operations can trust, with clear trade-offs: “90% coverage at h=14 with average width of 120 units.”

Ensembles often outperform single models because different families fail differently. ARIMA may excel in short-term autocorrelation, Prophet may capture holiday seasonality, and XGBoost may leverage complex calendar/lag interactions. The simplest ensemble—an unweighted average of forecasts—can be a strong baseline.

Simple averages: Average the point forecasts across your top candidates per horizon. This reduces variance and is surprisingly hard to beat. Ensure all forecasts are aligned (same frequency and horizon) and consider averaging on the natural scale (after inverse transforms).

Weighted blends: Use backtest performance to assign weights, e.g., inverse-MAE weights per horizon. Weights should be learned on a validation backtest set and then frozen for the final evaluation to avoid overfitting. If your business cares most about long horizons, compute weights with horizon-aware scoring.

Stacking cautions: Training a meta-model to combine forecasts can overfit because fold count is limited and forecasts are correlated. If you stack, generate out-of-fold predictions properly (meta-model never sees in-fold forecasts) and keep the meta-model simple (ridge regression is often sufficient). Avoid leaking future information by stacking on the same time slices used to train base models.

• Intervals for ensembles: do not average lower/upper bounds blindly unless you validate coverage; consider conformal wrapping on the ensemble point forecast.
• Champion–challenger: treat the ensemble as a challenger if it is harder to operate; require clear, stable gains across folds.
• Operational simplicity: a slightly worse single model may win if it is easier to retrain, explain, and monitor.

Practical outcome: a shortlist with (1) best single model, (2) best simple ensemble, and (3) a conservative fallback. This sets you up for a controlled promotion process rather than a one-time “winner.”

A model is “selected” only when it can be operated. Production readiness means you can retrain it on schedule, generate forecasts on time, detect when it degrades, and roll back safely. Build a lightweight checklist and monitoring plan alongside your backtests.

Retraining cadence: Decide how often to refit (daily/weekly/monthly) based on volatility and cost. Backtest under the same cadence: if you will retrain weekly, do weekly cutoffs. ARIMA/Prophet refits may be slower but straightforward; XGBoost retrains fast but requires feature pipeline correctness. Document training window length (expanding vs sliding) and keep it consistent.

Drift and degradation: Monitor both data drift (changes in input distributions like weekday mix, missingness rate, promo frequency) and performance drift (recent forecast error). Use a rolling backtest-like evaluation on the most recent periods once ground truth arrives. Track per-horizon MAE/MAPE and bias (mean error) because systematic bias is often the earliest failure signal.

• Alerting signals:
  • Accuracy: MAE or WAPE over the last N days exceeds a threshold relative to backtest baseline.
  • Bias: mean error deviates from zero for consecutive periods (persistent under-forecasting).
  • Coverage: 90% interval coverage drops materially (miscalibration).
  • Data quality: missing timestamps, sudden zeros, or outlier spikes beyond expected bounds.
• Champion–challenger process: keep a challenger trained in parallel (e.g., a simpler seasonal-naive or a new feature set). Promote only after it wins on a defined backtest window and passes stability and interval checks.
• Rollback plan: store model artifacts, code version, and feature definitions; ensure you can revert to the previous champion quickly.

Common mistakes: changing the pipeline without re-baselining metrics, monitoring only one aggregate KPI, and ignoring data issues that masquerade as model drift. The practical outcome is a forecasting system that is measurable, auditable, and resilient—where selection is not an event, but an ongoing, controlled practice.