Time Series Bootstrapping

MBB, CBB, Stationary Bootstrap & Parametric AR(1) Bootstrap

Created by Dr. Pedram Jahangiry | Enhanced with Claude

Website YouTube GitHub

⚠ Why Not Standard (i.i.d.) Bootstrap?

Standard bootstrap draws observations independently with replacement — this destroys the temporal dependence structure that defines time series data. Autocorrelations, trends, and seasonal patterns would be completely lost. Time series bootstrapping must preserve the local dependence by resampling in blocks or through model-based approaches.

Moving Block Bootstrap (MBB)

The MBB (Kunsch, 1989; Liu & Singh, 1992) samples overlapping blocks of fixed size m from the original series. Each block preserves the local dependence structure within it. Blocks are drawn uniformly from the T − m + 1 possible starting positions, then concatenated and trimmed to length T.

  • Block library: Bi = [yi, yi+1, ..., yi+m−1] for i = 1, ..., T−m+1
  • Draw k = ⌈T/m⌉ blocks uniformly with replacement
  • Concatenate and trim to length T
  • For non-stationary data: decompose into Trend + Seasonal + Residual, apply MBB to the residuals, then reconstruct
12
5

Key Insight — Edge Bias

Observations near the boundaries (y1 and yT) appear in fewer possible blocks than observations in the middle. For example, y1 only appears in B1, while a middle observation yt appears in up to m different blocks. This creates an edge bias where boundary observations are undersampled — a limitation addressed by CBB.

Circular Block Bootstrap (CBB)

The CBB (Politis & Romano, 1992) wraps the series into a circle so that yT+1 = y1, yT+2 = y2, etc. This creates exactly T possible blocks of any size m (vs. T−m+1 for MBB), ensuring all observations are sampled with equal probability.

  • Circular extension: ytcirc = y((t−1) mod T) + 1
  • Block library: T blocks (all starting positions valid)
  • Eliminates edge bias: every observation appears in exactly m blocks
12
5

Key Insight — Circular Wrap

By connecting yT back to y1, the CBB ensures that observations near the series boundaries have equal representation. This eliminates the edge bias of MBB. The trade-off is that the wrap-around junction is artificial — the connection from the end back to the start may not reflect genuine temporal dynamics, especially for non-seasonal data.

Stationary Bootstrap (SB)

The SB (Politis & Romano, 1994) uses random block lengths drawn from a Geometric(p) distribution with mean block length 1/p. At each step, with probability p a new random block begins, and with probability 1−p the current block continues (with circular wrapping).

  • Block length: L ~ Geometric(p), so E[L] = 1/p
  • The resampled series is itself stationary (unique property!)
  • No edge bias: all observations equally likely at any position
  • Less sensitive to the choice of mean block length than MBB/CBB are to fixed m
12
5

Key Insight — Stationarity Preservation

The SB is the only non-parametric block bootstrap that produces a stationary resampled series when the original is stationary. The random block lengths drawn from a memoryless Geometric distribution ensure no systematic position-dependent sampling patterns. MBB and CBB bootstrap distributions are not stationary even when applied to stationary data.

Parametric Bootstrap — Decomposition + AR(1)

The parametric bootstrap uses the same classical decomposition as the block methods (Trend + Seasonal + Residual on log data), but instead of resampling residuals in blocks, it fits an AR(1) model to capture the autocorrelation structure, then resamples the AR(1) innovations i.i.d. This is model-based rather than block-based.

  • Step 1: Decompose log(Y) = Trend + Seasonal + Residual (same as block methods)
  • Step 2: Fit AR(1) to residuals: rt = φ·rt−1 + εt
  • Step 3: Extract & center innovations ε̂t
  • Step 4: Resample innovations i.i.d. → ε*t, reconstruct r* via AR(1)
  • Step 5: Reconstruct: Y* = exp(Trend + Seasonal + r*)
5

Key Insight — Model Dependence

The parametric bootstrap is the most efficient when the model is correctly specified — it generates smooth, model-consistent paths without block-boundary discontinuities. However, if the ARIMA model is misspecified, the bootstrap paths will not faithfully represent the true data generating process. Block methods (MBB, CBB, SB) are model-free and don't suffer from this risk, making them more robust but less efficient.

Comparison of Methods

Property MBB CBB Stationary Bootstrap Parametric (ARIMA)
Block length Fixed m Fixed m Random ~ Geom(p) N/A (model-based)
Possible blocks T − m + 1 T Random N/A
Edge bias Yes No No No
Resampled series stationary No No Yes Yes (if model correct)
Equal obs. probability No Yes Yes Yes
Model-free Yes Yes Yes No
Block boundary artifacts Yes Yes Yes (smoothed) No
Can generate forecasts No No No Yes