ETC3550: Applied
forecasting for
business and
economics
Ch11. Advanced forecasting methods
[Link]/fpp2/
1
Outline
1 Complex seasonality
2 Vector autoregression
3 Neural network models
4 Bootstrapping and bagging
2
Examples
Weekly US finished motor gasoline products
Thousands of barrels per day
1990 1995 2000 2005 2010 2015
Year
3
Examples
5 minute call volume at North American bank
Call volume
400
300
200
100
0
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33
Weeks
400
Call volume
300
200
100
1 2 3 4
Weeks
4
Examples
Turkish daily electricity demand
25
Electricity Demand (GW)
20
15
10
2000 2002 2004 2006 2008
Year
5
TBATS model
TBATS
Trigonometric terms for seasonality
Box-Cox transformations for heterogeneity
ARMA errors for short-term dynamics
Trend (possibly damped)
Seasonal (including multiple and
non-integer periods)
6
TBATS model
yt = observation at time t
(y ω− 1)/ω if ω =6 0;
t
yt(ω) =
log yt if ω = 0.
M
yt(ω) = `t−1 + φbt−1 + (i)
X
st−mi + dt
i=1
`t = `t−1 + φbt−1 + αdt
bt = (1 − φ)b + φbt−1 + βdt
p
X q
X
dt = φi dt−i + θj εt−j + εt
i=1 j=1
ki s(i)
j,t =
(i)
sj,t−1 cos λj(i) + s∗(i) (i) (i)
j,t−1 sin λj + γ1 dt
s(i) s(i)
X
t = j,t ∗(i)
j=1 s(i) (i) (i) (i) (i)
j,t = −sj,t−1 sin λj + sj,t−1 cos λj + γ2 dt
7
TBATS model
yt = observation at time t
(y ω− 1)/ω
t if ω =6 0; Box-Cox transformation
yt(ω) =
log yt if ω = 0.
M
yt(ω) = `t−1 + φbt−1 + (i)
X
st−mi + dt
i=1
`t = `t−1 + φbt−1 + αdt
bt = (1 − φ)b + φbt−1 + βdt
p
X q
X
dt = φi dt−i + θj εt−j + εt
i=1 j=1
ki s(i)
j,t =
(i)
sj,t−1 cos λj(i) + s∗(i) (i) (i)
j,t−1 sin λj + γ1 dt
s(i) s(i)
X
t = j,t ∗(i)
j=1 s(i) (i) (i) (i) (i)
j,t = −sj,t−1 sin λj + sj,t−1 cos λj + γ2 dt
7
TBATS model
yt = observation at time t
(y ω− 1)/ω
t if ω =6 0; Box-Cox transformation
yt(ω) =
log yt if ω = 0.
M
yt(ω) = `t−1 + φbt−1 + (i)
st−mi + dt M seasonal periods
X
i=1
`t = `t−1 + φbt−1 + αdt
bt = (1 − φ)b + φbt−1 + βdt
p
X q
X
dt = φi dt−i + θj εt−j + εt
i=1 j=1
ki s(i)
j,t =
(i)
sj,t−1 cos λj(i) + s∗(i) (i) (i)
j,t−1 sin λj + γ1 dt
s(i) s(i)
X
t = j,t ∗(i)
j=1 s(i) (i) (i) (i) (i)
j,t = −sj,t−1 sin λj + sj,t−1 cos λj + γ2 dt
7
TBATS model
yt = observation at time t
(y ω− 1)/ω
t if ω =6 0; Box-Cox transformation
yt(ω) =
log yt if ω = 0.
M
yt(ω) = `t−1 + φbt−1 + (i)
st−mi + dt M seasonal periods
X
i=1
`t = `t−1 + φbt−1 + αdt global and local trend
bt = (1 − φ)b + φbt−1 + βdt
p
X q
X
dt = φi dt−i + θj εt−j + εt
i=1 j=1
ki s(i)
j,t =
(i)
sj,t−1 cos λj(i) + s∗(i) (i) (i)
j,t−1 sin λj + γ1 dt
s(i) s(i)
X
t = j,t ∗(i)
j=1 s(i) (i) (i) (i) (i)
j,t = −sj,t−1 sin λj + sj,t−1 cos λj + γ2 dt
7
TBATS model
yt = observation at time t
(y ω− 1)/ω
t if ω =6 0; Box-Cox transformation
yt(ω) =
log yt if ω = 0.
M
yt(ω) = `t−1 + φbt−1 + (i)
st−mi + dt M seasonal periods
X
i=1
`t = `t−1 + φbt−1 + αdt global and local trend
bt = (1 − φ)b + φbt−1 + βdt
p q
X X ARMA error
dt = φi dt−i + θj εt−j + εt
i=1 j=1
ki s(i)
j,t =
(i)
sj,t−1 cos λj(i) + s∗(i) (i) (i)
j,t−1 sin λj + γ1 dt
s(i) s(i)
X
t = j,t ∗(i)
j=1 s(i) (i) (i) (i) (i)
j,t = −sj,t−1 sin λj + sj,t−1 cos λj + γ2 dt
7
TBATS model
yt = observation at time t
(y ω− 1)/ω
t if ω =6 0; Box-Cox transformation
yt(ω) =
log yt if ω = 0.
M
yt(ω) = `t−1 + φbt−1 + (i)
st−mi + dt M seasonal periods
X
i=1
`t = `t−1 + φbt−1 + αdt global and local trend
bt = (1 − φ)b + φbt−1 + βdt
p q
X X ARMA error
dt = φi dt−i + θj εt−j + εt
i=1 j=1
ki s(i)
j,t =
(i)
sj,t−1 λj(i) + s∗(i)
cosFourier-like sin λ(i)
j,t−1 seasonal
(i)
j + γterms
1 dt
s(i) s(i)
X
t = j,t ∗(i)
j=1 s(i) (i) (i) (i) (i)
j,t = −sj,t−1 sin λj + sj,t−1 cos λj + γ2 dt
7
TBATS model
yt = observation at time t
(y ω− 1)/ω
t if ω =6 0; Box-Cox transformation
yt(ω) =
TBATS
log yt if ω = 0.
Trigonometric M
M seasonal periods
yt(ω) = `t−1 + φbt−1 + (i)
X
st−m + dt
Box-Cox i=1 i
`t = `t−1 + φbt−1 + αdt global and local trend
ARMA
bt = (1 − φ)b + φbt−1 + βdt
p Trend X q
X ARMA error
dt = φi dt−i + θj εt−j + εt
i=1Seasonal j=1
ki s(i)
j,t =
(i)
sj,t−1 λj(i) + s∗(i)
cosFourier-like sin λ(i)
j,t−1 seasonal
(i)
j + γterms
1 dt
s(i) (i)
X
t = sj,t ∗(i)
j=1 s(i) (i) (i) (i) (i)
j,t = −sj,t−1 sin λj + sj,t−1 cos λj + γ2 dt
7
Complex seasonality
gasoline %>% tbats() %>% forecast() %>% autoplot()
Forecasts from TBATS(1, {0,0}, −, {<52.18,12>})
10
level
80
.
8
95
1990 2000 2010 2020
Time
8
Complex seasonality
calls %>% tbats() %>% forecast() %>% autoplot()
Forecasts from TBATS(0.555, {0,0}, −, {<169,6>, <845,4>})
800
600
level
400 80
.
95
200
0 10 20 30
Time
9
Complex seasonality
telec %>% tbats() %>% forecast() %>% autoplot()
Forecasts from TBATS(0.005, {4,2}, −, {<7,3>, <354.37,7>, <365.25,3>})
25
level
20
80
.
95
15
10
2000 2004 2008
Time
10
TBATS model
TBATS
Trigonometric terms for seasonality
Box-Cox transformations for heterogeneity
ARMA errors for short-term dynamics
Trend (possibly damped)
Seasonal (including multiple and non-integer periods)
Handles non-integer seasonality, multiple seasonal
periods.
Entirely automated
Prediction intervals often too wide
11
Very slow on long series
Outline
1 Complex seasonality
2 Vector autoregression
3 Neural network models
4 Bootstrapping and bagging
12
Outline
1 Complex seasonality
2 Vector autoregression
3 Neural network models
4 Bootstrapping and bagging
13
Neural network models
Simplest version: linear regression
Input Output
layer layer
Input #1
Input #2
Output
Input #3
Input #4
14
Neural network models
Simplest version: linear regression
Input Output
layer layer
Input #1
Input #2
Output
Input #3
Input #4
Coefficients attached to predictors are called “weights”.
Forecasts are obtained by a linear combination of inputs.
Weights selected using a “learning algorithm” that
14
minimises a “cost function”.
Neural network models
Nonlinear model with one hidden layer
Input Hidden Output
layer layer layer
Input #1
Input #2
Output
Input #3
Input #4
15
Neural network models
Nonlinear model with one hidden layer
Input Hidden Output
layer layer layer
Input #1
Input #2
Output
Input #3
Input #4
A multilayer feed-forward network where each layer
of nodes receives inputs from the previous layers.
15
Inputs to each node combined using linear
Neural network models
Inputs to hidden neuron j linearly combined:
4
X
z j = bj + wi,j xi .
i=1
Modified using nonlinear function such as a sigmoid:
1
s(z) = ,
1 + e−z
This tends to reduce the effect of extreme input
values, thus making the network somewhat robust to
outliers.
16
Neural network models
Weights take random values to begin with, which
are then updated using the observed data.
There is an element of randomness in the
predictions. So the network is usually trained
several times using different random starting
points, and the results are averaged.
Number of hidden layers, and the number of
nodes in each hidden layer, must be specified in
advance.
17
NNAR models
Lagged values of the time series can be used as
inputs to a neural network.
NNAR(p, k): p lagged inputs and k nodes in the
single hidden layer.
NNAR(p, 0) model is equivalent to an
ARIMA(p, 0, 0) model but without stationarity
restrictions.
Seasonal NNAR(p, P, k): inputs
(yt−1 , yt−2 , . . . , yt−p , yt−m , yt−2m , yt−Pm ) and k
neurons in the hidden layer.
NNAR(p, P, 0)m model is equivalent to an
ARIMA(p, 0, 0)(P,0,0)m model but without
18
stationarity restrictions.
NNAR models in R
The nnetar() function fits an NNAR(p, P, k)m
model.
If p and P are not specified, they are
automatically selected.
For non-seasonal time series, default p = optimal
number of lags (according to the AIC) for a linear
AR(p) model.
For seasonal time series, defaults are P = 1 and p
is chosen from the optimal linear model fitted to
the seasonally adjusted data.
Default k = (p + P + 1)/2 (rounded to the nearest
integer). 19
Sunspots
Surface of the sun contains magnetic regions
that appear as dark spots.
These affect the propagation of radio waves and
so telecommunication companies like to predict
sunspot activity in order to plan for any future
difficulties.
Sunspots follow a cycle of length between 9 and
14 years.
20
NNAR(9,5) model for sunspots
fit <- nnetar(sunspotarea)
fit %>% forecast(h=20) %>% autoplot()
Forecasts from NNAR(9,5)
3000
2000
sunspotarea
1000
1900 1950 2000
Time 21
Prediction intervals by simulation
fit %>% forecast(h=20, PI=TRUE) %>% autoplot()
Forecasts from NNAR(9,5)
3000
2000
sunspotarea
level
80
95
1000
1900 1950 2000
Time
22
Outline
1 Complex seasonality
2 Vector autoregression
3 Neural network models
4 Bootstrapping and bagging
23
