Read and tidy up the data using the code provided.
Your aim in the first part of the assignment is to build an ARIMA model and use this to forecast. Recall the first step in ARIMA modelling is to stabilize the variance of your data. If you decided that your data required a transformation in the previous assignments you will be required to use the same transformation for what follows (unless you have a reason to change your mind - please do so if you think it is necessary).
Question 1
Visually inspect your data and decide on the transformation and what differencing is required to achieve stationarity. Plot the data at every step and comment on each plot justifying your actions. (No more than 50 words per plot). (6 marks)
myts |>autoplot(y) +ggtitle("Add you title here") +ylab("add y label")
Warning: Removed 13 rows containing missing values or values outside the scale range
(`geom_line()`).
Warning: Removed 13 rows containing missing values or values outside the scale range
(`geom_point()`).
Marking guide:
Transformation and discussion (2m)
If not needed, explain
Appropriate plots and explanation
Original plot (1m)
Explain what action needs to be taken to achieve stationarity (1m)
Stationary plot (1m)
Explain whether the series is stationary or not by providing the discussion on ACF (possibly unit root test), constant mean or any predictable pattern (1m)
Expectations
Transformation (or appropriate discussion)
Appropriate plots
Appropriate justification of differences
Common errors:
Fail to discuss the plot in each step.
Some students decided that only one seasonal difference was required, although a first order difference was still required due to a slowly decaying ACF() plot. (FOR US - LETS NOT BE TOO HARSH ON THIS - IF THEY JUSTIFY AND THEY POSSIBLE REVISE FURTHER BELOW IT IS OK - BUT IF CLEARLY NEEDING ANOTHER DIFFERENCE THAT IS FINE - PENALISE)
Question 2
Plot the ACF and PACF of the stationary data. Reading from these choose an appropriate ARIMA model. Make sure you justify your choice. (No more than 70 words in total – do not revise the theory – describe what you see in your plots and decide what ARIMA orders may be appropriate. Also note that it is highly likely that the ACF and PACF plots will be very messy. Do the best you can). (6 marks)
Oh yes very messy. I would go for an AR(2) due to the two non-seasonal spikes in the PACF and a seasonal MA(2) due to the two spikes at 12 and 24 in the ACF. So the suggested model is an ARIMA(2,1,0)(0,1,2)[12]
Marking guide:
ACF and PACF plot for stationary data (1m)
Identification of any AR/SAR or MA/SMA term with explanation (5m)
Expectation I. Appropriate plots provided II. Identification of potential models from the plots III. Graph based decision of most appropriate model
Common errors:
Did not identifying the appropriate model based on the plot, instead opting to select a model based on AICc.
Did not specify the ARIMA model based on the ACF/PACF plot but just briefly explain there were spikes outside the bounds.
Question 3
Check the whiteness of the residuals from the fitted ARIMA model. Based on these evaluate and if necessary review the ARIMA model specified in Q2. (No more than 50 words). (4 marks)
There are a couple of large residuals (see the long right tail in the histogram). Histogram looks close to normal (other than the outliers). The residuals are definitely not WN. The Ljung-Box test with 4 dof verifies the non-whiteness of the residuals as the Null of joint zero autocorrelation for the first 24 lags of the residuals is easily rejected with a p-value=0 at any reasonable level of significance.
However, I don’t think there is much I can do manually to improve the model by reading the ACF/PACF of the residuals. A combination of the non-seasonal AR and MA orders is probably needed and I will try those next.
Marking guide:
Plot/Comment on the residuals (1m)
Plot/Comment on the histogram of the residuals (1m)
Plot/Comment on the ACF/PACF plot (1m)
Perform LB test, with the Null hypothesis and correct dof. Comment on the result (1m)
Expectation:
Analysis of ACF/PACF or residuals
Ljung-box test with appropriate dof
Review of ARIMA model
Common errors:
Incorrectly used the Ljung-Box test, often miscalculating dof or not interpreting the test correctly.
Fail to review the existing model by just providing the conclusion of whiteness of the residuals.
No analysis on ACF/PACF plot in detail.
Question 4
Consider three (up to five if you think you need them) alternative ARIMA models based on your choice in Q2 and Q3. (Very briefly justify each choice with no more than 1 or 2 lines each). Use information criteria to choose the best model you have considered so far. (Very briefly justify each choice with no more than 1 or 2 lines each). (6 marks)
My ACF/PACF plots of both the original data and the residuals are very messy so I don’t have many alternatives here. My models are slight variations of the original model. I work mostly with the ACF/PACF of the original data. I select an MA(1), i.e., arima_011_012, as an alternative to the AR(2) for the one spike in the ACF. I also use an MA(4) for the (just) significnat spike in the ACF and then a combination of seasonal ARs and MAs.
I have a conflict in my information criteria with the lowest AICc selecting the arima_014_111 while the lowest BIC pointing to an arima_011_012.
Marking guide:
State at least 3 alternative ARIMA models and justify. (4m)
Choose the best model using the information criteria. (2m)
Expectation
Generation and justification of alternative ARIMA models
Choice of information criterion
Best model selection
Common errors:
Fail to elaborate on how they created the alternative ARIMA models. AICc can be used for comparison of models only with the same orders of differencing.
Question 5
Let the ARIMA() function choose a model. If you need to, make ARIMA() search harder exploring all possible options within the function. Perform a residual diagnostics analysis of the chosen model. How does this compare with your chosen model from Q4? (No more than 100 words). (6 marks).
The two models from the automated process (even with the wider search process as compared to the stepwise which skips some combinations) are not too different from the model I manually selected. For example I may have selected an AR(5) in place of the MA(4). The arima_auto2 is the model with the lowest AICc so I will go with this. The residuals still look messy and the Ljung-Box test still reject the null of the first 24 autocorrelations of the innovation residuals being jointly zero for any reasonable significance level.
Marking guide:
Run ARIMA() and explain how the model is chosen (using min AICc). (2m)
Compare with Q4 with the chosen ARIMA and state what is the different. If different number of differencing is used than AICc is not comparable. (2m)
Residual diagnostics (2m)
Expectation
Appropriate use of ARIMA()
Comparison with Q4 model, note that different differences makes the model IC are incomparable if these are different.
Residual diagnostics
Common errors: 1. Fail to compare models and state the difference. 2. Fail to discuss the residuals in detail. 3. Fail to mention how the ARIMA select the best model.
Question 6
Remember that you cannot use information criteria to compare between models with different orders of differencing. If necessary use an appropriate test set to choose the ARIMA model you want to use for forecasting. Which model have you selected and why? (No more than 50 words). (4 marks)
My models are all of the same order so I do not have to do another comparison.
Full marks if you didn’t need to do a test comparison and just added a comment.
Marking guide:
Perform test set RMSE based on appropriate test set. (2m)
Select the correct model (2m)
Expectation
Appropriate selection method for forecasting
Model selection and justification
Common errors:
Missing explanation on why RMSE test set is not needed in the case when all differencing are the same but straight away provided the answers on choosing the model with lowest AICc.
Question 7
Generate and plot forecasts and forecast intervals from your chosen ARIMA model for two years following the end of your sample. Comment on these. (No more than 50 words). (3 marks)
Forecasts seem reasonable both in terms of trend and variance. Prediction intervals seem reasonably wide given the cyclical nature of the data.
Marking guide:
Plot forecasts and forecast intervals (2m)
Comment on the forecast and intervals (1m)
Expectation
Generate forecasts
Plot forecasts
Comment
Common errors:
Fail to comment on the forecast.
Just mentioning the model forecast well is not sufficient, detail explanation such as discussion of the pattern is needed.
Question 8
You have now considered several modelling frameworks and built several models for your data set. In this part of the assignment you will evaluate these.*
Update your data from the ABS website https://www.abs.gov.au/statistics/industry/retail-and-wholesale-trade/retail-trade-australia. Explore your updated time series and comment on the series movement through and post the COVID-19 pandemic. Provide any necessary plots to support your analysis. Some States and Industries are unfortunately affected more than others. (6 marks)*
Successfully updates series (2) (do not deduct marks if series was not updated by ABS).
Plot data and visualise as necessary (at least time plot and season plot) (2)
Comments (2marks)
Some series show a drop after covid/during covid. The acceptable explanation can be:
My series showed significant drop between March and April 2020 and then followed a rapid recovery.
In this case the series shows and unusual spike in February 2020 - the beginning of the covid effect and closedowns in Australia. - Full recovery seems to had been returned to by July.
Bonus marks to whoever commented on lockdowns, etc.(2 marks)
Question 9
Generate forecasts for the period post 2022 until the end of your sample, from the models considered “best” in all assignments. More specifically, generate forecasts from the best benchmark, the best ETS and best ARIMA model. Plot the forecasts (both point forecasts and prediction intervals) together with the observed data and comment on these. (Make sure you can clearly visualise these. You may choose to plot on multiple graphs.) (6 marks)
Forecasts from both ETS and ARIMA models seem plausible. Given the strong trend in the data the SNAIVE forecasts are not suitable, hence removed from presenting prediction intervals.
Prediction intervals also seem very plausibe.
Marking guide
generate forecasts from all three model (2 marks)
plots and visualize (2 marks)
Comment on what you see or what you would have expected to see (2)
Question 10
Evaluate the accuracy of the point forecasts over the period post 2022. A table with accuracy measures will be necessary to be presented here. Comment on which forecasts are the most accurate. (4 marks)
# A tibble: 3 × 7
.model .type ME RMSE MAPE MASE RMSSE
<chr> <chr> <dbl> <dbl> <dbl> <dbl> <dbl>
1 ARIMA_011_212 Test 5.73 65.5 1.48 0.685 0.667
2 ETS_MAdM Test 88.7 95.9 2.51 1.15 0.977
3 snaive Test 321. 341. 8.96 4.17 3.47
COMMENT
ARIMA for me seems to do the best
Marking guide
Table (2)
Commenting on the observed or expected differences. (2)
Question 11
Use all three models to forecast the next 24-months of your updated series. Generate the necessary plots and comment on the forecasts. Make sure you can clearly visualise these. How have your models faired amidst the effects of COVID-19? (9 marks)
