Comparing Predictive Accuracy in the Presence of a Loss Function Shape Parameter

May 2020 | 909

Authors: Sander Barendse, Andrew J. Patton

We develop tests for out-of-sample forecast comparisons based on loss functions that contain shape parameters. Examples include comparisons using average utility across a range of values for the level of risk aversion, comparisons of forecast accuracy using characteristics of a portfolio return across a range of values for the portfolio weight vector, and comparisons using a recently-proposed “Murphy diagrams” for classes of consistent scoring rules. An extensive Monte Carlo study verifies that our tests have good size and power properties in realistic sample sizes, particularly when compared with existing methods which break down when then number of values considered for the shape parameter grows. We present three empirical illustrations of the new test.

JEL Codes: C53, C52, C12

Keywords: Forecasting, model selection, out-of-sample testing, nuisance parameters

View All Working Papers