Accuracy benchmark

In 2013, the accuracy of Indicio's algorithms were benchmarked towards practitioners in qualitative forecasting. The results were evaluated as an average of 40 observations of quarterly forecasts, on a rolling forecast basis. The study was based on an average of base metals registered at the London Metal Exchange (LME). The total time-horizon evaluated was thus 10 years, leaving some room for macroeconomic events biasing the result. The study is to be seen as an indication of method efficiency rather than a claim for absolute relationships.

During the study, the accuracy of metalprice forecasting at a Swedish industrial firm was investigated for a period of half a year. The industrial firm used a qualitative approach, where a forecaster assembled an initial forecast that was then communicated to concerned parts of the organization as basis for decision-making. In the study, the industrial firm had an average accuracy deviation of 32 %.

This was then benchmarked towards the reports of the 10 largest international banks. Banks use qualitative methods, similar to those used by the industrial firm, with the difference that the forecasts are assessed in order to suit the needs of the market rather than internal departments of an industrial firm. The investigated banks showed an accuracy deviation of 18 %.

The Theta model, which is one of the models found in the Indicio library, was then applied. The Theta model is a timeseries algorithm developed in the beginning of the 21th century as part of a competition to find the best timeseries algorithms. On the same data, the Theta model obtained an accuracy deviation of 8 %.