Criteria in choosing the holistic Forecasting System
1. Ability to Model Demand:
Ability of a forecasting system to generate forecast at the most granular level across Time, Geography, Product and Customer dimensions, with the highest accuracy. This will also decide how effectively the system has been able to model the business requirements.
2. Statistical Rigour:
Forecasting systems need to have exhaustive library of statistical models - starting from simplest to most complex. This can help in choosing the best forecasting model which truly represents demand, yet manages model complexity. For example, there are cases where a simple “Moving Average" model may be adequate, whereas there might be cases that demand more sophisticated models (such as “ARIMA"). Moreover, there might be occasions where a combined model is chosen with a weighted average of different models. As a result, the system should not only provide the means through which different models may be easily applied but also facilitate the collaboration between these models for a true representation of demand.
3. Accuracy and Forecast Generation Time:
The preliminary requirement of any forecasting system to generate accurate forecast may not be enough. For example, an organization requiring daily forecasts for the planning purpose may not be able to use the forecasting system if it takes 10 hours to generate forecasts, regardless of its accuracy. Timely availability of forecast is as important as the accuracy it provides.
4. Interpretability:
Statistical error measures (e.g. MAPE, MSE) are popular yet widely misunderstood and misinterpreted. Quite often, end-users are not equipped to interpret the forecasting accuracy through such error measures. Forecasting system should enable business users with dashboard capabilities that communicate such measures in visually interpretable mediums.
5. Accommodate external issues:
Selecting the best forecasting model may not be enough. In majority of the cases, accuracy could benefit from the consideration of external components. These components could entail information such as the dates of forthcoming national holidays and the occurrence of exceptional events such as marketing campaigns. Hence an automated system should be able to understand such components and should be able to seamlessly combine them with basic forecasting techniques.
6. Automatic self-tuning:
An automated collaboration of the various components generates an additional consideration. When various models are combined, collaborative operation and self tuning becomes a major issue. The challenge arises from the relationships among the models. Although manual operation is a solution, it is associated with two major problems. Firstly, efficiency is reduced due to the required time and secondly, selection of models may be compromised in order to choose simpler alternatives. Hence an automated process for self-tuning would increase the flexibility and efficiency of the system.
7. Generic data representation:
Different applications will be associated with different forecasting parameters. Typical examples of such parameters include geographical areas, types of product and priority levels of service. This list could be enriched as diverse application scenarios might be considered. As a result, the design of a generic and fully automated forecasting framework requires the definition of a generic data representation. This data type will hide the low level details and present an abstract view on which the generic forecasting framework may operate on.
Excerpt from the working paper – Shah M., Owusu G., Shoban B., Balkundi N., “Improving Forecasting Accuracy of Traditional Demand Planning System” (2008)
The original blog was published here.
