The Right Level for measuring the Right Forecast Errors
Many organizations have complex multi-level hierarchies for forecasting not only in the product dimension but also in geographical dimension. One of the standard practicies in most organizations is to do Bottom-up forecasting, followed by Middle-out forecasting and Top-down forecasting. The forecasting done at lower level of detail is very crucial since it determines the proportional factors for forecasts done at higher level. In other words, forecasts at lower level of detail helps come up with the right mix. Forecasting done at higher level helps come up with the right volume.
One of the predicaments of designing a sustainable self-correcting Forecasting system is to understand the level at which forecast errors should be measured and also what is the right type of forecast error. Based on two project experiences that I have been very closely associated with, I have found that
1. It makes sense to measure and monitor Forecast error at middle levels of product and geographical hierarchy. At lower level, usually data is very sparse and intermittant and any forecast error metric will tend to capture disproportionately long list of planning objects. This is counter-productive in an exception-based planning setup. At very high level, data may be smooth and forecastable, however it is disconnected from supply planning and any forecast error metric will tend to capture disproportionately short list of planning objects. Although planners tend to ask for error metric at each and every forecast level, it makes sense to monitor at only one level and this is in interest of time spent in a forecast cycle - given that most planners tend to spend lot of time negotiating and drawing consensus with various stakeholders.
2. Measure Bias errors on manually intervened and adjusted forecast. A statistical forecast is swayed by historical patterns and may lack some of the judgemental capabilities that planners bring in. However to make sure that these judgemental interventions from planners are moderated, a measure of bias error makes sure that they do not underforecast or overforecast over a period of time - thereby giving right signals to downstream supply planning applications.
3. Optimize the number of levels to manage and run forecast processes. Although business prefers flexibility in terms of being able to run and manage forecasts at all possible levels/combinations in a forecast hierarchy, it makes sense to only run forecast at handful of levels. Simple reason is that is manageable and sustainable.
It is important that the team does not go over-board on designing complex forecast solution that very few people understand. All complexities are better tucked in and hidden from general users. At the end of day, planners tend to use systems that are easy to use, work with and feel worth spending their productive time.
