Does your Forecaster end up being a Placebo?
By asking this question, I do not intend to question the efforts and intentions of the personnel from the Forecasting and Demand Planning organizations. However, I intend to question the role of the Forecaster and the philosophy adopted by the Forecasting and Demand Planning Team in your organization. I would want to state my stance at the onset that I do firmly believe in the benefits of a formal forecasting process and the key role the forecasting organization has played in the recent past to guide the subsequent planning processes in providing early guidance. But I do not want this to be considered at face value for all situations, but would rather want to question and constantly challenge the role of Forecaster for the different scenarios being planned... Again, the idea is to not doubt or question the intentions of the Forecasters, but with the premise that even noble intentions, if not managed well could lead to bad results, objectively evaluate the contribution made by the Forecaster and the Demand Planner.
My objective is to wean us away from the plausible "Placebo effect" - the ingrained position that it is the job of the forecaster to review and adjust the forecasts and that of the Demand Planner to enhance the same based on market intelligence, and once these are done, the end result, the final demand plan would be the most accurate that we can get. Do we ever really question - Do the changes made by the Forecaster and Demand Planner have any impact on the overall Forecast Accuracy. It is this phenomenon where the Forecaster actions lead to no or negative impact that I term as the 'Placebo effect', though I do not use the term in its conventional sense. The fact that we have a formal forecasting process leads us to believe that the decisions made by the various players are adding value in a positive way and thus improving the health of the overall planning process, whereas, in reality these interventions end up having no impact on the effectiveness of the process, if not further deteriorating the same.
So, how do we guard ourselves against this 'Placebo effect'? The old adage goes as - We get what we measure well... So if we want to measure the contribution of the Forecaster, we need to have the right measures in place... We do have the conventional Forecast Accuracy measures that have been so very well detailed in the blog by my colleague. These measures provide an indication of how well a particular forecast generated was compared to the actual historical demand for the same period. The question that we further need to ask ourselves is - how much better or worse would the forecast be if the Forecaster or the Demand Planner had not intervened - is the action provided by the role providing any positive contribution?
To answer this question, we need to take a step further and compare the Forecast Accuracies for the forecast numbers computed by the Forecasters with those of a forecast that would be computed with minimal or no effort - These are referred to as Naïve Forecast in the Makridakis text. There are various known variants of such Naïve Forecasts that one could consider - the simple moving average, incase there is no seasonality in the data pattern and the seasonal random walk, incase there is seasonality in the data pattern being forecasted.
We could thus come up with a new metric that would depict the contribution made by every step in the forecasting process. The formula for the same could be as simple as follows: Forecast Accuracy (Forecast Step 2) - Forecast Accuracy (Forecast Step 1) OR Forecast Accuracy (Forecast Step 2) / Forecast Accuracy (Forecast Step 1) For the first formula, a positive value for the metric would mean a positive contribution and a negative value would mean that the step in fact is deteriorating the Forecast Accuracy and a zero would mean that the step is redundant and provides no benefit in being executed. In case of the second formula for the metric, a value of 1 would mean that the step has not impacted the Forecast effectiveness in any way, a value greater than 1, would imply that the step is important and does bolster the effectiveness of the forecast. A value of less than 1 would mean that the step has impacted the overall process effectiveness negatively. We must take prompt actions to analyze the reasons for the same and accordingly decide on whether to improve on the step or eliminate it completely.
Some of the Key benefits that can be derived by using this approach: 1. This metric provides a very simple, yet powerful way in identifying the wasteful steps in your forecasting process and helps you plan to take steps to take corrective actions to improve the overall efficiency and effectiveness of your forecasting process. 2. This can be a key strategic input to the management for the effectiveness of the various steps in the planning process. This analysis can be further enhanced by additionally considering the attributes such as ABC Class, Lifecycle Status and product family to devise separate strategies for different sets of parts to be forecast accordingly and eliminate non-productive tasks altogether. An example of the same could be that we remove the step of Forecaster Adjustment for parts that are in their maturity stage or for the low runner items, but have this step as the most important step for parts that are in their Growth Stage or even A-class parts. This way, we could better segment our parts into different ways of Forecasting and Demand Planning, thereby making the complete process more efficient and effective. 3. This measure does consider the oft-missed consideration for Forecast Accuracy - the uncertainty involved in forecasting a particular data pattern, the concept that I had talked on in my earlier blog. By comparing the Forecast Accuracies of two Forecasts that have been generated amidst the same Uncertainty, we do neutralize the measure of any effect of the uncertainty involved and thus this measure is a true measure of the effectiveness of a Forecasting step or a process.
This philosophy has been further extended, as mentioned in the blog by our colleague to perform attribution analysis between the Planning functions and the Execution functions within an organization. Another variant that could be really effective is to compare the Forecasts computed by the planners against the 'Best-Fit' forecast generated by the system, after optimizing various forecast parameters and with the least forecast accuracy amongst various forecast models. This would be an acid test for the effectiveness of the interventions made by the Forecaster and the Demand Planner. With the advanced planning systems available, these could be implemented with no or very little effort - an opportunity that I would urge the Supply chain professionals to consider with the most leverage to improve on the effectiveness of the Demand Planning functions in your organizations.
Alas, this seems so intuitive and logical and yet as the concept discussed in my last blog, so less widely prevalent... There are many reasons for the same, some of the key being: - Such a metric could instill a sense if insecurity amongst the Forecasters and the Demand Planners - This could also increase or exacerbate the politics between the Forecasting and Demand Planning Organizations with the others. The list could go on and on... I would like to know your views on the same and if you had experience in implementing such similar metrics in your Planning organizations.
