The Infosys global supply chain management blog enables leaner supply chains through process and IT related interventions. Discuss the latest trends and solutions across the supply chain management landscape.

« Strategies for efficiently managing IT assets | Main | Competing on the true color of your Supply Chain »

How to Measure Forecast Accuracy?

Everyone who is associated with Demand Planning and Forecasting function invariably talks about a phrase called "Forecast Accuracy". It is a measure used for judging the efficiency of the Forecasting Process. At the back of the mind everyone knows that Forecast Accuracy is the comparison of Forecast Vs Actual.

But, if it is just the question of comparison of Forecast Vs Actual comparison then why we have so many Forecast Accuracy measures? To name a few, almost every Statistical Forecasting Tool has following error measure as a part of their standard offerings - Mean Absolute Deviation (MAD), Mean Absolute Percentage Error (MAPE), Root Mean Square Error (RMSE), Mean Percentage Error (MPE), Error Total (ET) and Mean Square Error (MSE)so on. What does all of these measures mean? And in what context which measure should be used?

Answers to above questions can be found by answering following 4 basic questions in your Business Context.

  1. Error Sign - Do you want to treat Positive and Negative Forecast Error same or different? - In simple words it means, will it make any difference to you if Forecast is more than Actual Sales or less than Actual Sales. This depends on what business you are in. If you are in business of say perishable products then you will always prefer Forecast to be less than Actual as surplus production is as good as loss. On the contrary if you are in a business of selling products which have reasonable shelf life then Forecast should be always greater than Actual as loss of sales is more costly than holding inventory.
  2. Reference to Base - Do you want reference to base value while measuring Forecasting Accuracy? - In simple words it means, whether a forecast error of 2 over a forecast value of 100 and an error or 2 over a forecast value of 10 mean the same to you. Answer to this question depends of what you are forecasting. If you are forecasting dispatches from factory where KPIs are "fulfillment" related like On Time In Full (OTIF), Order Fulfill Rate, Dollar Value Fulfilled on time etc then reference to base is important since you know 100% perfection is never possible. On the other hand if you are forecasting for Point of Sales (POS) then every unit stock out cost you same. 100% shelf availability is a must and there cannot be any compromise on this count. In such case reference to base does not mean anything. For example no point in achieving shelf availability of 98% and missing 2% availability during pick times.
  3. Error Spread - Does it matter to you if forecast error is concentrated on few points or do you want forecast error to be spread evenly?- 100% forecast accuracy is utopian. Some error is bound to happen. Question is what is your preference for distribution of error? Do we want it to be evenly distributed over all points or you don`t mind if you go horribly wrong on few points and look good if calculated for entire horizon. Answer to this depends upon type of products you are dealing in. If you are dealing with product that have very long shelf life which means unsold inventory from one period can be effectively put into use in subsequent periods then we don`t mind going horribly wrong with the forecast in one period as long as we make up for that in the horizon. Forecast Accuracy of the complete horizon is more important in such case than accuracy of an individual month or a week.
  4. Weightage to History Data Points - Do you want to treat error on every point equally or you want to differentiate between individual data points based on their place in history? - For example, when you are analyzing forecast accuracy do you care if total forecast error is contributed more by recent past than by distant past. In some fashion business it does make lot of difference. Fitment over recent past is more important than fitment over distant past. Again choice of this decision depends on what you are forecasting. If forecast is meant for long term investment planning in planning for facility then all data points means same. However if forecast is for POS and influenced greatly by consumer behavior then close fitment over recent history is more important than fitment over distant history.

Now let us map commonly offered Forecast Accuracy measures to each of the above criterions.


Measure Sensitive to Sign of Error?

No Yes No No No Yes
Measure has reference to Base? No No Yes No No Yes
Measure gives equal importance to all data points? No Yes Yes Yes Yes Yes
Measure sensitive to distribution of over data points? No No No Yes Yes No

Going forward if you face dilemma in selecting proper Forecast Accuracy Measure to suit your business context then make quick reference to above table and your problem will be solved.


Great article, very clear and compelling.

I would just suggest to consider a fifth question: is the demand continuative or sporadic? Are there zeroes in the demand?

In such a latter case, in fact, ratio-based measures (like MAPE and MPE) cannot be used, while other measures remain "mathematically valid", although not always meaningful.

I think that the problem of sporadic demand is still one of the most challenging and prominent in modern businesses.

Kindest regards.

Yes...You are right. Ratio based measures like MAPE, MPE fail with zero values in history. Generally all the standard softwares exclude these points while calculating these measures which makes the calculation even further wrong and use of MAPE and MPE even more unsuitable in such scenarios.In fact few of the forecasting algorithms also fail to calculate forecast in the first place if we have zero values in history.

Very Good article; i have one more point, Forecast accuracy vs. OTIF - is it necessary to have a correlation between OTIF and FA (FA based on MAPE)?

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Please key in the two words you see in the box to validate your identity as an authentic user and reduce spam.

Subscribe to this blog's feed

Follow us on

Blogger Profiles

Infosys on Twitter