Financial impact of the forecasting accuracy

By Joannes Vermorel, February 2012

Update (2019): the perspective presented in this article is dated to some extend. This article adopts a classic forecasting perspective, while probabilistic forecasting should be considered instead, as they yield better result in nearly all supply chain situations. In particular, the economic perspective on the forecasting accuracy is better tackled through approaches like the stock reward function.

More accurate demand forecasts are obviously good as far as inventory optimization is concerned. However, the quantitative assessment of the financial gains generated by an increase of the forecasting accuracy typically remains a fuzzy area for many retailers and manufacturers. This article details how to compute the benefits generated by an improved forecast.

The viewpoint adopted in this article is a best fit for high turnover inventories, with turnovers above 15. For high turnover values, the dominant effect is not so much stockouts, but rather the sheer amount of inventory, and its reduction through better forecasts. If such is not your case, you can check out our alternative formula for low turnover.

The formula

The detail of the proof is given below, but let’s start with the final result. Let’s introduce the following variables:

$${D}$$ the turnover (total annual sales).
$${m}$$ the gross margin.
$${\alpha}$$ the cost of stockout to gross margin ratio.
$${p}$$ the service level achieved with the current error level (and current stock level).
$${\sigma}$$ the forecast error of the system in place, expressed in MAPE (mean absolute percentage error).
$${\sigma_n}$$ the forecast error of the new system being benchmarked (hopefully lower than $${\sigma}$$).

The yearly benefit $${B}$$ of revising the forecasts is given by:

$${B = D (1 - p) m \alpha \frac{\sigma - \sigma_n}{\sigma}}$$

Download Excel sheet: accuracy-gains.xlsx (illustrated calculation)

It is possible to replace the MAPE error measurements by MAE (mean absolute error) measures within the formula. This replacement is actually strongly advised if slow movers exist in your inventory.

Practical example

Let’s consider a large retail network that can obtain a 10% reduction of the (relative) forecast error through a new forecasting system.

$$D=1,000,000,000$$€ (1 billion Euros)
$${m=0.2}$$ (i.e.gross margin of 20%)
$${p=0.97}$$ (i.e. service level of 97%)
$${\alpha=3}$$ (stockouts cost 3x the gross margin loss)
$${\sigma=0.2}$$ (MAPE of 20%)
$${\sigma_n=0.18}$$ (MAPE of 18% - relatively 10% lower than the previous error)

Based on the formula above, we obtain a gain at $$B=1,800,000$$€ per year. If we assume that the overall profitability of the retailer is 5%, then we see that a 10% improvement in forecasting accuracy already contribute to 4% of the overall profitability.

Proof of the formula

At a fundamental level, inventory optimization is a tradeoff between excess inventory costs vs. excess stockout costs.

Let’s assume, for now, that, for a given stock level, the stockout frequency is proportional to the forecasting error. This point will be demonstrated in the next section.

The total volume of sales lost through stockouts is simple to estimate: it’s $${D(1-p)}$$, at least for any reasonably high value of $${p}$$. In practice, this estimation is very good if $${p}$$ is greater than 90%.

Hence, the total volume of margin lost through stock-outs is $${D(1-p)m}$$.

Then, in order to model the real cost of the stock out, which is not limited to the loss of margin (think loss of customer loyalty for example), we introduce the coefficient $${\alpha}$$. So the total economical loss caused by stock outs becomes $${D(1-p)m\alpha}$$.

Based the assumption (demonstrated below) that stockouts are proportional to the error, we need to apply the factor $${(\sigma - \sigma_n) / \sigma}$$ as the evolution of the stockout cost caused by the new average forecast error.

Hence, in the end, we obtain:

$${B = D (1 - p) m \alpha \frac{\sigma - \sigma_n}{\sigma}}$$

Stockouts are proportional to the error

Let’s demonstrate now the statement that, for a given inventory level, stockouts are proportional to the forecasting error.

In order to do that, let’s start with service levels at 50% ($${p=0.5}$$). In this context, the safety stock formula indicates that safety stocks are at zero. Several variants exist for the safety stock formula, but they are all behaving similarly in this respect.

With zero safety stocks, it becomes easier to evaluate the loss caused by forecast errors. When the demand is greater than the forecast (which happens here 50% of the time by definition of $${p=0.5}$$), then the average percentage of sales lost is $${\sigma}$$. Again, this is only the consequence of $${\sigma}$$ being the mean absolute percentage error. However, with the new forecasting system, the loss is $${\sigma_n}$$ instead.

Thus, we see that with $${p=0.5}$$, stockouts are indeed proportional to the error. The reduction of the stockouts when replacing the old forecast with the new one will be $${\sigma_n / \sigma}$$.

Now, what about $${p \not= 0.5}$$? By choosing a service level distinct from 50%, we are transforming the mean forecasting problem into a quantile forecasting problem. Thus, the appropriate error metric for quantile forecasts becomes the pinball loss function, instead of the MAPE.

However, since we can assume here that the two mean forecasts (the old one, and the new one) will be extrapolated as quantile (to compute the reorder point), though the same formula, the ratio of the respective errors will remain the same. In particular, if the safety stock is small (say less than 20%) compared to the primary stock, then this approximation is excellent in practice.

Cost of stockouts (α)

The factor $${α}$$ has been introduced to reflect the real impact of a stockout on the business. A minima, we have $${α=1}$$ because the loss caused by an extra stockout is at least equal to the volume of gross margin being lost. Indeed, when considering the marginal cost of a stockout, all infrasture and manpower costs are fixed, hence the gross margin should be considered.

However, the cost for a stockout is typically greater than that the gross margin. Indeed, a stockout causes:

a loss of client loyaulty.
a loss of supplier trust.
more erratic stock movements, stressing supply chain capacities (storage, transport, …).
overhead efforts for downstream teams who try to mitigate stockouts one way or another.
…

Among several large food retail networks, we have observed that, as a rule thumb, practionners are assuming $${α=3}$$. This high cost for stockouts is also the reason why, in the first place, the same retail networks typically seek high service levels, above 95%.

Misconceptions about safety stocks

In this section, we debunk one recurrent misconception about the impact of an extra accuracy, which can be expressed as extra accuracy only reduces safety stocks.

Looking at the safety stock formula, one might be tempted to think that the impact of a reduced forecasting error will be limited to lowering the safety stock; all other variables remaining unchanged (stockouts in particular). This is a major misunderstanding.

Classical safety stock analysis splits inventory in two components:

the primary stock, equal to the lead demand, that is to say the average forecast demand multiplied by the lead time.
the safety stock, equal to the demand error multiplied by a safety coefficient that depends mostly of $${p}$$, the service level.

Let’s go back to the situation where the service level equals 50%. In this situation, safety stocks are at zero (as seen before). If the forecast error was only impacting the safety stock component, then it would imply that the primary stock was immune to poor forecast. However, since there is no inventory here beyond the primary stock, we end-up with the absurd conclusion that the whole inventory has become immune to arbitrarily bad forecasts. Obviously, this does not make sense. Hence, the initial assumption, that only safety stocks were impacted is wrong.

Despite being incorrect, the safety stock only assumption is tempting because when looking at the safety stock formula, it looks like one immediate consequence. However, one should not jump to conclusions too hastily: this is not the only one consequence. The primary stock is built on top of the demand forecast as well, and it’s the first one to be impacted by a more accurate forecast.

Advance topics

In section, we delve in further details that have been omitted in the discussion above for the sake of clarity and simplicity.

Impact of varying lead times

The formula above indicates that reducing the forecast error at 0% should bring stockouts at zero as well. On one hand, if customer demand could be anticipated with 100% accuracy 1 year in advance, achieving near-perfect inventory levels would seem less outstanding. One the other hand, some factors such as the varying lead time complicates the task. Even if the demand is perfectly known, an varying timing of delivery might generate further uncertainties.

In practice, we observe that the uncertainty related to the lead time is typically small compared to the uncertainty related to the demand. Hence, neglecting the impact of varying lead time is reasonable as long as forecasts remain somewhat inaccurate (say for MAPEs higher than 10%).