Mean absolute error

What Is Mean Absolute Error?

Mean Absolute Error (MAE) is a widely used metric in quantitative analysis that quantifies the average magnitude of errors in a set of predictions or forecasts, without considering their direction. It is a fundamental performance metric used in various fields, including finance, statistics, and machine learning, to evaluate the accuracy of predictive models. Essentially, MAE provides a direct measure of the average difference between predicted values and actual observed values, offering a clear and interpretable indication of how much, on average, a model's predictions deviate from reality. Lower MAE values indicate higher accuracy, meaning the predictions are closer to the true outcomes.

History and Origin

The Mean Absolute Error has a long-standing presence in the fields of statistics and machine learning. It is recognized as one of the earliest measures employed for error quantification in predictive modeling, valued for its simplicity and clear interpretability.⁹ Its intuitive nature has ensured its continued relevance across diverse domains where forecasting and prediction are essential, such as economics, meteorology, and data analysis. While more complex error metrics have emerged over time, the straightforwardness of MAE has maintained its position as a valuable tool for assessing model accuracy.

Key Takeaways

• Mean Absolute Error (MAE) measures the average magnitude of errors between predicted and actual values.
• It is expressed in the same units as the data being measured, making it highly interpretable.
• MAE treats all errors equally, regardless of their size, and does not disproportionately penalize larger errors.
• A lower MAE indicates a more accurate model, with zero MAE representing perfect predictions.
• It is a robust metric, less sensitive to outliers compared to squared-error metrics.

Formula and Calculation

The Mean Absolute Error (MAE) is calculated by taking the average of the absolute differences between each predicted value and its corresponding actual value.

The formula for MAE is:

$MAE = \frac{1}{n} \sum_{i=1}^{n} |y_i - \hat{y}_i|$

Where:

• (MAE) = Mean Absolute Error
• (n) = The number of observations or data points
• (y_i) = The actual (observed) value for the (i)-th observation
• (\hat{y}_i) = The predicted value for the (i)-th observation
• (|...|) = The absolute value of the difference, ensuring that negative and positive errors do not cancel each other out.

This formula essentially calculates the average deviation of the predictions from the true values.

Interpreting the Mean Absolute Error

Interpreting the Mean Absolute Error is straightforward due to its direct relationship with the units of the data. For instance, if MAE for a stock price prediction model is $2.50, it means that, on average, the model's predictions deviate by $2.50 from the actual stock prices. This ease of interpretation makes MAE particularly useful for communicating model performance to non-technical stakeholders or for applications where a direct understanding of prediction accuracy is crucial.

A lower MAE value signifies higher model accuracy; an MAE of zero would imply perfect predictions, though this is rarely achievable in real-world financial modeling. When evaluating models, analysts often compare the MAE against a baseline or against the MAE of other statistical models to gauge effectiveness. The context of the data and the specific application are vital for determining what constitutes an acceptable or "good" MAE value.

Hypothetical Example

Consider a scenario where a financial analyst uses a model to forecast the quarterly earnings per share (EPS) of a company. Below are the actual EPS values and the model's predicted EPS for five consecutive quarters:

| Quarter | Actual EPS ((y_i)) | Predicted EPS ((\hat{y}_i)) | Absolute Error ((|y_i - \hat{y}_i|)) |
|---------|-----------------------|---------------------------------|---------------------------------------|
| Q1 | $1.20 | $1.25 | (|1.20 - 1.25| = 0.05) |
| Q2 | $1.35 | $1.30 | (|1.35 - 1.30| = 0.05) |
| Q3 | $1.40 | $1.42 | (|1.40 - 1.42| = 0.02) |
| Q4 | $1.10 | $1.08 | (|1.10 - 1.08| = 0.02) |
| Q5 | $1.50 | $1.55 | (|1.50 - 1.55| = 0.05) |

To calculate the Mean Absolute Error:

1. Calculate the absolute error for each quarter:

   • Q1: (|1.20 - 1.25| = 0.05)
   • Q2: (|1.35 - 1.30| = 0.05)
   • Q3: (|1.40 - 1.42| = 0.02)
   • Q4: (|1.10 - 1.08| = 0.02)
   • Q5: (|1.50 - 1.55| = 0.05)

2. Sum the absolute errors:
   (0.05 + 0.05 + 0.02 + 0.02 + 0.05 = 0.19)

3. Divide by the number of observations (quarters):
   (MAE = \frac{0.19}{5} = 0.038)

The Mean Absolute Error for this EPS forecast model is $0.038. This means that, on average, the model's quarterly EPS predictions deviate from the actual EPS by $0.038. This concrete example demonstrates how MAE can be applied to evaluate the accuracy of financial predictions and provides a tangible measure of prediction error.

Practical Applications

Mean Absolute Error (MAE) finds numerous practical applications across finance and economics due to its intuitive nature and robust characteristics.

• Financial Forecasting: MAE is widely used to assess the accuracy of various financial forecasts, including stock prices, commodity prices, interest rates, and earnings. For example, investment analysts might use MAE to compare the predictive power of different investment strategies or asset pricing models.
• Risk Management: In risk management, models are used to estimate potential losses, credit risk, or market risk. MAE can help validate these models by measuring how closely their predictions align with actual outcomes. Regulatory bodies, such as the Office of the Comptroller of the Currency (OCC), emphasize rigorous model validation practices, where error metrics like MAE play a crucial role in assessing model soundness and performance.⁸
• Economic Forecasting: Government agencies and central banks often use MAE to evaluate the accuracy of their economic forecasts for indicators like GDP, inflation, and unemployment. For instance, analyses by the Federal Reserve Bank of San Francisco might use MAE to assess the historical accuracy of inflation or unemployment rate projections.⁷,⁶
• Inventory Management: In supply chain finance, businesses use MAE to measure the accuracy of demand forecasts, which directly impacts inventory levels and operational efficiency. Lower MAE in demand forecasting can lead to optimized inventory management and reduced carrying costs.
• Algorithmic Trading: Traders employing algorithmic trading strategies often rely on models that predict short-term price movements. MAE serves as a key metric to fine-tune and compare the performance of different trading algorithms.

These applications underscore MAE's role in providing a clear, unit-based measure of error, facilitating better decision-making in diverse financial and economic contexts.

Limitations and Criticisms

While Mean Absolute Error (MAE) offers advantages in interpretability and robustness, it also has limitations. One common criticism is that MAE does not penalize larger errors more heavily than smaller ones. This contrasts with metrics like Mean Squared Error (MSE) or Root Mean Squared Error (RMSE), which, by squaring the errors, give disproportionate weight to larger deviations.⁵ In scenarios where large errors are significantly more detrimental than small ones, MAE might not be the most appropriate metric for model evaluation.

Another drawback is its mathematical properties. The absolute value function is not differentiable at zero, which can complicate certain optimization algorithms used in model training, particularly in machine learning. While this is less of a concern for interpreting existing model performance, it can impact the technical implementation of models that aim to directly minimize MAE as a loss function.

Furthermore, MAE is a scale-dependent metric. This means its value is directly influenced by the scale of the data. An MAE of 10 for predicting stock prices ranging in hundreds of dollars is very different from an MAE of 10 for predicting bond yields that are typically single-digit percentages. Consequently, MAE cannot be directly used to compare forecast accuracy across datasets with different scales without some form of normalization. While methods exist to decompose MAE into components like bias and proportionality error for deeper insight, its raw value remains scale-dependent.⁴

Mean Absolute Error vs. Mean Squared Error

Mean Absolute Error (MAE) and Mean Squared Error (MSE) are both widely used metrics to evaluate the accuracy of predictive models, particularly in regression analysis, but they differ significantly in how they treat errors.

The choice between MAE and MSE often depends on the specific context and the cost associated with different magnitudes of errors. If large errors are particularly undesirable or costly, MSE might be preferred as it highlights them. However, if a straightforward, easily understood measure of average error is needed, MAE often stands out.

FAQs

Why is Mean Absolute Error preferred by some analysts?

MAE is preferred by some analysts because of its interpretability. Since it is expressed in the same units as the data, it provides a clear and intuitive understanding of the average prediction error.³ It's also more robust to outliers compared to squared error metrics.

Can Mean Absolute Error be negative?

No, Mean Absolute Error cannot be negative. The formula for MAE involves taking the absolute value of the differences between predicted and actual values, which always results in a non-negative number. The sum of these non-negative values, when averaged, will also always be non-negative. A perfect model would yield an MAE of zero.²

How does Mean Absolute Error relate to bias?

MAE measures the magnitude of the errors but does not indicate the direction of the errors (i.e., whether the model consistently over-predicts or under-predicts). While MAE provides an overall measure of accuracy, forecast bias specifically measures systematic over- or under-prediction. A model can have a low MAE but still exhibit bias if its errors consistently lean in one direction.

Is a lower Mean Absolute Error always better?

Generally, a lower Mean Absolute Error indicates better model performance, as it means the predictions are, on average, closer to the actual values.¹ However, what constitutes a "good" MAE value is relative to the specific problem, industry, and the scale of the data being modeled. It's often more informative to compare MAE values across different models for the same dataset or against established benchmarks in a given domain.