Underfitting

What Is Underfitting?

Underfitting occurs in machine learning and statistical model development when a model is too simple to adequately capture the underlying patterns within the data points it is trained on. In the broader field of Machine Learning in Finance, an underfit model will perform poorly not only on the training data but also on new, unseen data, indicating it has failed to learn the fundamental relationships. This typically happens when the model lacks sufficient complexity or is not trained long enough to identify the significant features in the dataset. Consequently, the model cannot make reliable predictions. Underfitting is a fundamental issue that prevents a model from being effective for its intended purpose.

History and Origin

The concept of underfitting, along with its counterpart, overfitting, emerged naturally with the development and increasing sophistication of statistical models and algorithms. As researchers and practitioners began using more complex computational methods for prediction and analysis, particularly in fields like econometrics and eventually machine learning, the challenges of model generalization became evident.

Early algorithmic approaches in finance, dating back to the 1970s, focused on automating trading processes based on pre-specified rules. The recognition of issues like underfitting became more pronounced as models aimed to learn patterns from data rather than simply execute fixed rules. The widespread adoption of machine learning in finance, particularly from the early 2000s onwards with increased computational power and data availability, brought the bias-variance tradeoff to the forefront. This trade-off, which underfitting exemplifies, highlights the balance between a model's simplicity (high bias, leading to underfitting) and its sensitivity to training data (high variance, leading to overfitting). The evolution of AI in finance has been propelled by research into robust model performance, directly addressing these challenges⁴.

Key Takeaways

• Underfitting occurs when a statistical model or machine learning algorithm is too simple to capture the underlying patterns in the training data.
• An underfit model performs poorly on both the training data and new, unseen data.
• It often indicates high bias, meaning the model makes strong assumptions about the data that do not reflect reality.
• Common causes include insufficient model complexity, lack of relevant feature engineering, or inadequate model training.
• Addressing underfitting typically involves increasing model complexity, adding more relevant features, or extending training.

Interpreting Underfitting

Underfitting is typically identified when a statistical model exhibits low accuracy or high error rates on the training data itself. This indicates that the model has not learned the fundamental relationships present in the dataset. If a model performs poorly on the data it was explicitly designed to understand, it will inevitably perform even worse on new, unseen data points.

In terms of the bias-variance tradeoff, underfitting corresponds to a model with high bias and low variance. High bias means the model makes overly simplistic assumptions about the data, leading to systematic errors regardless of the input. Low variance, in this context, means the model's performance is consistent across different subsets of the training data, but consistently bad because it fails to capture true patterns³. Recognizing underfitting early in the model development cycle is critical to ensure that the developed financial models are capable of providing meaningful insights or accurate predictions.

Hypothetical Example

Imagine a small investment firm wants to predict a company's stock price based on its quarterly revenue. A junior analyst decides to use a very simple linear regression model, assuming that stock price will increase directly and proportionally with revenue.

They gather historical data points for Company A, plotting revenue against stock price. However, Company A's stock price tends to react to revenue with a lag, and also factors in market sentiment, overall economic conditions, and industry trends, which are not captured by a simple linear relationship.

When the analyst trains their linear model, they find that even for the historical data, the model's predictions are consistently far off the actual stock prices. The line drawn by the linear model might barely skim the actual data, showing a clear pattern that it fails to capture. For example, if revenues doubled, the model might only predict a 10% increase in stock price, whereas historically, it has been closer to 30%, especially in certain market conditions. This consistent large error on the known data illustrates underfitting. The model is too basic for the complex relationship it's trying to model, making it ineffective for genuine stock price forecasting. The analyst might need to incorporate more features through feature engineering, such as revenue growth rates, profit margins, or market sentiment indicators, and possibly use a more complex statistical model.

Practical Applications

Underfitting can manifest in various practical applications within finance, often hindering the effectiveness of financial models:

• Algorithmic Trading Strategies: If a trading algorithm is underfit, it might fail to identify crucial market patterns or relationships between assets, leading to consistently poor trade signals and missed opportunities. For instance, a simple rule-based system might underperform if market dynamics become more complex.
• Credit Scoring Models: An underfit credit scoring model might rely on too few variables (e.g., only income, ignoring debt-to-income ratio or payment history), leading to inaccurate risk assessments. This could result in loaning money to high-risk individuals or denying credit to creditworthy applicants, impacting profitability and customer acquisition.
• Risk Management: In risk modeling, underfitting could mean a model fails to account for critical dependencies or tail risks in a portfolio. For example, a value-at-risk (VaR) model that is too simplistic might dramatically underestimate potential losses during market turmoil because it doesn't adequately capture non-linear relationships or extreme events.
• Fraud Detection: An underfit fraud detection system might be too general, failing to flag sophisticated or evolving fraud patterns because its underlying machine learning algorithm is not complex enough to learn the subtle indicators of fraudulent activity from legitimate transactions.

The ability to build robust and accurate models is paramount in finance, which is why institutions are increasingly focused on comprehensive model training and validation processes to mitigate issues like underfitting. Leveraging machine learning for financial modeling helps enhance analytical practices and create more reliable predictive models across various applications, including stock analysis, financial engineering, and quantitative analysis².

Limitations and Criticisms

The primary limitation of an underfit model is its inability to learn effectively from the available data points, rendering it ineffective for its intended purpose. This directly impacts its utility in real-world financial models and decision-making.

One common criticism is that underfitting can sometimes be overlooked, especially if the initial performance metrics are not rigorously evaluated or if there's an overemphasis on simplicity. A model that is too simple might be easier to understand and implement, but if it consistently fails to capture the essential patterns, its "simplicity" becomes a severe drawback rather than an advantage.

Mitigating underfitting often involves increasing model complexity, introducing more relevant features through feature engineering, or potentially extending the model training duration. However, there's a delicate balance: increasing complexity too much without proper cross-validation can lead to overfitting, which is the opposite problem. Financial institutions are guided by stringent regulations, such as the Federal Reserve's SR 11-7, which emphasizes comprehensive model risk management to address potential adverse consequences from incorrect or misused model outputs, including those arising from underfitting¹. The challenge lies in finding the optimal level of complexity that allows the model to learn adequately without memorizing noise.

Underfitting vs. Overfitting

Underfitting and overfitting represent two ends of the model complexity spectrum, both leading to poor model performance. The distinction is crucial for effective model training.

• Underfitting: An underfit model is too simple to capture the underlying patterns in the training data. It performs poorly on both the training data and new, unseen data. This indicates high bias-variance tradeoff and low variance. Think of it as a student who hasn't studied enough and performs poorly on both practice tests and the actual exam.
• Overfitting: Conversely, an overfitting model is too complex and learns not only the underlying patterns but also the noise and specific irregularities present only in the training data. It performs exceptionally well on the training data but poorly on new, unseen data because it struggles to generalize. This indicates low bias and high variance. This is like a student who memorizes every detail of practice tests, including typos, but cannot apply the knowledge to new questions on the actual exam.

The goal in machine learning is to achieve a "good fit," where the model captures the essential patterns without memorizing noise, striking a balance between underfitting and overfitting.

FAQs

What causes underfitting in financial models?

Underfitting in financial models can be caused by several factors: using a model that is too simple for the complexity of the data (e.g., a linear model for non-linear relationships), insufficient or irrelevant input feature engineering, or inadequate model training time or iterations. Poor data cleaning can also contribute, as critical information might be missing or misrepresented.

How can I detect underfitting?

Underfitting is typically detected by evaluating the model's performance on its own training data. If the model shows high error rates or low accuracy even on the data it was trained on, it indicates underfitting. This can be observed through performance metrics such as mean squared error (MSE) or R-squared values, where poor scores suggest the model hasn't adequately learned from the data. Comparing training error with validation error can also reveal underfitting (both will be high, whereas with overfitting, training error is low and validation error is high).

How can underfitting be prevented or fixed?

To prevent or fix underfitting, you can:

• Increase Model Complexity: Choose a more sophisticated statistical model or algorithm that can capture more complex relationships.
• Add More Features: Incorporate more relevant input variables through thorough feature engineering that better explain the target variable.
• Reduce Regularization: If excessive regularization (a technique to prevent overfitting) is applied, it can simplify the model too much, leading to underfitting.
• Increase Training Time/Epochs: Allow the model more iterations during model training to learn from the data, though this needs to be monitored with cross-validation to avoid overfitting.
• Address Data Quality: Ensure the data is clean, complete, and accurately represents the underlying patterns. This includes proper data cleaning and preprocessing.