Non response

What Is Non-response?

Non-response in the context of financial and economic research refers to the failure to obtain data from all individuals or entities selected for a sample in a survey or study. This issue falls under the broader field of survey methodology, which examines the design, collection, and analysis of survey data. When participants are unwilling or unable to provide information, it creates gaps that can significantly affect the accuracy and reliability of the findings. Non-response can occur in various forms, including a complete refusal to participate in a study (unit nonresponse) or the failure to answer specific questions within an otherwise completed survey (item nonresponse). Understanding and addressing non-response is crucial for maintaining data quality and ensuring that conclusions drawn from survey data are representative of the target population.

History and Origin

The challenge of non-response has been inherent in survey research since its inception. As formalized survey methods gained prominence in the early to mid-20th century, particularly in social sciences, economics, and market research, researchers quickly recognized that not everyone selected for a study would participate. Over time, particularly from the mid-1990s onward, survey response rates have seen a significant decline across various types of studies, including federal surveys. For instance, the Current Population Survey (CPS), a key source of U.S. labor market information conducted by the Census Bureau for the Bureau of Labor Statistics, has observed accelerated declines in response rates, especially since 2020, partly due to the COVID-19 pandemic.²⁷ This trend highlights a growing concern for statisticians and economists alike, as it complicates the process of gathering comprehensive and unbiased economic data. Addressing non-response has thus become a central focus for improving the integrity of survey-based insights.

Key Takeaways

• Non-response occurs when selected individuals or entities do not provide data in a survey or study.
• It encompasses both complete non-participation (unit non-response) and incomplete responses (item non-response).
• High rates of non-response can introduce bias, leading to inaccurate and unrepresentative conclusions.
• Factors contributing to non-response include personal concerns, survey length, and difficulty in contacting participants.
• Mitigation strategies involve follow-ups, incentives, and weighting adjustments.

Interpreting the Non-response

Interpreting non-response is less about a numerical value and more about assessing its potential impact on a study's validity. A high non-response rate alone does not automatically mean the results are biased; rather, it indicates a higher potential for non-response bias if the characteristics of non-respondents differ systematically from those who do respond. For instance, if a financial survey aimed at understanding investment habits receives responses primarily from individuals with higher incomes, while lower-income individuals are less likely to participate, the resulting data would skew towards the perspectives of the wealthier group. This misrepresentation can lead to flawed market research strategies or incorrect economic forecasts. Researchers often analyze the demographics and known characteristics of both respondents and the original sample to infer potential biases. This involves comparing the composition of the responding group to the known characteristics of the total target population or initial sample.

Hypothetical Example

Consider a hypothetical financial research firm conducting a survey on investor confidence in a particular stock market sector. They send out 1,000 questionnaires to a randomly selected group of investors. Only 300 investors complete and return the survey, resulting in a 70% non-response rate.

Upon reviewing the responses, the firm notices that the majority of respondents are highly active traders who invest frequently, while their initial sample included a significant proportion of long-term, passive investors. This disparity suggests potential non-response. The passive investors, perhaps less engaged with daily market fluctuations or less inclined to participate in surveys, may have been less likely to respond.

If the firm were to conclude from the 300 responses that "investor confidence in the sector is very high," it would be biased. The high confidence might only reflect the views of active traders, who may have different risk appetites or information access compared to passive investors. To mitigate this, the firm might attempt follow-up contacts with non-respondents, or apply statistical weighting adjustments based on known characteristics of the full sample, such as trading frequency or portfolio size, to make the results more representative of the original 1,000 selected investors.

Practical Applications

Non-response is a critical consideration in various areas of finance and economics where data collection relies on surveys:

• Economic Surveys: Federal agencies, such as the Bureau of Labor Statistics and the U.S. Census Bureau, regularly conduct surveys like the Current Population Survey (CPS) and the American Community Survey (ACS) to gather vital economic data. Non-response in these surveys can affect estimates of unemployment rates, income distribution, and poverty.²⁵, ²⁶ The Federal Reserve Board also conducts the Survey of Consumer Finances (SCF), which provides crucial insights into household wealth and financial behavior.²³, ²⁴ Non-response in the SCF, particularly from wealthy households, can pose challenges to data accuracy.²²
• Market Research and Public Opinion Polling: Businesses and financial institutions use surveys for market research to understand consumer sentiment, product demand, and investment preferences. If certain customer segments are underrepresented due to non-response, companies may develop strategies that fail to resonate with their broader audience.²¹
• Academic Research: Researchers in behavioral finance and other economic fields often rely on survey data. Unaddressed non-response can compromise the validity of their statistical analysis and the generalizability of their findings, potentially leading to incorrect conclusions about financial behaviors or market dynamics.¹⁸, ¹⁹, ²⁰
• Regulatory Reporting: In some cases, financial entities may be required to collect data from specific populations. Non-response can hinder compliance or obscure critical insights needed for effective regulatory oversight. Best practices for reporting on non-response bias are emphasized by organizations like the Federal Committee on Statistical Methodology (FCSM) for federal surveys to ensure data quality.¹⁷

Limitations and Criticisms

Despite efforts to mitigate it, non-response remains a significant challenge in survey-based research, and its inherent limitations warrant critical consideration. A primary criticism is that non-response can introduce bias, meaning the characteristics of respondents may differ systematically from non-respondents, leading to distorted results. For example, individuals with strong opinions (positive or negative) might be more inclined to respond to a survey, while those with moderate views or who are indifferent might not, creating a skewed representation.¹⁶

The assumption that non-response is random is often incorrect; sensitive questions, survey length, or privacy concerns can disproportionately affect who participates.¹⁴, ¹⁵ Financial surveys, particularly those asking about income or wealth, frequently encounter higher non-response rates due to the sensitive nature of the data requested.¹¹, ¹², ¹³ This can lead to an underestimation or overestimation of certain financial metrics in the population.

While techniques like weighting adjustments and imputation are used to compensate for missing data, their effectiveness depends on the accuracy of the assumptions made about non-respondents. If the underlying reasons for non-response are not well understood, these adjustments may not fully correct the bias.⁹, ¹⁰ Furthermore, a high non-response rate can reduce the effective sample size, increasing the sampling error and widening confidence intervals, which limits the precision of estimates.⁸ Some survey methodologists argue that the emphasis on achieving high response rates might sometimes lead to increased survey costs without necessarily improving data quality or reducing bias, as bias is item-specific and not solely determined by the overall response rate.⁶, ⁷

Non-response vs. Non-response Bias

While often used interchangeably, "non-response" and "non-response bias" refer to distinct but related concepts.

Non-response is the phenomenon itself—the failure to collect data from all units or items in a selected sample. It is a descriptive term for the absence of data. For example, if a company sends out 1,000 customer feedback surveys and only 400 are returned, the non-response rate is 60%.

Non-response bias, on the other hand, is a specific type of systematic error that can result from non-response. It occurs when the characteristics, opinions, or behaviors of those who do not respond to a survey are significantly different from those who do. The "bias" part implies a systematic distortion of results. For instance, if only highly satisfied customers respond to a survey, the survey results would inaccurately suggest universal satisfaction, because the dissatisfied customers (non-respondents) were not captured. The presence of non-response bias is a concern because it can lead to inaccurate conclusions and potentially misleading insights in areas like financial reports or policy decisions.

Essentially, non-response is a condition, while non-response bias is a potential consequence that arises when non-response is not random. All studies with non-response face the risk of non-response bias, but bias only occurs if there is a systematic difference between respondents and non-respondents.

FAQs

Why is non-response a concern in financial surveys?

Non-response in financial surveys is a concern because it can lead to inaccurate or misleading conclusions. If certain groups, such as those with particular income levels or investment behaviors, are less likely to respond, the survey data may not accurately represent the broader population. This can impact financial forecasting, market research, and policy decisions.

What causes individuals to not respond to surveys?

Individuals may not respond to surveys for various reasons, including the survey being too long or complex, concerns about privacy or the sensitivity of questions (especially regarding personal financial reports), inability to be contacted, or simply a lack of interest or time.

⁴, ⁵### How do researchers try to minimize non-response?
Researchers employ several strategies to minimize non-response. These include making surveys concise and easy to understand, providing incentives for participation, conducting follow-up attempts (e.g., reminders or re-contacts), using multiple contact methods (mail, phone, online), and reassuring participants about data confidentiality.

², ³### Can non-response bias be completely eliminated?
Completely eliminating non-response bias is often challenging, if not impossible, in most real-world surveys. The goal is typically to minimize it and to understand its potential impact through statistical analysis and adjustments like weighting adjustments, which attempt to make the responding sample more reflective of the overall population.¹