Missing income data skew poverty metrics: Can AI fix them?

In a groundbreaking new study published in the World Bank Economic Review, economist Paolo Verme explores the accuracy of poverty prediction models under varying conditions of missing income data. Titled Predicting Poverty, the study pits traditional econometric models against machine learning (ML) techniques and evaluates their performance across multiple real-world data challenges.

With poverty estimation central to policy design and international development goals, this research offers critical insight into how best to approach prediction when income data are incomplete or systematically biased.

Why missing income data jeopardize poverty estimates

Missing income data are pervasive and consequential. From the U.S. Census’s Current Population Survey (CPS) to the EU’s SILC datasets, item and unit non-response rates can reach up to 50%, and they tend to be Missing Not At Random (MNAR). This means the likelihood of missingness is itself correlated with income levels, typically higher-income or extremely low-income individuals are less likely to respond. As a result, poverty statistics built only on observed income are systematically biased, misrepresenting the extent and depth of poverty.

These inaccuracies ripple through all domains of poverty measurement - from proxy means testing and poverty mapping to vulnerability analysis. Even global poverty tracking systems such as PovCalNet and the Luxembourg Income Study are affected. The study emphasizes that unless these data gaps are intelligently addressed, entire policy frameworks could be built on unreliable poverty statistics.

How do prediction models compare when income data are incomplete?

To evaluate the robustness of different prediction models, Verme constructed a benchmark dataset based on Morocco’s 2007 ENNVM survey, which had minimal missing income data. He then introduced artificial patterns of missingness, both random and systematically biased (MNAR), to test eight models: traditional Ordinary Least Squares (OLS) and Logit regressions, and six machine learning models including Random Forest, Elastic Net, and Neural Networks, each tested with continuous and categorical dependent variables.

Key findings revealed that:

• Random Forest (categorical) outperformed all models across most scenarios, especially under difficult missing data conditions (like MAR-MNAR) and at a 25% poverty threshold.
• OLS models performed worst when uncorrected, due to their inability to predict income extremes (tails of the distribution).
• OLS with error-term adjustments significantly improved population-level poverty predictions but lacked precision at the household level, making them unsuitable for targeting.
• Neural networks and Elastic Net models required extensive fine-tuning to match the performance of Random Forests and incurred high computational costs.
• Categorical models generally performed better than their continuous counterparts in estimating who is poor, rather than just the aggregate poverty rate.

Model effectiveness also depended heavily on poverty line thresholds and the pattern of missing data. No single model dominated across all settings. For instance, while Random Forest excelled at the 25% and 50% poverty lines, its relative advantage diminished at the 75% line under MNAR conditions.

Furthermore, dichotomous dependent variable models (those predicting a binary poor/non-poor outcome) proved more reliable at correctly classifying households than continuous models, particularly when probability thresholds were optimized using ROC curve methodologies.

What should policymakers and practitioners do?

The study concludes with actionable recommendations for development economists, data scientists, and policymakers:

• Use Random Forests when missing data patterns are unknown or highly MNAR. These models consistently deliver robust predictions with default parameters and minimal calibration.
• Avoid uncorrected OLS models for poverty estimation unless using population-level data with post-estimation error adjustments.
• Calibrate ML models like Elastic Nets and Neural Networks when time and resources allow, but expect diminishing returns in predictive accuracy relative to Random Forests.
• Optimize classification thresholds in categorical models to fine-tune sensitivity and specificity, particularly when designing welfare targeting systems.
• Treat model selection as context-specific. Factors such as poverty line choice, error tolerance (exclusion vs. inclusion), computational resources, and policy objectives should guide the modeling approach.

Verme’s research emphasizes that accurate poverty prediction is not only a statistical challenge but a policy imperative. As governments and institutions increasingly rely on data-driven decision-making, ensuring these models are rigorously tested against the realities of missing and biased data is crucial. The finding that Random Forest models offer a resilient and accessible option is a key contribution to the poverty measurement toolbox.

With a rising emphasis on AI and big data in development economics, this paper provides a roadmap for deploying these tools responsibly and effectively. As Verme demonstrates, thoughtful model selection and calibration can significantly enhance our ability to understand and address poverty, even in the face of incomplete data.