How accurate are the World Food Programme's machine-learning "nowcasts" of hunger (HungerMap LIVE's estimates of the share of people with insufficient food consumption) in the places where they are actually used, that is, regions and countries where no recent phone or household survey exists, rather than in the places already surveyed that the published accuracy figures were measured on?
- Good it would do
- 2 / 5
- Can AI help
- 4 / 5
- How neglected
- 4 / 5
- Can we check it
- 4 / 5
Why it's on hold
A first leave-country-out run on WFP's public 2022 archive doubles the error (R2 0.77 to 0.16), so the published accuracy does not transfer to new countries. But the lab can only audit the 2022 method, not the April 2026 operational system; accuracy in never-surveyed places cannot be identified from outside; and no user who would act on the result has been named (Good done 2).
Why it's stuck
The model was published by its own builders (Martini, Omodei and colleagues, WFP and Central European University, Nature Food 2022), who report that it explains up to 81 percent of the variation in insufficient food consumption (The Register, 16 September 2022). Its full training data, code and trained models have been public on Zenodo under CC-BY 4.0 since August 2022, but its accuracy has not been re-tested from outside.
What we'd try
1) Rerun the published XGBoost models from the public Zenodo archive (doi 10.5281/zenodo.6953981), then re-test them leaving out whole regions and whole countries rather than the last two months, and compare with simple baselines (the country's last survey value, or a model built only on undernourishment and GDP; the archive already ships these baseline configurations). This gives the error that applies in never-surveyed places.
We'd look again if
WFP (or an advisor in contact with WFP) confirms that HungerMap LIVE still shows model-based insufficient-food-consumption estimates for areas without a recent survey using the 2022 method or a close descendant, and a named team says it would act on a measured out-of-country error (uncertainty flags, survey targeting); or archived HungerMap nowcasts by region and date become available.
Think we got it wrong?
Ruled-out problems can come back. Tell us what we missed.
Members can read the full record for this problem, check its sources and comment. Open the member view →