By Joe Grange
In the summer of 2021, I was supporting a USAID data quality initiative with the Bureau for Humanitarian Assistance. The Bureau had more than seven years of data on child malnutrition that had been collected by the Office of Food for Peace. As we looked more closely at this rich source of aggregated health data, we noticed a pattern: the reported ages of children were spiking at—exactly—one, two, three, four years of age. We came to find that this was a case of what’s known as ‘age heaping.’ As it turned out, paying attention to and understanding the data shed light not just on the dataset itself, but on how knowledge networks and localizing research, monitoring, and evidence-generating systems could improve data quality more broadly in global health.
My support to the Bureau for Humanitarian Assistance was made possible through the (recently renamed) Bureau for Planning, Learning, and Resource Management’s Program Cycle Mechanism. The Program Cycle Mechanism, led by Environmental Incentives, helps USAID put design, monitoring, evaluation, and learning policy into practice: in other words, it helps equip teams to adapt development programming with evidence. The Program Cycle Mechanism is also a model of sustaining and utilizing knowledge networks—its team of technical specialists regularly exchange experiences, knowledge, and tools with one another to amplify best practices and routinely champion opportunities and platforms for knowledge sharing throughout the Agency.
Without Context, Evidence Can Be Misleading
In this sense, I was well-prepared when a colleague I connected with through USAID put me in touch with UNICEF—a UN-mandated organization that supports, among other humanitarian issues, child health and nutrition—to share insights about child malnutrition data and population-based surveys. What these insights revolved around was that a seemingly simple survey question about a child’s date of birth often relied on a rounded estimate. This was because the question did not factor in context, where in some parts of the world, particularly those facing crises and conflict, there may not be dependable vital registration systems available to draw accurate birth dates from. Outside of medical records, culture also plays a role in how, and whether, birthdays are ‘marked’ as such, by celebrations or other forms of recognition.
In the absence of medical records or cultural touchstones, a researcher or a caregiver may estimate a child’s birth date. The consequence of birth dates being systematically estimated—and possibly ‘heaped,’ for example, on the nearest six- or twelve-month mark—is that it affects the accuracy of the data, and correlations that might be drawn from the data. Some studies have noted that the misidentification of a child’s birth date can lead to a misestimation of malnutrition prevalence by up to 30 percent. For an individual, a mischaracterization of their indicator value is not so serious, but when aggregated to the scale of a population it can lead to problematic resource allocation due to misinformed decision making.
Going Beyond Reporting
Sharing similar research priorities related to data in child health, UNICEF invited me to collaborate with them and lead the production of a technical brief related to this issue of biased malnutrition data. Improving data quality in the area of child malnutrition, and in the global health sector, will come down to more than one solution, but there are several, overarching considerations that will be key. One of these is the importance of localizing the monitoring and evidence-generating systems tied to health. The inaccuracies of the standard procedures in determining and recording birth dates show that cultural and other context needs to be taken into consideration to ensure the validity of the data. This can be done, for example, by co-developing local event calendars with local experts or other stakeholders to then be able to ‘sandwich’ birth dates between a recognized event before and after a child’s birth, thereby narrowing the estimable date range. Understanding these best practices in how to collect health information, as well as how it varies by culture and context, will be key to closing this knowledge and measurement gap.
Strengthening and utilizing knowledge networks can also improve global health data. My work with USAID through the Program Cycle Mechanism has deepened my appreciation for the direct impact of collaborating and sharing knowledge with colleagues and strengthened my skills in this area. My eventual collaboration with UNICEF would not have been possible without this background and without the appetite of all parties to come together to share and learn from each other and advance data quality and evidence.
In the same vein, data should ultimately go beyond reporting: it should be useable and actionable, a foundation for decision making and adapting programming. There are important instances where lives are temporarily distilled into numbers—such as in the case of individual and aggregated health data. Localized and adaptive health programming is what ultimately transforms those numbers back into real lives.
Photo by USAID-ACCESO/Fintrac Inc.
