Data matters. In light of the reports of inequitable data collection related to COVID-19 and the advocacy efforts to record data disaggregated by race, we need to think about what such data can tell us and why it is important.
First, the requirements for equitable data collection are complex. Data collection is just one area where inequities occur. Project design, analysis, interpretation and knowledge translation also have inequities embedded in them. Our biases start from the moment we initiate a project that involves data collection.
Second, in the recent Atlantic article, What’s Behind the COVID-19 Racial Disparity?, author Graeme Wood writes that our outrage with COVID-19 racial disparity is warranted, but “outrage unaccompanied by analysis is a danger in itself”. In other words, we need to understand that such omissions stem from specific sociocultural and historical issues, many of which fall into “social determinants of health”.
When we hear the calls to collect data disaggregated by race, we need to remember that collection of such data is not intended to be done for its own sake. Rather, categories of race, gender, age along with other determinants of health tell us a much more comprehensive picture of why certain groups are at a higher risk for COVID-19 and other conditions, including HIV and hepatitis C that are the focus of PAN’s work. There are multiple and often intersecting factors that contribute to a health inequity. For example, a combination of one’s age, immigration status, ethnicity, and geographic location (rural/urban) can create a unique set of conditions that make this individual more vulnerable to a specific health risk.
But no matter how much we account for the demographic characteristics in our data collection, we also need to develop ways of layering this data with additional data on the structural barriers and the broader conditions that foreground the deep-seated racism and discrimination embedded in the systems. By applying such an intersectional approach, we avoid shifting the responsibility and stigma on to the communities that experience these inequities.
Evaluation can offer some ways for ensuring that meaningful data is collected.
Evaluation looks at what is called “systems change” that encourages us to think about the complex interplay of factors, processes, attitudes, personal characteristics and policies and helps us develop the mechanisms for untangling the issue. In order to do this, we need to collect multiple types of data to get to the bottom of a systemic problem, such as discrimination and stigma.
Evaluating systems change efforts is not easy. As the figure 1 shows, such process is complex, haphazard and hardly linear.
What we can do as evaluators is continuously question versions of reality that sustain oppression. We also need to strive to understand the systemic drivers of problems so that our evaluation methodologies capture insights to solutions and give voice to those most marginalized.
 Hopson, R., & Cram, F. (Eds.). (2018). Tackling Wicked Problems in Complex Ecologies: The Role of Evaluation. Stanford University Press.
Questions? Feedback? Get in touch!
Alfiya Battalova, Evaluation Manager