Asking better questions to create more equitable outcomes 

How do we know whether a nonprofit program is effective? How do we know whether we’re asking the “right” questions, in the “right” way, when designing and evaluating programs? We know individuals’ experiences and outcomes can vary by race and ethnicity, gender, socioeconomic status, and other characteristics. The Annie E. Casey Foundation’s recently published Guiding Questions to Advance Equity in Evaluation and Research offers tools to strengthen evaluation and research so we have the data and evidence needed to address disparities and identify solutions that meet the needs of those most affected. These questions can help design research and evaluations that are more inclusive when determining what is studied, how it is studied, and how the findings are used within nonprofit organizations and beyond. 

Asking better questions that center the needs of those most affected 

The questions are designed to be applied to any research and evaluation method or design, at any point in the process. For example: asking “Are community members and stakeholders provided resources to participate in the project, such as training or compensation?” supports their involvement by reducing barriers to participation for those closest to the issue—a practice that’s easy to overlook.  

Strengthening research and evaluation to create better outcomes means centering the perspectives and interests of the people a program or practice is designed to benefit. The larger aim, however, is to ensure that all who are closest to the issue not only have a voice in shaping the study, but also feel comfortable and engaged as full contributors. This includes, for example, nonprofit staff.  

What does asking better questions look like in practice? 

Explicitly address racial and ethnic inequities and identify solutions that are relevant and build on community strengths. The Working Students Success Network was designed to provide wraparound support—such as job placement, access to public benefits and financial coaching—to improve community college student retention and credential completion. Through an evaluation we learned that even though intensive one-on-one support improved students’ retention and completion, only a third of students were actually receiving it and that racial disparities in credit completion remained. When the Casey Foundation worked with researchers to design a follow-up study, we asked: “Do research and evaluation questions explicitly address issues related to racial and ethnic inequities?” and “Are methods designed to examine stakeholder, individual, community, and system-level assets?” As a result, colleges were engaged to define equity and build that definition into their approach to serving students. The college also sought to understand students’ perspectives and whether there were specific supports that they relied on. One key finding was the important role that campus cultural associations play in connecting students to institutional, academic, work and income supports and in fostering a sense of belonging for students of color. 

Ensure community members lead and benefit from the research. Through the East Baltimore Research Project, the Casey Foundation wanted to know if residents, particularly residents of color, were benefiting from substantial capital and social investment or had access to the neighborhood resources that would help them thrive. An important question emerged: “Who decides what it means to ‘thrive’?” Residents expressed that they had experienced a long history of “being researched” and wanted instead to have access to and benefit from research conducted in their own community. Some of the Guiding Questions relevant in designing that study were: “Is it clear how key community members will be able to use the findings? Will they build community capacity?” The result was a resident-led research project where community members defined the questions, carried out the data collection and analysis, and shared the findings with the community to inform their own community change efforts. 

What’s in it for nonprofits? 

Nonprofit staff can use these guiding questions in their conversations with funders and researchers to strengthen studies and programs by incorporating the diverse voices and experience of staff and program participants. Usable outside of research and evaluation, the Guiding Questions can also apply to other contexts to create shared processes for decision making and ensure that programs and policies take the firsthand experiences and perspectives of communities into account throughout their design and implementation. 

The Guiding Questions resource is structured to help the user examine both the big picture and the small details. As you dive into examining specific aspects of a study, you also situate it in the process as a whole. This ensures at every step you’re building in practices that are more likely to lead to equitable outcomes. Asking better questions designed to remove barriers helps us envision a world where all young people and families can thrive, no matter where they live or what their background is.  

Photo credit: Chiaki Kawajiri, courtesy of the Annie E. Casey Foundation