Hello colleagues! Yesterday was Dr. Martin Luther King Jr. Day, and a day that holds a lot of meaning for our country, our communities, and our work at camp. But this day seems to hold particular meaning this year as our nation faces unprecedented ideological divisiveness and uncertainty. So, in recognition of this day and the hard work that lies ahead, I must write about the one topic you thought you were safe from in a blog about evaluation: politics.
You still here?
Good — because I am going to talk about the kind of politics that evaluators need to think about, especially evaluators who care about their work and the people they serve. Politics, in this sense, refers to the human tendencies and organizational pressures that prevent evaluation from being a purely objective, scientific practice.
Evaluation guru Michael Quinn Patton says that politics are the one constant in evaluation. He offers, in his foundational evaluation text Utilization-Focused Evaluation (2008) the following (among others) conditions under which evaluation is not political:
- When no one cares about the program
- When no one knows about the program
- When no money is at stake
- When no power or authority is at stake
According to these criteria, all evaluation is political, even when it is new, small-scale, or the results are not used (this might be when evaluation is the most political). At their core, evaluations are political because they always involve humans making decisions for and about other humans. And if you are a human like me, you know that we humans are imperfect, we want to do the right thing, we want to be accepted by others, and we are influenced by our own deeply-held world view. As long as we are human, our evaluation efforts will be political.
As onerous as that sounds, the solution is simple: we embrace it by acknowledging that political forces — such as those described below — direct and shape why and how we evaluate our programs, campers, staff, and parents.
We cannot avoid politics, but, as camp evaluators, we can ask ourselves:
- What are the sources and impacts of bias? Our biases, which are rooted in our culture, our gender, and our beliefs, to name just a few, are natural and unavoidable. As such, our evaluations are biased; however, we can reduce the negative impacts of bias by acknowledging how these biases might impact the questions we ask and our interpretation of our findings. Do this by making a list — at any point in an evaluation project — of the sources of bias inherent to you and your team and how these biases could possibly skew your results. Then, brainstorm ways to avoid biased questions and methods, like asking a third party to review a survey or interview questions.
- What are the sources and impacts of power? – Power, like politics, is a charged word, but, like bias, it is inherent in most social situations. Camps are social contexts that typically include a director, administrators, frontline staff, campers, and parents. Each of these roles engages in a different power dynamic: director-frontline staff, frontline staff-camper, and director-parent (who holds the power in this dynamic is another topic entirely!). Power influences evaluations when it prevents certain things from being asked or certain people from being involved. You can minimize the negative impact of power by inviting the group with less power- say campers or frontline staff — to help design and administer the evaluation, and to help interpret the results.
- What are the cultural assumptions? Think about the many different ways people greet one another around the world — and how this makes the world beautiful and interesting. Cultural norms like this influence our evaluations as well — from what we ask to how we ask it, to the fact that we are asking something at all. Consider carefully the people involved in your camp evaluation and their unique cultural norms. Better yet, invite people from less dominant cultures (this includes racial and ethnic cultures, gender- and income-based cultures, and geographic cultures), to help you design your evaluation to ensure it is appropriate and accessible for everyone.
If you are still with me, give yourself a pat on the back- we just talked about politics without once mentioning politicians or political parties. We did, however, talk about the politics of evaluation, which is a hefty topic. The things I’ve asked you to consider here are not easy, but are critical if we are to use evaluation to tell our best and most powerful camp stories.
To that end, I leave you with the following:
"Faith is taking the first step even when you don’t see the whole staircase." – Dr. Martin Luther King, Jr.
Thanks to our research partner, Redwoods.
Additional thanks goes to our research supporter, Chaco.