A New Year, a New Approach to Evaluation

January 2, 2018
Laurie Browne, PhD

Greetings camp evaluators, and Happy New Year! The year 2018 brings with it a chance to try things or look at things you are currently doing with a fresh set of eyes. Yes, of course, I am talking about evaluation, which can include anything you do at your camp to systematically gather information, analyze that information, and use what you learned to make decisions about your programs, staff, policies, or practices. Whether you do these things or not, the new year is an excellent time to consider trying something new or changing something that you’ve done in the past.

With that in mind, I’m going to suggest that you try one specific thing — or rather, approach — this year in your evaluation efforts. The approach is play, as in:

That’s right, I’m recommending that we approach our 2018 evaluation efforts with enjoyment and for the sake of recreation, and without a focus on serious or practical purpose. Odd, I know, to suggest that evaluation could be without practical purpose because evaluation is, by definition, all about practicality (unlike research, which tends toward the theoretical). But is also represents an opportunity to challenge this and other assumptions we hold about evaluation — that it is always practical, that it is always serious, and it is by no means an activity done for enjoyment and recreation.

With these assumptions out of the way, let’s think about how the ancient concept of play might inspire new and exciting evaluation efforts in 2018. We’ll use the concept of play outlined in this post as our guide.

Play Is Self-chosen and Self-directed; Players Are Always Free to Quit

Evaluation is often something we are compelled to do — either by administrators or funders or by our own interest in improving a program. In most cases, these compelling factors are not likely to change, nor are they likely to be impressed if you say, “Nope, I do not freely choose to evaluate my program. I quit.” To be more playful, then, we might need to think about our frame of mind. You might not be able to choose if you are going to evaluate your program, but your how, why, and when are likely better candidates for self-choice and self-direction. Let’s make evaluation more playful in 2018 by choosing to focus on all of the ways evaluation can be self-directed, and, if you choose a method that doesn’t work, you are always free to quit (that method).

Play Is Activity in Which Means Are More Valued than Ends

Here’s a radical idea: What if evaluation was less about the results and more about the process by which you achieved those results? Results — especially when they are in the form of numerical data — are, for some, the most compelling reason to engage in evaluation. Sure, funders and administrators, for example, want evidence of this sort to determine the value and success of your program, so we cannot throw out the results entirely. But we can, if we want to be more playful, spend as much time thinking about the process as we do the results. Instead of looking only at a final set of numbers, why not ask you got there? And, to be truly playful, you might even celebrate this process — the process in which you are trying new evaluation methods, learning from the process, and improving those methods next time around. The results are just part of the scenery.

Play Is Guided by Mental Rules

Think back to your playground days and the elaborate rules that guided kickball, the tire swing, and four-square. For me, those rules went far beyond the rules of the game; they included implicit rules about taking turns, lining up, and playing fair(ish) that everyone on the playground knew and upheld. The mental rules that guide play are organic, meaning they arise from the playing over time, and they are shared by the other kids on the playground. The rules of evaluation, on the other hand, are often more a priori (determined in advance or by someone else) and can be unclear, depending each individual’s level of involvement and buy-in for the evaluation effort. Let’s make the rules of evaluation more play-like in 2018: Allow them to emerge as you move through early, low-stakes evaluation efforts (test runs), and make sure that everyone on the team has a say in what they are and how they are governed.

Play Is Non-literal, Imaginative, Marked off in Some Way from Reality

To think about evaluation as imaginative might require some, um, imagination, but stay with me here. Consider this: who decides what and how to evaluate? Who convinces others that evaluation is important and teaches them how to do it? Who looks at the results and makes meaning of them, and then uses this interpretation to make decisions? The answer might be you, or it might be another person in your organization; either way, that person is likely a human, and that human likely has some degree of creative tendency. Instead of running from the very human aspects of evaluation, the concept of play teaches us that imagination might be the key to unlocking the things that are most visible to the literal eye. To be more play-like this year, let’s embrace the imaginative aspects of evaluation planning, interpretation, and use, and acknowledge the powerful role our innate creativity necessarily plays in our evaluation efforts.

Play Involves an Active, Alert, but Non-stressed Frame of Mind

And this is perhaps the biggest challenge for the upcoming year: To approach evaluation with a non-stressed frame of mind. For many camp professionals, evaluation is a high-stakes enterprise in which a camp’s evaluation results have important implications on their success in the future. Compounding the stressed frame of mind so many people feel when evaluating their programs are the complex, unpredictable, and ever-moving factors that bubble up any time we try to evaluate campers, staff, or programs. The challenge here is not to run from these factors nor to ignore them; rather, the concept of play teachers us to approach challenges with an active and alert mind, but not one tangled up in worry, anxiety, or dread. Yet again we have an opportunity to reset our frame of mind so we can approach evaluation in a more play-like way.

Evaluation is important business, but so is play. In fact, play is critical to young people’s development. So true for evaluation: Without play, we cannot create (and recreate), imagine, test, fail, learn, and improve. Let’s make 2018 a year for playful evaluation.


Laurie Browne, PhD, is the director of research at ACA. She specializes in ACA's Youth Outcomes Battery and supporting camps in their research and evaluation efforts. Prior to joining ACA, Laurie was an assistant professor in the Department of Recreation, Hospitality, and Parks Management at California State University-Chico.  Laurie received her Ph.D. from the University of Utah, where she studied youth development and research methods.\

Photo courtesy of Camp Weequahic in Lakewood, Pennsylvania

Thanks to our research partner, Redwoods.

Redwoods

Additional thanks goes to our research supporter, Chaco

Add comment

CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Image CAPTCHA
Enter the characters shown in the image.