Research 360: The Quality vs. Outcomes Smackdown

November 15, 2016
Laurie Browne, PhD

If you've been following this blog you know that the Research 360 goal is to build camp evaluation capacity. Thus far I've described what this means and why it is important, and in the most recent post I suggested that camp evaluation capacity is all about quality because quality camp experiences are more likely to produce positive camp outcomes.  In that post I also described the value of quality assessment, and suggested that, ideally, camps would assess both quality and measure outcomes. 

And that's where I stopped (the closest thing you'll get to a cliff-hanger in a blog all about research). Assess quality and measure outcomes? That probably sounds daunting, time consuming, and not at all fun. So, as promised, in this post I will put the two side by side: a smackdown if you will.  Why such a contentious approach?  Because, despite their synergies, quality assessment and outcomes measurement are often pitted against one another when talking about youth program evaluation.

First, let's look at what they are:

Quality assessment focuses on program practices- staff, activities, policies- and asks if those practices include the features known to promote positive youth development, and if those practices are improving over time. Outcomes measurement focuses on what campers get out of the camp experience – changes in skills, behaviors, beliefs, or attitudes that occurred because of camp.

Or, here is another way to think about it:

Process

Product

But these definitions alone do not explain why some people take a strong one-is-better-than-the-other stance to evaluating youth programs. For the sake of simplicity, we will compare the tools ACA offers to assess quality and measure outcomes:  the Camp Program Quality Assessment and the Youth Outcomes Battery. There are other tools available to do both of these things, but we'll use these tools because both were created specifically for use at camp.

The table below shows how these two tools stack up:

Camp Program Quality Assessment vs Youth Outcomes Battery

Note: The "criteria" used in this table are based on the criteria used in the Forum for Youth Investment's Soft Skills Hard Data, an analysis of common youth program assessment tools.

So, which one is the smackdown champion? You guessed it: both. Quality assessment is a great tool of staff training and program improvement and outcomes measurement is a great tool for advocating for the value of camp. Each one tells an important yet incomplete part of your campers' experience at camp, but together they allow you to a story about what campers are getting out of your camp and how

But, as we've discussed before, your camp's evaluation capacity is unique, so conducting both quality assessment and measuring outcomes might not be possible. In the next post, we will suggest some specific tips on using them independently and using them together so you can tell that more complete story.

Laurie Browne, Ph.D., is the director of research at ACA. She specializes in ACA's Youth Outcomes Battery and supporting camps in their research and evaluation efforts. Prior to joining ACA, Laurie was an Assistant Professor in the Department of Recreation, Hospitality, and Parks Management at California State University-Chico.  Laurie received her Ph.D. from the University of Utah, where she studied youth development and research methods.

Add comment

CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.