Picture this: You notice that a small sapling you planted a while ago is at last large enough that it no longer requires stakes to stay upright. It takes two hands to wrap around its now-sturdy trunk, and the branches, in midsummer, provide shade. It might even be a fruit tree, and you think that, after years of nurturing that tiny sapling, you might actually taste one of its fruits.

You are proud of this tree, and you should be — it takes care and effort and a whole lot of patience to grow a tree, and you did that. You tended the tree, and it grew. But can the tree survive on its own? Can it withstand a storm? A drought? A pandemic?

The metaphor, of course, is about trees and what they can teach us about evaluation. The tree you’ve nurtured for so long is a system of gathering information about your participants and/or your programs and using that information to support your work at camp. Maybe it’s a survey you developed, or maybe you use Camp Program Quality Assessment or ACA’s Youth Outcomes Battery. Whatever tools you are using, the most important part is that you are using what you learn for continuous program improvement — and it has taken a lot of work to get it to this point.

But what happens to that system in our current COVID-19 crisis? Your attention is understandably drawn to the most pressing concerns, namely the health and safety of your campers, your staff, and yourself. You might also be preparing for a drastic reduction in staff and revenue — how could you possibly think about evaluation during these uncertain times?

You might think about evaluation because you’re using new formats to deliver your programming and you don’t know yet how your new programming affects your participants. Program evaluation is a way for you to see how well you’re meeting your goals. Choosing to focus on short-term outcomes or measuring the outputs (i.e., numbers of people reached, activity types) of your efforts is a reasonable approach for brand-new programs. Evaluation results can be quickly synthesized to inform design of emerging programs, in a continuous cycle. You know all of this, yet it is hard to prioritize evaluation when there is so much uncertainty.

Our tree metaphor is one way to think about evaluation in the current environment. Trees just might be one of the few certainties we can look to—they are, after all, ancient organisms that have survived for a very long time. Below we consider trees and some ways they can teach us about nurturing evaluation, even in these uncertain times.

Trees Regulate Their Systems in Times of Stress

Trees conserve water during a drought, and they have many other ways of regulating their systems in times of stress—they drop leaves or produce smaller fruit or no fruit at all, but they stay alive.

What does this teach us about evaluation?

Our evaluation systems can expand and contract, but they do not go away completely. Evaluation is about building the capacity for learning more so than the results from a specific survey or data, and this learning capacity starts with you, your team, and everything you know and value about evaluation. That will not go away, even if you do not run a survey because your attention is needed elsewhere, or your program looks much different than you originally planned.

Trees Are a Single, Deeply Interconnected Organism

Some species, like aspen, are connected at their roots—a single organism that supports hundreds of thousands of what look like individual trees. Other species have complex biological systems of communication that allows trees facing threat to alert others in the species across hundreds of miles.

What does this teach us about evaluation?

You are not alone. You built your evaluation systems over time by engaging others from your team and possibly other camp professionals. This network does not disappear if you pause some aspects of your evaluation system; in fact, this network is the very place that will support your evaluation capacity, even as it constricts. Connecting with others — both internal to your camp program and externally, like through ACA Connect, is a powerful way to nurture your evaluation systems. Share what’s working for you and get inspired by others’ stories.

Trees Have Multiple Ecosystems — at the Soil, at the Canopy, Even in the Trunk Itself

At and in the soil are countless micro-organisms—mushrooms are a great example of the unique nature of trees and the ecosystem created by their roots and leaves and dropped fruit. Even fallen trees continue to nurture this ecosystem. Up top trees with expansive canopies support entire species of birds, insects, and micro-organisms. They even support their own microclimates!

What does this teach us about evaluation?

Trees and their numerous ecosystems are a way to think about your outcomes—your goals for camper growth and learning during their time at camp. You likely have several intended outcomes, and, like trees, each of these outcomes has its own ecosystem. These outcomes do not go away or change because you are adjusting your programming for this spring or summer. What are the aspects of these outcomes that you can retain as you consider reducing programming, or shifting to virtual programming?

These are uncertain times that undoubtably require prioritizing some things over others. Your health, and the health and safety of your camp community, are your focus right now. You are likely concerned about your camp’s financial health too. Evaluation is very likely somewhere down on your list.

Our tree metaphor teaches us that your evaluation capacity might expand or contract, but the tree will survive. Think about your systems and what you might conserve—you might do one less survey, or try something more simple, like using needs assessment interviews with key constituents or prioritizing counting numbers of participants or level of engagement in your new programming formats. Trees also remind us that we are deeply connected and our networks are vital to survival of systems that you might pause during the COVID-19 crisis. But don’t lose sight of the ecosystems that support your outcomes—those ecosystems are robust enough that you can take a different route to reaching those outcomes, whether it’s camp as usual or some completely new approach.

As researchers and evaluators, we are eager to support our camp community as they face these times of tremendous uncertainty. The next two blogs in this series will explore how we can evaluate virtual activities, first by looking at the logic model and what it can tell us about virtual programming and then by brainstorming specific ways we can evaluate virtual activities.

In the meantime, be well!

Laurie Browne, PhD, is the director of research at ACA. She specializes in ACA's Youth Outcomes Battery and supporting camps in their research and evaluation efforts. Prior to joining ACA, Laurie was an assistant professor in the Department of Recreation, Hospitality, and Parks Management at California State University-Chico.  Laurie received her PhD from the University of Utah, where she studied youth development and research methods.

Ann Gillard, PhD, has worked in youth development and camps for more than 25 years as a staff member, volunteer, evaluator, and researcher.

Thanks to our research partner, Redwoods.

Redwoods

Additional thanks goes to our research supporter, Chaco.