Spending time in nature is often therapeutic and provides many benefits, such as improving cognitive functioning (Berman, Jonides, & Kaplan, 2008) and improving memory (Berto, 2005). Spending time in a camp setting can cultivate honesty, respect, and trust (American Camp Association, 2005). No wonder, then, that several specialty camps have been developed over the past decades. Camps now exist for children with diabetes, cancer, and attention deficit disorder, just to mention a few. These specialty camps help campers to realize that they are not alone in experiencing their illness or condition, thus normalizing their circumstances.

One such specialty camp is the bereavement camp for children who have lost a parent or other loved one to death. These camps were first developed in the 1980s and grew in numbers in the 1990s and early 2000s (McClatchey & Wimmer, 2018). The camp setting lends itself especially well to the population of grieving children and adolescents. Children have a difficult time tolerating strong feelings for an extended time, and the opportunity to swim, hike, play ball, etc., gives these campers the message that it is all right to have fun and be children even if they have had the unfortunate experience of losing a person close to them.

Specialty camps are created with a certain mission in mind, and these camps are often supported by a nonprofit. Nonprofit organizations are dependent on donations from individuals and businesses, as well as grants from foundations. To stay active, it is important for these nonprofits to be able to show that what they do with the donated funds or grants meets the organization’s mission and has meaning to those who participate. They need to show a positive outcome, that camp creates a positive change in its participants. To do so, specialty camps should evaluate their programs.

Evaluation of practice seems to create some fear in those who run camp programs. However, the steps to evaluate camp outcomes do not have to be difficult, as the discussion that follows will show. (Note that while the focus here is on bereavement camps, the evaluation steps can be used for many specialty camps.)

Bereavement Camp Evaluation Background

Few outcome studies of bereavement camps are available in literature. Clute and Kobayashi (2013) examined outcome studies of bereavement camps and concluded that only eight studies were done in a way that could be said to measure bereavement camp effectiveness. Since this finding, The Moyer Foundation, Comfort Zone Camp, and the American Camp Association in collaboration with Kent State University have conducted a major long-term study of bereavement camps within the Camp Erin and Comfort Zone Camp networks (Richardson, Maxymiv, Freeman, Willis, & Taylor, 2014; Richardson, Maxymiv, & Freeman, 2014; Richardson, Ferguson, & Maxymiv, 2017). McClatchey and Raven (2017) have also contributed by studying posttraumatic growth among bereaved youth attending grief camps. Clute and Kobayashi (2013) stated, “Camp organizers must recognize the importance of ongoing evaluation of interventions implemented so that the most effective outcomes may be achieved.” There are hundreds of bereavement camps, and the organizations conducting these camps should evaluate their programs to ensure what they do is working. But do they?

To examine how many bereavement camps do indeed evaluate outcomes, or change, in their campers, McClatchey & Wimmer (2018) contacted bereavement camps throughout the country. Camps were located through an Internet search and 364 camps were identified. Out of these, the authors contacted those who were listed as overnight, or weekend camps, a total of 141 camps. Seventy-six of these camps responded to our questions. Ten percent of the responding camps did not evaluate their outcomes. However, 80 percent used satisfaction surveys, and another 10 percent evaluated the impact on campers as the outcome measure of camp.

What Is Evaluation?

Evaluation is often mistaken for research, but there is a distinct difference between the two. Because research sounds complicated, many organizations are hesitant to do what in reality are evaluations. According to Patton (2002), program evaluation is “the systematic collection of information about the activities, characteristics, and outcomes of programs to make judgments about the program, improve program effectiveness, and/or inform decisions about future programming.” The primary purpose of research, however, “is to generate or test theory and contribute to knowledge for the sake of knowledge . . . action is not the primary purpose” (Patton, 2002). Program evaluations can focus on what happened during the program delivery and perhaps identify why it happened (McClatchey & Wimmer, 2018). Once an organization knows what happened and/or why it happened, it can improve program services if the results are not what the agency wanted or expected them to be.

Satisfaction surveys are often mistaken for evaluations. Satisfaction surveys are questions asked of participants to find out how well the program ran and how well the participants liked the program. Satisfaction surveys may include statements such as, “The registration process ran smoothly,” “The counselors were helpful,” etc., which participants then rate on a scale from one to ten or one to five. Satisfaction surveys may also use open-ended statements that participants are asked to complete, such as “My favorite part of camp was . . . ,” or, “If I could change anything about camp it would be . . . .” However, rarely do the questions/answers on a satisfaction survey measure how participants have changed.

Why Evaluate

A camp program should evaluate its activities for several reasons. One of those reasons already mentioned is to know what happened. The reasons to want to know what happened are multiple. Foremost among those is the edict to do no harm. It may be difficult to imagine that camps can do harm bringing children to camp where the intention is for them to have fun, grow, make friends, and heal. However, it is possible. One of the author’s first two camp sessions 25 years ago proceeded without evaluations. Anecdotal feedback from campers and families was positive. At that point, when it was time to apply for grants and the program needed to show it was making a difference, two outcome measurements were applied: one on grief and another on posttraumatic stress disorder (PTSD) symptoms. To everyone’s horror, in spite of the fact that the grief symptoms decreased, the PTSD symptoms increased! This, of course, made the program examine what may have caused this situation and revamp the program. Originally the program only included activities based on grief theory, but after this discovery, trauma-focused and trauma-informed activities were added. Fortunately, further evaluations were positive.

Hand in hand with this reason to evaluate, to make sure to do no harm, is making sure the program is not wasting resources. If the camp does not benefit children and/or adolescents, then resources are clearly being wasted — including money and donations from grants, individuals, and businesses, as well as volunteer hours and camp facilities. Volunteers want to know that what they spend time on is meaningful. Without evaluating its activities, a program cannot tell volunteers what they do is important, nor tell sponsors that their money is well spent.

Who Wants to Know?

Naturally campers and their families want to know that they are sending their children and adolescents to a camp that is going to help or make a difference. The same goes for referral sources — they only want to refer to community resources they know will deliver a positive experience for the youth they refer. Staff and volunteers will want to know that what they spend valuable time on makes a meaningful difference to participants. And the board of directors of the bereavement camp put their names and reputations on the line by being on the board, making it imperative for them to know that what happens at camp is of value.

How to Evaluate

Outcome studies can be either qualitative or quantitative in nature. Qualitative studies use participants’ words to evaluate the outcome. Quantitative studies use numbers to quantify answers from participants often using a Likert scale, rating answers from one, “not at all,” to five, “all the time.” The simplest form is a survey, often created by the camp staff. Using a survey format, a qualitative outcome study of simply constructed fill-in-the-blank statements asks about behavior or various mental health situations, such as, “When I get sad and don’t want to get out of bed, I . . . ,” or “If somebody yells at me, I . . . .” The participants then simply fill in an answer. These questions can be given before and sometime after camp to see that a change occurred in the camper. Survey questions can also be quantified. Examples of quantified survey questions using the above questions may ask, “On a scale from one to five with one being ‘not true at all’ and five being ‘always true,’ when I get angry and want to hit somebody, I take deep breaths,” and “On a scale from one to five with one being ‘not true at all’ and five being ‘always true,’ if somebody yells at me, I walk away.”

A more sophisticated qualitative outcome evaluation involves a brief number of open-ended interview questions that may take up to an hour or more for participants to discuss. The questions are asked of a number of participants until the evaluator reaches saturation. Saturation is reached when consequent interviews are yielding no new information, and further evaluation data are not needed. To increase validity in a qualitative outcome evaluation, the camp evaluator may want to ask the same questions of different groups. For example, if the camp staff wants to evaluate changes in campers’ mood, they may ask a group of camp participants to talk about how their mood changed comparing before and after camp. In addition, they may ask the participants’ parents, and/or the participants’ teachers. This is called “triangulation.” Triangulation is defined by Creswell and Miller (2000) as “where researchers search for convergence among multiple and different sources of information to form themes or categories in a study.”

Instead of using self-composed quantitative surveys to evaluate change in campers, the camp staff may use questionnaires that have been authored to evaluate specific subjects, such as depression, loneliness, and cognitive growth. These questionnaires have been evaluated for validity and reliability and are easy to use. Fortunately, such questionnaires are also easy to find. Corcoran and Fischer (2013) have published a book, Measures for Clinical Practice and Research: A Sourcebook, with such measures that are for public use and can be used for free.

The advantage of qualitative outcome evaluations is that the programs get more information. When giving suggestions to answers in quantitative outcome evaluations, evaluators may miss information. The advantage of quantitative outcome surveys is that camp organizers can create averages of the answers, which is a less time-consuming way of finding out how camp impacted its participants. However, if there was a change in participants for the better, a quantitative outcome study will not tell you what at camp created the change. A qualitative outcome evaluation can be created to find out. Such an evaluation, one that measures outcome first using one method, a quantitative evaluation, followed up by a qualitative outcome evaluation, is called a mixed methods design.

What to Evaluate

Every nonprofit has a mission statement. The mission declares what the organization wants to accomplish. Mission statements for bereavement camps usually address one of two things: behavior and/or mental health. Often, the mission for bereavement camps is to provide healing or to normalize feelings surrounding loss, both of which look at mental health. Other missions include to give campers good coping skills, which addresses behavior. Unless camp organizations somehow evaluate the outcome of camp to see if the mission statement is accomplished, organizations have no way of knowing.

After deciding based on the mission statement what to measure, program staff need to decide how to measure. For example, if the program organizers have decided to measure coping skills based on the mission statement, an instrument has to be developed that addresses coping skills. Such an instrument may include questions such as, “When I get angry, I know what to do without hurting myself or others” with a scale from one (never) to five (all the time), and, “When I am sad, I know what to do without hurting myself or others” with the same scale. If the program has as a mission to “provide healing,” organizers must decide what that means. It may mean that depression among bereaved campers lessens after participating in camp. Therefore, the program staff can use an established questionnaire on depression. To measure change in camp participants, the questionnaire should be given immediately before camp and sometime after camp, usually three to four weeks later to allow the campers to process their experience.

Who Evaluates

The most common evaluators of camps are the camp organizers or staff. The advantage of having staff members evaluate the outcome is that they know the program and what happens at camp. However, they may also know it so well that they may interpret data in a more positive way than warranted. Also, when a staff member interviews participants, the participants may give responses they think the staff member wants to hear, often referred to as “response bias.” This problem can be solved by using an outside evaluator who is impartial to the outcome. Finances may preclude small organizations putting on bereavement camps from being able to hire outside evaluation help.

Ethics and Evaluations

When evaluating the outcome of camp, it is vital that organizations follow ethical procedures. Anybody who participates has to give consent prior to participation. Campers, who are underage, cannot give consent. However, their parents can. It is imperative, though, that the underage camper has a say in participating. Therefore, once a parent has given consent for their child to participate, the child should be approached for assent, or approval. Typically this should be done in writing after the evaluation exercise has been explained to the camper in age-appropriate language. Consents and assents both usually assure the participants of confidentiality — that is, their identity and answers will not be shared with anyone without their prior approval. At the same time, it is of utmost importance that the evaluator shares the limitations to confidentiality. If the camper shares with the evaluator that a parent or other adult is abusing them, the evaluator is under obligation to share that information with authorities. The same goes for information that divulges that the camper is planning serious injury to themselves or somebody else. Another limitation to confidentiality, although rare, is if the evaluator gets subpoenaed to court, in which case information may have to be shared.

Some evaluators obtain evaluations anonymously. This means that the evaluator does not know who said what — the participants’ names and identities are unknown to the evaluator. When using anonymous evaluations, the evaluator could run into a response that signals potential danger for a participant. For that reason, confidential evaluations may be preferred to anonymous ones.

Final Thoughts

In a world where time and money are hot commodities, it is vital for nonprofit organizations, bereavement camps included, and those who work/volunteer for them, to know that what they do is meaningful. Parents and caregivers of bereavement camp participants also want to know that camp has been beneficial. Outcome evaluations can definitively show this.

References

American Camp Association. (2005). Directions: Youth development outcomes of the camp experience. Martinsville, IN: American Camp Association.

Berman, M. G., Jonides, J., & Kaplan, S. (2008). The cognitive benefits of interacting with nature. Psychological Science, 19(12), 1207–1212.

Berto, R. (2005). Exposure to restorative environments helps restore attentional capacity. Journal of Environmental Psychology, 25, 249–259.

Clute, M. A., & Kobayashi, R. (2013). Are children’s grief camps effective? Journal of Social Work in End-of-Life & Palliative Care, 9, 43–57.

Corcoran, K., & Fischer, J. (2013). Measures for clinical practice and research: A sourcebook: Vol. 1. 5th Ed., Oxford, England: Oxford University Press.

Creswell, J. W., & Miller, D. L. (2000). Determining validity in qualitative inquiry. Theory into Practice, 39(3), 124–130.

McClatchey, I. S., & Raven, R. F. (2017). Adding trauma-informed care at a bereavement camp to facilitate posttraumatic growth: A controlled outcome study. Advances in Social Work, 18(1).

McClatchey, I. S., & Wimmer, J. S. (2018). Bereavement camps for children and adolescents: Planning, curriculum, and evaluation. New York, NY: Routledge.

Patton, M. Q. (2002). Qualitative research and evaluation methods. (3rd ed.). Thousand Oaks, CA: Sage.

Richardson, R. A., Maxymiv, S., & Freeman, P. (2014). Evaluation of bereavement camps: A positive youth development perspective. ADEC Forum: The quarterly publication of the Association for Death Education and Counseling, 40(4), 6–7.

Richardson, R. A., Maxymiv, S., Freeman, P., Willis, L., & Taylor, M. (2014, June). Understanding bereavement camps from a positive youth development perspective: A collaborative service provider-evaluator model. Presentation presented at the National Alliance for Grieving Children (NAGC) Annual Symposium on Children’s Grief, Atlanta, GA.

Richardson, R. A., Ferguson, P. A., & Maxymiv, S. (2017). Applying a positive youth development perspective to observation of bereavement camps for children and adolescents. Journal of Social Work in End-of-Life and Palliative Care, 13(2–3), 173–192. Retrieved from https://doi.org/10.1080/15524256.2017.1346544

Jane S. Wimmer, PhD, has a long career in US and international child welfare. Her specialties include program development and research, particularly in areas of adoption and child permanency. Jane was president of Child & Family Associated Consultants, doing research in varied areas of child and family well-being. She is a retired college professor and a volunteer with Camp MAGIK.