Do You Know What Your Outcomes Are?
The Impact of Oregon 4-H Residential Camp Programs on Positive Youth Development

by Mary Olszewski Arnold, Ph.D., Virginia D. Bourdeau, and Janet Nagele

As camp youth development professionals, we all try to keep up with the camp program research base. Research reported in these studies is vital in planning for the inputs required to reach targeted outcomes of our own camp programs.

We know that camp is comprised of three interrelated attributes: the outdoors, education, and community (Slatter 1984). These attributes combine to create the camp experience. Research on positive youth development emphasizes the importance of a positive, inclusive atmosphere in youth programming (Roth & Brooks-Gunn 2003). When youth participate in programs, they need to feel included and comfortable in order to benefit from the program. At 4-H camp the adult leaders, the camp counselors, who are often older 4-H teens, and the friendliness of other campers combine to play an instrumental role in ensuring a fun and inclusive camp community where youth can flourish.

While environmental education may not be what most people think of when 4-H is mentioned, developing environmental stewards is a targeted content area for 4-H programs nationwide; a great deal of this target is met through resident camp programs. Campers learn about nature and the outdoors, often returning from camp with a wider awareness and appreciation of the natural world (Smith 2001).

In addition to providing a caring community and content knowledge, all 4-H programs are designed to enhance the development of important life skills in youth (Hendricks 1996). There are thirty-five identified life skills from which 4-H staff can select in designing 4-H programs; they are divided into the four quadrants of Head, Heart, Hands, and Health.

In Oregon, we were convinced that 4-H camps were delivering quality programs as the camp literature base suggests. But could we rely on research at other sites to assure that our youth were reaching the outcomes we designed and desired for them? We knew we were doing lots of things well, but were there things we could be doing better? To explore developing a statewide, multiple-site camp evaluation that would document outcomes at Oregon's 4-H camps, state specialist staff presented a seminar at the 2003 spring 4-H staff development conference.

Methods and Procedures

Because one of the most important aspects of conducting program evaluations at multiple sites is buy-in from the site directors (Arnold 2003; Garst & Bruce 2003), the questionnaire to be used was developed with significant input from local camp directors — who were primarily 4-H agents in Oregon. This process began at the staff development conference where we invited participating agent directors to articulate the impact of their camp program on youth.

This session was a critical step in preparing for a meaningful evaluation. We asked the question this way, "How will your campers be different on Saturday when their parents pick them up from when they were dropped off on Sunday?" While each of the thirteen Oregon 4-H camps offer different programs at a variety of sites, the camps shared some common desired outcomes for personal growth, life skill development, and camper satisfaction, which were identified by the agent directors.

Information from this session was drafted into an initial two-page questionnaire. The questionnaire was then reviewed by three 4-H agents with significant camp expertise, as well as by a local 4-H county camp advisory council. Following the reviews, the questionnaire was finalized for use in the study.

The evaluation was conducted using a multi-site evaluation methodology, with each of the thirteen participating camps administering the same questionnaire and following the same evaluation protocol (Arnold 2003; Garst & Bruce 2003). The questionnaire included basic demographic information, eight questions related to the camper's perceived life skill development (Hendricks 1996); six retrospective pre/post questions related to growth during camp (Pratt, McGuigan, & Katzev 2000); and a set of four questions related to the camper's satisfaction with camp.

Each site agreed to designate time at the end of camp during which campers were asked to complete the questionnaire about their camp experience. The completed questionnaires were sent to the state 4-H office for data entry and analysis. By the end of the summer, each camp received a report of the results for its own camp from the state office. The data were then aggregated for use in this statewide analysis — a strategy that allows a more robust understanding of the impact of similar programs that are held at different sites (Arnold 2003; Straw & Herrell 2002).

Eight hundred forty-nine youth campers, entering grades four through nine, completed the questionnaire (100 percent response rate). Five hundred fifty-nine of the campers were girls (66 percent) and 238 were boys (34 percent). Three hundred campers (35 percent) came from urban communities. Five hundred forty-nine (65 percent) came from rural communities. Sixty-one percent (518) of the campers were 4-H members. This was the first time attending 4-H camp for 55 percent (467) of the campers.

Results

Opportunities for Personal Growth
To measure personal growth, campers were asked how they felt about six items related to the camp experience. For example, campers were asked about being away from home, managing their free time, living in nature, and doing skits or presentations in front of others. Using a retrospective pretest and a 1 to 4 scale, campers rated each item based on how they felt before camp and then after camp. A rating of 1 indicated the statement was "not true" and a rating of 4 indicated the statement was "very true." Before-and-after ratings were analyzed using a paired t-test. The analysis revealed a significant, positive change for each item (p < .05). Using Cronbach's Alpha the internal reliabilities for the personal growth scale were calculated at a =.56 for "before" and a =.61 for "after." Table 1 presents the pre- and post-camp mean scores and the results of the paired t-test analysis.

Life Skill Development
Campers were asked how much 4-H camp helped them to develop life skills selected from the Targeting Life Skills Model (Hendricks 1996). Campers rated each of eight skills on a 1 to 4 scale, with a rating of 1 indicating that camp contributed nothing to the development of that skill and a 4 indicating that camp contributed "a lot!" (Cronbach's Alpha =.89). Mean ratings ranged from 3.10 (working through disagreements) to 3.47 (learning new things I like to do). Table 2 presents the score range and mean ratings for each of the life skills.

Camper Satisfaction with Camp
Finally, in an effort to determine the extent of a positive atmosphere at camp, campers were asked about their counselors, friends, and whether camp was fun. Campers were asked to rate each of 4 items on a 1 to 4 scale, with 1 indicating "not true" and 4 indicating "very true" (a = 73). Mean ratings for camp satisfaction ranged from 3.32 (camp was one of the most fun things I have done) to 3.80 (I liked my camp counselors). Table 3 presents the score range and mean ratings for each item.

Conclusion

The results of the Oregon 4-H Residential Camp evaluation adds to the growing body of research in camp-based youth development suggesting that summer camps play an important role in the development of young campers. The results show that camp provides an opportunity for youth to grow socially, to develop important life skills, and experience nature, all in a fun, hands-on setting. At the end of camp, campers reported that camp helped them to feel good about themselves, learn new things, to make friends, and work together in a spirit of cooperation. Perhaps most importantly, the natural setting of camp allowed campers to enjoy learning about the natural world. This information, gleaned specifically at Oregon 4-H Camps, will be valuable to agents in communicating the values of 4-H camp to parents, grant funders, and stakeholders.

Generic "one-size-fits-all" evaluation instruments may seem like the answer to a camp director's dream. However, the commonalities in generic instruments may become so broad as to lose any ability to inform a camp's own on-site youth development practice. An evaluation, based on the camp's targeted outcomes, should identify not only what the camp is doing "right," but where there is room for improvement.

The Youth Development Outcomes of the Camp Experience Study found "that religiously affiliated camps appear most likely of the sponsoring groups to see statistically significant positive change in their most important outcome, which in most cases, was spiritual growth (Henderson & Scanlin 2004)." For programs that have identified this as an important outcome, this is the expected result. Similarly, we should not be surprised that camps that do not target environmental attitudes as an outcome showed "no growth" in this area in the study.

Not only should a camp integrate its most important outcomes into everyday camp life, these outcomes should be reflected on the camp's evaluation. To document the delivery of quality youth development programs, camps must intentionally identify outcomes that can be measured with an evaluation. Do you know what your outcomes are?

Mary Olszewski Arnold holds a Ph.D. in Adolescent Development and is a program planning and evaluation specialist with the Oregon 4-H program. Her work centers on helping 4-H youth educators articulate and measure the impacts of 4-H educational programs.

Virginia D. Bourdeau is an extension specialist with the Oregon State University Department of 4-H Youth Development. She has authored and co-authored five 4-H leader publications. Bourdeau has worked in the camp profession for twenty-seven years.

Janet Nagele is a 4-H youth development agent with the Oregon State University Extension Service. She is responsible for the Environmental Eduction and Latino Outreach initiatives in Clackamas County and serves on the state 4-H Natural Science committee. Nagele has worked in the youth development profession for eighteen years, developing and administering camp programs, natural science education, club programs, and after-school, oudoor education. Her e-mail is janet.nagele@oregonstate.edu.

A version of this article including an analysis of gender differences has been accepted for publicaton by and will be published in an upcoming issue of the Journal of Extension (JOE) <www.joe.org>. JOE has granted permission for publication of the present article.

Originally published in the 2005 September/October issue of Camping Magazine.