Frequently Asked Questions about the ACA Youth Outcomes Battery

Frequently asked questions about the ACA Youth Outcomes Battery

 

 

Why Were These Tools Constructed?

Many camp professionals are faced with the challenge of proving that they meet their program goals. The following tools — designed with camp professionals in mind — are:

  • Camp-specific
  • Easy to administer
  • Age-appropriate
  • Statistically tested (i.e., reliable and valid)
  • Individualized (able to select as you want to reflect intentional focus of your program)
  • Easily modified for use in other youth programs such as afterschool programs

Top of page

What about Reliability and Validity of the Measures?

Some individuals may be interested in the statistical details behind the questionnaires included in the YOB. The following information and references may be helpful to individuals who must document the strength and appropriateness of these outcome questionnaires. The following points highlight the construction and testing of the questionnaires:

  • The YOB was initially developed in 2006 and has been in continued use in various formats since this initial version was published by the American Camp Association. It has also been translated into several different languages including Russian, Spanish, and Italian.
  • The 2013 version of the YOB has 11 subscales: Friendship Skills (FS), Family Citizenship Behavior (FCB), Responsibility (RESP), Independence (IND), Teamwork Skills (TW), Perceived Competence (COMP), Affinity for Exploration (AE) Problem Solving Confidence (PS), Affinity for Nature (AN), Camp Connectedness (CC), and Spiritual Wellbeing (SWB). 
  • Subscales are generally available in two formats. The basic format assesses perceived changes over the duration of the program. The detailed format includes both a current status and a retrospective change scale and can be used as a standard pretest/posttest format or status + retrospective change format.
  • Psychometric evaluation included examination of the internal structure (reliability, item-to-total correlations, inter-item correlations) of the individual instruments as well as cross-structure analysis (intercorrelations among scale scores and correlations between scale scores and age). Each of these subscales has demonstrated excellent internal consistency reliability over repeated use (see references below).
  • In 2010 and 2011 a comprehensive examination of the 11 subscale YOB was conducted through a systematic examination of data from 3750 youth representing 37 summer camps. This study generally supported the construct, discriminant, and convergent validity of the YOB (Sibthorp et al., 2013).
  • The YOB is now one of the few assessment options with normative data and a track record of sustained use, adaptation, and translation. Recently the YOB was acknowledged as a promising tool for youth program assessment by the Forum for Youth Investment in From Soft Skills to Hard Data: Measuring Youth Program Outcomes (Wilson-Ahlstrom, Yohalem, Dubois, & Ji, 2011). The YOB is also now included in a number of tool data-banks such as Toolfind (http://www.toolfind.org/) from the United Way. 
  • Current work on counselor and parent version of the YOB are underway for use with youth populations that are unable to complete self-report measures. 

Publications related to the reliability and validity of the YOB and some of its specific subscales include:

Eastep, B., Cachelin, A., & Sibthorp, J. (2011). Affinity for nature in outdoor programming: Theoretical foundations, scale development, and importance. Journal of Outdoor Recreation, Education, and Leadership. 3(3), 126-136.

Ellis, G., Sibthorp, J., & Bialeschki, M. D. (2007). Development and validation of a camper outcomes battery (abstract). NRPA Leisure Research Symposium. (pp. 106-109). Indianapolis, IN.

Sibthorp, J., Bialeschki, D., Morgan, C., & Browne, L. (2013). Validating, norming, & utility of a youth outcomes battery for recreation programs and camps. Journal of Leisure Research. 45(4). 514-536.

Sibthorp, J. Browne, L., & Bialeschki, M.D., (2010). Problem solving and camp connectedness: Two new measures for the ACA Youth Outcomes Battery. Research in Outdoor Education. 10, 1-12.

Wilson-Ahlstrom, A., Yoholem, N., DuBois, D., & Ji, P. (2011). From soft skills to hard data: measuring youth program outcomes. Forum for Youth Investment, Funded by William T. Grant.
 

Top of page

How Can These Outcomes Tools Help Me?

The tools:

  • Help you evaluate your program goals
  • Document the changes in your campers so that information can be shared with key stakeholders (parents, funders, staff, etc.)
  • Demonstrate your commitment to quality programs that make a difference in people’s lives
  • Meet expectations for trust worthy instruments (high reliability and validity statistical checks prove the questionnaires accuracy)
  • Can be combined to measure socially relevant concepts (e.g., Environmental Leadership could be measured by combining the scales for teamwork, responsibility, independence, problem-solving, and affinity for nature)

Top of page

What Outcomes Can I Measure?

These tools focus on ten outcomes common to many camp programs. They include questions that ask if the camp experience has helped campers develop:

  • Friendship Skills (i.e., make friends and maintain relationships)
  • Independence (i.e., rely less on adults and other people for solving problems and for their day-to-day activities)
  • Teamwork (i.e., become more effective when working in groups of their peers)
  • Family Citizenship (i.e., encourage attributes important to being a member of a family)
  • Perceived Competence (i.e., believe that they can be successful in the things they do)
  • Interest in Exploration (i.e., be more curious, inquisitive, eager to learn new things)
  • Responsibility (i.e., learn to be accountable for their own actions and mistakes)
  • Affinity for Nature (i.e., develop feelings of emotional attraction toward nature)
  • Problem-Solving Confidence (i.e., believe they have abilities to resolve problems)
  • Camp Connectedness (i.e., feeling welcomed and supported at camp)
  • Spiritual Well-Being (i.e., having purpose and meaning in life, transcendence)

Top of page

What Are My Choices for Types of Questionnaires?

All of the questionnaires are age-appropriate measures that allow camp directors to choose the most relevant outcomes. One version was developed for 6-to-9-year-olds called Camper Learning. Two versions were constructed for 10-to-17-year-olds: Basic Camp Outcomes Questionnaires and Detailed Camp Outcomes Questionnaires.

Camper Learning Survey

  • Best for campers 6 to 9 years old
  • Staff help the campers do the survey
  • Only 14 total questions that combine the original 7 outcomes appropriately for young children into one survey
  • Uses an easy 4 point Likert scale

Basic Camp Outcomes Questionnaires (increase format)

  • For campers who are 10 to 17 years old (works especially well for 10-to-13-year-olds)
  • Questionnaires measure gains through the camp experience
  • Separate questionnaires that can be mixed and matched based upon intentional program efforts
  • Uses an easy 5 point Likert scale
  • Quick and easy to give (each of the questionnaires generally has 6 to 14 questions)

Detailed Camp Outcomes Questionnaires (status + change format)

  • For campers who are 10 to 17, but works especially well for 13-to-17-year-olds
  • Questionnaires measure gains through the camp experience, plus how much of that gain was due specifically to camp
  • Separate questionnaires that can be mixed and matched based upon intentional program efforts
  • Uses an easy 6 point Likert scale
  • Takes longer to give because each question has two parts
  • Each of the individual outcome questionnaires (same questions as the basic version) have between 6 to 14 questions

Top of page

How Do I Analyze the Data?

Microsoft Excel templates have been designed for use with these questionnaires. The following steps detail their use:

  • Download the template for whatever version chosen onto your computer (requires Excel).
  • Read the instruction sheet (notice the tabs on the bottom of the Excel page).
  • Go to the tab for the outcome(s) selected.
  • Enter the data.
  • Generate a report with the tables and charts.
  • You can always do your own analysis through Excel or other statistical packages.

Top of page

Can I Change the Questionnaires?

With the exception of the demographics questions and the specific word "camp," none of the individual questions can be altered. The statistical tests showed that these questions are the best predictors for that particular outcome. However, you are encouraged to add your own demographic questions (i.e., session, program, race/ethnicity, etc) to the basic list provided. Since the questions are not setting specific, the word "camp" can be replaced with other setting designators such as "afterschool program", etc.

Top of page

Steps to Follow to Use the Camper Outcomes Questionnaires

  1. Read the background information about the tools.
  2. Decide which version you want to use, then purchase the downloadable PDF(s).

Once PDF is downloaded:

  1. Click and print the instructions for staff.
  2. Click and print the instruction for campers.
  3. Click and print the outcome(s) questionnaire(s).
  4. Click and print the demographics questions (you list the demographics that you want to ask).
  5. If you want to combine several questionnaires into a larger survey or possibly combine with other evaluation measures, just cut and paste to make a survey that includes demographic questions or other questions specific to your camp. [Note: Please remember to acknowledge ACA by including the copyright information.]
  6. Make as many copies of the survey as you need.
  7. Give the survey to your campers (usually the second to the last day is good timing).
  8. Enter the data on the template tool provided at this website.
  9. Generate your report.
  10. Discuss your findings and how they help with program development or staff training
  11. Share your information with your key stakeholders (staff, Board, funders, etc.).

Top of page



 

 

Tags: