Happy New Year, Camp Evaluators! The year 2017 brings a lot of excitement for ACA’s Research 360 efforts, for both local camp evaluation and the 5-Year Impact Study. For local camps (the inside of the Research 360 circle), we will expand our evaluation training resources and initiate a Youth Outcomes Battery User’s Group. The 5-Year Impact Study (the outside of the circle) launches in Spring 2017, so we will be posting updates and preliminary results as they emerge.
With this much research and evaluation coming up, we are bound to wander into some geeky territory. You know — that place where you’ve been reading something technical and realize, after several pages, that you haven’t absorbed a single word. If you are reading research, chances are you are lost in complex concepts, too long sentences, and unfamiliar words. We call this geek speak.
So, to prepare you for the year ahead, we’re offering a crash course in geek speak: basic research terms that will help you engage with Research 360 every step of the way.
We are all biased, which means that we all have deep and often unconscious beliefs that guide our decisions. Bias is not a good thing in research, but it is unavoidable whenever humans are studying other humans. In the 5-Year Impact Study, the researchers will work to reduce bias and think about sources of bias that might impact their findings.
The 5-Year Impact Study begins by interviewing people who have attended camp as a child about their experiences and the impact of camp on their life. This is called exploratory research because the researchers do not have a hypothesis — a prediction — about what they will find. Exploratory research generally involves qualitative data (e.g., interviews) because this allows important themes to emerge rather than be imposed by the researcher.
Some research is meant to describe a specific group of people (or camps); other research tries to describe a larger population. When research is generalizable, its findings apply to a population beyond the specific participants (or camps) in the study. Findings from the 5-Year Impact Study will be generalizable to the larger ACA camp community because of the way the study is designed.
Most research looks at a group of people at a single point in time- we call this cross-sectional research. Longitudinal research looks at a group of people over time by collecting data from them several times over the course of days, weeks, or years. This type of research is less common because it is tricky to keep track of people over time, but it is a powerful tool, and one that will help the camp industry describe the long-term value of the camp experience.
You are likely familiar with pre-test post-test research, and if you’ve given surveys at the start of a program and again at the end, you are also familiar with the challenges with this approach. Retrospective surveys wrap the pre- and the post-test into a single survey by asking people to think about how they feel right now, and then to think about how they might have felt before the program started. It’s an approach we use in the YOB Detailed Version, and one that will be used in Phase 2 of the 5-Year Impact Study.
The sample is the group of people (or camps) selected to represent the larger population in the study. Choosing a sample is a critical step in research, and one that can be biased or limited. The 5-Year Impact Study will use a representative sample by selecting camps from each of the major camp types that make up the larger ACA camp community. This includes day camps and overnight camps, independent for-profit and nonprofit camps, camps affiliated with agencies, government, and religious organizations, and camps that serve youth with illnesses, disability, or other special needs. Camps within each of these categories will be selected randomly, to reduce bias, and then asked to participate in the study.
Congratulations — you are now officially a geek! Armed with just a few new research terms, we hope you will read and think critically about the 5-Year Impact Study over the course of the next year. In 2017, the Research 360 blog will continue to be the place where you can keep up on it all- from project updates to tips and tricks to expand your evaluation capacity. Also, be on the lookout for guest bloggers that will be adding their insight to Research 360. Until then, plan to make 2017 one enriched by powerful evidence of the good work you do.
Thanks to our research partner, Redwoods.
Additional thanks goes to our research supporter, Chaco.