Greetings Camp Evaluators! Today’s post comes from ACA National Board Member Lisa Westrich, who is the program development director at Challenge Success. In this role, Lisa works closely with schools and camps to improve programming, operations, and other initiatives through research and evaluation. We asked Lisa to write about logic models because this is one tool she has used with great success when working with camps. Here’s Lisa:
As 2017 gets under way, you may be thinking it’s a great time to begin planning your camp’s evaluation efforts. We know the evaluation process can feel a bit overwhelming and figuring out how to get started can be downright daunting. You may have heard that developing a logic model is a good way to ensure that your evaluation yields relevant and actionable information. But, you may be wondering, “What is a logic model, why do we need one, and how does our camp go about creating one??” In this Research 360 blog, we hope to provide camps with a better understanding of logic models and how they can be a valuable launching point for high quality evaluation and program improvement.
What is a Logic Model?
“A logic model is a systematic and visual way to present and share your understanding of the relationships among the resources you have to operate your program, the activities you plan, and the results you hope to achieve.” -- W.K Kellogg Logic Model Development Guide, 2004
Logic models illustrate how a program is supposed to work: what it does, what it’s trying to achieve, and what needs to be measured to demonstrate its effectiveness. They represent the assumptions an organization has about how its programs and activities lead to its intended goals and outcomes. In a nutshell, logic models create a clear picture of the steps necessary to achieve a program’s intended long-term impact. It bridges the gap between where you are now and where you intend to be.
Logic models provide an essential foundation for evaluation and can point to what data will be most useful to collect. They help an organization identify appropriate process measures to determine if it’s reaching the participants it says it will reach and doing what it says it’s doing (camp program quality). Logic models also help identify outcome measures that determine if the programs and activities are effectively leading to its intended outcomes and goals (camper impact).
In addition to laying the groundwork for evaluation, logic models can also:
- Help your staff stay focused on what you’re trying to achieve
- Help leadership stay focused on designing and maintaining programs and activities that are designed to lead to your intended outcomes
- Identify what parts of your program are working and what areas may be in need of rethinking or improvement
How do we develop our own Logic Model?
There are many variations on how a logic model can look visually, but the basic linear framework below is a great place to start.
Inputs: The resources needed to operate your program. This might include year-round and seasonal staff, materials, equipment, facilities or physical site, staff training, and value-added affiliations or partnerships.
Outputs: The program activities and services you deliver (types and quantity) as well as the participants in your program who receive them.
Outcomes: The changes or benefits participants receive as a result of the program. This may include changes in knowledge, beliefs, skills, attitudes, or behaviors, to name a few. Outcomes fall along a continuum from several short and intermediate-term outcomes to one long-term goal, or impact statement.
Building a logic model that reflects your program will take time, energy, and a lot of thoughtful discussion. Start by identifying your program’s one overarching, long-term goal. Be clear about what you’re trying to achieve in the long run. Next, conduct “backwards mapping” to identify the preconditions necessary to achieve that big goal. All of your short and intermediate-term outcomes should be clearly connected and logically lead to your long-term goal. Next, identify your main program activities and services and consider specifically how they lead to your outcomes. Finally, list your current resources. This will help you determine the extent to which you will be able to fully implement the program and achieve your intended goals and outcomes.
Effective logic models are:
- Meaningful — It represents action that is valued and worthy of doing.
- Plausible — Evidence and common sense suggest that the activities, if well implemented, will lead to the desired outcomes.
- Doable — The economic, technical, institutional, and human resources are available to carry out the program.
- Testable — It is specific and complete enough for an evaluator to track its progress in a way that is credible and useful.
- Changeable — It is a living document. Revisit and revise your logic model regularly!
Lisa Westrich, LCSW is the program development director at Challenge Success, a national school reform organization affiliated with Stanford University’s Graduate School of Education. She supports the organization’s core programs and research efforts. Previously, Lisa was a research and policy analyst at Stanford’s John W. Gardner Center where she partnered with educational institutions and community-based organizations to improve outcomes for youth. She has worked with children and families in a variety of education and nonprofit settings, with an emphasis on positive youth development, for twenty-five years.