This post was written by Chris Mai, Policy and Communications Manager for Every Hour Counts and originally appeared on their blog. Every Hour Counts, formerly the Collaborative for Building After-School Systems, is a coalition of citywide organizations that increase access to quality learning opportunities, particularly for underserved students.
Data can be a powerful tool, but only if you know how to use it.
For too long, the expanded learning field has struggled with the complex and elusive process of developing and adopting a common framework for measuring youth outcomes and the program and system practices that may influence them. In a webinar last week, we unveiled a new Measurement Framework, a tool to help communities set goals for their expanded learning systems and to help them assess their progress and make data-driven improvements.
How can the Measurement Framework help you? The Measurement Framework offers practitioners:
• A clear, simple set of outcomes that reflect priority measures of success for expanded learning systems. The Framework presents eight elements across the youth, program, and system levels that reflect high-priority focus areas for a thriving expanded learning system. Each element has corresponding outcomes designed to show whether systems and programs are functioning well. The Framework provides a description of measurement activities that can accompany each outcome, suggestions for how the data can be used, direction regarding how data on a given outcome may be linked to other levels within the Framework, and evidence on the value of each outcome.
• Tips for how to use data to drive improvement. Do you collect data but need guidance in how you use it to change practice? The Framework offers “data use goals” for each outcome to help improve the efficiency of your quality improvement and assessment efforts.
• A handful of social, and emotional “power skills” that are likely to drive student success. Education leaders, practitioners, and researchers are increasingly recognizing the importance of social and emotional skills in driving students’ long-term success. The Framework offers practitioners a variety of tools to evaluate the degree to which students are developing positive youth development skills like persistence and collaboration.
In our recent webinar, we gleaned new insights on the tool from two city leaders: Hillary Salmons, Executive Director of the Providence After School Alliance (PASA) and Barbara Couto Sipe, Executive Director of Middle School Renaissance 2020 (MSR2020) in Richmond, Virginia.
• Set realistic expectations. Be honest with yourself and with your partners about what outcomes they can reasonably expect in the short term. Improvements to the social and emotional skills of participating youth may not be visible in the data right away. Hillary Salmons advised system-builders to “use the tool to gain ground in conversations with leaders but at the same time be realistic about the time required to achieve some of these outcomes… persistence isn’t something you achieve and measure in a 6 week summer program, let’s get real!”
• The Framework doesn’t have to be adopted all at once. The Measurement Framework is an aspirational document. Every Hour Counts hopes communities ultimately adopt it in its entirety, but if you’re new to system-building or still developing data and quality systems, you don’t have to bite it off all at once. You might consider focusing on system and program level outcomes prior to assessing the impact on youth outcomes. Barbara Couto Sipe concurred, “We do want to get to the youth outcomes, and we will, but we need to pace ourselves, and really start with youth program quality, assessment, and supports for our providers.” Hillary Salmons added, “Keep it simple, pick a few variables that are pivotal, and build toward the more complex.”
• Communities across the country are finding it useful. Communities are finding the Framework valuable as they work to develop high-quality systems. Hillary Salmons said that PASA “bet on the fact that if we focus on good practice, we’re likely to achieve significant youth outcomes, and it’s now great to have a framework that validates this roadmap.” And Barbara Couto Sipe pointed out that the Measurement Framework has been helpful to MSR2020 efforts to define standards of quality and a measurement system.
The Measurement Framework is accompanied by a comprehensive Literature Review, which highlights the research we used in developing the Framework and defines key elements, processes, and outcomes of expanded learning systems.
This Measurement Framework is a work in progress. We’ll be sharing new learnings as we pilot test the tool in a few cities. Please let us know what you think of the Every Hour Counts Measurement Framework and how it can be helpful to your system-building efforts.