by Joseph V. Tomchak
As Director of Program Services and Outreach at the Boys and Girls Club of Pawtucket, my responsibilities extend to assessing the effectiveness of the programs that we deliver for thousands of kids here at the Club. Prior to implementing the Rhode Island Program Quality Assessment Tool (RIPQA), when I thought of measuring youth programs, I typically thought first of outcomes: the breadth, duration, and intensity of children attending; pre- and post-tests; surveys, etc. I assumed that good outcomes meant a quality program.
I was in for a surprise.
While these tried-and-true measurements are benchmarks for good programs, they miss the quality that happens at the point of service and leave several questions unanswered. Is there a safe and supportive environment for participants? Are conflicts addressed in a healthy and productive way? Do we have enough leadership opportunities? Are child/staff interactions meaningful?
Fortunately, a grant requirement enabled us to begin working with the Providence After School Alliance to implement RIPQA, which is based on a nationally recognized and validated tool developed by the High/Scope Educational Research Foundation. It has a supplementary component that addresses the unique needs and priorities of Rhode Island afterschool professionals. RIPQA measures and evaluates the quality of the youth program experience. We use the insight from the evaluation to create enhanced programs during which youth are truly motivated to attend and engage.
Although I couldn’t argue against RIPQA’s value, I originally believed the evaluation would be so complex and time-consuming, that it would be a burden to an already overworked staff. This was simply not true. In fact, the RIPQA evaluation is neither complex nor time-consuming. After training on how to use the tool, the formalized assessment is completed with an impartial quality advisor over a two-week time period, which includes program observations and a consensus meeting. The tool is easy to use and the scoring process is self-explanatory. It illustrates specific goals to meet and gives concrete examples on how to create meaningful experiences for children and youth.
RIPQA has helped our professional instructors to look at Club programs from a different perspective. We have always had a proven and reliable curriculum from which we could work and an expectation for how we wanted the program to look; however, until we began to implement RIPQA, we lacked the framework to specifically state our expectations to instructors. Sometimes what instructors think they do is different than what happens, and differs from what RIPQA indicates. After thoughtful self-assessment, staff members can now examine their own program design and rethink ways to foster greater impact. The process requires staff members to show evidence for meeting the targets for their program areas.
The Boys and Girls Club of Pawtucket offers a broad range of programming and activities from education and career development, character and leadership development, health and life skills, to the arts, and sports, fitness, and recreation. One might think that such a wide variety of programs would make a consistent assessment difficult; however, when utilizing RIPQA, this is not the case. RIPQA can be implemented for each of these programs. It reinforces the fact that a good learning environment is a good learning environment—regardless of the learning content or the specific setting.
All programs continually evolve over time, regardless of how structured they may be.
The evolution can be subtle or overt, through changes in staff and client populations, in response to financial constraints, or as a result of new information and identifiable concerns. Some changes are planned and implemented intentionally, while others are beyond our control. RIPQA gives us the tools to ensure that Club services continue to improve over the long haul, regardless of anticipated or unanticipated changes.
Because of RIPQA, the Club views evaluation not as a one-time event—rather, as an ongoing process. Here, all full-time program staff members are directly involved in the training and implementation of RIPQA. Since the tool literally teaches as it is utilized, our staff readily learns what a high-quality program should look like. They are able to look critically at one another’s programs and make informed observations through RIPQA on how programs can be improved. Staff members see the value, observe the results, and embed the assessment process into their program management efforts.
Therefore, RIPQA’s value is unquestioned. It serves as a highly useful tool to measure the effectiveness of a variety of programs while, simultaneously, being easy to learn, implement, and interpret. The bottom line is that RIPQA leads to better, stronger programs for our children and youth.
*Thanks to RIASPA for allowing us to reproduce this here on our site.