Program Staff Reflections on Evaluation

Program Staff Reflections on Evaluation in Youth Media

We asked the staff of Time Warner-funded youth media programs to share some of what they've learned through their experiences with program evaluation. Here are some of their thoughts:

  • Evaluation is a common concern among practitioners. Even those who have developed relatively complex systems for understanding the impact of their work cite areas that they are not satisfied with.
  • Evaluation can help programs define common values, goals, and languages. Use the process as an opportunity to engage a range of stakeholders in reflection on what is most important in your programs.
  • Evaluation practices vary widely in both format and content. Current practice among the grantees runs the gamut from informal feedback and simple staff review of information to highly structured data collection and analysis. There are innovative practices out there, most frequently in more established programs that have had more time to reflect on the results of their efforts and to think in a more intentional way about how to measure outcomes.
  • Evaluation needs to be embedded in everyday programming. Any evaluation practice is an evolving process in program implementation and sustainability. Start by defining what you already know and what capacity you have with regard to evaluation
  • Staff reflection is a key component of successful evaluation practices. Finding time for staff reflection is challenging for many practitioners. Dedicating time before, during, and after programs for dialogue and analysis is critical.

    •  "Through our evaluation practices, we have identified a need to provide more training opportunities for our younger staff."
    - Andy Paez, Youth Radio     

  • Youth can be more involved in the evaluation process. Many practitioners would like to find ways for program participants to be more centrally involved in the process of evaluation. This too takes planning and resources.
  • Community involvement can also be helpful in improving evaluation practices. Look for resources within your organization and community to strengthen evaluation practices.
  • Outside support may be necessary to put evaluation practices in place. Aside from funding earmarked specifically for program evaluation, practitioners would welcome greater support for evaluation from funders and value the support of outside parties.
  • Alumni are considered important for further study. The most frequently articulated wish amongst the Time Warner grantees was for follow-up with and further evaluation of program impact on alumni. Tracking youth for the long term is particularly challenging and requires ongoing outreach efforts.
  • Consider media products as useful tools for reflection on impact as well as recruitment and retention. Support the creation of multimedia products that demonstrate learning, and provide time for review and reflection. Have youth capture testimonials about what they've learned in your programs and share these with new participants.
  • Online survey tools can be useful. Sites like SurveyMonkey.com are useful resources in data collection and program feedback and can be accessible and engaging for stakeholders.