Taking the Bogeyman Out of Evaluation Planning

Careful, systematic planning is essential for producing useful and informative evaluations.  A well-conceived evaluation process is built on clear, thorough communication between the program administrators and the evaluator about components, constraints, and goals of both the program and the evaluation. All parties must be certain they understand each other’s needs completely before collecting the first piece of data. Such a full exchange of information makes it easier to plan an evaluation that covers all parts of a program and controls for any biases that might color the interpretation of outcomes.

This site provides a range of resources to help you build solid evaluation plans and judge the worthiness of existing evaluations.

  • A white paper describes different kinds of evaluations, discusses the importance of using rigorous research techniques, and synthesizes advice and information about design and implementation.
  • The Resources for Practitioners page provides links to guides and other publications offering how-to advice for planning and executing an evaluation.
  • A Glossary defines common terms in evaluation planning, design, and implementation.

Because science and medicine have used rigorous evaluation methods for years, this site includes several examples from those fields. While those examples have structural elements that apply to education programs (e.g. using rigorous research methods, ensuring that statistical models include sufficient controls, etc.), you may want to skip details that do not relate to your needs.

Before you start to work, hereís some advice from experts to help you avoid serious mistakes:

  • Start planning the evaluation as early as possible and document details, including budgets for money and time, in the full project plan. Involve the person doing the external evaluation in planning.
  • Formulate the evaluation questions carefully, based on the projectís goals, delivery methods, and stakeholders. And donít skip the logic-model stage. Both steps provide a foundation for designing the evaluation.
  • Identify data sources early and contact the owners to make sure needed data will be available. If the evaluation will involve participation by program stakeholders, involve them early to make sure they understand and agree to their expected role.
  • Devise plans for assessing whether the program is being enacted as designed so that adjustments can be made to improve fidelity or change elements that are difficult to implement, if necessary.
  • View evaluation as an integral, multi-stage part of the program that continues through the programís lifetime. That approach to evaluation will require you to plan timetables and procedures for ongoing data collection and formative processes for using the growing store of data Ė and the information it reveals Ė to fine tune the program and institute major changes if necessary.
  • Use rigorous methods. Rigorous evaluation is a critical component of a continuous improvement process. A well-designed evaluation can save money if it reveals that a crucial part of a program is not working and you correct the problem or discontinue the component.