Graduate Certificate of Applied Causal Evaluation of Policy and Innovation
A student review about the content offered in this program:
"[It] offers a survey into a unique and emerging form of research that most don't get the opportunity to learn and practice while in school but instead have to stumble their way through in practice."
- EII 604 (1 credit) Introduction to Applied Causal Evaluation of Policy and Innovation
Introduction to the certificate program topics including the meaning of causal evaluation, how it can inform decision-making and continuous improvement processes, and a review of relevant math and statistical tools.
- Describe how the historical context of causal policy research shapes the current educational conversations and climate.
- Explain the differences between research and evaluation with respect to generalizability and decision-making.
- Consider the applicability of causal evaluation in the setting(s) you typically encounter in your professional life.
- Demonstrate the capacity to correctly interpret the basic math and statistics tools used throughout the certificate program.
- EII 606 (2 credits) Methods in Causal Evaluation
Survey of research methods facilitating causal inference. Includes basic introduction to randomized control trials and quasi-experimental methods that can be used when a randomized control trial is not feasible.
- Explain the benefits and limitations of Randomized Control Trials (RCTs).
- Explain in non-technical terms how four quasi-experimental techniques approximate an RCT:
- Regression Discontinuity
- Instrumental Variables Estimation
- Propensity Score Matching
- EII 607 (1 credit) Ethical Data Management and Analysis I
Use theory to drive analytic decisions and demonstrate how to estimate, interpret, and communicate impact of Randomized Control Trial. Emphasis is on using Ordinary Least Squares regression.
- Demonstrate how theory drives modeling decisions through applied problems using Ordinary Least Squares regression.
- Demonstrate how to estimate and interpret the impact of a simple Randomized Control Trial (RCT).
- Communicate empirical results in an accessible way to the audience you typically encounter in your professional life.
- EII 608 (2 credits) Ethical Data Management and Analysis II
Learn to program and interpret results of applied problems using the quasi-experimental techniques learned in EII 606, including propensity score matching, difference-in-differences, and regression discontinuity.
- Demonstrate how to correctly estimate the impact of more complex RCTs, including those conducted at the cluster level, and make justifiable decisions about when to include covariates. Communicate the results in an accessible way to the audience you typically encounter in your professional life.
- Demonstrate how to correctly estimate the impact estimates of Difference-in-Differences causal models. Communicate the results in an accessible way to the audience you typically encounter in your professional life.
- Demonstrate how to estimate the impact estimates of Regression Discontinuity causal models. Communicate the results in an accessible way to the audience you typically encounter in your professional life.
- EII 609 (1 credit) Defining Interventions and Using Logic Models
Students will develop a logic map—a graphical representation that illustrates how the resources and strategies of an intervention are expected to translate into the desired outcomes.
- Compare well-defined interventions with those that are less well-defined. Convert an ill-defined intervention to a well-defined intervention that might be suitable for evaluation.
- Identify and justify appropriate short- and long-term outcome measures.
- Identify the best format of logic model for conveying the theory of change of a particular intervention.
- EII 610 (2 credits) Randomized Control Trials in Schools
Determine strategies for conducting Randomized Control Trials within schools considering timing, context, generalizability, strengths, and barriers. If a local RTC is not plausible, make use of existing research and evaluations to inform decision-making.
- Deduce appropriate lessons from existing applied research articles and evaluation reports to inform practice in the local context.
- Explain why an intervention in a particular setting is or is not appropriate for an RCT evaluation.
- Compare the pros and cons of conducting randomization at different levels for a particular intervention.
- Describe political and practical barriers to conducting RCTs in schools and strategies for overcoming those barriers. Communicate those strategies in a politically sensitive and collaborative way suitable for school boards, principals, teachers, and parents.
- Identify personally identifiable information (PII) and explain how it must be managed by schools, researchers, and other parties under the Family Educational Rights and Privacy Act (FERPA).
- EII 611 (1 credit) Process Evaluation
Students will apply a practical process to a theory-based framework to plan, implement, and use evaluation to critically examine the implementation of local educational initiatives.
- Identify key components for fidelity of implementation of an intervention.
- Develop a strategy for monitoring the extent to which each key component is in place at a given site using a variety of methods such as inventories, surveys, focus groups, direct observation, administrative data.
- Align specific fidelity aspects with the appropriate measurement tool and use that tool effectively.
- Determine the most efficient and appropriate sampling methods given available resources and goals.
- Using process-focused evaluation approaches in light of context as revealed through fidelity data combined with administrative data.
- EII 612 (1 credit) Data Visualization for Educators
Introduction to design principles and software for developing static and interactive data visualizations. Emphasis is on matching communication and data visualization strategies to target audiences.
- Match data visualization techniques to audiences and communication goals.
- Test-drive a variety of software packages for developing static and interactive data visualizations.
- Develop a checklist for reviewing and revising data visualizations.
- Apply data visualization techniques to a topic and audience you typically encounter in your professional life.
- EII 613 (1 credit) Research-Practice Partnerships
Framework for developing partnerships among education agencies and research institutions to inform and evaluate policy and innovative practices. Emphasis is on building long-term, sustainable partnerships.
- Identify situations when a partnership might be appropriate.
- Identify the characteristics of a reciprocal, mutually beneficial partnership.
- Delineate the critical components necessary to sustain a successful partnership.
- Propose strategies for overcoming common obstacles to well-functioning partnerships.
- EII 614 (1 credit) Introduction to Cost Studies
Introduction to identifying and understanding the costs of educational interventions. Overview of how to select the appropriate type of cost study: cost-analysis, cost-effectiveness, or cost-benefit.
- Differentiating between burdens from not having a treatment (e.g., school failure) and costs of an intervention or policy.
- Articulate why budgets are not enough to create a foundation for a cost-analysis and common reasons for cost inaccuracy.
- Take an ingredients method approach to identifying the resources needed to replicate an effect.
- Differentiate among and determine when to use cost-analysis, cost-benefit, or cost-effectiveness.
- EII 615 (2 credits) Application of Cost Analyses to Decision-Making
A case study approach to identifying costs, calculating cost ratios, and translating the findings into recommendations for decision-making.
- Identify cost of ingredients necessary to replicating an impact including pricing, discounting, and adjusting for inflation.
- Calculate cost-benefit and cost-effectiveness ratios.
- Compare results of cost-effectiveness and cost-benefit analyses.
- Translate findings into policy and practice recommendations.