Research on Evaluation New Directions for Evaluation, Number 148 J–B PE Single Issue (Program) Evaluation Series

This issue broaden the field’s understanding of Research on Evaluation (RoE) in practice, with a focus on empirical RoE. Considering the full collection of RoE studies, it defines RoE as systematic empirical inquiry resulting in original findings or in reexaminations of existing data about the practice, methods, or profession of program evaluation. RoE inquiries address:

• the inputs into evaluations,
• the contexts within which evaluations are conducted,
• the full range of methods and activities of evaluations,
• evaluation’s proximal or distal effects, or the professional issues that evaluators encounter.

The inquiries include case studies, reflective narratives, studies about evaluation methods, literature reviews, oral histories, bibliometric studies, meta-evaluations, experiments, longitudinal studies, simulations, and time-series studies.

This is the 148th issue in the New Directions for Evaluation series from Jossey-Bass. It is an official publication of the American Evaluation Association.

EDITOR-IN-CHIEF’S STATEMENT 1

EDITORS’ NOTES 3
Paul R. Brandon

1. Research-on-Evaluation Articles Published in the American Journal of Evaluation, 1998–2014 7
Lisa M. Vallin, Joanna Philippoff, Sena Pierce, Paul R. Brandon

The authors present a summary of the research-on-evaluation articles that were published during a 14-year period in the American Journal of Evaluation (the American Evaluation Association’s flagship journal), categorized by several descriptive characteristics.

2. Methodological Trends in Research on Evaluation 17
Matt Galport, Nicole Galport

The authors examine themes in the empirical literature on evaluation methodologies published in the American Journal of Evaluation since 1998 and consider directions for future research on evaluation methods.

3. Research on Evaluation Ethics: Reflections and an Agenda 31
Michael Morris

The author presents an overview of key findings from research on evaluation ethics during the past quarter century and offers suggestions for making ethics research more useful for evaluation practice.

4. Advancing Research on Evaluation Through the Study of Context 43
Anne T. Vo, Christina A. Christie

The authors describe a framework that seeks to organize ideas for conducting research on evaluation context and use it as a basis to generate ideas for possible studies of context.

5. Using Action Design Research to Research and Develop Evaluation Practice 57
Nick L. Smith

In this chapter, the author proposes the use of Action Design Research as a case-based, practical reasoning oriented methodology for self-corrective, real-time improvement of evaluation practice.

6. Assessing the Practice Impact of Research on Evaluation 73
J. Bradley Cousins, Kateryna Svensson, Barbara Szijarto, Celine Pinsent, Caroline Andrew, John Sylvestre

The authors propose an approach that might be used to evaluate the impact of research on evaluation in practice.

7. Evaluators’ Perspectives on Research on Evaluation 89
Nicole R. Lewis, George M. Harrison, Anna F. Ah Sam, Paul R. Brandon

In this chapter, the authors report the methods and results of a questionnaire survey that addressed evaluators’ attitudes toward, beliefs about, and knowledge of research on evaluation (RoE); time and opportunities to participate in RoE; application of RoE findings; and involvement in RoE.

8. Reflections on the Future of Research on Evaluation 103
Tarek Azzam, Miriam R. Jacobson

In this chapter, the authors reflect on the potential future of research on evaluation and offer suggestions for the increased implementation and use of research on evaluation theory and practice.

INDEX 117

Paul R. Brandon is a professor and director of the Curriculum Research & Development Group, College of Education, University of Hawaii at Manoa.