Developing Criteria to Identify Transformative Participatory Evaluators - Michael A. Harnar (The Evaluation Center at Western Michigan University, 2014)
Background: The evaluation discipline continues to evolve as more and more researchers study practice. The research described in this article further defines Transformative Participatory Evaluation (T-PE) by focusing on the key elements that practitioners and theorists agree define this evaluation practice. Purpose: A multi-stage, mixed-method approach was used to develop and examine a set of statements that serve two purposes: First, they can help identify a subset of participatory practitioners from others and, second, they further theory development by showing how T-PE practitioners differ from other evaluation practitioners on key indicators. Setting: In the first phase of this research, three prominent evaluation theorists comprised an expert panel to develop a set of statements that would identify T-PE practitioners.  The American Evaluation Association membership was used to test the statements in the research's subsequent phases. Intervention: NA   Research Design: A multi-stage, mixed-method approach was used to develop and test the statements. Data Collection and Analysis: The panel was engaged in a web-based wiki to jointly edit the statements; an online questionnaire with mostly closed-ended items was used to gather AEA member input; a unique online modeling software and webinars were used to further understand findings.  Analysis of variance was used to assess differences between groups and Rasch modeling and Wald tests were used to analyze the modeling data. Findings: The eight core statements that emerged had acceptable internal reliability and limited construct validity. Though the statements' discrimination strength was tenuous, quantitative comparisons of preferred evaluation practice models showed congruence with the predicted underlying philosophies and therefore supports the statements' ability to discern T-PE evaluators from P-PE evaluators.   Keywords: Transformative Participatory Evaluation; Participatory Evaluation; evaluation