Cathexis Partner, Melissa McGuire, contributed to the AEA 365 in 2012 by writing about using Group Priority Sort to select outcome indicators; one of “the top 10 most read non-staff articles from 2012”. We’ve included the full post here for you. You can also visit AEA’s 365 for the original post.
Melissa McGuire on Selecting Outcome Indicators for Evaluation – Leverage Divergent Points of View with Group Priority Sort
I’m Melissa McGuire, a Partner at Cathexis Consulting Inc., an evaluation practice in Toronto, Canada.
I was inspired by Ricardo Gomez’s recent post on Q methodology to share an adaptation of Q which we’ve been using in our practice that we call Group Priority Sort.
Lesson Learned: As many evaluators have experienced, it’s important to involve stakeholders in the selection of outcome indicators to get their buy-in and commitment. That said, we also know that consulting with stakeholders can stall the process, because different indicators are important to different groups. Worse still, sometimes organizations choose too many indicators in an attempt to appease everyone, which makes for a very cumbersome evaluation.
Rad Resource: Group Priority Sort is one method to rapidly get buy-in, build community, and find out which indicators are most important to measure, taking into account the perspectives of all stakeholder groups.
It is a hands-on exercise where stakeholders with different perspectives meet in person. The stakeholders are divided into small groups to sort the potential indicators (each printed on a cue card) along a continuum of relative “importance”.
Each group’s task is to determine the relative “importance” of all possible indicators through discussion and consensus building. The concrete outputs of Group Priority Sort are:
a) Consensus-based relative importance score for each indicator (for each group and averaged across groups).
b) Rich qualitative information about reasons given for relative importance of each indicator.
c) Engaged participants with a greater understanding of the perspectives of other stakeholders, greater understanding of the evaluation and greater ownership of the process.
The information obtained through Group Priority Sort can then be used to inform decisions about which indicators to ultimately include in your evaluation.
Lesson Learned: Carrying out a Group Priority Sort can be resource intensive and requires careful planning and execution. However, done right, the method makes it possible to quickly narrow down priorities in a sound and meaningful way.
If you find opportunity to use Group Priority Sort, we’d love to hear about your experience.