How to evaluate your collaboration | Article
Learning and evaluation are essential throughout all stages of collaboration to help determine whether the collaboration is on the right track to achieve the set goals, and to provide valuable insights for planning the next steps. However, evaluation of collaborations has long been recognised as a challenging area due to the complexity of their operation and the often-ambitious objectives.
Based on WIG’s Collaboration Playbook, these tips can help ensure you are evaluating your collaboration effectively with a view towards improvement.
Balance evaluating process and outcomes
Two types of evaluation are recommended for collaborations: formative and summative. A balance is needed between the two types of performance assessment.
- Formative refers to the functioning of the collaboration. Success can be partly measured through the progress in developing the functioning of the collaboration itself – what has been called the ‘process of collaboration’. The formative performance criteria can be internal to an organisation, referring to how it manages its engagement in the collaboration, or at the level of interorganisational integration, referring to how the partners function with each other in the collaboration on aspects such as shared decision-making.
- Summative refers to its performance in achieving the activities, outputs and outcomes that address the problem the collaboration was formed to solve. Evaluating impact should be a high priority for collaborations, as a failure to achieve collective impact, or to show returns for the investment, will undermine the partners’ justification for continued involvement.
Take a long-term view, keeping the stage of your collaboration in mind
Since collaborations are often looking to solve complex challenges, it often takes a long time to achieve outcomes. Collaborators need to acknowledge this timescale when evaluating their performance.
Timing the use of evaluation tactics correctly is crucial for effectively planning to improve a collaboration’s outcomes. For example, focusing on summative performance during a collaboration’s initial stages may come at the expense of efforts to cohere as a collaboration, leading to problems in forming relationships and maintaining legitimacy.
Evaluate achievements as a system; avoid narrowly focussing on individual contributions
In a collaboration, outcomes depend on the combined performance of the organisations within the collaboration. Focussing solely on the contribution of each individual organisation creates incentives for a partner to seek credit and funding in a zero-sum manner. Instead, achievements should be seen as collective impact.
Make sure your evaluation method can be adjusted for potential changes
Change is always possible for complex systems. Even the evaluation method itself will likely need to be adjusted to accommodate collaboration adaptation.
Professor Geoff Mulgan recommends ‘Triple loop’ thinking, a systematic approach to learning and adaptation at three different levels, illustrated in The Collaboration Playbook.
On the first loop, a conventional approach is taken to the challenge and solutions.
The second loop considers additional factors, placing the problem and solutions in a wider context to consider interconnected challenges and associated solutions.
The third loop focusses on radical rethinking, with entirely new ways of conceptualising the task and outcome being considered. Mulgan calls this third loop ‘rethinking how we think’. It will be the stage of reflection that will likely require the most creativity and diverse sources of information.
Using a method like this that plans for evaluating a changing approach can help ensure that the evaluation process remains effective even when the collaboration has adapted its strategy in response to environmental changes.
Learn more about how to make a collaboration successful in WIG’s Collaboration Playbook, researched and written by the University of Oxford’s Blavatnik School of Government.
Originally published: