Evaluation of the IKM Emergent Research Programme: Taking a Complexity Perspective to Evaluation

Describe The Approaches Utilized To Measure / Assess This KM Initiative: 

IKM Emergent has presented Chris Mowles of Red Kite Partners, UK, and Anita Gurumurthy of ICT4Change, India, with the challenge of evaluating the programme. The process is innovative in that uses a complexity perspective, examining the patterning of themes, paradoxes and power relationships within the programme and between programme participants and other actors in the development domain that have emerged as the programme develops. It is based on the understanding that evaluation means making judgements of value about the emerging understanding of programme participants as the programme develops. It was started in 2008 and will be completed in February 2011. Three interim reports have been produced along the way (see file attachment). It is employing the following methods: 1 Drawing together other evaluations already being undertaken by programme projects. 2 Encouraging reflection and reflexivity in working groups and amongst programme Participants on how their thinking has moved since being engaged in the programme. 3 Engaging a Southern evaluator for activities taking place in the South to gauge, amongst other things, their relevance to Southern audiences and to reflect on how N-S power relationships work through in practice. 4 Developing schematic maps and chronologies of how the programme has developed. 5 Attempting some kind of cost/benefit analysis or explain why this is not possible. 6 Exposing the evaluator’ methods, observations and reports to programme participants to promote the same kind of reflection and reflexivity that they are encouraging from everyone else in the anticipation that programme participants’ understanding of what they might mean by evaluation will also be moving as the programme develops.

What Do You Think Are The Main Unanswered Questions Or Challenges Related To This Field Of Work?: 

This approach implies the development of good relations between evaluators and those they are working alongside and issues of trust, detachment and impartiality are continuously being negotiated. How easy is this to square with the widely held, but largely unexplored concept of evaluator ‘objectivity’?

What Was The Purpose Or Motivation For Assessing This KM Initiative?: 

The evaluation is a continuous process which allows the steering group and programme management to learn from the feedback Chris and Anita give us as the programme progresses. They have also been asked to develop an evaluation methodology appropriate for the iterative nature of the programme and its interest in emergent issues.

What Were The Most Important Lessons Learned About The Assessment Process?: 

The evaluation is now nearing completion and the final report will be ready next month. The main lesson is that is has been useful to have ‘’impartial observers”, who are both part of the programme but not part of the programme at the same time, offering reflections, raising questions and summarising what they are observing over a 2-3 year period, and the course of the programme has been changed many times because of this.

What Would You Do Differently Next Time?: 

Include the proposed methodology within the main programme documents, right at the start of the programme but, at the same time, recognizing the fact that the evaluation itself is also an iterative process.

Image Upload: 

What Advice Would You Give To Others Based On Your Experience?: 

This has been a very useful process for inviting programme participants to take a detour via detachment. It should give others inspiration about how emergent evaluation is possible and offers an alternative to more traditional methodologies which can based on the assumption that development is a linear process and ignore unexpected changes, positive and negative, that can also arise.

Describe The KM Initiative: 

IKM Emergent is a research programme which explores information and knowledge processes within the international development sector. The programme is based on a critique of the assumptions behind more orthodox ways of working. It researches the development implications of cultural, economic and technical changes in the handling, use and exchange of information – known as informational developments - taking place in societies both “North” and “South”. Its work consists of a combination of action and scholarly research undertaken by an international network of academics and practitioners. It encompasses a large number of individual projects and sub-projects considering new information technologies, organisational KM, local knowledge and local content, multiple knowledges, and ways of connecting different kinds of knowledge. This approach also includes many forms of visualisation (such as the flag from Brazil - in the photo- which is a manifestation of local knowledge). IKM Emergent is a five year programme 2007-2011, funded by DGIS at the Royal Netherlands Ministry of Foreign Affairs.