Reflections of an Emerging Evaluator leading an evaluation: Power dynamics

11 March 2021 - Siyabonga Sibiya, Researcher, CLEAR-AA

Many debates and engagement on what constitutes a Young and Emerging Evaluator (YEE) in the context of the African Evaluation practice are raging. For instance, some Voluntary Organisations for Professional Evaluation (VOPE’s) such as the South African Monitoring and Evaluation (SAMEA) have coordinated platforms to engage on broad understandings of what defines or constitutes a YEE. The term continues to be used without a clear and commonly agreed conceptualisation in varying historical and geographical contexts. I have intentionally used the term ‘Emerging Evaluator’ to define myself in an emerging practice that has not yet convincingly defined the YEE concept.

In January 2020, I was assigned to ‘lead’ an evaluation of a civic education and active citizen participation project where a donor organisation ‘partners’ with community organisations to champion civic education and active citizen participation in several provinces in South Africa. A few days into the evaluation, I realised that one cannot overlook power relations that exist in the global ‘development’ arena, more particularly between donor organisations and the evaluator(s) as well as those between the evaluator and evaluand. I use the word power (even potential in its latent form) in the context of evaluation.

Donor organisations often have the power to determine who and what gets evaluated. Evaluators should constantly scrutinize and reflect on decisions made on who, how and what gets evaluated. The illusion of symmetrical power relations between the donor(s) and evaluator(s) was disapproved by this respective evaluation. In reflecting on decisions being made in the evaluation, we soon came to a realization that key stakeholder groupings who were mostly rural and often marginalized groups, were overlooked in evaluation processes. This particular evaluation had a pre-determined methodology and sampling frame which was prescribed by the commissioning organisation. Sampling prescriptions could either enrich or weaken the evaluation exercise. Evaluators should strive to interrogate commissioner prescriptions on elements such as methodologies, sample frames and sample groups etc. to ensure that these have no other basis either than scientific or practicality. Evaluators should continuously question prescriptions made in who, how and what gets evaluated as these have a potential to perpetuate notions of exploitation and disregard of parities whose insights and perspectives could enrich both the process and output of evaluations. Transforming the evaluation practice will require evaluators and other stakeholder groups to constantly reflect on decisions being made and how these perpetuate notions of exploitative power relations. 

The second lens of power experienced was that between the evaluators and vulnerable (often marginalised groups). The power that evaluators may have in the ‘field’ could either be exploitative or transformational. The immersion of an evaluator in the so called ‘field’ to do ‘fieldwork’ could either perpetuate exploitative or transformational evaluation practices. As a black urban evaluator evaluating a project in a peri-urban and rural community, it was important to reflect on the potential harm my background could exert on evaluation participants. The whole process of evaluation forced me to view and treat evaluation participants as humans who need to be treated with respect, listened to, and protected from foreseeable harms. Prior to data collection process we embarked on a power neutralization process which sought to empower and build confidence of the evaluation participants. This process resulted in the appreciation of my role as a facilitating agent in the evaluation and positioned evaluation participants as key agents in shaping the outcomes of the evaluation. 

It is very important for evaluators and evaluation teams to first acknowledge the potential harm that could be exerted in evaluation processes and constantly reflect on decisions being made that can have either an exploitative or transformational consequences. It is not sufficient to only acknowledge these but to adopt innovative approaches in ensuring that evaluations are not exploitative but take a transformative posture in ensuring that these are not harmful. One cannot exist within an instrument of power and be exempted from power, whether exploitative or transformative, evaluation is in itself an instrument of power and value judgement.