Why is our society so averse to failure, and what does this mentality mean as we are trying to create lasting positive change? Does it mean fewer creative ideas? Or does it contribute to a sense of fear as we step into the field? Or, perhaps it means projecting a false sense of confidence in our understanding of things as a “neutral”, “unbiased evaluator.”
There is an erred notion that it’s possible to set aside our privileges and worldviews simply because we choose a rigorous method of study in an evaluation. At the end of the day, we are still people with our own aspirations, hopes, fears, and views which influence the data we work with.
I believe we are averse to failure because admitting our mistakes means exposing ourselves, and therefore making ourselves vulnerable to others’ blame or criticism. It means peeling off the layers, standing naked in front of the mirror, and choosing to speak the truth each time. If though, we can see these failures as stepping-stones to a better version of ourselves as people and as professionals, there is so much we can gain.
The first time I was aware of failure and how it could shape my work, I was at the Minnesota Evaluation Studies Institute Training: Co-creating Evaluation Communities: Learning from Each Other. The Keynote speaker Michael Quinn Patton stressed how important it is to embrace failure as it comes, and to learn from it in evaluation. He said evaluators could benefit from sharing stories of failure with one another so they don’t replicate the mistakes of their peers. There is so much value to this.
As one of the newer converts to this doctrine, I’d like to share what I believe are some of the lessons I’ve learned from my own failures in a decade of international development and evaluation experience.
1. Community buy-in is everything
Sadly, in the past I’ve worked on projects where our goals as funders and program implementers didn’t align with community needs and aspirations. The danger of this is that without the support from community members, projects are often fragile and less sustainable over time.
I’ve used this experience now to self-select into work where I see strong community leadership and potential for growth. Some signs of community buy in are:
(a) when community members are the ones to approach an organization for a specific service or solution as opposed to the other way around
(b) when community members actively participate in the design of an intervention, and
(c) when there are clear channels for a community to provide consistent feedback and criticism to improve an intervention.
2. Transparency around roles is critical
Whether you’re collecting data as an internal or external evaluator, you are often perceived as someone with power who can influence future decision-making, and possibly bring more funding to a community. As an external evaluator, I’ve been stuck in many awkward conversations with interviewees who equate the evaluator role with the donor role. This isn’t their fault, as evaluation has evolved over time from a simple means of accountability and reporting, to an invitation for dialogue, learning and community building. It is best to give as much context as possible when conducting an evaluation. I also believe one must outwardly state the type of relationship you have with an organization and its funders before being prompted, so as to avoid misunderstandings.
3. People can relate to you better when you speak their language
A lot of international development work and evaluation takes place in developing countries. When we interview people for their opinions on a particular program, we’re sometimes asking them to reveal very personal information. In addition to their demographics, we might ask them about their dreams and aspirations, such as their hopes for a better school or for a safer community.
Such an evaluator’s role should not be taken lightly, and when given the chance to have these conversations with community members, it’s more humanizing to conduct them in their native language. It also removes the layer of complexity that comes with bringing in a local translator who may be from a higher social and economic class, and thus hold less trust or confidence from the community.
Ultimately, the best alternative here is to always opt for conducting interviews in people’s native language. If you don’t speak it, then finding a competent interviewer who does speak the language is the best route.
4. People’s experience in data collection matters
I’ve been in evaluations before where the priority was around collecting data, and not necessarily the experience participants from the community had during the evaluation. In these situations, we ended up designing a never-ending survey and/or coming unprepared to a data collection event and expecting people to sit still for several hours.
As I reflect on these experiences, it certainly wasn’t fair to the people that were providing us with valuable information for our evaluation. While as evaluators we always hope our contributions will help improve the quality of people’s lives, there is no guarantee that this will happen. Keeping that in mind, participants in data collection activities are setting aside precious time that could be spent elsewhere, and the least we can do is to provide a fun and memorable experience.
Budgeting time for icebreakers, games, and activities where people can bond has huge returns in the long run. Not only do participants get to relax and enjoy themselves, they’ll be in a much better state of mind to engage with the group and share their knowledge and experiences.
As I reflect on my experience in the field, I’m grateful for all these lessons. Taking an inventory of how my work influenced those around me both positively and negatively felt scary in the beginning, but it was a necessary step in aligning my consulting work to my own values. Through this process I’ve also been able to bond with other evaluators who have come across similar issues, which has helped us rethink the way we want to make change in the world.