When presenting data, one should not assume that everyone will understand and interpret it in the same way. In fact, most people tend to ignore what doesn’t fit with what they believe to be true. Confirmation bias is the habit of favoring information that supports our beliefs and ignoring or downplaying information that goes against them. This can lead to incorrect conclusions and missed opportunities for learning. How can we overcome confirmation bias to improve our use of evidence in decision-making?

A Common Scenario

Early in my career, I was working on a longitudinal study that followed people for two years after the earthquake in Nepal that struck near Kathmandu in 2015. We were trying to understand what factors led households to recover and rebuild their homes. When the results came in, I sat down with a senior member of the team to review the data. He was immediately drawn to one result that showed a small but positive effect on households’ ability to recover. He wanted to include an interpretation of the result that overstated the size of the finding. Perplexed, I dug further into the reasons behind his intense focus on this result, especially since other results were clearly more compelling. It turned out that a similar study in a different country that he had worked on had shown very positive results in that area. To him, the Nepal study proved what he already thought was true and provided the narrative he needed at the time.

Although this scenario may seem harmless, it highlights how easy it is to fall in a confirmation bias trap. In a higher stakes scenario, confirmation can have devestating consequences. In fact, many believe that confirmation bias played a large role in the 2008 housing market and finanical crisis¹. As I progressed in my career, I realized that taking proactive steps to combat confirmation bias was time well spent. Below are some ways you can address confirmation bias when analyzing and presenting data.

How to Address Confirmation Bias

Be clear about your asssumptions

It is helpful to provide a clear articulation of your theory of change before designing your evaluation or research study. A theory of change outlines the causal logic that explains how and why an intervention leads to a desired outcome or impact. In other words, it shows a chain of events you expect to see when implementing an intervention. It is important to remember that a theory of change is essentially a series of assumptions that you are testing. If the data show a difference from what you expected, flag it—don’t ignore it. You can then investigate these unexpected findings further by collecting or analyzing more data.

Triangulate your findings

Another way to reduce confirmation bias is to use triangulation. Speak with different people to verify your findings. Try various methods to tackle the same question. Involve many researchers in gathering and analyzing data to limit individual bias. If various data sources, methods, or researchers come to the same conclusions, it boosts the validity of your findings. It is still important not to disregard where there is disagreement between data sources or researchers. Triangulation aims to explore the complexities and contradictions in data—not force a unified conclusion.

Bring in multiple perspectives in the analysis

As principle investigators, we are not immune to confirmation bias. Bringing in many different partners to interpret data helps to diversity and ground truth a study’s conclusions. It also helps to elevate voices that are often left out of the analysis process. One way to bring in many perspectives into data analysis and interpretation is through participatory sensemaking. Participatory sensemaking is a collaborative approach to bring people together to unpack and make sense of data. This 2-minute video outlines the three basic steps involved in sensemaking:

• Step 1: Review the data in small groups.
• Step 2: Use participatory techniques to reflect on the unexpected results and consolidate key insights.
• Step 3: Identify actions as a group and document them in an action plan.

Key Takeaways

We are all prone to confirmation bias, but there are ways to combat it. Developing a theory of change will bring out the assumptions you have and what results you are expecting to see. Triangulation will strengthen the validity of findings and bring out where there is disagreement. Participatory sensemaking will lessen individual bias by including many viewpoints to interpret the data.

Citations

1. Michael D. Eriksen & Hamilton B. Fout & Mark Palim & Eric Rosenblatt, 2020. “Contract Price Confirmation Bias: Evidence from Repeat Appraisals,” The Journal of Real Estate Finance and Economics, Springer, vol. 60(1), pages 77-98, February. ↩︎