How to interpret your employee survey results: Avoiding bias
Interpreting and understanding employee data accurately is important, and modern survey reports give clear and easy to understand results. Having worked with many leaders on how to use survey results, I have seen many great examples of post-survey communication, reporting, and employee involvement.
However in the process of understanding what has caused our scores (and what to do about them) often we can be prone to biases, which can lead to incorrect assumptions, and potentially to make the wrong conclusions or actions.
In this article we explore some of the common psychological processes which can lead us to make false assumptions about the causes of events or outcomes, and how to avoid them.
We are experts at finding hidden meaning
We live in a world of constantly sent and received information – much of it incomplete, and we are pretty good at interpreting it. We see hidden patterns in complex events, make good guesses from imperfect data, and can guess at future events from our experiences. This helps us anticipate risks, or quickly understanding new or challenging situations.
This can be not ordering the fish because last time it was overcooked, or not buying a brand of car because a friend has one which breaks down. “Making assumptions” is often seen negatively, but in truth if we didn’t make assumptions, our path through the stream of data we encounter would be a lot more arduous.
While the psychological mechanisms that help us make assumptions are useful in many ways, they can also cause problems that lead us to make biased decisions, ignore the evidence, or draw false conclusions.
Biases and attributional errors
Two of the most common mental processes that cause bias, are Cognitive Dissonance (CD) and the Fundamental Attribution Error (FAE). CD occurs when what we believe about ourselves doesn’t match the external evidence we are presented with. This creates mental tension that we try and reduce – by reinterpreting the evidence to match our beliefs, e.g.
“I am a very good leader, and good leaders get good engagement scores. The engagement scores for my area are poor, therefore the scores must be inaccurate; or people didn’t understand the questions; or the results weren’t reported properly; or other factors over which I had no control caused the results”.
FAE leads us to give ourselves the benefit of the doubt when we encounter challenges, but to blame others when they encounter the same. For example:
“I got angry with my team member because I was provoked, you got angry with yours because you have a bad attitude”; “I was late for a meeting because something important came up, you were late for the meeting because you don’t care about your colleagues’ time.”
Biases in survey interpretation can be harmful
These types of biases can be very impactful at work, and can cause the same events to be interpreted very differently by different people. Imagine the scenario below:
In an organisation, Tom’s team has a high engagement score, but Jane’s team has a much lower score. Different interpretations could be made by the two managers, with very different implications. The divisional head may have a third, more-balanced viewpoint.
- The survey results are accurate and reliable.
- I am a great leader and my team respect me. Jane is a poor leader.
- I am likely to be more valued than Jane by others.
- Jane should learn from me to be a better leader.
- I am going to communicate the results widely with my team and the division.
- The survey results might not be reliable or accurate.
- Tom gets a lot more resources and support than me, and has an easier role.
- Others are going to unfairly value Tom more highly than me.
- Tom has nothing to teach me.
- I am not going to communicate the results widely in my team.
- Both of my Managers are good leaders with their own strengths.
- Tom’s team has higher scores, but they have declined since last year due to staff cuts. He does a great job.
- Jane’s team has low scores, but they have increased despite staff cuts. She does a great job.
- I am proud of both of my managers, but they should work more together to increase cross-functional working and share best practice to increase engagement for both teams next year.
The above example shows that we can’t take for granted the views of others, or that our own perceptions are not clouded by emotion or bias.
Do’s and don’ts for interpreting results to minimise bias
At People Insight, we think that interpreting survey data objectively and avoiding biases is so important, that the advice below forms a standard part of our action planning training sessions. We recommend that when interpreting survey data, that you:
- Do be aware of the biases that you and others may have when viewing survey results. Try to maintain an open attitude and a willingness to accept the results for what they are.
- Do understand that the results are employee perceptions – and may differ from your intentions.
- Do not seek to undermine the methods, accuracy, or scoring of results.
- Do focus equally on understanding the negatives and celebrating the positives!
- Do not take negative scores personally, take them as a guide for development and improvement.
- Consider other sources of data that may confirm or contradict your conclusions (e.g. anecdotal evidence, informal feedback, or other HR data.)
- Do share and seek the constructive views, input, and advice of your peers, especially to learn what good practice looks like.
- Don’t see survey results as departmental competition – every team has unique challenges. Support your colleagues with praise for strengths and ideas for improvements.
- Do seek to understand why people might feel this way, not “who said what”.
We all use cognitive shortcuts to help us make sense of the world, and often they help us to act and think fast, however an awareness of these biases and how they may affect us is very important in maintaining objectivity. This does not mean you should ignore your instincts, but remember that you do not need to “defend” against poor survey results. They are something to learn from, and your first reaction, especially when emotional, may not be the right one.
Thanks to Dr. George Margrove for this piece.
Delivered to you monthly.
Sign up to our monthly newsletter for all the latest in workplace culture and engagement
Thank you! You have been added to our mailing list.