How Biased Are Your Customers?

We all have biases. These biases are formed of our experiences in life and create prejudices in favor of, or against, different things. One of the many ways these biases manifest is in our responses to psychometric tools like ability tests, personality questionnaires, and even responses on Net Promoter Score (NPS) inquiries. These differences might be little, but when you group them together over a large set of data, you discover that they add up to some big biases.

Net Promoter Score (NPS) is used to determine how likely you are to recommend a product or service to one of your friends or family. Expert NPS analyst Brendan Rocks, Head of Data Science at Satmetrix, says demographic details of your respondents can bias the NPS. And you can detect these biases accurately – provided you have enough data.

Most people want to believe they aren’t biased, as bias is generally considered to be a negative thing. However, because we are human, by our very nature, we have prejudices or biases. When you consider how biases shape our behavior, it’s easy to see they can change the way we respond to certain stimuli, like, for example, a Net Promoter Score inquiry.

Rocks says that most of these biases are “small and noisy.” However, because Satmetrix has a large database to sort through, they can identify demographic biases in NPS at a higher level of understanding. For example, Rocks says that, all other things being equal, on average, men will rate a brand around four Net Promoter points lower than women (e.g. on a scale from -100 to 100). Also, he says the data shows that older consumers tend to give higher scores than younger ones.

“These little differences are cumulative across demographic dimensions, and can add up when you’re comparing segments in aggregate,” Rocks explains. “For example, say you have one brand marketed towards older women and another towards younger men. Are the differences you see in NPS attributable to performance or demographic bias? Often, it’s just not possible to answer questions like this with data from a single company. But by looking at data across several hundred companies, we can get a pretty good estimate of demographic differences, and get to the bottom of the performance question.”

The demographic bias is not just tied to gender or age, however. It can also be caused by where you live in the world. Rocks explains that similar demographic biases are often associated with B2B companies that look at NPS for different countries. When employee compensation structures are linked to NPS, it can cause a lot of problems for the various international regional managers, particularly if they are in charge of a region where the demographic bias for the nationality skews the scores lower!

Understanding these demographic biases and the reasons behind them is a complicated business, to be sure. Like many of the things associated with people’s behavior, however, it is essential for today’s businesses to use this understanding to help move their Customer Experience to the next level.

“Making data-driven decisions has become central to acquiring customers, and the same is increasingly true for retaining them. Customer experience practitioners are often put in a very tricky position. They are being asked to do things like establish causality with imperfect data and produce accurate predictions, all under considerable uncertainty, and for a business that’s constantly changing. This is where the skills of statisticians and data scientists come into play,” he says.

Rocks admits it is a complicated process to tease out the demographic bias in Net Promoter scores. However, the resulting understanding can help businesses make better decisions to optimize their business and improve NPS.