Why QA and CSAT Scores Don’t Always Match

This blog summarises key points from a recent article by Derek Corcoran at Scorebuddy, exploring why QA and CSAT scores differ, how scorecards can overlook the customer experience, and practical ways to bring the two measures closer together.

Why QA and CSAT Results Often Tell Different Stories

They’re Based on Different Sets of Interactions

Start by looking at coverage. Industry estimates generally put CSAT response rates at 5% to 10% of contacts, while manual QA typically reviews 2% to 5% of conversations.

If both groups are sampled independently, our rough calculations suggest that only around one in every 10 to 20 QA-scored conversations will also have a survey attached. In most cases, then, the two scores reflect different customer interactions.

Some teams do send surveyed contacts for QA review, but they tend to focus on dissatisfied customers. This can help identify what went wrong during a poor experience, but it also introduces bias because the QA sample contains a higher proportion of negative interactions.

Aaron Mulville, who led the CX Masterclass, summed up the issue with sampling: “Decisions need a population, not a sample… a 3% sample size is an anecdote.”

QA Often Assesses Processes That customers Cannot See

The criteria included in a QA scorecard can also explain why its results differ from CSAT. Many questions focus on internal procedures rather than the customer’s experience. Others directly reflect what customers encounter:

• Was I understood the first time?
• Did I have to repeat myself?
• Was I put on hold without being told why?
• Did I have to get in touch again about the same thing?

However, scorecards may also assess the precise wording of regulatory disclosures, call notes and dispositioning, CRM hygiene, and whether agents followed internal system procedures.

COPC’s research into aligning QA and CSAT highlights a similar disconnect. An assessor might pass an interaction because the agent followed the correct procedures, even though company policy prevented them from resolving the customer’s problem.

The customer, however, may view the same interaction as unsuccessful. Gartner’s guidance on modernizing customer service quality assurance also encourages organisations to look beyond individual agent performance and incorporate customer feedback and experience data into QA.

This doesn’t necessarily mean either team is measuring incorrectly. QA assesses performance against its chosen criteria, while CSAT captures the customer’s perception of the experience. Often, they’re also looking at different conversations.

Why a Strong Overall Score Can Mask Poor Customer Experiences

A COPC example shows how a single quality figure can give management an incomplete picture.

One contact centre reported an overall score of 86%, but breaking the results down revealed three very different outcomes: 60% of transactions had no customer-impacting errors, 70% were free from business-critical errors, and all transactions passed compliance checks.

With compliance performing perfectly, the headline figure looked healthier than the customer experience suggested. The gap between the overall score and the customer-impacting result was 26 percentage points.

COPC recommends presenting these measures individually rather than rolling them into one figure. The same principle matters when QA and CSAT scores don’t align.

Adding sentiment and resolution measures to an agent’s QA score might seem like a way to close the gap, but it risks creating another blended metric where customer satisfaction becomes harder to distinguish.

Keeping the measures separate gives leaders a clearer view of what the process checks show and how customers actually experience the service.

Practical Steps to Bring QA and CSAT Closer Together

Identify How Much of Your Scorecard Customers Can Actually Experience

Start with your existing QA scorecard. Review each question and decide whether the customer would notice the behaviour being assessed or whether it happens entirely behind the scenes.

Separate the questions into these two groups, then calculate how much of the total score is assigned to the second group.

This gives you your scorecard’s invisible weight: the proportion of marks based on activities customers cannot directly observe.

If that accounts for 40% of the score, for example, improvements in those areas might boost QA without producing a corresponding change in CSAT.

Working out which customer-facing behaviours have the strongest influence on satisfaction takes more than a calculator.

The regression workbook provided with the masterclass can help you explore the relationship using your own scorecard and survey data.

Review The Same Interactions That Generate CSAT Feedback

QA and CSAT become easier to compare when they relate to the same conversations. However, when manual QA covers just 2% to 5% of interactions, matching reviews to surveyed contacts can be difficult.

AI scoring can make wider coverage possible. In one high-volume support operation, only three or four chats were being reviewed each month.

After introducing AI Auto Scoring, the team was scoring almost 4,000 chats, increasing coverage from approximately 0.1% to 100%.

Human reviewers still have an important role, particularly when dealing with regulatory issues, high-risk calls or scores that are being challenged.

Teams need to decide which interactions require human oversight, and a guide to how frequently people should review AI-generated scores can help inform that approach.

Report QA and Customer Satisfaction Separately

Once you can score the conversations linked to survey responses, keep the two results visible alongside each other rather than combining them into one average.

Comparing the results also allows you to calculate a mismatch rate: the proportion of interactions that meet QA standards but still receive a negative customer assessment. The masterclass session explains how to calculate this for your own operation.

Conversation analytics and business intelligence can help make these comparisons more useful. Conversation analytics identifies topics and sentiment within completed interactions, while business intelligence brings quality scores and customer sentiment together in reports for analysis.

Involve The Person Responsible for CSAT Early

Don’t wait until your first QA and CSAT comparison is ready to present before involving the person who manages customer satisfaction.

If they see the report for the first time in a leadership meeting, it could come across as questioning the validity of their results.

Instead, work together from the outset. The CSAT owner understands how the survey is designed, where its limitations lie and how the feedback is collected.

Their input can help explain the differences between the measures and identify improvements that benefit both teams.

---

This post has been re-published by kind permission of ScorebuddyCX - view the original article.

Call Centre Helper is not responsible for the content of these guest blog posts. The opinions expressed in this article are those of the author, and do not necessarily reflect those of Call Centre Helper.

Reviewed by: Megan Jones