Obviously Don’t Worry About Sarcasm

Anya Korneyeva explains why you shouldn’t be concerned about your speech analytics solution detecting sarcasm.

Sarcasm is considered by some to be an “oh-no” moment in the contact centre. However, there is no speech analytics solution on the market that can effectively pinpoint sarcasm. Should you be concerned?

Not necessarily.

Although sarcasm is a glamorous research topic, it is not a significant driver of caller sentiment or top-line revenue. In addition, sarcasm isn’t necessary for evaluating agent performance.

Sarcasm in the call centre is actually quite rare

After mining calls for hundreds of customers, our analysts confirm that sarcasm in the call centre is actually quite rare and never in a customer’s top ten concerns about their call centre. They have found that accurate categories and call scoring are much more reliable in identifying agent-performance problems or in driving marketing revenue.

Sarcasm is also notoriously difficult for machines to detect. First off, it is not easy to define and is subject to competing theories from psychology, linguistics, and neuroscience.

Even humans struggle to distinguish what is sarcastic and what is not

Secondly, even humans struggle to distinguish what is sarcastic and what is not. It requires a non-trivial amount of knowledge about the world and about the speaker, and a high sensitivity to non-verbal cues.

When two annotators were asked to label spoken sentences as sarcastic or non-sarcastic, their agreement rate was not far above chance. The human judges of sarcastic messages on Twitter didn’t fare much better.

The fact that sarcasm is rare and hard to label is bad news for machine-learning algorithms, which need lots of examples from a relevant data set. Since sarcasm is not a statistically significant problem in the first place, it doesn’t make sense to make an investment in a costly process of collecting and annotating data.

If sarcasm is difficult to deal with in written form, then it should be much easier to spot it in a call recording, right? Not exactly: a highly cited study on sarcasm in spoken dialogue deals with only one phrase (“Yeah, right”) and was performed on manually transcribed audio.

Unfortunately, with acoustic features alone the system produced unbalanced results. Judging from its low F-measure, the system either retrieved few instances of sarcasm, or could not reliably identify the ones it retrieved as sarcastic, or both. An acoustics-only system would face much steeper challenges in a contact-centre environment with no manual transcription and millions of utterances to interpret.

Sarcasm should not be the focal point of your speech analytics implementation

Given that sarcasm is nearly impossible for a machine to reliably identify and that it doesn’t drive quality in the contact centre, sarcasm should not be the focal point of your speech analytics implementation.

Instead, focus on other aspects of speech analytics that provide much more accurate, reliable indicators of agent performance and top-line growth. Analyse 100% of your calls (no sampling), reliably categorise and score each call, and get that feedback into the front-line agent’s hands as quickly as possible.

With thanks to Anya Korneyeva at CallMiner

image – ©iStock.com/megancch