Beginners Guide to Speech Analytics


What is it?

Despite comments and speculation to the contrary, research completed by Ventana Research shows that one-on-one phone calls continue to dominate the volume of interactions between customers and companies. What this means is that there are large volumes of customer-related insights tied up in call records.

With the technology previously available, companies were only able to access a tiny percentage of this information because it was too expensive to record all calls, and assessing the content was largely a manual process. However, new recording techniques have made it cost effective to record all calls and speech analytics software means the data can be analysed automatically.

Using speech analytics tools your customer service operation can process and mine all calls based on the parameters you set and can deliver to your desktop advanced visual reporting and analysis tools providing managers with instant access to large volumes of data in simple intuitive reports.

Speech recognition software has been around for many years but only in the last decade have commercial-strength applications been developed. Nowadays there are many applications in the customer service world which can be classified as:

  • Applications such as biometric security, natural language IVR, situation intervention and guided scripting that are used ‘in call’.
  • Speech analytics applications that apply the technology to analyse data ‘post call’ for quality assurance and compliance or process improvement.

Typical uses

Call recording and speech analytics are generally used for one of four purposes: to prove compliance, to resolve disputes, to remove paper from the business process or to better train agents.

A key benefit quoted by all solution vendors is that of reduction in headcount. This comes from the ability to quickly identify the calls that need attention rather than sampling calls. In one large UK implementation, for example, the quality assurance team was reduced by 20 FTE almost overnight.

How does it work?

There are three different approaches to the underlying technology:

  • Phonetics based
  • Text to speech
  • Grammar based

Each vendor will sell the benefits of their solution, of course. However, there are technology differences and these should be considered as part of a selection process.

In essence, the recorded speech is analysed, indexed and stored in a database. Just like a normal database you run queries on the indexed data. These queries tend to be written like advanced Google searches.

You need to understand how to interpret the results, of course, and in general, will need to keep tuning the queries. This means you need a pretty skilled business analyst using the tools. You should expect to be obtaining close to 90 per cent accuracy on ‘well tuned’ queries.

The software will give you a probability that this call matches your query. You set the threshold below which the results are considered incorrect. What you need to consider in the results you get is how many ‘false positives’ occur and how many ‘miss hits’ occur. The better the tuning, the fewer of each you’ll have and the higher your threshold can be.

How do I use it?

If you use it for fairly static applications such as quality assurance or compliance, you will spend more time upfront working on the tuning to increase accuracy. The system will then highlight all the calls where the ‘rules’ have probably been broken. In this process you will want to spend time removing false positives but also listen to excluded calls to develop queries that you might have missed. Eventually you should reach over 90 per cent accuracy.

If you want to use the application to improve your business processes – helping you understand your issues, what customers are calling about and what you need to do to fix their issues – you’ll need to work in a more ad hoc manner. You need to put the tools in the hands of a skilled analyst who can use the query language and understands the business.

You’re not aiming for the same level of accuracy in this application – you just want to get close to the handful out of many thousands of calls that are going to tell you most about the issue you are investigating.

The challenge in this latter case is that if the information is not there you cannot find it! For example, if your advisors are not asking customers why they failed to login to the online service to make their payment but instead simply take the payment and end the call, you are no closer to understanding the root cause. So as with many root-cause analysis techniques, engagement of the advisors is essential.

How do I get started?

Most of the tools come configured with standard call centre queries that will help you to identify common ‘call language’ such as transfers, complaints, over talking and hold periods. Beyond this, queries need to be designed and then tuned for your product names, processes and industry terms. While there will be some common language used within an industry, do not take for granted that what works for one company will work for you. With a generic set of queries you’ll be looking at 60-70% accuracy.

At this point, your skilled business analyst and a period of tuning to the core business language is needed. Getting the tools ‘fit for purpose’ is therefore an essential stage that can take several weeks.

How do I buy it?

First of all you’ll need a voice recorder! There may be network-based VoIP providers who can record calls and offer speech analytics but I’m not aware of them. Your main system provider may offer a solution but it’s unlikely they will be experts at applying the technology to your business issues.

Most vendors operate through resellers who will either offer a SaaS (Software as a Service) option or sell you the platform to operate in your own business.

SaaS takes away all the issues of IT involvement and for one-off or small-scale uses is probably more cost effective. You lose the advantage of building the knowledge inside the business, however, and have to rely on the skills of the provider to tune the queries and analyse the results. At the entry point, you’ll find SaaS vendors who will analyse 200 hours of calls and write a report in a period of 2 weeks for £5,000. Others will build a dashboard for you.

If you buy the platform you have a significant IT investment to make! If you’re running large-scale quality assurance or compliance applications then this can be fine when offset against the saving in FTE. Be prepared for a spend of £100,000+ per year.

Are there things to watch out for?

Compression used in voice recorders may result in poor quality audio. We can understand it but the technology may not. This can increase the hours of calls that need to be analysed in order to get a good sample.

If you want to run the analytics on a continuous basis you’ll need sufficient processing power to analyse and index the calls. This may mean putting your ‘idle’ PC network to work overnight to process all the data. Depending on your IT department this may or may not be an issue!

Remember the tools are only as good as the analyst using them and the data being analysed. If your front line haven’t tried to understand from customers why problems have occurred, then analysing calls won’t get you too far.

Benefits of using speech analytics

There are three main areas where speech analytics can improve the performance of contact centres and the enterprise as a whole. One, it can identify agents that aren’t handling certain parts of calls or different call types. This allows companies to identify training and coaching requirements. Two, it can deliver extensive insights into customer preferences, issues, and feelings, allowing companies to modify customer acquisition, retention and up-sell opportunities. Three, it can identify business process issues, product and service issues, and even propensity of certain customer groups to defect to the competition, thereby allowing companies to change these root causes and thus improve business performance overall. The first two can both show considerable return on investment but the latter can go beyond that and identify survival techniques in ever more competitive markets.

Identifying the need for speech analytics

How do you decide if there is a need for speech analytics within your customer service operation? Think firstly about what your organisation is missing:

  • Do you know why your customers are calling? (caller intent)
  • What are your service reps and customers doing? (key behaviours)
  • What most impacts your relationship with your customers? (customer sentiment)

Analysis of speech and conversation will not only help answer the questions above it will improve operational efficiency, enhance customer experience whilst improving bottom-line contribution.

Looking at good and poor calls

In terms of customer service you will really only be interested in the information contained within the ‘poor’ and ‘good’ calls to help you address the issues raised above – but how can you identify and access those particular call recordings and consolidate the data in a way which provides genuine insight?

Speech analytics tools provide the ability to process vast amounts of data very quickly. For example, some solutions can now process more than 20,000 hours of audio data per day.


Analytics works by looking at the call flow to identify key phrases at the end of the call and write the results off to a database. Possible applications are:

  • Checking for mandatory compliance statements
  • Checking for agent scripting compliance
  • Checking for phrases or words that indicate a particular customer experience, motivation or activity and using them to analyse trends or influence how subsequent calls are handled
  • Cross referencing outcomes with agent activity to reverse engineer best practice

Legal Audio Discovery

The lack of a highly scalable, accurate audio discovery solution has hampered the discoverability of legal audio records. In the event of a lawsuit or regulatory action, the prohibitive cost and difficulty of retrieving these specific communications from amid vast numbers of more mundane audio files can force companies to accept a costly settlement.

Auditing professionals can locate critical evidence within valuable audio recordings within hours, not days. Automating the manual listening process eliminates the judgement errors caused by fatigue, boredom and other distractions that typically result from manual listening.

Speech analytics allows you to uncover relevant evidence that can prove the power of a legal argument.

Speech Analytics tools available

There are two main methods of processing the data, speech to text and phonetic processing.

Speech to text

Speech-to-text engines convert speech into text enabling complex queries to be carried out involving ‘word spotting’.

Phonetic indexing

Phonetic indexing is based on phonetics – the systematic study of sounds of human speech. It can also help detect emotion in the voice of a customer.

Further Reading


Author: Jonty Pearce

Published On: 14th Mar 2010 - Last modified: 4th Jan 2023
Read more about - Technology, , , ,

Follow Us on LinkedIn

Recommended Articles

An Introduction to... Contact Centre Analytics
Wooden cubes with speech bubbles linked to each other with lines.
Speech Analytics 101: What Is Speech Analytics?
A photo of someone presenting analytics findings
A Checklist for Implementing... Speech Analytics
The Call Recording and Speech Analytics Reference Guide