Evaluagent Updates AutoQA to Assess Conversations Like An Experienced QA Analyst

evaluagent has introduced new capabilities to its AutoQA platform, enabling contact centres to assess customer interactions against current company policies and incorporate evidence from beyond conversation transcripts.

The updates include integrations with Confluence, Notion and SharePoint, alongside additional scoring data from CRM systems, conversation metadata, audio analysis and response times.

Together, these capabilities aim to provide a more complete assessment of interaction quality by considering not only what was said during a conversation, but also whether the correct processes were followed and actions completed.

Traditional automated quality assurance typically relies on conversation transcripts and predefined scorecards. However, these may not reflect changes to internal policies or procedures, particularly when knowledge is maintained across separate systems.

evaluagent’s new Knowledge Base Integrations allow organisations to connect their existing documentation directly to AutoQA, helping ensure evaluations reference current policies and procedures.

The integrations support Confluence, Notion and SharePoint, with read-only access to selected documents. Organisations can choose which information to synchronise and how frequently the platform checks for updates.

Changes to source documents, including additions, edits and deletions, can then be reflected in the knowledge used by AI-powered scoring criteria.

For organisations using other knowledge management systems, evaluagent also provides a Knowledge Vault API for importing documentation.

Alongside the knowledge integrations, evaluagent has expanded the evidence available to its generative AI scoring criteria.

AutoQA can now incorporate additional information, including:

• CRM and platform metadata: Case status, conversation tags, wrap codes, channels and other recorded information.
• Audio analysis: Periods of silence and instances of overlapping speech.
• Response times: First-response waiting times and the longest intervals between replies across digital channels.

Quality teams can reference these fields when configuring scoring criteria using plain-English instructions.

For example, an evaluation could check whether an agent correctly categorised an interaction or responded within a specified service-level agreement.

The platform replaces these references with the corresponding conversation data during evaluation, with only the selected fields shared with the AI.

The combined capabilities allow evaluagent to assess whether actions taken during customer interactions align with an organisation’s documented procedures.

For example, if an agent promises a customer a callback, AutoQA can examine the conversation transcript, check the relevant case records to establish whether the callback was scheduled, and compare the outcome against the company’s callback policy.

This approach is intended to address situations where an interaction appears successful based on the transcript but required follow-up actions have not been completed.

To support transparency, evaluation results include the evidence examined, the values identified and a plain-English explanation of the AI’s assessment. Users can also access the underlying evidence, including instances where expected information is missing.

The developments are also intended to support organisations introducing AI agents into customer service operations.

As AI agents take responsibility for more interactions, quality teams need to establish whether automated systems have completed the actions communicated to customers, rather than assessing responses solely on their wording.

By combining conversation analysis with operational data and internal policies, evaluagent aims to provide a consistent framework for evaluating both human and AI agents.

The new capabilities are available through evaluagent’s AutoQA platform, with further knowledge base integrations planned.

Author: Alice Cuerden

Reviewed by: Xander Freeman