Auto QA: How to automate customer service quality assurance

Quality assurance (QA) is the process through which your support team can systematically improve their customer service. Customer expectations are evolving as fast as technology, necessitating a proactive approach to support quality. A QA program that leans on AI and quality assurance automation will help your team adapt and surpass rising expectations.

In fact, according to the Customer Service Quality Benchmark Report, 70 percent of organizations plan to invest more in support automation.

So, let’s talk about Auto QA and how you can get a sophisticated grasp on support quality.

In this guide:

• What is Auto QA?
• How to blend automated QA and manual reviews
• How does Zendesk QA help you find the critical conversations?
• How does autoscoring work?
• The scalability of combining automated and manual reviews

What is Auto QA?

Auto QA is short for automated quality assurance. On average, only 2 percent of conversations are reviewed manually—by using a tool to automatically assess support tickets, Auto QA gives you 100 percent coverage for a complete, unbiased overview of what is happening in your customer conversations. It takes into account multiple categories and languages to handle huge ticket volumes in less time.

Manually reviewing conversations to find problem areas is like taking a metal detector to the beach for treasure. It’s pretty long and tiring, with no guarantee of a result.

With Auto QA, you already have a map to find out where the treasure (or problem areas) are. If your reviewers use it to understand which areas need closer examination, they have more time to strategize for improvement and ensure that your QA insights go deeper than basic scoring.

Pay attention to how your quality management software provider defines Auto QA, though. While many quality assurance tools offer automation, not many can actually score your support conversation automatically like Zendesk QA.

How to blend automated QA and manual reviews

Let’s talk about the division of labor and define the roles of automated QA, AI, and human—in other words, your reviewers—in customer service quality assurance.

Auto QA reduces labor and helps you understand your customer service quality faster than you’ve ever been able to before.

However, manual reviews are still vital in the customer service QA process. As your quality assurance copilot, Auto QA ensures that any time you dedicate to manual quality assurance is time well spent. Instead of spending time on conversations with limited learning opportunities, reviewers can jump directly to the most impactful conversations using Spotlight and advanced filters.

Understanding the limits of manual review

The ‘human’ in the feedback loop is irreplaceable: your quality team is specialized at digging into nuanced situations, conducting in-depth analysis of dashboards and metrics, and constructing feedback sessions.
However, the most difficult aspect of the feedback loop lies in human behavior, specifically due to the following obstacles when it comes to manual QA reviews:

1. Limited scope
   On average, only 2 percent of conversations are reviewed manually, which means a vast majority of interactions go unchecked, potentially leading to missed insights or issues.
2. Bias
   Reviewers may harbor unconscious biases that can influence their evaluations, resulting in unfair assessments and decisions that may not accurately reflect performance.
3. Lack of scalability
   Manual reviews are not easily scalable, especially as the volume of customer interactions grows, which can lead to delays in providing feedback and improving customer service.
4. Human error
   Inevitable in any manual process, human error can introduce inaccuracies and distort data.
5. Inconsistency
   Different reviewers may have different standards or interpretations of what constitutes good customer service.
6. Time-consuming
   Manual reviews require a lot of time and effort, more so as your ticket volume swells. This, of course, has a ripple effect on your bottom line.

Auto QA serves as a vital bridge, addressing the gaps left by manual reviewing. However, manual reviews and automated QA reviews are not mutually exclusive. With the right software, you can master both.

Scoring straightforward conversations automatically

A large proportion of your conversations are very straightforward, which means they don’t contain enough nuance to dive into manually.

For example, an interaction in which there is a quick back-and-forth between agent and customer tells you nothing new about performance and processes. If only 2 percent of conversations are being reviewed, highlighting a conversation like this for manual review is a waste of your reviewers’ time.

That said, this does not mean the conversation is not worth scoring. Understanding review scores on aggregate gives you full coverage for analyzing trends and seeing the bigger picture.

Auto QA increases your reviewing capacity by 50x and covers all your QA bases for every single one of your conversations.

Without having to open a single interaction, you can understand the breadth of both overall customer sentiment and team performance—regardless of ticket volume.

Finding the signal among the noise

Having an analysis of every single conversation is step one in an advanced QA process. This is the stage that Auto QA covers.

However, there are still conversations that matter for quality improvement purposes and require human eyes. So it’s important not to let them stay mired in the status quo.

It is crucial that QA specialists are at hand to scrutinize critical conversations for several reasons:

1. They help identify specific areas of improvement and facilitate targeted training or support to help agents improve performance.

2. They can better analyze customer sentiment and address issues that are negatively impacting customer satisfaction—whether those problems lie in customer service or need to be communicated to other departments.

3. QA specialists can identify potential risks or compliance issues in customer interactions and deliver constructive feedback with review data.

How does Zendesk QA help you find the critical conversations?

Conversations where customers show negative sentiment, or churn risk, or on the other side of the coin have expressed experiencing exceptional service, are all valuable for quality assurance’s sake.

Spotlight is a unique conversation discovery feature that automatically samples the conversations critical for review. These are selected through a multilevel statistical analysis of your own communications metadata. In other words, based on criteria AI has customized for your support team.

By reviewing conversations highlighted by Spotlight, you can ensure your QA efforts are focused on conversations that are critical to review and contain the most influential learning moments.

Conversation discovery is much easier with AI. According to the Customer Service Quality Benchmark Report, a third of all customer service teams surveyed already use AI to assist in the selection of conversations for review.

How does autoscoring work?

Once you have your support conversations filtered and ready for review, it’s time to look into how Auto QA actually scores them.

Typically, when scoring conversations, you would use a QA scorecard—an evaluation form that speeds up the review processes, makes feedback specific and measurable, and empowers the support team to meet service standards.

Now, autoscoring is not the same as sentiment analysis or automatically generated conversation summaries. Auto QA can analyze every conversation based upon set categories, many of the common categories that might already be on your customer service scorecard.

Let’s get into what an automated scorecard looks like in Zendesk QA, and how the score is reached.

Solution

In customer support, providing a satisfactory solution is the ultimate goal—a definitive answer to the customer’s query or concern. It is, therefore, one of the most popular scorecard categories.

💡 88 percent of customer service teams score conversations by “Solution”.

How is ‘Solution’ scored by Auto QA:

In our Auto QA, the ‘Solution’ category assesses whether an agent offered a solution during the conversation.

This employs a binary scale. It boils down to a straightforward question:

Did the agent propose a solution to address the customer’s issue?

Yes = 👍
No = 👎

Unclear or inapplicable = NA

It’s crucial to understand that the Auto QA ‘Solution’ category identifies whether a solution was offered by the agent, but does not discern whether it was the correct solution. (However, support agents usually do give correct advice, so it’s a very worthwhile indicator!)

You always have the option to override Auto QA scores manually. If, for example, a solution was offered but it didn’t happen to align with internal standards.

Tone

Communication is not just about words; it’s also about how those words are delivered. Tone plays a pivotal role in shaping the emotional texture of a conversation.

💡 63 percent of customer service teams score conversations by “Tone”.

How is ‘Tone’ scored by Auto QA:

Our evaluation of ‘Tone’ understands that a conversation can be a tapestry of emotions, with various tones interwoven throughout. Each tone is assigned a different weight, where positive tones contribute positively and negative tones affect the score accordingly.

The ‘Tone’ category currently recognizes 27 distinct tones, including concerned, optimistic, and apologetic, among others.

The scoring system falls on a 5-point scale. This scale allows us to gauge the emotional resonance of the conversation, providing valuable insights into the customer-agent interaction.

Empathy

Empathy in customer support goes beyond just resolving issues—it’s about understanding and showing genuine concern for customers’ feelings and problems.

💡 49 percent of customer service teams score conversations by ‘Empathy’.

How is ‘Empathy’ scored by Auto QA:

Our ‘Empathy’ category is designed to assess a fundamental question: “Was the Agent empathetic towards the customer and their problems?”.

Empathy assessment using ChatGPT is translated as a binary:

Empathetic behavior found = 👍

Empathetic behavior not found = 👍

Undetected = N/A

In customer support, clear communication is paramount. Our ‘Spelling & Grammar’ category is designed to ensure that all written interactions meet high language standards. This category detects various types of language errors, which are grouped into three categories: grammar mistakes, misspelling mistakes, and style mistakes.

💡 74 percent of customer service teams score conversations according to ‘Spelling and Grammar.’

How is ‘Spelling and Grammar’ scored by Auto QA:

For each conversation, we aggregate scores for the entire conversation and the agents involved. A weighted score for mistakes is calculated, considering various types of grammar mistakes.

The final score is calculated on a scale of 1-5 for both the conversation and individual agents. While the scores are saved in the database as 1-5, they can be easily converted to different rating scales, such as binary or 3/4 scales, as needed.

Greeting

The initial moments of a customer interaction often set the stage for the entire experience. A warm and welcoming greeting can make customers feel valued and understood right from the start.

💡 7 percent of customer service teams score conversations according to ‘Greeting’.

How is ‘Greeting’ scored by Auto QA:

Our Greeting category is designed to assess whether the agent successfully initiated the conversation with a friendly and professional opening.

The Greeting category utilizes a binary scale, where the focus lies on whether the agent greeted the customer:

Greeting found = 👍
No greeting found = 👎

Closing

How you end a conversation matters. Our ‘Closing’ category is your tool to ensure that your agents conclude customer interactions on a high note.

💡 7 percent of customer service teams rate conversations according to ‘Closing’.

How is ‘Closing’ scored by Auto QA:

Auto QA takes into account the two elements of an effective close:

• Goodbye: The way agents bid farewell impacts how customers perceive the interaction’s conclusion. Using appropriate phrases like bye or farewell leaves a positive impression.

• Future contact encouragement: Encouraging customers to reach out again is essential for building lasting relationships. Phrases like ‘Feel free to reach out’ can be powerful in this context.

The scalability of combining automated and manual reviews

A purely manual QA process lacks the scalability essential for large or expanding teams to keep pace with customer expectations.

On the other hand, automated reviews can analyze a large volume of interactions quickly, but can miss the intricacies and subtleties of human communication.

We have officially entered the era in which humans work best when alongside machines. This is certainly true when it comes to customer service quality assurance. There are 5 key principles that, when followed, aid optimal human-machine collaboration:

1. Reimagining business processes

2. Embracing experimentation and employee involvement

3. Actively directing an AI strategy

4. Responsibly collecting data

5. Redesigning work to incorporate AI and cultivate employee skills.

Auto QA taps into each and every one of these principles when used by a QA specialist with the eye to pluck out critical conversations and knowledge to put the ensuing analytical data into action.

Automated reviews can quickly identify issues, while manual reviews can provide a more detailed analysis of interactions and identify areas for improvement. This approach provides an efficient and scalable way to ensure high-quality customer service across a large volume of interactions.

And ultimately, an easier, smarter way to keep customers happy.