Why Virtual Assistants Fail and How to Fix Them

We’re excited to bring Transform 2022 back in person on July 19 and virtually July 20 – August 3. Join AI and data leaders for insightful conversations and exciting networking opportunities. Learn more

Most of us have had frustrating experiences with a virtual assistant. This is not always the fault of the virtual assistant. When we call a company, we usually call to resolve some issue — an insurance claim we disagree with, a service interruption — and we’re often frustrated at first. The virtual assistant doesn’t quite understand the problem, and the person we end up talking to isn’t the right person to solve it, sending us through what people in the industry sometimes call the “spiral of misery.”

And yet, as the past two years have shown us, there will often be times when call centers with human resources will inevitably fall short, from companies transitioning tens of thousands of employees to work from home, to crisis response and providing public health information during a pandemic. . In these situations, human help should focus on solving more critical and complex problems, not answering the same four questions over and over.

In my years of experience building virtual assistants for large corporations and governments, I have learned that one of the main reasons many virtual assistants fall short is that they are built by developers who lack business or user context, or by business professionals who have not done that. don’t have the coding skills or system insights to build a sophisticated experience.

To deliver a useful experience to users and customers, a conversational AI must reflect the full breadth of the business’s knowledge, based on insights from customer service assistants, marketers, sales, experience design leaders, and more. If you’re considering using virtual assistants to help you solve customer service challenges, here are a few steps you can take to ensure you build a seamless experience for your end users.

Get everyone in the room

The COVID-19 pandemic has been a perfect storm for virtual assistants, with hundreds of thousands of people needing chat support. It was also a time when organizations had to build new services essentially overnight, as was the case with one hospital we worked with that needed a virtual assistant to help triage COVID-19 care. Fueled by a powerful combination of need and desire, the organization brought together all key stakeholders — physicians, lawyers, administrators, public health officials, developers — and collaborated through every element of the solution. Because all relevant parties were buy-in from the start, whenever an issue or point of confusion arose, it was addressed. By building in the right perspectives, they were able to quickly build a working solution.

Collect research data close to the source

Many virtual assistants also fail because the creators don’t do enough user research or take the time to collect the right data. Even if you have a clear idea of what the assistant is trying to accomplish, say something as simple as showing a customer movie times, that doesn’t mean you have the data on how people are asking for movie times. Some customer service areas, such as healthcare or financial services, can also be particularly confusing. Users may not know what documents they need, or what the document they need is even called. Without reliable data on how these users ask questions and explain their problems, a virtual assistant is only going to frustrate them.

The best place to find this data is to get close to the source: input created by a human in some sort of virtual assistant. Maybe you have an old or defunct chatbot with user data to mine, or you can create a simple one to collect the data needed to create a sophisticated product. If you can’t do that, you may have an on-site search engine or call logs from your existing customer service lines. By using expressions that people have actually used, you can train a more responsive and context-aware model for natural language processing. People talk differently to people, chatbots, and search engines, so you’ll need to adjust your initial training data over time.

Visualize your end user

Another reason it’s important to build many perspectives into your virtual assistant is that you need a clear visualization of your end user. Most companies design a virtual assistant with a focus on what they want to get out of it, rather than focusing on the customer who will be using the product and the situation they are in. A simple example is documentation. Most people don’t have their accounts and login details in front of them. They probably don’t remember the exact day they paid their last bill or when the transaction in question took place. In order for a virtual assistant to guide that customer through a transaction, the designers need to have a clear idea of that customer’s situation, where and how to call or type, and the likely reasons.

Automate your 80%

A good rule of thumb for the industry is the 80/20 rule. The idea is that 80% of customer requests relate to 20% of the topics you need to cover. In other words, 80% of the requests your virtual assistant receives are likely related to your top four or five most common question types. On the other hand, the remaining 20% of your chats could be one of a thousand different questions, so many that you could never code them all. Automating the most common customer interactions is the only way you can free up call centers for the interactions that are too complex to predict or code.

All text affects the system

As a technologist, I feel like I can say this: We tend to write terrible dialogue. When technologists create decision trees, they think way too much about details like ratings and ramifications, while neglecting the overall user experience. Frankly, I’ve noticed that our dialogue is also rather rude or overly direct.

On the other hand, non-experts are not very good at training models. You may have one intention with much more training data than the other, which can make your virtual assistant inaccurate. It’s critical to remember that every bit of text you add to a virtual assistant’s logic affects the system. Adding data at random can cause you to accidentally teach your model things you didn’t mean. Garbage in, garbage out. Dialogue should be empathetic and human, but it should also be written with a view to a balanced system that favors sampling. In other words, your model should be trained on representative data, so if half of your users use the word ‘password’ and the other half use the word ‘pin code’, training data should contain about 50% use of ‘pin code’, 50% of ‘password’, but not 100% of either.

Looking ahead to the future, there is reason to believe that we will be working with virtual assistants a lot more, even if we don’t experience a crisis. New kinds of natural language processing can learn how to interpret context, rather than relying on sprawling search word trees that must be rigorously maintained. We can now build virtual assistants that can interpret what you’re looking for, or what kind of problem you’re likely to experience, based on your browsing habits. As this technology continues to improve, we as an industry can work together to create AI that helps consumers solve once-frustrating everyday problems.

Thanks to a better understanding of these best practices around virtual assistants and conversational AI, virtual assistants have become much more powerful and intuitive in recent years. As the underlying AI that powers them continues to improve, we will see virtual assistants play an increasing role in helping people manage their lives.

Andrew R. Freed is a Senior Technical Staff Member and Master Inventor at IBM. He is the author of Conversational AI, published by Manning Publications and holds over 100 software patents.

DataDecision makers

Welcome to the VentureBeat Community!

DataDecisionMakers is where experts, including the technical people who do data work, can share data-related insights and innovation.

If you want to read about the latest ideas and up-to-date information, best practices and the future of data and data technology, join us at DataDecisionMakers.

You might even consider contributing an article yourself!