This post is about conversational chatbots, and some of the underlying technologies that make conversational chatbots work. There are different levels of chatbot sophistication; A conversational chatbot is, as the name suggests, one where an interactive speaking conversation takes place.
In other posts, we discuss OpBots. An conversational OpBot is one where a person can speak to the chatbot or machine learning engine via another communication tool, and there is integration to the backend system that allows deeper and broader access to a businesses’ data.
In order for a conversational chatbot to help, you have to first understand what they need help with. Machine learning can be useful in gaining a basic grasp on underlying customer intent, but it alone isn’t sufficient to gain a full understanding of what a user is requesting.
Machine learning is often mistakenly used as an umbrella term for the ‘AI’ in conversational AI. This is inaccurate.
A typical machine learning model for text classification, by contrast, uses only term frequency (i.e. the number of times a particular term appears in a data corpus) to determine the intent of a query. Often these are also only simple and ineffective keyword-based algorithms.
A rock-solid natural language foundation
In addition to machine learning, the addition of a NLP (Natural Language Processing) engine ensures that the interpretation is as robust as possible. Advanced components, such as language detection, spelling correction, entity extraction and stemming are the foundation of rock-solid NLP ensures that conversational AI is able to correctly process any questions, no matter how poorly they are composed.
An example of why this distinction matters:
– I cancelled my trip and I want the ticket refunded
– You have cancelled my trip, I assume the ticket gets refunded?
In the first sentence ‘I’, ‘cancelled’, ‘trip’, ‘ticket’ and ‘refunded’ will be marked as important. In the second sentence ‘you’, ‘cancelled’, ‘trip’, ‘ticket’ and ‘refunded’ will be marked as important.
Both ‘you’ and ‘I’ in the above sentences are known as stopwords and will be ignored by traditional algorithms. This will render the sentences essentially the same and unhelpful. Deep learning models (without the removal of stopwords) understand how these words are connected to each other and can, therefore, infer that the sentences are different.
The difference may be minimal for a machine, but the difference in outcome for a human is glaring and obvious. In the examples above, where the words used are the same for the two sentences, a simple machine learning model won’t be able to distinguish between the two. In terms of business value, automating this process incorrectly without sufficient natural language understanding (NLU) could be disastrous.
Learning to speak ‘human’
By understanding which words are important in a given context, these models are able to figure out the potential mistakes and can correct it (as long as the training data quality is sufficient). It’s an extra layer of understanding that reduces false positives to a minimum.
Recent Comments