Page 28 - MSDN Magazine, July 2018

COGNITIVE SERVICES
Improving LUIS Intent
Classifications
Zvi Topol
The Language Understanding Intelligence Service
(LUIS), which is part of Microsoft Cognitive Services, offers a machine learning solution for natural language understanding. There are many use cases for LUIS, including chat bots, voice interfaces and cognitive search engines.
In a nutshell, when given a textual user input, also known as an utterance, LUIS returns the intent detected behind the utter- ance, that is, what the user intends to ask about. It also detects the different entities—references to real-world objects—that appear in the utterance. Additionally, it outputs a confidence score for each intent and entity detected. Those are numbers in the range [0, 1], with 1 indicating the most confidence about the detection and 0 being the least confident about it.
Previous MSDN Magazine articles have covered the basics of LUIS in detail. In particular, I encourage you to refer to the article, “EnableNaturalLanguageInteractionwithLUIS,”byAshishSahu (msdn.com/magazine/mt745095) for additional information about how to get started with LUIS.
This article will focus on two open source tools, Scattertext and LIME, which can help you understand the detection and classifi- cation of intents by LUIS. (In what follows, I’ll use detection and classification interchangeably.)
In particular, I’ll show how such tools can be used to shed some light on the classification process and explain why LUIS is uncer- tain about its intent detection in some cases—typically situations in which the top intents detected for a given utterance have sim- ilar confidence scores, for example a 50-50 split between two in- tents. It’s more likely to output the wrong intent in such situations.
While LUIS currently supports some troubleshooting capabilities, including active learning to help identify and retrain utterances it’s uncertain about, there are no word-level visualization and analysis tools that can further help resolve such uncertainty. Scattertext and LIME can help in overcoming that limitation.
Now let’s take a look at a simple FinTech case that will serve as a running example. Imagine you work for a bank and you’ve been tasked with understanding user questions that fall into two categories:
• Questions about their personal bank accounts, such as: “What is my savings account balance?”
“What is the latest transaction in my checking account?” “I would like my savings statement to be sent again” “Have I received my April salary yet?”
“When was the last cell phone auto pay processed?” “What are annual rates for my savings accounts?” “What is the balance in my checking account?”
• Questions or requests about other banking services, including mortgages, auto loans, and so forth, such as:
This article discusses:
• LUIS intent classification
• Visualizing how utterances fit into your model’s intents
using Scattertext
• Explaining intent classification using LIME Technologies discussed:
LUIS, QnA Maker, Scattertext, Python, LIME
22 msdn magazine