There already are several articles on my blog around the subject of the Microsoft Cognitive Services. One that’s still missing, is LUIS – the Language Understanding Intelligent Service. So today, I’ll give you a brief introduction about LUIS so you’ll be able to know what it is and what it can do. As a demo, we’re trying to teach an AI to act like a restaurant where we want to order some food. Hello LUIS!

What is LUIS?

LUIS is part of the Microsoft Cognitive Services which means that it’s part of an extremely powerful yet easy set of APIs developers can use in order to tap into the massive power of machine learning. As the name suggests, LUIS is specifically made to understand language. Although it might look simple at first, it’s actually a lot harder than you might think. Sentences and utterances can be completely different, although they mean exactly the same. Consider the following examples:

• I want to order food
• Bring me a hamburger
• I want to place an order

The sentences look totally different, they have the same intent: order some food. Normally, you would try to parse each sentence (maybe through the usage of regular expressions), but that’ll cost you a lot of effort and you’ll probably won’t get it right. LUIS will help you with this problem, since it’s able to understand all the intents and handle accordingly.

Creating your first LUIS model

In order to do so, simply head over to LUIS.ai and navigate to My Apps. Select New App to start. Although the name “App” might be a little bit confusing here, since we’re essentially going to create a LUIS Model. LUIS supports a couple of different languages, so make sure you select the correct language before moving on.

Intents and Entities

Now that we have our LUIS model, we can start training. As stated at the beginning of the article, every utterance comes down an intent, the action you’re trying to achieve. Navigate to Intents, Add Intent and give it a name (in my example: OrderFood). Now start in typing Utterances that’ll be used to start your intent. In other words, simply write down a couple of sentences you would expect people to say.

But wait, do I need to add another utterance for each type of food that I sell? Luckily, you won’t need to do that. Simply select the word that’s variable (in my case the word hamburger) and add it as an Entity. LUIS can now be trained to understand different kind of foods when they are placed in the same kind of utterance.

Don’t forget to Save all your changes before moving on.

Train and test

Once you got your intents and entities in place, head over to Train & Test. This page will allow you to train LUIS to learn your intents, utterances and entities. Simply press the Train Application-button and let LUIS do it’s magic.

Now, we can use the Interactive Testing to check how the model behaves. Simply test some utterances, change some words and check how LUIS responds. The most importing thing here is the Top scoring intent, since that’s the intent that LUIS will assign as a matched utterance. Also note that LUIS changes words to the assigned entity automatically when it recognizes it.

In this example above, I’ve used the following utterance: i want to eat a hotdog. When I created OrderFood intent in the previous step, I have never used exact “i want to eat a” phrase. Also, I’ve never used the word hotdog. Still, LUIS is able to recognise this sentence as an OrderFood and even parses the hotdog as the correct entity. All because the system is learning from all kind of phrases and recognizes entities itself.

(Re)training through the API

In order to get your automated build integrated with LUIS, you’ll need to be able to call it’s Programmatic API. There are several methods available in order to do so. This will im- or export the LUIS model as you wish so you won’t be able to do so through the portal. Take note all these methods require a Ocp-Apim-Subscription-Key in the Request header, which holds your subscription key. These methods are most commonly used to (re)train the model.

• Export Application – Exports a LUIS application to JSON format
• Import Application – Imports an application to LUIS, the application’s JSON should be included in in the request body.
• Add Batch Labels – Adds a batch of labeled examples to the specified application
• Train – Gets the trained model predictions for the input example

You can use a tool like Postman or the API testing Console from Cognitive Services to call the API. Here you’ll find an example of Postman calling the Export Application on the API (note the {appId} and {key} have been removed in this example).

An exported model will be a JSON-file. It contains all the data from the LUIS model. Here’s an example of a (trimmed down) version from my RestaurantLuisModel.

{
    "luis_schema_version": "2.1.0",
    "versionId": "0.1",
    "name": "RestaurantLuisModel",
    "desc": "",
    "culture": "en-us",
    "intents": [ {
            "name": "OrderFood"
        } ],
    "entities": [ {
            "name": "food"
        } ],
    // Removed data
    "utterances": [ {
            "text": "i would like to order a hamburger",
            "intent": "OrderFood",
            "entities": [ {
                    "entity": "food",
                    "startPos": 24,
                    "endPos": 32 } ]
        }, {
            "text": "i want to place an order",
            "intent": "OrderFood",
            "entities": []
        }
    ]
}

Conclusion

LUIS makes is fairly easy to understand a language and identify Intents with utterances and Entities. Since it’s a system that you can train, you can gradually make it better and learn from the past. Although not discussed in this article, LUIS even has the possibility to check Suggested Utterances making your LUIS Model even better. Take note that not all supported languages have Prebuilt entity support. This might cause LUIS to act differently than you would expect.

In my next article, I’ll dive into the possibility to integrate LUIS with the Microsoft Bot Framework in order to create a smart chat bot. This combination is extremely powerful when you’re creating a chat bot that uses natural language. Stay tuned!

Want to learn more about this subject?
Join my “Weaving Cognitive and Azure Services“-presentation at TechDaysNL 2017!