natural language understanding (NLU)

What is natural language understanding (NLU)?

Natural language understanding is a branch of artificial intelligence that uses computer software to understand input in the form of sentences using text or speech.

NLU enables human-computer interaction. It is the comprehension of human language such as English, Spanish and French, for example, that allows computers to understand commands without the formalized syntax of computer languages. NLU also enables computers to communicate back to humans in their own languages.

The main purpose of NLU is to create chat- and voice-enabled bots that can interact with the public without supervision. Many major IT companies, such as Amazon, Apple, Google and Microsoft, and startups have NLU projects underway.

How does natural language understanding work?

NLU analyzes data to determine its meaning by using algorithms to reduce human speech into a structured ontology -- a data model consisting of semantics and pragmatics definitions. Two fundamental concepts of NLU are intent and entity recognition.

Intent recognition is the process of identifying the user's sentiment in input text and determining their objective. It is the first and most important part of NLU because it establishes the meaning of the text.

Entity recognition is a specific type of NLU that focuses on identifying the entities in a message, then extracting the most important information about those entities. There are two types of entities: named entities and numeric entities. Named entities are grouped into categories -- such as people, companies and locations. Numeric entities are recognized as numbers, currencies and percentages.

For example, a request for an island camping trip on Vancouver Island on Aug. 18 might be broken down like this: ferry tickets [intent] / need: camping lot reservation [intent] / Vancouver Island [location] / Aug. 18 [date].

NLU vs. NLP vs. NLG

NLU is a subset of natural language processing (NLP). NLP attempts to analyze and understand the text of a given document, and NLU makes it possible to carry out a dialog with a computer using natural language.

While both understand human language, NLU communicates with untrained individuals to learn to understand their intent. In addition to understanding words and interpret meaning, NLU is programmed to understand meaning despite common human errors, such as mispronunciations or transposed letters and words.

The other subset of NLP is natural language generation (NLG). NLG enables computers to automatically generate natural language text, mimicking the way humans naturally communicate -- a departure from traditional computer-generated text.

Generally, computer-generated content lacks the fluidity, emotion and personality that makes human-generated content interesting and engaging. However, NLG can use NLP so that computers can produce humanlike text in a way that emulates a human writer. This is done by identifying the main topic of a document, and then using NLP to determine the most appropriate way to write the document in the user's native language. Text is generated based on this decision.

For example, using NLG a computer can automatically generate a news article based on a set of data gathered about a specific event or produce a sales letter about a particular product based on a series of product attributes.

Natural language understanding applications

Here are examples of applications that are designed to understand language as humans do, rather than as a list of keywords. NLU is the basis of speech recognition software  -- such as Siri on iOS -- that works toward achieving human-computer understanding.

IVR and message routing. Interactive Voice Response (IVR) is used for self-service and call routing. Early iterations were strictly touchtone and did not involve AI. However, as IVR technology advanced, features such as NLP and NLU have broadened its capabilities and users can interact with the phone system via voice. The system processes the user's voice, converts the words to text, and then parses the grammatical structure of the sentence to determine the probable intent of the caller.

Customer support and service through intelligent personal assistants. NLU is the technology behind chatbots, which is a computer program that converses with a human in natural language via text or voice. Chatbots follow a script and can only answer questions in that script. These intelligent personal assistants can be a useful addition to customer service. For example, chatbots are used to provide answers to frequently asked questions. Accomplishing this involves layers of different processes in NLU technology, such as feature extraction and classification, entity linking and knowledge management.

Machine translation. Machine learning (ML) is a branch of AI that enables computers to learn and change behavior based on training data. Machine learning algorithms are also used to generate natural language text from scratch. In the case of translation, a machine learning algorithm analyzes millions of pages of text -- say, contracts or financial documents -- to learn how to translate them into another language. The more documents it analyzes, the more accurate the translation. For example, if a user is translating data with an automatic language tool such as a dictionary, it will perform a word-for-word substitution. However, when using machine translation, it will look up the words in context, which helps return a more accurate translation.

Data capture. Data capture is the process of gathering and recording information about an object, person or event. For example, if an e-commerce company used NLU, it could ask customers to enter their shipping and billing information verbally. The software would understand what the customer meant and enter the information automatically.

Conversational interfaces. Many voice-activated devices -- including Amazon Alexa and Google Home -- allow users to speak naturally. By using NLU, conversational interfaces can understand and respond to human language by segmenting words and sentences, recognizing grammar, and using semantic knowledge to infer intent.