Natural Language Processing for Digital Games　Section 1: What is NLP?

In SQUARE ENIX AI Division, we are working on R&D for make use of NLP for Digital games. On April 24th, we've released "SQUARE ENIX AI Tech Preview: THE PORTOPIA SERIAL MURDER CASE" as our Tech Preview. Thank you so much to everyone who has tried it and to those who have not yet experienced it but are interested in it.

We released "SQUARE ENIX AI Tech Preview: THE PORTOPIA SERIAL MURDER CASE" as a software to experience what is natural language processing. Now, what is "natural language processing" used in it? In several articles, including this one, I will talk about what "natural language processing" is and how we are working on it.


・What "Natural Language Processing" means

First, natural language refers to the "languages" we use in our daily communication (this text is also written in English , a natural language). Natural Language Processing (NLP) is a generic term for technologies that process natural language by computer, and is a field of AI.

NLP covers a broad and diverse range of tasks. It includes "summarizing" to summarize the content of long sentences, "question answering" to respond to input by words or sentences, "text classification" to identify the genre of documents, etc. Techniques for handling information specific to specialized fields, such as the field of law or the field of medicine, are also the subject of NLP research.

Human language has ambiguity; the same word can have multiple meanings and nuances depending on the situation and circumstances. To make computers comprehend human languages with such ambiguities, or Natural Language Understanding (NLU), is an essential topic of NLP. Natural Language Generation (NLG), which enables computers to output natural sentences, is expected to lead to a variety of real-world applications. Research has been conducted on AI that can answer questions on behalf of human operators and on AI that can respond to chatting.

When we speak of NLP, we are generally referring to text processing. Speech processing, such as Automatic Speech Recognition (ASR or Speech-To-Text, STT) and Text-To-Speech (TTS), is a field that requires different expertise because it deals with "sound," which has different characteristics from texts. This is a field that requires different expertise. Nevertheless, these technologies are closely related to NLP applications, such as converting speech into text by ASR and understanding that text by NLU, or converting text generated by NLG into speech by TTS.

In the past decade, NLP has evolved significantly, and AI-powered translation (called "machine translation"), for example, has dramatically improved in accuracy to a level where it is used on a daily basis and in business applications. Today, it is possible to produce sentences that are comparable to those written by humans, and even "creative" sentences, which were once considered difficult to produce, are now being produced at a high level of naturalness.  "Deep learning" has been the driving force behind this explosive evolution.

In a series of articles, I will explain the recent rapid progress of NLP and its background, as well as discuss how it can be applied to digital games.

In the first article, as a key point, I explained: "what natural language processing is."
In the next, I would like to give an overview of how natural language processing has developed over the past decade with the advent of deep learning.