2017 local time in California, February 5, AAAI conference held the top artificial intelligence, AAAI first set up this year AI in Practice (application of artificial intelligence) link, Baidu Vice President Wang Haifeng invited to do a "natural language processing Baidu "(Natural Language Processing at Baidu) keynote speech. Lei Feng Wang Haifeng live speech according to finishing the text, by sub Meng, Xia Rui, Sanchuan common editor.
Hello everyone, I am from Wang Haifeng of Baidu. Before introducing the work of Baidu NLP, I would like to talk about what the language means for the AI.
The ability to think and acquire knowledge is the result of today's human beings, which need to find the objects and methods of thought through language, and to transform our ability to see, hear, speak and act. And speech, visual, behavior and language is now an important field of AI research.
Language is one of the most important features of human beings that distinguish them from other creatures, as opposed to the ability to see, hear, and act. Language is the carrier of human thinking, usually our thinking language is the mother tongue. When we learn a foreign language, the teacher wants us to try to use a foreign language to think. On the other hand, from the beginning of human history, knowledge is recorded and transmitted in the form of language. The tools used to write language are continually improved: from oracle bone to paper, to today's Internet.
Therefore, we say that language is the carrier of thought and knowledge, and the treatment of language and understanding is particularly important. The purpose of natural language processing (NLP) in the computer field is to enable the computer to understand and generate human language.
In Baidu, we have developed knowledge maps based on the accumulation of large data, machine learning and linguistics. We analyze and understand query, text and emotion. We construct question and answer, machine translation and dialogue systems. NLP technology has been used in Baidu's many products, such as search, Feed, o2o and advertising.
Based on different application requirements, we established three kinds of knowledge map, including entity graph, attention graph and intent graph.
In the entity map, each node is an entity, each node has several properties, in this case, the connection between nodes is the relationship between entities. At present, our physical map already contains hundreds of millions of entities, tens of billions of properties and 100 billion relationship, which are from a large number of structured and unstructured data mining out.
Here's an example that searches for the question: Dou Jing Tong's ex-wife's father's ex-wife.
The characters involved in this sentence are very complex, however, our reasoning system can easily analyze the relationship between the entities, and eventually come to the correct answer.
In addition to the physical map, we have also created a focus map and an intent map, which I will present later in the chapter Understanding and Conversation.
Based on the techniques of entity recognition, grammar and semantic analysis, we developed query, text and perspective analysis and comprehension techniques. Next, I'll go further into query comprehension. We combine query dependency parsing (Dependency Parsing) and Semantic Understanding (Semantic Understanding) to achieve query comprehension.
In the example shown in the above figure, the query entered by the left user is & ldquo; I recommend a restaurant that can hear frogs. & Rdquo ;. We use dependency parsing techniques to analyze the syntactic structure of the sentence and help us find the various components of the sentence. For example, "recommended, restaurant" is the core component, indicating the user's main intent, and "hear, frog" is a modified composition, the user's intent was modified and limited.
The user's query on the right is "how much money my sister gets married to her sister and how much money", and explains how we can improve the semantic matching between the query and the page. First of all, we recognize the matching in the query based on dependency parsing, which can accurately represent the query semantics compared to a single word, which can be applied to the exact match between query and web page.
In addition, based on semantic understanding technology, we can understand the semantics of a query, to achieve semantic search, not just a literal match.
The first two sentences are:
Yingda's son who
Yingda is the son
These two sentences contain the same words, but the word order is different. If you use the traditional keyword-based search technology, we will get almost the same search results. However, through the analysis of semantic understanding technology, we can find that the semantics of these two sentences are completely different, correspondingly can be retrieved from the knowledge map to completely different answers. There is a third sentence:
Who is Yingda's father
In the literal sense, this is not the same as the second sentence, but after semantic understanding, we find that the two sentences are looking for the same object, so we can retrieve the same answer from the knowledge map.
We have also developedSemantic Understanding Technology Based on Deep Learning, To achieve a depth-based learning to calculate the query and text semantic association. We use more than 100 billion user data to train the model, for a query, including the user clicks positive and non-click negative example. We use the BOW, CNN and RNN models to learn the semantic representation of the language. In order to enhance the semantic representation of the model, we incorporate various syntactic and semantic constructs and integrate the "dependency structure" into the model.
The following figure shows the search results when the depth learning model is not applied, and the results are irrelevant.
After applying the depth learning model, the first three of the search results are relevant. DNN model has been applied since 2013, we have dozens of upgrades to this model iterative, DNN semantic features of Baidu search is a very important feature.
User access to information Another important channel is the Feed, which is personalized information, which, chapter understanding of technology has played an important role. Now, let me introduce some of our work in the understanding of the chapter.
We give the document a variety of labels, including: theme, topic and entity labels. Topic tags represent the concept of abstraction, where topic tags represent events that occur, and entity labels represent entity information such as people, places, and so on. These tags describe the content of a document from different perspectives to suit different application needs and are associated with different queries.
The integration of topic tags and entity tags, we formedAttention label map. This focus tag better describes the relationship between the user and the document, because it can simultaneously characterize the user and the document. We also build relationships between the different types of focus tags, so that we can reason about and compute the user's concerns. In the example shown in the following figure, "AI & rdquo; topic and & ldquo; Technology & rdquo; & ldquo;VR"And other topics and" Wuzhen summit "and other events associated with.
The following is an example of an application where the tag map is applied to a Baidu feed. In the left image, the tag represents the content of the article. Then the third map is based on the focus of the tag map personalized recommendations, better fit the user's concerns, bringing a higher click-through rate.
Emotional analysisIs another interesting topic in chapter understanding. Emotional analysis techniques, also called "opinion mining," are used to analyze human perceptions, emotions, and emotions about a variety of objects, such as products, organizations, and so on. Here are some of the work we've done in the "Viewpoint Mining" and "Summary of Opinions" section. "Hotel Reviews" as an example, we extract from the existing online comment data sentence, and then extract the user point of view. Based on these views, we can generate a summary of the viewpoints at the label level and a summary of the views at the sentence level. We can also use this as a basis for hotel recommendations.
Here are some examples of how emotion analysis can be applied to Baidu products. In the example on the left, we provide an evaluation of the dimensions of "Badaling Great Wall". In the example to the right , We in the analysis of the basis of the views of the user to provide a refined reason for the recommendation.
Automatic News Writing
Automatic news writing, which generates news articles from structured and unstructured data. This involves a total of four steps:
Data analysis: Identify the key information that is required to generate an article
Document planning: Determine the content and structure of the generated article
Micro-planning: Generating words, sentences, paragraphs, and headings
Article realization (surface realization): generate the final content of the article
The following is our automatic writing news. The first example on the left is the general news, the second is to generate the field of automotive news, the third is the sports news, now our automatic writing system has completed thousands of articles written in Baidu Feed products to Read by millions of users.
AI Basketball commentator
Our AI commentary system, like the human narrator, generates a live commentary of the game and interacts with the audience. There are four main steps to achieve this:
Information gathering (information gathering): real-time collection and extraction of online game key information
Generate structured data (structured data generation): based on different sources of competition information, generate structured data interpretation
Game scene inference: based on game data (such as scoring and statistics), infer the scene of the game scene
Generate live commentary generation: Generate live commentary based on the commentary model
The following is a commentary by our AI commentator about a real game.
In the middle of this figure, showing that the AI narrator can also be reasoned, which in one of the comments mentioned in the "test Sings grabbed the offensive rebounds", "offensive rebounds" This phrase shows that our AI narrator Through the existing knowledge of the test Sings where the team is currently in the offensive stage, after reasoning to arrive at the "offensive rebounds" conclusion.
The rightmost example illustrates how our AI narrator can answer multiple audience questions in addition to the narrative, which the human narrator can not.
Language generation technology can also be applied in another aspect: Chinese poetry is generated, and literary talent is not worse than the average poet. Chinese poetry has more than two thousand years of history, is an important part of Chinese culture, but for ordinary people, poetry is still very difficult.
We propose a two-step approach to the generation of Chinese poetry: first, the theme of each line of poetry to plan, and then the formation of specific verse.
For example, if the user wants to write a poem related to spring, then the poetic planning model will first generate a summary of the content, including spring, peach, Yan and Liu these four themes, and then by the RNN model based on the four Theme to generate four poems, to complete the creation of the first poem.
In the three poems shown below, the middle of this poem is completed by the AI poet, while the other two authors are ancient Chinese poet (Bai Juyi, Liu Yin). It is interesting to note that most of the people I asked were unable to tell which of the three poems actually came from the AI poet. History scholar and "Chinese Poetry Congress," guests, Professor Monmans also said, "the poet of artificial intelligence is a small superman poetry, and human poets can express the feelings of the poem in the same color. & Rdquo;
In addition, we have developed text summarization techniques. Specifically, it includes the general summarization and the query summarization based on the query as shown in the following table:
Text analysis (text analysis): analysis of the text structure
Sentence ranking (sentence ranking): through the sentence surface meaning and depth to achieve the order of the sentence
Sentence selection: Consider how sentences in the abstract are selected from the perspective of sentence importance, coherence between sentences, and redundancy.
Generation of abstracts: compresses selected sentences and integrates them into final results
General abstracts and query-based abstracts The difference between the two techniques is the "sort of sentences" link. In query-based abstracts, we calculate the query's characteristics so that the final digest reflects the relevance of the query.
Here are two examples of abstracts embodied in the search results. The left figure shows, enter the query "Why is the sky blue", and the system can pick out the relevant pages of this sentence, extract the summary and show it; the right example is the same reason.
Natural language processing application system
Here are three application systems for natural language processing: question answering, machine translation and dialogue systems.
Questions and Answers
When a user presents a question, the system can respond based on the knowledge map.
For example, when the user input in the search box, "Everest height", the page will appear on the Everest picture and its height description; users can also enter "Langya list", then the Baidu search page There will be a direct "Langya list" of the cast and photos.
In addition to the Q & A based on the knowledge map, we also designed a network-based depth question and answer system. The system analyzes the content of web search results and identifies keywords in user questions. Then the system will be analyzed from the page and the problem-related documents, from which the answers to questions extracted and displayed in the search results page at the top.
For example, the user can search for diabetes should eat what, then the system will answer "diet advice, food taboos" and so on. This information comes from the field of medical data, through information mining and matching, generate the answer presented in front of the user.
Today, machine translation based on neural networks is hot, but traditional machine translation methods are still valuable. Therefore, our system combines the old and new four methods:
Neural network machine translation (neural MT)
Rule-based machine translation (rule-based MT)
Example-based MT (example-based MT)
Statistical MT-based translation
May 2015, Baidu neural network machine translation technology will be applied to the Baidu online translation service, launched the world's first large-scale online translation system based on the depth of learning. In the same year, Baidu is still in the Baidu translation app on-line off-line translation feature that allows users to connect in the absence of the network can also use the translation service.
At present, Baidu translation can support the world's 28 languages, 756 translation between the translation, the number of daily translation of more than 100 million.
In addition to text translation, Baidu translates voice translations and uses OCR technology for picture content translation. So, after traveling to foreign countries do not have to worry about the language barrier of this problem. When you go to a restaurant for a meal, you can instantly translate it into the language you need by simply pressing your phone and taking a menu.
At the same time, we have more than 20,000 enterprises and developers to provide Baidu translation API, so that they enhance their product features, to provide users with better service.
When users enter a foreign language in the search box, Baidu search engine will automatically identify the translation needs and the translation results displayed in the top of the search results.
In the 2015 ACL meeting, Baidu's intelligent robot "small degree" also served as ACL Lifetime Achievement Award winner Professor Li Sheng simultaneous interpretation. In the Q & A session, a small degree of the audience asked the English question immediately translated into Chinese, and then translated into Chinese in English Professor Lee presented to the audience. The audience, most of whom are experts in natural language processing, was impressed by the performance of the small scale and was delighted with the current achievements in machine translation.
Next, I will introduce Baidu's dialogue system. The dialogue system can perform multi-turn interaction with the user. First of all, the user input through the natural language understanding (NLU) module, enter the dialogue management system. The system identifies the current dialogue state and determines the next step in the dialog action. Our dialog policy module contains a generic model and a domain model, the former dealing with generic interaction logic, and the latter dealing with domain-specific interaction logic. Finally, the system generates interactive replies for the user.
Here is an example, after the college entrance examination, Baidu intelligent assistant "degree secret" and the dialogue between users. When the user asks: "Which university can I enter?" "The secretaries asked him questions, to further understand the situation. Degree secret: "Are you liberal arts or science?" The other answer: "science." Degree secret then asked: "How many points did you test?" "He replied:" 620 points. "According to the degree of the Secretary of the information immediately, recommended for him to complete the volunteer schools. In 2016 the National College Entrance Examination period, the degree of secret handling 4.8 million 100 million users of the 30 million requests.
Next, I would like to talk about our intention map technology. Unlike the entity map I have talked about before, the nodes of the intent map represent one intent node. These "intent" relationships include requirements clarification (disambiguation), demand refinement (depth extension), demand for horizontal extension (breadth extension) and so on. In the example shown below, when "Alaska" means "Alaska", the associated intent is city, travel, and so on. When "Alaska" means "Alaskan Dogs", it extends the intent to pet dogs, pet dog care, and how to feed them.
This intention map can be used in man-machine dialogue system, let us look at a secret map based on the intention map user guide examples.
Users want to query on the "golden hair" information, based on the intention map, the degree of secret to provide users with general information on the golden hair; Then into the second round, the user clicks the "I want a golden" option, The secret will be able to guess the user will then want to know how to feed a golden hair, what kind of person is suitable for raising such dogs and other information, and these guide items to the user. Then the user clicks on the "easy to feed a golden hair" option. Dialogue to this round, the user's needs are basically met.
Above, I introduced Baidu in the field of NLP, including knowledge mapping, language understanding, language generation and several application systems (including Q & A, machine translation and dialogue), we have these technologies in Baidu products, We also support more products through the platform, such as our NLPC (NLP Cloud) platform, now has more than 20 kinds of NLP modules can be called more than 100 billion times a day.
Finally, I would like to say is that we are in the field of NLP in the exploration and pursuit, we will gradually realize the human dream of artificial intelligence have a crucial impact. thank you all.