Home > News content

What is the real AR translation of a real translator? Actually spike Google

via:博客园     time:2018/4/27 22:31:38     readed:642

When it comes to AR applications, your first thought may be Pokémon Go, which made people pick up the elf in the previous two years.


However, today's AR applications are not as imaginative. One of the reasons is that the AR applications in the market are relatively single and mostly concentrated in the game and entertainment fields.

From Pok & eacute;mon Go to Apple's ARKit, Alipay AR red envelopes, etc., these applications produced a lot of short-lived AR net red, but high-frequency utility tool AR function is rarely seen.


Actually there is still a great deal of room for development in the application scenario of AR. Youdao recently launched the real-world AR translation function, which can be used free of charge in the Daoist Translation Officer App, and supports the offline translation of Chinese, English, Japanese and Korean languages. It is the first real AR translation in China. Prior to this, AR translation by other vendors was actually a photo translation.


The biggest difference between real-world AR translation and photo translation is that there is no need to take photos and then identify image content. Instead, the text content in the camera is translated in real time. This involves complex techniques such as dynamic tracking, neural network translation, OCR segment analysis, and AR rendering. .


In general, the most high-frequency applications for translation apps are overseas tourism and Haitao shopping. Can AR translation improve the translation efficiency of these scenes?


In the past, Google Translate also introduced the instant translation function of mobile phone cameras. AppSo starts from some common scenarios and sees how real-world AR translation in a real translator can be used.

Foreign media news translation

Travel abroad When you want to understand the local news, especially after encountering some unexpected events, the best way is of course to read the news of foreign media directly.


Next, we will test the BBC’s report on a serious traffic accident in Toronto on April 23, using real-life AR translation and Google photo translation. The following is the original report:


The translation results are as follows:


(Left translation of the real-time AR translation of the Daoist translation official, real-time translation of Google's photo to the right)

News translation is one of the best touchstones for translating App, because often complex long sentences appear. To translate, it is necessary to take into consideration multiple aspects such as word meaning, syntax, and context.

From the translation results, there is a real AR translation is satisfactory. Basically, the main information of this news can be understood directly from the translation result, and phrases such as "standoff with" can be accurately translated.


Google's understanding of the context is much worse, and the Chinese reading habits are more difficult to read. It is difficult to understand the content of the original directly through the translation results, and the layout of the translated text is also confusing.

Haitao Product Description

Whether it is overseas shopping or online shopping, it is very necessary to understand the relevant information of the product, such as the description of the ingredients, notes and return instructions.


AppSo got a bottle of Korean skin care product this time. Here again, it translates product descriptions using realistic real-life AR translations and Google photo-interpretation. The content is how this skin care product clears the pores of the pores with a deep cleansing foam.

Take a look at the final translation:


(Left translation of the real-time AR translation of the Daoist translation official, real-time translation of Google's photo to the right)

The real translator of the Daoist Translation Agency has once again won over Google, and can translate this long description accurately and fluently. It is also easier to understand. Google’s real-time translation results are a bit incomprehensible. It seems that just converting each word into Chinese does not have the semantics of the context of contact.


Why does this happen? This is also a secret weapon that has a way. It is OCR's group segment analysis technology. Through millions of corpus trainings, the machine can read people's way of speaking and segmentation, and combine contextual translation directly, without a word or word. After the translation, they gathered together brutally.

Station street sign

Walking in a foreign city, the translation of unfamiliar place names and street signs is also a very high frequency demand. AR translation is much more convenient than manual input in the past. Let's take a look at the actual experience.

subway station


Landmark building


Shop business information


From the test of several scenes, the response speed of the camera from text-aligning to translation into the target language is faster, and real-time translation is achieved. Moreover, the text background is completely preserved while replacing the original text with the target language, which is faster than using PS.

Translation of dishes

Although restaurants in some countries have started to offer Chinese menus for Chinese tourists, they often encounter confusing situations when dining abroad, and they can see how the real translators of the Daoist Translation Agency understand the menu.


It can be seen that the translation of dish names is also in place.

to sum up

App that can maturely apply AR technology to translation is currently rare in the market because the technical difficulty behind it is that only Youdao and Google can implement this technology. In addition to AR technology, there are very high requirements for artificial intelligence technologies such as off-line neural network translation (NMT), motion tracking (Tracking), and image recognition (OCR).


Finally, we will summarize the measured experience of real-life AR translation:

  • The real translator AR's real-time AR translation can handle complex translations of long sentences, especially in the overseas scenes such as product packaging, signposts, and menus. The translation accuracy is high.
  • Authentic AR translation obviously understands Chinese more than Google Translate. This is thanks to the know-how of segmented OCR. It can read paragraphs naturally, and it can naturally “speak people”. This test is good in the passage of articles. It's very obvious.
  • The translation speed can be translated in real time, and there is basically no visible delay in following the subject.
  • The visual interaction experience of the AR translation is good, and the fusion of the flat text and the background is good. Each translation is equivalent to an automatic PS.
  • It is easy to use, and it supports free offline translation between China, UK, Japan and South Korea. It does not matter if there is no network, and it also saves traffic.

According to the introduction, the Daoist Translation Officer will continue to enhance the real-life AR translation experience. For example, the full-screen AR translation will soon be opened, and the automatic recognition language will be updated.


Nowadays, ordinary people are increasingly demanding translations. In the Translator App, the requirements for translation of texts, voices, dialogues, and photographs can all be translated. If you have a high demand for overseas travel, business trips and Haitao shopping, this time, the real-time AR translation function launched by Youdao Translation Co., Ltd. may be a try.

China IT News APP

Download China IT News APP

Please rate this news

The average score will be displayed after you score.

Post comment

Do not see clearly? Click for a new code.

User comments