1、 With the acceleration of smart phone 3D sensing penetration, TOF is expected to become the mainstream technology

1. 3D sensing penetration accelerates, leading mobile phone manufacturers join TOF camp

There are many routes of 3D sensing technology, different performance of different technologies, and different application fields. In the field of consumer electronics applications, there are two main 3D sensing technologies: 3D structure light and TOF (time of flight).

The commercial use of 3D structured light in the field of consumer electronics can be traced back to 2009. Microsoft and primesense, an Israeli 3D sensing company, jointly released the Kinect generation of body sensing equipment equipped with 3D structured light modules. After its launch in November 2010, the product became the fastest selling consumer electronic equipment in 2011. Despite the great success of the product, the accuracy, image resolution and response speed of the first generation Kinect are not ideal. Microsoft acquired 3Dv systems, an Israeli TOF camera company, and Canesta, a 3D gesture recognition company, in 2009 and 2010. In 2013, it terminated its cooperation with primesense and developed and launched the Kinect 2 generation product with TOF camera.

However, the good times are not long. Due to the lack of popular game applications, hardware loss sales and other problems, Microsoft said in October 2017 that it had stopped producing Kinect, and the cumulative sales volume since it went public in 2011 was only 35 million.

Despite Kinect's failure, the precipitation in the game market has made 3D sensing technology increasingly mature. In 2017, Apple released the iPhone x, which is equipped with 3D structured light module for the first time, enabling 3D face recognition technology, becoming the biggest innovation of apple in recent two years.

Previously, due to the limitations of semiconductor technology and other technologies, 3D sensing was difficult to be applied to mobile phones with very limited volume and low power consumption requirements. Therefore, the release of iPhone x is a major breakthrough in 3D structured light technology, and the market's enthusiasm for 3D structured light technology is rekindled. Apple's 3D structured light solution comes from primesense, which provides technical solutions for Microsoft's Kinect generation. In November 2013, Apple announced its acquisition of the company for $360 million.

Besides apple, major 3D structured light solution manufacturers include Intel, Qualcomm / HIMAX in the United States, mantis vision in Israel, Huawei and Obi Zhongguang in China.

In 2014, Intel released realsense, the world's first 3D depth of field camera embedded in a variety of intelligent devices, which uses 3D structured light technology and is used in Lenovo, Dell and other ultra-polar computers as well as UAVs and other devices. In the same year, Qualcomm announced that it cooperated with HIMAX, a vision IC design company, to provide a high-resolution, low-power 3D structured light module, slimtm.

After the release of iPhone x, Xiaomi, Huawei and oppo have also released the first smart phone equipped with 3D structured light module, among which Xiaomi adopts the solution of Mantis vision company in Israel, Huawei adopts the self-developed solution, and oppo adopts the solution of OBI Zhongguang company in China. Apple's new iPhone products in 2018 and 2019 are all equipped with 3D structured light modules.

At present, the smartphones with 3D structured light module have been released, including all models after Apple's iPhone x, Huawei's mate20 pro, glory Magic 2 and mate 30 pro, Xiaomi 8 exploration version and oppo findx. According to DIGITIMES, in 2018, there will be about 100 million smartphones equipped with 3D structured light, of which Apple accounts for about 88%.

The earliest commercial use of TOF can be traced back to July 2006. Mesa imaging company, derived from CSEM (Swiss electronic and Micro Technology Center), was founded and launched the commercial TOF camera product series swissranger, which was first applied to passive safety detection of automobiles. In 2014, mesa was acquired by heptagon, a micro optical device manufacturer in Singapore, and heptagon was acquired by AMS (AMIS semiconductor), a well-known sensor manufacturer in Austria, in 2016. It has certain advantages in the field of miniaturized TOF sensors.

In 2013, Microsoft adopted TOF technology in the second generation Kinect, and the scheme came from 3Dv systems, a TOF camera company acquired in 2010.

In 2015, Sony acquired Belgian gesture recognition technology company Softkinetic, which has a well-known depthsense TOF sensing system. Two years later, Sony released the world's smallest TOF module.

TOF technology was first applied to smart phones in 2016. Google and Lenovo jointly launched the world's first phab2 Pro smart phone equipped with TOF module, which adopts the TOF scheme of PMD / Infineon. The mobile phone can realize some simple AR applications such as 3D measurement, but it has not aroused great market reaction.

Infineon and PMD, a German 3D sensing company, have cooperated in TOF field for decades, and developed a famous real3 TOF sensor chip, in which PMD mainly provides TOF pixel matrix, Infineon mainly provides all functional components of SOC integration, and develops the corresponding manufacturing process, which is also used in the AR smart phone zenfone released by ASUS in 2017 Up.

▲ Infineon's real3 3D image sensor chip can realize a small TOF camera module, which is easy to integrate into a small electronic device (photo source: smart core)

On August 6, 2018, oppo held a TOF technical communication meeting in Beijing, and on August 23, it released its first mobile phone, oppo R17 pro, equipped with a TOF camera, adopting Sony's solution. Then in December 2018, vivo released its first dual screen version of vivo nex smart phone equipped with TOF camera, adopting Panasonic's solution; Huawei released its first smart phone equipped with TOF camera module, glory V20, using the same TOF scheme of oppo R17 pro.

After entering 2019, Android manufacturers have joined the TOF lens camp. In February 2019, Samsung released Galaxy S105G, with one TOF lens in front and one in back respectively; LG released LG G8 thinq, with rear TOF lens, with Infineon solution; Lenovo released Z6 Pro 5g mobile phone, with rear TOF lens. Huawei also installed a rear TOF lens on its mid tier Nova 5 pro in June.

At present, except Xiaomi, major Android phone manufacturers have released smartphones equipped with TOF module, among which Huawei and Samsung have released a relatively large number of models.

According to several media reports, such as Tencent Technologies, Collection Micronet and South Korean website TheElec, supply chain sources say Apple will carry TOF rear footage in its 2020 iPad Pro and two iPhones, while the front face recognition camera still follows the technology of 3D structured light. The report also said Apple or the use of custom CMOS to simulate human-eye features to achieve AR navigation and other applications, breaking through the current lack of TOF lens

We have assumed the penetration rate of TOF models of major brand mobile phone manufacturers in 2019 and 2020, and forecast that the global smartphone shipments carrying TOF modules in 2019 / 2020 will be 43 million and 150 million respectively. Considering that some high-end models such as Huawei and Samsung are equipped with front and rear TOF modules, it is predicted that the total number of TOF module combinations of global smart phones in 2019 / 2020 will be 57 million and 183 million.

2. 3D structured light vs TOF: why do mobile phone manufacturers choose TOF scheme?

Comparison of principle and system composition

The principle of 3D structured light scheme is to use infrared light source. The light emitted is projected on the object through certain coding. When these patterns are reflected back by the surface of the object, different deformation will occur with the different distance of the object. The image sensor will take pictures of the deformed patterns. Based on the triangle positioning method, the corresponding parallax can be obtained by calculating the deformation amount of each pixel in the photographed pattern, thus further obtaining the depth value.

The principle of TOF scheme is to use infrared light source to send high-frequency light pulse to the object, then receive the light pulse reflected back from the object, and calculate the distance between the measured object and the camera by detecting the flight (round-trip) time of the light pulse.

Compared with the 3D structured light module of iPhone and TOF module of oppo R17 pro, it can be seen that the composition and structure of the two are similar. The 3D structured light only has a dot matrix projector at the transmitter, but in fact, the floodlight used in the two schemes is quite different from the near-infrared camera. The most complex component in 3D structured light module is dot matrix projector, and the most complex component in TOF module is near infrared camera (TOF sensor).

The dot projector of 3D structured light module is composed of a high-power vcesel (vertical cavity surface emitting laser), a set of WLO lens (wafer level optics lens) and DOE (diffractive optical elements) for emitting specific encoded optical patterns. VCSEL: emit near-infrared light of specific wavelength (generally 880nm / 910nm / 940nm);

WLO lens:

The beam shaper includes beam shaper and projection lens, and the beam shaper includes beam expander and collimator. The function of the beam shaper is to change the output beam of VCSEL into a uniform collimating beam with a large cross-sectional area. The function of the beam expander is to expand the cross-sectional area of the laser so that it can cover the whole doe. The function of the collimating element is to readjust the laser after the beam expansion into a parallel beam. The projection lens is located behind the DOE, which is used to amplify the beam and make it reach a certain coverage;

DOE:

It refers to the optical element with step diffraction structure on the surface produced by photolithography, which is used to form a specific encoded optical pattern. It is the core part of the whole 3D structured light module, and the optical pattern is finally emitted through the projection lens.

Both the 3D structured light and the flood illuminator in TOF are composed of a VCSEL and a diffuser. The main difference between them is that the low-power VCSEL is used in the 3D structured light to supplement the light in the dark environment, so as to provide a complete depth map in the dark. The high-power VCSEL is used in TOF to emit light pulses to objects Need to be able to work day and night. The floodlights and TOF distance sensors in the iPhone X are packaged and supplied by STM. Both the 3D structured light and the near infrared camera in TOF are composed of an infrared CMOS sensor, a narrow band filter and a focus lens. The main difference between the two is that the performance of the infrared CMOS sensor is different.

VCSEL comparison:

There are great differences in the performance of VCSEL used in dot matrix projector, floodlight and floodlight in TOF. VCSEL of structured light needs to be made into specific patterns, which will have higher requirements on the consistency of pattern performance, high temperature drift of devices, heat performance, high temperature resistance of environment, etc., so as to test the design ability, process and product yield of suppliers. Only lumetum of the United States and Princeton Optronics acquired by AMS can achieve mass production in the world. The VCSEL output beam of floodlight in TOF does not need to be encoded, so the device is easier to manufacture and more VCSEL suppliers are available.

Diffuser:

Diffuser is a kind of DOE, also belongs to beam shaper, which is used to homogenize the input beam. By making a larger refraction angle with a larger diopter, the narrower beam can be extended to a wider angle range, and has a uniform illumination field. Diffuser in TOF is more difficult to design and manufacture than doe in 3D structured light spot array projector. There are only a few companies with advanced DOE design and manufacture in the world, mainly including CDA in Germany, silios in France and holoeye in Germany. Doe in iPhone x is designed by primesense. TSMC provides pattern micro nano processing and Caiyu provides ITO materials Materials, precision technology to provide device packaging. Diffuser has many suppliers, including Finisar (acquired by II-VI), PRC (acquired by viavi) and HIMAX.

Narrow band filter:

Since the emitter source VCSEL emits near-infrared light at a specific wavelength (850 nm/940 nm), narrow-band filters can emit ambient light beyond that wavelength

Comparison of near infrared cameras:

The 3D structured light's near-infrared camera requires less, its role is imaging, the iPhone X's near-infrared camera is provided by Italian semiconductor, using Soitec's Imager-SOI technology, with higher quantum efficiency and extremely low noise. The TOF's near-infrared camera requires much more light than the 3D structure, because the TOF emits high-frequency modulated pulses with a pulse frequency of up to 100 MHz, which makes the sensor's photosensitive time very short and nanosecond, so the single pixel size is much larger than that of the conventional camera, and the single pixel size of the TOF sensor is at least 10

With the continuous improvement of CMOS sensor technology by image sensor manufacturers, through the design of bis (backside illumination), CPAD (current assisted photon display) technology, and the synthesis of high-speed multi frame images into a single image to calculate the final depth, which reduces image noise and power consumption, making it possible to apply TOF to mobile phones, but the corresponding to The cost of f-sensor chip is also much higher.

Internet of things / VR / AR application drives TOF demand in 5g Era

Despite the many TOF handsets that have been introduced, the Mercure feature can be further optimized The depth of field,

In 5g era, the application of the Internet of animals is promoted, which brings the demand of all kinds of intelligent devices for three-dimensional perception ability. TOF with its cost advantage and excellent real-time perception ability will become the mainstream 3D sensing scheme. Especially in the field of VR / AR application, there is a high demand for time delay to avoid dizziness. TOF is the most suitable spatial positioning technology. In the field of automobile application, TOF can be used for gesture recognition, driver status monitoring in the vehicle, and object detection outside the vehicle, etc. In the field of industrial logistics application, TOF can be used to collect three-dimensional information of parcels, directly calculate the volume of different parcels, so as to reduce labor costs; in addition, TOF can also be used in obstacle avoidance systems, such as sweeping robots, and can be used for outdoor long-distance applications, such as logistics robots, guiding robots, etc., with its advantages of not sensitive to light conditions.

According to IHS Markit report, in 2018, the global TOF sensor market scale is 370 million US dollars, accounting for 33% of the whole 3D sensing market. In 2019, its market scale will increase by 35% year on year, reaching 500 million US dollars, accounting for about 40%. Based on the various advantages of TOF scheme, especially the cost advantage, it is estimated that the market size of TOF will reach 1.5 billion US dollars in 2022, accounting for about 50% of the whole 3D sensing market.

From the perspective of specific application fields, the main application market of TOF in the next three years will still come from the smartphone market. It is estimated that the corresponding market size will exceed 600 million US dollars in 2020, accounting for more than 90% of the whole market, followed by the tablet market, building detection, smart home, car central control, UAV and other application fields. In 2021, the tablet market will double. We think IHS is more conservative in the prediction of TOF market scale in 2020. According to our prediction in the first chapter, the global smartphone TOF module shipments in 2020 will be about 183 million, and the corresponding market scale will be about 1.8 billion US dollars based on the unit price of 10 US dollars.

2、 Domestic manufacturers have advantages in lens, filter and module

1. Panorama of industrial chain

The whole 3D sensing industry chain includes three links: 3D camera module (including 3D structured light and TOF), software algorithm and system overall solution provider. The 3D structured light and TOF module are basically the same, which are composed of illuminator, sensor array, optics and other components. According to yole's forecast, the market size of the whole 3D camera module will reach US $15.5 billion in 2023, including 14% of the light source market, 17% of the sensor market, 28% of the optical device market and 40% of the module market.

In terms of solution providers, 3D structured light manufacturers mainly include apple (purchase of prime sense), Intel (real sense products), AMS, Qualcomm / HIMAX, mantis vision, Huawei, Obi Zhongguang, etc. TOF camp manufacturers mainly include Microsoft, Sony, Panasonic, Infineon / PMD, ADI, AMS, St, Ti, melexis, espros, and domestic companies such as core microelectronics and torch intelligence.

AMS is the company with the most complete layout in the whole industry, with two solutions of 3D structured light and TOF, and basically the layout of the whole industry chain. The lattice projector in the iPhone x 3D structured light module is supplied by AMS, and the WLO lens (from heptagon, a subsidiary acquired in 2017) is also supplied by AMs. In the TOF scheme, the company has two series of floodlighting products, Tara and taraslim, which are respectively applicable to wide-angle gesture sensing and head tracking of home robots and automobiles, as well as face recognition in mobile phones.

(Note: for details of Infineon's TOF scheme and AMS's 3D structured light scheme, please refer to the previous article "3D sensing market is speeding up, who will be better at TOF and structured light?"? 》, for the TOF scheme of ADI, please refer to "the TOF 3D scheme of ADI has been widely used, and CMOS TOF sensor will be adopted later": http://www.icsmart.cn/33301/)

Sony is a leading manufacturer in the field of CIS (image sensor), so it has a first mover advantage in the field of TOF sensor. The company started to research and develop BSI (backlight) sensor technology in 2009, and combined with current assisted photon modulator (CAPD) developed by Softkinetic in 2015, it launched a new type of backlight TOF sensor with higher depth sensing performance and smaller volume Depthsense products. In 2017, the company launched the first back reflection TOF sensor imx456ql, which is only 1 / 2 inch (8mm) in size and has VGA resolution (300000 pixels), and the price is about 176 yuan. At present, most of TOF mobile phones in the market adopt Sony's TOF scheme. In addition, melexis, a well-known automotive semiconductor, signed a patent license agreement with Sony in 2015 and was allowed to apply Sony's depthsense TOF technology in its own products.

infineon is the world's leading semiconductor company, working with well-known tof manufacturer pmd to develop tof modules, in which pmd is mainly responsible for research and development of tof pixels and tof systems. infineon is mainly responsible for semiconductor process, product development and production. At present, the two sides have cooperated to launch a number of REAL3 series TOF image sensors, the latest version of the fourth generation REAL3 image sensor model is IRS 2771C, the chip area is only 4.6mm

STM is a well-known semiconductor company in the world. At present, it has launched three generations of TOF related products, vl6180, vl53l0x and vl53l1x. Since the launch of Apple's iPhone 7, the company has provided apple with a customized TOF distance sensor. In addition, in the iPhone x, the Italian French semiconductor also imports near-infrared camera image sensor and other components.

HUAWEI Mate 30 Pro adopts the Sony scheme. In addition, the module manufacturers of the two TOF cameras are mainly Ou Feiguang and sunny optics. Lens suppliers are Dali optics and sunny optics. Diffuser is supplied by American manufacturer Viavi and sunny optics in China. VCSEL is supplied by Lumentum and Zonghui.

2. Receiving end

sensor chip

The sensor chip of TOF receiver is still dominated by Sony. Sony and Samsung, as the leading companies of image sensor chip, have also accumulated profound experience in TOF market, and will become the main market leader in the future. At present, Howell technology, which is mainly acquired by Weil, has not made any obvious action in TOF field.

Optical lens

In the field of mobile lens, Taiwan's daliguang company is far ahead, occupying 38% of the global market share in 2017, followed by the domestic company sunny optics, occupying 17% of the market share. The lens suppliers of the two TOF cameras in the front and back of Huawei mate 30 Pro are mainly daliguang and sunny optics. We are optimistic about the impact of the TOF market on sunny optics' performance.

Narrow band filter

the thin film of the narrow band filter is generally composed of two kinds of films with low refractive index and high refractive index. the number of layers after superposition reaches tens of layers, and the parameter drift of each film may affect the final performance. and the transmittance of the narrow band filter is very sensitive to the loss of the film, so it is very difficult to prepare the filter with high peak transmittance, half bandwidth and very narrow. Currently the world's suppliers are only US company Vivi and domestic crystal Optoelectronics, crystalline Optoelectronics to Viavi to provide contract services. Viavi in the iPhone's 3D structured light module releases some of its orders to Crystal Optoelectronics later to reduce production costs, and the TOF module is expected to continue. In addition, on the android side, crystal optoelectronics is expected to be a direct supplier of narrow-band filters to samsung and Huawei, and gross margins will rise further.

3. Domestic listed companies

Domestic manufacturers have more obvious advantages in the receiver, including modules, narrow-band filters and lenses. We should be optimistic about the leading manufacturer of camera module of mobile phone, oufiguang, and the leading manufacturer of narrow-band filter, crystal optoelectronics. We suggest that we pay attention to the leading manufacturer of lens in China, Lianchuang electronics.

1. Ophiopogon

Ophiguang is the world's leading mobile camera module enterprise. The company's current business consists of two parts: one is the smart phone business, the main products include mobile camera module, touch display module and biometric module; the other is the smart car business, the main products include intelligent central control system, intelligent driving system (ADAS) and body electronics.

The company's revenue has maintained rapid growth in the past five years, and the mobile camera module is the company's main business, accounting for about 50% of the company's total revenue in 2017. In 2018, with the rise of in cell solutions, the company's external touch business has been greatly affected, which greatly hindered the performance of that year.

On November 19, 2019, the company announced that the touch display business related to non-A customers would be listed and developed independently in the form of equity transfer, and the optical business would be focused in the future. The company has leading advantages in technology and scale in the field of camera module, and is a module supplier of many TOF mobile phones that have been listed. Optimistic about the promotion of TOF module penetration to the company's performance.

2. Crystal photoelectricity

Crystal optoelectronics is a leading global filter manufacturer. Its main business includes imaging optics, new display, biometrics, reflective materials and sapphire substrate. Imaging optics mainly includes infrared filters and other products used in mobile camera, and new display mainly includes video glasses, ultra short focus projection, HUD (head up display) and micro imaging engine.

The company's operating income in 2018 was 2.326 billion yuan, up 8.39% year-on-year, a significant slowdown from previous years, mainly due to the steady growth of global consumer electronics, especially the mobile phone industry in 2018, and the decline in revenue from the sapphire substrate business as a result of overcapacity in the LED business. In the first three quarters of 2019, the company's operating income was 2.066 billion yuan, up 26.43% from the same period last year; its net profit was 360 million yuan, down 11.18% from the same period last year, mainly due to a decrease in non-recurrent profit and loss; and its net profit was 287 million yuan, up 14.32% from the same period last year.

As one of the only two narrowband filter suppliers in the world, in the future, with the increase of TOF penetration, the company will benefit from the release of some orders from viavi on the one hand, and the growth of demand from Android on the other hand. It will become the direct supplier of narrowband filters for Samsung and Huawei, and its gross margin will be further increased.

3. Lianchuang Electronics

The company is a leading manufacturer of optical lens, camera module and touch display products in China. Its products are applied in the field of consumer electronics such as smart phones, tablet computers, and other fields such as sports cameras, intelligent driving, smart home, VR / AR, etc. According to the official website information, the company has an annual production capacity of 100 million high-definition wide-angle lens and mobile phone lens, 60 million touch screens, 80 million display modules and 80 million touch display integrated films. In 2015-2017, the company's revenue and net profit maintained a high growth rate. In 2018, affected by external factors such as the decline of downstream consumer electronics demand, the company's revenue and net profit attributable to the parent decreased. In the first half of 2019, the company's optical business revenue increased by 88% year-on-year to 466 million yuan, driving the company's overall revenue back to the rising channel.

At present, the company's mobile lens customers are mainly Korean major customers. The collimation lens for 3D sensing has been mass produced and delivered to Huawei in small quantities. The company is the only supplier of the collimation lens of Huawei's 3D structured light module. The company said that it has the production capacity of TOF lens and module, and is optimistic about the promotion of TOF module penetration to the company's performance.