These two years, the high refresh rate screen in addition to the beginning of the popularity in the PC、 notebook end, also appeared as an important selling point on mobile phones, this year as the latest popular trend has become a new generation of hot spots, the market is constantly emerging screen refresh rate of 90 Hz、120Hz of new phones.
This screen, which accounts for the vast majority of the front of the phone, is a window to the hardware of the phone, providing a more fluid and realistic picture intuitively. The recently released Oppo Find X2Pro and one plus eight mobile phones, in addition to increasing the screen refresh rate to 120 Hz, also used a video clip technology called MEMC, which is why?
Unlike viewing real scenes, what we see through the screen is not
To give a simple example, the following graph clearly describes the difference between different frame rates: the leftmost square moving by 15 fps has a clear Carton sense, while the rightmost square moving by 60 fps is quite smooth. And what does this have to do with MEMC video frame insertion? are the videos we usually see low frame rate?
High refresh rate screen requires high frame rate content
Yeah, you got it right. Most of the online videos we usually see, or movies seen in cinemas and so on are presented in 24,25 or 30 fps, and a few content will appear in the form of 60 fps. What's the difference between them? Look carefully at the basketball in this picture below, do you think the right-most basketball is clear no matter how moving, and the left-most in the fall is blurred? By the way, this is the most intuitive benefit of high frame rate video, even if the main body in the video content is moving at high speed, it can let you see the details clearly.
Given the benefits of high-frame-rate video, why are our current videos still 24,25, or 30 fps? This is caused by historical reasons.
Think back to movies, tv shows, or home videos you've seen in a variety of places. They' re rarely presented in 60 fps, especially movies, which are basically all 24 fps. The film industry unified the frame rate of shooting and broadcasting into 24 in the 1930s fps, and then the television broadcasting system used 25 fps (PAL) or 30 fps (NTSC), which were made in the previous era, Although there are many producers and works with high frame rate in the film and television industry recently, it does not have much influence on the overall situation, and the film and television content with low frame rate will still occupy the absolute mainstream for a long time in the future.
The network video and short video production industry that follows the footsteps of the film and television industry is also following the experience of the former people, using the low frame rate production techniques. With the development of mobile Internet and smart devices, the contradiction between content and device hardware capability has gradually emerged: the refresh rate of mobile phone screen has gone to 90 Hz, or even 120 Hz, and only a small part of the video content we see on a daily basis uses 60 fps, most of which still stay on the low frame rate benchmark. Isn't it a waste that we ca n' t use watching video?
The reason is that, as mentioned above, there will not be a lot of content creators switching to make high frame rate content in a short period of time, but gradually using a production technique based on high frame rate, which will be a rather long transition process. In this long transition period, we have to use some transitional means, for example, we make the content of high frame rate, how to make it?
At this point, we have to ask the TV and PC have a more mature application of the plug-in technology, it can be low-frame content to complement it
Brief principle of frame insertion technology
After all that, we'll simply talk about what the interpolation technique is and how it works. Insert frame means insert new frame between original coherent two frame image, this frame can be duplicate frame of previous frame, can be black frame of all black, also can be middle frame between two frames by calculation.
Plugging repeat frame and inserting black frame are both relatively simple interpolation techniques, both can solve the problem of insufficient video frame rate, but the effect is general, in which the former is more common, some video players on the PC will automatically insert copy frame when playing video with less than 60 Hz frame rate to achieve synchronization. The insertion of a black frame is a better performance in reducing drag and improving the dynamic picture, but it will make the monitor/TV sacrifice a certain brightness.
And the last method of inserting frames
The logic of motion prediction and compensation algorithm is similar to that used in video coding. The common algorithm is to divide the picture into blocks of a certain size, then to find the moving blocks by comparing the differences between the two frames, and to analyze the motion trend of the image blocks, and then to obtain the intermediate frames by calculation.
We've got the technology
Before landing on the phone, binding technology has been used on some of the electronic devices around us, among which the better results and more common are the various television sets, such as Sony's introduction of Motionflow technology in its television set is the representative.
We can see from the schematic diagram that Motionflow technology combines the black frame with the middle frame to realize the characteristic of inserting the input source frame with the original frame rate of 60 fps to 240 fps. The application of frame insertion technology on TV sets effectively reduces drag and shadow, improves dynamic clarity and improves the experience of people watching movies. A lot of TV manufacturers have introduced similar Motionflow frame insertion technology, such as some domestic Internet TV brands to promote the technology as a major selling point.
On the PC end, frame insertion technology has already been used, even AMD directly correct such technology, graphics card hardware to provide relevant support, that is, Fluid Motion Video.. Because of the official support, it is convenient and effective, praised by many players, but the new generation of RDNA architecture has not yet supported this feature. There are also such interpolation software as Smooth Video Project (SVP), which can make full use of PC hardware computing power to insert frames for video. All in all, the purpose of inserting frames into video content is to improve the fluency of video.
High frame rate, star sea
If the high frame rate is the sea of stars, then frame insertion technology is the stormboat we rushed to this sea. Because the shift in the film and television industry is still very long, we need to insert frames in a short time
These new algorithms, supported by deep learning, have been widely used, from films up to a hundred years ago to new films in recent years, which can be well dealt with. For example, we have previously described the DAIN algorithms used to frame the train in station a hundred years ago to 60 fps, and its practical effect is quite excellent.
High frame rate has become the new sound of the times, whether it is high refresh rate screen or high frame rate content, are people in pursuit of new things. In this digital age of daily screen viewing, high frame rate applications directly and effectively enhance our sense of body when using the device, making the sight of the eyes more natural and reasonable.
As the hardware is integrated with the mobile phone, we can also enjoy the huge boost it brings to the visual experience. We can experience 60 or even 120 frames of video content every moment, which can be said to greatly reduce the threshold of high frame rate content and make that high refresh rate screen worth something.
There is no doubt that the technology of inserting frames into mobile phones is a good thing, because good technology needs to be widely promoted to be known to ordinary users in order to become truly excellent technology. If the previous binding technology is a niche plaything, it is expected to become a mass technology driven by mobile phone manufacturers, perceived by more people, at the visual level to provide a better experience, benefit from the daily use of mobile phones for entertainment. At the same time, the popularity of frame insertion technology can also allow more content creators to choose to make native high frame rate content to meet the new needs of users in the new era.