Facebook reality labs (FRL) is committed to many projects aimed at making the interaction between technology and people more approachable. FRL has previously studied technologies including VR helmets and AR glasses.Hand tracking technology is particularly important for virtual reality, it can make the technology more immersive, and strive to the most
In the beginning, FRL's work was to create more natural devices, such as touch controllers, which were designed to be comfortable to hold, with many sensors inside. The touch controller provided a lifelike sense of hand presence, making the basic VR interaction feel more like real objects.
On the other hand, FRL is trying to eliminate the need for hand controllers and instead sense the naked hand of humans. The lab began to study the hand tracking of VR more than five years ago, and no one has been able to develop a consumer quality, controller free hand tracking function. This means that the technology has to be created from scratch.
FRL is currently exploring new experimental text input forms, which is a common and critical task for productivity. FRL proposes a new method to realize touch typing on any plane without keyboard by using the hand movement of marker based hand tracking system. The system inputs the motion directly and decodes it into the text that the user intends to input.
The researchers point out that the technique is still in its early stages of research, but illustrates the potential of hand tracking in active scenes. Researchers must understand unstable typing patterns and use statistical decoding techniques in automatic speech recognition to predict keystrokes from hand movements using motion models. Using this method, typists only use their hands, a flat surface, and nothing else to type an average of 73 words per minute, with an uncorrected error rate of 2.4%. Compared with the actual keyboard, the new system achieves similar speed and accuracy.