It appears that gesture recognition will soon (finally) be coming to smartphones. An IC design startup in Taiwan called KaiKuTek is positioned to be one of the biggest beneficiaries in this new category of human-machine interface (HMI) technology. New gesture-recognition systems are based on 60 GHz radar, and while most vendors of 60 GHz radar ICs build their products using relatively expensive silicon germanium (SiG), KaiKuTek’s device is built using standard CMOS.

Google in a blog post at the end of July said it will support two new HMI features — facial recognition and motion sensing — in the upcoming Pixel 4 smartphone. The article pointed out that the new features in the Pixel 4 are the result of the company’s Project Soli, run by the company’s Advanced Technology and Projects (ATAP) team. The team first began touting its motion sensing technology in 2015.

In a 20-second YouTube video posted with the blog, Pixel 4's gesture control function was shown allowing a user to skip from one app to the next using hand gestures, without ever touching the phone. The blog emphasized that "these features are just the beginning. As Pixel gets better and better over time, Motion Sense will evolve."

After the long wait, from May 2015 to the second half of 2019, the market hopes that Google's Project Soli will finally be ready to add its new HMI capabilities, and spark a new market for chips enabling the features.

KaiKuTek (aka CoolTech), which was established in January 2017 and headquartered in Taipei, has developed a 3D gesture recognition solution based on 60GHz millimeter wave (mmWave) radar technology. Unlike some manufacturers in the market that produce 60GHz millimeter-wave radars on a high-cost SiGe process, the KaiKuTek’s gesture recognition solution is based on a more mature – and affordable – CMOS process.

The company’s single-chip SoC solution combines 1T3R antennas and millimeter-wave chips (MMICs). It also has a proprietary artificial intelligence (AI) accelerator integrated. The die is mounted in an advanced AiP (Antenna in Package) package.

The company claims its technology is capable of high-accuracy fine gesture recognition within 30 centimeters, while its AI accelerator draws less than 1 mW of power consumption. It will be officially put into mass production next year. KaiKuTek lists TSMC as a partner on its web site.

At present, about 30 employees of KaiKuTek are located in the "NanKang IC Design Incubation Center" (NKIC), which was set up by Taiwan’s Industrial Bureau of the Ministry of Economic Affairs and operated by the ITRI team. It has been in a low-key "stealth mode," but CEO Mike Wang said that as Google announces that Pixel 4 will be equipped with gesture control functions, major mobile phone manufacturers will also have to add the features in their next generation of products to remain competitive. The attention to the solution is bound to become higher and higher.

"It is time for more people to see our research and development results and value" Wang said. The company believes that in its two short years of existence, it has developed gesture recognition capabilities that at least match Google’s, if not exceed them.

Cooltech's strategic marketing director Griffon Lin added that Google's latest version of the mobile operating system Android Q (the beta trial version has been available for download) also included support for gesture control functions, which is likely to help accelerate the adoption of gesture recognition technology, feeding even more demand for the features. He also believes that a mobile phone will be the best tool for consumers to familiarize themselves with this new human-machine interface.

"It’s like when the smart phone was first introduced. People are more proficient in the use of touch interfaces because of games such as Angry Birds. When consumers become accustomed to using gestures on mobile phones, they can further extend this human-machine interface to other applications. Including cars, game consoles and smart homes, etc."

Why gesture?

It all begs the question: does anyone really need gesture recognition as an HMI?

At the moment, the touch interface has almost replaced all mechanical buttons, and voice control is commonly used for a variety of functions, including answer incoming calls, switching lights, playing music and searching for information on the Internet. Is there a clear benefit for gesture control?

"Gesture control is not a complete replacement for touch," said Wang. Using a touch interface to open various applications and answering calls on a mobile phone is an intuitive way to control those applications. However, using the touch interface during game play may obscure the touch screen. The screen affects the smoothness, and it is inconvenient to limit the screen size when inputting text in the instant messaging software.

Which brings up smart watches and other wearables that have screens significantly smaller than the screens on mobile phone. "In places where these touch interfaces cannot be fully utilized, that’s where gesture control has the advantage," Wang said.

As for voice control, although that already offers hands-free convenience, there are still many shortcomings in actual use. Lin pointed out that no matter where users are, there may be environmental noise interference affecting accuracy. Privacy can also be a big problem, not only when it comes to speaking commands in public, but also in instances when commands must be transmitted to the cloud for processing.

Wang added, "The characteristics of the 60 GHz millimeter wave technology are such that the signal does not go that far, but we have turned this shortcoming into an advantage. Since the radar detection is only at a close distance, and because a directional antenna is used, there is no problem of adjacent devices interfering with each other."

In the actual display, the solution can recognize subtle movements of the wrist, as well as finger movements. When it comes to responsiveness, there is almost no delay, because the recognition processes are not executed on the main processor of the mobile phone; that’s the domain of the AI processor.

"Our solution is not just edge computing, it can even be sensor-side computing;" This is also the main reason why its solution can greatly save mobile phone power, and Wang said: "We hope to put it on the mouse. All operational functions can be turned into gesture control."

Diversified market strategy

Considering the different needs of different applications for gesture manipulation, KaiKuTek will provide customized solutions it will develop with customers.

Wang expects that eventually the company will offer gesture training as a service in the future. For example, game developers might be interested in having the system recognize gestures specific to their games. A software developer might even want to support the ability of their customers to customize gestures.

In addition to preparing for the next generation of smart phones, KaiKuTek is also considering launching a dongle that allows mobile phones, laptops and other devices that do not have built-in millimeter-wave radar to support gesture recognition, and allow software developers to conceive a suite of development tools for more gesture recognition applications.

Wang emphasized that the development of gesture recognition is not easy, and requires expertise that won’t be easy for other companies to replicate any time soon. Wang, who went to the United States to study and work at a young age, has decades of experience in communication and wireless technology. He promoted the technology development process from the creation of the company, and is confident in the application prospect of 60GHz millimeter wave technology.

By: DocMemory
Copyright © 2023 CST, Inc. All Rights Reserved