Abstract:

Gesture-based human-machine interaction is becoming more and more important, which can provide users with a better experience in scenarios such as video games and virtual reality.In recent years, researchers have explored different sensing technologies to facilitate gesture recognition, including RF signal, acoustic signal, etc.Compared with these approaches, visible light-based gesture recognition is a more pervasive option.The basic principle is that different gestures will produce unique shadow patterns as they block the visible light, and gesture recognition can be achieved by capturing shadow changes through photoelectric sensors.To address the environment-dependent problem faced by existing solutions, a digit gesture recognition system was designed based on the photoelectric sensor array.In particular, by modeling recordings of the sensor array as images, the temporal and spatial correlation between different sensor recordings was discovered.An environment adaptive gesture recognition method was designed based on CNN-RNN by fusing the spatio-temporal features.To verify the effectiveness of the proposed method, a prototype gesture recognition system was designed, named Vi-Gesture.Experimental results show that the proposed method outperforms baselines by more than 10% in recognition accuracy.

Key words: visible light sensing, gesture recognition, environment adaptive, spatio-temporal feature, CNN-RNN