Abstract: An approach of robust accelerometer-based hand gesture recognition based on self-learning sparse representation was proposed. This method operated directly on the original acceleration signals by sparse representation without feature extraction and used the class-specific dictionary learning for sparse modeling to reduce the computing cost and time of recognition. The proposed system can easily add a novel gesture category as well as remove existing ones. Experiments on real-world database of 18 hand gestures validate the availability of the proposed algorithm.