Abstract:

3D dance is an indispensable form of virtual humans in the metaverse.It organically combines music and dance art, which greatly increases the interest in the metaverse.Previous work usually treats it as a simple sequence generation task, but it is difficult to match the dance movements with the music beat perfectly and the quality of long sequence dance generation is difficult to be guaranteed.Inspired by the process by which humans learn to dance, a novel 3D dance framework “Rhythm Dancer”to solve the above problems was proposed.The framework first uses VQ-VAE-2 to encode and quantify the dances in a hierarchical way, which effectively improves the quality of dance generation.Then, a key movement transition map was created using the core dance movements on the rhythm points, which not only ensures that the generated dance movements fit with the music beat, but also increases the diversity of dance movements.To ensure smooth and natural connections between the core dance moves, a poseinterpolation network was proposed to learn the transition movements between key moves.Extensive experiments demonstrate that the framework not only avoids the instability and uncontrollability problems of long sequence generation, but also achieves a higher match between dance movements and music rhythms, reaching state-of-the-art results.

Key words: 3D dance, metaverse, dance generation, deep learning