Big Data Research ›› 2023, Vol. 9 ›› Issue (1): 23-37.doi: 10.11959/j.issn.2096-0271.2023004
• TOPIC: METAVERSE AND BIG DATA • Previous Articles Next Articles
Yayun HE, Junqing PENG, Jianzong WANG, Jing XIAO
Online:
2023-01-15
Published:
2023-01-01
Supported by:
CLC Number:
Yayun HE, Junqing PENG, Jianzong WANG, Jing XIAO. Rhythm dancer: 3D dance generation by keymotion transition graph and pose-interpolation network[J]. Big Data Research, 2023, 9(1): 23-37.
"
方法 | 动作质量 | 动作多样性 | 节拍一致性得分↑ | 用户体验调查本模型胜出率 | |||
FIDk↓ | FIDg↓ | Distk↑ | Distg↑ | ||||
Li J M等人所提模型 | 86.43 | 20.58 | 6.85* | 4.93 | 0.161 | 100.00% | |
DanceNet | 69.18 | 17.76 | 2.86 | 2.72 | 0.143 | 95.10% | |
DanceRevolution | 73.42 | 31.01 | 3.52 | 2.46 | 0.195 | 87.20% | |
FACT | 35.35 | 12.40 | 5.94 | 5.30 | 0.221 | 94.30% | |
Bailando | 29.26 | 10.05 | 7.55 | 6.30 | 0.233 | 74.60% | |
节奏舞者 | 28.25 | 9.38 | 7.53 | 6.34 | 0.245 | — |
[1] | LI R L , YANG S , ROSS D A ,et al. AI choreographer:music conditioned 3D dance generation with AIST++[C]// Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2021. |
[2] | LI S Y , YU W J , GU T P ,et al. Bailando:3D dance generation by actor-critic GPT with choreographic memory[C]// Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2022: 11040-11049. |
[3] | AHN H , KIM J , KIM K ,et al. Generative autoregressive networks for 3D dancing move synthesis from music[J]. IEEE Robotics and Automation Letters, 2020,5(2): 3501-3508. |
[4] | ALEMI O , FRAN?OISE J , PASQUIER P . GrooveNet:real-time music-driven dance movement generation using artificial neural networks[C]// Proceedings of the 23rd ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York:ACM Press, 2017. |
[5] | GINOSAR S , BAR A , KOHAVI G ,et al. Learning individual styles of conversational gesture[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2019: 3492-3501. |
[6] | KAO H K , SU L . Temporally guided music-to-body-movement generation[C]// Proceedings of the 28th ACM International Conference on Multimedia. New York:ACM Press, 2020: 147-155. |
[7] | REN X C , LI H R , HUANG Z J ,et al. Self-supervised dance video synthesis conditioned on music[C]// Proceedings of the 28th ACM International Conference on Multimedia. New York:ACM Press, 2020: 46-54. |
[8] | RAZAVI A , OORD A V D , VINYALS O . Generating diverse high-fidelity images with VQ-VAE-2[J]. arXiv preprint, 2019,arXiv:1906.00446. |
[9] | KOVAR L , GLEICHER M , PIGHIN F . Motion graphs[C]// Proceedings of ACM SIGGRAPH 2008. New York:ACM Press, 2008: 1-10. |
[10] | LEE J , SHIN S Y . A hierarchical approach to interactive motion editing for humanlike figures[C]// Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques.[S.l.:s.n.], 1999: 39-48. |
[11] | PAPADOURAKIS A G. Motion capture and analysis:US20170348561A1[P]. 2017-12-07. |
[12] | HOLDEN D , SAITO J , KOMURA T . A deep learning framework for character motion synthesis and editing[J]. ACM Transactions on Graphics, 2016,35(4): 1-11. |
[13] | HOLDEN D , SAITO J , KOMURA T ,et al. Learning motion manifolds with convolutional autoencoders[C]// Proceedings of SIGGRAPH Asia 2015 Technical Briefs. New York:ACM Press, 2015: 1-4. |
[14] | HERNANDEZ A , GALL J , MORENO F . Human motion prediction via spatiotemporal inpainting[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2019: 7133-7142. |
[15] | LOPER M , MAHMOOD N , ROMERO J ,et al. Smpl[J]. ACM Transactions on Graphics, 2015,34(6): 1-16. |
[16] | OORD A V D , VINYALS O , Kavukcuoglu K . Neural discrete representation learning[J]. arXiv preprint, 2017,arXiv:1711.00937. |
[17] | LEE H Y , YANG X D , LIU M Y ,et al. Dancing to music[C]// Proceedings of the 33rd International Conference on Neural Information Processing Systems. New York:ACM Press, 2019: 3586-3596. |
[18] | LI B Y , ZHAO Y C , SHI Z L ,et al. DanceFormer:music conditioned 3D dance generation with parametric motion transformer[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022,36(2): 1272-1279. |
[19] | CHEN K , TAN Z P , LEI J ,et al. Choreomaster:choreography-oriented music-driven dance synthesis[J]. ACM Transactions on Graphics, 2021,40(4): 1-13. |
[20] | FERREIRA J P , COUTINHO T M , GOMES T L ,et al. Learning to dance:a graph convolutional adversarial network to generate realistic dance motions from audio[J]. Computers & Graphics, 2021,94(Feb.): 11-21. |
[21] | TANG T R , JIA J , MAO H Y . Dance with melody:an LSTM-autoencoder approach to music-oriented dance synthesis[C]// Proceedings of the 26th ACM international conference on Multimedia. New York:ACM Press, 2018: 1598-1606. |
[22] | WEST D B . Introduction to graph theory[M]// Discrete mathematics in statistical physics. Berlin: Springer, 2001. |
[23] | YAN W , ZHANG Y Z , ABBEEL P ,et al. VideoGPT:video generation using VQVAE and transformers[J]. arXiv preprint, 2021,arXiv:2104.10157. |
[24] | CHEN X L , HE K M . Exploring simple Siamese representation learning[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021. |
[25] | JIN Y H , ZHANG J K , LI M J ,et al. Towards the automatic anime characters creation with generative adversarial networks[J]. arXiv preprint, 2017,arXiv:1708.05509. |
[26] | VASWANI A , SHAZEER N , PARMAR N ,et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. New York:ACM Press, 2017: 6000-6010. |
[27] | ZHANG X J , XU Y , YANG S ,et al. Dance generation with style embedding:learning and transferring latent representations of dance styles[J]. arXiv preprint, 2021,arXiv:2104.14802. |
[28] | KINGMA D P , BA J . Adam:a method for stochastic optimization[C]// Proceedings of the 3rd International Conference for Learning Representations.[S.l.:s.n.], 2015. |
[29] | HEUSEL M , RAMSAUER H , UNTERTHINER T ,et al. GANs trained by a two time-scale update rule converge to a local Nash equilibrium[J]. arXiv preprint, 2017,arXiv:1706.08500. |
[30] | WON D G A J . fairmotion - tools to load,process and visualize motion capture data[Z]. 2020. |
[31] | MüLLER M , R?DER T , CLAUSEN M . Efficient content-based retrieval of motion capture data[C]// Proceedings of ACM SIGGRAPH 2005. New York:ACM Press, 2005: 677-685. |
[32] | LI J M , YIN Y H , CHU H ,et al. Learning to generate diverse dance motions with transformer[J]. arXiv preprint, 2020,arXiv:2008.08171. |
[33] | ZHUANG W L , WANG C Y , CHAI J X ,et al. Music2Dance:DanceNet for music-driven dance generation[J]. ACM Transactions on Multimedia Computing,Communications,and Applications, 2022,18(2): 1-21. |
[34] | HUANG Y H , ZHANG J J , LIU S Y ,et al. Genre-conditioned long-term 3D dance generation driven by music[C]// Proceedings of 2022 IEEE International Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE Press, 2022: 4858-4862. |
[1] | Hao WANG, Yushan PAN, Yi PAN. Generative AI empowered metaverse organisms: prospects and challenges [J]. Big Data Research, 2023, 9(3): 85-96. |
[2] | Yadong WU, Jiaming CHEN, Yan LUO, Xuefeng WANG, Dechun HUANG, Chao NI, Jiming LAN, Suiqun LI, Weihan ZHANG, Wei DAI. An overview of Caideng metaverse research [J]. Big Data Research, 2023, 9(3): 97-113. |
[3] | Yimin DENG, Xulong ZHANG, Shijing SI, Jianzong WANG, Jing XIAO. Human avatars synthesis technologies: a survey [J]. Big Data Research, 2023, 9(3): 114-139. |
[4] | Ye LIU, Wei CHENG, Yan LI, Yimeng YIN, Huijie SUN. Research on the construction of educational community from the perspective of metaverse [J]. Big Data Research, 2023, 9(1): 78-86. |
[5] | Yifei PENG, Zhen YUAN, Xulong ZHANG, Guilin JIANG, Yujiang LIU. Metaverse air pollutant concentration inference model based on digital twin technology [J]. Big Data Research, 2023, 9(1): 38-50. |
[6] | Zihang WANG, Xiangqun YU, Hongbiao SI, Simin FU, Xulong ZHANG, Shaoliang PENG. Design of the layered model of the metaverse based on computing network [J]. Big Data Research, 2023, 9(1): 51-62. |
[7] | Rui ZHU, Hongzhi WANG, Shuangshuang CUI, Kaixin ZHANG, Yu YAN. Cloud-edge-end collaborative big data management for metaverse [J]. Big Data Research, 2023, 9(1): 63-77. |
[8] | Bo HE. Research on the legal conundrums and regulation ideas of metaverse [J]. Big Data Research, 2023, 9(1): 87-102. |
[9] | Yang SHEN, Menglong YU. Metaverse and big data: data insight and value connection in spatio-temporal intelligence [J]. Big Data Research, 2023, 9(1): 103-110. |
[10] | Yumeng CUI, Jingya WANG, Shangyi YAN, Zhizhong TAO. Automatic key information extraction of police records based on deep learning [J]. Big Data Research, 2022, 8(6): 127-142. |
[11] | Zhitao ZHU, Shijing SI, Jianzong WANG, Jing XIAO. Survey on federated recommendation systems [J]. Big Data Research, 2022, 8(4): 105-132. |
[12] | Jie WANG, Songyan ZHANG, Jiye LIANG. A semi-supervised deep learning algorithm combining consistency regularization and manifold regularization [J]. Big Data Research, 2022, 8(3): 103-114. |
[13] | Kangting XU, Wei Song. A Chinese text sentiment analysis method combining language knowledge and deep learning [J]. Big Data Research, 2022, 8(3): 115-127. |
[14] | Chenhuizi WANG, Wei CAI. Digital economics in metaverse: state-of-the-art, characteristics, and vision [J]. Big Data Research, 2022, 8(3): 140-150. |
[15] | Zhitao ZHAO, Lijun ZHAO, Zheng ZHANG, Ping TANG. Integration of remote sensing intelligent processing algorithm using container cloud technology [J]. Big Data Research, 2022, 8(2): 58-74. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||
|