智能科学与技术学报 ›› 2022, Vol. 4 ›› Issue (3): 344-354.doi: 10.11959/j.issn.2096-6652.202246
郭超1, 鲁越1,2, 王晓1,3, 易达1, 王虓4, 王飞跃1,5
修回日期:
2022-08-08
出版日期:
2022-09-15
发布日期:
2022-09-01
作者简介:
郭超(1992- ),男,中国科学院自动化研究所复杂系统管理与控制国家重点实验室助理研究员,主要研究方向为机器学习、强化学习、机器艺术创作、图像生成、机器人绘画等基金资助:
Chao GUO1, Yue LU1,2, Xiao WANG1,3, Da YI1, Xiao WANG4, Fei-Yue WANG1,5
Revised:
2022-08-08
Online:
2022-09-15
Published:
2022-09-01
Supported by:
摘要:
随着人工智能探索领域的不断拓展,艺术创作成为人工智能发展和应用的重要研究热点。基于平行系统理论与 ACP 方法构建风格多样、内容逼真、笔触灵活和描述精准的平行艺术创作元宇宙,为提升人工智能的创造能力提供了一种可行的实现途径,并提供了应用案例。通过 AI 算法创作、人类筛选和评估、机器人执行,构建了人机物CPSS智能融合的平行创作架构,阐述了基于计算实验的绘画风格迁移、内容组合、笔触生成和图像描述等关键技术,并对所构建的平行创作系统进行了实验验证。平行创作系统融合了人、AI创作算法、机器人的优势,提升了人工智能艺术创作系统在虚拟和物理空间中的创作水平,促进了人机物协同艺术创作的发展。
中图分类号:
郭超, 鲁越, 王晓, 等. 人机物CPSS智能融合的平行创作架构与关键技术研究[J]. 智能科学与技术学报, 2022, 4(3): 344-354.
Chao GUO, Yue LU, Xiao WANG, et al. Architecture and key techniques of parallel creation through the fusion of human-cyber-physical intelligence in CPSS[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 344-354.
[1] | MCCARTHY J , MINSKY M , ROCHESTER N ,et al. A proposal for the Dartmouth summer research project on artificial intelligence,August 31,1955[J]. AI Mag, 2006,27: 12-14. |
[2] | BODEN M A . Creativity and artificial intelligence[J]. Artificial Intelligence, 1998,103(1/2): 347-356. |
[3] | SILVER D , HUANG A , MADDISON C J ,et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016,529(7587): 484-489. |
[4] | WANG F Y , ZHANG J J , ZHENG X H ,et al. Where does AlphaGo go:from church-Turing thesis to AlphaGo thesis and beyond[J]. IEEE/CAA Journal of Automatica Sinica, 2016,3(2): 113-120. |
[5] | 郭超, 鲁越, 林懿伦 ,等. 平行艺术:人机协作的艺术创作[J]. 智能科学与技术学报, 2019,1(4): 335-341. |
GUO C , LU Y , LIN Y L ,et al. Parallel art:artistic creation under human-machine collaboration[J]. Chinese Journal of Intelligent Science and Technology, 2019,1(4): 335-341. | |
[6] | RAMESH A , DHARIWAL P , NICHOL A ,et al. Hierarchical text-conditional image generation with CLIP latents[J]. arXiv preprint,2022,arXiv:2204.06125. |
[7] | GONTHIER N , GOUSSEAU Y , LADJAL S ,et al. Weakly supervised object detection in artworks[M]// Lecture notes in computer science. Cham: Springer International Publishing, 2019: 692-709. |
[8] | RODRIGUEZ C S , LECH M , PIROGOVA E . Classification of style in fine-art paintings using transfer learning and weighted image patches[C]// Proceedings of 2018 12th International Conference on Signal Processing and Communication Systems. Piscataway:IEEE Press, 2018: 1-7. |
[9] | CETINIC E , LIPIC T , GRGIC S . A deep learning perspective on beauty,sentiment,and remembrance of art[J]. IEEE Access, 2019,7: 73694-73710. |
[10] | DA YI , GUO C , BAI T X . Exploring painting synthesis with diffusion models[C]// Proceedings of 2021 IEEE 1st International Conference on Digital Twins and Parallel Intelligence. Piscataway:IEEE Press, 2021: 332-335. |
[11] | CETINIC E , SHE J . Understanding and creating art with AI:review and outlook[J]. ACM Transactions on Multimedia Computing,Communications,and Applications, 2022,18(2): 1-22. |
[12] | WANG F Y . Shadow systems:a new concept for nested and embedded co-simulation for intelligent systems[R]. 1994. |
[13] | 王飞跃 . 关于复杂系统研究的计算理论与方法[J]. 中国基础科学, 2004,6(5): 3-10. |
WANG F Y . Computational theory and method on complex system[J]. China Basic Science, 2004,6(5): 3-10. | |
[14] | 王飞跃 . 人工社会、计算实验、平行系统:关于复杂社会经济系统计算研究的讨论[J]. 复杂系统与复杂性科学, 2004,1(4): 25-35. |
WANG F Y . Artificial societies,computational experiments,and parallel systems:a discussion on computational theory of complex social-economic systems[J]. Complex Systems and Complexity Science, 2004,1(4): 25-35. | |
[15] | 王飞跃 . 平行系统方法与复杂系统的管理和控制[J]. 控制与决策, 2004,19(5): 485-489,514. |
WANG F Y . Parallel system methods for management and control of complex systems[J]. Control and Decision, 2004,19(5): 485-489,514. | |
[16] | 王飞跃, 史帝夫·兰森 . 从人工生命到人工社会:复杂社会系统研究的现状和展望[J]. 复杂系统与复杂性科学, 2004,1(1): 33-41. |
WANG F Y , LANSING J S . From artificial life to artificial societies—new methods for studies of complex social systems[J]. Complex Systems and Complexity Science, 2004,1(1): 33-41. | |
[17] | WANG F Y , WANG X , LI L X ,et al. Steps toward parallel intelligence[J]. IEEE/CAA Journal of Automatica Sinica, 2016,3(4): 345-348. |
[18] | 杨林瑶, 陈思远, 王晓 ,等. 数字孪生与平行系统:发展现状、对比及展望[J]. 自动化学报, 2019,45(11): 2001-2031. |
YANG L Y , CHEN S Y , WANG X ,et al. Digital twins and parallel systems:state of the art,comparisons and prospect[J]. Acta Automatica Sinica, 2019,45(11): 2001-2031. | |
[19] | GATYS L A , ECKER A S , BETHGE M . Image style transfer using convolutional neural networks[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 2414-2423. |
[20] | HERTZMANN A , . Painterly rendering with curved brush strokes of multiple sizes[C]// Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques.[S.l.:s.n.], 1998: 453-460. |
[21] | GOOCH B , COOMBE G , SHIRLEY P . Artistic vision:painterly rendering using computer vision techniques[C]// Proceedings of the 2nd International Symposium on Non-photorealistic Animation and Rendering.[S.l.:s.n.], 2002:83. |
[22] | HERTZMANN A , JACOBS C E , OLIVER N ,et al. Image analogies[C]// Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques.[S.l.:s.n.], 2001: 327-340. |
[23] | TOMASI C , MANDUCHI R . Bilateral filtering for gray and color images[C]// Proceedings of the 6th International Conference on Computer Vision. Piscataway:IEEE Press, 1998: 839-846. |
[24] | JOHNSON J , ALAHI A , LI F F . Perceptual losses for real-time style transfer and super-resolution[C]// Computer Vision - ECCV 2016.[S.l.:s.n.], 2016: 694-711. |
[25] | CHEN D W , YUAN L , LIAO J ,et al. Stylebank:an explicit representation for neural image style transfer[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 1897-1906. |
[26] | HUANG X , BELONGIE S . Arbitrary style transfer in real-time with adaptive instance normalization[C]// Proceedings of 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2017: 1510-1519. |
[27] | COLTON S , WIGGINS G A . Computational creativity:the final frontier?[C]// Proceedings of the 20th European Conference on Artificial Intelligence.[S.l.:s.n.], 2012: 21-26. |
[28] | ESSER P , ROMBACH R , OMMER B . Taming transformers for high-resolution image synthesis[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 12868-12878. |
[29] | RAMESH A , PAVLOV M , GOH G ,et al. Zero-shot text-to-image generation[J]. arXiv preprint,2021,arXiv:2102.12092. |
[30] | ZHENG Q Y , LI Z R , BARGTEIL A . Learning aesthetic layouts via visual guidance[J]. arXiv preprint,2021,arXiv:2107.06262. |
[31] | LI J N , YANG J M , HERTZMANN A ,et al. LayoutGAN:synthesizing graphic layouts with vector-wireframe adversarial networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021,43(7): 2388-2399. |
[32] | ZOU Z X , SHI T Y , QIU S ,et al. Stylized neural painting[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 15684-15693. |
[33] | HEGDE S , GATZIDIS C , TIAN F . Painterly rendering techniques:a state-of-the-art review of current approaches[J]. Computer Animation and Virtual Worlds, 2013,24(1): 43-64. |
[34] | HUANG Z W , ZHOU S C , HENG W . Learning to paint with model-based deep reinforcement learning[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2019: 8708-8717. |
[35] | VINYALS O , TOSHEV A , BENGIO S ,et al. Show and tell:a neural image caption generator[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2015: 3156-3164. |
[36] | HOSSAIN M Z , SOHEL F , SHIRATUDDIN M F ,et al. A comprehensive survey of deep learning for image captioning[J]. ACM Computing Surveys, 2019,51(6): 1-36. |
[37] | BAI S , AN S . A survey on automatic image caption generation[J]. Neurocomputing, 2018,311: 291-304. |
[38] | WANG W S , NA X X , CAO D P ,et al. Decision-making in driver-automation shared control:a review and perspectives[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(5): 1289-1307. |
[39] | LIU T , TIAN B , AI Y F ,et al. Parallel reinforcement learning-based energy efficiency improvement for a cyber-physical system[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(2): 617-626. |
[40] | WEI Q L , LI H Y , WANG F Y . Parallel control for continuous-time linear systems:a case study[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(4): 919-928. |
[41] | LI X S , LIU Y T , WANG K F ,et al. A recurrent attention and interaction model for pedestrian trajectory prediction[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(5): 1361-1370. |
[42] | LIU K H , YE Z H , GUO H Y ,et al. FISS GAN:a generative adversarial network for foggy image semantic segmentation[J]. IEEE/CAA Journal of Automatica Sinica, 2021,8(8): 1428-1439. |
[43] | LU J W , WEI Q L , WANG F Y . Parallel control for optimal tracking via adaptive dynamic programming[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(6): 1662-1674. |
[44] | SUN C , VIANNEY J M U , LI Y ,et al. Proximity based automatic data annotation for autonomous driving[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(2): 395-404. |
[45] | WANG S Y , HOUSDEN J , BAI T X ,et al. Robotic intra-operative ultrasound:virtual environments and parallel systems[J]. IEEE/CAA Journal of Automatica Sinica, 2021,8(5): 1095-1106. |
[46] | ZU C Y , YANG C , WANG J ,et al. Simulation and field testing of multiple vehicles collision avoidance algorithms[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(4): 1045-1063. |
[47] | TAN J Y , XU C L , LI L ,et al. Guidance control for parallel parking tasks[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(1): 301-306. |
[48] | 张俊, 许沛东, 王飞跃 . 平行系统和数字孪生的一种数据驱动形式表示及计算框架[J]. 自动化学报, 2020,46(7): 1346-1356. |
ZHANG J , XU P D , WANG F Y . Parallel systems and digital twins:a data-driven mathematical representation and computational framework[J]. Acta Automatica Sinica, 2020,46(7): 1346-1356. | |
[49] | JAYNES C , SEALES W B , CALVERT K ,et al. The metaverse:a networked collection of inexpensive,self-configuring,immersive environments[C]// Proceedings of the Workshop on Virtual Environments 2003 - EGVE’03. New York:ACM Press, 2003: 115-124. |
[50] | DUAN H H , LI J Y , FAN S Z ,et al. Metaverse for social good:a university campus prototype[C]// Proceedings of the 29th ACM International Conference on Multimedia. New York:ACM Press, 2021: 153-161. |
[51] | WANG F Y . Parallel intelligence in metaverses:welcome to Hanoi![J]. IEEE Intelligent Systems, 2022,37(1): 16-20. |
[52] | GUO C , BAI T X , LU Y ,et al. Skywork-daVinci:a novel CPSS-based painting support system[C]// Proceedings of 2020 IEEE 16th International Conference on Automation Science and Engineering. Piscataway:IEEE Press, 2020: 673-678. |
[53] | GUO C , BAI T X , WANG X ,et al. ShadowPainter:active learning enabled robotic painting through visual measurement and reproduction of the artistic creation process[J]. Journal of Intelligent & Robotic Systems, 2022,105(3): 1-17. |
[54] | KOTOVENKO D , WRIGHT M , HEIMBRECHT A ,et al. Rethinking style transfer:from pixels to parameterized brushstrokes[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 12191-12200. |
[55] | CORNIA M , STEFANINI M , BARALDI L ,et al. Meshed-memory transformer for image captioning[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2020: 10575-10584. |
[56] | LU Y , GUO C , DAI X Y ,et al. Image captioning on fine art paintings via virtual paintings[C]// Proceedings of 2021 IEEE 1st International Conference on Digital Twins and Parallel Intelligence. Piscataway:IEEE Press, 2021: 156-159. |
[57] | LU Y , GUO C , DAI X Y ,et al. Data-efficient image captioning of fine art paintings via virtual-real semantic alignment training[J]. Neurocomputing, 2022,490: 163-180. |
[1] | 缪青海, 吕宜生. 元宇宙下的平行交通系统[J]. 智能科学与技术学报, 2023, 5(1): 32-40. |
[2] | 康孟珍, 孙贺全, 王秀娟, 王飞跃. 系统农业:结合农业社会经济属性的建模和控制[J]. 智能科学与技术学报, 2023, 5(1): 41-50. |
[3] | 王晓, 杨林瑶, 胡斌, 侯家琛. 平行推理:一种基于ACP方法的虚实互动的知识协同框架[J]. 智能科学与技术学报, 2023, 5(1): 69-82. |
[4] | 田永林, 陈苑文, 杨静, 王雨桐, 王晓, 缪青海, 王子然, 王飞跃. 元宇宙与平行系统:发展现状、对比及展望[J]. 智能科学与技术学报, 2023, 5(1): 121-132. |
[5] | 康孟珍, 邱文忠, 陈自富, 王猛, 许沙沙, 王秀娟, 倪爱东, 蒋玉洁, 陈世超, DEREFFYE Philippe, 王飞跃. 平行圆明园:从数字孪生园林到元宇宙智慧遗址公园[J]. 智能科学与技术学报, 2022, 4(3): 301-307. |
[6] | 武强, 季雪庭, 吕琳媛. 元宇宙中的人工智能技术与应用[J]. 智能科学与技术学报, 2022, 4(3): 324-334. |
[7] | 吕秋云, 程绍鹏, 杨满智, 陈晓光, 王震. 面向元宇宙的数字公民身份认证方案[J]. 智能科学与技术学报, 2022, 4(3): 396-409. |
[8] | 李小双, 王晓, 杨林瑶, 田永林, 王雨桐, 张俊, 王飞跃. 元电网MetaGrid:基于平行电网的新一代智能电网的体系与架构[J]. 智能科学与技术学报, 2021, 3(4): 387-398. |
[9] | 李亚玲, 杨林瑶, 葛俊, 覃缘琪, 王晓. 博弈5.0:基于平行系统和机器博弈的社会认知平行博弈[J]. 智能科学与技术学报, 2021, 3(4): 507-520. |
[10] | 王飞跃, 蒋怀光. 平行电池:智能生态化电池技术与服务体系的框架和流程[J]. 智能科学与技术学报, 2021, 3(4): 521-531. |
[11] | 王春法, 王飞跃, 鲁越, 李华飙, 郭超. 平行博物馆:新时代博物馆运营的智能管理与控制[J]. 智能科学与技术学报, 2021, 3(2): 125-136. |
[12] | 吴宇震, 张俊, 高天露, 孙玉健, 刘金旭. 平行港口:智慧绿色时代下港口工业智联网新形态与体系结构[J]. 智能科学与技术学报, 2021, 3(2): 218-227. |
[13] | 王飞跃, 孟祥冰, 杜思聪, 耿征. 平行光场:基本框架与流程[J]. 智能科学与技术学报, 2021, 3(1): 110-122. |
[14] | 王飞跃. 平行控制与数字孪生:经典控制理论的回顾与重铸[J]. 智能科学与技术学报, 2020, 2(3): 293-300. |
[15] | 李浥东,张俊,陶耀东,王伟,顾元祥,王飞跃. 平行安全:基于CPSS的生成式对抗安全智能系统[J]. 智能科学与技术学报, 2020, 2(2): 194-202. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||
|