人机物CPSS智能融合的平行创作架构与关键技术研究

doi:10.11959/j.issn.2096-6652.202246

智能科学与技术学报 ›› 2022, Vol. 4 ›› Issue (3): 344-354.doi: 10.11959/j.issn.2096-6652.202246

人机物CPSS智能融合的平行创作架构与关键技术研究

郭超¹, 鲁越¹^,², 王晓¹^,³, 易达¹, 王虓⁴, 王飞跃¹^,⁵

¹ 中国科学院自动化研究所复杂系统管理与控制国家重点实验室，北京 100190
² 中国科学院大学人工智能学院，北京 100049
³ 青岛智能产业技术研究院平行智能创新中心，山东青岛 266114
⁴ 普渡大学, 美国西拉法叶 IN 47907-2050
⁵ 中国国家博物馆科学与艺术研究所，北京 100006

修回日期:2022-08-08 出版日期:2022-09-15 发布日期:2022-09-01
作者简介:郭超（1992- ），男，中国科学院自动化研究所复杂系统管理与控制国家重点实验室助理研究员，主要研究方向为机器学习、强化学习、机器艺术创作、图像生成、机器人绘画等
鲁越（1994- ），男，中国科学院自动化研究所复杂系统管理与控制国家重点实验室博士生，主要研究方向为机器学习、小样本学习、风格迁移等
王晓（1988- ），女，博士，中国科学院自动化研究所复杂系统管理与控制国家重点实验室副研究员，主要研究方向为群体行为的激发与汇聚激励、群体智能和社交网络挖掘与分析
易达（1996- ），男，中国科学院自动化研究所复杂系统管理与控制国家重点实验室硕士生，主要研究方向为机器学习、深度学习、绘画图像生成、艺术风格迁移等
王虓（1996- ），男，普渡大学计算机系博士生，主要研究方向为自监督学习、计算生物学和智能交通系统等
王飞跃（1961- ），男，中国科学院自动化研究所复杂系统管理与控制国家重点实验室研究员，国防科技大学军事计算实验与平行系统技术研究中心教授，中国科学院大学中国经济与社会安全研究中心主任，主要研究方向为智能系统和复杂系统建模、分析与控制
基金资助:
国家自然科学基金资助项目(61533019)

Architecture and key techniques of parallel creation through the fusion of human-cyber-physical intelligence in CPSS

Chao GUO¹, Yue LU¹^,², Xiao WANG¹^,³, Da YI¹, Xiao WANG⁴, Fei-Yue WANG¹^,⁵

¹ The State Key Laboratory for Management and Control of Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
² School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China
³ Parallel Intelligence Innovation Center, Qingdao Academy of Intelligent Industries, Qingdao 266114, China
⁴ Purdue University, West Lafayette IN 47907-2050, USA
⁵ Institute of Science and Art, National Museum of China, Beijing 100006, China

Revised:2022-08-08 Online:2022-09-15 Published:2022-09-01
Supported by:
The National Natural Science Foundation of China(61533019)

摘要/Abstract

摘要：

随着人工智能探索领域的不断拓展，艺术创作成为人工智能发展和应用的重要研究热点。基于平行系统理论与 ACP 方法构建风格多样、内容逼真、笔触灵活和描述精准的平行艺术创作元宇宙，为提升人工智能的创造能力提供了一种可行的实现途径，并提供了应用案例。通过 AI 算法创作、人类筛选和评估、机器人执行，构建了人机物CPSS智能融合的平行创作架构，阐述了基于计算实验的绘画风格迁移、内容组合、笔触生成和图像描述等关键技术，并对所构建的平行创作系统进行了实验验证。平行创作系统融合了人、AI创作算法、机器人的优势，提升了人工智能艺术创作系统在虚拟和物理空间中的创作水平，促进了人机物协同艺术创作的发展。

关键词: 平行系统, ACP, 艺术创作, 元宇宙, 人机物智能

Abstract:

With the expansion of the fields covered by AI, artistic creation will become the next hot spot for AI research and applications.Building a metaverse with diverse styles, realistic contents, flexible strokes and accurate descriptions based on parallel system theory and ACP approach will provide a feasible way to improve AI creation capability.The intelligence of human, AI, and robots were fused to develop a parallel creation architecture through the creation by AI, the evaluation by humans, and the execution by robots.The parallel creation with the key methods of style transfer, content combination, stroke generation and image captioning in computational experiments were explained.The parallel creation system was validated through the painting experiments.The parallel creation system will improve the creation capability of artificial intelligence in cyberspace and physical space, and promote the human-cyber-physical collaborative creation through the fusion of them.

Key words: parallel system, ACP, artistic creation, metaverse, human-cyber-physical intelligence

中图分类号:

TP391

郭超, 鲁越, 王晓, 等. 人机物CPSS智能融合的平行创作架构与关键技术研究[J]. 智能科学与技术学报, 2022, 4(3): 344-354.

Chao GUO, Yue LU, Xiao WANG, et al. Architecture and key techniques of parallel creation through the fusion of human-cyber-physical intelligence in CPSS[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 344-354.

图/表 7

图1

图2

图3

图4

图5

图6

图7

参考文献 57

[1]	MCCARTHY J , MINSKY M , ROCHESTER N ,et al. A proposal for the Dartmouth summer research project on artificial intelligence,August 31,1955[J]. AI Mag, 2006,27: 12-14.
[2]	BODEN M A . Creativity and artificial intelligence[J]. Artificial Intelligence, 1998,103(1/2): 347-356.
[3]	SILVER D , HUANG A , MADDISON C J ,et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016,529(7587): 484-489.
[4]	WANG F Y , ZHANG J J , ZHENG X H ,et al. Where does AlphaGo go:from church-Turing thesis to AlphaGo thesis and beyond[J]. IEEE/CAA Journal of Automatica Sinica, 2016,3(2): 113-120.
[5]	郭超, 鲁越, 林懿伦 ,等. 平行艺术:人机协作的艺术创作[J]. 智能科学与技术学报, 2019,1(4): 335-341.
	GUO C , LU Y , LIN Y L ,et al. Parallel art:artistic creation under human-machine collaboration[J]. Chinese Journal of Intelligent Science and Technology, 2019,1(4): 335-341.
[6]	RAMESH A , DHARIWAL P , NICHOL A ,et al. Hierarchical text-conditional image generation with CLIP latents[J]. arXiv preprint,2022,arXiv:2204.06125.
[7]	GONTHIER N , GOUSSEAU Y , LADJAL S ,et al. Weakly supervised object detection in artworks[M]// Lecture notes in computer science. Cham: Springer International Publishing, 2019: 692-709.
[8]	RODRIGUEZ C S , LECH M , PIROGOVA E . Classification of style in fine-art paintings using transfer learning and weighted image patches[C]// Proceedings of 2018 12th International Conference on Signal Processing and Communication Systems. Piscataway:IEEE Press, 2018: 1-7.
[9]	CETINIC E , LIPIC T , GRGIC S . A deep learning perspective on beauty,sentiment,and remembrance of art[J]. IEEE Access, 2019,7: 73694-73710.
[10]	DA YI , GUO C , BAI T X . Exploring painting synthesis with diffusion models[C]// Proceedings of 2021 IEEE 1st International Conference on Digital Twins and Parallel Intelligence. Piscataway:IEEE Press, 2021: 332-335.
[11]	CETINIC E , SHE J . Understanding and creating art with AI:review and outlook[J]. ACM Transactions on Multimedia Computing,Communications,and Applications, 2022,18(2): 1-22.
[12]	WANG F Y . Shadow systems:a new concept for nested and embedded co-simulation for intelligent systems[R]. 1994.
[13]	王飞跃 . 关于复杂系统研究的计算理论与方法[J]. 中国基础科学, 2004,6(5): 3-10.
	WANG F Y . Computational theory and method on complex system[J]. China Basic Science, 2004,6(5): 3-10.
[14]	王飞跃 . 人工社会、计算实验、平行系统:关于复杂社会经济系统计算研究的讨论[J]. 复杂系统与复杂性科学, 2004,1(4): 25-35.
	WANG F Y . Artificial societies,computational experiments,and parallel systems:a discussion on computational theory of complex social-economic systems[J]. Complex Systems and Complexity Science, 2004,1(4): 25-35.
[15]	王飞跃 . 平行系统方法与复杂系统的管理和控制[J]. 控制与决策, 2004,19(5): 485-489,514.
	WANG F Y . Parallel system methods for management and control of complex systems[J]. Control and Decision, 2004,19(5): 485-489,514.
[16]	王飞跃, 史帝夫·兰森 . 从人工生命到人工社会:复杂社会系统研究的现状和展望[J]. 复杂系统与复杂性科学, 2004,1(1): 33-41.
	WANG F Y , LANSING J S . From artificial life to artificial societies—new methods for studies of complex social systems[J]. Complex Systems and Complexity Science, 2004,1(1): 33-41.
[17]	WANG F Y , WANG X , LI L X ,et al. Steps toward parallel intelligence[J]. IEEE/CAA Journal of Automatica Sinica, 2016,3(4): 345-348.
[18]	杨林瑶, 陈思远, 王晓 ,等. 数字孪生与平行系统:发展现状、对比及展望[J]. 自动化学报, 2019,45(11): 2001-2031.
	YANG L Y , CHEN S Y , WANG X ,et al. Digital twins and parallel systems:state of the art,comparisons and prospect[J]. Acta Automatica Sinica, 2019,45(11): 2001-2031.
[19]	GATYS L A , ECKER A S , BETHGE M . Image style transfer using convolutional neural networks[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 2414-2423.
[20]	HERTZMANN A , . Painterly rendering with curved brush strokes of multiple sizes[C]// Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques.[S.l.:s.n.], 1998: 453-460.
[21]	GOOCH B , COOMBE G , SHIRLEY P . Artistic vision:painterly rendering using computer vision techniques[C]// Proceedings of the 2nd International Symposium on Non-photorealistic Animation and Rendering.[S.l.:s.n.], 2002:83.
[22]	HERTZMANN A , JACOBS C E , OLIVER N ,et al. Image analogies[C]// Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques.[S.l.:s.n.], 2001: 327-340.
[23]	TOMASI C , MANDUCHI R . Bilateral filtering for gray and color images[C]// Proceedings of the 6th International Conference on Computer Vision. Piscataway:IEEE Press, 1998: 839-846.
[24]	JOHNSON J , ALAHI A , LI F F . Perceptual losses for real-time style transfer and super-resolution[C]// Computer Vision - ECCV 2016.[S.l.:s.n.], 2016: 694-711.
[25]	CHEN D W , YUAN L , LIAO J ,et al. Stylebank:an explicit representation for neural image style transfer[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 1897-1906.
[26]	HUANG X , BELONGIE S . Arbitrary style transfer in real-time with adaptive instance normalization[C]// Proceedings of 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2017: 1510-1519.
[27]	COLTON S , WIGGINS G A . Computational creativity:the final frontier?[C]// Proceedings of the 20th European Conference on Artificial Intelligence.[S.l.:s.n.], 2012: 21-26.
[28]	ESSER P , ROMBACH R , OMMER B . Taming transformers for high-resolution image synthesis[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 12868-12878.
[29]	RAMESH A , PAVLOV M , GOH G ,et al. Zero-shot text-to-image generation[J]. arXiv preprint,2021,arXiv:2102.12092.
[30]	ZHENG Q Y , LI Z R , BARGTEIL A . Learning aesthetic layouts via visual guidance[J]. arXiv preprint,2021,arXiv:2107.06262.
[31]	LI J N , YANG J M , HERTZMANN A ,et al. LayoutGAN:synthesizing graphic layouts with vector-wireframe adversarial networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021,43(7): 2388-2399.
[32]	ZOU Z X , SHI T Y , QIU S ,et al. Stylized neural painting[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 15684-15693.
[33]	HEGDE S , GATZIDIS C , TIAN F . Painterly rendering techniques:a state-of-the-art review of current approaches[J]. Computer Animation and Virtual Worlds, 2013,24(1): 43-64.
[34]	HUANG Z W , ZHOU S C , HENG W . Learning to paint with model-based deep reinforcement learning[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2019: 8708-8717.
[35]	VINYALS O , TOSHEV A , BENGIO S ,et al. Show and tell:a neural image caption generator[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2015: 3156-3164.
[36]	HOSSAIN M Z , SOHEL F , SHIRATUDDIN M F ,et al. A comprehensive survey of deep learning for image captioning[J]. ACM Computing Surveys, 2019,51(6): 1-36.
[37]	BAI S , AN S . A survey on automatic image caption generation[J]. Neurocomputing, 2018,311: 291-304.
[38]	WANG W S , NA X X , CAO D P ,et al. Decision-making in driver-automation shared control:a review and perspectives[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(5): 1289-1307.
[39]	LIU T , TIAN B , AI Y F ,et al. Parallel reinforcement learning-based energy efficiency improvement for a cyber-physical system[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(2): 617-626.
[40]	WEI Q L , LI H Y , WANG F Y . Parallel control for continuous-time linear systems:a case study[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(4): 919-928.
[41]	LI X S , LIU Y T , WANG K F ,et al. A recurrent attention and interaction model for pedestrian trajectory prediction[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(5): 1361-1370.
[42]	LIU K H , YE Z H , GUO H Y ,et al. FISS GAN:a generative adversarial network for foggy image semantic segmentation[J]. IEEE/CAA Journal of Automatica Sinica, 2021,8(8): 1428-1439.
[43]	LU J W , WEI Q L , WANG F Y . Parallel control for optimal tracking via adaptive dynamic programming[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(6): 1662-1674.
[44]	SUN C , VIANNEY J M U , LI Y ,et al. Proximity based automatic data annotation for autonomous driving[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(2): 395-404.
[45]	WANG S Y , HOUSDEN J , BAI T X ,et al. Robotic intra-operative ultrasound:virtual environments and parallel systems[J]. IEEE/CAA Journal of Automatica Sinica, 2021,8(5): 1095-1106.
[46]	ZU C Y , YANG C , WANG J ,et al. Simulation and field testing of multiple vehicles collision avoidance algorithms[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(4): 1045-1063.
[47]	TAN J Y , XU C L , LI L ,et al. Guidance control for parallel parking tasks[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(1): 301-306.
[48]	张俊, 许沛东, 王飞跃 . 平行系统和数字孪生的一种数据驱动形式表示及计算框架[J]. 自动化学报, 2020,46(7): 1346-1356.
	ZHANG J , XU P D , WANG F Y . Parallel systems and digital twins:a data-driven mathematical representation and computational framework[J]. Acta Automatica Sinica, 2020,46(7): 1346-1356.
[49]	JAYNES C , SEALES W B , CALVERT K ,et al. The metaverse:a networked collection of inexpensive,self-configuring,immersive environments[C]// Proceedings of the Workshop on Virtual Environments 2003 - EGVE’03. New York:ACM Press, 2003: 115-124.
[50]	DUAN H H , LI J Y , FAN S Z ,et al. Metaverse for social good:a university campus prototype[C]// Proceedings of the 29th ACM International Conference on Multimedia. New York:ACM Press, 2021: 153-161.
[51]	WANG F Y . Parallel intelligence in metaverses:welcome to Hanoi![J]. IEEE Intelligent Systems, 2022,37(1): 16-20.
[52]	GUO C , BAI T X , LU Y ,et al. Skywork-daVinci:a novel CPSS-based painting support system[C]// Proceedings of 2020 IEEE 16th International Conference on Automation Science and Engineering. Piscataway:IEEE Press, 2020: 673-678.
[53]	GUO C , BAI T X , WANG X ,et al. ShadowPainter:active learning enabled robotic painting through visual measurement and reproduction of the artistic creation process[J]. Journal of Intelligent ＆ Robotic Systems, 2022,105(3): 1-17.
[54]	KOTOVENKO D , WRIGHT M , HEIMBRECHT A ,et al. Rethinking style transfer:from pixels to parameterized brushstrokes[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 12191-12200.
[55]	CORNIA M , STEFANINI M , BARALDI L ,et al. Meshed-memory transformer for image captioning[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2020: 10575-10584.
[56]	LU Y , GUO C , DAI X Y ,et al. Image captioning on fine art paintings via virtual paintings[C]// Proceedings of 2021 IEEE 1st International Conference on Digital Twins and Parallel Intelligence. Piscataway:IEEE Press, 2021: 156-159.
[57]	LU Y , GUO C , DAI X Y ,et al. Data-efficient image captioning of fine art paintings via virtual-real semantic alignment training[J]. Neurocomputing, 2022,490: 163-180.

人机物CPSS智能融合的平行创作架构与关键技术研究

Architecture and key techniques of parallel creation through the fusion of human-cyber-physical intelligence in CPSS

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 57

相关文章 15

Metrics

推荐阅读 0

[1]	缪青海, 吕宜生. 元宇宙下的平行交通系统[J]. 智能科学与技术学报, 2023, 5(1): 32-40.
[2]	康孟珍, 孙贺全, 王秀娟, 王飞跃. 系统农业：结合农业社会经济属性的建模和控制[J]. 智能科学与技术学报, 2023, 5(1): 41-50.
[3]	王晓, 杨林瑶, 胡斌, 侯家琛. 平行推理：一种基于ACP方法的虚实互动的知识协同框架[J]. 智能科学与技术学报, 2023, 5(1): 69-82.
[4]	田永林, 陈苑文, 杨静, 王雨桐, 王晓, 缪青海, 王子然, 王飞跃. 元宇宙与平行系统：发展现状、对比及展望[J]. 智能科学与技术学报, 2023, 5(1): 121-132.
[5]	康孟珍, 邱文忠, 陈自富, 王猛, 许沙沙, 王秀娟, 倪爱东, 蒋玉洁, 陈世超, DEREFFYE Philippe, 王飞跃. 平行圆明园：从数字孪生园林到元宇宙智慧遗址公园[J]. 智能科学与技术学报, 2022, 4(3): 301-307.
[6]	武强, 季雪庭, 吕琳媛. 元宇宙中的人工智能技术与应用[J]. 智能科学与技术学报, 2022, 4(3): 324-334.
[7]	吕秋云, 程绍鹏, 杨满智, 陈晓光, 王震. 面向元宇宙的数字公民身份认证方案[J]. 智能科学与技术学报, 2022, 4(3): 396-409.
[8]	李小双, 王晓, 杨林瑶, 田永林, 王雨桐, 张俊, 王飞跃. 元电网MetaGrid：基于平行电网的新一代智能电网的体系与架构[J]. 智能科学与技术学报, 2021, 3(4): 387-398.
[9]	李亚玲, 杨林瑶, 葛俊, 覃缘琪, 王晓. 博弈5.0：基于平行系统和机器博弈的社会认知平行博弈[J]. 智能科学与技术学报, 2021, 3(4): 507-520.
[10]	王飞跃, 蒋怀光. 平行电池：智能生态化电池技术与服务体系的框架和流程[J]. 智能科学与技术学报, 2021, 3(4): 521-531.
[11]	王春法, 王飞跃, 鲁越, 李华飙, 郭超. 平行博物馆：新时代博物馆运营的智能管理与控制[J]. 智能科学与技术学报, 2021, 3(2): 125-136.
[12]	吴宇震, 张俊, 高天露, 孙玉健, 刘金旭. 平行港口：智慧绿色时代下港口工业智联网新形态与体系结构[J]. 智能科学与技术学报, 2021, 3(2): 218-227.
[13]	王飞跃, 孟祥冰, 杜思聪, 耿征. 平行光场：基本框架与流程[J]. 智能科学与技术学报, 2021, 3(1): 110-122.
[14]	王飞跃. 平行控制与数字孪生：经典控制理论的回顾与重铸[J]. 智能科学与技术学报, 2020, 2(3): 293-300.
[15]	李浥东,张俊,陶耀东,王伟,顾元祥,王飞跃. 平行安全：基于CPSS的生成式对抗安全智能系统[J]. 智能科学与技术学报, 2020, 2(2): 194-202.