人机物CPSS智能融合的平行创作架构与关键技术研究

doi:10.11959/j.issn.2096-6652.202246

Abstract

Abstract:

With the expansion of the fields covered by AI, artistic creation will become the next hot spot for AI research and applications.Building a metaverse with diverse styles, realistic contents, flexible strokes and accurate descriptions based on parallel system theory and ACP approach will provide a feasible way to improve AI creation capability.The intelligence of human, AI, and robots were fused to develop a parallel creation architecture through the creation by AI, the evaluation by humans, and the execution by robots.The parallel creation with the key methods of style transfer, content combination, stroke generation and image captioning in computational experiments were explained.The parallel creation system was validated through the painting experiments.The parallel creation system will improve the creation capability of artificial intelligence in cyberspace and physical space, and promote the human-cyber-physical collaborative creation through the fusion of them.

Key words: parallel system, ACP, artistic creation, metaverse, human-cyber-physical intelligence

CLC Number:

TP391

Chao GUO, Yue LU, Xiao WANG, et al. Architecture and key techniques of parallel creation through the fusion of human-cyber-physical intelligence in CPSS[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 344-354.

Figures/Tables 7

References 57

[1]	MCCARTHY J , MINSKY M , ROCHESTER N ,et al. A proposal for the Dartmouth summer research project on artificial intelligence,August 31,1955[J]. AI Mag, 2006,27: 12-14.
[2]	BODEN M A . Creativity and artificial intelligence[J]. Artificial Intelligence, 1998,103(1/2): 347-356.
[3]	SILVER D , HUANG A , MADDISON C J ,et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016,529(7587): 484-489.
[4]	WANG F Y , ZHANG J J , ZHENG X H ,et al. Where does AlphaGo go:from church-Turing thesis to AlphaGo thesis and beyond[J]. IEEE/CAA Journal of Automatica Sinica, 2016,3(2): 113-120.
[5]	郭超, 鲁越, 林懿伦 ,等. 平行艺术:人机协作的艺术创作[J]. 智能科学与技术学报, 2019,1(4): 335-341.
	GUO C , LU Y , LIN Y L ,et al. Parallel art:artistic creation under human-machine collaboration[J]. Chinese Journal of Intelligent Science and Technology, 2019,1(4): 335-341.
[6]	RAMESH A , DHARIWAL P , NICHOL A ,et al. Hierarchical text-conditional image generation with CLIP latents[J]. arXiv preprint,2022,arXiv:2204.06125.
[7]	GONTHIER N , GOUSSEAU Y , LADJAL S ,et al. Weakly supervised object detection in artworks[M]// Lecture notes in computer science. Cham: Springer International Publishing, 2019: 692-709.
[8]	RODRIGUEZ C S , LECH M , PIROGOVA E . Classification of style in fine-art paintings using transfer learning and weighted image patches[C]// Proceedings of 2018 12th International Conference on Signal Processing and Communication Systems. Piscataway:IEEE Press, 2018: 1-7.
[9]	CETINIC E , LIPIC T , GRGIC S . A deep learning perspective on beauty,sentiment,and remembrance of art[J]. IEEE Access, 2019,7: 73694-73710.
[10]	DA YI , GUO C , BAI T X . Exploring painting synthesis with diffusion models[C]// Proceedings of 2021 IEEE 1st International Conference on Digital Twins and Parallel Intelligence. Piscataway:IEEE Press, 2021: 332-335.
[11]	CETINIC E , SHE J . Understanding and creating art with AI:review and outlook[J]. ACM Transactions on Multimedia Computing,Communications,and Applications, 2022,18(2): 1-22.
[12]	WANG F Y . Shadow systems:a new concept for nested and embedded co-simulation for intelligent systems[R]. 1994.
[13]	王飞跃 . 关于复杂系统研究的计算理论与方法[J]. 中国基础科学, 2004,6(5): 3-10.
	WANG F Y . Computational theory and method on complex system[J]. China Basic Science, 2004,6(5): 3-10.
[14]	王飞跃 . 人工社会、计算实验、平行系统:关于复杂社会经济系统计算研究的讨论[J]. 复杂系统与复杂性科学, 2004,1(4): 25-35.
	WANG F Y . Artificial societies,computational experiments,and parallel systems:a discussion on computational theory of complex social-economic systems[J]. Complex Systems and Complexity Science, 2004,1(4): 25-35.
[15]	王飞跃 . 平行系统方法与复杂系统的管理和控制[J]. 控制与决策, 2004,19(5): 485-489,514.
	WANG F Y . Parallel system methods for management and control of complex systems[J]. Control and Decision, 2004,19(5): 485-489,514.
[16]	王飞跃, 史帝夫·兰森 . 从人工生命到人工社会:复杂社会系统研究的现状和展望[J]. 复杂系统与复杂性科学, 2004,1(1): 33-41.
	WANG F Y , LANSING J S . From artificial life to artificial societies—new methods for studies of complex social systems[J]. Complex Systems and Complexity Science, 2004,1(1): 33-41.
[17]	WANG F Y , WANG X , LI L X ,et al. Steps toward parallel intelligence[J]. IEEE/CAA Journal of Automatica Sinica, 2016,3(4): 345-348.
[18]	杨林瑶, 陈思远, 王晓 ,等. 数字孪生与平行系统:发展现状、对比及展望[J]. 自动化学报, 2019,45(11): 2001-2031.
	YANG L Y , CHEN S Y , WANG X ,et al. Digital twins and parallel systems:state of the art,comparisons and prospect[J]. Acta Automatica Sinica, 2019,45(11): 2001-2031.
[19]	GATYS L A , ECKER A S , BETHGE M . Image style transfer using convolutional neural networks[C]// Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 2414-2423.
[20]	HERTZMANN A , . Painterly rendering with curved brush strokes of multiple sizes[C]// Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques.[S.l.:s.n.], 1998: 453-460.
[21]	GOOCH B , COOMBE G , SHIRLEY P . Artistic vision:painterly rendering using computer vision techniques[C]// Proceedings of the 2nd International Symposium on Non-photorealistic Animation and Rendering.[S.l.:s.n.], 2002:83.
[22]	HERTZMANN A , JACOBS C E , OLIVER N ,et al. Image analogies[C]// Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques.[S.l.:s.n.], 2001: 327-340.
[23]	TOMASI C , MANDUCHI R . Bilateral filtering for gray and color images[C]// Proceedings of the 6th International Conference on Computer Vision. Piscataway:IEEE Press, 1998: 839-846.
[24]	JOHNSON J , ALAHI A , LI F F . Perceptual losses for real-time style transfer and super-resolution[C]// Computer Vision - ECCV 2016.[S.l.:s.n.], 2016: 694-711.
[25]	CHEN D W , YUAN L , LIAO J ,et al. Stylebank:an explicit representation for neural image style transfer[C]// Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2017: 1897-1906.
[26]	HUANG X , BELONGIE S . Arbitrary style transfer in real-time with adaptive instance normalization[C]// Proceedings of 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE Press, 2017: 1510-1519.
[27]	COLTON S , WIGGINS G A . Computational creativity:the final frontier?[C]// Proceedings of the 20th European Conference on Artificial Intelligence.[S.l.:s.n.], 2012: 21-26.
[28]	ESSER P , ROMBACH R , OMMER B . Taming transformers for high-resolution image synthesis[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 12868-12878.
[29]	RAMESH A , PAVLOV M , GOH G ,et al. Zero-shot text-to-image generation[J]. arXiv preprint,2021,arXiv:2102.12092.
[30]	ZHENG Q Y , LI Z R , BARGTEIL A . Learning aesthetic layouts via visual guidance[J]. arXiv preprint,2021,arXiv:2107.06262.
[31]	LI J N , YANG J M , HERTZMANN A ,et al. LayoutGAN:synthesizing graphic layouts with vector-wireframe adversarial networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021,43(7): 2388-2399.
[32]	ZOU Z X , SHI T Y , QIU S ,et al. Stylized neural painting[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 15684-15693.
[33]	HEGDE S , GATZIDIS C , TIAN F . Painterly rendering techniques:a state-of-the-art review of current approaches[J]. Computer Animation and Virtual Worlds, 2013,24(1): 43-64.
[34]	HUANG Z W , ZHOU S C , HENG W . Learning to paint with model-based deep reinforcement learning[C]// Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE Press, 2019: 8708-8717.
[35]	VINYALS O , TOSHEV A , BENGIO S ,et al. Show and tell:a neural image caption generator[C]// Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2015: 3156-3164.
[36]	HOSSAIN M Z , SOHEL F , SHIRATUDDIN M F ,et al. A comprehensive survey of deep learning for image captioning[J]. ACM Computing Surveys, 2019,51(6): 1-36.
[37]	BAI S , AN S . A survey on automatic image caption generation[J]. Neurocomputing, 2018,311: 291-304.
[38]	WANG W S , NA X X , CAO D P ,et al. Decision-making in driver-automation shared control:a review and perspectives[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(5): 1289-1307.
[39]	LIU T , TIAN B , AI Y F ,et al. Parallel reinforcement learning-based energy efficiency improvement for a cyber-physical system[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(2): 617-626.
[40]	WEI Q L , LI H Y , WANG F Y . Parallel control for continuous-time linear systems:a case study[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(4): 919-928.
[41]	LI X S , LIU Y T , WANG K F ,et al. A recurrent attention and interaction model for pedestrian trajectory prediction[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(5): 1361-1370.
[42]	LIU K H , YE Z H , GUO H Y ,et al. FISS GAN:a generative adversarial network for foggy image semantic segmentation[J]. IEEE/CAA Journal of Automatica Sinica, 2021,8(8): 1428-1439.
[43]	LU J W , WEI Q L , WANG F Y . Parallel control for optimal tracking via adaptive dynamic programming[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(6): 1662-1674.
[44]	SUN C , VIANNEY J M U , LI Y ,et al. Proximity based automatic data annotation for autonomous driving[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(2): 395-404.
[45]	WANG S Y , HOUSDEN J , BAI T X ,et al. Robotic intra-operative ultrasound:virtual environments and parallel systems[J]. IEEE/CAA Journal of Automatica Sinica, 2021,8(5): 1095-1106.
[46]	ZU C Y , YANG C , WANG J ,et al. Simulation and field testing of multiple vehicles collision avoidance algorithms[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(4): 1045-1063.
[47]	TAN J Y , XU C L , LI L ,et al. Guidance control for parallel parking tasks[J]. IEEE/CAA Journal of Automatica Sinica, 2020,7(1): 301-306.
[48]	张俊, 许沛东, 王飞跃 . 平行系统和数字孪生的一种数据驱动形式表示及计算框架[J]. 自动化学报, 2020,46(7): 1346-1356.
	ZHANG J , XU P D , WANG F Y . Parallel systems and digital twins:a data-driven mathematical representation and computational framework[J]. Acta Automatica Sinica, 2020,46(7): 1346-1356.
[49]	JAYNES C , SEALES W B , CALVERT K ,et al. The metaverse:a networked collection of inexpensive,self-configuring,immersive environments[C]// Proceedings of the Workshop on Virtual Environments 2003 - EGVE’03. New York:ACM Press, 2003: 115-124.
[50]	DUAN H H , LI J Y , FAN S Z ,et al. Metaverse for social good:a university campus prototype[C]// Proceedings of the 29th ACM International Conference on Multimedia. New York:ACM Press, 2021: 153-161.
[51]	WANG F Y . Parallel intelligence in metaverses:welcome to Hanoi![J]. IEEE Intelligent Systems, 2022,37(1): 16-20.
[52]	GUO C , BAI T X , LU Y ,et al. Skywork-daVinci:a novel CPSS-based painting support system[C]// Proceedings of 2020 IEEE 16th International Conference on Automation Science and Engineering. Piscataway:IEEE Press, 2020: 673-678.
[53]	GUO C , BAI T X , WANG X ,et al. ShadowPainter:active learning enabled robotic painting through visual measurement and reproduction of the artistic creation process[J]. Journal of Intelligent ＆ Robotic Systems, 2022,105(3): 1-17.
[54]	KOTOVENKO D , WRIGHT M , HEIMBRECHT A ,et al. Rethinking style transfer:from pixels to parameterized brushstrokes[C]// Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2021: 12191-12200.
[55]	CORNIA M , STEFANINI M , BARALDI L ,et al. Meshed-memory transformer for image captioning[C]// Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2020: 10575-10584.
[56]	LU Y , GUO C , DAI X Y ,et al. Image captioning on fine art paintings via virtual paintings[C]// Proceedings of 2021 IEEE 1st International Conference on Digital Twins and Parallel Intelligence. Piscataway:IEEE Press, 2021: 156-159.
[57]	LU Y , GUO C , DAI X Y ,et al. Data-efficient image captioning of fine art paintings via virtual-real semantic alignment training[J]. Neurocomputing, 2022,490: 163-180.

Metrics

Recommended 0

No Suggested Reading articles found!

Architecture and key techniques of parallel creation through the fusion of human-cyber-physical intelligence in CPSS

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 57

Related Articles 15

Metrics

Recommended 0

[1]	Qinghai MIAO, Yisheng LYU. Parallel transportation systems in era of metaverse [J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(1): 32-40.
[2]	Mengzhen KANG, Hequan SUN, Xiujuan WANG, Fei-Yue WANG. Systems agriculture: modeling and control based on social and economic attributes of agriculture [J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(1): 41-50.
[3]	Xiao WANG, Linyao YANG, Bin HU, Jiachen HOU. Parallel reasoning: a virtual-real interactive knowledge collaboration framework based on ACP approach [J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(1): 69-82.
[4]	Yonglin TIAN, Yuanwen CHEN, Jing YANG, Yutong WANG, Xiao WANG, Qinghai MIAO, Ziran WANG, Fei-Yue WANG. Metaverses and parallel systems: the state of the art, comparisons and prospects [J]. Chinese Journal of Intelligent Science and Technology, 2023, 5(1): 121-132.
[5]	Mengzhen KANG, Wenzhong QIU, Zifu CHEN, Meng WANG, Shasha XU, Xiujuan WANG, Aidong NI, Yujie JIANG, Shichao CHEN, Philippe DEREFFYE, Fei-Yue WANG. Parallel Yuan-Ming Yuan Imperial Garden: from digital twin garden to metaverse smart heritage park [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 301-307.
[6]	Qiang WU, Xueting JI, Linyuan LYU. Artificial intelligence technologies and applications in the metaverse [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 324-334.
[7]	Qiuyun LYU, Shaopeng CHENG, Manzhi YANG, Xiaoguang CHEN, Zhen WANG. A metaverse-oriented digital citizen authentication scheme [J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(3): 396-409.
[8]	Xiaoshuang LI, Xiao WANG, Linyao YANG, Yonglin TIAN, Yutong WANG, Jun ZHANG, Fei-Yue WANG. MetaGrid: a parallel grids based approach for next generation smart power systems [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 387-398.
[9]	Yaling LI, Linyao YANG, Jun GE, Yuanqi QIN, Xiao WANG. Game 5.0: social cognitionparallel game based on the parallel systems and machine game [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 507-520.
[10]	Fei-Yue WANG, Huaiguang JIANG. Parallel battery: the framework and process for an intelligent and ecological battery system and related services [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(4): 521-531.
[11]	Chunfa WANG, Fei-Yue WANG, Yue LU, Huabiao LI, Chao GUO. Parallel museums: intelligent management and control of museum operations in the new era [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(2): 125-136.
[12]	Yuzhen WU, Jun ZHANG, Tianlu GAO, Yujian SUN, Jinxu LIU. Parallel port: new formation and system architecture of port industry Internet of minds in smart and green era [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(2): 218-227.
[13]	Fei-Yue WANG, Xiangbing MENG, Sicong DU, Zheng GENG. Parallel light field: the framework and processes [J]. Chinese Journal of Intelligent Science and Technology, 2021, 3(1): 110-122.
[14]	Fei-Yue WANG. Parallel control and digital twins:control theory revisited and reshaped [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(3): 293-300.
[15]	Yidong LI,Jun ZHANG,Yaodong TAO,Wei WANG,Yuanxiang GU,Fei-Yue WANG. Parallel security:generative adversarial systems for intelligent security in CPSS [J]. Chinese Journal of Intelligent Science and Technology, 2020, 2(2): 194-202.