一种高效鲁棒的元宇宙环境下的多场景智能医疗模型研究

doi:10.11959/j.issn.2096-0271.2023006

Abstract

Abstract:

Unbalanced medical and educational resources, low intelligence of the medical system, and reliance on individual experience in surgical operations are common in medical trade.The metaverse with immersive and interactive features is an effective tool to solve the problem.However, most of the existing solutions are based on a specific technology of virtual reality or artificial intelligence or a specific operation, and there is little systematic research on the multifunctional and multi-scenario medical metaverse.Therefore, a multi-scenario artificial intelligent medical model based on metaverse (MetaMed) was proposed, which elaborated the bottom-up implementations from four layers, including the access layer, data layer, technology layer and application layer.MetaMed was mathematically applied in five medical scenarios, i.e., intelligent surgery, online consultation, medical education, robotic surgery and outpatient registration scenarios, which provides references for the construction of medical metaverse in the future.

Key words: Metaverse, medical big data, artificial intelligence, intelligent healthcare

CLC Number:

Jiuwen ZHU, Yubing ZHOU, Hongbiao SI, Liang XU. An efficient and robust multi-scenario artificial intelligent medical model based on metaverse[J]. Big Data Research, 2024, 10(2): 122-139.

Figures/Tables 5

方法		应用	作者发表时间
综述类研究		探讨可应用于教育、远程护理、机器人手术等场景的脊柱元宇宙（spinemetaverse）	Chapman J Ret al^[2]2022
		基于AR、VR、区块链等技术探讨医疗保健元宇宙的发展与挑战	Mejia J M R and Rawat D B^[3] 2022
		描述元宇宙在医疗保健领域使用的物联网、区块链、人工智能等技术	MozumderMA I^[4]2022
		元宇宙在眼科领域远程医疗平台、医疗研学会议、数字教育、临床诊治的应用	TanT F et al^[5]2022
AR	手持显示型	基于AR显微镜检测转移性乳腺癌和前列腺癌	Chen D et al^[41]2019
		基于Curiscope的AR T恤的人体内部结构培训	Kye B^[15]2021
		3D打印结合AR应用于骨科肿瘤手术	Moreta-Martínez R et al^[42]2021
		AR技术结合身体传感器进行远程医疗	Mejia J M R and Rawat D B^[3]2022
	光学透视型	通过AR眼镜进行远程专家语音和临时视觉手术室操作指导	Cofano F^[43]2021
		基于HoloLens AR的临床手术和护理	Castelán E et al^[12]2021
	视频透视型	基于显微镜、头戴式显示器的AR系统应用于脊柱手术	AulogeP et al^[11]2019
		基于头戴式显示器导航AR系统的脊柱器械放置	Liu A et al^[13]2021
	投影显示型	基于AR医疗教育系统的解剖培训	Hoang T Net al^[14]2017
	综合	基于AR对医学生、医生与病人的培训指导	Campisi C et al^[44]2020
VR	沉浸式	基于虚拟现实暴露疗法的创伤后应激障碍治疗	Rizzo A A^[17-18]；Mozgai S^[19]2015,2017, 2021
		术前三维肝脏手术规划	Boedecker C et al^[21]2021
		基于VR的医学教育模式和传统模式的教学效率比较	Zhao G et al^[27]2021
		用于ADHD的VR训练系统	Kwan H-Y et al^[20]2022
		基于VR技术的ARC微创手术培训	Dinc F et al^[23]2022
	桌面式	经静脉拔铅（TLE）培训	Maytin M et al^[24]2015
	分布式	基于多人VR的心肺复苏训练	Creutzfeldt J et al^[25]2016
		通过沉浸式VR课程进行医患沟通培训	Real F et al^[26]2017
	综合	完全沉浸式VR医学培训与部分沉浸式医学培训效果比较	Gutiérrez F et al^[28]2007
AI	监督学习	血液数字形态分析	Kratz A et al^[30]2019
		分析疾病症状进行多病种药物推荐	Komal Kumar Aet al^[45]2021
	半监督学习	基于注意力的多任务半监督学习方法用于脑瘤和白质超密集区的图像分割	Chen S et al^[33]2019
		三维医学图像检测	Wang D et al^[31]2020
		基于联邦半监督学习方法对三维胸部电子计算机断层扫描中的新冠影响区域图像分割	Yang D et al^[32]2021
		基于异质电子病历数据测量病人相似度	Wang N et al^[34]2021
	无监督学习	骨分割	Chen J and Frey E C^[37]2020
		无监督学习用于生理或病理压力监测	Iqbal J et al^[36]2022
		无监督适应性学习基于心脏数据集、腹部数据集和大脑数据集的医疗数据分割	Xie Q et al^[38]2022
		自动分割肺部切片CT上的COVID-19病变	Sherwani M K et al^[39]2022
		医学图像分析	Zhang Y et al^[40]2022

References 53

[9]	HOU Y , XU W W . A survey of augmented reality technology[J]. Computer Measurement＆ Control, 2017,25(2): 1-7,22.
[10]	MAAS M J , HUGHES J M . Virtual,augmented and mixed reality in K–12 education:a review of the literature[J]. Technology,Pedagogy and Education, 2020,29(2): 231-249.
[11]	AULOGE P , CAZZATO R L , RAMAMURTHY N ,et al. Augmented reality and artificial intelligencebased navigation during percutaneous vertebroplasty:a pilot randomised clinical trial[J]. European Spine Journal, 2020,29(7): 1580-1589.
[12]	CASTELAN E , VINNIKOV M , ZHOU X A . Augmented reality anatomy visualization for surgery assistance with HoloLens:AR surgery assistance with HoloLens[C]// Proceedings of the Proceedings of the 2021 ACM International Conference on Interactive Media Experiences. New York:ACM, 2021: 329-331.
[13]	LIU A , JIN Y K , COTTRILL E ,et al. Clinical accuracy and initial experience with augmented reality-assisted pedicle screw placement:the first 205 screws[J]. Journal of Neurosurgery Spine, 2021,36(3): 351-357.
[14]	HOANG T , REINOSO M , JOUKHADAR Z ,et al. Augmented studio:projection mapping on moving body for physiotherapy education[C]// Proceedings of the Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. New York:ACM, 2017.
[15]	KYE B , HAN N , KIM E ,et al. Educational applications of metaverse:possibilities and limitations[J]. Journal of Educational Evaluation for Health Professions, 2021,18:32.
[16]	陈浩磊, 邹湘军, 陈燕 ,等. 虚拟现实技术的最新发展与展望[J]. 中国科技论文在线, 2011,6(1): 1-5,14.
	CHEN H L , ZOU X J , CHEN Y ,et al. Overview of the advance in virtual reality technology[J]. Sciencepaper Online, 2011,6(1): 1-5,14.
[17]	RIZZO A A . Bravemind:advancing the virtual iraq/afghanistan ptsd exposure therapy for MST[R]. 2015.
[18]	RIZZO A , ROY M J , HARTHOLT A ,et al. Virtual reality applications for the assessment and treatment of PTSD[M]// Handbook of Military Psychology. Cham: Springer, 2017: 453-471.
[19]	MOZGAI S , LEEDS A , KWOK D ,et al. Building BRAVEMIND Vietnam:usercentered design for virtual reality exposure therapy[C]// Proceedings of the 2021 IEEE International Conference on Artificial Intelligence and Virtual Reality. Piscataway:IEEE Press, 2021: 247-250.
[20]	KWAN H Y , LIN L , FAHY C ,et al. Designing VR training systems for children with attention deficit hyperactivity disorder (ADHD)[C]// Proceedings of the 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops. Piscataway:IEEE Press, 2022: 88-89.
[21]	BOEDECKER C , HUETTL F , SAALFELD P ,et al. Using virtual 3D-models in surgical planning:workflow of an immersive virtual reality application in liver surgery[J]. Langenbeck’s Archives of Surgery, 2021,406(3): 911-915.
[22]	ZHANG J , CHANG J , YANG X ,et al. Virtual reality surgery simulation:a survey on patient specific solution[C]// Proceedings of Next Generation Computer Animation Techniques. Cham:Springer, 2008: 220-233.
[23]	DINC F , OUMIMOUN K , KWABLA W ,et al. Towards real-time bone drilling simulation for anchor placement in VR based arthroscopic rotator cuff surgery simulation[J]. AMIA Joint Summits on Translational Science Proceedings AMIA Joint Summits on Translational Science, 2022,2022: 178-185.
[24]	MAYTIN M , DAILY T P , CARILLO R G . Virtual reality lead extraction as a method for training new physicians:a pilot study[J]. Pacing and Clinical Electrophysiology:PACE, 2015,38(3): 319-325.
[25]	CREUTZFELDT J , HEDMAN L , FELL?NDER-TSAI L . Cardiopulmonary resuscitation training by avatars:a qualitative study of medical students' experiences using a multiplayer virtual world[J]. JMIR Serious Games, 2016,4(2): e22.
[26]	REAL F J , DEBLASIO D , BECK A F ,et al. A virtual reality curriculum for pediatric residents decreases rates of influenza vaccine refusal[J]. Academic Pediatrics, 2017,17(4): 431-435.
[27]	ZHAO G J , FAN M J , YUAN Y B ,et al. The comparison of teaching efficiency between virtual reality and traditional education in medical education:a systematic review and meta-analysis[J]. Annals of Translational Medicine, 2021,9(3): 252.
[28]	GUTIéRREZ F , PIERCE J , VERGARA V M ,et al. The effect of degree of immersion upon learning performance in virtual reality simulations for medical education[J]. Studies in Health Technology and Informatics, 2007,125: 155-160.
[29]	RASHIDI H H , TRAN N , ALBAHRA S ,et al. Machine learning in health care and laboratory medicine:general overview of supervised learning and Auto-ML[J]. International Journal of Laboratory Hematology, 2021,43(Suppl 1): 15-22.
[30]	KRATZ A , LEE S H , ZINI G ,et al. Digital morphology analyzers in hematology:ICSH review and recommendations[J]. International Journal of Laboratory Hematology, 2019,41(4): 437-447.
[31]	WANG D , ZHANG Y , ZHANG K X ,et al. FocalMix:semi-supervised learning for 3D medical image detection[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2020: 3950-3959.
[32]	YANG D , XU Z Y , LI W Q ,et al. Federated semi-supervised learning for COVID region segmentation in chest CT using multi-national data from China,Italy,Japan[J]. Medical Image Analysis, 2021,70:101992.
[33]	CHEN S , BORTSOVA G , GARCíAUCEDA JUáREZ A , et al . Multitask attention-based semi-supervised learning for medical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham:Springer, 2019: 457-465.
[34]	WANG N , HUANG Y Q , LIU H L ,et al. Study on the semi-supervised learning-based patient similarity from heterogeneous electronic medical records[J]. BMC Medical Informatics and Decision Making, 2021,21(Suppl 2): 58.
[35]	RAZA K , SINGH N K . A tour of unsupervised deep learning for medical image analysis[J]. Current Medical Imaging, 2021,17(9): 1059-1077.
[36]	IQBAL T , ELAHI A , WIJNS W ,et al. Exploring unsupervised machine learning classification methods for physiological stress detection[J]. Frontiers in Medical Technology, 2022,4:782756.
[37]	CHEN J Y , FREY E C . An unsupervised learning model for medical image segmentation[EB]. arXiv preprint, 2020,arXiv:2001.10155.
[38]	XIE Q S , LI Y X , HE N J ,et al. Unsupervised domain adaptation for medical image segmentation by disentanglement learning and selftraining[J]. IEEE Transactions on Medical Imaging, 2024,43(1): 4-14.
[39]	SHERWANI M K , MARZULLO A , DE MOMI E ,et al. Lesion segmentation in lung CT scans using unsupervised adversarial learning[J]. Medical ＆Biological Engineering ＆ Computing, 2022,60(11): 3203-3215.
[40]	ZHANG Y M , LI H L , HOU Y W ,et al. Consecutive knowledge meta-adaptation learning for unsupervised medical diagnosis[J]. Knowledge-Based Systems, 2024,291:111573.
[41]	CHEN P H C , GADEPALLI K , MACDONALD R ,et al. An augmented reality microscope with real-time artificial intelligence integration for cancer diagnosis[J]. Nature Medicine, 2019,25(9): 1453-1457.
[42]	MORETA-MARTINEZ R , POSE-DíEZDE-LA-LASTRA A , CALVO-HARO J A ,et al. Combining augmented reality and 3D printing to improve surgical workflows in orthopedic oncology:smartphone application and clinical evaluation[J]. Sensors, 2021,21(4): 1370.
[43]	COFANO F , DI PERNA G , BOZZARO M ,et al. Augmented reality in medical practice:from spine surgery to remote assistance[J]. Frontiers in Surgery, 2021,8:657901.
[44]	CAMPISI C A , LI E H , JIMENEZ D E ,et al. Augmented reality in medical education and training:from physicians to patients[M]. Augmented reality in education. Cham: Springer, 2020:111138.
[45]	KOMAL KUMAR N , VIGNESWARI D . A drug recommendation system for multidisease in health care using machine learning[C]// Proceedings of International Conference on Advanced Communication and Computational Technology. Singapore:Springer, 2021: 1-12.
[46]	DHAYNE H , HAQUE R , KILANY R ,et al. In search of big medical data integration solutions - a comprehensive survey[J]. IEEE Access, 2019,7: 91265-91290.
[47]	XIONG J H , HSIANG E L , HE Z Q ,et al. Augmented reality and virtual reality displays:emerging technologies and future perspectives[J]. Light,Science ＆Applications, 2021,10(1): 216.
[48]	NGUYEN H X , TRESTIAN R , TO D ,et al. Digital twin for 5G and beyond[J]. IEEE Communications Magazine, 2021,59(2): 10-15.
[49]	SAAD W , BENNIS M , CHEN M Z . A vision of 6G wireless systems:applications,trends,technologies,and open research problems[J]. IEEE Network, 2020,34(3): 134-142.
[50]	SHEN S , WANG P Z , LI X Y ,et al. Preoperative simulation using a threedimensional printing model for surgical treatment of old and complex tibial plateau fractures[J]. Scientific Reports, 2020,10(1): 6044.
[51]	WAGNER J P , COCHRAN A L , JONES C ,et al. Professional use of social media among surgeons:results of a multiinstitutional study[J]. Journal of Surgical Education, 2018,75(3): 804-810.
[1]	LEE L H , BRAUD T , ZHOU P Y ,et al. All one needs to know about metaverse:a complete survey on technological singularity,virtual ecosystem,and research agenda[EB]. arXiv preprint, 2021.
[2]	CHAPMAN J R , WANG J C , WIECHERT K . Into the Spine Metaverse:reflections on a future Metaspine (Uni-) verse[J]. Global Spine Journal, 2022,12(4): 545-547.
[3]	RUIZ MEJIA J M , RAWAT D B . Recent advances in a medical domain metaverse:status,challenges,and perspective[C]// Proceedings of the 2022 Thirteenth International Conference on Ubiquitous and Future Networks. Piscataway:IEEE Press, 2022: 357-362.
[4]	MOZUMDER M A I , SHEERAZ M M , ATHAR A ,et al. Overview:technology roadmap of the future trend of metaverse based on IoT,blockchain,AI technique,and medical domain metaverse activity[C]// Proceedings of the 2022 24th International Conference on Advanced Communication Technology. Piscataway:IEEE Press, 2022: 256-261.
[5]	TAN T F , LI Y , LIM J S ,et al. Metaverse and virtual health care in ophthalmology:opportunities and challenges[J]. AsiaPacific Journal of Ophthalmology, 2022,11(3): 237-246.
[6]	KALA D N . Revolutionizing medical education with metaverse[J]. International Journal of Scientific Research in Computer Science,Engineering and Information Technology, 2022: 26-32.
[7]	BRUYNSEELS K , SANTONI DE SIO F , VAN DEN HOVEN J . Digital twins in health care:ethical implications of an emerging engineering paradigm[J]. Frontiers in Genetics, 2018,9:31.
[8]	D’SOUZA M , GENDREAU J , FENG A ,et al. Robotic-assisted spine surgery:history,efficacy,cost,and future trends[J]. Robotic Surgery (Auckland), 2019,6: 9-23.
[9]	侯颖, 许威威 . 增强现实技术综述[J]. 计算机测量与控制, 2017,25(2): 1-7,22.
[52]	WISE S , DUFFIELD C , FRY M ,et al. A team mental model approach to understanding team effectiveness in an emergency department:a qualitative study[J]. Journal of Health Services Research ＆ Policy, 2022,27(1): 14-21.
[53]	BASHKANOV O , SAALFELD P , GUNASEKARAN H ,et al. VR multi-user conference room for surgery planning[C]// Proceedings of Annual Meeting of the German Society of Computer and RobotAssisted Surgery.[S.l.:s.n.], 2019: 264-268.

Metrics

Recommended 0

No Suggested Reading articles found!

An efficient and robust multi-scenario artificial intelligent medical model based on metaverse

RichHTML

PDF下载

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 5

References 53

Related Articles 15

Metrics

Recommended 0

[1]	Guojie LI. Big data and computing models [J]. Big Data Research, 2024, 10(1): 9-16.
[2]	Zhi WANG, Shutao XIA, Rui MAO. Edge-intelligence-based immersive metaverse:key technologies and prospects [J]. Big Data Research, 2024, 10(1): 35-45.
[3]	Hao WANG, Yushan PAN, Yi PAN. Generative AI empowered metaverse organisms: prospects and challenges [J]. Big Data Research, 2023, 9(3): 85-96.
[4]	Yadong WU, Jiaming CHEN, Yan LUO, Xuefeng WANG, Dechun HUANG, Chao NI, Jiming LAN, Suiqun LI, Weihan ZHANG, Wei DAI. An overview of Caideng metaverse research [J]. Big Data Research, 2023, 9(3): 97-113.
[5]	Yimin DENG, Xulong ZHANG, Shijing SI, Jianzong WANG, Jing XIAO. Human avatars synthesis technologies: a survey [J]. Big Data Research, 2023, 9(3): 114-139.
[6]	Jingwen LI, Yawen LI. Research on algorithm and application risk and its governance [J]. Big Data Research, 2023, 9(3): 140-149.
[7]	Ye LIU, Wei CHENG, Yan LI, Yimeng YIN, Huijie SUN. Research on the construction of educational community from the perspective of metaverse [J]. Big Data Research, 2023, 9(1): 78-86.
[8]	Yayun HE, Junqing PENG, Jianzong WANG, Jing XIAO. Rhythm dancer: 3D dance generation by keymotion transition graph and pose-interpolation network [J]. Big Data Research, 2023, 9(1): 23-37.
[9]	Yifei PENG, Zhen YUAN, Xulong ZHANG, Guilin JIANG, Yujiang LIU. Metaverse air pollutant concentration inference model based on digital twin technology [J]. Big Data Research, 2023, 9(1): 38-50.
[10]	Zihang WANG, Xiangqun YU, Hongbiao SI, Simin FU, Xulong ZHANG, Shaoliang PENG. Design of the layered model of the metaverse based on computing network [J]. Big Data Research, 2023, 9(1): 51-62.
[11]	Rui ZHU, Hongzhi WANG, Shuangshuang CUI, Kaixin ZHANG, Yu YAN. Cloud-edge-end collaborative big data management for metaverse [J]. Big Data Research, 2023, 9(1): 63-77.
[12]	Bo HE. Research on the legal conundrums and regulation ideas of metaverse [J]. Big Data Research, 2023, 9(1): 87-102.
[13]	Yang SHEN, Menglong YU. Metaverse and big data: data insight and value connection in spatio-temporal intelligence [J]. Big Data Research, 2023, 9(1): 103-110.
[14]	Chenhuizi WANG, Wei CAI. Digital economics in metaverse: state-of-the-art, characteristics, and vision [J]. Big Data Research, 2022, 8(3): 140-150.
[15]	Zhitao ZHAO, Lijun ZHAO, Zheng ZHANG, Ping TANG. Integration of remote sensing intelligent processing algorithm using container cloud technology [J]. Big Data Research, 2022, 8(2): 58-74.