一种高效鲁棒的元宇宙环境下的多场景智能医疗模型研究

doi:10.11959/j.issn.2096-0271.2023006

大数据 ›› 2024, Vol. 10 ›› Issue (2): 122-139.doi: 10.11959/j.issn.2096-0271.2023006

• 研究 • 上一篇

一种高效鲁棒的元宇宙环境下的多场景智能医疗模型研究

朱玖闻, 周玉冰, 斯洪标, 徐亮

湖南财信金融控股集团有限公司，湖南长沙 410015

出版日期:2024-03-01 发布日期:2024-03-01
作者简介:朱玖闻（1997‒ ），女，中国科学院计算技术研究所计算机硕士，曾任腾讯算法工程师、华为算法工程师，在MedicalImageAnalysis等期刊上发表多篇学术论文，主要研究方向为人工智能、计算机视觉、医学影像分析和元宇宙等。
周玉冰（1997‒ ），女，中南大学管理科学与工程硕士。在深度强化学习领域发表核心期刊，主要研究方向为动态定价、人工智能和元宇宙等。
斯洪标（1966‒ ），男，现任湖南财信金融控股集团副总经理，北京中国网传播公司董事，湖南财信资产管理公司董事，长沙市首批赴美高级公共管理培训班、美国加州圣何塞大学MPA，工学硕士，副研究员。政协湖南省委员会第九届、第十届、第十二届委员，中国民主建国会湖南省第六届、第七届参政议政委员会副主任委员，第九届财政与金融委员会主任委员，长沙市大数据（地理信息）产业特聘专家。曾任中南大学政府与社会资本合作研究中心研究员。
徐亮（1981‒ ），男，中国科学院软件研究所博士，先后在中国科学院软件研究所、基础软件国家工程研究中心、湖南师范大学、湖南财信金融控股集团等高校科研院所和企业工作。现任湖南财信金融控股集团信息技术部总经理，IT治理委员会办公室主任。研究方向涉及软件理论、形式化方法、安全操作系统、智能语义分析、信息化管理、企业数字化转型、金融科技等领域。

An efficient and robust multi-scenario artificial intelligent medical model based on metaverse

Jiuwen ZHU, Yubing ZHOU, Hongbiao SI, Liang XU

Hunan Chasing Financial Holdings Co., Ltd., Changsha 410015, China

Online:2024-03-01 Published:2024-03-01

摘要/Abstract

摘要：

现今医疗行业普遍存在医疗资源和教育资源不均衡、医疗体系智能化水平低、手术操作依赖个体经验等问题，拥有沉浸、互动特点的元宇宙为以上问题提供了解决方案。但现有的解决方案多基于虚拟现实或人工智能中的一种技术，针对特定的手术或疾病诊断问题进行探索，少有针对多功能、多场景应用的医疗元宇宙进行的系统研究。基于此，提出了元宇宙环境下的多场景智能医疗模型框架（MetaMed），从接入层、数据层、技术层、应用层自底向上阐述了模型的框架设计。该框架在智能手术、线上会诊、医疗培训、机器人手术和门诊挂号5种应用场景中给出了数学描述，并为未来医疗元宇宙的构建提供参考。

关键词: 元宇宙, 医疗大数据, 人工智能, 智能医疗

Abstract:

Unbalanced medical and educational resources, low intelligence of the medical system, and reliance on individual experience in surgical operations are common in medical trade.The metaverse with immersive and interactive features is an effective tool to solve the problem.However, most of the existing solutions are based on a specific technology of virtual reality or artificial intelligence or a specific operation, and there is little systematic research on the multifunctional and multi-scenario medical metaverse.Therefore, a multi-scenario artificial intelligent medical model based on metaverse (MetaMed) was proposed, which elaborated the bottom-up implementations from four layers, including the access layer, data layer, technology layer and application layer.MetaMed was mathematically applied in five medical scenarios, i.e., intelligent surgery, online consultation, medical education, robotic surgery and outpatient registration scenarios, which provides references for the construction of medical metaverse in the future.

Key words: Metaverse, medical big data, artificial intelligence, intelligent healthcare

中图分类号:

朱玖闻, 周玉冰, 斯洪标, 徐亮. 一种高效鲁棒的元宇宙环境下的多场景智能医疗模型研究[J]. 大数据, 2024, 10(2): 122-139.

Jiuwen ZHU, Yubing ZHOU, Hongbiao SI, Liang XU. An efficient and robust multi-scenario artificial intelligent medical model based on metaverse[J]. Big Data Research, 2024, 10(2): 122-139.

图/表 5

表1

基于元宇宙相关技术的医疗研究"

方法		应用	作者发表时间
综述类研究		探讨可应用于教育、远程护理、机器人手术等场景的脊柱元宇宙（spinemetaverse）	Chapman J Ret al^[2]2022
		基于AR、VR、区块链等技术探讨医疗保健元宇宙的发展与挑战	Mejia J M R and Rawat D B^[3] 2022
		描述元宇宙在医疗保健领域使用的物联网、区块链、人工智能等技术	MozumderMA I^[4]2022
		元宇宙在眼科领域远程医疗平台、医疗研学会议、数字教育、临床诊治的应用	TanT F et al^[5]2022
AR	手持显示型	基于AR显微镜检测转移性乳腺癌和前列腺癌	Chen D et al^[41]2019
		基于Curiscope的AR T恤的人体内部结构培训	Kye B^[15]2021
		3D打印结合AR应用于骨科肿瘤手术	Moreta-Martínez R et al^[42]2021
		AR技术结合身体传感器进行远程医疗	Mejia J M R and Rawat D B^[3]2022
	光学透视型	通过AR眼镜进行远程专家语音和临时视觉手术室操作指导	Cofano F^[43]2021
		基于HoloLens AR的临床手术和护理	Castelán E et al^[12]2021
	视频透视型	基于显微镜、头戴式显示器的AR系统应用于脊柱手术	AulogeP et al^[11]2019
		基于头戴式显示器导航AR系统的脊柱器械放置	Liu A et al^[13]2021
	投影显示型	基于AR医疗教育系统的解剖培训	Hoang T Net al^[14]2017
	综合	基于AR对医学生、医生与病人的培训指导	Campisi C et al^[44]2020
VR	沉浸式	基于虚拟现实暴露疗法的创伤后应激障碍治疗	Rizzo A A^[17-18]；Mozgai S^[19]2015,2017, 2021
		术前三维肝脏手术规划	Boedecker C et al^[21]2021
		基于VR的医学教育模式和传统模式的教学效率比较	Zhao G et al^[27]2021
		用于ADHD的VR训练系统	Kwan H-Y et al^[20]2022
		基于VR技术的ARC微创手术培训	Dinc F et al^[23]2022
	桌面式	经静脉拔铅（TLE）培训	Maytin M et al^[24]2015
	分布式	基于多人VR的心肺复苏训练	Creutzfeldt J et al^[25]2016
		通过沉浸式VR课程进行医患沟通培训	Real F et al^[26]2017
	综合	完全沉浸式VR医学培训与部分沉浸式医学培训效果比较	Gutiérrez F et al^[28]2007
AI	监督学习	血液数字形态分析	Kratz A et al^[30]2019
		分析疾病症状进行多病种药物推荐	Komal Kumar Aet al^[45]2021
	半监督学习	基于注意力的多任务半监督学习方法用于脑瘤和白质超密集区的图像分割	Chen S et al^[33]2019
		三维医学图像检测	Wang D et al^[31]2020
		基于联邦半监督学习方法对三维胸部电子计算机断层扫描中的新冠影响区域图像分割	Yang D et al^[32]2021
		基于异质电子病历数据测量病人相似度	Wang N et al^[34]2021
	无监督学习	骨分割	Chen J and Frey E C^[37]2020
		无监督学习用于生理或病理压力监测	Iqbal J et al^[36]2022
		无监督适应性学习基于心脏数据集、腹部数据集和大脑数据集的医疗数据分割	Xie Q et al^[38]2022
		自动分割肺部切片CT上的COVID-19病变	Sherwani M K et al^[39]2022
		医学图像分析	Zhang Y et al^[40]2022

表1

图1

图2

图3

图4

参考文献 53

[9]	HOU Y , XU W W . A survey of augmented reality technology[J]. Computer Measurement＆ Control, 2017,25(2): 1-7,22.
[10]	MAAS M J , HUGHES J M . Virtual,augmented and mixed reality in K–12 education:a review of the literature[J]. Technology,Pedagogy and Education, 2020,29(2): 231-249.
[11]	AULOGE P , CAZZATO R L , RAMAMURTHY N ,et al. Augmented reality and artificial intelligencebased navigation during percutaneous vertebroplasty:a pilot randomised clinical trial[J]. European Spine Journal, 2020,29(7): 1580-1589.
[12]	CASTELAN E , VINNIKOV M , ZHOU X A . Augmented reality anatomy visualization for surgery assistance with HoloLens:AR surgery assistance with HoloLens[C]// Proceedings of the Proceedings of the 2021 ACM International Conference on Interactive Media Experiences. New York:ACM, 2021: 329-331.
[13]	LIU A , JIN Y K , COTTRILL E ,et al. Clinical accuracy and initial experience with augmented reality-assisted pedicle screw placement:the first 205 screws[J]. Journal of Neurosurgery Spine, 2021,36(3): 351-357.
[14]	HOANG T , REINOSO M , JOUKHADAR Z ,et al. Augmented studio:projection mapping on moving body for physiotherapy education[C]// Proceedings of the Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. New York:ACM, 2017.
[15]	KYE B , HAN N , KIM E ,et al. Educational applications of metaverse:possibilities and limitations[J]. Journal of Educational Evaluation for Health Professions, 2021,18:32.
[16]	陈浩磊, 邹湘军, 陈燕 ,等. 虚拟现实技术的最新发展与展望[J]. 中国科技论文在线, 2011,6(1): 1-5,14.
	CHEN H L , ZOU X J , CHEN Y ,et al. Overview of the advance in virtual reality technology[J]. Sciencepaper Online, 2011,6(1): 1-5,14.
[17]	RIZZO A A . Bravemind:advancing the virtual iraq/afghanistan ptsd exposure therapy for MST[R]. 2015.
[18]	RIZZO A , ROY M J , HARTHOLT A ,et al. Virtual reality applications for the assessment and treatment of PTSD[M]// Handbook of Military Psychology. Cham: Springer, 2017: 453-471.
[19]	MOZGAI S , LEEDS A , KWOK D ,et al. Building BRAVEMIND Vietnam:usercentered design for virtual reality exposure therapy[C]// Proceedings of the 2021 IEEE International Conference on Artificial Intelligence and Virtual Reality. Piscataway:IEEE Press, 2021: 247-250.
[20]	KWAN H Y , LIN L , FAHY C ,et al. Designing VR training systems for children with attention deficit hyperactivity disorder (ADHD)[C]// Proceedings of the 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops. Piscataway:IEEE Press, 2022: 88-89.
[21]	BOEDECKER C , HUETTL F , SAALFELD P ,et al. Using virtual 3D-models in surgical planning:workflow of an immersive virtual reality application in liver surgery[J]. Langenbeck’s Archives of Surgery, 2021,406(3): 911-915.
[22]	ZHANG J , CHANG J , YANG X ,et al. Virtual reality surgery simulation:a survey on patient specific solution[C]// Proceedings of Next Generation Computer Animation Techniques. Cham:Springer, 2008: 220-233.
[23]	DINC F , OUMIMOUN K , KWABLA W ,et al. Towards real-time bone drilling simulation for anchor placement in VR based arthroscopic rotator cuff surgery simulation[J]. AMIA Joint Summits on Translational Science Proceedings AMIA Joint Summits on Translational Science, 2022,2022: 178-185.
[24]	MAYTIN M , DAILY T P , CARILLO R G . Virtual reality lead extraction as a method for training new physicians:a pilot study[J]. Pacing and Clinical Electrophysiology:PACE, 2015,38(3): 319-325.
[25]	CREUTZFELDT J , HEDMAN L , FELL?NDER-TSAI L . Cardiopulmonary resuscitation training by avatars:a qualitative study of medical students' experiences using a multiplayer virtual world[J]. JMIR Serious Games, 2016,4(2): e22.
[26]	REAL F J , DEBLASIO D , BECK A F ,et al. A virtual reality curriculum for pediatric residents decreases rates of influenza vaccine refusal[J]. Academic Pediatrics, 2017,17(4): 431-435.
[27]	ZHAO G J , FAN M J , YUAN Y B ,et al. The comparison of teaching efficiency between virtual reality and traditional education in medical education:a systematic review and meta-analysis[J]. Annals of Translational Medicine, 2021,9(3): 252.
[28]	GUTIéRREZ F , PIERCE J , VERGARA V M ,et al. The effect of degree of immersion upon learning performance in virtual reality simulations for medical education[J]. Studies in Health Technology and Informatics, 2007,125: 155-160.
[29]	RASHIDI H H , TRAN N , ALBAHRA S ,et al. Machine learning in health care and laboratory medicine:general overview of supervised learning and Auto-ML[J]. International Journal of Laboratory Hematology, 2021,43(Suppl 1): 15-22.
[30]	KRATZ A , LEE S H , ZINI G ,et al. Digital morphology analyzers in hematology:ICSH review and recommendations[J]. International Journal of Laboratory Hematology, 2019,41(4): 437-447.
[31]	WANG D , ZHANG Y , ZHANG K X ,et al. FocalMix:semi-supervised learning for 3D medical image detection[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2020: 3950-3959.
[32]	YANG D , XU Z Y , LI W Q ,et al. Federated semi-supervised learning for COVID region segmentation in chest CT using multi-national data from China,Italy,Japan[J]. Medical Image Analysis, 2021,70:101992.
[33]	CHEN S , BORTSOVA G , GARCíAUCEDA JUáREZ A , et al . Multitask attention-based semi-supervised learning for medical image segmentation[C]// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham:Springer, 2019: 457-465.
[34]	WANG N , HUANG Y Q , LIU H L ,et al. Study on the semi-supervised learning-based patient similarity from heterogeneous electronic medical records[J]. BMC Medical Informatics and Decision Making, 2021,21(Suppl 2): 58.
[35]	RAZA K , SINGH N K . A tour of unsupervised deep learning for medical image analysis[J]. Current Medical Imaging, 2021,17(9): 1059-1077.
[36]	IQBAL T , ELAHI A , WIJNS W ,et al. Exploring unsupervised machine learning classification methods for physiological stress detection[J]. Frontiers in Medical Technology, 2022,4:782756.
[37]	CHEN J Y , FREY E C . An unsupervised learning model for medical image segmentation[EB]. arXiv preprint, 2020,arXiv:2001.10155.
[38]	XIE Q S , LI Y X , HE N J ,et al. Unsupervised domain adaptation for medical image segmentation by disentanglement learning and selftraining[J]. IEEE Transactions on Medical Imaging, 2024,43(1): 4-14.
[39]	SHERWANI M K , MARZULLO A , DE MOMI E ,et al. Lesion segmentation in lung CT scans using unsupervised adversarial learning[J]. Medical ＆Biological Engineering ＆ Computing, 2022,60(11): 3203-3215.
[40]	ZHANG Y M , LI H L , HOU Y W ,et al. Consecutive knowledge meta-adaptation learning for unsupervised medical diagnosis[J]. Knowledge-Based Systems, 2024,291:111573.
[41]	CHEN P H C , GADEPALLI K , MACDONALD R ,et al. An augmented reality microscope with real-time artificial intelligence integration for cancer diagnosis[J]. Nature Medicine, 2019,25(9): 1453-1457.
[42]	MORETA-MARTINEZ R , POSE-DíEZDE-LA-LASTRA A , CALVO-HARO J A ,et al. Combining augmented reality and 3D printing to improve surgical workflows in orthopedic oncology:smartphone application and clinical evaluation[J]. Sensors, 2021,21(4): 1370.
[43]	COFANO F , DI PERNA G , BOZZARO M ,et al. Augmented reality in medical practice:from spine surgery to remote assistance[J]. Frontiers in Surgery, 2021,8:657901.
[44]	CAMPISI C A , LI E H , JIMENEZ D E ,et al. Augmented reality in medical education and training:from physicians to patients[M]. Augmented reality in education. Cham: Springer, 2020:111138.
[45]	KOMAL KUMAR N , VIGNESWARI D . A drug recommendation system for multidisease in health care using machine learning[C]// Proceedings of International Conference on Advanced Communication and Computational Technology. Singapore:Springer, 2021: 1-12.
[46]	DHAYNE H , HAQUE R , KILANY R ,et al. In search of big medical data integration solutions - a comprehensive survey[J]. IEEE Access, 2019,7: 91265-91290.
[47]	XIONG J H , HSIANG E L , HE Z Q ,et al. Augmented reality and virtual reality displays:emerging technologies and future perspectives[J]. Light,Science ＆Applications, 2021,10(1): 216.
[48]	NGUYEN H X , TRESTIAN R , TO D ,et al. Digital twin for 5G and beyond[J]. IEEE Communications Magazine, 2021,59(2): 10-15.
[49]	SAAD W , BENNIS M , CHEN M Z . A vision of 6G wireless systems:applications,trends,technologies,and open research problems[J]. IEEE Network, 2020,34(3): 134-142.
[50]	SHEN S , WANG P Z , LI X Y ,et al. Preoperative simulation using a threedimensional printing model for surgical treatment of old and complex tibial plateau fractures[J]. Scientific Reports, 2020,10(1): 6044.
[51]	WAGNER J P , COCHRAN A L , JONES C ,et al. Professional use of social media among surgeons:results of a multiinstitutional study[J]. Journal of Surgical Education, 2018,75(3): 804-810.
[1]	LEE L H , BRAUD T , ZHOU P Y ,et al. All one needs to know about metaverse:a complete survey on technological singularity,virtual ecosystem,and research agenda[EB]. arXiv preprint, 2021.
[2]	CHAPMAN J R , WANG J C , WIECHERT K . Into the Spine Metaverse:reflections on a future Metaspine (Uni-) verse[J]. Global Spine Journal, 2022,12(4): 545-547.
[3]	RUIZ MEJIA J M , RAWAT D B . Recent advances in a medical domain metaverse:status,challenges,and perspective[C]// Proceedings of the 2022 Thirteenth International Conference on Ubiquitous and Future Networks. Piscataway:IEEE Press, 2022: 357-362.
[4]	MOZUMDER M A I , SHEERAZ M M , ATHAR A ,et al. Overview:technology roadmap of the future trend of metaverse based on IoT,blockchain,AI technique,and medical domain metaverse activity[C]// Proceedings of the 2022 24th International Conference on Advanced Communication Technology. Piscataway:IEEE Press, 2022: 256-261.
[5]	TAN T F , LI Y , LIM J S ,et al. Metaverse and virtual health care in ophthalmology:opportunities and challenges[J]. AsiaPacific Journal of Ophthalmology, 2022,11(3): 237-246.
[6]	KALA D N . Revolutionizing medical education with metaverse[J]. International Journal of Scientific Research in Computer Science,Engineering and Information Technology, 2022: 26-32.
[7]	BRUYNSEELS K , SANTONI DE SIO F , VAN DEN HOVEN J . Digital twins in health care:ethical implications of an emerging engineering paradigm[J]. Frontiers in Genetics, 2018,9:31.
[8]	D’SOUZA M , GENDREAU J , FENG A ,et al. Robotic-assisted spine surgery:history,efficacy,cost,and future trends[J]. Robotic Surgery (Auckland), 2019,6: 9-23.
[9]	侯颖, 许威威 . 增强现实技术综述[J]. 计算机测量与控制, 2017,25(2): 1-7,22.
[52]	WISE S , DUFFIELD C , FRY M ,et al. A team mental model approach to understanding team effectiveness in an emergency department:a qualitative study[J]. Journal of Health Services Research ＆ Policy, 2022,27(1): 14-21.
[53]	BASHKANOV O , SAALFELD P , GUNASEKARAN H ,et al. VR multi-user conference room for surgery planning[C]// Proceedings of Annual Meeting of the German Society of Computer and RobotAssisted Surgery.[S.l.:s.n.], 2019: 264-268.

一种高效鲁棒的元宇宙环境下的多场景智能医疗模型研究

An efficient and robust multi-scenario artificial intelligent medical model based on metaverse

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 5

参考文献 53

相关文章 15

Metrics

推荐阅读 0

[1]	李国杰. 大数据与计算模型[J]. 大数据, 2024, 10(1): 9-16.
[2]	王智, 夏树涛, 毛睿. 基于边缘智能的沉浸式元宇宙关键技术与展望[J]. 大数据, 2024, 10(1): 35-45.
[3]	王皓, 潘昱杉, 潘毅. 生成式人工智能大模型赋能的元宇宙生命体：前瞻和挑战[J]. 大数据, 2023, 9(3): 85-96.
[4]	吴亚东, 陈家鸣, 罗焱, 王学锋, 黄德春, 倪超, 蓝集明, 李随群, 张巍瀚, 代唯. 彩灯元宇宙研究综述[J]. 大数据, 2023, 9(3): 97-113.
[5]	邓钇敏, 张旭龙, 司世景, 王健宗, 肖京. 虚拟人形象合成技术综述[J]. 大数据, 2023, 9(3): 114-139.
[6]	李婧文, 李雅文. 算法应用风险与治理研究[J]. 大数据, 2023, 9(3): 140-149.
[7]	刘烨, 成伟, 李焱, 尹依梦, 孙慧杰. 元宇宙视域下教育社区构建研究[J]. 大数据, 2023, 9(1): 78-86.
[8]	贺亚运, 彭俊清, 王健宗, 肖京. 节奏舞者：基于关键动作转换图和有条件姿态插值网络的3D舞蹈生成方法研究[J]. 大数据, 2023, 9(1): 23-37.
[9]	彭一非, 袁贞, 张旭龙, 姜桂林, 刘逾江. 基于数字孪生技术的元宇宙空气污染物浓度推断模型[J]. 大数据, 2023, 9(1): 38-50.
[10]	王子航, 禹向群, 斯洪标, 傅思敏, 张旭龙, 彭绍亮. 基于算力网络的元宇宙分层处理模型设计[J]. 大数据, 2023, 9(1): 51-62.
[11]	朱锐, 王宏志, 崔双双, 张恺欣, 燕钰. 面向元宇宙的云边端协同大数据管理[J]. 大数据, 2023, 9(1): 63-77.
[12]	何波. 元宇宙的法律难题与规制思路研究[J]. 大数据, 2023, 9(1): 87-102.
[13]	沈阳, 余梦珑. 元宇宙与大数据：时空智能中的数据洞察与价值连接[J]. 大数据, 2023, 9(1): 103-110.
[14]	王陈慧子, 蔡玮. 元宇宙数字经济：现状、特征与发展建议[J]. 大数据, 2022, 8(3): 140-150.
[15]	赵智韬, 赵理君, 张正, 唐娉. 基于容器云技术的典型遥感智能解译算法集成[J]. 大数据, 2022, 8(2): 58-74.