基于边缘计算的多摄像头视频协同分析方法

doi:10.11959/j.issn.1000-436x.2023150

通信学报 ›› 2023, Vol. 44 ›› Issue (8): 14-26.doi: 10.11959/j.issn.1000-436x.2023150

• 学术论文 • 上一篇

基于边缘计算的多摄像头视频协同分析方法

期治博¹^,², 杜磊³, 霍如³^,⁴, 杨帆¹^,⁴, 黄韬¹^,⁴

¹ 北京邮电大学网络与交换国家重点实验室，北京 100876
² 中国信息通信研究院工业互联网与物联网研究所，北京 100083
³ 北京工业大学信息学部，北京 100124
⁴ 网络通信与安全紫金山实验室，江苏南京 211111

修回日期:2023-07-10 出版日期:2023-08-01 发布日期:2023-08-01
作者简介:期治博（1987- ），男，河南郑州人，北京邮电大学博士生，中国信息通信研究院工业互联网与物联网研究所工程师，主要研究方向为工业互联网、物联网和区块链领域的技术及产业开发等
杜磊（1995- ），男，山西晋中人，北京工业大学博士生，主要研究方向为工业互联网、边缘计算、物联网和深度学习算法等
霍如（1988- ），女，黑龙江哈尔滨人，博士，北京工业大学讲师，主要研究方向为未来网络、工业互联网、边缘计算、网络资源管理、区块链等
杨帆（1981- ），男，黑龙江哈尔滨人，博士，北京邮电大学工程师，主要研究方向为软件定义网络、高性能路由交换技术等
黄韬（1980- ），男，重庆人，博士，北京邮电大学教授，主要研究方向为未来网络体系架构、软件定义网络、网络虚拟化等
基金资助:
国家重点研发计划基金资助项目(2018YFB1800602);2020年工业互联网创新发展工程基金资助项目（工业互联网标识资源搜索系统）

Multi-camera video collaborative analysis method based on edge computing

Zhibo QI¹^,², Lei DU³, Ru HUO³^,⁴, Fan YANG¹^,⁴, Tao HUANG¹^,⁴

¹ State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
² Department of Industrial Internet Institute, China Academy of Information and Communication, Beijing 100083, China
³ Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China
⁴ Purple Mountain Laboratories, Nanjing 211111, China

Revised:2023-07-10 Online:2023-08-01 Published:2023-08-01
Supported by:
The National Key Research and Development Program of China(2018YFB1800602);The MIIT of China 2020 (Identification Re-source Search System for Industrial Internet of Things)

摘要/Abstract

摘要：

为了减少智慧城市场景下多摄像头实时视频数据的处理量，提出了基于机器学习算法的边缘端视频协同分析方法。首先，针对各摄像头检测到的重要目标物体，设计了不同的关键窗口来筛选视频的感兴趣区域，缩减视频数据量并提取其特征。然后，根据提取的数据特征，对不同摄像头视频中的相同目标物体进行标注，并设计了摄像头之间关联程度值的计算策略，用于进一步缩减视频数据量。最后，提出了基于图卷积网络和重识别技术的 GC-ReID 算法，旨在实现多摄像头视频协同分析。实验结果表明，与现有的视频分析方法相比，所提方法能够有效降低系统时延和提高视频压缩率，并保证较高的准确率。

关键词: 边缘计算, 机器学习, 视频协同分析, 感兴趣区域标注, 多摄像头关联性

Abstract:

In order to reduce the processing volume of multi-camera real-time video data in smart city scenarios, a video collaborative analysis method based on machine learning algorithms at the edge was proposed.Firstly, for the important objects detected by each camera, different key windows were designed to filter the region of interest (RoI) in the video, reduce the video data volume and extract its features.Then, based on the extracted data features, the same objects in the videos from different cameras were annotated, and a strategy for calculating the association degree value between cameras was designed for further reducing the video data volume.Finally, the GC-ReID algorithm based on graph convolutional network (GCN) and re-identification (ReID) was proposed, aiming at achieving the collaborative analysis of multi-camera videos.The experimental results show that proposed method can effectively reduce the system latency and improve the video compression rate while ensuring the high accuracy, compared with the existing video analysis methods.

Key words: edge computing, machine learning, video collaborative analysis, region of interest annotation, association between cameras

中图分类号:

TN919.85

期治博, 杜磊, 霍如, 杨帆, 黄韬. 基于边缘计算的多摄像头视频协同分析方法[J]. 通信学报, 2023, 44(8): 14-26.

Zhibo QI, Lei DU, Ru HUO, Fan YANG, Tao HUANG. Multi-camera video collaborative analysis method based on edge computing[J]. Journal on Communications, 2023, 44(8): 14-26.

图/表 8

图1

表1

边缘系统模型的参数和含义"

参数	含义
N	摄像头集合
Ms	主边缘服务器
M	直连边缘服务器集合
T	时间段，组成单元为时隙
$r_{n_{z}}$	摄像头n_z 的传输速率
$r_{m_{z}}$	直连边缘服务器m_z的传输速率
$W_{t}$	摄像头n_z 在t时隙的上行带宽
$P_{n_{z}}$	摄像头n_z 的传输功率
$G_{n_{z}}$	摄像头n_z 到直连边缘服务器的信道增益
$σ^{2}$	背景噪声方差
$D_{n_{z}}^{tran}$	摄像头n_z 到直连边缘服务器的传输时延
$D_{n_{z}}^{tran}$	直连边缘服务器n_z 到主边缘服务器或云服务器的传输时延
$D_{n_{z}}^{com}$	计算时延

表1

图2

表2

图2中区域块的集合"

时隙	检测到的目标物体	区域块集合
t	$L_{t} = {l_{1}, l_{2}, l_{3}, l_{4}, l_{5}, l_{6}, l_{7}, l_{8}, l_{9}, l_{10}}$	${RS}_{t}^{1} = {{R_{1, 9}, R_{1, 10}, R_{1, 13}, R_{1, 14}}, {R_{2, 1}, R_{2, 2}, R_{2, 5}, R_{2, 6}}}$
		${RS}_{t}^{2} = {{R_{1, 5}, R_{1, 6}, R_{1, 9}, R_{1, 10}, R_{1, 14}}, {R_{2, 2}, R_{2, 6}}}$
		${RS}_{t}^{3} = {{R_{1, 2}, R_{1, 6}}, {R_{2, 3}, R_{2, 4}, R_{2, 7}, R_{2, 8}}, {R_{4, 6}, R_{4, 10}, R_{4, 14}}}$
		${RS}_{t}^{4} = {{R_{1, 2}, R_{1, 6}}, {R_{2, 5}, R_{2, 7}}, {R_{4, 7}, R_{4, 11}}}$
		${RS}_{t}^{5} = {{R_{1, 2}, R_{1, 6}}, {R_{2, 5}, R_{2, 7}}}$
		${RS}_{t}^{6} = {{R_{2, 2}, R_{2, 6}}}$
		${RS}_{t}^{7} = {{R_{3, 5}, R_{3, 6}, R_{3, 9}, R_{3, 10}}, {R_{4, 7}, R_{4, 8}, R_{4, 11}, R_{4, 12}}}$
		${RS}_{t}^{8} = {{R_{3, 5}, R_{3, 9}}, {R_{4, 8}, R_{4, 12}}}$
		${RS}_{t}^{9} = {{R_{3, 7}, R_{3, 11}, R_{3, 15}}}$
		${RS}_{t}^{10} = {{R_{4, 2}, R_{4, 6}}}$

表2

图3

图4

图5

图6

参考文献 23

[7]	YANG P , LYU F , WU W ,et al. Edge coordinated query configuration for low-latency and accurate video analytics[J]. IEEE Transactions on Industrial Informatics, 2020,16(7): 4855-4864.
[8]	WANG C , ZHANG S , CHEN Y ,et al. Joint configuration adaptation and bandwidth allocation for edge-based real-time video analytics[C]// Proceedings of 2020 IEEE Conference on Computer Communications. Piscataway:IEEE Press, 2020: 257-266.
[9]	RAN X K , CHEN H , ZHU X D ,et al. DeepDecision:a mobile deep learning framework for edge video analytics[C]// Proceedings of IEEE Conference on Computer Communications. Piscataway:IEEE Press, 2018: 1421-1429.
[10]	HUNG C C , ANANTHANARAYANAN G , BODIK P ,et al. VideoEdge:processing camera streams using hierarchical clusters[C]// Proceedings of 2018 IEEE/ACM Symposium on Edge Computing (SEC). Piscataway:IEEE Press, 2018: 115-131.
[11]	GUO Y D , LIU Z Y , LUO H . Multi-person multi-camera tracking for live stream videos based on improved motion model and matching cascade[J]. Neurocomputing, 2022,492: 561-571.
[12]	CHUNG N M , LE H D A , NGUYEN V A ,et al. Multi-camera multi-vehicle tracking with domain generalization and contextual constraints[C]// Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Piscataway:IEEE Press, 2022: 3326-3336.
[13]	LI F , WANG Z , NIE D ,et al. Multi-camera vehicle tracking system for AI city challenge 2022[C]// Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Piscataway:IEEE Press, 2022: 3264-3272.
[14]	LIU Y M , ZHANG X C , ZHANG B Z ,et al. Multi-camera vehicle tracking based on occlusion-aware and inter-vehicle information[C]// Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Piscataway:IEEE Press, 2022: 3256-3263.
[15]	LI Y Q , PADMANABHAN A , ZHAO P Z ,et al. Reducto:on-camera filtering for resource-efficient real-time video analytics[C]// Proceedings of the Annual Conference of the ACM Special Interest Group on Data Communication on the Applications,Technologies,Architectures,and Protocols for Computer Communication. New York:ACM Press, 2020: 359-376.
[16]	RAY D , KOSAIAN J , RASHMI K V ,et al. Vantage:optimizing video upload for time-shifted viewing of social live streams[C]// Proceedings of the ACM Special Interest Group on Data Communication. New York:ACM Press, 2019: 380-393.
[17]	DU L , HUO R . Real-time video transmission optimization based on edge computing in IIoT[C]// Proceedings of 2021 IEEE 29th International Conference on Network Protocols (ICNP). Piscataway:IEEE Press, 2021: 1-6.
[1]	JIANG J , ANATHANARAYANAN G , BODIK P ,et al. Chameleon:scalable adaptation of video analytics[C]// Proceedings of 2018 ACM SIGCOMM. New York:ACM Press, 2018: 253-266.
[2]	黄韬, 刘江, 汪硕 ,等. 未来网络技术与发展趋势综述[J]. 通信学报, 2021,42(1): 130-150.
	HUANG T , LIU J , WANG S ,et al. Survey of the future network technology and trend[J]. Journal on Communications, 2021,42(1): 130-150.
[3]	YANG K , YI J , LEE K ,et al. FlexPatch:fast and accurate object detection for on-device high-resolution live video analytics[C]// Proceedings of IEEE Conference on Computer Communications. Piscataway:IEEE Press, 2022: 1898-1907.
[4]	赵羽, 杨洁, 刘淼 ,等. 面向视频监控基于联邦学习的智能边缘计算技术[J]. 通信学报, 2020,41(10): 109-115.
	ZHAO Y , YANG J , LIU M ,et al. Federated learning based intelligent edge computing technique for video surveillance[J]. Journal on Communications, 2020,41(10): 109-115.
[5]	GAO C S , WANG Y , HAN Y H ,et al. IVP:an intelligent video processing architecture for video streaming[J]. IEEE Transactions on Computers, 2023,72(1): 264-277.
[6]	CHEN Y , ZHANG S , JIN Y B ,et al. Multi-server multi-user game at edges for heterogeneous video analytics[C]// Proceedings of 2022 IEEE International Conference on Communications. Piscataway:IEEE Press, 2022: 841-846.
[18]	RANADHEERA S , MAGHSUDI S , HOSSAIN E . Minority games with applications to distributed decision making and control in wireless networks[J]. IEEE Wireless Communications, 2017,24(5): 184-192.
[19]	SCARSELLI F , GORI M , TSOI A C ,et al. The graph neural network model[J]. IEEE Transactions on Neural Networks, 2009,20(1): 61-80.
[20]	WANG Y , SUN Y B , LIU Z W ,et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2018,38(5): 1-12.
[21]	LI P , ZHANG J B , ZHU Z ,et al. State-aware re-identification feature for multi-target multi-camera tracking[C]// Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Piscataway:IEEE Press, 2020: 1506-1516.
[22]	ALAMEDA-PINEDA X , STAIANO J , SUBRAMANIAN R ,et al. SALSA:a novel dataset for multimodal group behavior analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016,38(8): 1707-1720.
[23]	REDMON J , FARHADI A . YOLOv3:an incremental improvement[J]. arXiv Preprint,arXiv:1804.02767, 2018.

基于边缘计算的多摄像头视频协同分析方法

Multi-camera video collaborative analysis method based on edge computing

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 23

相关文章 15

Metrics

推荐阅读 0

[1]	唐冬, 黄栩蔚, 罗至威, 赵赛, 黄高飞. 无线供能智能反射面辅助移动边缘计算系统设计与优化[J]. 通信学报, 2023, 44(9): 79-92.
[2]	陈晓霖, 昝道广, 吴炳潮, 关贝, 王永吉. 面向纵向联邦学习的对抗样本生成算法[J]. 通信学报, 2023, 44(8): 1-13.
[3]	巩小雪, 庞嘉豪, 张琦涵, 徐长乐, 秦文帅, 郭磊. 基于机器学习的光网络干扰攻击检测、识别与恢复方法[J]. 通信学报, 2023, 44(7): 159-170.
[4]	谢人超, 文雯, 唐琴琴, 刘云龙, 谢高畅, 黄韬. 轨道交通移动边缘计算网络安全综述[J]. 通信学报, 2023, 44(4): 201-215.
[5]	苏新, 张桂福, 行鸿彦, Zenghui Wang. 基于平衡生成对抗网络的海洋气象传感网入侵检测研究[J]. 通信学报, 2023, 44(4): 124-136.
[6]	鲁蔚锋, 李宁, 徐佳, 徐力杰, 徐建. 多接入边缘计算中相关性任务的联合调度算法[J]. 通信学报, 2023, 44(4): 87-98.
[7]	戴千一, 张斌, 郭松, 徐开勇. 基于多分类器集成的区块链网络层异常流量检测方法[J]. 通信学报, 2023, 44(3): 66-80.
[8]	余雪勇, 邱礼翔, 宋家宁, 朱洪波. 无人机辅助边缘计算中安全通信与能效优化策略[J]. 通信学报, 2023, 44(3): 45-54.
[9]	赵庶旭, 韦萍, 王小龙. 多任务并发边缘计算环境中最优联盟结构生成策略[J]. 通信学报, 2023, 44(2): 172-184.
[10]	范伟, 彭诚, 朱大立, 王雨晴. 移动边缘计算网络下基于静态贝叶斯博弈的入侵响应策略研究[J]. 通信学报, 2023, 44(2): 70-81.
[11]	龙隆, 刘子辰, 陆在旺, 张玉成, 李蕾. 移动边缘网络下服务缓存与资源分配联合优化策略[J]. 通信学报, 2023, 44(1): 64-74.
[12]	张宇, 程旻. NDN中边缘计算与缓存的联合优化[J]. 通信学报, 2022, 43(8): 164-175.
[13]	王子园, 杜瑞忠. 边缘环境下基于无证书公钥密码的数据完整性审计方案[J]. 通信学报, 2022, 43(7): 62-72.
[14]	莫梓嘉, 高志鹏, 杨杨, 林怡静, 孙山, 赵晨. 面向车联网数据隐私保护的高效分布式模型共享策略[J]. 通信学报, 2022, 43(4): 83-94.
[15]	王继锋, 王国峰. 边缘计算模式下密文搜索与共享技术研究[J]. 通信学报, 2022, 43(4): 227-238.