VR全景视频时空切片传输的流量评估与建模

doi:10.11959/j.issn.1000-436x.2022139

摘要/Abstract

摘要：

为了刻画虚拟现实（VR）全景视频的传输流量需求，针对时空切片传输模式，首先，基于VR全景视频观看数据集，对传输流量评估与空间切块方式和时间分段方式之间的关系进行仿真。然后，基于仿真结果，对VR 全景视频的传输流量进行统计建模，建立传输流量与空间切块数量和时间分段长度之间的函数关系。最后，基于得到的传输流量模型，提出一种VR全景视频时空切片方式的嵌套优化方法，其外层对空间切块数量进行遍历寻优，内层直接计算最优的时间分段长度。研究和仿真结果表明，传输流量近似服从正态分布，其均值和方差与时空切片方式有关；通过对不同类型的视频分别进行传输流量统计建模，基于所建立的流量模型优化得到的时空切片方式与通过仿真搜索得到的最优切片方式性能接近。

关键词: 虚拟现实, 全景视频, 流量模型, 时空切片

Abstract:

To characterize the traffic load of virtual reality (VR) panoramic video under the spatio-temporal slicing transmission mode, firstly, based on a real-word VR video viewing dataset, the relationship between the traffic load and the spatio-temporal slicing strategy was evaluated via simulations.Secondly, on the basis of the simulation results, a statistical traffic model was established, describing the interaction between the traffic load and the number of spatial tiles as well as the duration of temporal chunks.Finally, based on the established model, a hierarchical spatial-temporal slicing strategy optimization method was proposed, where the outer layer found the optimal number of spatial tiles by exhaustive searching while the inner layer could directly compute the optimal duration of temporal chunks.Research and simulation results show that the traffic load approximately follows a normal distribution, where the mean and variance are functions of the spatio-temporal slicing strategy.By establishing the traffic model for each type of videos, the optimized slicing strategy based on the established model performs close to the optimal strategy obtained by searching by simulations.

Key words: virtual reality, panoramic video, traffic model, spatio-temporal slicing

中图分类号:

TN92

韩圣千, 娄函, 王君来. VR全景视频时空切片传输的流量评估与建模[J]. 通信学报, 2022, 43(7): 102-112.

Shengqian HAN, Han LOU, Junlai WANG. Traffic evaluation and modeling for spatio-temporal slicing based VR panoramic video transmission[J]. Journal on Communications, 2022, 43(7): 102-112.

图/表 18

图1

表1

图2

图3

图4

图5

图6

图7

图8

表2

表3

图9

图10

图11

图12

图13

图14

图15

参考文献 30

[1]	华为iLab. 面向VR业务的承载网络需求白皮书[R]. 2016.
	HUAWEI iLab. White paper on bearer network requirements for VR services[R]. 2016.
[2]	3GPP. Virtual reality (VR) media services over 3GPP:TR 26.918[S]. 2017.
[3]	ZINK M , SITARAMAN R , NAHRSTEDT K . Scalable 360° video stream delivery:challenges,solutions,and opportunities[J]. Proceedings of the IEEE, 2019,107(4): 639-650.
[4]	HOSSEINI M , SWAMINATHAN V . Adaptive 360 VR video streaming based on MPEG-DASH SRD[C]// Proceedings of 2016 IEEE International Symposium on Multimedia. Piscataway:IEEE Press, 2016: 407-408.
[5]	GADDAM V R , RIEGLER M , EG R ,et al. Tiling in interactive panoramic video:approaches and evaluation[J]. IEEE Transactions on Multimedia, 2016,18(9): 1819-1831.
[6]	MISRA K , SEGALL A , HOROWITZ M ,et al. An overview of tiles in HEVC[J]. IEEE Journal of Selected Topics in Signal Processing, 2013,7(6): 969-977.
[7]	ALFACE P R , MACQ J F , VERZIJP N . Interactive omnidirectional video delivery:a bandwidth-effective approach[J]. Bell Labs Technical Journal, 2012,16(4): 135-147.
[8]	ZARE A , AMINLOU A , HANNUKSELA M M ,et al. HEVC-compliant tile-based streaming of panoramic video for virtual reality applications[C]// Proceedings of the 24th ACM International Conference on Multimedia. New York:ACM Press, 2016: 601-605.
[9]	TU Z Z , ZONG T Y , XI X L ,et al. Content adaptive tiling method based on user access preference for streaming panoramic video[C]// Proceedings of 2018 IEEE International Conference on Consumer Electronics. Piscataway:IEEE Press, 2018: 1-4.
[10]	NGUYEN D V , TRAN H T T , THANG T C . An evaluation of tile selection methods for viewport-adaptive streaming of 360-degree video[J]. ACM Transactions on Multimedia Computing,Communications,and Applications, 2020,16(1): 1-24.
[11]	SINGLA A , G?RING S , RAAKE A ,et al. Subjective quality evaluation of tile-based streaming for omnidirectional videos[C]// Proceedings of the 10th ACM Multimedia Systems Conference. New York:ACM Press, 2019: 232-242.
[12]	AN C L , YEN S C , HUANG C Y ,et al. On the optimal encoding ladder of tiled 360° videos for head-mounted virtual reality[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021,31(4): 1632-1647.
[13]	CHEN X L , WU D , AHMAD I . Optimized viewport-adaptive 360-degree video streaming[J]. CAAI Transactions on Intelligence Technology, 2021,6(3): 347-359.
[14]	BETHANABHOTLA D , CAIRE G , NEELY M J . WiFlix:adaptive video streaming in massive MU-MIMO wireless networks[J]. IEEE Transactions on Wireless Communications, 2016,15(6): 4088-4103.
[15]	HAN S Q , SU H T , YANG C Y ,et al. Proactive edge caching for video on demand with quality adaptation[J]. IEEE Transactions on Wireless Communications, 2020,19(1): 218-234.
[16]	SUN Y P , CHEN Z Y , TAO M X ,et al. Communications,caching,and computing for mobile virtual reality:modeling and tradeoff[J]. IEEE Transactions on Communications, 2019,67(11): 7573-7586.
[17]	WEI X , YANG C Y , HAN S Q . Prediction,communication,and computing duration optimization for VR video streaming[J]. IEEE Transactions on Communications, 2021,69(3): 1947-1959.
[18]	XU M , LI C , ZHANG S Y ,et al. State-of-the-art in 360° video/image processing:perception,assessment and compression[J]. IEEE Journal of Selected Topics in Signal Processing, 2020,14(1): 5-26.
[19]	WU C L , TAN Z H , WANG Z ,et al. A dataset for exploring user behaviors in VR spherical video streaming[C]// Proceedings of the 8th ACM on Multimedia Systems Conference. New York:ACM Press, 2017: 193-198.
[20]	YAQOOB A , MUNTEAN G M . A combined field-of-view prediction-assisted viewport adaptive delivery scheme for 360° videos[J]. IEEE Transactions on Broadcasting, 2021,67(3): 746-760.
[21]	BAN Y X , XIE L , XU Z M ,et al. CUB360:exploiting cross-users behaviors for viewport prediction in 360 video adaptive streaming[C]// Proceedings of 2018 IEEE International Conference on Multimedia and Expo. Piscataway:IEEE Press, 2018: 1-6.
[22]	XIE L , ZHANG X G , GUO Z M . CLS:a cross-user learning based system for improving QoE in 360-degree video adaptive streaming[C]// Proceedings of the 26th ACM International Conference on Multimedia. New York:ACM Press, 2018: 564-572.
[23]	ZHANG Y X , ZHAO P Y , BIAN K G ,et al. DRL360:360-degree video streaming with deep reinforcement learning[C]// Proceedings of IEEE Conference on Computer Communications. Piscataway:IEEE Press, 2019: 1252-1260.
[24]	NGUYEN A , YAN Z S , NAHRSTEDT K . Your attention is unique:detecting 360-degree video saliency in head-mounted display for head movement prediction[C]// Proceedings of the 26th ACM International Conference on Multimedia. New York:ACM Press, 2018: 1190-1198.
[25]	FFMPEG Team . FFmpeg software[EB]. 2022.
[26]	RICHARDSON I E . G.H.264/MPEG-4 part 10 white paper[R]. 2003.
[27]	BENTO4. Bento4 MP4,DASH,HLS,CMAF SDK and tools[EB]. 2020.
[28]	PERFECTO C , ELBAMBY M S , SER J D ,et al. Taming the latency in multi-user VR 360°:a QoE-aware deep learning-aided multicast framework[J]. IEEE Transactions on Communications, 2020,68(4): 2491-2508.
[29]	FEI Z S , WANG F , WANG J ,et al. QoE evaluation methods for 360-degree VR video transmission[J]. IEEE Journal of Selected Topics in Signal Processing, 2020,14(1): 78-88.
[30]	ZHENG C , LIU S H , HUANG Y M ,et al. Hybrid policy learning for energy-latency tradeoff in MEC-assisted VR video service[J]. IEEE Transactions on Vehicular Technology, 2021,70(9): 9006-9021.

视频编号	类型	t= 2	t= 3	t=4	t= 5	t=6
视频2	运动类	0.77%	9.28%	5.70%	2.86%	11.92%
视频5	运动类	0.58%	9.11%	4.87%	2.52%	10.48%
视频8	运动类	0.36%	9.55%	4.04%	1.79%	11.54%
视频1	表演类	0.33%	4.81%	2.97%	1.10%	5.84%
视频4	表演类	0.31%	4.40%	2.67%	1.07%	5.37%
视频7	表演类	0.23%	6.87%	2.98%	1.39%	8.61%

系数	数值	系数	数值
p₀	-0.15	p_{μ ,5}	3.39×10^-4
p₁	3.723	p_{μ ,6}	-2.68×10^-5
p_{μ ,-3}	7.217	p₇	-1.97×10^-4
p_{μ ,-2}	-17.17	p₈	0.01
p_{μ ,-}1	14.81	p₉	0.077
p_{μ ,0}	-4.62	p₁₀	0.32
p_{μ ,1}	0.81	p₁₁	-0.41
p_{μ ,2}	-0.066	p₁₂	1.48×10^-4
p_{μ ,3}	0.002	p₁₃	2.41×10^-5
p_{μ ,4}	3.69×10^-4	—	—

切块参数t	数值/KB
1	763
2	766
3	774
4	777
5	778
6	788
7	805
8	803
9	806
10	808
11	828