结合CRF与ShapeBM形状先验的图像标记

doi:10.11959/j.issn.1000-0801.2017004

电信科学 ›› 2017, Vol. 33 ›› Issue (1): 67-76.doi: 10.11959/j.issn.1000-0801.2017004

结合CRF与ShapeBM形状先验的图像标记

王浩,郭立君(),王亚东,张荣

宁波大学信息科学与工程学院，浙江宁波 315211

修回日期:2016-09-30 出版日期:2017-01-01 发布日期:2017-06-04
作者简介:王浩（1992-），男，宁波大学信息科学与工程学院硕士生，主要研究方向为计算机视觉与模式识别。|郭立君（1970-），男，博士，宁波大学教授，主要研究方向为计算机视觉与模式识别、移动互联网及其应用。|王亚东（1990-），男，宁波大学信息科学与工程学院硕士生，主要研究方向为计算机视觉与模式识别。|张荣（1974-），女，博士，宁波大学副教授，主要研究方向为计算机视觉与信息安全。
基金资助:
国家自然科学基金资助项目(61175026);浙江省自然科学基金资助项目(LY17F030002);宁波市自然科学基金资助项目(2014A610031);浙江省“信息与通信工程”重中之重学科开放基金资助项目(xkxl1516);浙江省“信息与通信工程”重中之重学科开放基金资助项目(xkxl1521)

CRF combined with ShapeBM shape priors for image labeling

Hao WANG,Lijun GUO(),Yadong WANG,Rong ZHANG

School of Electrical Engineering and Computer Science,Ningbo University,Ningbo 315211,China

Revised:2016-09-30 Online:2017-01-01 Published:2017-06-04
Supported by:
The National Natural Science Foundation of China(61175026);Zhejiang Provincial Natural Science Foundation of China(LY17F030002);Ningbo Municipal Natural Science Foundation(2014A610031);Open Research Fund of Zhejiang First-Foremost Key Subject-Information and Communications Engineering of China(xkxl1516);Open Research Fund of Zhejiang First-Foremost Key Subject-Information and Communications Engineering of China(xkxl1521)

摘要/Abstract

摘要：

条件随机场（CRF）是一种强大的图像标记模型，适合描述图像相邻区域间（例如超像素）的相互作用。然而，CRF没有考虑标记对象的全局约束。对象的整体形状可以作为对象标记的一种全局约束，利用形状玻尔兹曼机（ShapeBM）在建模对象的整体形状方面的优势，提出了一种CRF与ShapeBM相结合的标记模型。标记模型建立在超像素的基础上，并通过pooling技术在CRF的超像素层与ShapeBM的输入层间建立对应关系，增强了 CRF 与 ShapeBM 结合的有效性，提高了标记准确率。在 Penn-Fudan Pedestrians 数据集和 Caltech-UCSD Birds 200数据集上的实验结果表明，联合模型明显地改善了标记结果。

关键词: 条件随机场, 形状玻尔兹曼机, 联合模型, 超像素, 图像标记

Abstract:

Conditional random field (CRF) is a powerful model for image labeling,it is particularly well-suited to model local interactions among adjacent regions (e.g.superpixels).However,CRF doesn't consider the global constraint of objects.The overall shape of the object is used as a global constraint,the ShapeBM can be taken advantage of modeling the global shape of object,and then a new labeling model that combined the above two types of models was presented.The combination of CRF and ShapeBM was based on the superpixels,through the pooling technology was wed to establish the corresponding relationship between the CRF superpixel layer and the ShapeBM input layer.It enhanced the effectiveness of the combination of CRF and ShapeBM and improved the accuracy of the labeling.The experiments on the Penn-Fudan Pedestrians dataset and Caltech-UCSD Birds 200 dataset demonstrate that the model is more effective and efficient than others.

Key words: CRF, ShapeBM, join model, superpixels, image labeling

中图分类号:

TP391

王浩,郭立君,王亚东,张荣. 结合CRF与ShapeBM形状先验的图像标记[J]. 电信科学, 2017, 33(1): 67-76.

Hao WANG,Lijun GUO,Yadong WANG,Rong ZHANG. CRF combined with ShapeBM shape priors for image labeling[J]. Telecommunications Science, 2017, 33(1): 67-76.

图/表 8

图1

图2

图3

图4

表1

表2

图5

图6

参考文献 26

[1]	LAFFERTY J , MCCALLUM A , PEREIRAF C , et al. Conditional random fields:probabilistic models for segmenting and labeling sequence data [C]// ICML,June 28-July 1,2001,Williams College,UK. New Jersey: IEEE Press, 2001.
[2]	BORENSTEIN E , SHARON E , ULLMAN S , et al. Combining top-down and bottom-up segmentation [C]//Conference on Computer Vision and Pattern Recognition,June 27-July 2,2004,Washington,DC,USA. New Jersey: IEEE Press, 2004.
[3]	HE X , ZEMEL R S , CARREIRAPERPINAN M A , et al. Multi-scale conditional random fields for image labeling[C]//Conference on Computer Vision and Pattern Recognition,June 27-July 2,2004,Washington,DC,USA. New Jersey: IEEE Press, 2004.
[4]	HE X , ZEMEL R S , RAY D , et al. Learning and incorporating top-down cues in image segmentation[C]// European Conference on Computer Vision,May 7-13,2006,Graz,Austria. New Jersey: IEEE Press, 2006:338-351.
[5]	SHOTTON J , WINN J , ROTHER C , et al. Texton boost for image understanding:multi-class object recognition and segmentation by jointly modeling texture,layout,and context[J]. International Journal of Computer Vision, 2009,81(1):2-23.
[6]	ZHANG L . A unified probabilistic graphical model and its application to image segmentation[J]. Rensselaer Polytechnic Institute, 2009(3).
[7]	SMOLENSKY P . Information processing in dynamical systems:foundations of harmony theory[M]. Cambride: MIT Press, 1986(1):194-281.
[8]	SALAKHUTDINOV R , HINTON G E . Deep Boltzmann machines[J]. Journal of Machine Learning Research, 2009,5(2):1967-2006.
[9]	ESLAMI S M , HEESS N , WILLIAMS C K , et al. The shape Boltzmann machine:a strong model of object shape[J]. International Journal of Computer Vision, 2014,107(2):155-176.
[10]	KAE A , SOHN K , LEE H , et al. Augmenting CRFs with Boltzmann machine shape priors for image labeling [C]//Conference on Computer Vision and Pattern Recognition,June 23-28,2013,Portland,Oregon,USA. New Jersey: IEEE Press, 2013.
[11]	KAE A , MARLIN B M , LEARNEDMILLER E G , et al. The shape-time random field for semantic video labeling [C]//Conference on Computer Vision and Pattern Recognition,June 23-28,2014,Columbus,OH,USA. New Jersey: IEEE Press, 2014.
[12]	CHEN F , YU H , HU R , et al. Deep learning shape priors for object segmentation [C]//Conference on Computer Vision and Pattern Recognition,June 23-28,2013,Portland,Oregon,USA. New Jersey: IEEE Press, 2013.
[13]	CREMERS D , SCHMIDT F R , BARTHEL F , et al. Shape priors in variational image segmentation:Convexity,Lipschitz continuity and globally optimal solutions [C]//Conference on Computer Vision and Pattern Recognition,June 24-26,2008,Anchorage,Alaska,USA. New Jersey: IEEE Press, 2008.
[14]	ARBELAEZ P , HARIHARAN B , GU C , et al. Semantic segmentation using regions and parts [C]//Conference on Computer Vision and Pattern Recognition,June 16-21,2012,Providence,RI,USA. New Jersey: IEEE Press, 2012.
[15]	MURPHY K , WEISS Y , JORDAN M I , et al. Loopy beliefpropagation for approximate inference:an empirical study[C]//15th Conference on Uncertainty in Artificial Intelligence,July 30-August 1,1999,Stockholm,Sweden. New Jersey: IEEE Press, 1999.
[16]	SCHMIDT M . minFunc:unconstrained differentiable multivariate optimization in Matlab[EB/OL]. 2016-07-10.
[17]	SAUL L K , JAAKKOLA T S , JORDAN M I , et al. Mean field theory for sigmoid belief networks[J]. Journal of Artificial Intelligence Research, 1996(13).
[18]	MNIH V , LAROCHELLE H , HINTON G E , et al. Conditional restricted Boltzmann machines for structured output prediction[C]//Conference on Uncertainty in Artificial Intelligence,Aug 15-17,2012,Catalina Island,USA. New Jersey: IEEE Press, 2012.
[19]	WANG L , SHI J , SONG G , et al. Object detection combining recognition and segmentation[C]//Asian Conference on Computer Vision,November 18-22,2007,Tokyo,Japan. New Jersey: IEEE Press, 2007.
[20]	WELINDER P , BRANSON S , MITA T , et al. Caltech-UCSD Birds 200[J]. California Institute of Technology, 2010.
[21]	YANG J , SAFAR S , YANG M H . Max-margin Boltzmann machines for object segmentation [C]//IEEE Conference on Computer Vision and Pattern Recognition,June 23-28,2014,Columbus,OH,USA. New Jersey: IEEE Press, 2014.
[22]	ACHANTA R P , SHAJI A , SMITH K M , et al. SLIC superpixels compared to state-of-the-art superpixelmethods[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012,34(11):2274-2282.
[23]	MALIK J , BELONGIE S , SHI J , et al. Textons,contours and regions:cue integration in image segmentation [C]//ICCV,September 20-25,1999,Kerkyra,Corfu,Greece. New Jersey: IEEE Press, 1999.
[24]	MARTIN D R , FOWLKES C C , MALIK J , et al. Learning to detect natural image boundaries using brightness and texture[C]//Conference on Neural Information Processing Systems,December 8-13,2003,Providence,USA. New Jersey: IEEE Press, 2003.
[25]	HUANG G B , NARAYANA M , LEARNEDMILLER E G , et al. Towards unconstrained face recognition [C]//Conference on Computer Vision and Pattern Recognition,June 24-26,2008,Anchorage,Alaska,USA. New Jersey: IEEE Press, 2008.
[26]	COHN T . Efficient inference in large conditional random Fields[M]. Berlin: Springer, 2006:606-613.

方法	超像素标记准确率	误识率
CRF	83.97%	参照
SCRF	86.31%	14.60%
ShapeBM	85.01%	6.49%
本文方法	87.90%	24.52%

方法	超像素准确率	误识率
CRF	78.85%	参照
SCRF	81.93%	14.60%
ShapeBM	81.27%	11.44%
本文方法	83.34%	21.23%

结合CRF与ShapeBM形状先验的图像标记

CRF combined with ShapeBM shape priors for image labeling

在线阅读

PDF下载

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 26

相关文章 15

Metrics

推荐阅读 0

[1]	金宏辉, 简志华, 杨曼, 吴超. 采用圆周局部三值模式纹理特征的合成语音检测方法[J]. 电信科学, 2023, 39(6): 85-95.
[2]	马辉, 王瑞琴, 杨帅. 一种渐进式增长条件生成对抗网络模型[J]. 电信科学, 2023, 39(6): 105-113.
[3]	卢敏, 胡娟, 张先超, 丁伟健, 乐光学. 基于用户多特征融合的个性化推荐模型[J]. 电信科学, 2023, 39(5): 101-115.
[4]	张永, 刘纪奎, 柯文龙. 基于并行可分离卷积和标签平滑正则化的脑电情感识别[J]. 电信科学, 2023, 39(5): 116-128.
[5]	邓琨, 蒋庆丰, 刘星妍. 融合节点分析与边分析的复杂网络社区识别算法[J]. 电信科学, 2023, 39(4): 87-100.
[6]	冶莉娟, 王亦婷, 朱励程. 基于细胞自动机模型电力网络攻击预测技术[J]. 电信科学, 2023, 39(4): 173-179.
[7]	韩一士, 徐雨欣, 卢甜甜. 一种基于耦合网络的RD-IHSAT网络谣言传播模型[J]. 电信科学, 2023, 39(2): 118-131.
[8]	徐嘉, 简志华, 金宏辉, 吴超, 游林, 吴迎笑. 基于中心对称局部二值模式的合成伪装语音检测方法[J]. 电信科学, 2023, 39(1): 72-78.
[9]	任华健, 郝秀兰, 徐稳静. 融合递增词汇选择的深度学习中文输入法[J]. 电信科学, 2022, 38(12): 56-64.
[10]	周薇娜, 刘露. 复杂场景下多尺度船舶实时检测方法[J]. 电信科学, 2022, 38(10): 67-78.
[11]	金楠, 王瑞琴, 陆悦聪. 基于艾宾浩斯遗忘曲线和注意力机制的推荐算法[J]. 电信科学, 2022, 38(10): 89-97.
[12]	杨帅, 王瑞琴, 马辉. 基于多通道的边学习图卷积网络[J]. 电信科学, 2022, 38(9): 95-104.
[13]	赵东明. 电信运营商知识图谱技术体系研究及应用实践[J]. 电信科学, 2022, 38(8): 151-162.
[14]	于佳祺, 简志华, 徐嘉, 游林, 汪云路, 吴超. 基于联合特征与随机森林的伪装语音检测[J]. 电信科学, 2022, 38(6): 91-99.
[15]	申情, 郭文宾, 楼俊钢, 余强国. 考虑多层次潜在特征的个性化推荐模型[J]. 电信科学, 2022, 38(2): 71-83.