大数据典型相关分析的云模型方法

doi:10.3969/j.issn.1000-436x.2013.10.015

Abstract

Abstract:

The complexity of traditional CCA methods is too high meet the requirements to analyze big data due to their huge scale which is reaching the level of peta-byte.A novel approach to CCA was proposed to mine the big data by introducing the cloud model which is a brand-nowel theory about the uncertainty artificial intelligence.A distributed ar-chitecture based on cloud computing was established.All of the clouds distributing on the nodes of the distributed archi-tecture were combined to a center cloud via cloud operation (whe cloud is a synopsis of data and which is a concept coming from the cloud theory).A type of virtual sample of data called cloud drops created based on the center cloud.Fi-nally the computing of CCA was imposed on the cloud drops.The CCA was impose on the cloud drops with less volume,which improves the efficiency.Experimental results on real data sets indicate the effectiveness of this method.

Key words: big data, canonical correlation analysis (CCA), cloud model, cloud operation, cloud computing

Jing YANG,Wen-ping LI,Jian-pei ZHANG. Canonical correlation analysis of big data based on cloud model[J]. Journal on Communications, 2013, 34(10): 121-134.

Figures/Tables 16

References 33

[1]	MINNESOTA M . Big data:science in the petabyte era[J]. Nature, 2008,455(7209): 1-136.
[2]	SAKAR C O , KURSUN O . A method for combining mutual informa-tion and canonical correlation analysis:predictive mutual information and its use in feature selection[J]. Expert Systems wi h Applications, 2012,39(3): 3333-3344.
[3]	OLCAY K , ETHEM A , OLEG V ,et al. Canonical correlation analysis using within-class coupling[J]. Pattern Recognition Letters, 2011,32(2): 134-144.
[4]	KAMALIKA C , SHAM M K , KAREN L ,et al. Multi-view clustering via canonical correlation analysis[A]. Proc of the 26th International Conference on Machine Learning[C]. New York,ACM,USA, 2009. 129-136.
[5]	杨静, 李文平, 张健沛 . 基于秩2更新的多维数据流典型相关跟踪算法[J]. 电子学报, 2012,40(9): 1765-1774. YANG J , LI W P , ZHANG J P . A tracking algorithm based on rank two modifications for canonical correlation analysis of mu idimensional data streams[J]. Acta Electronica Sinica, 2012,40(9): 1765-1774.
[6]	顾鑫, 徐正全, 刘进 . 基于云理论的可信研究及展望[J]. 通信学报, 2011,32(7): 176-181. GU X , XU Z Q , LIU J . Review of cloud based trust model[J]. Journal on Communications, 2011,32(7): 176-181.
[7]	黄海生, 王汝传 . 基于隶属云理论的主观信任评估模型研究[J]. 通信学报, 2008,29(4): 13-19. HUANG H S , WANG R C . Subjective trust evaluation model based on membership cloud theory[J]. Journal on Communications, 2008,29(4): 13-19.
[8]	蒋嵘, 李德毅 . 基于形态表示的时间序列相似性搜索[J]. 计算机研究与发展, 2000,37(5): 601-608. JIANG R , LI D Y . Similarity search based on shape representation in time-series data sets[J]. Journal of Computer Research ＆ Development, 2000,37(5): 601-608.
[9]	许凯, 秦昆, 黄伯和 ,等. 基于云模型的图像区域分割方法[J]. 中国图象图形学报, 2010,15(5): 757-763. XU K , QIN K , HUANG B H ,et al. A new method of region based on image segmentation based on cloud model[J]. Journal of Image and Grphics, 2010,15(5): 757-763.
[10]	HOTELLING H . Relations between two sets of variates[J]. Biometri-ka, 1936,28(3): 321-377.
[11]	彭岩, 张道强 . 半监督典型相关分析算法[J]. 软件学报, 2008,19(11): 2822-2832. PENG Y , ZHANG D Q . Semi-supervised canonical correlation analy-sis algorithm[J]. Journal of Software, 2008,19(11): 2822-2832.
[12]	顾晶晶, 陈松灿, 庄毅 . 用局部保持典型相关分析定位无线传感器网络节点[J]. 软件学报, 2010,21(11): 2883-2891. GU J J , CHEN S C , ZHUANG Y . Localization in wireless sensor net-work using locality preserving canonical correlation analysis[J]. Jour-nal of Software, 2010,21(11): 2883-2891.
[13]	LI D Y , HAN J W . Knowledge representation and discovery based on linguistic atoms[J]. Knowledge-based Systems, 1998,7(10): 431-440.
[14]	PHILIP R . Big Data Analytics[R]. TDWI Best Parctices Report, 2011.3-38.
[15]	BENJAMIN H B , MARK R B , KEITH A S ,et al. Large-scale elec-trophysiology:acquisition,comprression,encryption,storage of big data[J]. Journal of Neurosience Methods, 2009,180(1): 185-192.
[16]	ARONOVA E , BAKER K , ORESKES N . Big science and big dat in biology:from the international geophysical year through the international biological program to the long term ecological research (LTER)network[J]. Historical Studies in the Natural Sciences, 2010,40(8): 183-224.
[17]	WERNER C . Scienti?c perspectivism:a philosopher of science's response to the challenge of big data biology[J]. Studies in History and Philosophy of Biological and Biomedical Sciences, 2012,43(1): 69-80.
[18]	ALFREDO C , YEOL S , KAREN C D . Analytics over largescale mul-tidimensional data:the big data revolution[A]. Proc of the DOLAP'11[C]. Glasgow, 2011. 101-103.
[19]	STEVEN C H H , WANG J L , ZHAO P L ,et al. Online feature selec-tion for mining big data[A]. Proc of the Big-Mine'12[C]. New York:ACM,USA, 2012. 93-100.
[20]	SIMON B , DUODUO L . On clusterization of ''big data'' streams[A]. Proc of the 3rd International Conference on Computing Geospatial Research and Applications[C]. New York:ACM,USA, 2012. 1-6.
[21]	JOHN L . Parallel machine learning on big data[J]. XRDS, 2012,19(1): 60-62.
[22]	THOMAS C , PEGGY H , MELANIE M ,et al. Building a big data research program at a small university[J]. JCSC, 2012,28(2): 95-102.
[23]	YU C , CHENG J Q , FLORIN R . GLADE:big data analytics made easy[A]. Proc of the SIGMOD'12[C]. New York:ACM,USA, 2012. 697-700.
[24]	KYUSEOK S . MapReduce algorithms for big data analysis[A]. Proc of the 38th International Conference on Very Large Data Bases(VLDB)[C]. New York:ACM,USA, 2012. 2016-2017.
[25]	JENS D , JORGE A . Efficient big data processing in hadoop MapRe-duce[A]. Proc of the 38th International Conference on Very Large Data Bases(VLDB)[C]. New York:USA,ACM, 2012. 2014-2015.
[26]	DIVYAKANT A , SUDIPTO D , AMR E A . Big data and cloud compu-ting:current state and future opportunities[A]. Proc of the EDBT 2011[C]. New York:ACM,USA, 2011. 530-533.
[27]	XU H Q , LI Z , GUO S M ,et al. CloudVista:interactive and economi-cal visual cluster analysis for big data in the cloud[A]. Proc of the 38th International Conference on very Large Data Bases(VLDB)[C]. New York:USA,ACM, 2012. 1886-1889.
[28]	COLIN T , DIGITAL P . Big data security[J]. Network Security, 2012,7(2): 5-8.
[29]	SOTIRIS K . Combining bagging,boosting,rotation forest and random subspace methods[J]. Artificial Intelligence Review, 2011,35(3): 223-240.
[30]	李德毅, 杜鷁 . 不确定性人工智能[M]. 北京 : 国防工业出版社, 2005. 224-227. LI D Y , DU Y . Artificial Intelligence with Uncertainty[M]. Beijing: National Defence Industry PressPress, 2005. 224-227.
[31]	TAVALLAEE M , BAGHERI E , LU W ,et al. A detailed analysis of the KDD CUP 99 data set[A]. Proc of the Second IEEE International Conference on Computational Intelligence for Security and Defense Applications[C]. Ottawa,Canada, 2009. 53-58.
[32]	WANG Y L , ZHANG G X , QIAN J B . ApproxCCA:an approximate correlation analysis algorithm for multidimensional data streams[J]. Knowledge-Based Systems, 2011,24(7): 952-962.
[33]	SUN L , JI S W . Canonical correlation analysis for multilabel classifi-cation:a least-squares formulation,extensions,and analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intellige 2011,33(1): 194-200.

Metrics

Recommended 0

No Suggested Reading articles found!

组数						数据总容量（×1）0⁷
			第1典型系数						第2典型系数
		PAMAP2			IDS			PAMAP2			IDS
	BDCCA	ApproxCCA	LS-CCA	BDCCA	ApproxCCA	LS-CCA	BDCCA	ApproxCCA	LS-CCA	BDCCA	ApproxCCA	LS-CCA
1	0.098 6	0.067 5	0.031 9	0.125 0	0.077 0	0.020 5	0.064 8	0.030 1	0.009 6	0.125 1	0.086 6	0.083 0
2	0.119 2	0.136 8	0.092 5	0.136 9	0.108 3	0.070 8	0.059 7	0.049 7	0.035 0	0.136 4	0.090 8	0.086 6
3	0.119 3	0.157 4	0.105 3	0.132 6	0.155 1	0.109 7	0.087 2	0.082 6	0.051 4	0.141 9	0.136 2	0.122 5
4	0.121 0	0.170 9	0.153 7	0.157 5	0.211 5	0.159 0	0.096 6	0.109 0	0.082 0	0.145 3	0.182 6	0.174 5
5	0.137 1	0.268 6	0.213 0	0.166 2	0.315 9	0.219 3	0.102 1	0.116 7	0.095 2	0.153 3	0.231 8	0.194 2
6	0.127 2	0.272 9	0.225 9	0.172 3	0.353 3	0.255 0	0.099 7	0.213 6	0.113 7	0.162 4	0.336 3	0.279 8
7	0.129 6	0.287 0	0.228 9	0.164 7	0.366 2	0.273 3	0.121 6	0.231 9	0.153 1	0.159 5	0.351 9	0.301 0
8	0.133 5	0.336 0	0.270 1	0.157 5	0.423 9	0.287 1	0.113 9	0.297 5	0.163 4	0.161 9	0.396 0	0.311 6
9	0.124 9	0.352 1	0.284 2	0.160 5	0.432 7	0.333 2	0.111 8	0.319 9	0.178 7	0.163 0	0.433 9	0.329 2
10	0.145 9	0.408 7	0.314 2	0.173 9	0.438 6	0.408 1	0.116 9	0.365 1	0.196 0	0.179 7	0.444 5	0.342 7

Canonical correlation analysis of big data based on cloud model

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 16

References 33

Related Articles 15

Metrics

Recommended 0

[1]	Ling MA, Qiliang FAN, Ting XU, Guanchen GUO, Shenglin ZHANG, Yongqian SUN, Yuzhi ZHANG. Scheduling framework based on reinforcement learning in online-offline colocated cloud environment [J]. Journal on Communications, 2023, 44(6): 90-102.
[2]	Wei JIN, Fenghua LI, Mingjie YU, Yunchuan GUO, Ziyan ZHOU, Liang FANG. HDFS-oriented cryptographic key resource control mechanism [J]. Journal on Communications, 2022, 43(9): 27-41.
[3]	Huaqun WANG, Zhe LIU, Debiao HE, Jiguo LI. Identity-based provable data possession scheme for multi-source IoT terminal data in public cloud [J]. Journal on Communications, 2021, 42(7): 52-60.
[4]	Yimin MAO, Qianhu DENG, Zhigang CHEN. Parallel association rules incremental mining algorithm based on information entropy and genetic algorithm [J]. Journal on Communications, 2021, 42(5): 122-136.
[5]	Jianhong ZHANG, Menglong WU, Jing WANG, Pei LIU, Zhengtao JIANG, Changgen PENG. Secure and verifiable multi-keyword searchable encryption scheme in cloud [J]. Journal on Communications, 2021, 42(4): 139-149.
[6]	Ruiqi LI, Chunfu JIA, Yafei WANG. Multi-key homomorphic proxy re-encryption scheme based on NTRU and its application [J]. Journal on Communications, 2021, 42(3): 11-22.
[7]	Jiawei ZHANG, Jianfeng MA, Zhuo MA, Teng LI. Time-based and privacy protection revocable and traceable data sharing scheme in cloud computing [J]. Journal on Communications, 2021, 42(10): 81-94.
[8]	Wenjuan WANG, Xuehui DU, Dibin SHAN. Construction method of attack scenario in cloud environment based on dynamic probabilistic attack graph [J]. Journal on Communications, 2021, 42(1): 1-17.
[9]	Youliang TIAN,Qin LUO. Verifiable multi-keyword search scheme based on improved Merkle-Tree authentication method [J]. Journal on Communications, 2020, 41(9): 118-129.
[10]	Na WANG,Kun ZHENG,Junsong FU,Jian LI. Method of ciphertext retrieval in mobile edge computing based on block segmentation [J]. Journal on Communications, 2020, 41(7): 95-102.
[11]	Lindong ZHAO,Wenqin ZHUANG,Jianxin CHEN,Liang ZHOU. Hierarchical task offloading in heterogeneous cellular network:modeling and optimization [J]. Journal on Communications, 2020, 41(4): 34-44.
[12]	Liang YUAN, Xiao YU, Enjie DING, Xiaohu ZHAO, Shimin FENG, Da ZHANG, Tongyu LIU, Weidong WANG, Yanqiu HUANG. Research on key technologies of human-machine-environment states perception in mine Internet of things [J]. Journal on Communications, 2020, 41(2): 1-12.
[13]	Bing LIANG,Wen JI. Multiuser computation offloading for edge-cloud collaboration using submodular optimization [J]. Journal on Communications, 2020, 41(10): 25-36.
[14]	. Dual-architecture Internet supporting intelligent governance of cyber content [J]. Journal on Communications, 2019, 40(9): 1-14.
[15]	SU Mingfeng,WANG Guojun,LI Renfa. Multidimensional QoS cloud computing resource scheduling method based on stakeholder perspective [J]. Journal on Communications, 2019, 40(6): 102-115.