Traditional relational data model cannot meet the demand of big data applications for expressing and processing wide varieties of data.As a result,a number of non-relational data types have become popular in practice,among which JSONlike tree-structured data types have been widely adopted.Tree-structured data types have important theoretical and practical values.A systematic description of tree-structured data types was provided,and the way to efficiently support data analysis operations on tree-structured data was investigated.
In view of bottlenecks faced by big data analysis and application in China,such as weak big data mining and analysis ability,low big data algorithm application and comprehensive ability,the overall technical framework of big data analysis and application technology innovation platform was systematically introduced.The shortcomings and solutions of the five major common technologies in China's big data analysis and application were analyzed in detail.The design ideas and application directions of the four supporting platforms in the innovation platform were introduced.Finally,the future development direction and key work of the National Engineering Laboratory for Big Data Analysis and Application were introduced.
With the wide application of bar code,two-dimensional code,RFID,industrial sensor,automatic control system,industrial Internet,ERP and CAD/CAM/CAE techniques in industry,a large number of data related to industrial production are collected and stored in information system in real time.Analyzing those data can help to improve the production techniques,optimize the production process,reduce the production costs,laying the foundation for intelligent manufacturing.Therefore,the industrial big data analysis has drawn much attention of both industry and academia in recent years.Models and algorithms are two core issues of big data analysis theory and techniques.The concept of industrial big data analysis was introduced,and the applications of several popular models and the research results of the corresponding algorithms were reviewed,and future research directions in this area were explored.
In view of the bottlenecks in common technology and engineering practice faced by big data system software and applications development,the key technological innovations of the National Engineering Laboratory for Big Data System Software(NEL-BDSS) were introduced systematically,including the technical architecture and domain applications of the big data system software "Tsinghua Dataway Platform",which addresses issues such as:massive multi-source heterogeneous data integration management,interactive heterogeneous data analysis framework,data visualisation and intelligent data engineering,validation and verification of hybrid source big data software,and domain-specific big data applications development and run-time environment.The big data system software eco-system,state-of-the-art big data technology and systems,domain applications,as well as future challenges were summarized systematically.The NEL-BDSS focuses on supporting demonstrative applications of industrial big data,environmental big data as well as meteorological big data.
The rapid development of big data industry has posed a big challenge to cultivate big data talents.How to effectively integrate interdisciplinary and cross-disciplinary knowledge and build a teaching system for big data education is a problem facing the current major construction of big data majors in universities.From the perspective of emerging engineering education,the cultivating requirements of big data talents,curriculum system,teaching material system and practical educating system were discussed.Then some cases of integration of industry,government and university were introduced.Finally,some key points of major construction of big data were discussed.
The training of big data professionals is the foundation of a new round of scientific and technological contest in the world.Colleges and universities assume the responsibility of training big data talents.As a typical “new engineering” major,the major of big data is still in the exploratory stage in the construction of curriculum system.Firstly,the difficulties in the construction of big data courses were analyzed,and then the big data course system built by Xiamen University was introduced,including introductory courses,advanced courses and training courses.Also the experience and methods of the course construction of the principles and applications of big data technology were introduced,which includes course orientation,training objectives,preparatory knowledge,knowledge partitioning between big data and cloud computing courses,course content and arrangement,teaching material,experimental environment construction,matching resources construction,online service platform,offline training and communication,and so on.