Abstract:

The signaling data of telecom operators,which have a wide range of applications and great of values,is big data in real sense.In order to serve the real-time marketing scenarios better,the real-time processing and analysis ability of signaling data must be improved effectively.Firstly,the scheme of signaling collection and processing based on FKS was introduced,which combined the advantages of components such as Flume,Kafka and Spark Streaming,and played an important supporting role for operators to build real-time decision center.Furthermore,the Kafka monitoring program for topic consumption of consumer groups was proposed,which provided effective means for warning faults of real-time flow collection.The monitoring solution could effectively reduce the message queue congestion,avoid the risk of data loss,and also could obtain good application effects in practice.

Key words: real-time streaming, monitoring, Kafka, big data, stream computing