Abstract: With the rapid growth of internet data, system architecture based on distributed platform to effectively crawl was proposed and the data was analyzed. The architecture consists of three modules, crawler module, Web module and distributed platform module. Among them, the crawler module is responsible for data collection, Web module processes the simple job and gives a visual display for the analysis result, and the distributed platform module is for data storage and complicated job computing. The combination of three modules provides an excellent solution for mass data collection, storage and analysis on the internet. The effectiveness of the proposed framework was verified in the development of a public opinion mining system.