Detecting popular subjects and optimizing microblog performance using the Hybrid Hadoop Framework

Abstract

Active users of social media networks all over may dynamically create intolerable stuff. Big data helps to sustain the enormous volume of information on social media channels. Big data is advanced on Hadoop-based cloud systems using its fault-tolerance and dependability. Hadoop is the basic platform for big data analytics. Using Hadoop has a main disadvantage in terms of handling the enormous number of configuration metrics handling. Driven by cloud-based Apache Spark, the hybrid Hadoop Framework is proposed in this paper to enhance big data processing by means of key parameter regulation including workload, response time, network bandwidth, and the hot topic detection mechanism especially tailored for the microblog into the big data. To manage the big volumes, we deliberately construct the MapReduce jobs to precisely identify hot subjects. According to the experimental results, the proposed system's accuracy is quite high when compared to related methods.