Twitter announced that in September, will make open source the Storm, his stream-processing framework for the Conference Strange Loop. The Storm was developed by BackType, a company acquired by Twitter in July.
The Storm is designed to bring the power of distributed processing to the processing of real-time streams of data. Although superficially similar to systems such as Hadoop MapReduce, the Storm is different. Hadoop systems are oriented toward discrete tasks, with the work sent to the network to be collected later results.
Already with the Storm, the computer never ends: the network continuously process messages and produces results. These messages may be new data for analysis, which then will update databases in real time. For example, Twitter messages being analyzed for trending topics and passing this information to the client systems within the architecture.
More details on the Storm can be found here .
Comments
Post a Comment