The days of data generated from a system being stored in a database only to be processed at a later stage may soon be over. The ever-increasing need for real-time data has made it imperative to process data as and when it is produced, thus marking the inception of (data) stream processing to generate outputs almost instantaneously. Stream processing systems typically generate results in the time range of sub-milliseconds to a few seconds or minutes, depending on the functionality.
In the wake of humongous amounts of data being collected by the operations technology (OT) domain, Fujitsu Laboratories, recently announced plans to develop a Dracena (Dynamically Reconfigurable Asynchronous Consistent Event-processing Architecture) to enable addition or processing of content without powering off the system. The architecture proves to be a marked departure from the conventional stream processing architectures in that it does not require the processing activity to be temporarily stopped when changing existing content or adding new data to the system. Fujitsu's solution separates the stream processing activity from data ingestion, which allows the system to automatically begin processing a given data stream following the completion of a running program. Simulation results for a connected car use case confirm the capability of Fujitsu’s architecture to process a few dozens of data bytes released each second from a million cars.
Fujitsu plans to productize the technology in 2018 as an additional module to its IoT platform for mobility. The buzz is that the company plans to further extend the capabilities of the stream processing architecture to all areas that require processing of data streamed at a high frequency, in real-time.
