Hitachi’s Advanced Technology for Better and Faster Database Management System

TOKYO: Hitachi announces the development of a database management system generated for the high-speed embedded memory in the hardware and technology for high performance of FPGAs.

These two technologies are developed by combining with ‘Pentaho Business Analytics’, a business analytics software developed by Pentaho Corporation, to visualize business analytics results. With the expansion of the flash storage for data storage, it endeavours to create a prototype real-time data analytics system, which helps realize self-service data analytics, enabling employees of the particular field execute data analytics on massive business data faster.

Self-service analytics allows employees in a specific field conduct big data analytics easily, without depending on experts such as data scientists. But the data analytics system for self-service data analytics needs to have high processing capabilities to execute data read and data analysisprocesses to produce results quickly.

Using flash storage instead of a hard disk drive to store data increases the data read performance. But data analysis performance is not able to keep up with data read performance. To beat this issue, Hitachi is developing a database management system optimized for the high-speed embedded memory in the hardware and technology to conduct high speed parallel data processing in the FPGAs. The new technologies increase the speed of data analytics by up to 100 times. Combining these two technologies with Pentaho Business Analytics, a real-time data analytics system prototype is being built.

Technology for high-speed parallel data processing by the hardware

The column processing method in the new technology for high performance of FPGAs enables a set number of columns to be processed in turn rather than processing of one column at a time. Parallel data processing has been made successful using this method with a data filter circuit to select the data for analytics, and an aggregation circuit to group the data and calculate values such as total or average.

Database management system optimized for high-speed memory in hardware

The new technology for high-speed embedded memory in the hardware subdivides database management system into multiple data segments so that the management information of each data segments could be handled by the FPGA’s internal memory, stored in the flash storage, and processed within the FPGA.