Microsoft Exhibits Spark on Azure HDInsight to Expand its Cloud Based Big Data Analytics

Microsoft has worked with Hortonworks to add high availability features to the platform via updates to the YARN resource manager.

Apac CIOOutlook | Thursday, June 09, 2016

Stay ahead of the industry with exclusive feature stories on the top companies, expert insights and the latest news delivered straight to your inbox. Subscribe today.

FREMONT, CA: Microsoft officially rolls out Apache Spark for Azure HDInsight cloud-hosted service for big data analytics, bringing the advantages of the Spark platform for processing of data to better support real-time analysis.

Azure HDInsight has been developed in partnership with Hadoop specialist Hortonworks and is based on the Hortonworks Data Platform. Microsoft has worked with Hortonworks to add high availability features to the platform via updates to the YARN resource manager.

Spark runs data analytics processes in-memory, and can deliver queries up to 100 times faster than traditional big data solutions. With general availability of Spark for Azure HDInsight,Microsoft will be offering Apache Spark v1.6.1 along with improvements to the service to make it more enterprise-ready and easier for users to access.

R server for HDInsight

In addition, Microsoft announces the general availability of R server for HDInsight. It achieves unprecedented scale and performance by combining enterprise-scale R analytics software with the power of Hadoop and Spark.

Microsoft also announces R server for Hadoop installations. R server for Hadoop will help scale analysis transparently by distributing work across nodes without complex programming.

The new Microsoft R Client, freely available tool for data scientists. The software helps to analyze remote big data and scale out the analytics by pushing the computation to a production instance of Microsoft R Server such as SQL Server R Services, R Server for Hadoop and HD Insight with Spark.

For data visualization front Microsoft announces Power BI. This allows users to publish real-time events from Spark Streaming directly into one of the fastest growing visualization tools in the market.