shutterstock_162650075 Pivotal has released Pivotal HD 2.0, an update of its Apache Hadoop distribution. Pivotal’s freshly upgraded distro is rebased on Hadoop 2.2, which reached its first general-release milestone last October. Hadoop 2.2 improvements include YARN (a general-purpose resource management system) and increased integration with other open-source projects. Numerous companies and developers rely on the Hadoop open-source framework to run data applications on large hardware clusters, although enterprises with titanic amounts of information to manage—most notably Facebook and Netflix—have moved to building their own customized frameworks in some cases. Given Hadoop’s popularity—research firms predict the software will become a multibillion-dollar market over the next few years—it’s unsurprising that tech firms large and small have begun issuing Hadoop distributions, often loaded up with proprietary analytics tools. Pivotal is no different in that regard: the company also announced the release of Pivotal GemFire XD, an in-memory database designed for integration with Pivotal HD 2.0. In theory, the combined platform—working in conjunction with other Pivotal products, such as the HAWQ query engine—can provide developers and data scientists with flexible and efficient means for mining insight from large datasets. And Pivotal has something to prove, having spent the entirety of its short existence trumpeting its desire to become a major player in the so-called “Big Data” segment. As a mash-up of assets from EMC and its VMware subsidiary (and headed by former EMC chief strategy officer Paul Maritz), Pivotal offers a variety of cloud-infrastructure and analytics tools that it’s attempted to blend into a more holistic platform. The crown jewel in that effort is Pivotal One, released late last year, which Pivotal bills as a “comprehensive” collection of application and data services running atop its enterprise version of the open-source Cloud Foundry. If Pivotal can take a healthy chunk of the commercial Hadoop market, that could likewise help make it a serious competitor.   Image: Pressmaster/