In-Memory Data Fabric

– Nikita Ivanov, CTO of GridGain says:

Much of what human beings experience as commonplace today – social networking, on-line gaming, mobile and wearable computing — was impossible a decade ago. One thing is certain: we’re going to see even more impressive advances in the next few years. However, this will be the result of a fundamental change in computing, as current methods have reached their limit in terms of speed and volume. Traditional disk-based storage infrastructure is far too slow to meet today’s data demands for speed at volume, which are growing exponentially. In-memory computing (IMC) offers the only practical way to scale to today’s computing demands. Although IMC may once have been the domain of only the most well-funded organizations, the falling cost of RAM means that it can now be accomplished by any organization on commodity hardware.

As Mike Matchett, Senior Analyst, The Taneja Group said: “Clearly the big data ecosystem is evolving, tackling more kinds of analytics and addressing a wider variety of applications, including enterprise operational intelligence (EOI) apps that require faster interactive latencies, predictive machine learning, split-second decision-making, and high-volume processing. In-memory computing is key to profiting from new competitively differentiating big data solutions like these.”

Spinning disks will certainly be around for years to come, but the growing demands of Big Data, SaaS, mobile computing and Internet scale, coupled with dramatic improvements in RAM economics and availability, have led to a surge in new applications that access and process data in memory. As a result, IMC is quickly moving from esoteric concept to mainstream adoption. Like any game-changing technology, developers need to be able to learn it, experiment with it and test its limits in order to fulfill its promise.

As you may know, GridGain has been at the cutting edge of developing a comprehensive in-memory data processing framework for real-time transactional and analytical use cases for many years. Hundreds of organizations around the globe now use its proven open source technology under the Apache 2.0 license. This work has culminated in the recent release of its application- and data-agnostic GridGain In-Memory Data Fabric. GridGain’s in-memory data fabric code base shares the goals of easy accessibility and free redistribution with other memory-centric, open source data exchange frameworks that have emerged — most notably the Apache Spark project in the Hadoop world.

Today we are thrilled to announce that the GridGain In-Memory Data Fabric has been accepted into the Apache Incubator program under the name “Apache Ignite.” Our decision to contribute our pride and joy to the Apache Software Foundation (ASF) will ensure that this robust and battle-tested software not only remains broadly available, but will continue gaining community support to become the standard upon which in-memory computing’s promise will be realized. While GridGain will continue to be a strong contributor to the Ignite code base and offer new and innovative enterprise-grade features to its commercial product, the platform’s core code will be managed by the non-profit ASF, home of the world’s most successful open source projects.

Mike Matchett, The Taneja Group, continues: “GridGain has now put their key IP out for the world to use as Apache Ignite. We predict an enthusiastic adoption curve since GridGain’s approach leverages current developer skills and accelerates existing Hadoop apps without modification, providing an easy transition into in-memory computing.”

In order to explain why now is the right time to trust our code base to the ASF, it is first necessary to explain why we went to an open source model less than a year ago–and for that matter, why we spent years developing this software in the first place.

Earlier this year, we took an important step in order to make the transformation to IMC truly accessible by shifting to an open source model through an Apache 2.0 license. Our motivation was simple: remove all barriers to IMC adoption and accelerate the innovation that’s poised to occur through this technology. We wanted to introduce IMC to the mass market in a way that would give developers maximum freedom to experiment with it and discover the amazing things it allows you to  do – for example, accelerating drug discovery for diseases such as cancer. The first step was to make our mature and tested IMC platform open source, so that the technology would be as widely available and easily adoptable as possible. So we put the IMC code up on places like GitHub, which resulted in a more than 2000 percent increase in downloads in just a few months, clearly validating the huge interest from the community in the software.

The second step — turning over the keys to the Apache Software Foundation — will drive true community-building beyond just open source consumption, and of course will ensure the in-memory data fabric’s viability for the long term. By doing this, we hope to continue to fast-track both the adoption and adaptation of this technology.

Ilya Sterin, Sr. Director of Engineering, Chronotrack Systems, summed up the advantages of this strategy: “GridGain’s In-Memory Data Fabric enables us to solve our data and compute scale challenges, which become increasingly complex as our systems grow to support more events and athletes. With its track record behind open source projects, the Apache Software Foundation’s backing of GridGain’s open source platform gives us confidence that we can rely on it for the long term, and that we can expect to see ongoing innovation as this project is opened to thousands of contributors.”

Ultimately, community-driven adoption of open source in-memory data fabric technology will enable a new class of transactional, analytical and hybrid real-time applications that will allow even the smallest organizations to gain a competitive advantage from fast, data-driven decisions and operations. We believe Apache Ignite has all the right ingredients to become for the Fast Data world of the future what Hadoop is for Big Data today.

Let the innovation begin!