The ability to rapidly process data can be achieved in a variety of ways, including the use of an in-memory database.
Data is stored in main memory in an in-memory database, which results in faster response times than those of relational databases. This allows for a distributed database deployment model that also leads to low latency and high throughput.
Among the various in-memory database platforms is Apache Ignite, an open source technology that got its start in 2014 and is backed by commercial vendor GridGain.
GridGain, founded in 2007 and based in Foster City, Calif., competes against vendors including Red Hat and Hazelcast, which has an in-memory processing tier as part of its platform. GridGain has expanded its technology over the last eight years and on Aug. 2 appointed Eoin O'Connor its new president and CEO. He replaced Abe Kleinfeld, who retired.
While the company has a new CEO, technical leadership and direction continues to benefit from the firm's founder and current CTO, Nikita Ivanov.
Ivanov, creator of Apache Ignite, recently took time to talk about the challenges and opportunities for in-memory database technology.
What was the impetus for creating Apache Ignite and backing it with GridGain?
Nikita IvanovFounder and CTO, GridGain
Nikita Ivanov: The fundamental idea is very simple.
Databases typically keep data on storage devices, and the reason is the amount of memory you have on any computer is small. That works great, but it has one particular limitation in that it's ungodly slower than using memory.
With in-memory computing, essentially we deal with the stored data in memory of many computers clustered together. The single amount of memory in a single computer is not enough, but if you string together hundreds of computers, all of a sudden you have a big enough memory pool.
If you have software that allows you to store data in memory, you can gain tremendous performance benefits.
What is the intersection between in-memory databases and a caching layer where data is stored in a cache?
Ivanov: Caching is just one use case for in-memory computing.
Modern in-memory database computing goes beyond that because caching is very simple. Modern systems also look at transactions. You don't store data just for the sake of storage, unless you are in the backup business. Most of the time, you store data for it to be present and used. Apache Ignite and GridGain have a focus on how to process data that is in memory.
Our customers -- like American Airlines and American Express -- tend to use data for critical systems, including enabling authentication and transaction.
What are common challenges that organizations still face with respect to database transaction performance?
Ivanov: The amount of data is growing, and companies have realized that just collecting data means basically nothing for a business.
What they have realized is that instead of just collecting the data, they need the ability to process data in a reasonable time frame. We have seen more of a focus on performance because data drives a lot of real-time decisions. When I started, in-memory computing was mostly being used just by financial services organizations. Today, it's being used for cloud authentication, ticketing and scheduling.
Milliseconds now matter when it comes to data processing performance.
Editor's note: This Q&A has been edited for clarity and conciseness.