Future of real-time enterprise IoT depends on in-memory computing powered HTAP
Just a couple of years ago, the internet of things was more vision than reality. Today, enterprises are increasingly deploying IIoT technologies as part of digital transformation initiatives designed to minimize maintenance costs and maximize asset utilization. Examples include instrumenting commercial aircraft, large commercial buildings, power plants and truck fleets. But large-scale IIoT isn’t easy. It requires the real-time processing and analysis of massive amounts of data, which is a significant challenge for almost any enterprise. To meet this challenge and ensure real-time responsiveness at scale, enterprises have begun taking advantage of hybrid transactional/analytical processing (HTAP) powered by in-memory computing.
The IoT growth projections are staggering. Ericsson expects there will be approximately 18 billion connected devices related to IoT by 2022 and that between 2016 and 2022, IoT devices will increase at a compound annual growth rate (CAGR) of 21%. Machina Research predicts the total number of IoT connections will grow from 6 billion in 2015 to 27 billion in 2025. IIoT investment could surpass $1 trillion over the next 10 years, and GE Digital anticipates $60 trillion in connected industrial assets by 2030. And overall, Nokia Bell Labs predicts IoT’s value will be 36 times that of today’s entire internet, based on the number of devices connected and how users perceive and experience the value of IoT devices and applications.
It’s important to recognize that this growth depends on the wide range of IoT and enterprise IoT use cases delivering as promised — not just in the early implementation stage, but at scale, with ever-soaring data flows and data-intensive real-time analytics.
Digital twins
Consider the concept of the digital twin. A digital twin uses advanced analytics to model the current state of real-world manufacturing and industrial assets — from an automobile to an aircraft to a power plant — using large numbers of IoT sensors on the real-world devices, combined with feeds from other relevant data sources, such as weather, temperature and moisture, plus historical data. All this data is combined and analyzed in real time. By creating this digital model of a complex system, companies can plan maintenance to minimize costs and maximize utilization.
For example, a digital twin can determine if a jet engine requires maintenance without requiring a physical inspection. Inspecting a digital twin instead of the physical object can also reduce the risk of inspecting items in potentially dangerous environments, such as with underwater pumps or power plant cooling systems. Companies have been using modeling for years, but creating a real-time digital twin of a complex physical system requires that sensors be deployed and that the sensor data feeds into a system with the processing power and bandwidth required to benefit from the data. Gartner is predicting that by 2022, IoT will save consumers and businesses $1 trillion a year in maintenance, services and consumables, and digital twins can play an important role in that.
Many other enterprise and industrial IoT use cases have been equally challenging for organizations. For example, production tracking, inventory maintenance, logistics and patient monitoring have all required a tremendous investment in compute power as the applications scale, prohibiting many companies from realizing their IoT goals.
All of that is changing thanks to HTAP powered by in-memory computing.
HTAP and in-memory computing
The key to successful real-time IoT processing and analysis at scale is the ability to implement HTAP powered by in-memory computing. Traditional approaches rely on an outdated, bifurcated database structure. Online transaction processing (OLTP) databases are designed to handle only operational data. Separate online analytical processing (OLAP) databases handle the analytical processes. To bridge the two systems, extract, transform and load (ETL) processes periodically move data from the OLTP database to the OLAP database — which introduces a delay which cannot support the real-time analytical demands of IIoT.
HTAP eliminates the delay associated with ETL. It enables real-time analysis on the operational data set without impacting performance. However, until recently HTAP was too expensive for most enterprise budgets, requiring huge hardware and software investments.
Today’s in-memory computing platforms make HTAP affordable. In-memory computing platforms maintain data in RAM in order to process and analyze data without requiring the delays inherent in reading data from a disk-based database. Architected for Massively Parallel Processing across a cluster of commodity servers, these platforms can easily be inserted between existing application and data layers with no rip-and-replace of the existing database. They can also be easily and cost-effectively scaled out by adding new nodes to the cluster, which automatically takes advantage of the added RAM and CPU processing power. The benefits of in-memory computing platforms include massive performance gains, the ability to scale to petabytes of in-memory data and high availability thanks to distributed computing.
In-memory computing isn’t new, but until recently, the cost of RAM and the lack of affordable technologies severely limited adoption. However, the cost of RAM has dropped steadily, approximately 10% per year for decades, and mature, easy-to-install in-memory computing platforms are now available.
This makes in-memory computing perfect for HTAP because the entire transactional data set is already in RAM and ready for analysis. By applying massively parallel processing to the data, sophisticated in-memory computing platforms can run fast, distributed analytics across the data set without impacting transaction processing.
According to “Market Guide for HTAP-Enabling In-Memory Computing Technologies,” in-memory computing-enabled HTAP can have a transformational impact on a business. It will also help power a new generation of IoT platforms, which are on-premises software suites or cloud services that monitor and manage various types of IoT endpoints. IoT platforms will eventually make IoT projects easier to launch and simpler to manage, and Gartner predicts that by 2020, 65% of companies adopting IoT will use an IoT platform for at least one project.
Call it a wave or a deluge, refer to it as an era or an age, but IIoT will dramatically change how businesses manufacture, deliver and maintain products. Just looking at the revenue side, Machina Research also predicts overall IoT growth from $750 billion in 2015 to nearly $3 trillion in 2025, growing at a 15% CAGR. IoT platforms and middleware revenue will grow from about $50 billion in 2015 to $250 billion in 2025. But IoT requires a critical technology infrastructure reinvention, which can be enabled only by HTAP powered by in-memory computing. Enterprises developing IIoT strategies should immediately begin exploring the power of integrating in-memory computing with HTAP to help make those strategies a reality.
All IoT Agenda network contributors are responsible for the content and accuracy of their posts. Opinions are of the writers and do not necessarily convey the thoughts of IoT Agenda.