Neo4j's latest targets graph database performance at scale
Infinigraph enables the vendor's platform to handle the scale of hybrid transactional/analytical processing workloads used to help inform AI and analytics applications.
Neo4j on Thursday launched Infinigraph, a new architecture that enables the vendor's graph database to run operational and analytical workloads together in one system without suffering performance declines or forcing users to build duplicate infrastructures.
In addition, Infinigraph guarantees full ACID (atomicity, consistency, isolation and durability) compliance so that data remains valid despite any changes.
An operational workload is the real-time processing of transactional data related to an enterprise's daily operations and supports short, simple queries. An analytical workload, meanwhile, is the processing of data used for historical analysis and involves lengthy, complex queries.
Many databases enable hybrid transactional/analytical processing (HTAP). However, with the different workload types competing for system resources, not all can do so without the database's performance slowing down.
As a result, Infinigraph, which is now generally available as part of Neo4j's self-managed database with availability in its fully managed database soon, is a valuable addition for the vendor's customers, according to William McKnight, president of McKnight Consulting Group.
"Infinigraph could be a significant addition by delivering on the promise of enabling the database to handle both operational and analytical workloads at scale without compromising performance," he said. "This would eliminate some data silos and simplify data architecture."
Specifically, it would reduce the need for extract, transform and load (ETL) pipelines and building architectures for different workload types, McKnight continued.
Based in San Mateo, Calif., Neo4j is a graph technology specialist. The vendor's database enables users to discover relationships between data points in different ways than traditional relational databases so they can be used for applications such as fraud detection and recommendation engines.
AI models and applications require large amounts of high-quality data to reduce the likelihood of hallucinations and engender trust from end users. When an enterprise's data is isolated in different systems, it becomes difficult to integrate to provide AI tools with enough relevant data, and the resulting lack of accuracy prevents the application from being usable.
Neo4j's Infinigraph architecture aims to address isolated data by uniting transactional systems and analytical tools without lowering database performance.
Previously, users had to link separate transactional and analytical databases, synchronize systems not predesigned to work together or push single platforms to the point that their performance declined, all of which drove up the cost of building AI and analytics applications.
Like McKnight, Stephen Catanzano, an analyst at Enterprise Strategy Group, now part of Omdia, called Infinigraph's addition "significant" for Neo4j's database users.
Infinigraph … solves the challenge of running both operational and analytical workloads in a single system at scale.
Stephen CatanzanoAnalyst, Enterprise Strategy Group
"Infinigraph … solves the challenge of running both operational and analytical workloads in a single system at scale," he said. "This eliminates the need for separate systems, ETL processes and synchronization delays that have traditionally plagued enterprise data architectures."
Meanwhile, customer feedback provided Neo4j with the impetus for developing a new graph database architecture, according to Sudhir Hasbe, the vendor's president of technology.
"The primary motivation for Infinigraph was customers who were creating graphs with lots of property data and bumping up against practical limitations on machine resources," he said.
Infinigraph utilizes sharding -- a database architecture technique that divides large databases into small, more manageable parts called shards -- to distribute a graph's property data across different nodes. By doing so, the graph remains whole, but nodes operate independently when called upon to better manage workloads.
Catanzano noted that while competing graph database providers such as TigerGraph and ArangoDB offer multimodel capabilities so users can store and process various data types, Neo4j's use of sharding to enable HTAP workloads may be unique.
"Neo4j's implementation of sharding to preserve graph structure while enabling both workload types at massive scale appears distinctive," he said. "Neo4j had previously introduced parallel runtime capabilities for faster analytics, but Infinigraph takes this to scale."
Key benefits of the new database architecture include the following, according to Neo4j:
Handling 100 TB of data or more without requiring developers to rewrite code.
Performance across both transactional and analytical workloads.
Eliminating development delays and increased costs related to building ETL pipelines, synchronizing systems and duplicating storage.
Scalability, simplicity and ACID compliance are perhaps most valuable, according to Catanzano.
"These solve critical pain points for enterprises dealing with massive connected data sets that need both real-time transaction processing and deep analytical capabilities," he said.
McKnight similarly highlighted handling high volumes of data and ACID compliance as substantial benefits. In addition, he noted that Infinigraph has the potential to help differentiate Neo4j from other graph database vendors.
"Neo4j is a leader in the graph database market, with Infinigraph furthering their position … as an innovator," McKnight said.
Looking ahead
With Infinigraph now available in Neo4j's Enterprise Edition and coming soon for AuraDB, the vendor is focused on continuing to build capabilities that help customers create intelligent applications, according to Hasbe.
"We will also broaden our technology partner ecosystem and ensure that every product has 'ease of use' at its core," he said.
McKnight noted that Neo4j is already providing data scientists with key capabilities. However, he suggested that the vendor still has room to support more advanced graph neural networks, graph-based anomaly detection and graph-based recommendation systems.
"While it's already doing a lot for data science workloads, Neo4j could continue to expand its Graph Data Science Library to include more algorithms, models and tools for advanced graph analytics and machine learning," McKnight said.
Catanzano, meanwhile, recommended that Neo4j continue focusing on enabling AI development by improving its vector embedding and retrieval-augmented generation capabilities. In addition, industry-specific graph database templates could broaden Neo4j's appeal, according to Catanzano.
"Creating industry-specific solution templates that combine Infinigraph's scale with domain knowledge would help attract new customers who need guidance implementing graph technology for their specific use cases," he said.
Eric Avidon is a senior news writer for Informa TechTarget and a journalist with more than 25 years of experience. He covers analytics and data management.
