Bigeye brings new visibility to data observability
The startup continues to build out its cloud platform that enables data engineers to identify and remediate data pipeline quality and availability problems.
In April the vendor secured a $17 million Series A funding round and followed it up with a $45 million Series B round in September.
Over the course of the year the vendor grew its user base and identified data pipeline and DataOps challenges with visibility.
The need to better track problems with alerts and notifications is one such challenge Bigeye identified and is now addressing with the beta feature update, unveiled this week.
Alongside the issue tracking feature is a new dashboard that provides a graphical visualization for data pipeline metrics.
Data observability is helping meet enterprises' desire to shift detection and resolution of data-related problems to earlier in the process, said Paige Bartley, an analyst at S&P Global Market Intelligence's 451 Research.
The idea is to address potential problems before they manifest into much more serious challenges downstream in data consumption.
"Particularly as self-service data programs expand in scope, there are often simply more workers that depend on the access and leverage of relevant data than there are resources to pipeline and stage that data," Bartley said. "By often catering to data engineering roles, data observability technologies are helping catch potential data problems early, leveraging skills that are most appropriate to proactively remediate issues."
The intersection of DataOps and data observability
The concepts of DataOps and data observability are closely related, said Ventana Research analyst Dave Menninger.
This is what data observability is all about -- collecting, examining and reacting to the metadata associated with data processes.
Dave MenningerAnalyst, Ventana Research
DataOps provides agility in data processes and the ability to deal with changes in those data processes quickly and easily, often automatically, Menninger explained. In his view, the only way to do that is to examine the metadata associated with those data processes.
"In essence, this is what data observability is all about -- collecting, examining and reacting to the metadata associated with data processes," Menninger said. "Bigeye has recognized the value of this metadata and has engineered ways to expand the metadata that is collected and analyzed."
Looking at the issues of data observability
Until now, the Bigeye platform has provided users with a series of automated capabilities, among them the ability to automatically detect potential problems in data pipelines.
While Bigeye was able to identify issues, Kyle Kirwan, the vendor's co-founder and CEO, said the platform only flagged the problem for users and didn't go much further than that at the time in helping them actually walk through a resolution process.
The Bigeye data observability platform is introducing a pair of beta features to help users track issues and get data metrics on a dashboard.
With the new issues-tracking feature that is now in beta, Bigeye is providing more visibility and tracking for data problem resolution.
"When we identify a problem we create a ticket inside Bigeye, and then users can resolve those tickets as they fix data quality issues," Kirwan said.
As problems are resolved, the system creates documentation. So the next time a similar issue occurs, Kirwan said the Bigeye platform can remind users how they fixed the issue in the past.
The issues system also feeds a machine learning system that helps Bigeye identify data quality problems.
A dashboard view for data observability
Bigeye is now also providing a beta of its dashboard, which provides a visual way for users to understand data visibility.
Among the items included in the dashboard is tracking for service-level agreement expectations about data. The dashboard also makes it easier for users to understand what the top issues are as Bigeye monitors the data.
Looking forward, Kirwan said a goal for the vendor in 2022 is to expand the automation capabilities of the platform.
While Bigeye now can automatically identify data source as well as potential data issues, it doesn't currently have the ability to automatically remediate problems.
"So in my mind, like 2022, the big thing for Bigeye is how much more intelligent can we get the platform," Kirwan said. "I think that's really the next chapter for us … starting to get more into automation space and collecting all the signals necessary to know what types of automations to try to apply in different scenarios."
