kentoh - Fotolia

Elastic 7.9 platform improves data observability

Elastic updated its platform with a new unified agent for collecting data from different sources and a newly enhanced view for data observability across different types of data.

Elastic updated its namesake Elastic Stack data platform with new features designed to improve data observability, ingest and security.

The Elastic 7.9 release became generally available on Aug. 19, providing enhanced capabilities for the Elastic Stack platform.

The platform is built on a foundation that includes ElasticSearch for search, Logstash for log management and Kibana for data visualization. Among the new capabilities in the update is a technology preview for the new unified agent that helps enable data collection and security. The release also includes an enhanced ingest manager with a simplified approach to getting data from different sources into Elastic.

Among the customers of the open source search and analytics vendor that are optimistic about the 7.9 release and the new ingest manager is financial services provider CDL, based in Stockport, Englang. The company deploys Elastic for several applications in the insurance and financial services sector, including the Hummingbird data intelligence system.

CDL's service ingests large amounts of information in real time to analyze patterns of behavior, using the search capabilities of Elastic to look at data such as active insurance quotations and policy history. CDL also uses Elastic for central logging and audit functions.

"Our applications process huge volumes of transactions on a daily basis and are highly connected systems, so having central visibility through a system that renders the data accessible through search is critical for us," said Robert Trueman, head of software engineering at CDL.

The new ingest manager is particularly interesting to Truman, he said. He noted that having one-click data ingestion is the type of new feature CDL needs to help manage data, as ingest is one of the more demanding areas, especially with data coming from many sources in different formats from a growing set of sources. 

Screenshot of the Elastic Ingest Manager
The Elastic Ingest Manager provides users with pre-built integrations for different sources of log and metrics data.

Elastic 7.9 brings a single agent, improving data observability and security

Elastic CEO and founder Shay Banon said he sees the new unified agent approach in the Elastic Stack as being a major advance. The single agent in Elastic 7.9 is still only a preview as it isn't yet fully configured for every potential source of data.

The promise of the single agent is that it can be used for different use cases including crawling logs for data observability, infrastructure metrics or application performance. The same single agent can also be used to enable security and endpoint protection. Elastic acquired endpoint protection security vendor Endgame in October 2019 for $234 million.

Our applications process huge volumes of transactions on a daily basis and are highly connected systems, so having central visibility through a system that renders the data accessible through search is critical for us.
Robert TruemanHead of software engineering, CDL

"We have a saying at Elastic, 'While you observe, why not protect?'" Banon said. "So we want to bring not only a one-click experience for users to add data, but also one click to add protection to anywhere that you have an agent installed."

Elastic data observability improvements

Another key improvement in the update is a new observability view in the Kibana data visualization component. The observability view provides a consolidated view of incoming logs and metrics from different application and server sources.

There has often been a distinction between different types of logs and metrics coming from applications and infrastructure. Banon noted that Elastic is trying to help blur the line between the log and metrics data to get to the broader concept of data observability.

"Observability is a concept that is happening because at the end of the day, data is data," Banon said. "The outcome that users want to have, whether they get data from instrumenting applications or collecting logs from servers or monitoring metrics from a system that they deployed, is to just to make sure that they can observe all of that."

Elastic workplace search set free

Elastic offers several editions of its platform including free, enterprise and cloud deployment models.

As part of the Elastic 7.9 platform, the vendor is now making the Workplace search feature available to its free tier of users. Workplace search was unveiled April 23, during the Elastic{ON} Americas East virtual conference. Workplace search enables Elastic users to perform a unified search across their data sources to query and find relevant information.

"People now are spending a lot of time online and we think our workplace search product is highly applicable to the current state of how people work," Banon said.

Next Steps

Cribl aims to ease data observability with LogStream update

Monte Carlo Incident IQ looks to improve data observability

Cribl brings in $200M to advance data lake observability

Dig Deeper on Data management strategies

Business Analytics
Content Management