Know your options for Elasticsearch services on AWS
Determine which Elasticsearch tool is right for your organization with a comparison of key features in Amazon's Elasticsearch Service and Elastic's Elasticsearch.
Elasticsearch has become one of the most widely used tools to quickly weave different kinds of search capabilities into other applications. AWS users can pick from two managed versions, and their choice will likely depend on how the software fits into their broader architectural needs.
ElasticSearch is a tool that implements search engines on a variety of data sources for full text search. In addition to just indexing text, Elasticsearch also makes it easier to organize a data stream. It can help you analyze log data for clickstream analytics, application monitoring and security analytics. Shay Banon released an open source implementation in 2010 and also founded the search company elastic in 2015 to sell a commercially licensed version of the technology.
There are two popular approaches to implement managed Elasticsearch capabilities on AWS: Amazon Elasticsearch Service (Amazon ES) and Elastic Cloud Elasticsearch Service. There's also a self-managed approach, in which an organization would install the open source Elasticsearch software on an EC2 instance. This approach gives an organization the most flexibility but requires more technical expertise and management resources.
Amazon ES vs. Elastic Cloud
Amazon ES and Elastic Cloud offer similar functionality but differ in key ways. They share capabilities, such as support for Kibana, a tool that visualizes and analyzes data processed with Elasticsearch. They also work with other open source tools, such as Beats and Logstash, which simplify the aggregation and transformation of log data from raw logs. Both provide easy deployment and scalability, and they require less human overhead than if you manually deployed Elasticsearch on EC2 or on premises.
If you can get by with the features AWS provides, I usually try to recommend sticking with the AWS native services.
Alex HigginsDevOps technical lead, Candid Partners
Amazon ES offers better integration with other AWS tools, such as Kinesis Streams, Kinesis Firehose and CloudWatch Logs. However, Elastic Cloud makes it easier to port to other cloud platforms or on-premises infrastructure.
Elastic Cloud supports a variety of newer capabilities that Amazon ES currently lacks. They include:
Data Rollups use less storage space to summarize and store historical analytics.
Elasticsearch SQL provides new tools to query dates and times and generate histograms.
Graph features help analyze relationships in your data to make recommendations and protect against threats.
Kibana Spaces make it easier to create and organize dashboards.
Elastic Commercial Plugins help use Elasticsearch for security, monitoring and machine learning.
Custom plugins simplify management of new enterprise-specific functionality.
The experts weigh in on managed Elasticsearch options
Alex Higgins
The Elastic Cloud service gives organizations more control but also requires a bit more work to maintain, said Alex Higgins, DevOps technical lead for Candid Partners, a cloud consultancy.
"If you can get by with the features AWS provides, I usually try to recommend sticking with the AWS native services, as both the cost in running the software, as well as maintenance, is likely to be lower," he said.
Todd Matters, co-founder and chief architect at RackWare, a hybrid cloud management platform, said the extra capabilities provided by Elastic's implementation are not a significant differentiator since both vendors provide the same basic Elasticsearch features and functions needed for most common uses.
"The only real difference is that, if you are already in the AWS cloud, the AWS service is easier to set up and use, and it provides better resiliency," Matters said.
On the other hand, the biggest reason to use the Elastic Cloud Elasticsearch Service is to avoid vendor lock-in.
"It may seem trivial, but the more services you are using in AWS, the harder it will be to get out of there if economics or other IT reasons encourage you to do so," Matters said.
Elastic's Elasticsearch Service is also more flexible. You choose the infrastructure you run the service on and how to configure it, and the setup and configuration are simple enough for any competent IT person or tech-savvy user.
However, organizations that heavily use Elasticsearch and want the most control over their workloads should deploy and manage Elasticsearch on their own.
Alex Higgins
