AWS adds logs to CloudWatch, under-prices third-party tools

Amazon Web Service’s new CloudWatch Logs may give IT pros reason to ditch more expensive third party tools such as Splunk, Loggly and Logstash.

NEW YORK -- Amazon Web Services added Logs to its CloudWatch interface, enabling IT teams to detect and prevent system performance trends as third-party management tool vendors worry about their future.

AWS CloudWatch records some simple metrics about CPU utilization and reads/writes for Elastic Compute Cloud (EC2) instances, and then displays this information as a graph in the AWS Console. But there’s more happening on EC2 instances and collecting this data can be a painful experience.

An entire ecosystem of tools including Loggly, Logstash, and Splunk have evolved to fill those CloudWatch gaps.

CloudWatch Logs, introduced here at the AWS Summit this week, is an attempt to flesh out the basic CloudWatch functionality, according to Brian Tarbox, a software engineer at Cabot Research in Boston. CloudWatch Logs automatically monitors log files from AWS services such as EC2 and stores it in the highly scalable Kinesis

This data gives administrators insight into all associated logs to help detect patterns of possible performance issues or security concerns, for example. Adding logging capabilities to CloudWatch could give customers a window into all the events happening on their AWS platform.

While some say CloudWatch logging capability still falls short of what many third-party tools offer, it has grabbed IT pros’ attention.  

[CloudWatch] is going to make a few companies very unhappy, like Splunk.
Harold BondyCEO of Microserve

“[AWS is] making it easier to do something with your logs and to take action in some way,” said Elias Kamaras, manager of application development for a New York-based financial services company. “It seems pretty basic at this point compared to third parties that have [those] capabilities. The notifications for CloudWatch is something we’ll look into.”

Using a single management platform appeals to others.

“It’s one place to look for things, said Michael Schabert, infrastructure engineer at Global Link Communications. “The biggest advantage of Amazon is it’s already getting the logs from application programming interfaces. It’s an easier starting point.”

Third-party tools battle for traction

What this added CloudWatch capability means for third-party vendors like Splunk is uncertain.

“[CloudWatch] is going to make a few companies very unhappy, like Splunk,” said Harold Bondy, CEO of Microserve, a phone and internet service provider based in New York. “They’re gone now.”

A representative from Splunk at the AWS Summit declined to comment.

Third-party monitoring tools are clunky, cumbersome and expensive in comparison to the new AWS tool, Bondy added.

Loggly, for example, has a Standard version for $49 per month for 1 GB, and a Pro version for $349 for 7GB. AWS' CloudWatch Logs provides 5 GB free for the first month; the service is pay as you go thereafter at $0.50 per GB for Ingested and $0.03 per GB Archived logs.

Michelle Boisvert is Executive Site Editor with TechTarget and can be reached at [email protected]. Adam Hughes is a News Editor and can be reached via email at [email protected] or on Twitter at @AdamHughesTT.

Former Site Editor Katherine Martin contributed to this report.

Dig Deeper on AWS CloudWatch and application performance monitoring

App Architecture
Cloud Computing
Software Quality