adam121 - Fotolia
How do CloudWatch logging features track resources?
Between our on-premises and in-cloud resources, our hybrid infrastructure produces a multitude of logs. How can we use Amazon CloudWatch to aggregate and visualize logs?
One key to managing any IT environment is the availability of logs -- rolling files that capture and record an array of events within systems, applications and actions. Admins can access and review log files to identify and troubleshoot problems and trends for long-term planning.
Amazon CloudWatch provides a feature called "Logs," which captures and stores log files from Amazon Elastic Compute Cloud (EC2) instances and AWS CloudTrail. Through CloudWatch logging, IT teams can collect and aggregate content from resources outside of AWS. And Amazon EC2 instances export any event log, event trace or log text file for CloudWatch logging. For example, a Windows-based instance can send Windows Server system, security, application or Internet Information Server information for CloudWatch logging and then apply CloudWatch metrics to report them.
Admins can check CloudWatch Logs for a variety of different activities, including near real-time troubleshooting as well as can filtering for events, phrases, values or behaviors that occur. For example, a technician can receive alarms corresponding to log errors that exceed a selected threshold, and then review the log entries to learn the underlying cause. Similarly, CloudWatch logging results can be viewed graphically over a longer period of time, allowing enterprise IT to make better strategic decisions about changes, such as infrastructure improvements or adjusting AWS products to boost workload performance.
CloudWatch retention settings enable admins to decide how long to store log events. Log data is stored indefinitely by default, but users can also set a log expiration, allowing any older log data or events to be automatically deleted. This helps limit storage use and reflects established corporate data retention policies.
CloudWatch Logs have storage, configuration and traffic limitations. The service provides 5 GB of incoming data and 5 GB of data archiving for free, though additional storage will require services like Amazon Glacier. Users can configure up to 500 log groups and filter up to 100 CloudWatch metrics per log group.
CloudWatch logging is available up to 256 KB and can handle batched log events up to 1 MB. Log events can only be sent -- "PutLogEvents" -- up to five requests per second, per log stream; log events can only be received -- "GetLogEvents" -- up to 10 requests per second for the entire AWS account. This can pose a problem when processing a large number of logs for multiple business units.
Amazon EC2 instances running Linux, Ubuntu, CentOS or RHEL must have a CloudWatch Logs agent installed. Windows Server can also use a CloudWatch Logs agent, but Windows Server instances, such as Amazon Machine Instances, already include the EC2Config service, which can send event or Web server request logs to CloudWatch Logs.
Dig Deeper on AWS infrastructure
Related Q&A from Stephen J. Bigelow
What is data separation and why is it important in the cloud?
Some enterprises avoid the public cloud due to its multi-tenant nature and data security concerns. Learn what data separation is and how it can keep ... Continue Reading
NAS vs. object storage: What's best for unstructured data storage?
There are advantages and disadvantages to using NAS or object storage for unstructured data. Find out what to consider when it comes to scalability, ... Continue Reading
Do hypervisors limit vertical scalability?
Knowing hardware maximums and VM limits ensures you don't overload the system. Learn hypervisor scalability limits for Hyper-V, vSphere, ESXi and ... Continue Reading