AWS Data Pipeline (Amazon Data Pipeline)

AWS Data Pipeline is an Amazon Web Services (AWS) tool that enables an IT professional to process and move data between compute and storage services on the AWS public cloud and on-premises resources. 

AWS Data Pipeline manages and streamlines data-driven workflows, which includes scheduling data movement and processing. The service is useful for customers who want to move data along a defined pipeline of sources, destinations and data-processing activities.

Using a Data Pipeline template, an IT pro can access information from a data source, process it and then automatically transfer results to another system or service. Access to the Data Pipeline is available through the AWS Management Console, the command-line interface or service APIs.

An activity is an action that AWS Data Pipeline performs, such as a SQL query or command-line script. A developer can associate an optional precondition to a data source or activity, which ensures that it meets specified conditions before running an activity. AWS Data Pipeline includes several standard activities and preconditions for services like Amazon DynamoDB and Amazon Simple Storage Service (S3).

A developer can manage resources or let AWS Data Pipeline manage them. AWS-Data-Pipeline-managed resource options include Amazon EC2 instances and Amazon Elastic MapReduce (EMR) clusters. The service provisions an instance type or EMR cluster, as needed, and terminates compute resources when the activity finishes.