freshidea - Fotolia

Tip

AWS CodeCommit triggers bolster use of Git

AWS CodeCommit allows admins to manage Git repositories. AWS recently added support for triggers, giving IT teams the ability to respond to events and accelerate app deployments.

AWS CodeCommit was launched in 2015, allowing developers to run repositories of Git on AWS. But the announcement was mostly quiet because it didn't add any special features. However, I suspected it marked a first step in integrating a cloud-based workflow for Git on AWS. That has now come to fruition -- with support for triggers based on events in a Git repository in AWS CodeCommit.

Triggers allow IT teams to respond to events that happen in a repository, such as a developer pushing out new code. The GitFlow methodology, along with trigger use, allow developers to properly implement both continuous integration -- testing code as it is committed -- and continuous delivery -- deploying code as soon as it is verified and committed. With CodeCommit, developers can use Git on AWS to deploy new versions to both development and production environments entirely by pushing code to specified branches.

One very common use case for triggers is to automatically build new releases of code pushed to either a development or master branch of a repository. Developers can completely automate testing and deploy a Node.js application from AWS CodeCommit directly to AWS Elastic Beanstalk.

The Lambda test function

Make sure code validates a given set of tests before deploying it from a repository. Unit tests or even general "lint" compilations can prevent simple syntax errors. For Node, I prefer to use the simple ESLint script -- installable through npm. This "linter" checks to make sure basic syntax is obeyed. It also checks for common errors like typos and the use of reserved keywords where they're not allowed.

Before AWS CodeCommit executes a Lambda function, it must have the appropriate access. Developers need to create a new JSON permission file, like this one:

{

   "FunctionName": "MyCodeCommitFunction",

   "StatementId": "1",

   "Action": "lambda:InvokeFunction",

   "Principal": "codecommit.amazonaws.com",

   "SourceArn": "arn:aws:codecommit:us-east-1:80398EXAMPLE:MyDemoRepo",

   "SourceAccount": "80398EXAMPLE"

}

 Then, upload it through the AWS command-line interface.

aws lambda add-permission --cli-input-json file://AllowAccessfromMyDemoRepo.json

The data your Lambda function will receive looks like:

{ Records: [

 {

   awsRegion: 'us-east-1',

   codecommit: {

    references: [ {

      commit: '0000000000000000000000000000000000000000',

      ref: 'refs/heads/all'

    } ]

   },

   eventId: '123456-7890-ABCD-EFGH-IJKLMNOP',

   eventName: 'TriggerEventTest',

   eventPartNumber: 1,

   eventSource: 'aws:codecommit',

   eventSourceARN: 'arn:aws:codecommit:us-east-1:80398EXAMPLE:MyDemoRepo',

   eventTime: '2016-03-08T20:29:32.887+0000',

   eventTotalParts: 1,

   eventTriggerConfigId: ‘123456-7890-ABCD-EFGH-IJKLMNOP',

   eventTriggerName: 'MyCodeCommitFunction',

   eventVersion: '1.0',

   userIdentityARN: 'arn:aws:sts::80398EXAMPLE:assumed-role/DevOps/cmoyer'

} ] }

There are some important fields to observe here. The "userIdentityARN" indicates the user who initiated the push. At a minimum, the Lambda function should log this so developers know who initiated the build request. But developers can also authorize who is allowed to initiate new build requests. For example, the Lambda function can be designed to only build new versions to production initiated by developers who are trusted to push production code.

AWS CodeCommit was launched in 2015, allowing developers to run repositories of Git on AWS.

The second important field to note here is under "code commit/references/ref," which shows the branch or branches that were committed.

This check needs to examine code and run a custom command, which may end up taking longer than five minutes. Instead, I use my Lambda function to execute an EC2 Container Service (ECS) task. This also allows developers to trigger other events, such as building and deploying new releases right through an ECS task.

This Lambda function triggers an ECS task:

/**

 * Execute an ESLint Task

 * to check the Code that was committed

 */

var AWS = require('aws-sdk');

var ecs = new AWS.ECS({ region: 'us-east-1' });

 

exports.handler = function(data, context){

   console.log(JSON.stringify(data));

 

   var counter = data.Records.length;

   function done(){

      counter--;

      if(counter === 0){

         context.succeed('OK');

      }

   }

 

   data.Records.forEach(function processRecord(record){

      console.log('CHANGES from', record.userIdentityARN);

      record.codecommit.references.forEach(function(ref){

       counter++;

       ecs.runTask({

          taskDefinition: ‘ECSBuilder',

          overrides: {

             containerOverrides: [

               {

                  command: [

                     ‘./checkBuild',

                     record.eventSourceARN.split(':')[5],

                    ref.ref.replace(‘refs/heads‘,''),

                    ref.commit,

                  ],

                  name: ‘ECSBuilder',

               },

             ],

          },

          startedBy: 'ESLint: ' + record.userIdentityARN.split('/')[1],

       }, function(err, resp){

          if(err){

             console.error('ERROR', err);

          } else {

             console.log('Successfully started', resp);

          }

          done();

      });

      });

     done();

   });

}

Note the use of a "counter" function; a single push event could actually trigger multiple repository updates. This code makes certain to test them all.

Adding triggers to a CodeCommit repository

After creating the Lambda function, developers configure CodeCommit to fire the Lambda function on specific events. This can be configured in multiple ways, but it is generally best to make sure the CodeCommit repository fires the event for any push events to the repository. The function can also be configured to filter pushes to specific branches.

Click on the newly added "triggers" option and choose "Create trigger" to get started.

Create a trigger in AWS CodeCommit
Developers can create a trigger for a Lambda function in AWS CodeCommit.

Next, fill out the details to create the trigger:

Configure the Lambda trigger.
Fill in the details to set up the trigger for the Lambda function.

In this example, the function only executes on a push to existing branches. If a development cycle uses GitFlow, developers may also need to include "Create branch or tag" to make sure new release branches also trigger this function. In both cases, make sure to fill out the branch names either to "All branches" or choose specific branches. Choose "AWS Lambda" as the service to send to, and select the Lambda function. Once everything is set, use the "Test trigger" option to make sure the code repository has access. If it doesn't, retrace the steps to authorize the CodeCommit repository to call Lambda functions.

Creating an ECS task

The final step is to create an ECS task and authorize the Lambda role to execute it. ECS tasks execute Docker repositories from Amazon EC2 Container Registry (ECR), so the easiest method is to push a Docker image up to ECR where the task can run it.

A simple Docker script may look like this:

FROM node:5.6.0

 

# make sure apt is up to date

RUN apt-get update

 

# Install global packages

RUN npm install --global grunt-cli eslint

 

# install Git and curl

# Python is required by the "memcached" node module

RUN apt-get install -y Git Git-core curl python build-essential

 

# Create a bashrc

RUN touch ~/.bashrc

 

# Copy our bundled code

COPY . /usr/local/app

 

# Set the working directory to where we installed our app

WORKDIR /usr/local/app

This script needs to be in a directory with the script to check the build within the repository. In the Lambda script we created above, we run a script called "checkBuild" that contains the last part of the repository's name -- "eventSourceARN" -- as well as a reference to the commit branch and the exact commit ID. With these three items, developers can build a check script that examines the exact version that pushes the trigger.

The checkBuild.sh script should look like this:

#!/bin/sh

REPOSITORY=$1

BRANCH=$2

COMMIT=$3

 

# Add the Known Host

ssh-keyscan -H Git-codecommit.us-east-1.amazonaws.com >> ~/.ssh/known_hosts

 

# Check out the repository

Git clone ssh://[email protected]/v1/repos/${REPOSITORY} build -b ${BRANCH}

 

cd build && Git checkout ${COMMIT} && eslint .

Make sure to replace "USERNAME" with a valid Identity and Access Management (IAM) user that has secure shell (SSH) access to the AWS CodeCommit repositories you're testing. It's best to create a new IAM user specifically for this build service, give it access to the repositories and upload an SSH public key for it.

Once this is set, developers can build and deploy the Docker image to the ECR and then use that to create the ECS task. The Lambda function sets up the command, so the task just needs to point to the Docker repository for the image.

Although this code runs "ESlint" on the checked out code, it neither notifies anyone of the results nor does it automatically deploy anything if the build succeeds. Unit tests can also be executed here to make sure everything passes. A good way to do this is to build notifications right into grunt to make sure the results are sent to developers through integrations with Slack, Flowdock or email notifications.

This one new option from AWS for adding basic hook support for CodeCommit can open up a whole new world of opportunities for using Git on AWS for continuous integration and deployment.

Dig Deeper on AWS cloud development

App Architecture
Cloud Computing
Software Quality
ITOperations
Close