In the previous article, you learned how to use CloudFormation to describe a production-ready infrastructure for an EC2 based app. In this article you will learn to:
- Automate the creation of an AMI that contains the app with Packer
- Deploy a CloudFormation stack based
infrastructure/ec2.yml
with AWS CodePipeline - Run the Acceptance tests on AWS CodeBuild against the infrastructure created in the previous step
- Deploy another CloudFormation stack for the production environment
You can follow step by step or get the full source code here: https://github.com/widdix/aws-velocity
The pipeline is based on the CI/CD Pipeline as Code part of this series.
AWS Velocity Series
Most of our clients use AWS to reduce time-to-market following an agile approach. But AWS is only one part of the solution. In this article series, I show you how we help our clients to improve velocity: the time from idea to production. Discover all posts!
Additionally, the EC2 based app pipeline contains:
- add a BuildAMI action to the Build stage.
- add an Acceptance stage
- add a Production stage
Copy the deploy/pipeline.yml
file to deploy/pipeline_ec2.yml
to get the starting point right. If you don’t have the deploy/pipeline.yml
file you can download it from https://github.com/widdix/aws-velocity.
BuildAMI Action
EC2 instances start from an image (AMI) that contains the operating system including all the files that are needed. You can create your own AMI as well. Usually, you take one of the available AMIs like the Amazon Linux, make your modifications, and then create a new image from that. This whole procedure can also be automated with a tool called Packer. You will now see how you can run Packer in CodeBuild to create a new AMI that contains the app.
Packer itself needs configuration files. Create a file infrastructure/packer.json
with the following content that build a new AMI based on ami-c51e3eb6
(Amazon Linux) and a Bash script that you will create later:
{ |
Packer is configured to run a Bash script to provision the AMI and to upload the app
folder. Create a file infrastructure/packer.sh
with the following content to:
- install the latest patches
- install Node.js 6.x
- install the CloudWatch Logs agent
- install
forever
, a tool to run Node.js script in the background
#!/bin/bash -ex |
The script also moves the application files to the right place.
To integrate Packer into the pipeline, add the following resources to the Resources
section of deploy/pipeline_ec2.yml
to create a CodeBuild project to run Packer with the above configuration. Packer also needs a bunch of IAM permissions which are also added.
# Packer needs a set of access rights as defined in https://www.packer.io/docs/builders/amazon.html |
You also need to make a small modification to the exiting BuildSpec
in the AppProject
resource to include the packer files into the App
artifact. This is a hack because CodeBuild does not support multiple input artifacts at the moment. Change the artifacts
section to:
artifacts: |
Now the CodeBuild project needs to be called in the pipeline, therefore change the Pipeline
resource in the file deploy/pipeline_ec2.yml
and add a new build action:
Pipeline: |
Now, a new AMI is automatically created whenever the pipeline runs. The AMI will include the app and the latest patches.
It’s time to deploy the app to the acceptance stage and too see if the app works.
Acceptance stage
The acceptance stage consists of a CloudFormation stack based on infrastructure/ec2.yml
and the execution of the acceptance tests. To create the CloudFormation stack, you first have to provide a few parameters. Create a file infrastructure/ec2.json
with the following content:
{ |
If you don’t have a VPC stack based of our Free Templates for AWS CloudFormation (https://github.com/widdix/aws-cf-templates/tree/master/vpc) create a VPC stack first. Make sure to change the ParentVPCStack
parameter in the infrastructure/ec2.yml
accordingly. Also change the value of the AdminEmail
parameter. The other values can be stay as they are. Look at the ImageId
parameter value. This is the way of getting a value out of a JSON artifact file in CoePipeline.
To run the acceptance tests, you also need another CodeBuild project, add the following resources to the Resources
section of deploy/pipeline_ec2.yml
:
ExtendedCodeBuildRole: |
Now the CodeBuild project needs to be called in the pipeline, therefore change Pipeline
resource in the file deploy/pipeline_ec2.yml
to:
- deploy the CloudFormation stack suffixed with
-acceptance
- run the acceptance tests
Pipeline: |
The acceptance stage is now ready.
Production stage
The production stage is pretty simple, just one CloudFormation stack. Change the Pipeline
resource in the file deploy/pipeline_ec2.yml
to add a new stage that looks familiar to the acceptance stage:
Pipeline: |
Now the AMI containing the application is deployed to production with confidence and without disturbing the users. Try it and run the pipeline!
Summary
Let’s use my production-ready definition to summarize how each point is implemented:
- Highly available: The load balancer (which is HA) sits in front of a fleet of EC2 instances managed by the Auto Scaling Group for maximum availability. In case of an unhealthy instance, the Auto Scaling Group will replace that instance.
- Scalable: If the CPU utilization gets over 70%, a Cloud Watch Alarm triggers a Scaling Policy to add new instances automatically.
- Frictionless deployment: To deploy a new version of the app, a new AMI is created. This AMI is then rolled out to the acceptance environment by updating the CloudFormation stack with the new ImageId parameter. CloudFormation and the Auto Scaling Group perform a rolling update to avoid the application being down during deployment. If the application can not be started the Rolling Update fails and CloudFormation rolls back.
- Secure: During AMI creation, the latest patches are applied. You must ensure that the pipeline runs often enough to keep up with new patches. Besides that, Security Groups control network traffic to the EC2 instances. When following the bastion host approach, you get maximum security. The EC2 instance is only allowed to send logs to CloudWatch Logs by following the least privileges approach.
- Operations: All logs are stored in CloudWatch Logs, important metrics are monitored and alarms are defined.
If you now have the impression that running an app on EC2 is a lot of work you are right. In the next two articles, you will learn about other options with fewer responsibilities.
Series
- Set the assembly line up
- Local development environment
- CI/CD Pipeline as Code
- Running your application
- EC2 based app
a. Infrastructure
b. CI/CD Pipeline (you are here) - Containerized ECS based app
a. Infrastructure
b. CI/CD Pipeline - Serverless app
- Summary
You can find the source code on GitHub.
Leave a Reply