Configuration and Change Management with Packer & Ansible

This article explains the process of automating configuration and change management using Packer to build a machine image & using Ansible as a provisioner to the Packer script. The image will be built on Amazon AWS and subsequently used to provision an Amazon EC2 instance for a basic express application. We will also use NGINX for reverse-proxy.

Configuration & Change Management

Configuration and Change Management:

Software development requires a continuous cycle of update of features or addition of new features and this often translate to rescaling of the product infrastructure by DevOps Engineers to ensure proper deployment and functioning of the product.
Configuration and change management are two different but complementary concepts. Configuration Management serves as a version control system for a product, processes, plans, baselines, and takes care of how changes should be handled when scaling up or down. The changes includes the versions and updates that have been applied to installed software packages and the locations and network addresses of hardware devices. Change Management focuses on how any change to the process, or controls should be done.

Benefits of Configuration and Change Management for Servers:

Reliability: It ensures that the configuration of a systems is well defined, rather than relying on vague ideas from a DevOps engineer or team. The system ensures clarity of requirements needed to handle changes thereby avoiding problems. In the event of failures, the system can be recover speedily due to proper documentation and automation of processes.
Organisation: With a documented system, connecting past records of the infrastructure and making informed decisions to address changes in the future becomes pretty easy to come by.
Cost and Risk Reduction: A substantial knowledge base of the configuration of the system ensures effective maintenance of the system and prevents wasteful duplication of infrastructure.

Tools for Configuration Management:

The choice of configuration management tools to be adopted is relative to the operating environment and the problems to be addressed. Below are some configuration management tools:
Ansible: This is an open-source automation software, built on the popular Python language, for provisioning, configuration management, and application deployment.
Vagrant: This is an open-source tool that can be used to create and configure reproducible virtual development environments. It built on Ruby and works on top of virtualisation products such as VirtualBox, VMware etc. There are a host of other configuration management tools which are not mentioned here.

BUILDING A MACHINE IMAGE WITH PACKER AND ANSIBLE ON AWS

If you’re using Homebrew on a macOS, install Packer using

$ brew install packer

If you're using Chocolatey on a windows machine, use

$ choco install packer

We would be building an AWS machine image using a packer template obtained from packer hashicorp website and modified to suit the application described in this write up.
First is to create a .env file that would contain the secrete key and access id of your AWS account. Follow this format:

export AWS_ACCESS_KEY_ID=
export AWS_SECRET_ACCESS_KEY=
export AWS_REGION=

Note: Create a .gitignore file and add the .env file to it, so that it doesn’t get pushed to Github repository. If you're using VSCODE, you can install ignoreit, an extension I created to help ensure that your .env file is never pushed to source control (it also automatically creates a .env.example file for you).
The next step is to create a directory named packer and create a file named template.json. The file should contain the script below:

{
  "variables": {
    "aws_access_key": "{{env `AWS_ACCESS_KEY_ID`}}",
    "aws_secret_key": "{{env `AWS_SECRET_ACCESS_KEY`}}"
    "aws_region": "{{env `AWS_REGION`}}"
  },
  "builders": [{
    "type": "amazon-ebs",
    "access_key": "{{user `aws_access_key`}}",
    "secret_key": "{{user `aws_secret_key`}}",
    "region": "{{user `aws_region`}}",
    "source_ami_filter": {
      "filters": {
        "virtualization-type": "hvm",
        "name": "ubuntu/images/*ubuntu-xenial-16.04-amd64-server-*",
        "root-device-type": "ebs"
      },
    "owners": ["099720109477"],
    "most_recent": true
  },
  "instance_type": "t2.micro",
  "ssh_username": "ubuntu",
  "ami_name": "frontend-app"
  }],
  "provisioners": [{
    "type": "shell",
    "script": "../ansible/install_ansible.sh"
   }, {
    "type": "ansible",
    "playbook_file": "../ansible/ansible_playbook.yml"
  }]
}

The template file above comprises several key-value pairs representing specifics about the application environment. The keywords are explained in packer docs here.
As mentioned earlier, Ansible will be used with Packer to provision the image.
Create a directory named ansible. It should contain two files; install_ansible.sh and ansible_playbook.yml. As the name implies, shell file will contain shell script that will install ansible into the Amazon EC2 instance that we want to be provision while the machine image is being created. This makes it possible for ansible modules to be available for use in the EC2 instance. The playbook file should contain:

installAnsible () {
  sudo apt-get update -y
  sudo apt-get install software-properties-common -y
  sudo apt-add-repository ppa:ansible/ansible -y
  sudo apt-get update -y
  sudo apt-get install ansible -y
}
installAnsible

Fill up the ansible_playbook with the script below:

---
- hosts: all
  remote_user: ubuntu
  become_method: sudo
  become: true
  vars:
    project_name: "Basic Express App"
    project_path: /home/ubuntu/basic-express-app
    sites_available: /etc/nginx/sites-available
    sites_enabled: /etc/nginx/sites-enabled"
    sites_available_express: /etc/nginx/sites-available/express
    sites_enabled_express: /etc/nginx/sites-enabled/express
    PM2_PATH: $PATH:/usr/bin /usr/lib/node_modules/pm2/bin/pm2 startup systemd -u ubuntu --hp /home/ubuntu
  tasks:
    - name: Reset contents of apt list
      shell: |
        sudo rm -rf /var/lib/apt/lists/*
        sudo apt-get update
    - name: Get Nodejs gpg key
      apt_key:
        url: "https://deb.nodesource.com/gpgkey/nodesource.gpg.key"
        state: present
    - name: Install Nodejs version 10 binary
      apt_repository:
        repo: "deb https://deb.nodesource.com/node_10.x {{ ansible_distribution_release }} main"
        state: present
    - name: Install Node
      apt:
        name: nodejs
        state: present
    - name: Clone the repository
      git:
        repo: https://github.com/marcdomain/basic-express-app.git
        dest: "{{ project_path }}"
    - name: Install node packages
      shell: |
        npm install
      args:
        chdir: "{{ project_path }}"
    - name: Install nginx
      apt:
        name: nginx
        state: latest
    - name: Delete nginx default file
        file:
          path: "{{ sites_available }}/default"
          state: absent
    - name: Configure nginx server
      shell: |
        echo "
          server  {
            listen 80;
            location / {
              proxy_pass http://127.0.0.1:3000;
            }
          }
        " > {{ sites_available_express }}
    - name: Update nginx symlink
      file:
        src={{ sites_available_express }}
        dest={{ sites_enabled_express }}
        state=link
    - name: Start nginx
      service:
        name: nginx
        state: started
    - name: Install pm2 to run app in background
      shell: npm install pm2 -g
    - name: Create pm2 start script
      shell: |
        cd /home/ubuntu/basic-express-app
        echo '
          {
            "apps": [{
               "name": "authors-haven",
               "script": "npm",
               "args": "start"
            }]
          }
        ' > start_script.config.json
    - name: Start app with pm2
      shell: |
        cd /home/ubuntu/basic-express-app
        sudo pm2 start ./start_script.config.json
        sleep 10
        sudo pm2 startup
        sudo env PATH={{PM2_PATH}}
        sudo pm2 save

Working with ansible playbook is easy because it has built-in modules that can be applied to implement different task with just a few codes. A comprehensive list and description of the modules can be found here.
To build the image, create a file called build.sh and fill it with.

. .env
packer build packer/template.json

The first line above would source the env file, while the second line would build the machine image. To trigger the build, use bash to run the command below and watch the output on your terminal as the image is being built.

$ bash build.sh

Provisioning EC2 instance with the AMI you created

After the image is built, go to your Amazon EC2 dashboard and click the AMI tab. From here, you can launch the image and access the application with the public ip address assigned to the EC2 instance. Follow the guides below:

• From your AWS console (ensure you’re in the same region the image was created), click on Services at the top menu and select EC2.
• Navigate to the left hand menu, under IMAGES, click on AMI.
• From the list of images presented to you, identify the image you just created and select it. Click on launch.
• Choose an instance type, and then choose Next: Configure Instance Details. t2.micro will be selected by default since the image was built with this instance_type.
• Click Next: Add Storage
• Click Next: Add Tags
• Add tags (optional) and proceed to create security group.
• In the port range column, input the port your app is running on.
• Set the source IP to 0.0.0.0/0 to allow traffic from all sources.
• Click “Review and Launch” to review your settings.
• Then click launch.
• Select an existing key pair or create a new key pair, select the acknowledge agreement box, and then choose Launch Instances.
• Navigate to EC2 dashboard and click on instances to view the instance you just created.
• Click on the instance from the list presented.
• Copy the IP address and past it in your browser URL to access the application.

GitHub repo link