How To Build Docker Image With Ansible and Packer

Build Image from Ansible code and persist them on local or in AWS ECR

In a previous article, we have seen how to create a testable Ansible role with Molecule and Docker. This role installs and configures WordPress in a Docker image. This part was only covered tests using Molecule. Here we are going to go further to reuse this work to build the Docker image with Packer.

We will see how to configure Packer through manifests. There will be two manifests, one to build the image locally and the other remotely. We will persist the remote image in AWS Elastic Registry (ECR). For both manifests, we will use the Ansible provisioner and Docker post-processors. The provisioner will reuse the WordPress role. The post-processors will build persist the image in the Docker registry.

We will use Terraform to deploy and configure AWS ECR. Then we will create a Packer manifest to build the image on local. A docker-compose file will run the image to ensure all is working well. Finally, we will create the second Packer manifest to tag and push the image to AWS ECR.

Install The Prerequisites

The prerequisites presented here are for MacOSX. Adapt versions to your platform architecture.

Install Packer:

$ wget https://releases.hashicorp.com/packer/1.7.2/packer_1.7.2_darwin_amd64.zip -P $HOME/.local/bin
$ unzip $HOME/.local/bin/packer_1.7.2_darwin_amd64.zip -d $HOME/.local/bin

Installation of the Packer binary

Install Terraform :

$ wget https://releases.hashicorp.com/terraform/0.15.4/terraform_0.15.4_darwin_amd64.zip -P $HOME/.local/bin
$ unzip $HOME/.local/bin/terraform_0.15.4_darwin_amd64.zip -d $HOME/.local/bin/

Installation of the Terraform binary

Assume you have python and pip installed and install Ansible :

$ pip install --user ansible==2.9.0

Installation of Ansible using pip

Install docker-compose:

$ pip install --user docker-compose

Installation of Docker-Compose using pip

For Docker, I use Docker for Desktop

Deploy AWS Elastic Container Registry (ECR)

At the project root, create the terraform structure and the aws-ecr module:

$ mkdir -p terraform/environments terraform/modules
$ mkdir -p terraform/environments/dev/aws-ecr

Creation of the aws-ecr layer for development environment

The module state is stored in AWS S3. You have to create your own S3 bucket and replace the value in terraform/environments/dev/aws-ecr/main.tf:

terraform {
  required_version = "> 0.15.0"
  backend "s3" {
	bucket  = "<YOUR_S3_BUCKET>"
	key     = "dev/ecs-ansible-packer-terraform-wordpress/aws-ecr.tf"
	encrypt = true
	region  = "us-east-1"
  }
}

module "aws-ecr" {
  source = "../../../modules/aws-ecr"
  region = "us-east-1"
  tags   = {
	environment = "dev"
	project     = "ecs-wordpress"
	terraform   = true
  }
}

output "arn" {
  value = module.aws-ecr.arn
}

output "repository_url" {
  value = module.aws-ecr.repository_url
}

terraform/environments/dev/aws-ecr/main.tf

Just below, you have the content of the module aws-ecr files:

resource "aws_ecr_repository" "default" {
  name = "${var.tags["environment"]}-${var.tags["project"]}"
  tags = var.tags
}

main.tf

output "arn" {
  description = "The ECR ARN"
  value       = aws_ecr_repository.default.arn
}

output "repository_url" {
  description = "The ECR repository URL"
  value       = aws_ecr_repository.default.repository_url
}

outputs.tf

terraform {
  required_providers {
	aws = {
	  source  = "hashicorp/aws"
	  version = "~> 3.46.0"
	}
  }
}

provider "aws" {
  region = var.region
}

providers.tf

Initialize and apply the layer:

$ cd terraform/environments/dev/aws-ecr
$ terraform init
$ terraform apply

Initialization and apply of the aws-ecr layer

Connect to AWS web console and ensure the registry is ready:

Create The Ansible File Hierarchy

At the root of the project, create the ansible directory:

$ mkdir -p ansible/playbooks ansible/roles

Creation of the Ansible roles and playbooks directories

Create the WordPress playbook in ansible/playbooks/wordpress.yml:

---
- hosts: all
  become: yes

  roles:
    - wordpress

ansible/playbooks/wordpress.yml

The WordPress ansible of the previous article needs to be put into ansible/roles .

Persist The Docker Image Locally

Create the packer directory:

$ mkdir packer

Creation of the packer directory

We create a script to install ansible in the container before running the Ansible provisioned:

$ mkdir scripts

Creation of the scripts directory

Here the scripts/install-ansible.sh :

#!/bin/sh

# Hide warnings, we'll use aptitude instead of apt later
apt update -y 2>/dev/null | grep packages | cut -d '.' -f 1
apt install -y aptitude 2>/dev/null | grep packages | cut -d '.' -f 1

aptitude install -y bash python3 python3-pip
python3 -m pip install --upgrade pip wheel setuptools
# Disable rust due to https://github.com/pyca/cryptography/issues/5776
python3 -m pip install ansible==${ANSIBLE_VERSION}

scripts/install-ansible.sh

There is also another script scripts/cleanup.sh to clean up the container after Ansible has finished :

#!/bin/sh

/usr/bin/yes | python3 -m pip uninstall ansible
aptitude remove -y python3-pip
@guivin

scripts/cleanup.sh

Here is the Packer manifest to build the image:

{
  "variables": {
	"ansible_playbook_dir": "./ansible",
	"ansible_playbook_file": "./ansible/playbooks/wordpress.yml",
	"ansible_version": "2.9",
	"docker_base_image": "php:7.2-apache",
	"docker_image_version": "{{env `IMAGE_VERSION`}}"
  },
  "builders":[{
	"type": "docker",
	"image": "{{user `docker_base_image`}}",
	"commit": true,
	"changes": [
	  "VOLUME /var/www/html",
	  "ENTRYPOINT [\"/opt/entrypoint.sh\"]",
	  "CMD [\"apache2-foreground\"]"
	]
  }],
  "provisioners": [
	{
	  "type": "shell-local",
	  "command": "ansible-galaxy install -p ../ansible/roles -r ansible/requirements.yml"
	},
	{
	  "type": "shell",
	  "script": "scripts/install-ansible.sh",
	  "environment_vars": [
		"ANSIBLE_VERSION={{user `ansible_version`}}"
	  ]
	},
	{
	  "type": "ansible-local",
	  "playbook_dir": "{{user `ansible_playbook_dir`}}",
	  "playbook_file": "{{user `ansible_playbook_file`}}"
	},
	{
	  "type": "shell",
	  "script": "scripts/cleanup.sh"
	}
  ],
  "post-processors": [
	[
	  {
		"type": "docker-tag",
		"repository": "guivin/wordpress",
		"tag": "{{user `docker_image_version`}}"
	  }
	]
  ]
}

packer/wordpress-local.json

Build the image on local docker. The image version is passed as a variable :

$ IMAGE_VERSION=latest packer build packer/wordpress-local.json

Build the docker image on local with Packer

You can check the image is present:

$ docker images | grep -i guivin/wordpress

List the Docker image built by Packer

Test Local Image With Docker-Compose

---
wordpress:
  image: guivin/wordpress
  ports:
    - 80:80
  links:
    - db:mysql
  dns: 8.8.8.8
  environment:
    WP_VERSION: 5.4.2
    TZ: Europe/Paris
db:
  image: mysql:5.7
  environment:
    MYSQL_ROOT_PASSWORD: xxxxxx

docker-compose.yml

$ docker-compose up

Deploy the docker-compose stack

Once WordPress is ready, you can access it on localhost:

Persist The Image In AWS ECR

Comparing to the previous manifest, the one for AWS ECR adds docker-push post-processor :

{
  "variables": {
	"aws_access_key_id": "{{env `AWS_ACCESS_KEY_ID`}}",
	"aws_secret_access_key": "{{env `AWS_SECRET_ACCESS_KEY`}}",
	"ansible_playbook_dir": "./ansible",
	"ansible_playbook_file": "./ansible/playbooks/wordpress.yml",
	"ansible_version": "2.9",
	"docker_base_image": "php:7.2-apache",
	"docker_repository": "{{env `DOCKER_REPOSITORY`}}",
	"docker_image_version": "{{env `IMAGE_VERSION`}}"
  },
  "builders":[{
	"type": "docker",
	"image": "{{user `docker_base_image`}}",
	"commit": true,
	"changes": [
	  "VOLUME /var/www/html",
	  "ENTRYPOINT [\"/opt/entrypoint.sh\"]",
	  "CMD [\"apache2-foreground\"]"
	]
  }],
  "provisioners": [
	{
	  "type": "shell-local",
	  "command": "ansible-galaxy install -p ../ansible/roles -r ansible/requirements.yml"
	},
	{
	  "type": "shell",
	  "script": "scripts/install-ansible.sh",
	  "environment_vars": [
		"ANSIBLE_VERSION={{user `ansible_version`}}"
	  ]
	},
	{
	  "type": "ansible-local",
	  "playbook_dir": "{{user `ansible_playbook_dir`}}",
	  "playbook_file": "{{user `ansible_playbook_file`}}"
	},
	{
	  "type": "shell",
	  "script": "scripts/cleanup.sh"
	}
  ],
  "post-processors": [
	[
	  {
		"type": "docker-tag",
		"repository": "{{user `docker_repository`}}",
		"tag": "{{user `docker_image_version`}}"
	  },
	  {
		"type": "docker-push",
		"ecr_login": true,
		"aws_access_key": "{{user `aws_access_key_id`}}",
		"aws_secret_key": "{{user `aws_secret_access_key`}}",
		"login_server": "https://{{user `docker_repository`}}"
	  }
	]
  ]
}

packer/wordpress.json

Build the image. The docker repository URL is taken from the outputs of the aws-ecr module and passed as a variable :

$ DOCKER_REPOSITORY=`terraform -chdir=terraform/environments/dev/aws-ecr output -json | jq -r .repository_url.value` IMAGE_VERSION=${VERSION} packer build packer/wordpress.json

Build and push the Docker image to the AWS ECR

Conclusion

We have seen how to reuse an existing Ansible role to create Docker images with Packer. Here we have worked with a local Docker registry and a remote one in AWS ECR deployed with Terraform. The two methods have their own Packer manifest. We have checked the generated image was working with docker-compose.

Now all is in place to keep working on AWS. In the next article of this series, we will see how to deploy the WordPress image in AWS. We will use managed services like AWS Relational Database Service (RDS) and AWS Elastic Container Service (ECS).