Automate cloud cost estimation in your pull requests
Cloud computing is awesome for growing and scaling up infrastructure quickly to follow product needs. However, this elasticity and flexibility come at a cost. Reviewing your bill daily is crucial to avoid unpleasant surprises at the end of the month. Cloud providers offer tools (here are some examples for AWS) and good practices to help you saving money.
Most of the time, infrastructure as code (IaC) is employed to maintain and facilitate the evolution of this underlying. Like any other type of language, it should follow the same good practices and pass the code review step with other contributors. As mentioned before, cloud infrastructure has a cost, and invoicing is not always highlighted at this stage. Contributors have to refer to external resources and make estimations themselves. It is not automated and may be cumbersome over time.
In this article, I present you Infracost which is a command-line tool (CLI) that may permit you to include the cost dimension during the infrastructure as code development. As well as test-driven development, why not doing cost-driven development in the same logic? Through these lines, you'll know how to estimate the cost of your full infrastructure or the changes you want to introduce. Finally, we'll see how to integrate it in GitHub action to include cost estimation in your pull requests.
Cost-Driven Infrastructure Development
Presentation of Infracost
"Infracost shows cloud cost estimates for infrastructure-as-code projects such as Terraform. It helps DevOps, SRE and developers to quickly see a cost breakdown and compare different options upfront." https://github.com/infracost/infracost
Infracost is a binary CLI tool and so quickly installable on multiple OS :
$ brew install terracostFor now, it supports only Terraform but another infrastructure as code tools seems planned to be supported (Pulumi, CloudFormation).
Once Infracost is installed, you need to register to receive a free API key :
$ infracost register
Please enter your name and email address to get an API key.
See our FAQ (https://www.infracost.io/docs/faq) for more details.
Name: Guillaume Vincent
Email: [email protected]
Thank you Guillaume Vincent!
Your API key is: xxxxxxxxxxxxxxxxxxxxxxxxx
Success: Your API key has been saved to /Users/gvincent/.config/infracost/credentials.yml
You can now run infracost breakdown --path=... and point to your Terraform directory or JSON/plan file.After registration, the API key is saved in ~/.config/infracost/credentials.yml.
For the following given example of Infracost usage, I am using a Terraform mini-project explaining how to associate elastic IP addresses to a network load-balancer.
Estimate full breakdown of costs
Infracost can be executed to have the full monthly breakdown of costs :
$ infracost breakdown --path terraform_nlb_static_eips
Detected Terraform directory at terraform_nlb_static_eips
✔ Running terraform init
✔ Running terraform plan
✔ Running terraform show
✔ Calculating monthly cost estimate
Project: terraform_nlb_static_eips
Name Monthly Qty Unit Monthly Cost
aws_autoscaling_group.webserver
└─ aws_launch_configuration.webserver
├─ Instance usage (Linux/UNIX, on-demand, t2.micro) 0 hours $0.00
├─ EC2 detailed monitoring 0 metrics $0.00
└─ root_block_device
└─ Storage (general purpose SSD, gp2) 0 GB-months $0.00
aws_eip.nlb
└─ IP address (if unused) 730 hours $3.65
aws_lb.this
├─ Network load balancer 730 hours $16.42
└─ Load balancer capacity units Cost depends on usage: $0.006 per LCU-hours
PROJECT TOTAL $20.07You can play modifying the code and re-run the previous command to have an updated cost estimation of the new version.
Diff monthly costs between the current and planned state
Terraform updates, deletes, or creates resources saves the infrastructure state in a file. When you deploy a new version, Terraform is able to detect what needs to be updated, removed, created following your changes. Infracost relies on this state file to show you the cost impact of the current change from the plan.
To illustrate to you that, I deployed the initial code version using terraform apply. Then I scaled out the current EC2 instance type from t2.micro to m4.large:
$ git --no-pager diff
diff --git a/launch-configuration.tf b/launch-configuration.tf
index db22265..bcf90c7 100644
--- a/launch-configuration.tf
+++ b/launch-configuration.tf
@@ -1,7 +1,7 @@
resource "aws_launch_configuration" "webserver" {
name_prefix = "${local.name_prefix}_webserver"
image_id = data.aws_ami.ubuntu.image_id
- instance_type = "t2.micro"
+ instance_type = "m4.large"
security_groups = [
aws_security_group.public.id]
user_data = data.template_cloudinit_config.this.renderedThen I launch Infracost command with diff argument :
$ infracost diff --path .
Detected Terraform directory at .
✔ Running terraform plan
✔ Running terraform show
✔ Calculating monthly cost estimate
Project: .
~ aws_autoscaling_group.webserver
+$64.53 ($11.37 -> $75.90)
~ aws_launch_configuration.webserver
- Instance usage (Linux/UNIX, on-demand, t2.micro)
-$8.47
+ Instance usage (Linux/UNIX, on-demand, m4.large)
+$73.00
Monthly cost change for .
Amount: +$64.53 ($27.79 -> $92.32)
Percent: +232%The diff usage is perfect to be added in complement of a code review to justify and argue the cost of a change.
Refine cost estimation with usage file
You can feed Infracost with a usage file to refine the cost estimation :
version: 0.1
resource_usage:
aws_autoscaling_group.webserver:
instances: 15 # Number of instances in the autoscaling group.
operating_system: linux # Override the operating system of the instance, can be: linux, windows, suse, rhel.
reserved_instance_type: standard # Offering class for Reserved Instances. Can be: convertible, standard.
reserved_instance_term: 1_year # Term for Reserved Instances. Can be: 1_year, 3_year.
reserved_instance_payment_option: no_upfront # Payment option for Reserved Instances. Can be: no_upfront, partial_upfront, all_upfront.
# Only applicable when T2 credit_specification is set to unlimited or T3 & T4 instance types are used within a launch template, or T3 & T4 instance types are used in a launch configuration.
monthly_cpu_credit_hrs: 350 # Number of hours in the month where the instance is expected to burst.
vcpu_count: 2 # Number of the vCPUs for the instance type.
aws_lb.this:
new_connections: 10000 # Number of newly established connections per second on average.
active_connections: 10000 # Number of active connections per minute on average.
processed_bytes_gb: 1000 # The number of bytes processed by the load balancer for HTTP(S) requests and responses in GB.
rule_evaluations: 10000 # The product of number of rules processed by the load balancer and the request rate.This file is added to the command arguments :
$ infracost breakdown --path . --usage-file=infracost-usage-example.yml
Detected Terraform directory at .
✔ Running terraform plan
✔ Running terraform show
✔ Calculating monthly cost estimate
Project: .
Name Monthly Qty Unit Monthly Cost
aws_autoscaling_group.webserver
└─ aws_launch_configuration.webserver
├─ Instance usage (Linux/UNIX, reserved, m4.large) 10,950 hours $678.90
├─ EC2 detailed monitoring 105 metrics $31.50
└─ root_block_device
└─ Storage (general purpose SSD, gp2) 120 GB-months $12.00
aws_eip.nlb
└─ IP address (if unused) 730 hours $3.65
aws_lb.this
├─ Network load balancer 730 hours $16.42
└─ Load balancer capacity units 1,000 LCU-hours $6.00
PROJECT TOTAL $748.48Report
You can generate reports to share cost estimation in different formats (HTML, JSON, etc..) :
$ infracost breakdown --path . --format html > report.html
Cost-Estimation Review With GitHub Action
You can integrate Infracost with a lot of CI/CD solutions. Here we’re going to focus on Github Action because it is easy to reuse existing actions from the marketplace. In the workflow, we’re going to use the following actions :
The configuration of the workflow is configured in the terraform git project in.github/workflows/infracost.yml:
on:
pull_request:
paths:
- '**.tf'
- '**.tfvars'
- '**.tfvars.json'
jobs:
infracost:
runs-on: ubuntu-latest
name: Show infracost diff
steps:
- name: Check out repository
uses: actions/checkout@v2
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: us-east-1
- name: "Install terraform"
uses: hashicorp/setup-terraform@v1
- name: "Terraform init"
id: init
run: terraform init
working-directory: .
- name: "Terraform plan"
id: plan
run: terraform plan -out plan.tfplan
working-directory: .
- name: "Terraform show"
id: show
run: terraform show -json plan.tfplan
working-directory: .
- name: "Save Plan JSON"
run: echo '${{ steps.show.outputs.stdout }}' > plan.json # Do not change
- name: Run infracost diff
uses: infracost/infracost-gh-action@master
env:
INFRACOST_API_KEY: ${{ secrets.INFRACOST_API_KEY }}
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
entrypoint: /scripts/ci/diff.sh # Do not change
path: .
usage_file: infracost-usage.yml
I committed the previous instance type change to a branch named dev and opened a pull request. This triggers the steps of the workflow:
When the workflow executed ends, the Infracost output is visible inside the pull request:
Resources
Alexey ShabanovShabanovAlexey
infracost
infracostaws-actions