Automate cloud cost estimation in your pull requests
Cloud computing is awesome for growing and scaling up infrastructure quickly to follow product needs. However, this elasticity and flexibility come at a cost. Reviewing your bill daily is crucial to avoid unpleasant surprises at the end of the month. Cloud providers offer tools (here are some examples for AWS) and good practices to help you saving money.
Most of the time, infrastructure as code (IaC) is employed to maintain and facilitate the evolution of this underlying. Like any other type of language, it should follow the same good practices and pass the code review step with other contributors. As mentioned before, cloud infrastructure has a cost, and invoicing is not always highlighted at this stage. Contributors have to refer to external resources and make estimations themselves. It is not automated and may be cumbersome over time.
In this article, I present you Infracost which is a command-line tool (CLI) that may permit you to include the cost dimension during the infrastructure as code development. As well as test-driven development, why not doing cost-driven development in the same logic? Through these lines, you'll know how to estimate the cost of your full infrastructure or the changes you want to introduce. Finally, we'll see how to integrate it in GitHub action to include cost estimation in your pull requests.
Cost-Driven Infrastructure Development
Presentation of Infracost
"Infracost shows cloud cost estimates for infrastructure-as-code projects such as Terraform. It helps DevOps, SRE and developers to quickly see a cost breakdown and compare different options upfront." https://github.com/infracost/infracost
Infracost is a binary CLI tool and so quickly installable on multiple OS :
For now, it supports only Terraform but another infrastructure as code tools seems planned to be supported (Pulumi, CloudFormation).
Once Infracost is installed, you need to register to receive a free API key :
After registration, the API key is saved in ~/.config/infracost/credentials.yml.
Infracost can be executed to have the full monthly breakdown of costs :
You can play modifying the code and re-run the previous command to have an updated cost estimation of the new version.
Diff monthly costs between the current and planned state
Terraform updates, deletes, or creates resources saves the infrastructure state in a file. When you deploy a new version, Terraform is able to detect what needs to be updated, removed, created following your changes. Infracost relies on this state file to show you the cost impact of the current change from the plan.
To illustrate to you that, I deployed the initial code version using terraform apply. Then I scaled out the current EC2 instance type from t2.micro to m4.large:
Then I launch Infracost command with diff argument :
The diff usage is perfect to be added in complement of a code review to justify and argue the cost of a change.
Refine cost estimation with usage file
You can feed Infracost with a usage file to refine the cost estimation :
instances: 15 # Number of instances in the autoscaling group.
operating_system: linux # Override the operating system of the instance, can be: linux, windows, suse, rhel.
reserved_instance_type: standard # Offering class for Reserved Instances. Can be: convertible, standard.
reserved_instance_term: 1_year # Term for Reserved Instances. Can be: 1_year, 3_year.
reserved_instance_payment_option: no_upfront # Payment option for Reserved Instances. Can be: no_upfront, partial_upfront, all_upfront.
# Only applicable when T2 credit_specification is set to unlimited or T3 & T4 instance types are used within a launch template, or T3 & T4 instance types are used in a launch configuration.
monthly_cpu_credit_hrs: 350 # Number of hours in the month where the instance is expected to burst.
vcpu_count: 2 # Number of the vCPUs for the instance type.
new_connections: 10000 # Number of newly established connections per second on average.
active_connections: 10000 # Number of active connections per minute on average.
processed_bytes_gb: 1000 # The number of bytes processed by the load balancer for HTTP(S) requests and responses in GB.
rule_evaluations: 10000 # The product of number of rules processed by the load balancer and the request rate.
This file is added to the command arguments :
You can generate reports to share cost estimation in different formats (HTML, JSON, etc..) :
Cost-Estimation Review With GitHub Action
You can integrate Infracost with a lot of CI/CD solutions. Here we’re going to focus on Github Action because it is easy to reuse existing actions from the marketplace. In the workflow, we’re going to use the following actions :