Simulate and test your code interactions with AWS
Do you need to test some pieces of code interacting with AWS? In this article, Iโm going to introduce the Moto library whose aim is to mock out AWS services. That means you wonโt have to deploy anything in the AWS cloud to test your code.
In the first part, weโll create a simple piece of code interacting with the Elastic Compute (EC2) service. Weโll examine the different ways of mocking provided by Moto. This will give you an overview of its possibilities.
Next, weโll couple the Moto library and pytest framework to have clean and scalable unit tests.
Prerequisites
First of all, we need to install the different Python libraries required using pip
:
$ python3 -m pip install boto3 moto pytest
Overview of the Moto Library
โMoto is a library that allows your tests to easily mock out AWS Services.โ โ moto on GitHub
Letโs create a main.py
containing a simple function that creates EC2 instances:
import boto3
def create_instances(region_name, ami_id, count):
client = boto3.client('ec2', region_name=region_name)
client.run_instances(ImageId=ami_id, MinCount=count, MaxCount=count)
@guivin
Moto is not limited to the EC2 service. It supports many other AWS services to mock out. Also, the library offers multiple usages that weโll see in the following sections.
We will be going over two tests:
- Assert that there is the instance count as expected.
- Assert that the EC2 instance is running the correct Amazon Machine Image (AMI).
Decorator
AWS services can be mocked out using a simple decorator:
import boto3
from moto import mock_ec2
from main import create_instances
@mock_ec2
def test_create_instances_with_decorator(self):
instance_count = 1
image_id = 'ami-02ed82f3a38303e6f'
create_instances('us-east-1', image_id, instance_count)
client = boto3.client('ec2', region_name='us-east-1')
instances = client.describe_instances()['Reservations'][0]['Instances']
assert len(instances) == instance_count
assert instances[0]['ImageId'] == image_id
Context manager
You can also use Moto mocks as a context manager:
import boto3
from main import create_instances
from moto import mock_ec2
def test_create_instances_with_context_manager(self):
with mock_ec2():
instance_count = 1
image_id = 'ami-02ed82f3a38303e6f'
create_instances('us-east-1', image_id, instance_count)
client = boto3.client('ec2', region_name='us-east-1')
instances = client.describe_instances()['Reservations'][0]['Instances']
assert len(instances) == instance_count
assert instances[0]['ImageId'] == image_id
@guivin
Raw use
And finally, you can go to the raw approach:
import boto3
from main import create_instances
from moto import mock_ec2
def test_create_instances_raw(self):
mock = mock_ec2()
mock.start()
instance_count = 1
image_id = 'ami-02ed82f3a38303e6f'
create_instances('us-east-1', image_id, instance_count)
client = boto3.client('ec2', region_name='us-east-1')
instances = client.describe_instances()['Reservations'][0]['Instances']
assert len(instances) == instance_count
assert instances[0]['ImageId'] == image_id
mock.stop()
Couple Moto With Pytest
Pytest is a framework to create small and scalable unit tests. It provides test fixtures that are functions running before the test functions. It is great to feed some data or set up a database connection, for example. Here, weโre going to define fixtures to configure S3 mocking and fake AWS credentials. By default, pytest loads the content of conftest.py
:
import boto3
import pytest
import os
from moto import mock_s3
pytest.aws_region = 'us-east-1'
@pytest.fixture(scope='function')
def aws_credentials():
os.environ['AWS_ACCESS_KEY_ID'] = 'testing'
os.environ['AWS_SECRET_ACCESS_ID'] = 'testing'
os.environ['AWS_SECURITY_TOKEN'] = 'testing'
os.environ['AWS_SESSION_TOKEN'] = 'testing'
@pytest.fixture(scope='function')
def s3(aws_credentials):
with mock_s3():
yield boto3.client('s3', region_name=pytest.aws_region)
In test.py
, we use a context manager to create S3 before running the tests present in a dedicated class. There is a test checking if the S3 bucket exists and another testing the placement of an object:
from contextlib import contextmanager
@contextmanager
def s3_bucket(s3, bucket_name):
s3.create_bucket(Bucket=bucket_name)
yield
class TestClassS3:
def test_create_bucket(self, s3, bucket_name='testing'):
with s3_bucket(s3, bucket_name):
result = s3.list_buckets()
assert len(result['Buckets']) == 1
assert result['Buckets'][0]['Name'] == bucket_name
def test_put_object(self, s3, bucket_name='testing', key='testing', body='testing'):
with s3_bucket(s3, bucket_name):
s3.put_object(Bucket=bucket_name, Key=key, Body=body)
object_response = s3.get_object(Bucket=bucket_name, Key=key)
assert object_response['Body'].read().decode() == body
Launch the unit tests with the pytest
command:
$ pytest test.py
Conclusion
In this article, weโve seen how to mock AWS responses using Moto. This allows for free and quick testing of your Python code. Coupled with pytest, it provides scalable and clean unit tests for AWS interactions.
Resources
