How To Mock AWS Services in Python Unit Tests

Do you need to test some pieces of code interacting with AWS? In this article, I’m going to introduce the Moto library whose aim is to mock out AWS services. That means you won’t have to deploy anything in the AWS cloud to test your code.

In the first part, we’ll create a simple piece of code interacting with the Elastic Compute (EC2) service. We’ll examine the different ways of mocking provided by Moto. This will give you an overview of its possibilities.

Next, we’ll couple the Moto library and pytest framework to have clean and scalable unit tests.

Prerequisites

First of all, we need to install the different Python libraries required using pip:

$ python3 -m pip install boto3 moto pytest

Overview of the Moto Library

“Moto is a library that allows your tests to easily mock out AWS Services.” — moto on GitHub

Let’s create a main.py containing a simple function that creates EC2 instances:

import boto3


def create_instances(region_name, ami_id, count):
	client = boto3.client('ec2', region_name=region_name)
	client.run_instances(ImageId=ami_id, MinCount=count, MaxCount=count)
@guivin

main.py

Moto is not limited to the EC2 service. It supports many other AWS services to mock out. Also, the library offers multiple usages that we’ll see in the following sections.

We will be going over two tests:

Assert that there is the instance count as expected.
Assert that the EC2 instance is running the correct Amazon Machine Image (AMI).

Decorator

AWS services can be mocked out using a simple decorator:

import boto3
  
from moto import mock_ec2
from main import create_instances
  
@mock_ec2
def test_create_instances_with_decorator(self):
  instance_count = 1
  image_id = 'ami-02ed82f3a38303e6f'
  create_instances('us-east-1', image_id, instance_count)
  client = boto3.client('ec2', region_name='us-east-1')
  instances = client.describe_instances()['Reservations'][0]['Instances']
  assert len(instances) == instance_count
  assert instances[0]['ImageId'] == image_id

test.py

Context manager

You can also use Moto mocks as a context manager:

import boto3

from main import create_instances
from moto import mock_ec2

def test_create_instances_with_context_manager(self):
  with mock_ec2():
    instance_count = 1
    image_id = 'ami-02ed82f3a38303e6f'
    create_instances('us-east-1', image_id, instance_count)
    client = boto3.client('ec2', region_name='us-east-1')
    instances = client.describe_instances()['Reservations'][0]['Instances']
    assert len(instances) == instance_count
    assert instances[0]['ImageId'] == image_id
@guivin

test.py

Raw use

And finally, you can go to the raw approach:

import boto3

from main import create_instances
from moto import mock_ec2

def test_create_instances_raw(self):
  mock = mock_ec2()
  mock.start()
  instance_count = 1
  image_id = 'ami-02ed82f3a38303e6f'
  create_instances('us-east-1', image_id, instance_count)
  client = boto3.client('ec2', region_name='us-east-1')
  instances = client.describe_instances()['Reservations'][0]['Instances']
  assert len(instances) == instance_count
  assert instances[0]['ImageId'] == image_id
  mock.stop()

test.py

Couple Moto With Pytest

Pytest is a framework to create small and scalable unit tests. It provides test fixtures that are functions running before the test functions. It is great to feed some data or set up a database connection, for example. Here, we’re going to define fixtures to configure S3 mocking and fake AWS credentials. By default, pytest loads the content of conftest.py:

import boto3
import pytest
import os

from moto import mock_s3

pytest.aws_region = 'us-east-1'


@pytest.fixture(scope='function')
def aws_credentials():
  os.environ['AWS_ACCESS_KEY_ID'] = 'testing'
  os.environ['AWS_SECRET_ACCESS_ID'] = 'testing'
  os.environ['AWS_SECURITY_TOKEN'] = 'testing'
  os.environ['AWS_SESSION_TOKEN'] = 'testing'


@pytest.fixture(scope='function')
def s3(aws_credentials):
  with mock_s3():
    yield boto3.client('s3', region_name=pytest.aws_region)

conftest.py

In test.py, we use a context manager to create S3 before running the tests present in a dedicated class. There is a test checking if the S3 bucket exists and another testing the placement of an object:

from contextlib import contextmanager


@contextmanager
def s3_bucket(s3, bucket_name):
  s3.create_bucket(Bucket=bucket_name)
  yield


class TestClassS3:
  def test_create_bucket(self, s3, bucket_name='testing'):
    with s3_bucket(s3, bucket_name):
      result = s3.list_buckets()
      assert len(result['Buckets']) == 1
      assert result['Buckets'][0]['Name'] == bucket_name

  def test_put_object(self, s3, bucket_name='testing', key='testing', body='testing'):
    with s3_bucket(s3, bucket_name):
      s3.put_object(Bucket=bucket_name, Key=key, Body=body)
      object_response = s3.get_object(Bucket=bucket_name, Key=key)
      assert object_response['Body'].read().decode() == body

test.py

Launch the unit tests with the pytest command:

$ pytest test.py

Conclusion

In this article, we’ve seen how to mock AWS responses using Moto. This allows for free and quick testing of your Python code. Coupled with pytest, it provides scalable and clean unit tests for AWS interactions.

Resources

pytest: helps you write better programs — pytest documentation

Logo

boto/boto3

AWS SDK for Python. Contribute to boto/boto3 development by creating an account on GitHub.

GitHubboto

spulec/moto

A library that allows you to easily mock out tests based on AWS infrastructure. - spulec/moto

GitHubspulec