Leverage the power of Kubernetes Operators to deploy your own RabbitMQ Cluster
RabbitMQ is the most widely deployed open-source message broker. Before talking about RabbitMQ, let's define what a message broker is. It allows applications and services to communicate with each other and exchange data. The diagram below presents the use case of an event bus using a message broker:

Message brokers are very useful to decouple communications between your components. It gives you the opportunity to create fully distributed and modern architectures. In the message broker family, you can find other popular names like Apache Kafka, Apache ActiveMQ.
RabbitMQ is lightweight and easy to deploy on-premise or in the cloud. It offers interesting features:
- It supports multiple messaging protocols like AMQP, STOMP, MQTT, etc...
- It can be deployed in distributed and federated configurations to meet high-scale and high availability requirements.
- It provides a UI for management and an HTTP API for monitoring.
- It can be extended with many available plugins.
- There are implemented clients in many programming languages: Python, Javascript, Java, Go, Ruby, etc.
In this tutorial, we will focus on how to deploy a high-available RabbitMQ cluster in a Kubernetes cluster. The choice will be to use Kubernetes Operators to achieve this goal efficiently. We will see the advantages of using them and how to deploy RabbitMQ resources step by step.
Why Use The Kubernetes Operators?
"Operators are software extensions to Kubernetes that make use of custom resources to manage applications and their components." https://kubernetes.io/docs/concepts/extend-kubernetes/operator/
Operators in Kubernetes allow you to extend the cluster's behavior without modifying the code of Kubernetes itself. In our case, the behavior of RabbitMQ will be delegated to the operators. This will save us a lot of time on our side.
The RabbitMQ team develops and maintains two Kubernetes operators :
- RabbitMQ Cluster Kubernetes Operator to automate provisioning, management, and operations of RabbitMQ clusters running on Kubernetes.
- RabbitMQ Messaging Topology Operator to manage the topology of the clusters (Permissions, Users, etc...)
The Operators are installed through CRDs (Custom Resource Definition) in the Kubernetes cluster. Once installed, new resources are known in the cluster such as classical kinds (Pod, Deployment, etc...). You just have to create the YAML manifests to invoke them.
Deploying The RabbitMQ Cluster Kubernetes Operator
The CRD is available in the repository releases. Install it with the kubectl
command:
$ kubectl apply -f "https://github.com/rabbitmq/cluster-operator/releases/latest/download/cluster-operator.yml"
namespace/rabbitmq-system created
customresourcedefinition.apiextensions.k8s.io/rabbitmqclusters.rabbitmq.com created
serviceaccount/rabbitmq-cluster-operator created
role.rbac.authorization.k8s.io/rabbitmq-cluster-leader-election-role created
clusterrole.rbac.authorization.k8s.io/rabbitmq-cluster-operator-role created
rolebinding.rbac.authorization.k8s.io/rabbitmq-cluster-leader-election-rolebinding created
clusterrolebinding.rbac.authorization.k8s.io/rabbitmq-cluster-operator-rolebinding created
deployment.apps/rabbitmq-cluster-operator created
The new resources are inside the rabbitmq-system
namespace:
$ kubectl get all -n rabbitmq-system
NAME READY STATUS RESTARTS AGE
pod/rabbitmq-cluster-operator-7cbf865f89-9pq6n 1/1 Running 4 2d21h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/rabbitmq-cluster-operator 1/1 1 1 2d21h
NAME DESIRED CURRENT READY AGE
replicaset.apps/rabbitmq-cluster-operator-7cbf865f89 1 1 1 2d21h
Ensure the resources are all ready to go next.
Create The RabbitMQ Cluster
The RabbitMQ Cluster Operator provides a RabbitmqCluster
kind. You can now create as many RabbitMQ clusters as you want in your Kubernetes cluster. You have just to define them in a YAML manifest like this:
apiVersion: rabbitmq.com/v1beta1
kind: RabbitmqCluster
metadata:
labels:
app: rabbitmq
name: rabbitmq
spec:
replicas: 3
image: rabbitmq:3.9.11
service:
type: ClusterIP
persistence:
storageClassName: hostpath
storage: 10
resources:
requests:
cpu: 256m
memory: 1Gi
limits:
cpu: 256m
memory: 1Gi
rabbitmq:
additionalPlugins:
- rabbitmq_management
- rabbitmq_peer_discovery_k8s
additionalConfig: |
cluster_formation.peer_discovery_backend = rabbit_peer_discovery_k8s
cluster_formation.k8s.host = kubernetes.default.svc.cluster.local
cluster_formation.k8s.address_type = hostname
vm_memory_high_watermark_paging_ratio = 0.85
cluster_formation.node_cleanup.interval = 10
cluster_partition_handling = autoheal
queue_master_locator = min-masters
loopback_users.guest = false
default_user = guest
default_pass = guest
advancedConfig: ""
Let's recap the content of the manifest above:
- We create a resource of kind Β
Rabbitmqcluster
. It is defined by the CRD we installed in the previous part. - The cluster has 3 replicas for high availability and fault tolerance.
- Each pod has clearly defined resources.
- Added plugins:
rabbitmq_management
for management UI andrabbitmq_peer_discovery_k8s
for cluster formation. You can find other plugins here: https://www.rabbitmq.com/plugins.html. - The cluster configuration is done in the
additionalConfig
field: https://www.rabbitmq.com/configure.html#config-file. - The type of service used is
ClusterIP
. You can configure aLoadBalancer
type if your application is outside the cluster. - The persistence is configured. The given example runs on a local Kubernetes cluster so the storage class is
hostpath
. You can look at the classes available in Kubernetes that will fit your needs: https://kubernetes.io/docs/concepts/storage/storage-classes/
$ kubectl create -f rabbitmq.yaml
$ kubectl get pods -l app.kubernetes.io/name=rabbitmq
NAME READY STATUS RESTARTS AGE
rabbitmq-server-0 1/1 Running 0 4m41s
rabbitmq-server-1 1/1 Running 0 4m41s
rabbitmq-server-2 1/1 Running 0 4m41s
Deploying the RabbitMQ Topology Operator
Let's place to RabbitMQ Topology Operator now. It allows you to manage RabbitMQ specific resources like users and permissions for example: https://www.rabbitmq.com/kubernetes/operator/using-topology-operator.html.
This Operator needs cert-manager to work. Deploy it and ensure all pods are healthy:
$ kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.3.1/cert-manager.yaml
$ kubectl get pods -n cert-manager
NAME READY STATUS RESTARTS AGE
cert-manager-7dd5854bb4-vt5jn 1/1 Running 0 47s
cert-manager-cainjector-64c949654c-j2npq 1/1 Running 0 47s
cert-manager-webhook-6b57b9b886-f6gwt 1/1 Running 0 47s
Install the CRD:
kubectl apply -f https://github.com/rabbitmq/messaging-topology-operator/releases/latest/download/messaging-topology-operator-with-certmanager.yaml
Here is an example of a manifest to configure the permissions for the guest user to do tests with the authentication:
apiVersion: rabbitmq.com/v1beta1
kind: Permission
metadata:
name: rabbitmq-guest-permission
spec:
vhost: /
user: guest
permissions:
write: ".*"
configure: ".*"
read: ".*"
rabbitmqClusterReference:
name: rabbitmq
Install The RabbitMQ Cluster Operator Kubectl Plugin
The RabbitMQ team also provides a kubectl plugin to use the RabbitMQ Cluster Operator. This plugin will automate many interactions with the Kubernetes API and the RabbitMQ cluster operator.
The plugin needs krew
plugin manager installed (Installation here is for MacOSX, check the documentation and adapt for your OS)
(
set -x; cd "$(mktemp -d)" &&
OS="$(uname | tr '[:upper:]' '[:lower:]')" &&
ARCH="$(uname -m | sed -e 's/x86_64/amd64/' -e 's/\(arm\)\(64\)\?.*/\1\2/' -e 's/aarch64$/arm64/')" &&
KREW="krew-${OS}_${ARCH}" &&
curl -fsSLO "https://github.com/kubernetes-sigs/krew/releases/latest/download/${KREW}.tar.gz" &&
tar zxvf "${KREW}.tar.gz" &&
./"${KREW}" install krew
)
Update your PATH
environment variable with the location of krew
:
export PATH="${KREW_ROOT:-$HOME/.krew}/bin:$PATH"
Check the installation is ok:
$ kubectl krew
Install the RabbitMQ plugin:
$ kubectl krew install rabbitmq
Check the RabbitMQ plugin:
$ kubectl rabbitmq help
You can retrieve your cluster in the list:
$ kubectl rabbitmq list
NAME ALLREPLICASREADY RECONCILESUCCESS AGE
rabbitmq True True 4d20h
You can also open directly the management UI:
$ kubectl rabbitmq manage rabbitmq


Conclusion
Through this tutorial, we saw the advantage of using Operators to deploy Kubernetes. They allow transparency and provisioning of RabbitMQ clusters easily. It automates a large part of the management and allows to define the RabbitMQ topology through YAML manifests. Finally, we have seen how to set up the kubectl plugin to interact with the operator.