Replace Docker Desktop on Mac with a concrete setup alternative (Kubernetes included)
Docker announced on August 31 2021 that they will update their subscription. There are some changes on the Docker Desktop side:
"Docker Desktop remains free for small businesses (fewer than 250 employees AND less than $10 million in annual revenue), personal use, education, and non-commercial open source projects." from the Docker blog
This is a good opportunity to think about an alternative to Docker Desktop.
In my case of daily use, I've long been looking for an alternative to Docker Desktop. When I am developing Ansible roles, I do tests with the Molecule framework and Docker. But the problem here is that there is no support for systemd because it doesn't exist in MacOSX.
To have a Linux host I was running a vagrant virtual machine with a docker inside. Rather heavy as a working environment! But recently I found a turnkey and easy-to-set-up solution to have Docker and Kubernetes on Mac without going through Docker Desktop. We will see a toolset to achieve that.
Lima-VM
Lima is a project that allows you to launch Linux virtual machines on MacOS with automatic file sharing, port forwarding, and containerd. To install containerd on Mac, you have to compile sources and this can be tedious and long. This is the main motivation for the project:
"The goal of Lima is to promote containerd including nerdctl (contaiNERD ctl) to Mac users, but Lima can be used for non-container applications as well. " https://github.com/lima-vm/lima#motivation
Lima can be compared to a kind of Windows Subsystem for Linux (WSL) but for Mac and a containerd for Mac. It wraps QEMU hypervisor with containerd runtime and nerdctl a replacement for the docker command. It is super simple to install with Homebrew:
brew install limaThe VM can be launched like this:
$ limactl start
? Creating an instance "default" Proceed with the default configuration
INFO[0007] Downloading "https://github.com/containerd/nerdctl/releases/download/v0.11.2/nerdctl-full-0.11.2-linux-amd64.tar.gz" (sha256:27dbb238f9eb248ca68f11b412670db51db84905e3583834400305b2149915f2)
174.89 MiB / 174.89 MiB [----------------------------------] 100.00% 37.45 MiB/s
INFO[0013] Downloaded "nerdctl-full-0.11.2-linux-amd64.tar.gz"
INFO[0015] Attempting to download the image from "~/Downloads/hirsute-server-cloudimg-amd64.img"
INFO[0015] Attempting to download the image from "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-amd64.img"
558.19 MiB / 558.19 MiB [----------------------------------] 100.00% 22.66 MiB/s
INFO[0041] Downloaded image from "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-amd64.img"
INFO[0043] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/gvincent/.lima/default/serial.log")
INFO[0044] SSH Local Port: 60022
INFO[0044] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0054] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0081] [hostagent] Waiting for the essential requirement 1 of 4: "ssh"
INFO[0088] [hostagent] The essential requirement 1 of 4 is satisfied
INFO[0088] [hostagent] Waiting for the essential requirement 2 of 4: "sshfs binary to be installed"
INFO[0129] [hostagent] Waiting for the essential requirement 2 of 4: "sshfs binary to be installed"
INFO[0129] [hostagent] The essential requirement 2 of 4 is satisfied
INFO[0129] [hostagent] Waiting for the essential requirement 3 of 4: "/etc/fuse.conf to contain \"user_allow_other\""
INFO[0150] [hostagent] The essential requirement 3 of 4 is satisfied
INFO[0150] [hostagent] Waiting for the essential requirement 4 of 4: "the guest agent to be running"
INFO[0150] [hostagent] The essential requirement 4 of 4 is satisfied
INFO[0150] [hostagent] Mounting "/Users/gvincent"
INFO[0151] [hostagent] Mounting "/tmp/lima"
INFO[0151] [hostagent] Waiting for the optional requirement 1 of 2: "systemd must be available"
INFO[0151] [hostagent] Forwarding "/run/user/501/lima-guestagent.sock" (guest) to "/Users/gvincent/.lima/default/ga.sock" (host)
INFO[0151] [hostagent] The optional requirement 1 of 2 is satisfied
INFO[0151] [hostagent] Waiting for the optional requirement 2 of 2: "containerd binaries to be installed"
INFO[0151] [hostagent] Not forwarding TCP 127.0.0.53:53
INFO[0151] [hostagent] Not forwarding TCP 0.0.0.0:22
INFO[0151] [hostagent] Not forwarding TCP [::]:22
INFO[0154] [hostagent] The optional requirement 2 of 2 is satisfied
INFO[0154] READY. Run `lima` to open the shell.The VM is up and running. The host filesystem is mounted into the VM through sshfs and it has the same path as the host. You can execute commands by prefixing them with lima :
$ lima uname -a
Linux lima-default 5.11.0-37-generic #41-Ubuntu SMP Mon Sep 20 16:39:20 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux You can also use nerdctl with the containerd present in the Lima-VM to control containers:
$ lima nerdctl run -it --rm alpine sh -c "cat /etc/os-release"
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.14.2
PRETTY_NAME="Alpine Linux v3.14"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://bugs.alpinelinux.org/"Install the Docker Daemon in Lima-VM
We will still install Docker in the VM to prepare because we need it for the Kubernetes part that comes after :
$ lima sudo apt install docker-io
$ lima sudo docker version
Client:
Version: 20.10.7
API version: 1.41
Go version: go1.13.8
Git commit: 20.10.7-0ubuntu1~21.04.1
Built: Fri Aug 6 14:14:21 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server:
Engine:
Version: 20.10.7
API version: 1.41 (minimum version 1.12)
Go version: go1.13.8
Git commit: 20.10.7-0ubuntu1~21.04.1
Built: Wed Aug 4 12:24:19 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.5.5
GitCommit: 72cec4be58a9eb6b2910f5d10f1c01ca47d231c0
runc:
Version: 1.0.2
GitCommit: v1.0.2-0-g52b36a2d
docker-init:
Version: 0.19.0
GitCommit:Use Kubernetes in Docker (Kind)
kind is a tool for running local Kubernetes clusters using Docker container nodes. In a previous article, I presented to you how to integrate it into your CI pipelines. Here we are going to use it for local development purposes.
Open a session inside the VM and go to $HOME:
$ lima
$ cdInstall go 1.11+ :
$ sudo apt install golangInstall kind :
$ GO111MODULE="on" go get sigs.k8s.io/[email protected]
$ sudo cp $HOME/go/bin/kind /usr/local/bin/kind
Create a brand new Kubernetes cluster:
$ sudo kind create cluster
Creating cluster "kind" ...
✓ Ensuring node image (kindest/node:v1.21.1) 🖼
✓ Preparing nodes 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
Set kubectl context to "kind-kind"
You can now use your cluster with:
kubectl cluster-info --context kind-kind
Thanks for using kind! 😊With the version argument you can use a specific Kubernetes version: kind create cluster --image kindest/node:v1.21.1Download kubectl with a version compliant with the cluster version:
$ sudo curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.21.1/bin/linux/amd64/kubectl
Make the kubectl binary executable:
$ sudo chmod +x ./kubectlMove the binary to your PATH:
$ sudo mv ./kubectl /usr/local/bin/kubectlTest to ensure the installed version is up to date:
$ sudo kubectl version --clientCheck the cluster is ready:
$ sudo kubectl cluster-info
Kubernetes control plane is running at https://127.0.0.1:39635
CoreDNS is running at https://127.0.0.1:39635/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.Launch a busybox pod interactive mode:
$ sudo kubectl run -i --rm --tty debug --image=busybox --restart=Never -- sh
If you don't see a command prompt, try pressing enter.
/ #Conclusion
We have seen how to replace Docker Desktop with lima to create a linux VM easily that natively integrates containerd. We installed the docker daemon to be able to use Kubernetes with kind.
The only drawback for the moment is that I did not find how to launch docker in non-root mode. Adding the user to the docker group does not give the rights to the docker socket
With this setup, you can also use Kubernetes and create clusters of different versions. This is really ideal for a development workstation. It's what was missing I think and it opens new possibilities for a Mac environment.
Resources
Scott Johnston
lima-vmkubernetes-sigs