Replace Docker Desktop on Mac with a concrete setup alternative (Kubernetes included)
Docker announced on August 31 2021 that they will update their subscription. There are some changes on the Docker Desktop side:
"Docker Desktop remains free for small businesses (fewer than 250 employees AND less than $10 million in annual revenue), personal use, education, and non-commercial open source projects." from the Docker blog
This is a good opportunity to think about an alternative to Docker Desktop.
In my case of daily use, I've long been looking for an alternative to Docker Desktop. When I am developing Ansible roles, I do tests with the Molecule framework and Docker. But the problem here is that there is no support for systemd because it doesn't exist in MacOSX.
To have a Linux host I was running a vagrant virtual machine with a docker inside. Rather heavy as a working environment! But recently I found a turnkey and easy-to-set-up solution to have Docker and Kubernetes on Mac without going through Docker Desktop. We will see a toolset to achieve that.
Lima is a project that allows you to launch Linux virtual machines on MacOS with automatic file sharing, port forwarding, and containerd. To install containerd on Mac, you have to compile sources and this can be tedious and long. This is the main motivation for the project:
"The goal of Lima is to promote containerd including nerdctl (contaiNERD ctl) to Mac users, but Lima can be used for non-container applications as well. " https://github.com/lima-vm/lima#motivation
Lima can be compared to a kind of Windows Subsystem for Linux (WSL) but for Mac and a containerd for Mac. It wraps QEMU hypervisor with containerd runtime and nerdctl a replacement for the docker command. It is super simple to install with Homebrew:
brew install lima
The VM can be launched like this:
$ limactl start ? Creating an instance "default" Proceed with the default configuration INFO Downloading "https://github.com/containerd/nerdctl/releases/download/v0.11.2/nerdctl-full-0.11.2-linux-amd64.tar.gz" (sha256:27dbb238f9eb248ca68f11b412670db51db84905e3583834400305b2149915f2) 174.89 MiB / 174.89 MiB [----------------------------------] 100.00% 37.45 MiB/s INFO Downloaded "nerdctl-full-0.11.2-linux-amd64.tar.gz" INFO Attempting to download the image from "~/Downloads/hirsute-server-cloudimg-amd64.img" INFO Attempting to download the image from "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-amd64.img" 558.19 MiB / 558.19 MiB [----------------------------------] 100.00% 22.66 MiB/s INFO Downloaded image from "https://cloud-images.ubuntu.com/hirsute/current/hirsute-server-cloudimg-amd64.img" INFO [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/gvincent/.lima/default/serial.log") INFO SSH Local Port: 60022 INFO [hostagent] Waiting for the essential requirement 1 of 4: "ssh" INFO [hostagent] Waiting for the essential requirement 1 of 4: "ssh" INFO [hostagent] Waiting for the essential requirement 1 of 4: "ssh" INFO [hostagent] The essential requirement 1 of 4 is satisfied INFO [hostagent] Waiting for the essential requirement 2 of 4: "sshfs binary to be installed" INFO [hostagent] Waiting for the essential requirement 2 of 4: "sshfs binary to be installed" INFO [hostagent] The essential requirement 2 of 4 is satisfied INFO [hostagent] Waiting for the essential requirement 3 of 4: "/etc/fuse.conf to contain \"user_allow_other\"" INFO [hostagent] The essential requirement 3 of 4 is satisfied INFO [hostagent] Waiting for the essential requirement 4 of 4: "the guest agent to be running" INFO [hostagent] The essential requirement 4 of 4 is satisfied INFO [hostagent] Mounting "/Users/gvincent" INFO [hostagent] Mounting "/tmp/lima" INFO [hostagent] Waiting for the optional requirement 1 of 2: "systemd must be available" INFO [hostagent] Forwarding "/run/user/501/lima-guestagent.sock" (guest) to "/Users/gvincent/.lima/default/ga.sock" (host) INFO [hostagent] The optional requirement 1 of 2 is satisfied INFO [hostagent] Waiting for the optional requirement 2 of 2: "containerd binaries to be installed" INFO [hostagent] Not forwarding TCP 127.0.0.53:53 INFO [hostagent] Not forwarding TCP 0.0.0.0:22 INFO [hostagent] Not forwarding TCP [::]:22 INFO [hostagent] The optional requirement 2 of 2 is satisfied INFO READY. Run `lima` to open the shell.
The VM is up and running. The host filesystem is mounted into the VM through sshfs and it has the same path as the host. You can execute commands by prefixing them with lima :
$ lima uname -a Linux lima-default 5.11.0-37-generic #41-Ubuntu SMP Mon Sep 20 16:39:20 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
You can also use
nerdctl with the containerd present in the Lima-VM to control containers:
$ lima nerdctl run -it --rm alpine sh -c "cat /etc/os-release" NAME="Alpine Linux" ID=alpine VERSION_ID=3.14.2 PRETTY_NAME="Alpine Linux v3.14" HOME_URL="https://alpinelinux.org/" BUG_REPORT_URL="https://bugs.alpinelinux.org/"
Install the Docker Daemon in Lima-VM
We will still install Docker in the VM to prepare because we need it for the Kubernetes part that comes after :
$ lima sudo apt install docker-io $ lima sudo docker version Client: Version: 20.10.7 API version: 1.41 Go version: go1.13.8 Git commit: 20.10.7-0ubuntu1~21.04.1 Built: Fri Aug 6 14:14:21 2021 OS/Arch: linux/amd64 Context: default Experimental: true Server: Engine: Version: 20.10.7 API version: 1.41 (minimum version 1.12) Go version: go1.13.8 Git commit: 20.10.7-0ubuntu1~21.04.1 Built: Wed Aug 4 12:24:19 2021 OS/Arch: linux/amd64 Experimental: false containerd: Version: v1.5.5 GitCommit: 72cec4be58a9eb6b2910f5d10f1c01ca47d231c0 runc: Version: 1.0.2 GitCommit: v1.0.2-0-g52b36a2d docker-init: Version: 0.19.0 GitCommit:
Use Kubernetes in Docker (Kind)
kind is a tool for running local Kubernetes clusters using Docker container nodes. In a previous article, I presented to you how to integrate it into your CI pipelines. Here we are going to use it for local development purposes.
Open a session inside the VM and go to
$ lima $ cd
Install go 1.11+ :
$ sudo apt install golang
$ GO111MODULE="on" go get sigs.k8s.io/[email protected] $ sudo cp $HOME/go/bin/kind /usr/local/bin/kind
Create a brand new Kubernetes cluster:
$ sudo kind create cluster Creating cluster "kind" ... ✓ Ensuring node image (kindest/node:v1.21.1) 🖼 ✓ Preparing nodes 📦 ✓ Writing configuration 📜 ✓ Starting control-plane 🕹️ ✓ Installing CNI 🔌 ✓ Installing StorageClass 💾 Set kubectl context to "kind-kind" You can now use your cluster with: kubectl cluster-info --context kind-kind Thanks for using kind! 😊
With the version argument you can use a specific Kubernetes version:
kind create cluster --image kindest/node:v1.21.1
kubectl with a version compliant with the cluster version:
$ sudo curl -LO https://storage.googleapis.com/kubernetes-release/release/v1.21.1/bin/linux/amd64/kubectl
kubectl binary executable:
$ sudo chmod +x ./kubectl
Move the binary to your
$ sudo mv ./kubectl /usr/local/bin/kubectl
Test to ensure the installed version is up to date:
$ sudo kubectl version --client
Check the cluster is ready:
$ sudo kubectl cluster-info Kubernetes control plane is running at https://127.0.0.1:39635 CoreDNS is running at https://127.0.0.1:39635/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
Launch a busybox pod interactive mode:
$ sudo kubectl run -i --rm --tty debug --image=busybox --restart=Never -- sh If you don't see a command prompt, try pressing enter. / #
We have seen how to replace Docker Desktop with lima to create a linux VM easily that natively integrates containerd. We installed the docker daemon to be able to use Kubernetes with kind.
The only drawback for the moment is that I did not find how to launch docker in non-root mode. Adding the user to the docker group does not give the rights to the docker socket
With this setup, you can also use Kubernetes and create clusters of different versions. This is really ideal for a development workstation. It's what was missing I think and it opens new possibilities for a Mac environment.