Lately we’ve been working on a new Docker and Kubernetes instructor-led training class that we’ll be running onsite at several companies this year. The class uses Docker Desktop and the Kubernetes features it provides for several of the chapters. We needed to get the local cluster students will use to match as closely as possible to a cloud-based Kubernetes cluster that would be found on Azure, AWS, or GCP. The class covers using AKS as well, but most of the lab exercises rely on Kubernetes in Docker Desktop so running key features like the dashboard and Metrics API was important.
The majority of the Kubernetes functionality on Docker Desktop works great out of the box. You can run standard kubectl commands, work with various Service types including LoadBalancer (it supports localhost), run Deployments (against a single Node), and more, but getting kubectl top commands to work was challenging.
It turns out that Metrics Server isn’t installed by default with Docker Desktop. You do get it automatically if you install Kubernetes using kube-up.sh. To work around that, we installed Metrics Server by following the directions at https://kubernetes.io/docs/tasks/debug-application-cluster/core-metrics-pipeline and https://github.com/kubernetes-incubator/metrics-server#deployment, but running kubectl top commands resulted in “no metrics available” messages. Definitely frustrating.
By running kubectl logs [metrics-server-pod-name] -n kube-system we could see that the Pod/Container was there, but it looked like some unexpected issues were coming up.
After doing some research (translated: Google Fu), I came across a Github issue that seemed to solve the problem and enabled the kubectl top command to start reporting information about Nodes and Pods on Docker Desktop/Kubernetes. Here are the steps that fixed the issue.
Enabling Metrics Server in Docker Desktop
1. Clone or download the Metrics Server project.
2. Open the deploy/1.8+/metrics-server-deployment.yaml file in an editor.
3. Add the following command values into the containers property (it should be at the same level as the image property).
NOTE: DO NOT enable kubelet-insecure-tls on a cluster that will be accessed externally. This is only being done for a local Docker Desktop cluster.
command: - /metrics-server - --kubelet-insecure-tls
The full container definition ends up looking like the following:
containers: - name: metrics-server image: k8s.gcr.io/metrics-server-amd64:v0.3.1 imagePullPolicy: Always # Added the following lines command: - /metrics-server - --kubelet-insecure-tls
4. Run kubectl create -f deploy/1.8+ as shown on the Metrics Server repo to create the deployment, services, etc.
5. To see how things are going, first get the name of your Metrics Server Pod by running kubectl get pods -n kube-system.
6. Now run kubectl logs [metrics-server-pod-name] -n kube-system and the logs should show it starting up and the API being exposed successfully.
7. Give it a little time and you should now be able to run kubectl top commands!
There are almost always multiple ways to accomplish the same goal so if you know of an alternate technique for getting Metrics Server going on Docker Desktop Kubernetes please leave a comment!
I’m hoping that at some point this functionality will ship directly in Docker Desktop, but for now you have to install it to get it running.