Introduction
In my previous blog, I explored various options for running Kubernetes locally on a laptop, and we came to the conclusion that k3d is a solid choice for lightweight Kubernetes clusters. That test was done on Ubuntu, where everything ran smoothly. However, when I tried setting up the same environment on a Mac, I noticed a different behavior altogether. Today, I will walk you through this difference and show you how to solve the challenge of accessing Kubernetes containers on macOS when running k3d.
Behavior on Linux OS
On a Linux machine, things work like a charm. Docker runs directly on the host system, making networking smooth and efficient. This allows you to access your Kubernetes containers directly from the host. So, when you expose a pod via NodePort, you can access it from your laptop without any special tweaks.
Let’s see this in action:
➜ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 22.04.5 LTS
Release: 22.04
Codename: jammy
➜ kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE IP NODE
test-pod 1/1 Running 0 3d15h 10.42.1.46 k3d-trinity-cluster1-agent-1
➜ kubectl expose pod test-pod --port 80 --name test-svc --type NodePort
service/test-svc exposed
➜ kubectl get node -o wide |awk '{print $1,"\t",$6,"\t",$5}'
NAME INTERNAL-IP VERSION
k3d-trinity-cluster1-agent-0 172.28.0.2 v1.30.4+k3s1
k3d-trinity-cluster1-agent-1 172.28.0.4 v1.30.4+k3s1
k3d-trinity-cluster1-server-0 172.28.0.3 v1.30.4+k3s1
~ …
➜ curl http://172.28.0.4:31902
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
[...]As you can see, the Internal-IP of the nodes is accessible directly from the host machine, and we can communicate with the Kubernetes cluster seamlessly. Why? Because on Linux, Docker sets up a network bridge(docker0) that enables direct communication between your host machine and containers:
➜ ip l show type bridge docker0
4: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default
link/ether 02:42:06:59:6e:ef brd ff:ff:ff:ff:ff:ff
➜ ip r
default via 192.168.50.1 dev wlp61s0 proto dhcp metric 600
[...]
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1
[...]
➜ traceroute 172.28.0.4
traceroute to 172.28.0.4 (172.28.0.4), 30 hops max, 60 byte packets
1 172.28.0.4 (172.28.0.4) 0.219 ms 0.107 ms 0.097 msBehavior on macOS
When I tried the same setup on macOS, things didn’t go as smoothly. I couldn’t access the applications running inside the k3d Kubernetes cluster in the same way. Curious about the reason, I did some digging and discovered that Docker behaves differently on macOS.
Why is that?
Unlike Linux, Docker doesn’t run natively on macOS. Instead, Docker Desktop on macOS creates a tiny Linux VM and runs Docker inside it. This adds an additional networking layer, and as a result, you can’t directly connect to the container IPs from your Mac. The VM isolates the containers from the macOS host, which is why traditional access methods fail.
Here is a breakdown of how this looks on macOS when creating a k3d cluster:
~ on 🐳 v27.2.0 (desktop-linux) via 🐍 system took 3s ➜ k3d cluster create trinity-cluster
INFO[0000] Prep: Network
INFO[0000] Created network 'k3d-trinity-cluster'
INFO[0000] Created image volume k3d-trinity-cluster-images
INFO[0000] Starting new tools node...
INFO[0001] Creating node 'k3d-trinity-cluster-server-0'
INFO[0002] Starting node 'k3d-trinity-cluster-tools'
INFO[0002] Creating LoadBalancer 'k3d-trinity-cluster-serverlb'
INFO[0003] Using the k3d-tools node to gather environment information
INFO[0004] Starting new tools node...
INFO[0005] Starting node 'k3d-trinity-cluster-tools'
INFO[0007] Starting cluster 'trinity-cluster'
INFO[0007] Starting servers...
INFO[0008] Starting node 'k3d-trinity-cluster-server-0'
INFO[0025] All agents already running.
INFO[0025] Starting helpers...
INFO[0026] Starting node 'k3d-trinity-cluster-serverlb'
INFO[0035] Injecting records for hostAliases (incl. host.k3d.internal) and for 3 network members into CoreDNS configmap...
INFO[0037] Cluster 'trinity-cluster' created successfully!
INFO[0038] You can now use it like this:
kubectl cluster-info
~ on 🐳 v27.2.0 (desktop-linux) via 🐍 system took 39s ➜ kubectl get node -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
k3d-trinity-cluster-server-0 Ready control-plane,master 50s v1.30.4+k3s1 172.20.0.3 <none> K3s v1.30.4+k3s1 6.10.4-linuxkit containerd://1.7.20-k3s1
## Created nginx pod and exposed it on NodePort:
~ on 🐳 v27.2.0 (desktop-linux) via 🐍 system ➜ kubectl run --image nginx test-pod
pod/test-pod created
~ on 🐳 v27.2.0 (desktop-linux) via 🐍 system ➜ kubectl expose pod test-pod --name test-svc --type NodePort --port 80
service/test-svc exposed
~ on 🐳 v27.2.0 (desktop-linux) via 🐍 system ➜ kubectl get svc test-svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
test-svc NodePort 10.43.129.238 <none> 80:30693/TCP 10s
## Now, when I try to access an exposed service via NodePort, things break down:
~ on 🐳 v27.2.0 (desktop-linux) via 🐍 system ➜ curl http://172.20.0.3:30693
^C
~ on 🐳 v27.2.0 (desktop-linux) via 🐍 system took 39s ➜ ping 172.20.0.3
PING 172.20.0.3 (172.20.0.3): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
^C
--- 172.20.0.3 ping statistics ---
3 packets transmitted, 0 packets received, 100.0% packet lossNotice that I can’t ping the Kubernetes node IP from the macOS host. However, accessing the service from within the Kubernetes pod itself works fine:
~ on 🐳 v27.2.0 (desktop-linux) via 🐍 system ➜ kubectl exec -it test-pod -- sh
# curl http://172.20.0.3:30693
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
[...]So, it’s clear that the service is running perfectly within the cluster, but we are unable to access it directly from the macOS host.
Solution
After some exploration, I found the perfect solution: docker-mac-net-connect. This utility bridges the gap by allowing you to connect directly to Docker containers via IP on macOS. Once installed, it fixes the networking issue, allowing you to access Kubernetes pods like you would on a Linux machine. You can check the implementation details from the repo.
Here is how things look after installing docker-mac-net-connect:
~ on 🐳 v27.2.0 (desktop-linux) via 🐍 system ➜ kubectl get pod
NAME READY STATUS RESTARTS AGE
test-pod 1/1 Running 0 16s
~ on 🐳 v27.2.0 (desktop-linux) via 🐍 system took 2s ➜ kubectl get svc test-svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
test-svc NodePort 10.43.131.28 <none> 80:30266/TCP 26s
~ on 🐳 v27.2.0 (desktop-linux) via 🐍 system ➜ curl 172.20.0.3:30266
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
[...]Additionally, if you are running a LoadBalancer type service with MetalLB, you can even expose services with an external IP:
~/Desktop/myData on 🐳 v27.2.0 (desktop-linux) took 3s ➜ kubectl expose pod test-pod --name test-svc1 --type LoadBalancer --port 80
service/test-svc1 exposed
~/Desktop/myData on 🐳 v27.2.0 (desktop-linux) ➜ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 29m
test-svc NodePort 10.43.131.28 <none> 80:30266/TCP 8m40s
test-svc1 LoadBalancer 10.43.241.45 172.20.0.101 80:30494/TCP 4s
~/Desktop/myData on 🐳 v27.2.0 (desktop-linux) ➜ curl http://172.20.0.101
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
[...]Conclusion
Networking with Docker on macOS is a bit different due to the Linux VM layer, which complicates direct access to container IPs. Thankfully, tools like docker-mac-net-connect exist to solve these problems. With this fix, you can enjoy a smoother experience running Kubernetes on macOS using k3d, and your local dev environment should feel a lot more like it does on Linux.
If you are working with Kubernetes on macOS and hit similar issue, give this solution a try.
If you have any query related to this topic, please add a comment or ping me on LinkedIn. Stay curious and keep exploring!
