Some Thoughts on k8s on WSL2

It’s getting to the point now where I rarely touch a Powershell prompt. My WSL2 system has taken over the computer, and it thinks it runs Linux now. The last thing for me to get running on my local “Linux” dev environment is Kubernetes. I use the docker-desktop Kubernetes cluster, since I am already using Docker Desktop and it’s as simple as checking a box. Minikube comes recommended as well as k3s.

I initially installed the snap for kubectl as per the suggestion from the CLI when I typed the non-existent kubectl command. I’d suggest installing it from the package repository instead. In any case, obtaining kubectl is not that hard; just don’t use the snap.

The real difficulty with running a local k8s environment is that your localhost interface fills up awfully fast with ports. It would be much easier if you could assign your external IPs to an aliased interface and then you could not worry about port collisions as much. Enter MetalLB, a virtual load balancer for your k8s cluster. Installation instructions are at https://metallb.org/. MetalLB will create the necessary virtual interfaces for you, and assign all of your LoadBalancer ports to one or more address pools. This is great – all of your load balancers in one place.

The only problem is that Windows can’t reach them by default. At least our MetalLB configuration is based on static addresses, because our WSL address will move every time WSL2 starts up. Not only that, but we’ll get an address in a different network! This makes setting up static routes a little difficult, but thankfully not impossible:

$wsl_gw=$($(netsh interface ipv4 show addresses "vEthernet (WSL)" | findstr "IP") -replace "^.*?(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?).*$","`$1.`$2.`$3.`$4")

try {
    route delete 10.100.0.0/24 > Out-Null 2>&1
} catch {

}

try {
    route add 10.100.0.0/24 $wsl_gw > Out-Null 2>&1
} catch {

}

The magic comes from the first line, a one-liner for grabbing the IP address of the WSL gateway interface. The regex is excruciatingly correct, but it works every time. It’s probably worth unpacking the declaration a bit:

Show the address information for the interface named “vEthernet (WSL)”. This is the default name of the virtual network installed to host WSL. The output looks like:

Configuration for interface "vEthernet (WSL)"
    DHCP enabled:                         No
    IP Address:                           172.17.192.1
    Subnet Prefix:                        172.17.192.0/20 (mask 255.255.240.0)
    InterfaceMetric:                      5000

Now that we have the output, it seems that the address we are interested in is all by its lonesome on the line labelled “IP Address”. This sounds like a job for a regular expression. So, we match the output against an entire line containing a single IP address and replace the line with the IP address. We use findstr to narrow the output to a single line in the first place.

And finally, we fire and forget a couple of network commands that should (occassionally) give us a route from our Windows machine to our MetalLB address pool.

There’s a lot that can go wrong here. Most notably, we cannot run anything related to WSL without the user logging in. So, we’re stuck with startup scripts in the Windows Startup folder to get the job done, and the one above goes in there.

Second, we need to trigger the Linux-side scripting on startup too. One clever thing you can do is to install systemd-genie and then start terminal upon login. Set Ubuntu to your default terminal and logging in will cause systemd to kick in and start those services. With any luck, both the route and the additional interface will be allocated without issue. But if you expect it to work without occasional (frequent) intervention, you’re crazy. Microsoft, if you’re listening, please have your systemd guy give us a real systemd! 🙂

But, as expected, other than the fiddling with IPs and ports, k8s runs very well on “WSL2” (remember – it’s running in a different Linux container on your Windows host). The Docker Desktop implementation of Kubernetes is fine.

One last thought. Deployment to Kubernetes is usually achieved using a CI/CD pipeline. I’d suggest you do the same for your local k8s cluster as well. Install an Azure DevOps agent in WSL2 and connect it to your DevOps project. Now you can manage your k8s builds from DevOps instead of tinkering from the command line. This turns out to be excellent for other reasons – it lets you break the build dozens of times in a pipeline whose stats don’t matter. Then you can simply clone the pipeline and update your variable groups for test/prod.

As usual, I’m late to the party on Kubernetes. But I believe it’s a key architectural component that allows us to avoid vendor lock-in. Building cloud-native apps for Kubernetes increases agility and reduces lock-in because you can easily deploy your application to other clouds, giving you the ability to rapidly move from cloud to cloud with minimal disruption. This leaves you in a much better negotiating position with your current cloud vendor.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: