Konubinix' opinionated web of thoughts



Tool to ease playing with k3s inside docker, done by rancher.

It uses flannel to deal with networking. Flannel does not handle networkpolicy.

Flannel doesn’t control how containers are networked to the host, only how the traffic is transported between hosts and doesn’t implement network policy controller. For network policy, other projects such as Calico can be used.


It seems like there is a minimal implementation of networkpolicy inside k3s, but it does not appear to deal with even basic use cases.

running Cilium in K3D

important k3s arguments here are --disable-network-policy and --flannel-backend=none.


cannot use a cilium init-container in the k3d/k3s combo, because it tries to run a script in the context of the above containers using the /bin/bash interpreter (but the k3s images are based on busybox, and only have a /bin/sh interpreter available).


docker exec -it k3d-foo-agent-0 mount bpffs /sys/fs/bpf -t bpf docker exec -it k3d-foo-agent-0 mount –make-shared /sys/fs/bpf

docker exec -it k3d-foo-server-0 mount bpffs /sys/fs/bpf -t bpf docker exec -it k3d-foo-server-0 mount –make-shared /sys/fs/bpf


helm repo add cilium https://helm.cilium.io/

helm install cilium cilium/cilium –version 1.9.1 \ –namespace kube-system \ –set kubeProxyReplacement=partial \ –set hostServices.enabled=false \ –set externalIPs.enabled=true \ –set nodePort.enabled=true \ –set hostPort.enabled=true \ –set bpf.masquerade=false \ –set image.pullPolicy=IfNotPresent \ –set ipam.mode=kubernetes

helm upgrade cilium cilium/cilium –version 1.9.1 \ –namespace kube-system \ –reuse-values \ –set hubble.listenAddress=":4244" \ –set hubble.relay.enabled=true \ –set hubble.ui.enabled=true


k3d with cgroup v2

I think I’m seeing the same or similar issue. When I rollback to rancher/k3s:v1.19.7-k3s1, the cluster starts fine.


k3d cluster create –verbose –trace –image rancher/k3s:v1.19.8-k3s1 wsop Doesn’t work on Fedora 33 as there is different error

> docker logs –follow k3d-wsop-server-0 2>&1 … time=“2021-03-03T11:52:51.432307543Z” level=fatal


Hi @fr33ky , thanks for your input. I guess when you run docker info, you see that Cgroup Version is 2? I’m on the same docker version on Ubuntu, but using cgroup v1… no problems here


Using Debian Sid, in the meantime, I personally switched back to cgroup v1. I added systemd.unified_cgroup_hierarchy=0 to my GRUB_CMDLINE_LINUX_DEFAULT (/etc/default/grub) and then ran update-grub


k3d: unable to start built container

Basically it seems like k3d is not a good environment for kubectl-buildkit, and the only option for it is if you are using a registry, as described by @pdevine, although that eliminates one of the use cases (not transferring bytes up and down from the registry, and needing a registry)


install a custom cni with k3d

Custom CNI

Run K3s with –flannel-backend=none and install your CNI of choice


install calico alongside flannel

Installing Calico for policy and flannel (aka Canal) for networking


Notes linking here