Konubinix' opinionated web of thoughts

Kubernetes

Fleeting

run locally (for development purpose)

k0s

k3d

Tool to ease playing with k3s inside docker, done by rancher.

It uses flannel to deal with networking. Flannel does not handle networkpolicy.

Flannel doesn’t control how containers are networked to the host, only how the traffic is transported between hosts and doesn’t implement network policy controller. For network policy, other projects such as Calico can be used.

— https://banzaicloud.com/docs/pipeline/security/network-policy/network-plugins/

It seems like there is a minimal implementation of networkpolicy inside k3s, but it does not appear to deal with even basic use cases.

running Cilium in K3D

cannot use a cilium init-container in the k3d/k3s combo, because it tries to run a script in the context of the above containers using the /bin/bash interpreter (but the k3s images are based on busybox, and only have a /bin/sh interpreter available).

— https://sandstorm.de/de/blog/post/running-cilium-in-k3s-and-k3d-lightweight-kubernetes-on-mac-os-for-development.html

docker exec -it k3d-foo-agent-0 mount bpffs /sys/fs/bpf -t bpf docker exec -it k3d-foo-agent-0 mount –make-shared /sys/fs/bpf

docker exec -it k3d-foo-server-0 mount bpffs /sys/fs/bpf -t bpf docker exec -it k3d-foo-server-0 mount –make-shared /sys/fs/bpf

— https://sandstorm.de/de/blog/post/running-cilium-in-k3s-and-k3d-lightweight-kubernetes-on-mac-os-for-development.html

helm repo add cilium https://helm.cilium.io/

helm install cilium cilium/cilium –version 1.9.1 \ –namespace kube-system \ –set kubeProxyReplacement=partial \ –set hostServices.enabled=false \ –set externalIPs.enabled=true \ –set nodePort.enabled=true \ –set hostPort.enabled=true \ –set bpf.masquerade=false \ –set image.pullPolicy=IfNotPresent \ –set ipam.mode=kubernetes

helm upgrade cilium cilium/cilium –version 1.9.1 \ –namespace kube-system \ –reuse-values \ –set hubble.listenAddress=":4244" \ –set hubble.relay.enabled=true \ –set hubble.ui.enabled=true

— https://sandstorm.de/de/blog/post/running-cilium-in-k3s-and-k3d-lightweight-kubernetes-on-mac-os-for-development.html

k3d with cgroup v2

k3d: unable to start built container

install a custom cni with k3d

install calico alongside flannel

Installing Calico for policy and flannel (aka Canal) for networking

— https://projectcalico.docs.tigera.io/getting-started/kubernetes/flannel/flannel

kind

kind, a local k8s

known issues

too many open files

  • External reference: https://kind.sigs.k8s.io/docs/user/known-issues/#pod-errors-due-to-too-many-open-files Pod errors due to “too many open files”

    This may be caused by running out of inotify resources. Resource limits are defined by fs.inotify.max_user_watches and fs.inotify.max_user_instances system variables. For example, in Ubuntu these default to 8192 and 128 respectively, which is not enough to create a cluster with many nodes.

    To increase these limits temporarily run the following commands on the host:

$ sudo sysctl fs.inotify.max_user_watches=524288 $ sudo sysctl fs.inotify.max_user_instances=512

To make the changes persistent, edit the file /etc/sysctl.conf and add these lines:

fs.inotify.max_user_watches = 524288 fs.inotify.max_user_instances = 512

— https://kind.sigs.k8s.io/docs/user/known-issues/#pod-errors-due-to-too-many-open-files

ingress

Point d’entrée http(s) de kubernetes

ingress class

default

You can mark a particular IngressClass as default for your cluster. Setting the ingressclass.kubernetes.io/is-default-class annotation to true on an IngressClass resource will ensure that new Ingresses without an ingressClassName field specified will be assigned this default IngressClass.

— https://kubernetes.io/docs/concepts/services-networking/ingress/

some ingress controllers, that work without the definition of a default IngressClass. For example, the Ingress-NGINX controller can be configured with a flag –watch-ingress-without-class. It is recommended though, to specify the default IngressClass:

— https://kubernetes.io/docs/concepts/services-networking/ingress/

cronjob

trigger manually

kubectl create job –from=cronjob/pgdump pgdump-manual-001

— https://www.craftypenguins.net/blog/how-to-trigger-a-kubernetes-cronjob-manually/

taint

remove

if we want to taint a node we use kubectl taint as follows

— https://pet2cattle.com/2021/09/k8s-node-untaint

We can use kubectl taint but adding an hyphen at the end to remove the taint (untaint the node)

— https://pet2cattle.com/2021/09/k8s-node-untaint

$ kubectl taint nodes minikube application=example:NoSchedule- node/minikubee untainted

— https://pet2cattle.com/2021/09/k8s-node-untaint

If we don’t know the command used to taint the node we can use kubectl describe node to get the exact taint we’ll need to use to untaint the node

— https://pet2cattle.com/2021/09/k8s-node-untaint

Notes linking here