Konubinix' opinionated web of thoughts

Configure Liveness, Readiness and Startup Probes

Fleeting

In short,

  • readiness probe and liveness probe starts together1,
  • liveness probe is used to restart when something appears to hang2,
    • with a counter indicating each time the containers are up again3
  • readiness probe is used to detect when the pod can receive traffic4,
    • without specification, it becomes ready when all the containers have started5
  • startup probe6 is used to delay the liveness probe and readiness probe7, 8, to mitigate for slow starting pods,
    • that way, you can define frequent liveness checks that won’t restart the pod when its startup takes time9
    • if you don’t need to wait some specific event but some time you can simply use initialDelaySeconds10

Here is a copy of the possible values, because I often forget them and have a hard time finding them back.

Configure Probes Probes have a number of fields that you can use to more precisely control the behavior of startup, liveness and readiness

  • checks:initialDelaySeconds: Number of seconds after the container has started before startup, liveness or readiness probes are initiated. Defaults to 0 seconds. Minimum value is 0.
  • periodSeconds: How often (in seconds) to perform the probe. Default to 10 seconds. Minimum value is 1.
  • timeoutSeconds: Number of seconds after which the probe times out. Defaults to 1 second. Minimum value is 1.
  • successThreshold: Minimum consecutive successes for the probe to be considered successful after having failed. Defaults to 1. Must be 1 for liveness and startup Probes. Minimum value is 1.
  • failureThreshold: After a probe fails failureThreshold times in a row, Kubernetes considers that the overall check has failed: the container is not ready / healthy / live. For the case of a startup or liveness probe, if at least failureThreshold probes have failed, Kubernetes treats the container as unhealthy and triggers a restart for that specific container. The kubelet takes the setting of terminationGracePeriodSeconds for that container into account. For a failed readiness probe, the kubelet continues running the container that failed checks, and also continues to run more probes; because the check failed, the kubelet sets the Ready condition on the Pod to false.
  • terminationGracePeriodSeconds: configure a grace period for the kubelet to wait between triggering a shut down of the failed container, and then forcing the container runtime to stop that container. The default is to inherit the Pod-level value for terminationGracePeriodSeconds (30 seconds if not specified), and the minimum value is 1. See probe-level terminationGracePeriodSeconds for more detail.

https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

  • host: Host name to connect to, defaults to the pod IP. You probably want to set “Host” in httpHeaders instead.
  • scheme: Scheme to use for connecting to the host (HTTP or HTTPS). Defaults to HTTP.
  • path: Path to access on the HTTP server. Defaults to /.
  • httpHeaders: Custom headers to set in the request. HTTP allows repeated headers.
  • port: Name or number of the port to access on the container. Number must be in the range 1 to 65535

https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

Notes linking here


  1. Readiness and liveness probes can be used in parallel for the same container. Using both can ensure that traffic does not reach a container that is not ready for it, and that containers are restarted when they fail

    https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

     ↩︎
  2. liveness probes to know when to restart a container

    https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

     ↩︎
  3. RESTARTS counter increments as soon as a failed container comes back to the running state

    https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

     ↩︎
  4. readiness probes to know when a container is ready to start accepting traffic

    https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

     ↩︎
  5. Pod is considered ready when all of its containers are ready.

    https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

     ↩︎
  6. startup probes to know when a container application has started

    https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

     ↩︎
  7. it disables liveness and readiness checks until it succeeds, making sure those probes don’t interfere with the application startup

    […]

    Once the startup probe has succeeded once, the liveness probe takes over

    https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

     ↩︎
  8. set up a startup probe with the same command, HTTP or TCP check, with a failureThreshold * periodSeconds long enough to cover the worse case startup time.

    […]

    Thanks to the startup probe, the application will have a maximum of 5 minutes (30 * 10 = 300s) to finish its startup

    https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

     ↩︎
  9. used to adopt liveness checks on slow starting containers, avoiding them getting killed by the kubelet before they are up and running

    https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

     ↩︎
  10. Liveness probes do not wait for readiness probes to succeed. If you want to wait before executing a liveness probe you should use initialDelaySeconds or a startupProbe

    https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

     ↩︎