Document VPA behavior for sidecar containers and how to change it #7645
Conversation
Signed-off-by: Omer Aplatony <[email protected]>
k8s-ci-robot
added
kind/documentation
k8s-ci-robot
added
area/vertical-pod-autoscaler
size/L
| @@ -0,0 +1,173 @@ | |||
| # VPA Sidecar Container Management | |||
|
|
|||
| The Vertical Pod Autoscaler (VPA) has specific behavior when dealing with sidecar containers that are injected into pods via admission webhooks. This document explains the default behavior and how to customize it for your needs. | |||
There was a problem hiding this comment.
Does "Sidecar" refer to "A container that isn't the main one", or to the formal Sidecar definition (ie: https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/)
I assume it's the former. It may be good to clarity this.
Co-authored-by: Adrian Moisey <[email protected]>
|
Documentation is hard, so thanks for doing this. I'm wondering how this page fits into the broader layout of the documents. My best guess is that it's a "FAQ" (ie: https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/docs/faq.md). I'm not entirely sold on that though. Having the document exist is a great first step, we can move it later if needed. |
Signed-off-by: Omer Aplatony <[email protected]>
I was wondering about that as well. Perhaps @voelzmo might have some ideas on how the overall layout should be structured |
|
/retest |
| This default "manage everything" approach can cause problems with sidecars because: | ||
| - Sidecar containers often have their own resource requirements set by their injection webhooks | ||
| - VPA's automatic management may conflict with these requirements | ||
| - Without proper handling, this can lead to resource conflicts and pod instability |
There was a problem hiding this comment.
| - Without proper handling, this can lead to resource conflicts and pod instability | |
| - Without proper handling, this can lead to problems like endless eviction loops |
| Without proper handling of sidecar containers, the following problematic sequence could occur: | ||
|
|
||
| 1. VPA admission controller sets resources for all containers | ||
| 2. Sidecar webhook injects a new container with its own resource requirements |
There was a problem hiding this comment.
| 2. Sidecar webhook injects a new container with its own resource requirements | |
| 2. Sidecar webhook reconciles the injected sidecar container to the original resource requirements |
|
|
||
| ### Option 1: Webhook Ordering | ||
|
|
||
| To have VPA manage sidecar resources, ensure your webhook names follow this pattern: |
There was a problem hiding this comment.
| To have VPA manage sidecar resources, ensure your webhook names follow this pattern: | |
| If you know that you only use sidecar injecting webhooks which _don't_ reconcile Container resources, you can choose to have VPA manage sidecar resources. Ensure your webhook names follow this pattern, resulting in the VPA admission-controller webhook to be executed last: |
| @@ -0,0 +1,177 @@ | |||
| # VPA Sidecar Container Management | |||
|
|
|||
| In this document, "sidecar container" refers to any additional container that isn't the main application container in a pod. This is distinct from the formal [Kubernetes sidecar pattern](https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/), which describes a specific design pattern where a container enhances or extends the main container's functionality. Our usage here applies to all additional containers, regardless of their architectural pattern or purpose. | |||
There was a problem hiding this comment.
| In this document, "sidecar container" refers to any additional container that isn't the main application container in a pod. This is distinct from the formal [Kubernetes sidecar pattern](https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/), which describes a specific design pattern where a container enhances or extends the main container's functionality. Our usage here applies to all additional containers, regardless of their architectural pattern or purpose. | |
| In this document, "sidecar container" refers to any additional Container that isn't the main application Container in a Pod. This is distinct from the [native Kubernetes sidecar pattern](https://kubernetes.io/docs/concepts/workloads/pods/sidecar-containers/), which makes use of `initContainers`. Our usage here applies to all additional regular `containers` only, as VPA does not support `initContainers` yet. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: omerap12 The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
What type of PR is this?
/kind documentation
What this PR does / why we need it:
Document VPA behavior for sidecar containers and how to change it
Which issue(s) this PR fixes:
Fixes #7641
Special notes for your reviewer:
Does this PR introduce a user-facing change?
Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: