Skip to content

Commit

Permalink
Merge pull request #496 from baalooos/use-built-in-priorit-class
Browse files Browse the repository at this point in the history
modify documentation on Daemonset to use built-in Priority Class
  • Loading branch information
jul-dan authored Jan 7, 2025
2 parents a9a2f18 + ccde766 commit 2cf06f0
Show file tree
Hide file tree
Showing 3 changed files with 46 additions and 64 deletions.
55 changes: 23 additions & 32 deletions website/guides/advanced/deploy-daemonset-with-karpenter.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_on: "2024-11-27"
last_modified_on: "2024-12-06"
$schema: "/.meta/.schemas/guides.json"
title: Deploy a DaemonSet in a Karpenter context
description: How to ensure your DaemonSet is well deployed when you are using Karpenter.
Expand All @@ -12,7 +12,7 @@ import Alert from '@site/src/components/Alert';
import Steps from '@site/src/components/Steps';
import Assumptions from '@site/src/components/Assumptions';

[Karpenter](https://karpenter.sh/) is a great way to cut your AWS bill. It provides an easy and flexible way to scale and optimize your resource consumption. But there is a known [issue](https://github.com/kubernetes-sigs/karpenter/issues/731) with capacity planning when you deploy DaemonSets. In this guide, I will present the issue and explain how to avoid it by using Priority Class.
[Karpenter](https://karpenter.sh/) is a great way to cut your AWS bill. It provides a simple and flexible way to scale and optimize your resource consumption. But there is a known [issue](https://github.com/kubernetes-sigs/karpenter/issues/731) with capacity planning when deploying DaemonSets. In this guide, we will expolore this issue and demonstrate how to resolve it using PriorityClass.

<!--
THIS FILE IS AUTOGENERATED!
Expand All @@ -26,7 +26,10 @@ import Assumptions from '@site/src/components/Assumptions';

A DaemonSet in Kubernetes is a specialized controller used to ensure that a copy of a particular pod runs on all nodes in a cluster. It is particularly useful for deploying background tasks or system-level services that need to run on every node, such as log collectors, monitoring agents, or network components.

When nodes are added to the cluster, the DaemonSet automatically schedules the specified pod on the new nodes, ensuring consistent deployment across the entire infrastructure. Similarly, when nodes are removed, the DaemonSet takes care of cleaning up the pods that were running on those nodes.
Key features of DaemonSets include:

- Automatic Scheduling: When nodes are added to the cluster, the DaemonSet schedules the specified pod on the new nodes automatically.
- Cleanup: When nodes are removed, the DaemonSet cleans up the pods running on those nodes.

This makes DaemonSets a powerful tool for maintaining uniformity and reliability in the operation of essential services across a Kubernetes cluster.

Expand All @@ -36,48 +39,36 @@ There is a [known issue](https://github.com/kubernetes-sigs/karpenter/issues/731

This forces operators to over-provision their nodes, resulting in inefficient resource utilization and higher costs. While the Kubernetes community and Karpenter developers are working on solutions, users currently need to manually adjust resource allocations and monitor node utilization to mitigate these issues.

A way to resolve this problem is to use a Priority Class and attach it to the DaemonSet we are creating.
A way to resolve this problem is to use a PriorityClass and attach it to the DaemonSet we are creating.

## How to resolve it?

### What is a Priority Class?

A PriorityClass in Kubernetes is a resource used to assign a priority level to pods. This resource helps the scheduler make decisions during resource contention.

- Higher-priority pods are scheduled before lower-priority ones
- In case of resource shortages, lower-priority pods may be preempted (evicted) to make room for higher-priority pods.
### What is a PriorityClass?

This ensures that critical workloads receive the necessary resources to run effectively.
A PriorityClass in Kubernetes is a resource used to assign priority levels to pods. It helps the Kubernetes scheduler make decisions during resource contention by determining which pods should be scheduled first or evicted in case of resource shortages.

### Deploy a new Priority Class using Helm
- Pods with higher priorities are scheduled before those with lower priorities.
- During resource shortages, lower-priority pods may be preempted (evicted) to free up resources for higher-priority pods.

I created a [simple repository](https://github.com/Qovery/qovery-charts-examples/tree/main/karpenter-daemonset-priority-class-main) you can clone to follow along.
This ensures that critical workloads have the resources they need to run effectively.

Create the karpenter-priority-class service in the Qovery environment where you want to deploy your DaemonSet by following [this documentation][docs.using-qovery.configuration.helm] and these values:
### Use PriorityClass built-in Helm Charts

* General:
* Service name: `karpenter-priority-class`
* Source:
* Helm source: `Git Provider`
* Git account: `Public repository` (Change if you are not using GitHub)
* Repository: `https://github.com/Qovery/qovery-charts-examples.git` (Replace by the name of your repository)
* Branch: `main`
* Root application path: `/karpenter-daemonset-priority-class-main`
* Allow cluster-wide resources :heavy_check_mark:
Many Helm charts include built-in values to automatically create and configure PriorityClasses. We have examples in our documentation:

Click on Continue
- [Kubernetes observability and monitoring with Datadog
](https://hub.qovery.com/guides/tutorial/kubernetes-observability-and-monitoring-with-datadog/)
- [Deploy JupyterHub using Helm
](https://hub.qovery.com/guides/tutorial/deploy-jupyterhub-qovery/)

* Values override as file:
* File source: `None`
### Use Qovery's dedicated PriorityClass

Then, you can:

- deploy this helm service to add the priority class on your cluster
- Modify your DaemonSet configuration to use the new priority class an redeploy it
When deploying Qovery on a cluster, a dedicated PriorityClass named `qovery-standard-priority` is created automatically.
You can use this PriorityClass when deploying a new Helm chart or DaemonSet to ensure that DaemonSets deploy correctly, even during resource contention.

## Conclusion

Even if Karpenter is a great way of reducing your AWS bill, sometimes you will have to do some manual lifting. This issue is a good example. A single Priority Class is enough to avoid a complex resource allocation problem.
While tools like Karpenter are excellent for optimizing costs and reducing your AWS bill, some manual configuration is sometimes required. This issue demonstrates that adding a single PriorityClass can effectively prevent complex resource allocation problems.



[docs.using-qovery.configuration.helm]: /docs/using-qovery/configuration/helm/
51 changes: 21 additions & 30 deletions website/guides/advanced/deploy-daemonset-with-karpenter.md.erb
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,16 @@ import Alert from '@site/src/components/Alert';
import Steps from '@site/src/components/Steps';
import Assumptions from '@site/src/components/Assumptions';

[Karpenter](https://karpenter.sh/) is a great way to cut your AWS bill. It provides an easy and flexible way to scale and optimize your resource consumption. But there is a known [issue](https://github.com/kubernetes-sigs/karpenter/issues/731) with capacity planning when you deploy DaemonSets. In this guide, I will present the issue and explain how to avoid it by using Priority Class.
[Karpenter](https://karpenter.sh/) is a great way to cut your AWS bill. It provides a simple and flexible way to scale and optimize your resource consumption. But there is a known [issue](https://github.com/kubernetes-sigs/karpenter/issues/731) with capacity planning when deploying DaemonSets. In this guide, we will explore this issue and demonstrate how to resolve it using PriorityClass.

## What is a DaemonSet?

A DaemonSet in Kubernetes is a specialized controller used to ensure that a copy of a particular pod runs on all nodes in a cluster. It is particularly useful for deploying background tasks or system-level services that need to run on every node, such as log collectors, monitoring agents, or network components.

When nodes are added to the cluster, the DaemonSet automatically schedules the specified pod on the new nodes, ensuring consistent deployment across the entire infrastructure. Similarly, when nodes are removed, the DaemonSet takes care of cleaning up the pods that were running on those nodes.
Key features of DaemonSets include:

- Automatic Scheduling: When nodes are added to the cluster, the DaemonSet schedules the specified pod on the new nodes automatically.
- Cleanup: When nodes are removed, the DaemonSet cleans up the pods running on those nodes.

This makes DaemonSets a powerful tool for maintaining uniformity and reliability in the operation of essential services across a Kubernetes cluster.

Expand All @@ -27,45 +30,33 @@ There is a [known issue](https://github.com/kubernetes-sigs/karpenter/issues/731

This forces operators to over-provision their nodes, resulting in inefficient resource utilization and higher costs. While the Kubernetes community and Karpenter developers are working on solutions, users currently need to manually adjust resource allocations and monitor node utilization to mitigate these issues.

A way to resolve this problem is to use a Priority Class and attach it to the DaemonSet we are creating.
A way to resolve this problem is to use a PriorityClass and attach it to the DaemonSet we are creating.

## How to resolve it?

### What is a Priority Class?

A PriorityClass in Kubernetes is a resource used to assign a priority level to pods. This resource helps the scheduler make decisions during resource contention.

- Higher-priority pods are scheduled before lower-priority ones
- In case of resource shortages, lower-priority pods may be preempted (evicted) to make room for higher-priority pods.

This ensures that critical workloads receive the necessary resources to run effectively.
### What is a PriorityClass?

### Deploy a new Priority Class using Helm
A PriorityClass in Kubernetes is a resource used to assign priority levels to pods. It helps the Kubernetes scheduler make decisions during resource contention by determining which pods should be scheduled first or evicted in case of resource shortages.

I created a [simple repository](https://github.com/Qovery/qovery-charts-examples/tree/main/karpenter-daemonset-priority-class-main) you can clone to follow along.
- Pods with higher priorities are scheduled before those with lower priorities.
- During resource shortages, lower-priority pods may be preempted (evicted) to free up resources for higher-priority pods.

Create the karpenter-priority-class service in the Qovery environment where you want to deploy your DaemonSet by following [this documentation][docs.using-qovery.configuration.helm] and these values:
This ensures that critical workloads have the resources they need to run effectively.

* General:
* Service name: `karpenter-priority-class`
* Source:
* Helm source: `Git Provider`
* Git account: `Public repository` (Change if you are not using GitHub)
* Repository: `https://github.com/Qovery/qovery-charts-examples.git` (Replace by the name of your repository)
* Branch: `main`
* Root application path: `/karpenter-daemonset-priority-class-main`
* Allow cluster-wide resources :heavy_check_mark:
### Use PriorityClass built-in Helm Charts

Click on Continue
Many Helm charts include built-in values to automatically create and configure PriorityClasses. We have examples in our documentation:

* Values override as file:
* File source: `None`
- [Kubernetes observability and monitoring with Datadog
](https://hub.qovery.com/guides/tutorial/kubernetes-observability-and-monitoring-with-datadog/)
- [Deploy JupyterHub using Helm
](https://hub.qovery.com/guides/tutorial/deploy-jupyterhub-qovery/)

Then, you can:
### Use Qovery's dedicated PriorityClass

- deploy this helm service to add the priority class on your cluster
- Modify your DaemonSet configuration to use the new priority class an redeploy it
When deploying Qovery on a cluster, a dedicated PriorityClass named `qovery-standard-priority` is created automatically.
You can use this PriorityClass when deploying a new Helm chart or DaemonSet to ensure that DaemonSets deploy correctly, even during resource contention.

## Conclusion

Even if Karpenter is a great way of reducing your AWS bill, sometimes you will have to do some manual lifting. This issue is a good example. A single Priority Class is enough to avoid a complex resource allocation problem.
While tools like Karpenter are excellent for optimizing costs and reducing your AWS bill, some manual configuration is sometimes required. This issue demonstrates that adding a single PriorityClass can effectively prevent complex resource allocation problems.
4 changes: 2 additions & 2 deletions website/guides/tutorial/deploy-jupyterhub-qovery.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
last_modified_on: "2024-12-05"
last_modified_on: "2024-12-06"
$schema: "/.meta/.schemas/guides.json"
title: Deploy JupyterHub using Helm
description: How to deploy JupyterHub on Qovery using the official Helm chart.
Expand Down Expand Up @@ -92,7 +92,7 @@ proxy:
</TabItem>
<TabItem value="EKS with karpenter">
<TabItem value="karpenter">
To avoid [a known issue](https://hub.qovery.com/guides/advanced/deploy-daemonset-with-karpenter/) with Karpenter, we need to add a priorityClass to our pods. It is possible to do this using built-in configuration keys:
Expand Down

0 comments on commit 2cf06f0

Please sign in to comment.