Updates for Karpenter 0.34.x using v1beta1 APIs

[NicholasRaymondiSpot]

TL:DR;

Updates CRDs-catalog for Karpenter 0.34.x configurations

Issue

Karpenter has made some changes to their CRDs between versions 0.32.x & 0.33.x, deprecating the v1alphax APIs and replacing them with v1beta1. (more details) Karpenter deployments running release v0.32.x allow clusters to operate both alpha & beta APIs for safer onboarding updates of applications.

Starting in Karpenter 0.34.x, the nodepool.spec.disruption.budgets spec is introduced which the current CRDs-catalog has no references for. We've introduced those values to our configuration and hit this error when running validation for the beta APIs:

karpenter-nodepools.yaml - NodePool test-node-pool is invalid: problem validating schema.
Check JSON formatting: jsonschema: '/spec/disruption' does not validate with https://raw.githubusercontent.com/InfinityFlowApp/CRDs-catalog/main/karpenter.sh/nodepool_v1beta1.json#/properties/spec/properties/disruption/additionalProperties: additionalProperties 'budgets' not allowed

Solution

I referenced the 0.33.5 & 0.34.6 CRDs published for the v1beta1 EC2NodeClass, NodeClaim and NodePool APIs and compared the configurations to one another. I also went back to 0.32.10 to ensure there were no major changes introduced there either. Finding the changes for budgets in the NodePools manifest, I found some other small changes that were also needed to make the configurations match.

Now we can use my forked CRDs-catalog configuration and we get valid manifests produced and updated with the new values as expected:

kubeconform -schema-location default -schema-location 'https://raw.githubusercontent.com/NicholasRaymondiSpot/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json' "$output_path/$chart_name.yaml"

Changes in karpenter-nodepools.yaml:
-    expireAfter: Never
+    expireAfter: 720h
+    budgets:
+      - nodes: "10%"

Changes Made

• All three manifests have had their descriptions reformatted with newline character breaks.
• EC2NodeClass
  ** instanceProfile was added, it was missing since 0.32.x
  ** instanceStorePolicy is added with a default RAID0 enumeration
• NodePools
  ** Added budgets configuration with it's defaults
  ** Added maxProperties: 0 to the spec.template.requirements.resources

Notes

This was tested with Karpenter 0.34.6, there is no guarantee that these changes will work with newer releases of the addon. I'll be adding more changes soon for versions 0.35.x & 0.36x if they're needed.

Resources & References

https://raw.githubusercontent.com/aws/karpenter/v0.32.10/pkg/apis/crds/karpenter.k8s.aws_ec2nodeclasses.yaml
https://raw.githubusercontent.com/aws/karpenter/v0.33.5/pkg/apis/crds/karpenter.k8s.aws_ec2nodeclasses.yaml
https://raw.githubusercontent.com/aws/karpenter/v0.34.6/pkg/apis/crds/karpenter.k8s.aws_ec2nodeclasses.yaml

https://raw.githubusercontent.com/aws/karpenter/v0.32.10/pkg/apis/crds/karpenter.sh_nodeclaims.yaml
https://raw.githubusercontent.com/aws/karpenter/v0.33.5/pkg/apis/crds/karpenter.sh_nodeclaims.yaml
https://raw.githubusercontent.com/aws/karpenter/v0.34.6/pkg/apis/crds/karpenter.sh_nodeclaims.yaml

https://raw.githubusercontent.com/aws/karpenter/v0.32.10/pkg/apis/crds/karpenter.sh_nodepools.yaml
https://raw.githubusercontent.com/aws/karpenter/v0.33.5/pkg/apis/crds/karpenter.sh_nodepools.yaml
https://raw.githubusercontent.com/aws/karpenter/v0.34.6/pkg/apis/crds/karpenter.sh_nodepools.yaml

Upgrading to v0.34.0+

Related PRs

chore: update karpenter crds to version v0.35.0 #280
Updating Karpenter v1beta1 Spec #318

[NicholasRaymondiSpot]

@ElanHasson this is ready for review.

[ElanHasson]

@NicholasRaymondiSpot Thanks! on it!

[ElanHasson]

LGTM!

Thank you for the PR!