Updates for Karpenter 0.35.x CRDs, fixes from previous updates

[NicholasRaymondiSpot]

TL:DR;

Updates CRDs-catalog for Karpenter 0.35.x configurations, other minor changes

Issue

The AWS CRDs have existed since the Karpenter 0.19.0 release, going back through these configurations I was not able to find a number of the additionalProperties claims that exists in these CRDs-catalog definitions. There are also new changes needed that were previously missed in order for the CRDs-catalog to catch-up to the live configurations.

Solution

I used the CRD Extractor script to export the 0.35 CRDs for Karpenter. I based my testing changes on a provided example config. I was able to confirm validation by adding a new minValue to our manifest prior to see it fail, here's a snippet from the manifest:

apiVersion: karpenter.sh/v1beta1
kind: NodePool
spec:
  template:
    spec:
      requirements:
      - key: karpenter.k8s.aws/instance-category
        operator: In
        values:
        - c
        - m
        - r
        - t
        minValues: 3

This produced the following error:

kubeconform -schema-location default -schema-location 'https://raw.githubusercontent.com/InfinityFlowApp/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json' "$output_path/$chart_name.yaml"

karpenter-nodepools.yaml
NodePool testing-node-pool is invalid: 
  problem validating schema. Check JSON formatting: 
    jsonschema: '/spec/template/spec/requirements/3' does not validate with 
    https://raw.githubusercontent.com/InfinityFlowApp/CRDs-catalog/main/karpenter.sh/nodepool_v1beta1.json#/properties/spec/properties/template/properties/spec/properties/requirements/items/additionalProperties: additionalProperties 'minValues' not allowed

Once I updated my feature branch and ran validation using it, we're given a successful outcome:

kubeconform -schema-location default -schema-location 'https://raw.githubusercontent.com/NicholasRaymondiSpot/CRDs-catalog/karpenter-035/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json' "$output_path/$chart_name.yaml"

Changes in karpenter-nodepools.yaml:

@@ -62,6 +62,7 @@ spec:
         - m
         - r
         - t
+        minValues: 3
       - key: karpenter.k8s.aws/instance-generation
         operator: Gt
         values:

Changes Made

• NodeClaim
  ** Added minValues to properties spec, updated the requirements description to include it's usage
  ** Updated kubernetes validation rules for requirements
  ** Added spec requirement to NodeClaim definition
• NodePool
  ** Added minValues to properties spec, updated the requirements description to include it's usage
  ** Updated kubernetes validation rules for requirements
  ** Added spec requirement to NodePool & NodePool Template spec definitions
  ** Added template requirement to NodePoolSpec definition
  ** Adds maxProperties: 0 to the requirements.resources spec

Notes

This was tested with Karpenter 0.35.5, there is no guarantee that these changes will work with newer CRDs of the addon. I'll be adding more changes soon for version 0.36.x if they're needed.

Resources & References

https://raw.githubusercontent.com/aws/karpenter/v0.34.6/pkg/apis/crds/karpenter.k8s.aws_ec2nodeclasses.yaml
https://raw.githubusercontent.com/aws/karpenter/v0.35.5/pkg/apis/crds/karpenter.k8s.aws_ec2nodeclasses.yaml

https://raw.githubusercontent.com/aws/karpenter/v0.34.6/pkg/apis/crds/karpenter.sh_nodeclaims.yaml
https://raw.githubusercontent.com/aws/karpenter/v0.35.5/pkg/apis/crds/karpenter.sh_nodeclaims.yaml

https://raw.githubusercontent.com/aws/karpenter/v0.34.6/pkg/apis/crds/karpenter.sh_nodepools.yaml
https://raw.githubusercontent.com/aws/karpenter/v0.35.5/pkg/apis/crds/karpenter.sh_nodepools.yaml

Upgrading to v0.35.0+

Related PRs

chore: update karpenter crds to version v0.35.0 #280
Updating Karpenter v1beta1 Spec #318
fix(karpenter): update v1beta1 spec

[ElanHasson]

@NicholasRaymondiSpot thanks for the PR--

Question-- how are you generating these?

I'm wondering how we can automate this stuff.

[ElanHasson]

Was this a regression from #12 ?

[NicholasRaymondiSpot]

@NicholasRaymondiSpot thanks for the PR--

Question-- how are you generating these?

I'm wondering how we can automate this stuff.

I'm doing it by hand, using VSCode to diff the public CRDs. Then I look at each block and compare it to what's in the CRDs-catalog to make sure they match in order. I also used the other public PRs that were made for reference as well. My suggestion to start would be to create a pipeline that collects the current Karpenter release CRDS and values.yaml and compares them against the newer versions. This will give a starting point for what changes are needed.

Then perhaps generate a manifest with every configuration type defined as a validation test step.

Was this a regression from #12 ?

This is in addition to that release, #12 was missing some newer 0.35 changes and my original PR should have included these removals & reorganization changes but I was only focused on the validation passing and adding the new additions. I now went back and compared the entire contents of the file.

[ElanHasson]

I see.

Can you try using https://github.com/InfinityFlowApp/CRDs-catalog?tab=readme-ov-file#crd-extractor ?

You point it at a cluster and it extras all CRDs and formats them correctly for here.

[NicholasRaymondiSpot]

Can you try using https://github.com/InfinityFlowApp/CRDs-catalog?tab=readme-ov-file#crd-extractor ?

You point it at a cluster and it extras all CRDs and formats them correctly for here.

Thanks for this, @ElanHasson. Obviously a much easier and more reliable approach. 😄 I've updated the PR comments, these changes are ready for review.

[ElanHasson]

Thanks for the PR!