Control plane init generates userdata larger than max user data size in AWS

[ttreptow]

What happened:

Creating a new cluster with rke2 provider v0.8.0, the AWSMachine fails to come up due to the error

"failed to create AWSMachine instance: failed to run instance: InvalidParameterValue: Encoded User data is limited to 25600 bytes"

It mostly seems to affect the first control plane node since the userdata for that node (control plane init) is larger.

Note that the limit is 25600 base64 encoded or around 16k unencoded. This is easily exceeded due to the certs included in the user data as well as rke2 config and any start up scripts needed.

What did you expect to happen:

First node comes up without error

How to reproduce it:

Create an RKE2 cluster using rke2 provider v0.8.0 and cluster-api AWS (CAPA)

Anything else you would like to add:
cloud-init supports gzipped userdata. If i manually replace the generated data with base64 encoded gzipped data, then the node comes up fine. I think the fix would be to always gzip cloud-init userdata.

Environment:

• rke provider version: v0.8.0
• OS (e.g. from /etc/os-release): ubuntu 22.04 LTS

[ttreptow]

I have a branch with a fix for this (ignore the closed PR, that was incomplete)

I have a design question first:

My fix preemptively gzips all cloudinit userdata. Alternatively we could have a threshold where if the size is less than the threshold it will leave it untouched but then you'd need to figure out a good threshold value across various infrastructure providers. Always compressing is much simpler but it will be a noticeable change to people (and tools?) expecting uncompressed data.

I've been running this change in my clusters without issue

[Danil-Grigorev]

@ttreptow Have you tried individually gzip+base64 encoding certificate files in the cloudinit and specifying it in Encoding section? It is supported by the current bootstrapping implementation, and does not suffer from the downside you mentioned.