Always use Credential Plugins for Registry Access

[kferrone]

In our case, we are on EKS and using ECR images. In the world of aws you "login" to a registry using the ecr plugin helper from here: https://github.com/awslabs/amazon-ecr-credential-helper

We are forced to have an image pull secret in k8s with the credentials. To do this, you basically need to
first run this for creds aws ecr get-login-password
second we inject those creds to a k8s secret with this kubectl create secret docker-registry
Unfortunately, the credentials only last about 15 minutes, so we need to constantly rebuild this secret, for now we had to make a cronjob.

Normally when you pull an image, the ecr credential helper will use the aws api to get those creds and give directly to whatever is pulling/pushing the image.

Here is how kaniko can use the creds helper:

mkdir -p /kaniko/.docker
echo "{\"credsStore\":\"ecr-login\"}" > /kaniko/.docker/config.json

With this, kaniko doesn't need a secret with registry creds, the ecr cred helper will use the standard AWS SDK login flow, then use the aws api to get the registry password. When running on an EC2 instance, the ecr helper will use the aws instance profile. So no creds need to be configured at all. Magick!

I was told the actual issue is the KMM code itself is using a library called crane. This crane lib is what needs the credentials to manually make a call to the registry to check if the image exists in the first place. So maybe somehow the code in this case can use the credential helpers.

These credential helpers exist for GCP, Azure, and AWS who all have their own registries which can be accessed by using their api to gain credentials. KMM should use these so it's not difficult to access a registry.

Here is the original slack conversation:
https://kubernetes.slack.com/archives/C037RE58RED/p1736364086193159