[feature] Cache Key Customization

[zzmao]

Feature Area

/area backend
/area sdk

What feature would you like to see?

Setting custom cache key at the task level

What is the use case or pain point?

Currently, the cache key in KFP is generated internally, and users are unable to customize or set it.
However, some users have expressed the need for more granular control over caching behavior by setting custom cache keys at the task level.

Is there a workaround currently?

N/A

KFP community meeting discussion for this feature

https://www.youtube.com/watch?v=I5iynMg_DNk

Proposal:

https://docs.google.com/document/d/1oNgYyFYondaVSFf9Pd3Q9uVzaqBr5wrOMHKgX9MMa78/edit?tab=t.0

---

Love this idea? Give it a 👍.

[TheSwarnim]

Hi @HumairAK @zzmao, I would like to work on this issue as my first contribution in the community.

[HumairAK]

That would be great! /assign @TheSwarnim

[TheSwarnim]

Hey @HumairAK, thanks for assigning it to me. I usually work on weekend for the opensource project due to a full time job which I'm also pursuing. I'll raise the PR with all the changes by this coming weekend.

[Garvit-77]

Hey as it's more than 4 weeks and no PR was raised
@TheSwarnim are you still working on this issue ?

[TheSwarnim]

Hi @Garvit-77 a PR was raised for this change some time ago. So was not checking out this
#11466
Please let me know if pick this PR and update with the relevant changes raised by the reviewer