Sync Catalog: Utilize EndpointSlice conditions when endpoints are updated

[jukie]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

Is your feature request related to a problem? Please describe.

Sync Catalog currently doesn't account for an Endpoint's condition state. As a result, an endpoint that is in terminating state will remain being synced to Consul until it's fully removed. This can cause issues in environments where a user may want to healthily wait for connections to drop off on a pod (e.g. the synced consul instance) before fully terminating or the opposite use case of not wanting to register the Endpoint in Consul until it's deemed ready.

Feature Description

With #3693 Sync Catalog changed from using Endpoints to EndpointSlices. This was mostly done to provide access to topology information but another feature of EndpointSlices is Conditions which expose additional state about whether the endpoint is ready, serving, and/or terminating.
Sync Catalog could use this extra information to enhance existing behavior and only register Endpoint's that are marked as ready and clean up Endpoints once they are marked as terminating.

Use Case(s)

• I'm rolling a new replicaset for my Kubernetes Deployment and my workload has a long initialization phase. I don't want it to be registered to Consul until all readiness checks have passed and the underlying Endpoint is marked as ready or serving.
• I'm using consul-template to add consul instances as backend targets in an external system. It would be ideal to have these Consul instances updated with some state information or removed whenever the pod is being torn down to allow for a buffer period and any open connections to complete.

Contributions

#3874 adds this feature

[jukie]

I've opened a PR for this under #3874

[jukie]

bumpety bump in case anyone's watching

[tholcman]

Hmm I have also started digging into it:
#4080

looks like this repo is ignored by Hashicorp :/

Anyway ignoring "not-ready" pods is not the right implementation, better is to change "Status". For example in our implementation, envoy if it sees certain percentage of unhealthy endpoints, it starts routing part of the traffic to a different cluster (different region in our case) - so even knowing about not-ready pods is helpful information.

[blake]

@tholcman This issue is being reviewed by our team. There have been more recent comments on the PR. #3874 (comment)