Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement CLUSTER PUBSUB #866

Draft
wants to merge 19 commits into
base: main
Choose a base branch
from
Draft

Implement CLUSTER PUBSUB #866

wants to merge 19 commits into from

Conversation

vazois
Copy link
Contributor

@vazois vazois commented Dec 9, 2024 •
edited
Loading

This PR adds support for PUBSUB in Cluster mode.

  • Implement SPUBLISH and SSUBSCRIBE APIs
  • Add published message forwarding using the active gossip connections
  • Expose parameters to allow configuring max outstanding PubSub forwarding tasks
  • Augment latency and throughput benchmarks with PUBLISH and SPUBLISH support.

@vazois vazois force-pushed the vazois/cluster-pubsub branch 4 times, most recently from 2f789f5 to 422532b Compare December 12, 2024 01:28
vazois added 15 commits January 6, 2025 11:15
@vazois
move key spec extraction to clusterProvider
9b4001e
@vazois
implement CLUSTER PUBLISH api
ae8ffcc
@vazois
implement SSUBSCRIBE api
47c942e
@vazois
implement SPUBLISH api
46707ed
@vazois
implement separate code path for CLUSTER SPUBLISH
3eb8b25
@vazois
expose max number of outstanding pubsub forwarding tasks parameter
e3156a4
@vazois
add API to forward published messages using GarnetClient
4941cde
@vazois
add methods to extract nodeIds of all nodes in cluster and all nodes …
3371827 
…in a shard
@vazois
hook up published message forwarding
98dbf5c
@vazois
PUBLISH and SPUBLISH benchmark addition
e6163f5
@vazois
fix check for getting most recent config instance
87f7d98
@vazois
optimization 1: calculate candidates for published message forwarding…
f2c9570 
… only when config gets updated
@vazois
optimization2: ignore response from cluster publish
43e3580
@vazois
add publish BDN
a187c6d
@vazois
minimize task creation
b889fa5
@vazois vazois force-pushed the vazois/cluster-pubsub branch from de84b93 to 802b69f Compare January 6, 2025 19:15
@vazois vazois force-pushed the vazois/cluster-pubsub branch from 802b69f to 919f90c Compare January 7, 2025 00:17
@vazois vazois force-pushed the vazois/cluster-pubsub branch from 0f02ac5 to 4945338 Compare January 7, 2025 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@vazois