Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JetStream Ordered consumer #169

Merged
merged 11 commits into from
Nov 6, 2023
Merged

JetStream Ordered consumer #169

merged 11 commits into from
Nov 6, 2023

Conversation

mtmk
Copy link
Collaborator

@mtmk mtmk commented Oct 25, 2023

Ordered consumer implementation based on https://github.com/nats-io/nats-architecture-and-design/blob/main/adr/ADR-17.md

  • Ordered consumer creation parameters
  • Wrapper around the existing pull consume / fetch logic

API change discussion:

Simplified consumer api to just consume, fetch and next methods removing the *All* versions. Now consume and fetch methods (renamed from the *All* versions) return IAsyncEnumerable only. This reduces the API footprint as well as making it consistent between 'Ordered Consumer' and 'Consumer'.

  • Removed consume and fetch methods exposing channel reader.
  • Renamed *All* methods e.g. FetchAllAsync() to FetchAsync().
  • Also removed the interfaces exposing the channel readers.

resolves #32

mtmk added 5 commits October 25, 2023 15:11
Simplified consumer api to just consume, fetch and next methods
removing the *All* versions. Now consume and fetch methods return
IAsyncEnumerable only. This reduces the API foot-print as well as
making it consistent between 'Ordered Consumer' and 'Consumer'.

* Removed consume and fetch methods exposing channel reader.
* Renamed *All* methods e.g. FetchAllAsync() to FetchAsync().
* Also removed the interfaces exposing the channel readers.
@mtmk mtmk changed the title [WIP] Ordered consumer JetStream Ordered consumer Oct 26, 2023
@mtmk mtmk marked this pull request as ready for review October 26, 2023 11:11
@mtmk mtmk requested a review from caleblloyd October 26, 2023 11:11
@mtmk
Copy link
Collaborator Author

mtmk commented Nov 3, 2023

What can we do about the disparity with core subscribe?

Subscribe call on core on the other hand still returns an interface INatsSub<T> which exposes a channel reader for messages among a few others:

public interface INatsSub<T> : IAsyncDisposable
{
    ChannelReader<NatsMsg<T>> Msgs { get; }
    string Subject { get; }
    string? QueueGroup { get; }
    public ValueTask UnsubscribeAsync();
}

An idea here maybe to introduce a different call to match JetStream calls:

// New matching call
IAsyncEnumerable<NatsMsg<T>> SubscribeAsync<T>(string subject, ...);

// Existing call renamed
ValueTask<INatsSub<T>> SubscribeCoreAsync<T>(string subject, ...);

if (!string.IsNullOrWhiteSpace(request.Config.Name))
{
subject += $".{request.Config.Name}";
request.Config.Name = default!;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why mutate name here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spot on! no need for it.

Comment on lines 195 to 199
if (!string.IsNullOrWhiteSpace(request.Config.FilterSubject))
{
subject += $".{request.Config.FilterSubject}";
}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the name was not set, but the filter subject was set? With this logic the subject token for name is not present, and the filter subject token is appended to where the name subject should go

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FilterSubject is never set actually. Dead code. I think it might be my copy & paste error. Sorry about that!

Copy link
Collaborator

@caleblloyd caleblloyd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mtmk mtmk merged commit 548b15c into main Nov 6, 2023
9 checks passed
@mtmk mtmk deleted the ordered-consumer branch November 6, 2023 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Ordered Consumer
2 participants