Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add the # of messages prioritized in the queue to the http resp… #5043

Merged
merged 13 commits into from
Jan 6, 2025

Conversation

kamiyaa
Copy link
Collaborator

@kamiyaa kamiyaa commented Dec 18, 2024

Description

Add a mpsc channel between relayer and message retry handler, so message retry handler knows the retry results from the relayer.

POST /message_retry endpoint now returns a JSON with details on the retry request:

#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct MessageRetryResponse {
    /// ID of the retry request
    pub uuid: String,
    /// how many pending operations were processed
    pub processed: usize,
    /// how many of the pending operations matched the retry request pattern
    pub matched: u64,
}

Drive-by changes

Related issues

Fixes #4059

Backward compatibility

Testing

Updated unit tests to return a response so the handler is not blocked

Copy link

changeset-bot bot commented Dec 18, 2024

⚠️ No Changeset found

Latest commit: 2cb99b3

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

rust/main/Cargo.toml Outdated Show resolved Hide resolved
Copy link

codecov bot commented Dec 18, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.53%. Comparing base (99c91d8) to head (2cb99b3).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #5043   +/-   ##
=======================================
  Coverage   77.53%   77.53%           
=======================================
  Files         103      103           
  Lines        2110     2110           
  Branches      190      190           
=======================================
  Hits         1636     1636           
  Misses        453      453           
  Partials       21       21           
Components Coverage Δ
core 87.80% <ø> (ø)
hooks 79.39% <ø> (ø)
isms 83.68% <ø> (ø)
token 91.27% <ø> (ø)
middlewares 79.80% <ø> (ø)

Copy link
Collaborator

@tkporter tkporter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good stuff!

excited that this task is fairly meaty, lots of interesting concurrency and touching a lot of systemic things in the relayer :)

rust/main/agents/relayer/src/msg/op_queue.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/server/message_retry.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/server/message_retry.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/server/message_retry.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/server/message_retry.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/msg/op_queue.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/server/message_retry.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/msg/op_queue.rs Outdated Show resolved Hide resolved
 - also updating tracing calls to use structured logs
 - update variable names to receiver/transmitter instead of rx/tx
 - this creates a nice separation between each retry request
 and won't cause multiple requests to conflict with each others
 responses
Copy link
Contributor

@daniel-savu daniel-savu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😎 awesome work, hitting the ground running from the first PR!!

rust/main/agents/relayer/Cargo.toml Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/msg/op_queue.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/msg/op_queue.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/msg/op_queue.rs Show resolved Hide resolved
rust/main/agents/relayer/src/msg/op_queue.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/server/message_retry.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/server/message_retry.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/server/message_retry.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/msg/op_queue.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/server/message_retry.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/server/message_retry.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/server/message_retry.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/server/message_retry.rs Outdated Show resolved Hide resolved
rust/main/utils/run-locally/src/server.rs Outdated Show resolved Hide resolved
rust/main/utils/run-locally/src/main.rs Outdated Show resolved Hide resolved
 - update test cases to (1, 1)
 - add error handling
 - panic if failed to send retry request
@daniel-savu
Copy link
Contributor

🙏 go @kamiyaa! You need to fix clippy for CI to pass btw: cargo clippy -- -D warnings

@kamiyaa
Copy link
Collaborator Author

kamiyaa commented Jan 2, 2025

cargo clippy -- -D warnings

Ah, let me do that before merging!

Copy link
Collaborator

@tkporter tkporter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mostly stylistic nits but I think the reading responses is prone to a race condition

rust/main/agents/relayer/src/msg/op_queue.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/msg/op_queue.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/msg/op_queue.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/msg/op_queue.rs Outdated Show resolved Hide resolved
rust/main/agents/relayer/src/msg/op_queue.rs Outdated Show resolved Hide resolved
rust/main/utils/run-locally/src/server.rs Outdated Show resolved Hide resolved
rust/main/utils/run-locally/src/server.rs Show resolved Hide resolved
rust/main/utils/run-locally/src/main.rs Outdated Show resolved Hide resolved
 - fix nits
 - add comments to functions
 - remove thread::spawn
@tkporter tkporter added this pull request to the merge queue Jan 6, 2025
Merged via the queue into hyperlane-xyz:main with commit c5d4bd6 Jan 6, 2025
39 of 42 checks passed
daniel-savu added a commit that referenced this pull request Jan 8, 2025
if retry_req.pattern.op_matches(&op) {
debug!(uuid = retry_req.uuid, "Matched request");
retry_response.matched += 1;
Some(retry_req.uuid.clone())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like this isn't used anymore? think we can go back to using .any instead of filter_map and avoid this allocation


// This channel is only created to service this single
// retry request so we're expecting a single response
// from each transmitter end, hence we are using a channel of size 1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: comment is stale

tiendn pushed a commit to tiendn/hyperlane-monorepo that referenced this pull request Jan 11, 2025
hyperlane-xyz#5043)

### Description

Add a mpsc channel between relayer and message retry handler, so message
retry handler knows the retry results from the relayer.

`POST /message_retry` endpoint now returns a JSON with details on the
retry request:

```rust
#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct MessageRetryResponse {
    /// ID of the retry request
    pub uuid: String,
    /// how many pending operations were processed
    pub processed: usize,
    /// how many of the pending operations matched the retry request pattern
    pub matched: u64,
}
```

### Drive-by changes

<!--
Are there any minor or drive-by changes also included?
-->

### Related issues

Fixes hyperlane-xyz#4059

### Backward compatibility

<!--
Are these changes backward compatible? Are there any infrastructure
implications, e.g. changes that would prohibit deploying older commits
using this infra tooling?

Yes/No
-->

### Testing

Updated unit tests to return a response so the handler is not blocked
tiendn pushed a commit to tiendn/hyperlane-monorepo that referenced this pull request Jan 11, 2025
hyperlane-xyz#5043)

### Description

Add a mpsc channel between relayer and message retry handler, so message
retry handler knows the retry results from the relayer.

`POST /message_retry` endpoint now returns a JSON with details on the
retry request:

```rust
#[derive(Clone, Debug, Deserialize, Serialize)]
pub struct MessageRetryResponse {
    /// ID of the retry request
    pub uuid: String,
    /// how many pending operations were processed
    pub processed: usize,
    /// how many of the pending operations matched the retry request pattern
    pub matched: u64,
}
```

### Drive-by changes

<!--
Are there any minor or drive-by changes also included?
-->

### Related issues

Fixes hyperlane-xyz#4059

### Backward compatibility

<!--
Are these changes backward compatible? Are there any infrastructure
implications, e.g. changes that would prohibit deploying older commits
using this infra tooling?

Yes/No
-->

### Testing

Updated unit tests to return a response so the handler is not blocked
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Relayer retry API: add the # of messages prioritized in the queue to the http response
3 participants