Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add latency stats around cluster config file operations #1534

Merged
merged 2 commits into from
Jan 11, 2025

Conversation

enjoy-binbin
Copy link
Member

When the cluster changes, we need to persist the cluster configuration,
and these file IO operations may cause latency.

Copy link

codecov bot commented Jan 9, 2025

Codecov Report

Attention: Patch coverage is 84.61538% with 4 lines in your changes missing coverage. Please review.

Project coverage is 70.95%. Comparing base (b207b42) to head (8efc449).
Report is 7 commits behind head on unstable.

Files with missing lines Patch % Lines
src/cluster_legacy.c 84.61% 4 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##           unstable    #1534      +/-   ##
============================================
+ Coverage     70.92%   70.95%   +0.03%     
============================================
  Files           120      120              
  Lines         65004    65055      +51     
============================================
+ Hits          46104    46162      +58     
+ Misses        18900    18893       -7     
Files with missing lines Coverage Δ
src/cluster_legacy.c 87.14% <84.61%> (+0.44%) ⬆️

... and 16 files with indirect coverage changes

When the cluster changes, we need to persist the cluster configuration,
and these file IO operations may cause latency.

Signed-off-by: Binbin <[email protected]>
@enjoy-binbin enjoy-binbin force-pushed the cluster_config_latency branch from 33047ce to 523f1e3 Compare January 9, 2025 04:39
@madolson madolson added the release-notes This issue should get a line item in the release notes label Jan 9, 2025
@hpatro hpatro changed the title Add latency around cluster config file save Add latency stats around cluster config file operations Jan 9, 2025
Copy link
Collaborator

@hpatro hpatro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we also add some sanity tests?

@enjoy-binbin
Copy link
Member Author

start_cluster 1 1 {tags {"latency-monitor cluster external:skip needs:latency"} overrides {latency-monitor-threshold 1}} {
    test "Cluster config file latency" {
        puts [r latency latest]
    }
}

i can get this ouput in my local mac, but it is not certain that they will show in CI. Do you have a better idea?

{command 1736483355 1 1} {cluster-config-fsync 1736483355 6 6} {cluster-config-rename 1736483355 1 1} {cluster-config-open 1736483355 1 1} {cluster-config-close 1736483355 1 1} {fork 1736483355 1 1} {cluster-config-dir-fsync 1736483355 5 5}
[ok]: Cluster config file latency (1 ms)

@hpatro
Copy link
Collaborator

hpatro commented Jan 10, 2025

i can get this ouput in my local mac, but it is not certain that they will show in CI. Do you have a better idea?

Didn't follow. Could you explain why it won't show up in CI?

@enjoy-binbin
Copy link
Member Author

Didn't follow. Could you explain why it won't show up in CI?

If it is 0ms, it will not show up in the tests.

@enjoy-binbin enjoy-binbin force-pushed the cluster_config_latency branch from 5e6bda7 to 671a100 Compare January 11, 2025 03:01
Signed-off-by: Binbin <[email protected]>
@enjoy-binbin enjoy-binbin force-pushed the cluster_config_latency branch from 671a100 to 8efc449 Compare January 11, 2025 03:01
@enjoy-binbin enjoy-binbin merged commit 11cb8ee into valkey-io:unstable Jan 11, 2025
47 checks passed
@enjoy-binbin enjoy-binbin deleted the cluster_config_latency branch January 11, 2025 03:03
proost pushed a commit to proost/valkey that referenced this pull request Jan 17, 2025
When the cluster changes, we need to persist the cluster configuration,
and these file IO operations may cause latency.

Signed-off-by: Binbin <[email protected]>
Signed-off-by: proost <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-notes This issue should get a line item in the release notes
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

4 participants