Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: introduce /load endpoint for self-reported quantised NymNode load #5326

Merged
merged 3 commits into from
Jan 16, 2025

Conversation

jstuczyn
Copy link
Contributor

@jstuczyn jstuczyn commented Jan 9, 2025

this PR introduces a new /load endpoint on a NymNode to return its current load.

it returns the following data:

pub struct NodeLoad {
    pub total: Load,
    pub machine: Load,
    pub network: Load,
}

where Load is quantised into the following buckets / tiers:

pub enum Load {
    Negligible, // 0 - 0.1
    Low,        // 0.1 - 0.3
    Medium,     // 0.3 - 0.6
    High,       // 0.6 - 0.8
    VeryHigh,   // 0.8 - 0.95
    AtCapacity, // >= 0.95
}

the actual values forNodeLoad are determined as follows:

  • for network we approximate current rx/tx rates on all eth interfaces and scale them to the range of 0-1Gbps (for this initial iteration I assume the maximum network speed is 1Gbps which would be treated as fully saturated). so for example, if the node is sending at 0.5Gbps, it would get a Load of 0.5 and thus value of Load::Medium, 0.9Gbps would get Load of 0.9 and value of Load::VeryHigh, etc. we take the bigger value between rx and tx
  • for machine load there's a bit more logic in there:
    • firstly we determine what I call a "base load" of the machine. we do this by taking the average load from the last 5min (via getloadavg) and dividing it by the number of CPUs in the machine. for example if the average load of the machine in the last 5min was 1.23 and it has 2 CPUs, then it's Load would be Load::High (1.23 / 2 = 0.615)
    • however, whilst CPU utilisation is one of the most important factors, it does not tell the whole story. I tried to also take memory/swap utilisation into consideration (whilst not making it the main factor)
    • thus we calculate two additional auxiliary Load values, for memory usage and swap usage, i.e.: used_memory / total_memory and used_swap / total_swap respectively.
    • then we check whether either of the MemoryLoad or SwapLoad is bigger than the current base Load of the machine we have determined, if so, it's increased by one tier / bucket. For example, say the current machine load is Load::Low, but the memory usage is at 90% (Load::VeryHigh). that would result in the reported Load being bumped up to Load::Medium instead. The same logic applies with swap load, however, only if the total swap > 1GB. this is to prevent weird edge cases where the machine has hardly any swap.
  • finally, the .total Load uses the same "tier bumping" behaviour using the .total and .network loads, i.e. if network > machine, then total = machine + 1. for example if machine Load is Load::Low, but network Load is Load::Medium, then the total Load is set to Load::Medium instead.

This change is Reviewable

Copy link

vercel bot commented Jan 9, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

2 Skipped Deployments
Name Status Preview Comments Updated (UTC)
docs-nextra ⬜️ Ignored (Inspect) Visit Preview Jan 16, 2025 2:35pm
nym-next-explorer ⬜️ Ignored (Inspect) Visit Preview Jan 16, 2025 2:35pm

@jstuczyn jstuczyn added this to the Hu milestone Jan 10, 2025
@jstuczyn jstuczyn requested a review from octol January 10, 2025 09:27
@jstuczyn jstuczyn force-pushed the feature/self-reported-load branch from 9370dd3 to 7ce120e Compare January 16, 2025 14:34
@jstuczyn jstuczyn merged commit 03c4895 into develop Jan 16, 2025
18 of 21 checks passed
@jstuczyn jstuczyn deleted the feature/self-reported-load branch January 16, 2025 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant