Issue 5954 - Disable Transparent Huge Pages v2

[vashirov]

Bug Description:
THP can have negative effects on DS performance when large caches are
used.

Fix Description:

• Add a new variable for ns-slapd THP_DISABLE.
  When THP_DISABLE is set to 1, THP is disabled for ns-slapd process
  via prctl(2). With any other value, THP settings are untouched.

Before:

$ grep THP /proc/$(pidof ns-slapd)/status
THP_enabled:    1

After

$ grep THP /proc/$(pidof ns-slapd)/status
THP_enabled:    0

• Add a new healthcheck linter, that checks if THP is disabled system-wide
  or per instance. In case THP is enabled for both the system and the
  process, it prints recommendations how to disable THP.

Fixes: #5954

Reviewed-by: ???

[tbordaz]

Thanks LGTM
Let's see which option is preferred

[Firstyear]

My concern here is we are addressing the symptom not the problem. I think if we disable THP then we'll likely never turn it back on again, and we could be losing out on benefits. Shouldn't we pursue a fix to the underlying issue?

[vashirov]

I understand your concern. I think that the underlying issue is that THP is not applicable to certain workloads, where data access patterns are not contiguous, with a lot of small allocations and deallocations.
Some examples:
https://www.pingcap.com/blog/transparent-huge-pages-why-we-disable-it-for-databases/
https://www.mongodb.com/docs/manual/tutorial/transparent-huge-pages/
https://redis.io/docs/management/optimization/latency/#latency-induced-by-transparent-huge-pages
https://www.postgresql.org/docs/current/runtime-config-resource.html#GUC-HUGE-PAGES
https://docs.oracle.com/en/database/oracle/oracle-database/19/ladbi/disabling-transparent-hugepages.html

Every optimization is a tradeoff and comes with a cost. THP can reduce page faults, but in our case it also causes memory fragmentation, CPU spikes (khugepaged has a non-zero overhead) and latency spikes. If we can't take advantage of this optimization due to our workload and usage patterns, I see no reason to keep it enabled.

[Firstyear]

Every optimization is a tradeoff and comes with a cost. THP can reduce page faults, but in our case it also causes memory fragmentation, CPU spikes (khugepaged has a non-zero overhead) and latency spikes. If we can't take advantage of this optimization due to our workload and usage patterns, I see no reason to keep it enabled.

Okay, lets disable it then.

[vashirov]

@Firstyear, thanks!

@tbordaz, any thoughts on the healthcheck part? Do you think it's still needed?

[tbordaz]

@Firstyear, thanks!

@tbordaz, any thoughts on the healthcheck part? Do you think it's still needed?

IIUC, with the patch the server requests to disable THP for itself. So the benefit of healthcheck would be to double check that the request was successful. I think it worthes doing that check if it is not too complex to implement.

[vashirov]

I've added healthcheck linter and tests for it, please review.
Thanks!

[vashirov]

Newly added test failed, investigating.

[vashirov]

I fixed the test, now it passes. @tbordaz, please let me know if you're happy with the healthcheck changes, thanks.

[tbordaz]

I fixed the test, now it passes. @tbordaz, please let me know if you're happy with the healthcheck changes, thanks.

Yes that looks perfect to me