[BUG] build_disk_index does not respect memory limit

[gopalrs]

Expected Behavior

build_disk_index should not exceed the memory limit set in the command line.

Actual Behavior

It consumes all available memory instead.

Dataset Description

All datasets

Error

Runs out of memory occasionally

Your Environment

• Operating system (Windows 11 Pro)
• DiskANN version: main

Additional Details

Most likely it is because tcmalloc isn't releasing memory freed by DiskANN back to the OS.

[waxd0000]

Hi, I was wondering if this issue has been resolved or if the root cause has been identified? If possible, could you share any updates or insights? Thank you for your time!