You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This can free up the memory if we want to switch the models for inference. Also semantically, it doesn't make sense to continue occupying the memory after the model is unregistered
A real application is that if we want to inference the base model -> do post training -> inference the finetuned model. If we can not free up the memory occupied by the base model with unregister command, it's easy to be OOMed.
Other thoughts
No response
The text was updated successfully, but these errors were encountered:
SLR722
changed the title
Free-up GPU memory after unregistering model in meta reference inference
Free up GPU memory after unregistering model in meta reference inference
Jan 15, 2025
🚀 Describe the new functionality needed
as title
💡 Why is this needed? What if we don't build it?
This can free up the memory if we want to switch the models for inference. Also semantically, it doesn't make sense to continue occupying the memory after the model is unregistered
A real application is that if we want to inference the base model -> do post training -> inference the finetuned model. If we can not free up the memory occupied by the base model with unregister command, it's easy to be OOMed.
Other thoughts
No response
The text was updated successfully, but these errors were encountered: