-
Notifications
You must be signed in to change notification settings - Fork 1k
feature matrix
Stella Biderman edited this page Feb 7, 2021
·
2 revisions
| GPT-NeoX | NVIDIA Megatron | DeepSpeed Megatron | |
|
|
|
|
|
| model parallel | ? | ? | ? |
| data parallel | y | ? | ? |
| pipeline parallel | y | ? | ? |
| other optimizations | ZeRO | ? | ? |
| benchmarks |