Singel machine vs multi-machine validation accuracy

[sakh251]

Hi,

I ran the code on single machine with 3 gpus and 2 machines with 6 gpus (each 3).
The validation accuracy of single machine is 3 percent more than multi-machine.
So I can not reproduce the reported results.

Is there any clue to found out the problem.