Skip to content
This repository has been archived by the owner on Aug 3, 2021. It is now read-only.

Deep speech 2 training time #541

Open
anhtuanluu opened this issue Jun 16, 2020 · 0 comments
Open

Deep speech 2 training time #541

anhtuanluu opened this issue Jun 16, 2020 · 0 comments

Comments

@anhtuanluu
Copy link

I used my data abt 1000h speech and trained on 8x Tesla V100-16GB using horovod and mixed precision. With batch_size_per_gpu equals to 32, the training time is abt 3.2s per step, it took 3.5h for 1 epoch. Is it expected? Can I reduce the training time?
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@anhtuanluu