Query: Parallel_batches support for CNN and NLP model

[avinashcpandey]

I see WideDeep has parallel batch support where no. of input samples can be broken into N chunks and run parallely.

Eg. if we are giving parallel_batchess 28, inter=28 intra and num_threads=1 for CSL single socket.
Benchmark will launch 28 graph execution parallely and each will work on provided BS. Please correct me here.

This is the reference I followed.
https://github.com/IntelAI/models/blob/master/benchmarks/recommendation/tensorflow/wide_deep_large_ds/inference/fp32/Advanced.md

Now I want to do the same for CNN model (eg. inceptionV3)
For example Given BS is 280, and given parallel batches is 28. Then 28 graph execution will be created where each graph execution will work on 10 images.

Is this supported? If yes, how can I use it?
If No, Is it there in plan?

I want to do the same for BERT also.

[sramakintel]

@avinashcpandey do you still need help with this issue?