Can GPU be used for batched inference in Concrete ML

[Laser_beam]

from concrete.compiler import check_gpu_available,
Can we use GPU for parallel batched inference on encrypted inputs?

[andrei-stoian-zama]

Concrete ML models will executed some layers with batched GPU computation. However, the batching is done on layer breadth and Concrete ML will not batch multiple examples like PyTorch.