Why is the inference speed for CIFAR-10 suddenly got much faster?

Zama Concrete ML
1

Hi,

I realized that the inference speed got around 40 seconds not, which is more than 3 times faster than before:

https://github.com/zama-ai/concrete-ml/blob/main/use_case_examples/cifar/cifar_brevitas_training/README.md

We ran the FHE inference over 10 examples and achieved 100% similar predictions between the simulation and FHE. The overall accuracy for the entire data-set is expected to match the simulation. The original model (no rounding) with a maximum of 13 bits of precision runs in around 9 hours on the specified hardware. Using the rounding approach, the final model ran in 40 seconds . This significant performance improvement demonstrates the benefits of the rounding operator in the FHE setting.

I wonder how you achieved this great speed up. I think something changed in the circuit. How did the circuit achieve this much speedup?

2

Hello @Ronny_Ko,
Great to see that you have noticed this improvement !

The short answer is: we used a new feature from Concrete called approximate rounding. We are planning on publishing some blog posts in the next few days that will explain exactly this. I will therefore ping you when there will become available :wink:

1 Like
3

Hello again,
We have released some blog posts for our recent release where we mention this new feature :

If you have more questions about this, don’t hesitate !