Hi,
We’ve been testing Concrete ML and TFHE-rs for research purposes. We would love to create a small project around it to test its capabilities. One of our ideas is to use Object Detection to solve the famous puzzle “Where is Wally?”. Is it possible to use Object Detection from Tensor Flow? Or how can we adapt these models to Concrete ML?
Thanks in advance!
TF Object Detection algorithms such as EfficientDet, Faster-RCNN, SSD and RFCN usually have a rather complex architecture, and are based on backbones such as MobileNet, ResNet and EfficientNet.
Currently Concrete-ML was not tested with these backbones, in a first step a quantization aware training (QAT) implementation of these backbones would be necessary. Second the object detection specific layers (box regressors) are usually harder to quantize, which adds to the difficulty.
I would suggest you look at converting a one-stage anchor-free detection model, such as FCOS, to QAT using Brevitas. If you can get it to train it should work in Concrete-ML. We strongly encourage the use of Brevitas (built upon Pytorch) for QAT.