We have an API access to compile and test (in FHE)and do keygen ,encrypt and decrypt ,Can we also get access to the low level implementations of the functions,so that we see the process them ,how things are getting encrypted ,if at all we can compute some homomorphic function of cipher text .
Concrete ML is built on top of the Concrete stack which compiles an intermediary representation of FHE programs to machine code. Thus the encrypt, decrypt functions of Concrete ML models call call compiled code so you won’t be able to see the code in human readable form.
If you want to take a look at the code corresponding to such functions I would suggest you look at the Concrete source code, more precisely at the concrete-cpu backend.
I am pasting some code excerpts from the script(CNN on MNIST)(https://github.com/zama-ai/concrete-ml/blob/release/1.1.x/docs/advanced_examples/ConvolutionalNeuralNetwork.ipynb)
q_module_fhe = compile_brevitas_qat_model(
net,
x_train,
)
once we have created the quantized module using brevitas
we use,
q_module.fhe_circuit.keygen()( where do i see the details of the function fhe_circuit.keygen())
(Does this generate public ,private evaluation keys?)
# Run inference in FHE on a single encrypted example
mini_test_dataset = TensorDataset(torch.Tensor(x_test[[0], :]), torch.Tensor(y_test[[0]]))
mini_test_dataloader = DataLoader(mini_test_dataset)
t = time.time()
test_with_concrete(
q_module_fhe,
mini_test_dataloader,
use_sim=False,
)
print(f"Time per inference in FHE: {(time.time() - t) / len(mini_test_dataset):.2f}")
How is the example encrypted it is loading a single example from Tensor dataset ,creating a dataloader(torch.Tensor(x_test[[0], :]), torch.Tensor(y_test[[0]] I suppose this this is not a encrypted ciphertext and a label)
,It is not clear how is cipher text produced with the key( can i get the definition of all intermediate functions involved)
Do we have the defn of test_with_concrete?
Are we using lwe cipher texts?
If we produce image cipher texts can he apply custom filters over it homomorphically?
- q_module.fhe_circuit.keygen()( where do i see the details of the function fhe_circuit.keygen())
(Does this generate public ,private evaluation keys?)
This function produces the secret key (for encryption and decryption) and the circuit evaluation key which is public and is shared with the server.
- How is the example encrypted it is loading a single example from Tensor dataset ,creating a dataloader(torch.Tensor(x_test[[0], :]), torch.Tensor(y_test[[0]] I suppose this this is not a encrypted ciphertext and a label)
In test_with_concrete that function you will see a call to y_pred = quantized_module.forward(data, fhe=fhe_mode). If you step into that function you’ll see it calls Concrete Python encrypt_run_decrypt which encrypts, runs in FHE and decrypts the result
- It is not clear how is cipher text produced with the key( can i get the definition of all intermediate functions involved)
As suggested before, please look at the Concrete compiler source code
-
Yes we are using LWE ciphertexts
-
Yes, you can apply custom filters on images homomorphically, we show a demo of that here: Image Filtering On Encrypted Data Using Fully Homomorphic Encryption - a Hugging Face Space by zama-fhe with full source code
I was thinking in the client -server setting where the client sends (encrypted image ,encrypted labels) and then we evaluate using compiled FHE(Quantized model) the cipher text and label ,can I execute all this process
distinctively(If so which separate methods should I use)( I believe the FHE compiling of a quantized aware trained model uses defns in concrete ,is that so?)
There is an example for encrypting/decrypting separately from the computation step in this documentation section.
I had some more doubts regarding in which space does the encrypted inputs lie before we serialize
encrypted_input = fhemodel_client.quantize_encrypt_serialize(clear_input)
can we do programmable bootstrapping(separately) ( for the unserialized encrypted inputs)?
Can we deserialization explicitly from the encrypted_input ?
One more thing in the client server setting how does the client send the description of the model(how does he hide from the server) from which he wants encrypted inference?
Also in the encrypted image filtering I have a confusion are you are (server)sending the encrypted filters and after decrypting the client is applying filters or the server is homomorphically computing the conv filters over encrypted images and then sending the results?
Serialization is simply writing a datastructure in memory to a file. It is not related to encryption. We simply use it to store encrypted data and keys in order to implement the client/server protocol.
In
encrypted_input = fhemodel_client.quantize_encrypt_serialize(clear_input)
before “serialization”, the data is encrypted. Thus encrypted data is serialized.
Deserialization can be done on encrypted_input to load encrypted data from a file. Deserialization will return encrypted data in this case.
The model, and thus the weights, are not encrypted. The server has the model in the clear. Thus the client does not need to send the model to the server. The filters in encrypted image filtering are not encrypted, only the images are encrypted and the filtering is done homomorphically.
clear_input = X_client[[-1], :]
clear_input = clear_input.reshape(1, 1, 8, 8)
encrypted_input = fhemodel_client.quantize_encrypt_serialize(clear_input)
encrypted_input(b’\x01\x06\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x08\x00\x00\x00\x00\x00\x00\x00\x08\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x01\x08\x00\x00\x00\x00\x00\x00@\x00\x00\x00\x00\x00\x00\x00\x009W\xfe\xd1\x9f@\xad\xa2y\x0b\xe5S\x0b\xc0\xc0\x89x&\xcf\x9aby\xbf\xf8\xd4?\xc6W+s\xc1.d\xd1Y\xdd\x…so on ) This is serialized output i got then i tried,
deserialized_input = Value.deserialize(encrypted_input)
Print the deserialized input:
print(deserialized_input)
output(<main.Value object at 0x7bed174f7e50>) ,is this the lwe cipher text encoding ?
one more thing you already explained about serialization beautifully,
Is addition defined in concrete ML for cipher texts also be used for encrypted serialized inputs that means suppose i have two encrypted serialized inputs a and b ,is a+b a valid serialized cipher text and pbs(a) a valid operation?
actually i tried adding two encrypted serialized inputs for two images using a network with capacity 1216 MAC
but i got
IOPub data rate exceeded.
The notebook server will temporarily stop sending output
to the client in order to avoid crashing it.
To change this limit, set the config variable
--NotebookApp.iopub_data_rate_limit.
Current values:
NotebookApp.iopub_data_rate_limit=1000000.0 (bytes/sec)
NotebookApp.rate_limit_window=3.0 (secs)
It looks like you printed a serialized LWE encrypted value which I think is what you wanted.
If you want to perform addition of ciphertexts you need a compiled function to do it. Please see the this Getting Started page
No its not about printing the cipher text ,if you can explain me what is this value object once we print a cipher text ? what can we retrieve from this object and how?
Hello @Rish ,
Could you elaborate a bit more on your last questions ? It is not very clear for us what your are trying to achieve ! 
Just trying to learn the basics and seeking assistance in the process, I hope I am not annoying with basic questions which might sound trivial ,and yes thankyou to all the team for helping me till now to ease out my journey with concrete ML
print(deserialized_input)
output(<main .Value object at 0x7bed174f7e50>)
.Value object at 0x7bed174f7e50>,what is this value object ? please point towards any class defn or so if there
Hi @Rish,
Let’s track it together!
- It’s defined here Value
- But it’s just a wrapper around this compiler class NativeValue
- Which is just another wrapper around this class SharedScalarOrTensorData which comes from pybind11
- And that itself wraps this class ScalarOrTensorData which can be either ScalarData or TensorData
- ScalarData and TensorData are just containers for to store an integer or a tensor of integers.
So, fhe.Value is just a bunch of integers 
Hope this helps!
1 Like
Can someone explain how can we build python bindings for low level primitive operations using some simple example
In addition can you elaborate on this : _server_lambda function