# Differences in Performance and Accuracy between Concrete Numpy and FHE in Concrete Library

Hello everyone,

I’ve been experimenting with the Concrete library for a project that involves secure computation. Initially, I opted for Concrete Numpy (import concrete.numpy as cnp) due to its compatibility with my requirements, especially for executing basic logical operations (AND, XOR, etc.) on binary arrays. However, when I switched to using the core FHE functionalities of the Concrete library (from concrete import fhe), simply substituting cnp operations with fhe, I noticed significant differences in both execution time and accuracy of the results.

Execution Time: With Concrete Numpy, the computation time for each array ranged between 30 to 40 seconds. Surprisingly, when performing the same logic using the core FHE functionalities, the time reduced dramatically to approximately 1.5 seconds per array.
Accuracy of Results: While the results obtained from Concrete Numpy were accurate, I encountered inaccuracies in some of the outcomes derived using the FHE operations.
Could someone help clarify if there’s a misunderstanding on my part regarding the use of these two components of the Concrete library? Is there a specific reason behind these discrepancies in performance and accuracy between Concrete Numpy and the core FHE operations?

Could you share your function so we can investigate more?

Thanks!

Thanks, @umutsahin, for your prompt response. Here is the code snippet where I’ve been using Concrete Numpy:

import concrete.numpy as cnp
#from concrete import fhe
import numpy as np
import csv
import time

def _PSI_impl(g_key, given_key):
intersect = np.bitwise_xor(g_key, given_key)
print(“Encrypted Intersection is :”, intersect)
if_match = (np.sum(np.bitwise_xor(g_key, given_key))==0)
is_sub_match = (np.sum(np.bitwise_and(g_key, given_key)))
if_sub_match = np.logical_xor(is_sub_match, 0)
if_no_match = (np.sum(np.bitwise_xor(g_key, given_key))==32)
match_status = if_no_match * 2 + (1 + if_sub_match) - if_match

``````return match_status
``````

class HomomorphicPSIS:
_PSI_circuit: cnp.Circuit

``````      def __init__(self):
configuration = cnp.Configuration(
enable_unsafe_features=True,
use_insecure_key_cache=True,
insecure_key_cache_location=".keys",
)

PSI_compiler = cnp.Compiler(
_PSI_impl,
{"g_key": "encrypted", "given_key": "encrypted"}
)

inputset_PSI = [
(
np.ones(32, dtype=np.int32),
np.ones(32, dtype=np.int32),
)
]

print("Compiling PSI circuit...")
start = time.time()
self._PSI_circuit = PSI_compiler.compile(inputset_PSI, configuration)
end = time.time()
print(f"(took {end - start:.3f} seconds)")

with open(csv_file_path, 'r') as csv_file:

def PSI(self, g_key, given_key):

print("\nProcessing keys:", g_key, given_key)
start = time.time()
encrypted_keys = self._PSI_circuit.encrypt(g_key, given_key)
result_keys = self._PSI_circuit.run(encrypted_keys)
end = time.time()
print(f"Encryption (took {end - start:.3f} seconds)")
start = time.time()
decrypted_result = self._PSI_circuit.decrypt(result_keys)
end = time.time()
print(f"Decryption (took {end - start:.3f} seconds)")
print("Decrypted results for intersecting keys:", decrypted_result)
return decrypted_result
``````

if name == “main”:
server = HomomorphicPSIS()
decrypted_results_array =

``````for i in range(5):
start = time.time()
end = time.time()
print(f"Load g_key from documet{i} (took {end - start:.3f} seconds)")

start = time.time()
end = time.time()
print(f"Load given key from document (took {end - start:.3f} seconds)")

print("    g_key is:", g_key)
print("given_key is:", given_key)

start = time.time()
print("encrypt and serialize data")
decrypted_result = server.PSI(g_key, given_key)
decrypted_results_array.append(decrypted_result)
end = time.time()
print(f"Pair encryption and serialization (took {end - start:.3f} seconds)")

# Iterate over intersection_array with document index
documents_with_full_match = []
documents_with_sub_match = []
documents_without_match = []
for i, intersection_result in enumerate(decrypted_results_array):
if intersection_result == 1:
documents_with_full_match.append(i)
elif intersection_result == 2:
documents_with_sub_match.append(i)
elif intersection_result == 3:
documents_without_match.append(i)

# Print the lists to see the categorized documents
print("Documents with full match:", documents_with_full_match)
print("Documents with sub match:", documents_with_sub_match)
print("Documents without match:", documents_without_match)
``````

This function is designed to compute the encrypted intersection of two sets represented by `g_key` and `given_key` , and determine the match status based on logical operations. Additionally, this script has been executed as described previously, utilizing both Concrete Numpy(cnp) and Concrete FHE(fhe), with noted differences in performance and accuracy outcomes.

Thank you in advance for your help and insights on this matter.

1 Like

Okay, so here is the issue:

• Your inputset only contains ones.
• Which makes `np.sum(np.bitwise_xor(g_key, given_key))` always 0.
• So it’s assigned `uint1`.
• But in the runtime, if you give ones for `g_key` and zeros for `given_key`, it results in a value which wouldn’t fit to `uint1`.
• So it overflows.

What you can do is to improve your inputset, and use hinting!

The idea is:

``````sum_xor = np.sum(g_key ^ given_key)
fhe.hint(sum_xor, can_store=g_key.size)
return sum_xor == 0
``````

or

``````return fhe.hint(np.sum(g_key ^ given_key), can_store=g_key.size) == 0
``````

You need to hint all intermediate values appropriately, or create an inputset which would capture the extreme values of all intermediate values.

I’d recommend hinting as it’s easy to do for your use case.

Hope this helps!

1 Like

Thank you, @umutsahin, for your suggested solution, which resolved the issue I was facing with incorrect results for some inputs. However, I have two follow-up questions for a better understanding:

1. Why didn’t the overflow issue occur when the same code was executed with Concrete Numpy and I always got accurate results?

2. Also, why is Concrete (fhe modules) noticeably faster than Concrete Numpy? My limited understanding, shaped by discussions within the community, hints that it might be due to how Numpy uses the fhe compiler.
Your answer to these questions will help me understand which of them is more compatible with me. Initially, I thought Concrete Numpy was better based on my requirements, but now I’m a bit confused about the abilities and the reasons we should select one over the other.

I need to see the graphs produced by `cnp` and `fhe` to give an accurate answer, but it could be due to the fact that bitwise operations in `cnp` not being multi precision, or `cnp` itself being single precision.