Concrete-python installation on raspberry pi

Zama Concrete
42

I used to encounter this error when libConcretelangRuntime.so was linked to my system llvm library rather than the one we compile. If you compiled the bindings from the compiler Makefile, the -DLLVM_LINK_LLVM_DYLIB=on line should fix that for you.

Are you building on main ? Are you using the makefile to compile the python bindings ?

43

Yes on main. I am using the makefile and checked in it and has -DLLVM_LINK_LLVM_DYLIB=on \ set

44

I think I need to start again and make sure I am going through the correct process. Goin to take a break and will try again tomorrow. If you have an suggestions in the mean time all this help is greatly appreciated :slight_smile: From a high level what is the sequence I should go through from the very beginning?

45

I have tried to start from the start and have switched to Ubuntu on raspberry pi as opposed to Raspbian. When I run pytest I get a different error but I believe that can be ignored?. The issue is now when I try to install the wheel and the test program I get an LLVM dump. Any suggestions on what I could try next?

Here are the steps I go through for a build:

# Ubuntu Server running on raspberry pi 5 with 8GB memory
# Ubuntu 23.10 (GNU/Linux 6.5.0-1013-raspi aarch64)
# Recommend ubuntu is installed and booted from USB 3 SSD or PICe SSD as the SD card will be very slow. 

# Make sure all packages are up to date. 

sudo apt update
sudo apt upgrade

# Reboot after this may be needed depending on packages updated. 

# Setup Swap as test workers will crash due to not enough memory.
sudo dd if=/dev/zero of=/mnt/8GiB.swap bs=1024 count=8388608
sudo chmod 600 /mnt/8GiB.swap
sudo mkswap /mnt/8GiB.swap
sudo swapon /mnt/8GiB.swap
echo '/mnt/8GiB.swap swap swap defaults 0 0' | sudo tee -a /etc/fstab

# Install packages (screen install optional so can easily detach if removing into Ubuntu)

sudo apt install cmake screen build-essential ninja-build python3-dev ccache python3-pip git python3-venv libboost-all-dev libgoogle-perftools-dev patchelf graphviz graphviz-dev valgrind libssl-dev python-is-python3


# create link for python to python3 so Makefile does not need to be updated to use python3. (Not needed if python-is-python3 is installed).
#sudo ln -s /usr/bin/python3 /usr/bin/python

# install rust - Accept the default.

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh


# Logout and log back in again and then get the nightly toolchain
rustup toolchain install nightly-2024-01-31

# may need this if get the error again
# rustup target add aarch64-unknown-linux-gnu --toolchain nightly-2024-01-31


mkdir newbuild
cd new build

git clone https://github.com/zama-ai/concrete.git --recursive
cd concrete

cd frontends/concrete-python
make venv
source .venv/bin/activate
cd ../../compilers/concrete-compiler/compiler


# see if this fixes the dataflow error
export DATAFLOW_EXECUTION_ENABLED=ON


# Run ā€œmake python-bindingsā€ in background with nohup as can take a few hours to compile, output is stored in nohup.out, tail -f to see progress I.e ā€œtail -f nohup.outā€
nohup make python-bindings &

# once complete do the following:

export COMPILER_BUILD_DIRECTORY=$(pwd)/build
echo "export COMPILER_BUILD_DIRECTORY=$(pwd)/build" >> ~/.bashrc

cd ../../../frontends/concrete-python

#edit the Makefile and change the following so number of threads is 1
#        pytest tests -svv -n 1 \


make pytest 

#Edit the Makefile and change from x86_64 to aarch64 so reads ā€œfor PLATFORM in manylinux_$${GLIBC_VER}_aarch64 linux_aarch64; do \ā€

make whl

Here is the failed test:

=================================== FAILURES ===================================
______________________ tests/compilation/test_circuit.py _______________________
[gw0] linux ā€“ Python 3.11.6 /home/pi/newbuild/concrete/frontends/concrete-python/.venv/bin/python
worker ā€˜gw0ā€™ crashed while running ā€˜tests/compilation/test_circuit.py::test_dataflow_circuitā€™
=============================== warnings summary ===============================
ā€¦/ā€¦/compilers/concrete-compiler/compiler/build/tools/concretelang/python_packages/concretelang_core/concrete/init.py:1
ā€¦/ā€¦/compilers/concrete-compiler/compiler/build/tools/concretelang/python_packages/concretelang_core/concrete/init.py:1
ā€¦/ā€¦/compilers/concrete-compiler/compiler/build/tools/concretelang/python_packages/concretelang_core/concrete/init.py:1
/home/pi/newbuild/concrete/compilers/concrete-compiler/compiler/build/tools/concretelang/python_packages/concretelang_core/concrete/init.py:1: DeprecationWarning: pkg_resources is deprecated as an API. See Package Discovery and Resource Access using pkg_resources - setuptools 69.2.0.post20240313 documentation
import(ā€œpkg_resourcesā€).declare_namespace(name)

ā€¦/ā€¦/compilers/concrete-compiler/compiler/build/tools/concretelang/python_packages/concretelang_core/concrete/init.py:1
ā€¦/ā€¦/compilers/concrete-compiler/compiler/build/tools/concretelang/python_packages/concretelang_core/concrete/init.py:1
ā€¦/ā€¦/compilers/concrete-compiler/compiler/build/tools/concretelang/python_packages/concretelang_core/concrete/init.py:1
/home/pi/newbuild/concrete/compilers/concrete-compiler/compiler/build/tools/concretelang/python_packages/concretelang_core/concrete/init.py:1: DeprecationWarning: Deprecated call to pkg_resources.declare_namespace('concrete').
Implementing implicit namespace packages (as specified in PEP 420) is preferred to pkg_resources.declare_namespace. See Keywords - setuptools 69.2.0.post20240313 documentation
import(ā€œpkg_resourcesā€).declare_namespace(name)

concrete/init.py:7
concrete/init.py:7
concrete/init.py:7
/home/pi/newbuild/concrete/frontends/concrete-python/concrete/init.py:7: DeprecationWarning: pkg_resources is deprecated as an API. See Package Discovery and Resource Access using pkg_resources - setuptools 69.2.0.post20240313 documentation
import(ā€œpkg_resourcesā€).declare_namespace(name) # pragma: no cover

concrete/init.py:7
concrete/init.py:7
concrete/init.py:7
/home/pi/newbuild/concrete/frontends/concrete-python/concrete/init.py:7: DeprecationWarning: Deprecated call to pkg_resources.declare_namespace('concrete').
Implementing implicit namespace packages (as specified in PEP 420) is preferred to pkg_resources.declare_namespace. See Keywords - setuptools 69.2.0.post20240313 documentation
import(ā€œpkg_resourcesā€).declare_namespace(name) # pragma: no cover

ā€“ Docs: How to capture warnings ā€” pytest documentation

---------- coverage: platform linux, python 3.11.6-final-0 -----------
---------------------- coverage: failed workers ----------------------
The following workers failed to return coverage data, ensure that pytest-cov is installed on these workers.
gw0
Name Stmts Miss Cover Missing

concrete/fhe/compilation/artifacts.py 108 2 98% 295, 302
concrete/fhe/compilation/circuit.py 155 1 99% 202
concrete/fhe/compilation/client.py 59 1 98% 99
concrete/fhe/compilation/compiler.py 226 45 80% 188-189, 360-383, 507-517, 532-537, 544-549, 556-561, 568-573, 580-581, 596-597, 603-628
concrete/fhe/compilation/configuration.py 405 70 83% 39-40, 44-48, 64-74, 258-271, 575-588, 736-752, 787-801, 859-872, 1259-1265, 1275-1281, 1285-1286, 1289-1290
concrete/fhe/compilation/decorators.py 59 3 95% 49, 59-64
concrete/fhe/compilation/keys.py 65 32 51% 52, 96-103, 114-128, 147-154, 167-172, 188-195
concrete/fhe/compilation/module.py 182 25 86% 104, 107, 121-152, 222, 540-546, 568, 575-577, 584-585, 603-604, 610, 681
concrete/fhe/compilation/module_compiler.py 235 42 82% 59-71, 79-91, 115-117, 133, 162-168, 179-194, 197, 200, 205-210, 232-233, 497, 499-500, 524-525, 568-571
concrete/fhe/compilation/server.py 231 25 89% 130, 201, 253-265, 304-310, 744-783
concrete/fhe/compilation/utils.py 288 13 95% 59-62, 846-859
concrete/fhe/dtypes/integer.py 55 5 91% 95-100, 106, 150
concrete/fhe/dtypes/utils.py 38 18 53% 36, 41-67
concrete/fhe/extensions/multivariate.py 35 3 91% 51, 94-95
concrete/fhe/extensions/ones.py 16 1 94% 56
concrete/fhe/extensions/relu.py 16 9 44% 32-48
concrete/fhe/extensions/round_bit_pattern.py 111 8 93% 132, 147-155
concrete/fhe/extensions/table.py 67 12 82% 43, 56-57, 67-68, 74-79, 123-124, 130-131
concrete/fhe/extensions/univariate.py 27 4 85% 68-75
concrete/fhe/mlir/context.py 1394 137 90% 1035-1096, 1207-1226, 1302-1321, 1529, 1939-1949, 1953, 1961, 2104, 2107-2124, 2267, 2324, 2335, 2381, 2430-2431, 2453-2477, 2877, 2886, 2927-2930, 2939-2942, 2952-2960, 2985-3038, 3066, 3121, 3144-3164, 3293, 3640
concrete/fhe/mlir/converter.py 406 50 88% 139-146, 166-204, 504-505, 508-509, 733-737, 781-782
concrete/fhe/mlir/processors/assign_bit_widths.py 245 8 97% 338-350
concrete/fhe/mlir/utils.py 146 25 83% 43, 96-139, 162
concrete/fhe/representation/graph.py 308 2 99% 392, 1029
concrete/fhe/representation/node.py 165 29 82% 178, 200-203, 206-207, 212-215, 218-224, 237-240, 243-248, 257-262, 265-270, 301-303, 316-319, 362, 365
concrete/fhe/representation/utils.py 33 3 91% 100, 133-134
concrete/fhe/tracing/tracer.py 388 48 88% 187, 192-193, 378-379, 384-387, 433-437, 459-460, 474-476, 557-560, 630-647, 650-654, 658, 708, 715, 725, 781-785, 788-790, 795-810, 813-814, 817-826
concrete/fhe/values/value_description.py 60 5 92% 61, 66-69, 95, 110

TOTAL 6680 626 91%

33 files skipped due to complete coverage.

FAIL Required test coverage of 95% not reached. Total coverage: 90.63%
=========================== short test summary info ============================
FAILED tests/compilation/test_circuit.py::test_dataflow_circuit
========== 1 failed, 2241 passed, 12 warnings in 20180.25s (5:36:20) ===========
make: *** [Makefile:45: pytest-default] Error 1

Here is the the new error when running the test script

(dave) pi@pi-fhe:~$ python test.py
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var LLVM_SYMBOLIZER_PATH to point to it):
0 libLLVM-17git-2a4f05ac.so 0x0000ffffa4ef0bd8 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 72
1 libLLVM-17git-2a4f05ac.so 0x0000ffffa4eee560 llvm::sys::RunSignalHandlers() + 64
2 libLLVM-17git-2a4f05ac.so 0x0000ffffa4eee6b8
3 linux-vdso.so.1 0x0000ffffaf0a47e0 __kernel_rt_sigreturn + 0
4 libstdc++.so.6 0x0000ffff8f18ffb0
5 libstdc++.so.6 0x0000ffff8f19213c
6 libstdc++.so.6 0x0000ffff8f192290 std::codecvt<char16_t, char, __mbstate_t>::do_in(__mbstate_t&, char const*, char const*, char const*&, char16_t*, char16_t*, char16_t*&) const + 80
7 libstdc++.so.6 0x0000ffff8f2011d0 std::ostream& std::ostream::_M_insert(unsigned long) + 176
8 libtorch_cpu.so 0x0000ffff83d13154 torch::Library::_fallback(torch::CppFunction&&) & + 1824
9 libtorch_cpu.so 0x0000ffff83a88194
10 libtorch_cpu.so 0x0000ffff83a8bdd8
11 libtorch_cpu.so 0x0000ffff83a29eb4
12 ld-linux-aarch64.so.1 0x0000ffffaf06b7e8
13 ld-linux-aarch64.so.1 0x0000ffffaf06b8f4
14 ld-linux-aarch64.so.1 0x0000ffffaf068370 _dl_catch_exception + 208
15 ld-linux-aarch64.so.1 0x0000ffffaf07195c
16 ld-linux-aarch64.so.1 0x0000ffffaf0682fc _dl_catch_exception + 92
17 ld-linux-aarch64.so.1 0x0000ffffaf071ccc
18 libc.so.6 0x0000ffffaedef6d4
19 ld-linux-aarch64.so.1 0x0000ffffaf0682fc _dl_catch_exception + 92
20 ld-linux-aarch64.so.1 0x0000ffffaf06841c
21 libc.so.6 0x0000ffffaedef154
22 libc.so.6 0x0000ffffaedef7b0 dlopen + 144
23 python 0x00000000005e7634
24 python 0x00000000005e5318
25 python 0x00000000004d36d4
26 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
27 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
28 python 0x00000000004ca1e8
29 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
30 python 0x000000000050ae60 PyImport_ImportModuleLevelObject + 2124
31 python 0x00000000004b23fc _PyEval_EvalFrameDefault + 21964
32 python 0x00000000005a4b74 PyEval_EvalCode + 180
33 python 0x00000000005c34f8
34 python 0x00000000004be1b0
35 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
36 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
37 python 0x00000000004ca1e8
38 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
39 python 0x000000000050ae60 PyImport_ImportModuleLevelObject + 2124
40 python 0x00000000004b23fc _PyEval_EvalFrameDefault + 21964
41 python 0x00000000005a4b74 PyEval_EvalCode + 180
42 python 0x00000000005c34f8
43 python 0x00000000004be1b0
44 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
45 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
46 python 0x00000000004ca1e8
47 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
48 python 0x000000000050ae60 PyImport_ImportModuleLevelObject + 2124
49 python 0x00000000004b23fc _PyEval_EvalFrameDefault + 21964
50 python 0x00000000005a4b74 PyEval_EvalCode + 180
51 python 0x00000000005c34f8
52 python 0x00000000004be1b0
53 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
54 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
55 python 0x00000000004ca1e8
56 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
57 python 0x000000000050ae60 PyImport_ImportModuleLevelObject + 2124
58 python 0x00000000004b23fc _PyEval_EvalFrameDefault + 21964
59 python 0x00000000005a4b74 PyEval_EvalCode + 180
60 python 0x00000000005c34f8
61 python 0x00000000004be1b0
62 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
63 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
64 python 0x00000000004ca1e8
65 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
66 python 0x000000000050ae60 PyImport_ImportModuleLevelObject + 2124
67 python 0x00000000004b23fc _PyEval_EvalFrameDefault + 21964
68 python 0x00000000005a4b74 PyEval_EvalCode + 180
69 python 0x00000000005c34f8
70 python 0x00000000004be1b0
71 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
72 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
73 python 0x00000000004ca1e8
74 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
75 python 0x000000000050ae60 PyImport_ImportModuleLevelObject + 2124
76 python 0x00000000004b23fc _PyEval_EvalFrameDefault + 21964
77 python 0x00000000005a4b74 PyEval_EvalCode + 180
78 python 0x00000000005c34f8
79 python 0x00000000004be1b0
80 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
81 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
82 python 0x00000000004ca1e8
83 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
84 python 0x000000000050ae60 PyImport_ImportModuleLevelObject + 2124
85 python 0x00000000005282b8
86 python 0x00000000004be1b0
87 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
88 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
89 python 0x00000000004ca1e8
90 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
91 python 0x000000000050aed8 PyImport_ImportModuleLevelObject + 2244
92 python 0x00000000004b23fc _PyEval_EvalFrameDefault + 21964
93 python 0x00000000005a4b74 PyEval_EvalCode + 180
94 python 0x00000000005cacac
95 python 0x00000000005c6714
96 python 0x00000000005e0060
97 python 0x00000000005dfc48 _PyRun_SimpleFileObject + 392
98 python 0x00000000005df9b4 _PyRun_AnyFileObject + 84
99 python 0x00000000005dd388 Py_RunMain + 712
100 python 0x00000000005958ac Py_BytesMain + 40
101 libc.so.6 0x0000ffffaed97584
102 libc.so.6 0x0000ffffaed97658 __libc_start_main + 152
103 python 0x0000000000595730 _start + 48
PLEASE submit a bug report to Issues Ā· llvm/llvm-project Ā· GitHub and include the crash backtrace.
Segmentation fault (core dumped)

46

Is HPX installed? Itā€™s required since you have:

export DATAFLOW_EXECUTION_ENABLED=ON
47

I have tried with and without. Does HPX need to be installed if it is set to OFF?

48

It shouldnā€™t be. What happens when you set it off?

49

I think it was the same error, but I will just do it again to confirm, I have three raspberry Piā€™s sitting there with the same config trying different options . Have tried a lot of combinations but will confirm in about 3 hrs :slight_smile:

1 Like
50

Tried it with set to OFF and I still get the same issue:

(venv) pi@pi-fhe:~$ python test.py
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var LLVM_SYMBOLIZER_PATH to point to it):
0 libLLVM-17git-2a4f05ac.so 0x0000ffff89900bd8 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 72
1 libLLVM-17git-2a4f05ac.so 0x0000ffff898fe560 llvm::sys::RunSignalHandlers() + 64
2 libLLVM-17git-2a4f05ac.so 0x0000ffff898fe6b8
3 linux-vdso.so.1 0x0000ffff93aaf7e0 __kernel_rt_sigreturn + 0
4 libstdc++.so.6 0x0000ffff73b8ffb0
5 libstdc++.so.6 0x0000ffff73b9213c
6 libstdc++.so.6 0x0000ffff73b92290 std::codecvt<char16_t, char, __mbstate_t>::do_in(__mbstate_t&, char const*, char const*, char const*&, char16_t*, char16_t*, char16_t*&) const + 80
7 libstdc++.so.6 0x0000ffff73c011d0 std::ostream& std::ostream::_M_insert(unsigned long) + 176
8 libtorch_cpu.so 0x0000ffff68713154 torch::Library::_fallback(torch::CppFunction&&) & + 1824
9 libtorch_cpu.so 0x0000ffff68488194
10 libtorch_cpu.so 0x0000ffff6848bdd8
11 libtorch_cpu.so 0x0000ffff68429eb4
12 ld-linux-aarch64.so.1 0x0000ffff93a767e8
13 ld-linux-aarch64.so.1 0x0000ffff93a768f4
14 ld-linux-aarch64.so.1 0x0000ffff93a73370 _dl_catch_exception + 208
15 ld-linux-aarch64.so.1 0x0000ffff93a7c95c
16 ld-linux-aarch64.so.1 0x0000ffff93a732fc _dl_catch_exception + 92
17 ld-linux-aarch64.so.1 0x0000ffff93a7cccc
18 libc.so.6 0x0000ffff937ff6d4
19 ld-linux-aarch64.so.1 0x0000ffff93a732fc _dl_catch_exception + 92
20 ld-linux-aarch64.so.1 0x0000ffff93a7341c
21 libc.so.6 0x0000ffff937ff154
22 libc.so.6 0x0000ffff937ff7b0 dlopen + 144
23 python 0x00000000005e7634
24 python 0x00000000005e5318
25 python 0x00000000004d36d4
26 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
27 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
28 python 0x00000000004ca1e8
29 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
30 python 0x000000000050ae60 PyImport_ImportModuleLevelObject + 2124
31 python 0x00000000004b23fc _PyEval_EvalFrameDefault + 21964
32 python 0x00000000005a4b74 PyEval_EvalCode + 180
33 python 0x00000000005c34f8
34 python 0x00000000004be1b0
35 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
36 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
37 python 0x00000000004ca1e8
38 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
39 python 0x000000000050ae60 PyImport_ImportModuleLevelObject + 2124
40 python 0x00000000004b23fc _PyEval_EvalFrameDefault + 21964
41 python 0x00000000005a4b74 PyEval_EvalCode + 180
42 python 0x00000000005c34f8
43 python 0x00000000004be1b0
44 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
45 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
46 python 0x00000000004ca1e8
47 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
48 python 0x000000000050ae60 PyImport_ImportModuleLevelObject + 2124
49 python 0x00000000004b23fc _PyEval_EvalFrameDefault + 21964
50 python 0x00000000005a4b74 PyEval_EvalCode + 180
51 python 0x00000000005c34f8
52 python 0x00000000004be1b0
53 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
54 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
55 python 0x00000000004ca1e8
56 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
57 python 0x000000000050ae60 PyImport_ImportModuleLevelObject + 2124
58 python 0x00000000004b23fc _PyEval_EvalFrameDefault + 21964
59 python 0x00000000005a4b74 PyEval_EvalCode + 180
60 python 0x00000000005c34f8
61 python 0x00000000004be1b0
62 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
63 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
64 python 0x00000000004ca1e8
65 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
66 python 0x000000000050ae60 PyImport_ImportModuleLevelObject + 2124
67 python 0x00000000004b23fc _PyEval_EvalFrameDefault + 21964
68 python 0x00000000005a4b74 PyEval_EvalCode + 180
69 python 0x00000000005c34f8
70 python 0x00000000004be1b0
71 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
72 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
73 python 0x00000000004ca1e8
74 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
75 python 0x000000000050ae60 PyImport_ImportModuleLevelObject + 2124
76 python 0x00000000004b23fc _PyEval_EvalFrameDefault + 21964
77 python 0x00000000005a4b74 PyEval_EvalCode + 180
78 python 0x00000000005c34f8
79 python 0x00000000004be1b0
80 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
81 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
82 python 0x00000000004ca1e8
83 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
84 python 0x000000000050ae60 PyImport_ImportModuleLevelObject + 2124
85 python 0x00000000005282b8
86 python 0x00000000004be1b0
87 python 0x00000000004b18bc _PyEval_EvalFrameDefault + 19084
88 python 0x00000000004e1380 _PyFunction_Vectorcall + 416
89 python 0x00000000004ca1e8
90 python 0x000000000050c5d0 PyObject_CallMethodObjArgs + 256
91 python 0x000000000050aed8 PyImport_ImportModuleLevelObject + 2244
92 python 0x00000000004b23fc _PyEval_EvalFrameDefault + 21964
93 python 0x00000000005a4b74 PyEval_EvalCode + 180
94 python 0x00000000005cacac
95 python 0x00000000005c6714
96 python 0x00000000005e0060
97 python 0x00000000005dfc48 _PyRun_SimpleFileObject + 392
98 python 0x00000000005df9b4 _PyRun_AnyFileObject + 84
99 python 0x00000000005dd388 Py_RunMain + 712
100 python 0x00000000005958ac Py_BytesMain + 40
101 libc.so.6 0x0000ffff937a7584
102 libc.so.6 0x0000ffff937a7658 __libc_start_main + 152
103 python 0x0000000000595730 _start + 48
PLEASE submit a bug report to Issues Ā· llvm/llvm-project Ā· GitHub and include the crash backtrace.
Segmentation fault (core dumped)

51

Hi Dave,
Itā€™s difficult to tell what is going wrong as the backtrace doesnā€™t contain all the symbols, but I was wondering if the Pi has sufficient RAM to execute this. Itā€™s just a hunch as this seems to blow within an operation that could be trying to allocate memory.
While itā€™s not necessarily exactly the same, the memory usage of a given test should be in the same ballpark on different machines, so if this test works on another platform that you have, you could measure the high water mark (e.g., using /usr/bin/time -v) and make sure that would comfortably fit within the Piā€™s memory.

1 Like
52

Hi, I think i was gettgin further on Raspbian as opposed to Ubuntu.

I now get the following error when running the test script which seems to be much further along:

(venv) pi@pi-fhe3:~ $ python test.py 
Compiling...
Traceback (most recent call last):
  File "/home/pi/test.py", line 10, in <module>
    circuit = compiler.compile(inputset)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/venv/lib/python3.11/site-packages/concrete/fhe/compilation/compiler.py", line 521, in compile
    mlir_module = GraphConverter(self.configuration).convert(self.graph, mlir_context)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/venv/lib/python3.11/site-packages/concrete/fhe/mlir/converter.py", line 129, in convert
    return self.convert_many({name: graph}, mlir_context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/venv/lib/python3.11/site-packages/concrete/fhe/mlir/converter.py", line 60, in convert_many
    self.process(graphs)
  File "/home/pi/venv/lib/python3.11/site-packages/concrete/fhe/mlir/converter.py", line 240, in process
    processor.apply_many(graphs)
  File "/home/pi/venv/lib/python3.11/site-packages/concrete/fhe/mlir/processors/assign_bit_widths.py", line 115, in apply_many
    new_bit_width = model[bit_width].as_long()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/venv/lib/python3.11/site-packages/z3/z3.py", line 3023, in as_long
    return int(self.as_string())
           ^^^^^^^^^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: ''
53

Could you provide us with test.py content ? Is it a file from the repository, or something you implemented yourself ? Itā€™s hard to tell what goes wrong from this trace only.

54

Sorry, should have metioned it. It is the getting started file:

pi@pi-fhe3:~ $ cat test.py 
from concrete import fhe

def add(x, y):
    return x + y

compiler = fhe.Compiler(add, {"x": "encrypted", "y": "encrypted"})
inputset = [(2, 3), (0, 0), (1, 6), (7, 7), (7, 1), (3, 2), (6, 1), (1, 7), (4, 5), (5, 4)]

print(f"Compiling...")
circuit = compiler.compile(inputset)

print(f"Generating keys...")
circuit.keygen()

examples = [(3, 4), (1, 2), (7, 7), (0, 0)]
for example in examples:
    encrypted_example = circuit.encrypt(*example)
    encrypted_result = circuit.run(encrypted_example)
    result = circuit.decrypt(encrypted_result)
    print(f"Evaluation of {' + '.join(map(str, example))} homomorphically = {result}")
55

No worries @DaveCT :slight_smile: !

So I have been building the python-bindings on my computer (not a raspberry-pi) from main, and running what you sent me does not produce any error ā€¦ Do you get the same kind of errors when running the tests ?

56

Hi, the only error in the test is the orginal one (copied below), all the other tests pass:

tests/compilation/test_circuit.py::test_dataflow_circuit WARNING: The generated random seed is not crypto secure
/home/pi/newbuild3/concrete/frontends/concrete-python/.venv/bin/python: **symbol lookup error: /tmp/tmpyzoeh276/sharedlib.so: undefined symbol: _dfr_register_work_function**