You can implement the convolution directly in Python, as shown in the example here. This will become automated in a future release of Concrete Numpy, which will include an equivalent to the nn.Conv2d operator.
Let us explain here how to do it:
Suppose you have a convolution filter already prepared (e.g. trained with pytorch or even a simple gaussian filter), that has sufficiently low precision not to overflow a 7 bit accumulator. For example for 2 bit inputs and 2 bit weights the minimal worst case number of weights you can have in a filter is 14. Depending on the distribution of the weights and intermediary activations you can maybe increase this number 3-4 fold before overflowing.
Then, for 2d convolution, you can reshape your input (M, N) matrix I to matrix A of size (1, MxN). You can then build a matrix B that has size = (MxN, OutH x OutW) where OutH and OutW are the output width and height computed as below. You can then obtain the result of the convolution by multiplying A x B, obtaining a flattened version of size (1, OutH x OutW) of the 2d activation map.
def compute_conv_out_shape(x_shape, padding, filter_w, filter_h, stride): N, C, H, W = x_shape assert (H + 2 * padding - filter_h) % stride == 0 assert (W + 2 * padding - filter_w) % stride == 0 out_height = (H + 2 * padding - filter_h) // stride + 1 out_width = (W + 2 * padding - filter_w) // stride + 1 return (N, C, out_height, out_width)
Each column of B is the flattened 2d matrix with the filter splatted at positions corresponding to output cell i,j (i in 0…OutH, j in 0…OutW). You can build the matrix in the following manner:
def make_1c_2d_conv_filter_matrix(input_shape, weights, padding, stride): out_shape1 = compute_conv_out_shape(input_shape, padding, weights.shape, weights.shape, stride ) matrix = np.zeros( (input_shape * input_shape, out_shape1 * out_shape1), dtype=np.uint8 ) col_idx = 0 for j in range(-padding, input_shape + padding - weights.shape, stride): for i in range(-padding, input_shape + padding - weights.shape, stride): tmp_filt_mat = np.zeros((input_shape, input_shape), np.uint8) for fj in range(weights.shape): for fi in range(weights.shape): if ( j + fj < 0 or j + fj >= input_shape or i + fi < 0 or i + fi >= input_shape ): continue tmp_filt_mat[j + fj, i + fi] = weights[fj, fi] matrix[:, col_idx] = tmp_filt_mat.flatten() col_idx += 1 return matrix
In a further version of the framework, convolutions will be added.