How do I do a convolution with Concrete Numpy?

Zama Concrete
1

You can implement the convolution directly in Python, as shown in the example here. This will become automated in a future release of Concrete Numpy, which will include an equivalent to the nn.Conv2d operator.

Let us explain here how to do it:

Suppose you have a convolution filter already prepared (e.g. trained with pytorch or even a simple gaussian filter), that has sufficiently low precision not to overflow a 7 bit accumulator. For example for 2 bit inputs and 2 bit weights the minimal worst case number of weights you can have in a filter is 14. Depending on the distribution of the weights and intermediary activations you can maybe increase this number 3-4 fold before overflowing.

Then, for 2d convolution, you can reshape your input (M, N) matrix I to matrix A of size (1, MxN). You can then build a matrix B that has size = (MxN, OutH x OutW) where OutH and OutW are the output width and height computed as below. You can then obtain the result of the convolution by multiplying A x B, obtaining a flattened version of size (1, OutH x OutW) of the 2d activation map.

    def compute_conv_out_shape(x_shape, padding, filter_w, filter_h, stride):
        N, C, H, W = x_shape
        assert (H + 2 * padding - filter_h) % stride == 0
        assert (W + 2 * padding - filter_w) % stride == 0
        out_height = (H + 2 * padding - filter_h) // stride + 1
        out_width = (W + 2 * padding - filter_w) // stride + 1
        return (N, C, out_height, out_width)

Each column of B is the flattened 2d matrix with the filter splatted at positions corresponding to output cell i,j (i in 0…OutH, j in 0…OutW). You can build the matrix in the following manner:

def make_1c_2d_conv_filter_matrix(input_shape, weights, padding, stride):
        out_shape1 = compute_conv_out_shape(input_shape, padding, weights.shape[0], weights.shape[1], stride )

        matrix = np.zeros(
            (input_shape[2] * input_shape[3], out_shape1[2] * out_shape1[3]), dtype=np.uint8
        )

        col_idx = 0
        for j in range(-padding, input_shape[2] + padding - weights.shape[0], stride):
            for i in range(-padding, input_shape[3] + padding - weights.shape[1], stride):
                tmp_filt_mat = np.zeros((input_shape[2], input_shape[3]), np.uint8)
                for fj in range(weights.shape[0]):
                    for fi in range(weights.shape[1]):
                        if (
                            j + fj < 0
                            or j + fj >= input_shape[2]
                            or i + fi < 0
                            or i + fi >= input_shape[3]
                        ):
                            continue
                        tmp_filt_mat[j + fj, i + fi] = weights[fj, fi]
                matrix[:, col_idx] = tmp_filt_mat.flatten()
                col_idx += 1

        return matrix

In a further version of the framework, convolutions will be added.