NeuralNetClassifier

Zama Concrete ML
1

Hello,

i have some questions for the NeuralNetClassifier.

  1. is it possible to set a weight decay (L2 penalty) in the built in NN classifier?

  2. how can I define the hidden layer sizes? I know there is a parameter called “n_hidden_neurons_multiplier” which is dependent on the input layer. So if i understand it right: input size = 20, module__n_layers = 2, n_hidden_neurons_multiplier = 0.5 the hidden layer sizes are (10,10). Or is the neurons multiplier dependent on the layer in front, hence the second hidden layer is half of the first hidden layer (10,5)?

  3. is there a way to generate loss and accuracy curves for train and validation data? the metrics are already printed from skorch and there is a history_ attribute which is empty though

  4. how to get rid of the following warning? [W shape_type_inference.cpp:1974] Warning: The shape inference of onnx.brevitas::Quant type is missing, so it may result in wrong shape inference for the exported graph. Please consider adding it in symbolic function. (function UpdateReliable)

Thanks in advance :slight_smile:

2
  1. yes, I think it’s possible to set it. Concrete ML uses skorch for the built-in neural network classifiers. See here for the skorch documentation on this.
  2. Currently Concrete ML does not support setting the number of hidden neurons per layer. The neuron multiplier depends on the input size (not the previous layer).
  3. Could you try this solution ? The history class is documented here. If that’s the solution you already tried, I’m afraid I can’t see any other one for now
train_loss = net.history[:, 'train_loss']
  1. Unfortunately that very annoying warning is generated by the pytorch ONNX exporter and there is no known way to disable it. It occurs with Pytorch 2.0 but not with Pytorch 1.3.1 which is the default one in Concrete ML.

If you need fine grained control on the network architecture or on the training behavior I suggest you use custom models as shown in this example .

3

thanks for your answers!

  1. adding {‘optimizer__weight_decay’: 0.0001} to my kwargs worked

  2. for the history. although net has an attribute called ‘history_’ there are no values stored there. what works though is accessing the history over the sklearn_model attribute → net.sklearn_model.history

1 Like