You can use any activation function if you implement your neural network directly in Concrete or Numpy.
However, the automatic PyTorch concersion only supports nn.Sigmoid and nn.ReLU6 for the for the moment. We’ll add more of them in further releases.