Actually Concrete ML works with torch nn.Modules. You can train them either with your own training code, as shown in the CNN Example or by using the wrappers in the concrete.ml.sklearn.qnn module.
Therefore, Concrete ML does not replace torch in any way, it just complements it by allowing you to convert your models to FHE.
If you plan on using Concrete ML with models that you train yourself, you need to ensure these models obey FHE constraints as detailed here, notably the accumulator bitwidth limitations.