RobustNeuralNetworks.jl Documentation

A Julia package for robust neural networks.

Welcome to the documentation for RobustNeuralNetworks.jl! This package contains neural network models that are constructed to naturally satisfy robustness constraints, all in native Julia. Check out our GitHub repository here.

Why Robust Models?

Modern machine learning relies heavily on rapidly training and evaluating neural networks in problems ranging from image classification to robotic control. Most neural network architectures have no robustness certificates, and can be sensitive to adversarial attacks, poor data quality, and other input perturbations. Many solutions that address this brittle behaviour rely on explicitly enforcing constraints during training to smooth or stabilise the network response. While effective on small-scale problems, these methods are computationally expensive, making them slow and difficult to scale up to complex real-world problems.

Recently, we proposed the Recurrent Equilibrium Network (REN) and Lipschitz-Bounded Deep Network (LBDN) or sandwich layer model classes as computationally efficient solutions to these problems. The REN architecture is flexible in that it includes many common neural network models, such as multi-layer-perceptrons (MLPs), convolutional neural networks (CNNs), and recurrent neural networks (RNNs). The weights and biases in RENs are directly parameterised to naturally satisfy a set of user-defined robustness metrics constraining the internal stability and input-output sensitivity of the network. When a network is guaranteed to satisfy a robust metric, we call this a robustness certificate. An example is a Lipschitz bound, which restricts the network’s amplification of input perturbations in its outputs. LBDNs are specializations of RENs with the specific feed-forward structure of deep neural networks like MLPs or CNNs, and built-in restrictions on the Lipschitz bound.

The direct parameterisation of RENs and LBDNs means that we can train models with standard, unconstrained optimisation methods (such as stochastic gradient descent) while also guaranteeing their robustness. Achieving the "best of both worlds" in this way is the main advantage of the REN and LBDN model classes, and allows the user to freely train robust models for many common machine learning problems, as well as for more challenging real-world applications where safety is critical.

Introduction

• Getting Started
• Package Overview
• Contributing to the Package

Examples

• Fitting a Curve with LBDN
• Image Classification with LBDN
• Reinforcement Learning with LBDN
• Observer Design with REN
• (Convex) Nonlinear Control with REN

Library

• Model Wrappers
• Model Parameterisations
• Functions

Citing the Package

If you use RobustNeuralNetworks.jl for any research or publications, please cite our work as necessary.

@article{barbara2025robustneuralnetworksjl, 
   title = {RobustNeuralNetworks.jl: a Package for Machine Learning and Data-Driven Control with Certified Robustness}, 
   author = {Nicholas H. Barbara and Max Revay and Ruigang Wang and Jing Cheng and Ian R. Manchester}, 
   journal = {Proceedings of the JuliaCon Conferences},
   publisher = {The Open Journal}, 
   year = {2025}, 
   volume = {7}, 
   number = {68}, 
   pages = {163}, 
   doi = {10.21105/jcon.00163}, 
   url = {https://doi.org/10.21105/jcon.00163}, 
}

Research Papers

RobustNeurlaNetworks.jl is built on the REN and LBDN model classes described in the following two papers (respectively):

M. Revay, R. Wang, and I. R. Manchester, "Recurrent Equilibrium Networks: Flexible Dynamic Models with Guaranteed Stability and Robustness" IEEE Trans Automat Contr 1–16 (2023) doi:10.1109/TAC.2023.3294101.

R. Wang and I. R. Manchester, "Direct parameterization of Lipschitz-bounded deep networks" in Proceedings of the 40th International Conference on Machine Learning (PMLR, 2023) 202:36093-36110.

The REN parameterisation was extended to continuous-time systems in (yet to be implemented):

D. Martinelli, C. L. Galimberti, I. R. Manchester, L. Furieri, and G. Ferrari-Trecate, "Unconstrained Parametrization of Dissipative and Contracting Neural Ordinary Differential Equations," Proceedings of the 62nd IEEE Conference on Decision and Control (2023). doi: https://doi.org/10.1109/CDC49753.2023.10383704.

See below for a collection of projects and papers using RobustNeuralNetworks.jl.

N. H. Barbara, R. Wang, and I. R. Manchester, "Learning Over Contracting and Lipschitz Closed-Loops for Partially-Observed Nonlinear Systems," Proceedings of the 62nd IEEE Conference on Decision and Control (2023). doi: https://doi.org/10.1109/CDC49753.2023.10383269.

P. Pauli, R. Wang, I. R. Manchester, and F. Allgöwer, "Lipschitz-Bounded 1D Convolutional Neural Networks using the Cayley Transform and the Controllability Gramian," Proceedings of the 62nd IEEE Conference on Decision and Control (2023). doi: https://doi.org/10.1109/CDC49753.2023.10383534.