Home
Deutsch
Advanced Search
Browse by ...
Publication overview
Contact persons
Help
Postprint
 |
|
as soon as is submitted to ZB.
Kernel normalized convolutional networks.
Trans. Machine Learn. Res. 2024, accepted (2024)
Existing convolutional neural network architectures frequently rely upon batch normalization (BatchNorm) to effectively train the model. BatchNorm, however, performs poorly with small batch sizes, and is inapplicable to differential privacy. To address these limi-tations, we propose the kernel normalization (KernelNorm) and kernel normalized convolutional layers, and incorporate them into kernel normalized convolutional networks (KNConvNets) as the main building blocks. We implement KNConvNets corresponding to the state-of-the-art ResNets while forgoing the BatchNorm layers. Through extensive exper-iments, we illustrate that KNConvNets achieve higher or competitive performance compared to the BatchNorm counterparts in image classification and semantic segmentation. They also significantly outperform their batch-independent competitors including those based on layer and group normalization in non-private and differentially private training. Given that, KernelNorm combines the batch-independence property of layer and group normalization with the performance advantage of BatchNorm1.
Additional Metrics?
Edit extra informations
Login
Publication type
Article: Journal article
Document type
Scientific Article
ISSN (print) / ISBN
2835-8856
e-ISSN
2835-8856
Quellenangaben
Volume: 2024
Publisher
Journal of Machine Learning Research Inc.
Non-patent literature
Publications
Reviewing status
Peer reviewed
Institute(s)
Institute for Machine Learning in Biomed Imaging (IML)