Nonlinearity Enhanced Adaptive Activation Functions
David Yevick
3/29/2024
cs.LGcs.CVcs.NE
No confident code implementation found for this paper. 2 possible matches can be shown below.
Abstract
A general procedure for introducing parametric, learned, nonlinearity into activation functions is found to enhance the accuracy of representative neural networks without requiring significant additional computational resources. Examples are given based on the standard rectified linear unit (ReLU) as well as several other frequently employed activation functions. The associated accuracy improvement is quantified both in the context of the MNIST digit data set and a convolutional neural network (CNN) benchmark example.
