کنفرانس مهندسی برق ایران

صفحه اصلی / سی و سومین کنفرانس بین المللی مهندسی برق

Improving Artificial Neural Network Performance Using Hybrid Activation Function

نویسندگان :

Morteza Taheri¹ Sajad Haghzad Klidbary²

1- زنجان 2- زنجان

کلمات کلیدی :

Activation Function, Hybrid Activation Function, Neural Networks, Deep Learning.،Hybrid Activation Function،Cifar and Mnist Dataset.

چکیده :

Abstract— Choosing the optimal activation function for the training of deep neural networks has always posed a significant challenge due to its substantial impact on the network performance and training speed. Despite the impressive performance demonstrated by various nonlinear non-monotonic activation functions, such as rectified linear units, hyperbolic tangent, sigmoid, swish, mish, Smish, and Logish, only a restricted subset of these functions is commonly embraced in most applications, primarily because of inherent inconsistencies or other limitations. This research introduces a new hybrid activation functions designed to surpass other activation functions. The results from our experiments suggest that the new activation function(AF) we introduced performs well across diverse datasets. Also, These findings indicate that the activation function we developed are better suited for intricate deep-learning models. In comparison to Sigmoid, Relu and other AFs, our proposed activation function has increased accuracy on MNIST, Cifar10, Cifar100 dataset. Significantly, our purposed AF has the maximum accuracy (100%) compared to other well-known activation functions on MNIST dataset.