THE PURSUIT FOR NOVEL ACTIVATION FUNCTION

From the very start, activation functions in the neural network have been a key research area. More and more people are trying to come up with the state of the art activation functions for various purposes. These functions play an important role in deciding how good the model performs.

Deep Learning allows us to create various complex learning models that are developed by algorithms capable of multi-level representation. It uses representation learning, which allows it to learn from the raw data and is capable of performing tasks like detection and classification. The functions play an important role in deciding the performance of a simple MLP to a very complex Deep Neural Networks. With the invention of every new activation function, we have seen a performance boost in various neural network architecture ranging from object detection to various Natural language applications too. The aim of the invented activation function is also focused on trying to further boost the performance of these Deep Neural networks, thereby providing a better user experience with increased performance in various Deep Learning-based applications.