A general insight into the effect of neuron structure on classification

Abstract

This paper gives a general insight into how the neuron structure in a multilayer<br><br>perceptron (MLP) can affect the ability of neurons to deal with classification. Most of the<br><br>common neuron structures are based on monotonic activation functions and linear input<br><br>mappings. In comparison, the proposed neuron structure utilizes a nonmonotonic activation<br><br>function and/or a nonlinear input mapping to increase the power of a neuron. An MLP of these<br><br>high power neurons usually requires a less number of hidden nodes than conventional MLP<br><br>for solving classification problems. The fewer number of neurons is equivalent to the smaller<br><br>number of network weights that must be optimally determined by a learning algorithm. The<br><br>performance of learning algorithm is usually improved by reducing the number of weights,<br><br>i.e., the dimension of the search space. This usually helps the learning algorithm to escape<br><br>local optimums, and also, the convergence speed of the algorithm is increased regardless of<br><br>which algorithm is used for learning. Several 2-dimensional examples are provided manually<br><br>to visualize how the number of neurons can be reduced by choosing an appropriate neuron<br><br>structure. Moreover, to show the efficiency of the proposed scheme in solving real-world<br><br>classification problems, the Iris data classification problem is solved using an MLP whose<br><br>neurons are equipped by nonmonotonic activation functions, and the result is compared with<br><br>two well-known monotonic activation functions.