Neurons With Paraboloid Decision Boundaries for Improved Neural Network Classification Performance
In mathematical terms, an artificial neuron computes the inner product of a <inline-formula> <tex-math notation="LaTeX">$d$ </tex-math></inline-formula>-dimensional input vector <inline-formula> <tex-math notation="LaTeX">${x}$ </tex-math></inline-formula> with its weight vector <inline-formula> <tex-math notation="LaTeX">${w}$ </tex-math></inline-formula>, compares it with a bias value <inline-formula> <tex-math notation="LaTeX">$w_{0}$ </tex-math></inline-formula> and fires based on the result of this comparison. Therefore, its decision boundary is given by the equation <inline-formula> <tex-math notation="LaTeX">${w}^{T}{x}+w_{0}=0$ </tex-math></inline-formula>. In this paper, we propose replacing the linear hyperplane decision boundary of a neuron with a curved, paraboloid decision boundary. Thus, the decision boundary of the proposed paraboloid neuron is given by the equation <inline-formula> <tex-math notation="LaTeX">$({h}^{T}{x}+h_{0})^{2}-||{x}-{p}||_{2}^{2}=0$ </tex-math></inline-formula>, where <inline-formula> <tex-math notation="LaTeX">$h$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$h_{0}$ </tex-math></inline-formula> denote the parameters of the directrix and <inline-formula> <tex-math notation="LaTeX">$p$ </tex-math></inline-formula> denotes the coordinates of the focus. Such paraboloid neural networks are proven to have superior recognition accuracy in a number of applications.