Auto-Rotating Perceptrons

This paper proposed an improved design of the perceptron unit to mitigate the vanishing gradient problem at deep neural networks. The results show that models with Auto-Rotating Perceptrons (ARP) can achieve better learning performance than equivalent networks with classic perceptrons.

The modification consists of adding the scalar ρ to the value that enters the activation function, as is depicted below. Geometrically, it represents a rotation of the hyperplane present in the latent space of the perceptron. The ARP is a generalization of the classic perceptrons.

Work presented as an oral exposition and poster session at the LatinX in AI Workshop co-located with NeurIPS 2019.