Behaviour in 0 of the Neural Networks Training Cost

    loading  Checking for direct PDF access through Ovid

Abstract

We study the behaviour in zero of the derivatives of the cost function used when training non-linear neural networks. It is shown that a fair number of first, second and higher order derivatives vanish in zero, validating the belief that 0 is a peculiar and potentially harmful location. These calculations are related to practical and theoretical aspects of neural networks training.

Related Topics

    loading  Loading Related Articles