Behaviour in 0 of the Neural Networks Training Cost

    loading  Checking for direct PDF access through Ovid


We study the behaviour in zero of the derivatives of the cost function used when training non-linear neural networks. It is shown that a fair number of first, second and higher order derivatives vanish in zero, validating the belief that 0 is a peculiar and potentially harmful location. These calculations are related to practical and theoretical aspects of neural networks training.

Related Topics

    loading  Loading Related Articles