Deep CNN models often have worse performance. This is not caused by overfitting, and adding more layers can lead to higher training errors. One explanation is that during the training process, gradient vanishing happens in some layers, causing all its preceding layers to fail to learn.

Residual connection can help reduce model complexity. DenseNet, in which all the layers are connected through the residual signal, can achieve the same prediction accuracy with fewer model parameters.