[KDD 2020] AutoGrow: Automatic Layer Growing in Deep Convolutional Networks
Aug 13, 20203 views
Depth is a key component of Deep Neural Networks (DNNs), however, designing depth is heuristic and requires many human efforts.,We propose,AutoGrow,to automate depth discovery in DNNs: starting from a shallow seed architecture,,AutoGrow,grows new layers,if the growth improves the accuracy; otherwise, stops growing and,thus discovers the depth. We propose robust growing and stopping policies to generalize to different network architectures and,datasets. Our experiments show that by applying the same policy,to different network architectures,,AutoGrow,can always discover,near-optimal depth on various datasets of MNIST, FashionMNIST,,SVHN, CIFAR10, CIFAR100 and ImageNet. For example, in terms of,accuracy-computation trade-off,,AutoGrow,discovers a better depth,combination in,ResNets,than human experts. Our,AutoGrow,is efficient. It discovers depth within similar time of training a single DNN.,Our code is available at https://github.com/wenwei202/autogrow.