SpineNet: Learning Scale-Permuted Backbone for Recognition and Localization (Paper Explained)

#machinelearning #ai #google The high-level architecture of CNNs has not really changed over the years. We tend to build high-resolution low-dimensional layers first, followed by ever more coarse, but deep layers. This paper challenges this decades-old heuristic and uses neural architecture search to find an alternative, called SpineNet that employs multiple rounds of re-scaling and long-range skip connections. OUTLINE: 0:00 - Intro & Overview 1:00 - Problem Statement 2:30 - The Problem with Current Architectures 8:20 - Scale-Permuted Networks 11:40 - Neural Architecture Search 14:00 - Up- and Downsampling 19:10 - From ResNet to SpineNet 24:20 - Ablations 27:00 - My Idea: Attention Routing for CNNs 29:55 - More Experiments 34:45 - Conclusion & Comments Papers: https://arxiv.org/abs/1912.05027 Code: https://github.com/tensorflow/tpu/tree/master/models/official/detection Abstract: Convolutional neural networks typically encode an input image into a series of intermediate features with decreasing resolutions. While this structure is suited to classification tasks, it does not perform well for tasks requiring simultaneous recognition and localization (e.g., object detection). The encoder-decoder architectures are proposed to resolve this by applying a decoder network onto a backbone model designed for classification tasks. In this paper, we argue encoder-decoder architecture is ineffective in generating strong multi-scale features because of the scale-decreased backbone. We propose SpineNet, a backbone with scale-permuted intermediate features and cross-scale connections that is learned on an object detection task by Neural Architecture Search. Using similar building blocks, SpineNet models outperform ResNet-FPN models by ~3% AP at various scales while using 10-20% fewer FLOPs. In particular, SpineNet-190 achieves 52.5% AP with a MaskR-CNN detector and achieves 52.1% AP with a RetinaNet detector on COCO for a single model without test-time augmentation, significantl