MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks

Review of paper by Zhiqiang Shen and Marios Savvides, Carnegie Mellon University, 2020

The authors used a version of the recently suggested MEAL technique (which involves knowledge distillation from multiple large teacher networks into a smaller student network via adversarial learning) to increase the top-1 accuracy of ResNet-50 on ImageNet with 224×224 input size to 80.67% without external training data or network architecture modifications.

CONTINUE READING >

End-to-End Object Detection with Transformers

Review of paper by Nicolas Carion, Francisco Massa, Gabriel Synnaeve et al, Facebook AI Research, 2020

This paper describes a completely automated end-to-end object detection system combining convolutional networks and Transformers. The new model shows competitive performance on par with Faster R-CNN and can be generalized to other tasks such as panoptic segmentation.

CONTINUE READING >

Supervised Contrastive Learning

Review of paper by Prannay Khosla, Piotr Teterwak, Chen Wang et al, Google Research, 2020

The authors used contrastive loss, which has recently been shown to be very effective at learning deep neural network representations in the self-supervised setting, for supervised learning, and achieved better results than those obtained with the cross-entropy loss for ResNet-50 and ResNet-200.

CONTINUE READING >

ResNeSt: Split-Attention Networks

Review of paper by Hang Zhang1, Chongruo Wu2, Zhongyue Zhang1 et al, 1Amazon and 2UC Davis, 2020

The authors suggest a new ResNet-like network architecture that incorporates attention across groups of feature maps. Compared to previous attention models such as SENet and SKNet, the new attention block applies the squeeze-and-attention operation separately to each of the selected groups, which is done in a computationally efficient way and implemented in a simple modular structure.

CONTINUE READING >

No more pages to load