Introduction to Transformers for NLP: With the Hugging Face Library and Models to Solve Problems

Appendix A: Vision Transformers

Before the advent of vision transformers (ViTs), all tasks related to image-based machine learning like image classification, object detection, Q&A on images, image-to-caption mapping, etc. were taken care of by mostly CNNs and related neural architectures. With vision transformers there emerged an alternate means of handling such image-related tasks with better results.

Among the papers released on vision transformers, the one released on October 22, 2020, by Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, and Thomas Unterthiner is one that’s particularly noteworthy. Their approach is based on Vaswani’s “Attention Is All You Need” paper, which is widely used in natural language ...