今日arXiv精选 | 近期必读的5篇Transformers相关论文
?關于?#今日arXiv精選?
這是「AI 學術前沿」旗下的一檔欄目，編輯將每日從arXiv中精選高質量論文，推送給讀者。
Fastformer: Additive Attention is All You Need
Category: NLP
Link:?https://arxiv.org/abs/2108.09084
Abstract
Transformer is a powerful model for text understanding. It is inefficient due to its quadratic complexity to input sequence length. In Fastformer, instead of modeling the pair-wise interactionsbetween tokens, we first use additive attention mechanism to model global contexts.
Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text ?Models
Category: NLP
Link:?https://arxiv.org/abs/2108.08877
Abstract
Sentence embeddings are broadly useful for language processing tasks. While T5 achieves impressive performance on language tasks cast assequence-to-sequence mapping problems, it is unclear how to produce sentences from encoder-decoder models. We investigate three methods for extracting T5 sentences.
Do Vision Transformers See Like Convolutional Neural Networks?
Category: Computer Vision
Link:?https://arxiv.org/abs/2108.0881
Abstract
Recent work has shown that (Vision) Transformer models (ViT) can achieve superior performance on image classification tasks. Are they acting like convolutional networks, or learning entirely different visual representations?
Is it Time to Replace CNNs with Transformers for Medical Images?
Category: Computer Vision
Link:?https://arxiv.org/abs/2108.09038
Abstract
Vision transformers (ViTs) have appeared as a competitive alternative to CNNs. ViTs possess several interestingproperties that could prove beneficial for medical imaging tasks. While CNNs perform better when trained from scratch, ViTs are on par with CNNs when pretrained on ImageNet.
Video Relation Detection via Tracklet based Visual Transformer
Category: Computer Vision
Link:?https://arxiv.org/abs/2108.08669
Abstract
Video Visual Relation Detection (VidVRD), has received significant attention over recent years. We apply the state-of-the-art video object tracklet detection pipeline MEGA and deepSORT. Then we perform VidVRD in a tracklet-based manner.
·
總結
以上是生活随笔為你收集整理的今日arXiv精选 | 近期必读的5篇Transformers相关论文的全部內容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： 2022年农村什么生意好做 考虑这些未
- 下一篇： 关于NLP相关技术全部在这里：预训练模型