2026-11-10 · America/Los_Angeles · 论文 · #15

Training Vision Transformers to Focus: Emergence of Spatial Attention Head Specialization

Self-supervised learning in Vision Transformer models, such as DINO, enables robust visual perception without requiring labeled data by leveraging large-scale image datasets. However, these models lack a focusing mechanism during training, resulting in perceptual representations that are often ambiguous or overly similar. In this work, we introduce two visual focusing loss functions designed to establish correspondence between the model's attention and specific regions within images. Specifically, we leverage the multi-head attention mechanism within the model to selectively steer the…

热度 30.9 / 100；排名与评分保留该期记录。