PROTOSCORE

Prototypicality Bias Reveals Blindspots in Multimodal Evaluation Metrics

arXiv:2601.04946v3 Announce Type: replace Abstract: Automatic metrics are widely used to evaluate text-to-image models, often replacing human judgment in benchmarking, model selection, and large-scale data filtering. Yet they may reward images that look plausible or prototypical rather than images that faithfully satisfy the prompt. We identify prototypicality bias as a systematic blindspot in multimodal evaluation: metrics can prefer a semantically incorrect but visually or socially...