Any-Order Modality Interactions

GRAMformer: Any-Order Modality Interactions via Volumetric Multimodal Cross-Attention

arXiv:2606.06249v1 Announce Type: new Abstract: Transformer-based multimodal models rely on attention mechanisms to integrate information across heterogeneous modalities. Despite their success, existing multimodal attention formulations compute their scores through collections of pairwise dot-product interactions or by concatenating all the modalities into the keys, even when multiple modalities should be jointly involved. As a consequence, current approaches either incur quadratic...