Modality Alignment Supercharges Audio

The Alignment Curse: Modality Alignment Supercharges Audio Attacks via Text Transfer

Announce Type: replace Abstract: Recent advances in end-to-end trained omni-models have substantially improved audio capabilities by strengthening text-audio modality alignment. However, whether such alignment inadvertently facilitates the transfer of safety vulnerabilities across modalities remains underexplored. This question is critical as text-based jailbreak attacks are considerably more mature than audio-based ones; if they transfer systematically, current audio safety evaluations may...