Outlier Dimension in LLMs and Multidmodal-LLMs: Mechanisms for Task Adaptation and Factual Recall

Day - Time: 04 April 2025, h.10:30

Place: Area della Ricerca CNR di Pisa - Room: Aula Faedo

Speakers

• William Rudman (Brown University)

Referent

Giovanni Puccetti

Abstract

Understanding how models embed tokens in vector space is critical for interpreting their behavior. This talk explores the geometric properties of Large Language Model (LLM) embeddings and Multimodal-LLM (MLLM) embeddings through three studies.The first study introduces IsoScore, a novel metric for measuring isotropy, i.e., how uniformly variance is distributed in the embedding space. This study finds that LLM representations are dominated by a small set of "outlier dimensions," defined as dimensions with exceedingly high variance and magnitude. We use IsoScore to demonstrate that reducing isotropy correlates strongly with improved LLM classification performance.Next, this talk examines how LLMs adapt their embeddings to encode task-specific knowledge, showing that outlier dimensions play a central role in storing such information.Finally, I will present ongoing work on the role of outlier dimensions in storing factual associations in MLLMs. We first propose VisualCounterfact, which consists of tuples that alter specific visual properties—color, size, and texture—of common objects. For instance, given (banana, color, yellow), we create a counterfact image (banana, color, purple) by modifying the object's pixels. Using VisualCounterfact, we locate a mechanism, dominated by outlier dimensions, for reliably controlling whether a model will answer with the counterfactual property present in the image or retrieve the world-knowledge answer from its weights.