2026-10-19 · America/Los_Angeles · 论文 · #18

Lexical Reference Frames: Interpreting LM Residual Streams in Native Semantics

The vocabulary of a language model is its native semantics: the embedding and the unembedding fix the meaning of a state at the two ends of the residual stream. Between them, existing methods either reuse a fixed lens or treat the states as a point cloud, and neither reads a state the way the model does. We argue that every hidden state has a native reference frame, induced by the layers that read it next and adaptive to the context. Lexical reference frames approximate it by reading a state through the model's own remaining layers with a stated number of rounds of attention between…

热度 50.1 / 100；排名与评分保留该期记录。