2026-10-16 · America/Los_Angeles · 论文 · #13

Behavioral Compositionality of Sparse Autoencoder Representations

Sparse autoencoders (SAEs) represent model activations with sparse, linearly decoded features, but the downstream effects of those features need not compose additively. We introduce SAE Additivity Residuals (SAR), which measure the behavioral variation left unexplained above a chosen interaction order. Under independent feature subset sampling, population SAR is exactly the normalized energy of interactions above that order and bounds coalition ordering error for the optimal low order approximation. To distinguish compositionality associated with the SAE representation from that induced by…

热度 55.7 / 100；排名与评分保留该期记录。