2026-10-06 · America/Los_Angeles · 社区动态 · #19

Transformers vs RNNs vs SSMs: Where Does Memory Actually Live? [D]

Someone who has always loved looking at the space between different AI techniques, this time I went a little deeper into the memory trade-offs between RNNs, Transformers and SSMs. I found it interesting because once you start looking at these architectures through the lens of working memory, a lot of the differences become easier to understand. Where does the memory actually live? Is it a compact recurrent state, a growing KV cache, or something closer to the network itself? RNNs keep memory in a recurrent hidden state, which is pretty elegant because the state carries forward step by step.…

热度 48.8 / 100；排名与评分保留该期记录。