2026-09-19 · America/Los_Angeles · 社区动态 · #14

Experimenting with hypersurface-constrained dynamic weight updating [P]

Hey everyone! I want to share the results of a small language model architecture experiment I’ve been running as a side project. The ultimate goal is to test an approach to reduce the number of the model's training parameters, since the main bottleneck in training is VRAM. The core idea is very similar to Universal Transformer. Let's take just a single decoder block and iteratively pass the input through it L times in a loop. But instead of updating the input with the loop iteration depth information, my model dynamically updates the weights of the base layer. How is it implemented? The…

热度 44.1 / 100；排名与评分保留该期记录。