#ai

Attention Residuals: How Kimi Is Rethinking Transformer Depth

Kimi's Attention Residuals replace fixed residual connections with learned layer aggregation. What it means for LLM depth.