Attention Residuals

Moonshot AI@Kimi_MoonshotMar 15, 2026

Introducing π‘¨π’•π’•π’†π’π’•π’Šπ’π’ π‘Ήπ’†π’”π’Šπ’…π’–π’‚π’π’”: Rethinking depth-wise aggregation. Residual connections have long relied on fixed, uniform accumulation. Inspired by the duality of time and depth, we introduce Attention Residuals, replacing standard depth-wise recurrence with https://t.co/gcWyzhZVc0

Views5.07M
Comments334
Reposts2.04k
Likes13.4k
Launched Mar 15, 2026View post

More from Moonshot AI

1

🌘 Meet Kimi K2.7 Code HighSpeed! A high-speed mode of our latest open-source multimodal coding model, Kimi K2.7 Code. ⚑️ Up to 6Γ— faster: Around 180 tok/s on coding tasks with median-length inputs, and up to 260 tok/s on shorter-context tasks. πŸ”· Rolling out to Kimi Code Beta https://t.co/syOOgIdtI4

Jun 15, 2026
Views519.0k
Comments166
Reposts326
View post
2

Introducing Kimi Code, an open-source coding agent under the Apache 2.0 License. πŸ”Ή Python-based, easy to extend. πŸ”Ή Fully transparent β€” clear, safe, reliable. πŸ”Ή Seamlessly integrates with VS Code, Cursor, JetBrains, Zed, and more. πŸ”Ή Fully-featured & out-of-the-box ready. https://t.co/HtEvvH3amI

Jan 26, 2026
Views258.1k
Comments61
Reposts212
View post
3

Introducing Goal Mode in Kimi Work Goal lets your desktop agent run 24/7 until the task is done, built for long-horizon tasks and complex multi-step workflows. https://t.co/qD9utKsEQr

Jun 18, 2026
Views200.4k
Comments97
Reposts184
View post
4

See all Moonshot AI launches β†’