Designing Hardware-Aware Algorithms with Kimi Linear: Kimi Delta Attention

**Designing Hardware-Aware Algorithms with Kimi Linear: Kimi Delta Attention**

Introduction Moonshot AI has done it again. We were impressed with their release of Kimi-K2 and their post-training approach. Now, in addition to Kimi-K2-Thinking (which we encourage you to check out), they also released Kimi Linear, a hybrid linear attention architecture where they introduce a new attention mechanism, Kimi Delta Attention (KDA). The release features an open-source KDA kernel (written in triton), vLLM implementations, as well as the pre-trained and instruction-tuned model checkpoints (48B total parameters, 3B activated parameters, 1 million context length). In this article, we are going to discuss key findings from the Kimi Linear paper and show how you can run the model with DigitalOcean...

---

**[devsupporter 해설]**

이 기사는 DigitalOcean에서 제공하는 최신 개발 동향입니다. 관련 도구나 기술에 대해 더 알아보시려면 원본 링크를 참고하세요.