WASTE Inference Engine Enables Kimi K3 2.78T Model to Run Locally on 64GB Laptop

TechFlow news, August 1, according to a trending project on Hacker News, the open-source inference engine WASTE has achieved local inference of the full Kimi K3 model on a MacBook Pro equipped with 64GB of memory.

According to the introduction, WASTE is developed using the C language. By keeping the model backbone resident in memory and streaming MoE expert weights from disk on demand, it compresses the minimum memory usage to approximately 29.05 GiB, with an inference speed of about 0.49 to 0.54 tokens per second.

The full Kimi K3 model possesses approximately 2.78 trillion parameters, with a model container size of about 982 GiB, and was previously generally considered difficult to run on consumer-grade hardware. WASTE bypasses VRAM limits through a streaming loading mechanism, enabling laptops with standard configurations to run full model weights without relying on quantized or pruned versions.

Currently, the project has gained widespread attention within the Hacker News community, and the relevant code has been open-sourced.