WASTE Inference Engine Open Sourced to Run Kimi K3 on 64GB MacBook

SQLite AI releases WASTE, enabling full-layer Kimi K3 execution on 64GB hardware via SSD offloading. The 1TB model achieves 0.49-0.54 TPS with 3-bit expert quantization.

Woofun AI reports that SQLite AI has open-sourced the WASTE inference engine, facilitating the execution of the Kimi K3 model on a 64GB MacBook Pro without layer reduction. The system retains approximately 27GB of the model backbone in memory while storing over 80,000 experts on the built-in SSD, reading only the invoked expert for each generated token.

The converted model totals approximately 1TB in size and generates tokens at a speed of 0.49 to 0.54 per second. This implementation avoids distillation, pruning, or expert removal, instead applying 3-bit quantization to expert weights and 4-bit or 8-bit quantization to the model backbone.