**[2.78 Trillion-Parameter Kimi K3 Achieves 8GB Memory Operation, Developer Open-Sources Lightweight C Inference Engine]** BlockBeats News, August 8: A developer recently open-sourced the project `kimi-k3-in-c`, attempting to run the Kimi K3 model, which has 2.78 trillion parameters, on devices with only 8GB of memory. The project is just 176KB in size, written entirely in pure C99, and does not rely on GPU, CUDA, PyTorch, or BLAS, completing model inference solely via CPU. This approach leverages the MoE (Mixture of Experts) architecture of Kimi K3. Although the model's total parameter size reaches 2.78T, only 16 out of 896 experts per layer are activated. As a result, the developer did not load the full ~1.56TB model weights into memory but instead stored most of the expert weights on NVMe drives, reading them in real-time based on inference needs. Additionally, some dense layers (dense trunk) are also loaded in a layer-by-layer streaming manner. However, this solution currently faces significant performance limitations. In 8GB memory mode, generating a single token takes approximately 32.7 seconds and requires nearly 1.7TB of high-speed storage space. The developer stated that this approach is more of an experimental exploration of optimizing infrastructure for large-model inference rather than being suitable for practical production use. Nonetheless, its method of "streaming from disk + sparse activation via MoE" provides a new perspective for running ultra-large-scale models at low cost in the future. [Original Link]