Homelabber runs 2.8T-parameter Kimi K3 on CPU with second-gen Optane memory

A Reddit homelab builder successfully ran Moonshot AI’s 2.8-trillion-parameter Kimi K3 model using only a CPU and discontinued second-generation Optane persistent memory, according to the published report This demonstration highlights an alternative path for high-capacity memory access when traditional DRAM becomes prohibitively expensive for individual users.reported

What was released or claimed

The builder utilized Intel’s Optane Persistent Memory 200 Series, which launched in 2020 for third-generation Xeon Scalable processors. according to the published report, the setup employs a 40-core CPU and avoids GPU acceleration entirely.reported The system operates in App Direct Mode, allowing software to map the Optane memory directly rather than using it as a cache for DRAM. This configuration provides more control over where model tensors reside, though it requires active cooling with dual 12-watt blower fans to manage heat.reported

Who it affects

This development primarily impacts hobbyists and homelab builders seeking to run large AI models without expensive enterprise hardware. It also interests users facing high DRAM prices, as the builder noted that secondhand Optane costs a fraction of equivalent DRAM capacity. The community of users interested in CPU-only inference for massive models may find this approach viable for specific, non-real-time workloads.

What is still unknown

It remains unclear how broadly this specific configuration can be replicated with other large models or if performance scales linearly with different quantization levels. The long-term reliability of running these discontinued memory modules at high thermal loads is not documented in the report. Additionally, the total cost of ownership, including electricity for cooling, is not detailed.

What to make of it

The experiment demonstrates that massive memory capacity can be achieved through legacy enterprise hardware, bypassing current DRAM market constraints. However, the trade-off is significant performance degradation compared to GPU-accelerated or DRAM-heavy setups. The builder’s advice to treat active DIMM cooling as mandatory underscores the thermal challenges of this approach.

What buyers should weigh

Buyers considering similar setups must account for the severe speed limitations, with inference at 0.85 tokens per second.reported They should also evaluate the availability and condition of discontinued Optane PMem 200 modules, which come in 128GB, 256GB, and 512GB capacities.reported The necessity for robust cooling solutions adds complexity and cost to what is otherwise a cheaper memory alternative.