ByteDance Begins Training 10T Parameter Model, Rivaling Anthropic Scale

ByteDance initiates pre-training for a 10 trillion-parameter model, aiming to match US flagship scales. The project prioritizes independent R&D over distillation, signaling a major shift in Chinese AI strategy.

Woofun AI reports that ByteDance has commenced pre-training a large language model with up to 10 trillion parameters. This scale exceeds Kimi K3 by more than three times and approaches industry estimates for Anthropic’s Mythos 5, which holds approximately 8 trillion parameters. The pre-training phase, typically lasting three to six months, precedes necessary fine-tuning before release. Zhang Yiming has directed the team to reject distilling competitor models, emphasizing independent research to achieve global leadership despite potential short-term setbacks.