深潮TechFlow|Aug 19, 2026 11:33
[Zhiyuan Founder Discusses GLM-5.3: Expansion Laws Go Beyond Parameter Scale, Post-Training Becomes a Key Variable]
DeepFlow TechFlow News, August 19 — Zhiyuan founder Tang Jie published an article stating that large model expansion should not focus solely on the number of parameters but must also consider data scale, allocation of computational resources, inference costs, and actual operating conditions. The article reviewed the evolution from Kaplan's scaling laws to Chinchilla's compute-optimal theory, highlighting that the early approach of 'parameter growth outpacing data growth' led to resource mismatches in some ultra-large models. Subsequent research demonstrated that model parameters and training data should grow more proportionally.
The article further pointed out that as models are frequently invoked, inference costs have become the core of lifecycle costs, with optimal strategies shifting toward smaller models and longer training durations. Additionally, in mixture-of-experts models, total parameters primarily affect knowledge capacity, while activated parameters and effective depth more significantly influence inference capabilities. The single ratio of 'training tokens per parameter' is no longer universally applicable.
Regarding GLM-5.3, the article noted that it uses the same base model, architecture, total parameters, and activated parameters as GLM-5.2, but achieved significant improvements through one month of extended long-range environment training and reinforcement learning during post-training. The article argued that parameter scale, pre-training data, forward computation, and post-training are all independently adjustable expansion variables. This experiment demonstrated that post-training still has considerable room for improvement, and the expansion process is far from over.
Share To
HotFlash
APP
X
Telegram
CopyLink