TechFlow news — On January 9, according to TechCrunch, Elon Musk stated in a live conversation with Stagwell chairman Mark Penn that the sum of human knowledge available for training AI was essentially exhausted by 2024. This echoes the "data peak" theory proposed by former OpenAI chief scientist Ilya Sutskever at the NeurIPS conference in December.
Musk believes synthetic data will become a key pathway for future AI development. Currently, tech giants including Microsoft, Meta, OpenAI, and Anthropic have already adopted synthetic data training approaches in their flagship AI models. For instance, Microsoft's newly open-sourced Phi-4, Google's Gemma model, Anthropic's Claude 3.5 Sonnet, and Meta's latest Llama series models all use synthetic data for training or fine-tuning.
From a cost perspective, AI startup Writer developed its Palmyra X 004 model using nearly entirely synthetic data at a cost of just $700,000—significantly lower than OpenAI's $4.6 million cost for a similarly sized model. However, research indicates synthetic data may lead to model collapse, resulting in less creative outputs and increased bias, as biases and limitations in the original training data are amplified during the synthesis process. According to Gartner, approximately 60% of data used in AI and analytics projects in 2024 was synthetically generated.
