TechFlow reports, on November 23, OpenAI Chief Scientist Jakub Pachocki wrote an article suggesting that developers of frontier models set a shared AI development cap based on metrics such as perplexity, and should not exceed this cap until their ability to understand and monitor AI behavior reaches specific targets.
He proposed that the industry establish common standards for model evaluation, red teaming, and transparency, and use perplexity as a reference metric for cross-developer comparison of pretraining progress. He also noted that datasets and training data vary across laboratories, and perplexity cannot capture all capability advancements beyond pretraining.
Pachocki anticipates that as log perplexity enters the range of approximately 0.68 to 0.7, models may exhibit new transformative capabilities, though the specific abilities remain unclear at this time.





