Sam Altman révèle les défis actuels et les orientations futures d'OpenAI
TechFlow SélectionTechFlow Sélection
Sam Altman révèle les défis actuels et les orientations futures d'OpenAI
Cet article provient d'un récent entretien accordé par Sam Altman, PDG d'OpenAI, à Raza Habib, PDG de Humanloop. Lors de cet entretien, Sam s'est exprimé très franchement sur les difficultés actuelles, la limitation des ressources en GPU, la longueur du contexte, la multimodalité ainsi que sur la question de l'open source.
Dédié à des analyses Web3 approfondiesIntroduction
Last week, Raza Habib, CEO of Humanloop, had the privilege of sitting down with Sam Altman and 20 other developers to discuss OpenAI's API and their product roadmap. Sam was remarkably candid. The conversation covered both practical developer concerns and broader questions related to OpenAI’s mission and the societal impact of AI. Here are the key takeaways:
OpenAI is currently heavily dependent on GPUs
A recurring theme throughout the discussion was that OpenAI is currently extremely reliant on GPUs, which has delayed many of their short-term plans. The top customer complaint revolves around the reliability and speed of the API. Sam acknowledged these concerns and explained that most issues stem from GPU shortages.
Longer 32k contexts cannot yet be rolled out to more users. OpenAI has not overcome the O(n^2) scaling of attention, so while they may soon have context windows of 100k to 1M tokens (this year), anything larger will require a research breakthrough.
Fine-tuning APIs are limited by GPU availability. They are not yet using efficient fine-tuning methods like Adapters or LoRa, making fine-tuning runs and management highly computationally expensive. Better fine-tuning support is coming in the future. They might even host a marketplace for community-contributed models.
Dedicated capacity is constrained by GPU availability. OpenAI also offers dedicated capacity, providing customers with private instances of their models. To use this service, clients must commit upfront to spending $100,000.
OpenAI's Short-Term Roadmap
2023:
Cheaper, faster GPT-4—This is their top priority. Overall, OpenAI aims to minimize the cost of "intelligence," so they will continue working to reduce API costs.
Longer context windows—Context windows up to one million tokens are possible in the near future.
Fine-tuning API—The fine-tuning API will expand to the latest models, but its exact form will depend on what developers indicate they truly want.
A stateful API—When you call the chat API today, you must repeatedly pass the same conversation history and pay for the same tokens again. Future versions of the API will remember conversation history.
2024:
Multimodality—This was demonstrated as part of the GPT-4 release, but cannot be scaled to everyone until more GPUs come online.
Plugins haven't achieved PMF and may not come to the API soon
Many developers are interested in calling ChatGPT plugins externally via the API, but Sam said he doesn’t believe they’ll release this functionality anytime soon. Beyond browsing, current plugin usage suggests they haven’t yet achieved product-market fit (PMF). He noted that many assumed they wanted apps to be used inside ChatGPT, but what people actually want is ChatGPT capabilities embedded within their own applications.
OpenAI will avoid competing with its customers—except with ChatGPT
Many developers expressed concern about building applications using OpenAI’s API, fearing OpenAI might launch competing products. But Sam stated that OpenAI won’t release any additional products beyond ChatGPT. He explained that a great platform company typically has one killer application, and ChatGPT allows them to improve the quality of their API by being their own customer. The vision for ChatGPT is to become a super-intelligent assistant at work, but OpenAI will not venture into other GPT use cases.
Regulation is needed, but so is open source
Although Sam is calling for regulation of future models, he does not consider existing models dangerous and believes regulating or banning them would be a major mistake. He reiterated his belief in the importance of open source and mentioned that OpenAI is considering open-sourcing GPT-3. One reason they haven’t done so yet is his skepticism about how many individuals and companies currently have the capability to host and serve large models.
Scaling laws still hold
Recently, numerous articles have claimed that “the era of big models is over.” This misrepresents the original intent.
Internal data at OpenAI shows that scaling laws for model performance remain valid—making models larger continues to yield better performance. The rate of scaling cannot be sustained indefinitely, as OpenAI has already increased model size by millions of times within just a few years, and continuing at that pace is unsustainable. This doesn’t mean OpenAI will stop trying to scale up, only that increases may be limited to doubling or tripling each year rather than growing by multiple orders of magnitude.
The fact that scaling remains effective has significant implications for the timeline to AGI. The scaling hypothesis suggests we may already possess most of the components needed to build AGI, with the remaining work largely consisting of scaling existing methods to larger models and datasets. If the era of scaling were truly over, we should expect AGI to be much further away. The continued validity of scaling laws strongly suggests a shorter timeline to AGI.
Bienvenue dans la communauté officielle TechFlow
Groupe Telegram :https://t.me/TechFlowDaily
Compte Twitter officiel :https://x.com/TechFlowPost
Compte Twitter anglais :https://x.com/BlockFlow_News
