Meta's chief AI scientist Yann LeCun once again attacks generative AI
TechFlow Selected TechFlow Selected
Meta's chief AI scientist Yann LeCun once again attacks generative AI
Open-source models are slowly but steadily surpassing closed-source models.
Navigating Web3 tides with focused insightsSource: China Entrepreneurs Magazine
Image source: Generated by Wujie AI
"Abandon generative models, stop researching LLMs (large language models). We cannot achieve human-level intelligence in AI through text-only training," recently Yann LeCun, Meta's Chief AI Scientist, once again criticized generative AI at the 2025 Action Summit on Artificial Intelligence in Paris, France.
LeCun believes that although current large models operate efficiently, their reasoning process is divergent—generated tokens may fall outside the range of reasonable answers, which explains why some large models produce hallucinations. While today’s generative models enable AI to pass bar exams and solve math problems, they still can't do household chores. Tasks that humans perform without thinking are extremely complex for generative AI.
He also stated that generative models are fundamentally unsuitable for video generation. The AI models currently capable of generating videos do not truly understand the physical world; they merely generate aesthetically pleasing images. LeCun supports models that can understand the physical world, proposing a Joint Embedding Predictive Architecture (JEPA) better suited for predicting video content. He firmly believes only when AI genuinely understands the physical world can we achieve artificial intelligence on par with human intelligence.
Finally, LeCun emphasized the necessity of open-source AI platforms. In the future, we will have general virtual assistants that manage all our interactions with the digital world. These assistants must speak all the world’s languages, understand every culture, value system, and center of interest. Such AI systems cannot emerge from just a few companies in Silicon Valley—they must be collaboratively developed in an effective manner.
Key insights below:
1. We need human-level intelligence because we are accustomed to interacting with people. We expect AI systems with human-level intelligence. Future ubiquitous AI assistants will serve as bridges between humans and the digital world, enhancing human-digital interaction.
2. We cannot achieve human-level intelligence in AI through text-only training—it's simply impossible.
3. At Meta, we refer to this kind of AI achieving human-level intelligence as Advanced Machine Intelligence. We dislike the term "AGI" (Artificial General Intelligence), preferring "AMI," which sounds like "friend" in French.
4. Generative models are fundamentally unsuitable for video creation. You may have seen AI models that generate videos, but they don’t truly understand physics—they just generate pretty pictures.
5. If you're interested in human-level AI and work in academia, don't research LLMs. You’re competing against hundreds of teams with tens of thousands of GPUs—it’s meaningless.
6. AI platforms must be shared. They must speak all the world’s languages, understand all cultures, value systems, and centers of interest. No single company worldwide can train such foundational models; collaboration in an effective way is essential.
7. Open-source models are slowly and steadily outperforming closed-source models.
Full transcript of the talk (edited):
Why We Need Human-Level AI
It's well known that we need human-level artificial intelligence—not only as an interesting scientific challenge but also as a product requirement. In the future, we’ll wear smart devices like smart glasses, allowing us to access AI assistants anytime and interact with them.
We need human-level intelligence because we are accustomed to interacting with people. We expect AI systems with human-level intelligence. Future ubiquitous AI assistants will serve as bridges between humans and the digital world, helping humans interact better with the digital realm. However, compared to humans and animals, current machine learning is still very poor. We haven't built machines with human-like learning ability, common sense, or understanding of the physical world. Animals and humans can act based on common sense—behaviors that are inherently goal-driven.
Therefore, nearly all widely used AI systems today lack the characteristics we desire. They recursively generate one token after another, using marked tokens to predict the next. These systems are trained by placing information at the input end and attempting to reconstruct it at the output end. It's a causal structure—no cheating allowed, no using specific inputs to predict themselves—only neighboring tokens can be considered. Thus, it's highly efficient, leading people to call them universal large models usable for generating text and images.
But this reasoning process is divergent: each time you generate a token, it might fall outside the valid answer space, potentially taking you further from the correct answer. Once this happens, there's no way to fix it later—this is why some large models produce hallucinations and nonsense.
Current AI cannot replicate human intelligence—we can't even replicate the intelligence of cats or mice, which understand the rules of the physical world and perform common-sense actions without planning. A 10-year-old child can effortlessly clear dishes and wipe tables; a 17-year-old can learn to drive in 20 hours. Yet we still can't build a home robot. This shows our AI research and development are missing something crucial.
Our current AI can pass bar exams, solve math problems, and prove theorems, but cannot do housework. The things we consider effortless are extremely complex for AI robots, while tasks we regard as uniquely human—like language, chess, poetry writing—are easily accomplished by current AI and robots.
We cannot achieve human-level intelligence in AI through text-only training—it's simply impossible. Some vested interests claim AI intelligence will reach PhD-level next year, but that's utterly unrealistic. AI may reach human PhD-level in specific domains like chess or translation, but general large models cannot achieve this. If you train AI models specialized for certain domain questions, they can generate answers within seconds if the question is standard. But if you slightly rephrase the question, the AI may give the same answer because it hasn't truly thought about the problem. So achieving human-level AI will still take time.
Not "AGI" but "AMI"
At Meta, we call this kind of AI capable of human-level intelligence Advanced Machine Intelligence. We don't like the term "AGI" (Artificial General Intelligence); instead, we use "AMI," which sounds like "friend" in French. We need models that gather information through senses and learn—models that can manipulate representations in their minds and learn 2D physics from videos. Examples include systems with persistent memory, hierarchical action planning, and reasoning capabilities—systems made controllable and safe through design rather than fine-tuning.
Now, I know the only way to build such systems is to change how current AI systems perform reasoning. Current LLM reasoning works by running a fixed number of neural network layers (Transformers) to generate a token and input it, then repeating this process. The issue with this method is that whether you ask a simple or complex question, the system spends the same amount of computation answering "yes" or "no." So people constantly cheat by instructing the system how to respond. Humans know this reasoning trick: make the system generate more tokens, thus consuming more compute power to answer questions.
Actually, reasoning doesn't work this way. In classical statistical AI, structural prediction, and many other fields, reasoning works as follows: you have a function measuring compatibility or incompatibility between observations and outputs. The reasoning process involves finding values that minimize compression in the information space and produce output. We call such functions energy functions. When results don't meet requirements, the system performs optimization and reasoning. If the reasoning task is harder, the system spends more time reasoning—in other words, it thinks longer about complex problems.
In classical AI, much revolves around reasoning and search, so optimizing any computational problem can be reduced to a reasoning or search problem. This type of reasoning resembles what psychologists call System 2—thinking through actions before taking them—while System 1 refers to automatic, subconscious behaviors.
Source: Video screenshot
Let me briefly explain energy-based models: we capture dependencies between variables via energy functions. Suppose observation X and output Y—when X and Y are compatible, the energy function takes a low value; when incompatible, it takes a high value. You don’t want to simply compute Y from X—you want an energy function measuring incompatibility. Given X, you just find a Y with lower energy.
Now let’s examine in detail how such a world model architecture is built and its relation to thinking or planning. The system observes the world through a perception module that summarizes the world's state. Of course, the world state isn’t fully observable, so you may need to combine it with memory—the content of your thoughts about the world state. Together, these form a world model.
What is a world model? A world model provides a summary of the current world state. Within an abstract representation space, it imagines a sequence of actions, and your world model predicts the world state after executing those actions. If I tell you to imagine a cube floating in front of you, now rotate it vertically by 90°, what would it look like? You can easily visualize its appearance after rotation.
I believe we will achieve human-level intelligence before we have truly functional audiovisual systems. If we have such a world model capable of predicting outcomes of action sequences, we can feed it into a task objective measuring how well the predicted final state satisfies our self-defined goals. This is just a goal function. We can also set constraints—viewed as requirements for safe system operation. With these constraints, system safety is guaranteed—you cannot bypass them; they are hard-coded, outside the scope of training and reasoning.
Now, a sequence of actions should repeatedly use a world model across multiple time steps. After executing the first action, it predicts the resulting state; after the second action, it predicts the next state, continuing along this trajectory. You can also define task objectives and constraints. If the world isn't fully deterministic or predictable, the world model may require latent variables to account for unobserved aspects of the world, introducing prediction bias. Ultimately, we want a system capable of hierarchical planning. It may have several abstraction levels: at lower levels, we plan low-level actions like basic muscle control; at higher levels, we plan abstract macro-actions. For example, sitting in my NYU office, I decide to go to Paris. I can break this task into subtasks: getting to the airport and catching the flight. Then plan each step in detail: grab bag, leave office, hail taxi, take elevator, buy ticket…
We often aren’t consciously aware of performing hierarchical planning—it’s mostly subconscious—but we don’t know how to teach machines to do this. Almost every machine learning process involves hierarchical planning, but each level’s prompts are manually entered. We need to train an architecture that learns these abstract representations autonomously—not just world states, but also predictive world models and abstract actions across different abstraction levels—so machine learning can unconsciously perform hierarchical planning like humans.
How to Make AI Understand the World
With all these reflections, three years ago I wrote a long paper explaining the areas I believe AI research should focus on. I wrote this paper before ChatGPT became popular, and to this day, my views haven’t changed—ChatGPT has changed nothing. That paper discussed the path toward autonomous machine intelligence, which we now call Advanced Machine Intelligence because the word "autonomous" scares people. I've presented it in various talks.
To make systems understand how the world operates, a common method is applying the same process we used to train natural language systems to video: if a system can predict what happens in a video—given a short clip, predict what comes next—training it to make predictions can actually help the system understand the underlying structure of the world. This works for text because predicting words is relatively simple—there are finite words and limited tokens. We can’t accurately predict which word follows another or which word is missing in text, but we can calculate probabilities for each possible word in the dictionary.
But we can't do this for images or videos—we lack good methods to represent the distribution of video frames. Every attempt usually runs into mathematical difficulties. So you could try solving this using statistics and mathematics invented by physicists—or better yet, completely abandon the idea of probabilistic modeling.
Because we can't precisely predict what will happen in the world. Training a system to predict just one frame won't work well. So the solution is to develop a new architecture I call Joint Embedding Predictive Architecture (JEPA). Generative models are fundamentally unsuitable for video production. You may have seen AI models that generate videos, but they don’t truly understand physics—they just generate pretty pictures. The idea behind JEPA is running observations and outputs simultaneously, so instead of just predicting pixels, it predicts what happens in the video.
Source: Video screenshot
Let’s compare these two architectures. On the left is the generative architecture: you input X (observation) into an encoder and predict Y—a simple prediction. On the right, the JEPA architecture runs both X and Y (possibly through same or different encoders), then predicts Y’s representation based on X’s representation in this abstract space. This leads the system to essentially learn an encoder that eliminates everything unpredictable—this is exactly what we aim to do.
When filming in a room, if the camera moves, neither humans nor AI can predict who will appear in the next frame or what the texture of walls or floors will be—many things are fundamentally unpredictable. Therefore, instead of insisting on probabilistic predictions for things we can't predict, we should abandon predicting them and learn a representation where all these details are essentially eliminated—making prediction much simpler and simplifying the problem.
JEPA architecture comes in various forms. Here, let’s skip discussion of latent variables and focus on action-conditioned versions—the most interesting part because they truly function as world models. You have an observation X representing the current world state, input your planned action into the encoder (which serves as the world model), and it predicts the representation of the world state after performing that action—this is how planning works.
Recently, we conducted in-depth research on Video JEPA. How does this model work? For example, first extract 16 consecutive frames from a video as input samples, then mask and corrupt some frames, feeding these partially damaged video frames into the encoder while synchronously training a predictor module to reconstruct complete video representations from incomplete visual information. Experiments show this self-supervised learning method has significant advantages—its learned deep features can be directly transferred to downstream tasks like video action classification, achieving excellent performance across multiple benchmarks.
One fascinating thing is if you show this system something very strange happening in a video, the system actually tells you its prediction error is skyrocketing. You record a video, take 16 frames to measure the system’s prediction error—if something strange occurs, like an object spontaneously disappearing or changing shape, the prediction error rises. It indicates that despite its simplicity, the system has learned a degree of common sense—it can tell you when something very strange happens in the world.
I’d like to share our latest work—DINO-WM (a novel method for building visual dynamics models without reconstructing the visual world). Train a predictor using an image of the world, run it through a DINO encoder, then perhaps let a robot perform an action to get the next video frame. Feed this new image back into the DINO encoder to obtain a new image, then train your predictor to forecast what will happen based on the action taken.
Planning is very simple: observe an initial state, run it through the DINO encoder, then use imagined actions to run the world model across multiple time points and steps. You have a goal state represented by a target image—for instance, you run that image through the encoder, giving you a target state in the representation space. Then compute the gap between the predicted state and the target image state in the representation space, finding the action sequence with minimal execution cost.
Source: Video screenshot
This is a very simple concept, yet highly effective. Suppose you have a small T-shaped pattern you want to push to a specific location. You know where it needs to go because you’ve placed that location’s image into the encoder, giving you a target state in the representation space. As you execute a planned sequence of actions, what actually happens in the real world is shown alongside the system’s internal mental prediction of the action sequence. Feeding this into a decoder produces a graphical representation of the internal state.
Please Abandon Research on Generative Models
Finally, I have some advice to share. First, abandon generative models. This is currently the most popular approach—everyone is working on it. Instead, study JEPA—these aren’t generative models; they predict what will happen in the world within a representation space. Abandon reinforcement learning—I've said this for a long time—it's inefficient. If you're interested in human-level AI and work in academia, don’t research LLMs. You're competing against hundreds of teams with tens of thousands of GPUs—it makes no sense. Academia still faces many unsolved challenges: planning algorithms are inefficient, we must devise better methods. JEPA with latent variables in uncertain hierarchical planning remains completely unresolved—scholars are welcome to explore these areas.
In the future, we will have general virtual assistants accompanying us constantly, managing all our interactions with the digital world. We cannot allow these AI systems to come from just a few companies in Silicon Valley or China. This means the platforms we build these systems on must be open-source and widely accessible. Training these systems is expensive, but once you have a base model, fine-tuning for specific applications becomes relatively cheap and affordable for many.
AI platforms must be shared. They must speak all the world’s languages, understand all cultures, all value systems, and all centers of interest. No single company worldwide can train such foundational models—they must be collaboratively developed effectively.
Therefore, open-source AI platforms are necessary. The crisis I see in Europe and elsewhere is geopolitical competition inducing some national governments to essentially outlaw the release of open-source models, wanting to keep scientific secrets to maintain leadership. This is a huge mistake. When you conduct research secretly, you fall behind—that’s inevitable. What will happen is that the rest of the world adopts open-source technology, and we will surpass you. This is already happening—open-source models are slowly and steadily outperforming closed-source models.
Join TechFlow official community to stay tuned
Telegram:https://t.me/TechFlowDaily
X (Twitter):https://x.com/TechFlowPost
X (Twitter) EN:https://x.com/BlockFlow_News
