What is Software 2.0? — OpenAI Founding Team Member
TechFlow Selected TechFlow Selected
What is Software 2.0? — OpenAI Founding Team Member
When the day comes that we develop Artificial General Intelligence (AGI), it will certainly be using Software 2.0.
Navigating Web3 tides with focused insightsAuthor: Andrej Karpathy (Founding team member of OpenAI, former Head of AI at Tesla)
Translation: OneFlow Community, ywh1bkansf
Andrej Karpathy was one of the first people to clearly explain why the AI wave truly matters. He believes that AI represents a new and powerful way to program computers, and if we ever develop Artificial General Intelligence (AGI), it will almost certainly be through Software 2.0.
Below is the full text of Andrej Karpathy's article originally published in 2017:
I've noticed that sometimes people think of neural networks as "just another tool in the machine learning toolbox." They have pros and cons, are useful in certain domains, and can help you win Kaggle competitions. Unfortunately, this view completely misses the forest for the trees. Neural networks aren't just another classifier—they mark the beginning of a fundamental shift in how we develop software. This shift is called Software 2.0.
We're familiar with Software 1.0—software written in traditional programming languages like Python or C++. It consists of explicit instructions given by programmers to computers. By writing each line of code, the programmer selects a specific point in program space that exhibits some desired behavior.
In contrast, Software 2.0 is developed using more abstract, human-unreadable languages—such as the weights inside neural networks. No one can directly write such code because it involves millions of parameters, and (I’ve tried) manually setting weights is practically impossible.
Instead, we specify objectives for what the program should do (e.g., “match input-output pairs from a dataset,” or “win the game of Go”), define a skeleton of the program (e.g., the architecture of a neural network), thereby defining a searchable subset within the entire program space, and then use our computational resources to search for viable programs within this space.
For neural networks, we constrain the search to a continuous subspace of program space and use backpropagation with stochastic gradient descent—a surprisingly effective search method.
More concretely, Software 1.0 compiles hand-designed source code (e.g., .cpp files) into efficient binary executables. In contrast, Software 2.0 source code typically has two components: 1) a dataset defining the desired behavior; and 2) a neural network architecture providing a rough structure that needs to be filled in. Training a neural network is the process of compiling the dataset into a binary—the final neural network. Today, most practical applications involve increasingly standardized neural network architectures and training systems, making them commodity-like. As a result, most active “software development” now revolves around organizing, expanding, tuning, and cleaning labeled datasets. This fundamentally changes our software iteration paradigm, splitting development teams into two groups: Software 2.0 programmers (data labelers) who edit and expand datasets, and a smaller group maintaining infrastructure for training, along with interfaces for analysis, visualization, and annotation.
It turns out that for many real-world problems, collecting data (or more broadly, specifying desired behaviors) is much easier than explicitly writing programs. Due to these advantages—and others I’ll discuss—we are witnessing a massive industrial shift from Software 1.0 to Software 2.0. Software 1.0 is eating the world, and Software 2.0 (AI) is eating Software 1.0.
The Shift Is Underway
Let’s look at concrete examples of this shift across different fields. Over recent years, we’ve abandoned attempts to solve complex problems by writing explicit code and instead moved toward Software 2.0.
-
Image Recognition: Image recognition used to rely heavily on feature engineering, with only a small amount of machine learning at the end (e.g., SVMs). Then, using larger datasets (like ImageNet) and searching over convolutional neural network architectures, we discovered more powerful visual features. Recently, we’ve even stopped trusting our hand-designed architectures and started automatically searching for optimal ones.
-
Speech Recognition: Traditional speech recognition involved extensive preprocessing, Gaussian Mixture Models, and Hidden Markov Models. Now, it’s almost entirely neural networks. There’s a famous quote related to this, attributed to Fred Jelinek in 1985: “Every time I fire a linguist, the performance of the speech recognizer improves.”
-
Speech Synthesis: Historically, speech synthesis relied on various concatenation techniques. Today, state-of-the-art large convolutional networks (like WaveNet) generate raw audio signals directly.
-
Machine Translation: Machine translation used to employ phrase-based statistical methods, but neural networks are rapidly taking over. My favorite architectures involve multilingual training—one model translating between any source and target language, often with weak or even fully unsupervised supervision.
-
Gaming: For a long time, people wrote chess-playing programs by hand. Now, AlphaGo Zero—a convolutional network observing only the raw board state—is the strongest Go player. I predict similar outcomes in other games like DOTA 2 and StarCraft.
-
Databases: Even traditionally non-AI systems show early signs of shifting toward Software 2.0. For example, in the “Learned Index Structures” paper, neural networks replace core database components, achieving up to 70% faster speeds than optimized B-trees while saving an order of magnitude in memory.
You may have noticed that many of the works linked above come from Google. That’s because Google is currently leading the charge in converting its own massive codebase into Software 2.0. The vision of “one universal model” aims to unify previously scattered statistical approaches across domains into a single system capable of understanding the world holistically.
Advantages of Software 2.0
Why are we moving complex programs to Software 2.0? Clearly, one answer is that it simply works better in practice. But there are many other compelling reasons. Let’s compare the benefits of Software 2.0 (represented by convolutional neural networks) against Software 1.0 (represented by production-grade C++ codebases):
-
Homogeneous Computation: A typical neural network combines only two operations: matrix multiplication and rectified linear units (ReLU). Compared to the diverse and heterogeneous instructions in traditional software, this simplicity makes correctness and performance verification significantly easier—only a few core routines (e.g., matrix multiplication) need to be implemented correctly in Software 1.0.
-
Chip-Friendly: Because neural networks require a relatively small set of operations, they’re easier to implement directly on hardware—using custom ASICs, neuromorphic chips, etc. When low-power intelligent devices become ubiquitous, the world will change dramatically. For instance, pre-trained convolutional networks, speech recognizers, and WaveNet synthesizers could run on cheap, tiny devices that connect to everything else.
-
Constant Runtime: Each forward pass of a typical neural network requires nearly constant computation (FLOPs). Unlike handwritten C++ code with unpredictable branching, Software 2.0 avoids such variability. Even with dynamic graphs, execution flow remains tightly constrained. As a result, we can almost guarantee no unexpected infinite loops.
-
Constant Memory Usage: Relatedly, without dynamic memory allocation, swapping with disk is rare and memory leaks are virtually nonexistent.
-
Highly Portable: A sequence of matrix multiplications is far easier to run across diverse computing environments than traditional binaries or scripts.
-
Agile Development: If you're writing C++ and someone asks you to double your development speed (at the cost of performance), adapting the system isn’t trivial. But in Software 2.0, you simply remove half the paths in the computation graph and retrain—you get slightly lower accuracy but twice the speed. Conversely, with more data and compute, you can immediately scale up the graph and retrain for better performance. This agility is magical.
-
End-to-End Optimization: Traditional software is broken into modules communicating via APIs or shared functions. In Software 2.0, even if two interacting modules were initially trained separately, we can easily perform backpropagation across the entire system. Imagine your browser automatically designing low-level instructions to speed up page loading, or your computer vision library (like OpenCV) adapting its behavior based on your specific data—this is standard operation in Software 2.0.
-
Better Than You: Finally and most importantly, in many verticals, neural networks produce code superior to anything you or I could write—especially today in areas involving images, video, and audio.
Disadvantages of Software 2.0
Software 2.0 also has drawbacks. After optimization, we obtain large networks that work well in practice but are difficult to interpret. In many domains, we must choose between a less accurate (90%) but interpretable model, or a highly accurate (99%) but opaque one.
Software 2.0 can make unintuitive, embarrassing errors—or worse, fail silently. For example, if biased data is quietly absorbed during training, analyzing the root cause becomes extremely difficult when dealing with million-sample datasets.
Finally, strange properties of Software 2.0 continue to emerge—such as adversarial and attack samples—which further highlight its lack of interpretability.
Programming in Software 2.0
Software 1.0 code is written by humans. Software 2.0 code emerges from optimization under evaluation criteria (e.g., “correctly classify training data”). Any program whose principles are unclear but whose performance can be repeatedly evaluated is a candidate for this shift, because optimization often finds solutions far superior to human-written code.
Our perspective on trends matters. Once you realize neural networks aren't just another handy classifier in the machine learning toolkit, but rather the emergence of a whole new programming paradigm—Software 2.0—the implications become obvious, and it’s clear there’s still much work ahead.
Specifically, we’ve built numerous tools to assist Software 1.0 developers—powerful IDEs with syntax highlighting, debuggers, profilers, symbol jumping, integrated git, and more. In Software 2.0, programming happens through accumulating, organizing, and cleaning datasets. For example, when a neural network fails in edge cases, we don’t fix it by writing code—we add more data representing those scenarios.
Who will build the first Software 2.0 IDE? It should support all workflows related to datasets: data collection, visualization, cleaning, labeling, and generation. Perhaps it flags images suspected of incorrect labels based on per-sample loss, assists labeling by predicting suggested tags, or recommends samples for labeling based on prediction uncertainty.
Similarly, GitHub succeeded enormously in the Software 1.0 era. Can we envision a GitHub for Software 2.0? Repositories would store datasets, and commits would consist of adding or editing data labels. Traditional package managers and deployment tools like pip, conda, and Docker made software installation and deployment easier. In the Software 2.0 era, how do we efficiently deploy, share, import, and run models? What is the equivalent of conda for neural networks?
In short, Software 2.0 will grow increasingly dominant in domains where evaluation is cheap and repeated, and where algorithms are hard to design explicitly. When we consider the entire development ecosystem and how to adapt to this new programming paradigm, many exciting opportunities arise. Long-term, this paradigm has a bright future—because it’s becoming increasingly clear that if we ever build AGI, it will be through Software 2.0.
Join TechFlow official community to stay tuned
Telegram:https://t.me/TechFlowDaily
X (Twitter):https://x.com/TechFlowPost
X (Twitter) EN:https://x.com/BlockFlow_News
Andrej Karpathy
