Sequoia Capital In-Depth Analysis of Large Language Models: How Enterprises Can Make AI Applications a Reality
TechFlow Selected TechFlow Selected
Sequoia Capital In-Depth Analysis of Large Language Models: How Enterprises Can Make AI Applications a Reality
This analysis is extremely worth paying attention to for investors, entrepreneurs, and startups alike.
Navigating Web3 tides with focused insights
Sequoia Capital, one of the world's leading venture capital firms, has consistently been at the forefront of technological innovation thanks to its exceptional investment insights and extensive industry experience. Recently, Sequoia released an in-depth analysis on large language models (LLMs), exploring how LLMs are driving enterprise innovation, giving rise to new technology stacks, and gradually merging trends of open-source development with custom model training.
This analysis is highly valuable for investors, entrepreneurs, and founders alike. For investors, it reveals which industries and specific technologies may experience rapid growth in the future. For entrepreneurs and founders, the report highlights the rise of LLMs and the growing trend toward customization—offering a fresh perspective on how natural language interactions can be leveraged to innovate products.
Below is the summary and full translation prepared by our AI assistant, ChatGPT:
Summary:
Innovation through Large Language Models (LLMs): LLMs like ChatGPT are fueling innovation across industries—from marketing and law to search—with more and more companies integrating natural language interaction into their products.
Emergence of New Tech Stacks: The new technology stack for these applications includes language model APIs, retrieval mechanisms, and orchestration frameworks like LangChain. There is also a growing trend toward open-source adoption and custom model training, as companies increasingly seek to tailor models to their specific needs.
Customization of LLMs: Companies customize LLMs by training models from scratch, fine-tuning foundational models, or using pre-trained models combined with retrieval of relevant context. This enables more precise and contextually relevant natural language interactions based on proprietary data.
Convergence of Tech Stacks: The LLM API stack and the custom model training stack are expected to merge over time, as interest grows in training and fine-tuning proprietary models. This convergence will likely offer greater flexibility and adaptability for language model applications.
Improved Developer Accessibility: Tools like LangChain are making the tech stack increasingly accessible to general developers—not just machine learning teams. These tools help solve common challenges, avoid vendor lock-in, and foster a more inclusive and diverse developer community.
Full Translation:
How Enterprises Are Bringing AI Applications to Life
ChatGPT has unleashed a massive wave of innovation, accelerating the development of large language models (LLMs). More and more enterprises are integrating the power of natural language interaction into their products. The adoption of language model APIs has sparked a revolution in new technology stacks. To better understand the types of applications being built and the technical stacks being used, we spoke with 33 companies within the Sequoia network—ranging from early-stage startups to large public companies. We first engaged them two months ago and followed up last week to fully capture the rapid pace of change in this space.
Given that many founders and developers are still formulating their own AI strategies, we hope to share our findings—even as this field continues to evolve rapidly.
1. Nearly every company in the Sequoia network is integrating language models into their products
We’ve seen magical code autocompletion features (e.g., Sourcegraph, Warp, GitHub) and auto-complete capabilities in data science (e.g., Hex). We’ve witnessed significantly improved chatbots, not only for customer support and employee assistance but also for consumer entertainment.
Other companies are reimagining entire workflows from an AI-first perspective, spanning visual art (e.g., Midjourney), marketing (e.g., HubSpot, Attentive, Drift, Jasper, Copy, Writer), sales (e.g., Gong), contact centers (e.g., Cresta), legal (e.g., Ironclad, Harvey), accounting (e.g., Pilot), productivity tools (e.g., Notion), data engineering (e.g., dbt), search (e.g., Glean, Neeva), grocery shopping (e.g., Instacart), consumer payments (e.g., Klarna), and travel planning (e.g., Airbnb). These are just a few examples—and they’re only the beginning.
2. The new tech stack for these applications centers around language model APIs, retrieval, and orchestration—but open-source usage is also growing
-
In our sample, 65% of companies are now deploying applications in production, up from 50% two months ago; the rest remain in experimentation mode.
-
94% of companies are using foundational model APIs. Among them, OpenAI’s GPT is clearly the most popular, used by 91%, while Anthropic has seen significant interest over the past quarter, reaching 15% (some companies use multiple models).
-
88% of companies believe retrieval mechanisms (e.g., vector databases) will remain a key part of their tech stack. By providing contextual information to models, retrieval helps improve output quality, reduce “hallucinations” (inaccuracies), and address data freshness issues. Some companies use specialized vector databases (e.g., Pinecone, Weaviate, Chroma, Qdrant, Milvus), while others rely on pgvector or AWS offerings.
-
38% of companies are exploring LLM orchestration and application development frameworks like LangChain. Some use them for prototyping, others in production. Adoption has increased over recent months.
-
Fewer than 10% are actively seeking tools to monitor LLM outputs, costs, performance, or conduct A/B testing. As more large enterprises and regulated industries adopt language models, we expect interest in these areas to grow.
-
A small number of companies are experimenting with complementary generative technologies—such as combining generated text and speech. We also see this as a promising growth area.
-
15% of companies are building custom language models from scratch or using open-source tools, often alongside LLM APIs. Custom model training has seen notable growth in recent months. This requires assembling their own tech stack using compute resources, model libraries, hosting services, training frameworks, and experiment tracking tools from widely used providers such as Hugging Face, Replicate, Foundry, Tecton, Weights & Biases, PyTorch, and Scale.
After speaking with each practitioner, we found that AI is evolving so quickly that confidence in any final tech stack remains low. However, there is consensus that LLM APIs will remain a core pillar, followed by retrieval mechanisms and development frameworks like LangChain. Training and tuning of open-source and custom models also appear to be on the rise. Other parts of the stack are important but less mature.
3. Enterprises want to customize language models to their unique environments
While general-purpose language models are powerful, they often fail to distinguish or meet the needs of many specific use cases. Enterprises want to enable natural language interactions over their own data—such as developer documentation, product inventory, HR or IT policies. In some cases, they also want to personalize models based on user-specific data—like personal notes, design layouts, data metrics, or code repositories.
Currently, there are three main approaches to customizing language models (for deeper technical explanations, see Andrej Karpathy’s recent “State of GPT” talk at Microsoft Build):
-
Training a custom model from scratch: Highest difficulty. This is the classic—and most challenging—approach. It typically requires skilled ML scientists, large volumes of domain-specific data, training infrastructure, and computational resources. This is one major reason why NLP innovation historically happened primarily within large tech companies. BloombergGPT is a great example of a custom model project executed outside big tech, leveraging resources from Hugging Face and other open-source tools. As open-source tools improve and more companies creatively leverage LLMs, we expect to see increased use of custom and pre-trained models.
-
Fine-tuning a base model: Moderate difficulty. This involves updating the weights of a pre-trained model using additional proprietary or domain-specific data. Open-source innovations are making this approach more accessible, though it still usually requires a high-level team. Some practitioners privately admit that fine-tuning is harder than it sounds and can lead to unintended consequences—such as model drift or “breaking” other model capabilities without warning. While this method is likely to become more common, it remains out of reach for most companies today—though this is changing rapidly.
-
Using a pre-trained model with retrieval of relevant context: Lowest difficulty. Many people assume they need a fine-tuned model, when in fact they simply want the model to reason over their information at the right time. There are many ways to provide the correct information at the right moment: issuing structured queries to SQL databases, searching product catalogs, calling external APIs, or using embedding-based retrieval. Embedding-based retrieval excels at enabling natural language search over unstructured data. Technically, this works by converting data into embedding vectors and storing them in a vector database. When a query occurs, the system searches these embeddings to find the most relevant context and feeds it to the model. This approach helps overcome the model’s limited context window, is cost-effective, solves data freshness issues (e.g., ChatGPT doesn’t know about events after September 2021), and can be implemented by a single developer without formal ML training. Vector databases are useful because, at scale, they simplify storage, search, and updates of embeddings. So far, we observe that larger enterprises tend to rely on their enterprise cloud agreements and use tools provided by cloud vendors, while startups prefer purpose-built vector databases. However, this space is evolving rapidly. Context windows are growing larger (OpenAI recently expanded to 16K; Anthropic launched a model with a 100K-token context window). Foundational models and cloud databases may soon embed retrieval capabilities directly into their services. We’re closely watching this market.
4. Currently, the tech stack for LLM APIs and the stack for custom model training may feel separate—but they will gradually converge over time
Sometimes, it feels like there are two parallel stories: one centered on the LLM API stack (more closed-source, developer-focused), and another focused on custom language model training (more open-source, traditionally requiring complex ML teams).
Some have questioned whether easy access to LLMs via APIs means companies will do less custom training. But so far, the opposite is true. As interest in AI grows and open-source development accelerates, more companies are becoming interested in training and fine-tuning their own models.
We believe the LLM API and custom model training stacks will gradually converge. For instance, a company might train its own language model from open-source components but use a vector database for retrieval to address data freshness. Meanwhile, smart startups building tools for the custom model stack are expanding their offerings to align more closely with the LLM API revolution.
5. The tech stack is becoming increasingly developer-friendly
Language model APIs have made powerful off-the-shelf models accessible to general developers—not just ML teams. Now, the group of developers working with LLMs has expanded dramatically, encompassing all developers. We expect to see even more developer-centric tools emerge.
For example, LangChain helps developers build LLM applications by abstracting away common problems: composing models into higher-level systems, chaining multiple model calls, connecting models to tools and data sources, building agents that can operate those tools, and simplifying the process of switching between language models to avoid vendor lock-in. Some use LangChain for prototyping, while others continue using it in production.
6. For broad adoption, language models need to become more reliable—in terms of output quality, data privacy, and security
Before fully integrating LLMs into their applications, many companies want better tools to manage data privacy, isolation, security, copyright, and monitoring of model outputs. Regulated industries—from fintech to healthcare—are especially concerned and say it’s difficult to find software solutions that adequately address these issues (a promising area for startups). Ideally, software should flag—or even prevent—the generation of incorrect, hallucinated, discriminatory, or harmful content.
Some companies are also concerned about how data shared with models is used for training: for example, few realize that ChatGPT Consumer data is, by default, used for training, whereas ChatGPT Business and API data are not. As policies become clearer and stronger safeguards are put in place, language models will gain greater trust—and we may witness another leap in adoption.
7. LLM applications will become increasingly multimodal
Companies are already discovering compelling ways to combine multiple generative models: chatbots that blend text and voice generation create entirely new levels of conversational experiences. Text and voice models can work together to help you quickly correct mistakes in video recordings without re-recording the entire clip.
Models themselves are also becoming more multimodal. We can envision rich consumer and enterprise AI applications in the future that combine text, speech/audio, and image/video generation to deliver more engaging user experiences and accomplish more complex tasks.
8. We’re still in the early stages
AI has only just begun to permeate every aspect of technology. In our survey, only 65% of companies are in production—and many of these are relatively simple applications. As more enterprises launch LLM-powered apps, new challenges will emerge, creating opportunities for founders. The infrastructure layer will continue to evolve rapidly over the next few years.
If even half of the demos we've seen successfully translate into real products, we're in for an exciting journey. It's inspiring to see founders—from our earliest Arc investments to companies like Zoom—all focused on the same mission: delighting users through AI.
Join TechFlow official community to stay tuned
Telegram:https://t.me/TechFlowDaily
X (Twitter):https://x.com/TechFlowPost
X (Twitter) EN:https://x.com/BlockFlow_News
