Last night, Google launched its "AI super suite," but its biggest innovation was preempted by OpenAI.
TechFlow Selected TechFlow Selected
Last night, Google launched its "AI super suite," but its biggest innovation was preempted by OpenAI.
OpenAI specifically chose to release GPT-4o one day before the Google I/O conference kicked off, perhaps aiming to counter Google.
Navigating Web3 tides with focused insightsAuthors: Zheng Yue, Xinxin
Editor: Zheng Xuan
As expected, Google has been targeted by OpenAI.
After a series of smoke screens, yesterday OpenAI stunned the world with an AI assistant called "GPT-4o" that can see, hear, and engage in real-time, zero-latency conversations about the real world. By deliberately scheduling the GPT-4o launch one day before Google's I/O keynote, media outlets immediately speculated that OpenAI was aiming directly at Google.
Sure enough, during Google’s I/O keynote held at 10 a.m. U.S. local time on the 14th, Google unveiled a suite of Gemini-powered “AI products”—including Gemini 1.5 Pro with support for 2 million tokens context, the new lightweight model Gemini 1.5 Flash, a Sora-like video generation model Veo, as well as multiple AI applications such as AI-enhanced search and AI-integrated Gmail.
However, the most anticipated announcements were Project Astra—the so-called universal assistant toward AGI, described by Demis Hassabis, head of Google DeepMind and leader of Google AI—and Live, a new voice chat feature for Gemini Advanced subscribers. Live is expected to gain camera functionality later this year, enabling AI to converse based on users’ real-world surroundings.
Both products bear strong resemblance to GPT-4o. While it remains to be seen how they compare in actual performance, from a product development standpoint, OpenAI currently holds the lead. That said, OpenAI isn’t guaranteed to win. For now, smartphones remain the most critical hardware platform for these super-smart assistants, and Google, with control over the Android ecosystem, enjoys a natural advantage. In this light, recent rumors of collaboration between Apple and OpenAI may stem precisely from their shared interest in countering Google’s dominance.
Regardless, super-intelligent assistants capable of “real-world perception (input)” and “low-latency speech output” have become the new battleground in the AI arms race. Going forward, more large-model companies, cloud platforms, smartphone makers, and AI application developers will be drawn into the competition.
01. Project Astra—Real-Time Video Q&A, Swearing It’s Not Faked
At I/O, Google released an incredibly impressive demo video of its AI assistant—and swore this time it hadn’t been forged or altered in any way.
Demis Hassabis said Project Astra represents the prototype of the AI assistant he’s dreamed of for decades—the future of AI assistants.
Project Astra is a real-time, multimodal AI assistant that interacts with the world by receiving information, remembering what it sees, processing data, and understanding contextual details. Its voice interaction is more natural than current versions of Google Assistant, with no delay or latency, allowing it to answer questions or help with nearly anything.
In the demo, when asked to identify the source of a sound, the assistant promptly responded: “I see a speaker producing the sound.”
The interaction occurred via real-time voice without wake words like “Hey Google” or “Gemini.” When the user asked what code shown on a monitor did, Project Astra explained instantly, with zero lag.
Project Astra remembers fleeting glasses | Image Source: Google
Project Astra’s visual processing might be the highlight of the entire event. When asked, “Do you remember where I put my glasses?” it replied, “Your glasses are next to the red apple on the table.” It even remembered the glasses briefly passing through the camera frame and accurately recalled their location. There may be some hidden message here—perhaps those glasses also run Project Astra.
From the demo, Project Astra processes incoming information rapidly, using continuous video frame encoding and combining audio and video into a timeline of events, caching data for later recall. Google says some of these capabilities will be added to the Gemini app and other products later this year.
Project Astra names this band Golden Stripes | Image Source: Google
Google says the DeepMind team is still researching optimal ways to integrate multimodal models and balance ultra-large general models with smaller, more focused ones.
Yesterday, OpenAI’s release of GPT-4o also marked a major breakthrough in multimodality—making it feel like déjà vu. The key difference lies in visual processing: while GPT-4o currently handles only static images, Astra can already process video.
Underlying Project Astra, the Gemini series of large models also received updates. At the Google Cloud Next 2024 event last month, Google launched Gemini 1.5 Pro with native audio understanding, system instructions, JSON mode, and support for up to 1 million tokens of context, now open to developers worldwide.
Now introducing Gemini 1.5 Flash, which addresses a key cost issue. Positioned between Gemini 1.5 Pro and Gemini 1.5 Nano, Flash targets developers. Pricing details were shared: Gemini 1.5 Flash costs $0.35 per million tokens—significantly cheaper than GPT-4o’s $5 per million tokens.
Gemini 1.5 Flash | Image Source: GeekPark
Google also previewed upcoming upgrades, announcing plans to double the context window to 2 million tokens later this year—enabling simultaneous processing of 2 hours of video, 22 hours of audio, over 60,000 lines of code, or more than 1.4 million words.
Gemini 1.5 Pro | Image Source: GeekPark
Last December, Google faced accusations of “faking” a pre-recorded demo that misleadingly exaggerated Gemini’s video processing capabilities through editing. But now, these capabilities are real.
02. Progress in AI Application Ecosystem Based on Large Models
At this I/O conference, Google also upgraded its AI application ecosystem built on large models, covering search, photos, content creation, tools, and workplace applications.
Search was Google’s foundational product 25 years ago. A year ago, Google declared the future of search is AI. Now, AI-powered search has truly arrived—with the slogan: “Let Google do the Googling for you.”
CEO Sundar Pichai announced at I/O that AI-generated search summaries, now called “AI Overviews,” will roll out to all users in the U.S. “this week,” with expansion to more countries and regions soon.
Compared to before, AI Search can handle more complex queries. Google gave an example: if a user searches for a new yoga studio that is locally popular, convenient for commuting, and offers discounts, AI Search can deliver results in a single query.
Additionally, users can adjust generated AI summaries, opt for simplified versions, or choose to view more details.
This is powered by custom Gemini models that combine multi-step reasoning, planning, multimodality, and search systems to summarize web content and display answers—reportedly with AI even designing and populating the result page layout.
But here’s a concern: if AI curates search results for users, what happens to Google’s paid advertising business?
According to Google executives, links within AI summaries receive higher click-through rates compared to traditional web listings. “Ads will continue to appear in dedicated spots on the page, clearly labeled to distinguish organic and sponsored results.”
Beyond answering questions, Google’s AI Search can create plans—from meal plans to travel itineraries. For example, searching “create an easy-to-prepare 3-day group meal plan” triggers AI to generate a customized plan. Users can then modify a dinner option to vegetarian and export the plan to a document or Gmail.
Search isn’t limited to text input anymore. Google’s visual search has evolved to accept video queries. For instance, if a user has a secondhand record player with a misaligned needle but doesn’t know the correct terminology, they can simply use video search to get an AI summary with step-by-step solutions and relevant links—saving time and effort in describing the problem.
Using Google AI Search to create a meal plan | Image Source: Google
Google Photos is also evolving. Sundar Pichai noted that nine years ago they launched this app, and now over 6 billion photos and videos are uploaded daily.
In the coming months, it will introduce a new “Ask Photos” feature powered by Gemini, enabling users to search people, pets, locations, and more in photos and videos using natural language. It understands context and themes, retrieving specific memories without endless scrolling.
For example: “Show me the best photo from each national park I’ve visited,” “Where did I camp last year?” “When does my coupon expire?” “When did Lucia learn to swim?” “Show Lucia’s swimming progress,” “What was the theme of Lena’s birthday party?” Finally, it can even draft travel highlights or personalized captions for social media sharing.
Google emphasized: “Unless in rare cases involving abuse or harm prevention, we do not use personal data from Google Photos to train any generative AI products outside of Google Photos—including other Gemini models and services.”
Using Ask Photos to inquire about birthday party details | Image Source: Google
On the creative front, Google unveiled VideoFX—an AI video generation tool—at I/O, alongside updates to its AI image and music tools, ImageFX and MusicFX.
VideoFX allows filmmakers to write prompts to build cinematic shots. ImageFX adds image editing controls, while MusicFX introduces a new DJ mode.
Powered by Veo, Google DeepMind’s most advanced video generation model, VideoFX includes a storyboard mode for scene-by-scene iteration and adding music to final videos.
According to Google executives, Veo understands cinematic terms like “time-lapse,” generates 1080p videos in various styles, and supports clips longer than one minute. Veo will be available via waitlist for early preview.
Notably, all content created by VideoFX, ImageFX, and MusicFX is watermarked with SynthID—a digital signature designed to prevent misuse of AI content and curb misinformation.
Using Google’s AI video tool VideoFX | Image Source: Google
Beyond creative tools, Google’s evolution in workplace apps is also noteworthy. The latest Gemini 1.5 Pro model will be integrated into Google Workspace apps—including Gmail, Calendar, Docs, Sheets, Slides, and Drive—as a sidebar virtual assistant.
Clicking the Gemini icon in the sidebar brings up options like “Summarize this email,” “List next steps,” or “Suggest reply.” Users can ask it to summarize school emails from their child, extract action items, and draft responses.
Gemini summarizing emails in Gmail sidebar | Image Source: Google
Meanwhile, in yesterday’s GPT-4o demo by OpenAI, there was an educational scenario where GPT-4o acted as a tutor, guiding a teenager step-by-step through solving a geometry problem.
Google also announced LearnLM—a set of fine-tuned models based on Gemini, specifically designed for education—alongside demos of AI learning features in Search and YouTube.
For example, on YouTube, users can ask questions while watching videos or follow up with deeper inquiries. On Android, Circle to Search lets users circle math or physics problems to get instant help with difficult concepts.
Asking questions about a YouTube learning video | Image Source: Google
In this AI arms race, one of Google’s key advantages in catching up to OpenAI is its ownership of the Android operating system and vast app ecosystem.
But as generative AI fully integrates into Google’s ecosystem—taking over search, helping recall memories, planning trips, replacing human tutors—how reliable will it truly be? We’ll have to wait and see.
Join TechFlow official community to stay tuned
Telegram:https://t.me/TechFlowDaily
X (Twitter):https://x.com/TechFlowPost
X (Twitter) EN:https://x.com/BlockFlow_News
极客公园
