Multimodal Live API demo: GenExplainer

Google for Developers · Intermediate ·🧠 Large Language Models ·1y ago

Key Takeaways

Demonstrates the Multimodal Live API with GenExplainer using Gemini 2.0 Flash

Full Transcript

hi I'm Trudy and this is a demo of the multimodal live API which lets you use real time two-way streaming in your apps all powered by Gemini and the easiest way to try this out is in AI studio so we can go ahead and talk to Gemini can you give me a super quick poem about coding sure here's a short poem about coding lines of logic a digital art crafting with code from start to start okay that was pretty nice and just to be clear AI studio is where you can try out the multimodal live API but now I'm going to show you how you can start building with it so this is Gen explainer which is a boiler plate boilerplate template for you as a developer to clone and modify um so that you can start building with the multimodal live API and the way that it works is we can ask the Modo to explain any Topic in the style of anything else so let's explain black holes in the style of a cooking metaphor okay I can do that imagine a black coal is like a cosmic oven it has so much gravity that it sucks everything in like ingredients thrown into a mixing bowl not even light the fastest thing we know of can escape once it gets too close it's like the oven is so hot that nothing can get away from the heat the point of no return is called The Event Horizon anything that crosses it is done for okay and then we can actually pick uh another style maybe a sports commentator all right I'm ready to switch things up so picture this a black hole is like the ultimate star player but instead of scoring points it's all about sucking in matter it's got a gravitational pole that's out of this world like the strongest defense in the league this thing has an area called The Event Horizon and once you're past it there is no turning back it's game over the black hole just swallows everything around it and in the boilerplate code you can see this is where we actually send the request to the model which you can actually come in and change and modify you want so whenever we update the topic or the style we just send a message to the client and say explain the new topic in the style of the new style and don't change the style until I ask you to to start building with the multimodal live API visit ai. gooogle dodev Gemini API or visit the Google Gemini GitHub page

Original Description

The Multimodal Live API lets developers build real-time multimodal applications powered by Gemini 2.0 Flash. In this starter demo, you can pick any topic, then have it explained to you in a variety of fun styles through real-time conversation. Start building with Gemini 2.0 at ai.google.dev. Read our Gemini 2.0 developer blog post→https://goo.gle/3Bl23dQ Get the code for this demo at → https://goo.gle/3ZOoUbd Subscribe to Google for Developers → https://goo.gle/developers Products Mentioned: Gemini
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Google for Developers · Google for Developers · 0 of 60

← Previous Next →
1 Developer Journey - Sunnyvale DSC Summit ‘19
Developer Journey - Sunnyvale DSC Summit ‘19
Google for Developers
2 How Google is working with students - Sunnyvale DSC Summit ‘19
How Google is working with students - Sunnyvale DSC Summit ‘19
Google for Developers
3 Starting your career in the Cloud - Sunnyvale DSC Summit ‘19
Starting your career in the Cloud - Sunnyvale DSC Summit ‘19
Google for Developers
4 The Solution Challenge  - Sunnyvale DSC Summit ‘19
The Solution Challenge - Sunnyvale DSC Summit ‘19
Google for Developers
5 Firebase - Sunnyvale DSC Summit ‘19
Firebase - Sunnyvale DSC Summit ‘19
Google for Developers
6 Cloud Hero - Sunnyvale DSC Summit ‘19
Cloud Hero - Sunnyvale DSC Summit ‘19
Google for Developers
7 Panel discussion  - Sunnyvale DSC Summit ‘19
Panel discussion - Sunnyvale DSC Summit ‘19
Google for Developers
8 The art of negotiation - Sunnyvale DSC Summit ‘19
The art of negotiation - Sunnyvale DSC Summit ‘19
Google for Developers
9 Courage to care, solve and share - Sunnyvale DSC Summit ‘19
Courage to care, solve and share - Sunnyvale DSC Summit ‘19
Google for Developers
10 Version 9 of Angular, Glass Enterprise Edition 2, path to DX deprecation, & more!
Version 9 of Angular, Glass Enterprise Edition 2, path to DX deprecation, & more!
Google for Developers
11 [DEPRECATING] Introducing a new series (Assistant for Developers Pro Tips)
[DEPRECATING] Introducing a new series (Assistant for Developers Pro Tips)
Google for Developers
12 Detecting memory bugs with HWASan, Bazel 2.1, Next ‘20 session guide, & more!
Detecting memory bugs with HWASan, Bazel 2.1, Next ‘20 session guide, & more!
Google for Developers
13 Why Podcast.app chose a .app domain name
Why Podcast.app chose a .app domain name
Google for Developers
14 Machine Learning Bootcamp Jakarta 2019
Machine Learning Bootcamp Jakarta 2019
Google for Developers
15 Android Studio 3.6, Android 11 Developer Preview, Kubeflow 1.0, & more!
Android Studio 3.6, Android 11 Developer Preview, Kubeflow 1.0, & more!
Google for Developers
16 [DEPRECATING]  Importance of community (Assistant on Air)
[DEPRECATING] Importance of community (Assistant on Air)
Google for Developers
17 Why the Flutter team switched from .io to a .dev domain name
Why the Flutter team switched from .io to a .dev domain name
Google for Developers
18 3 website-building tips from .dev creators
3 website-building tips from .dev creators
Google for Developers
19 Why NimbleDroid chose a .app domain name
Why NimbleDroid chose a .app domain name
Google for Developers
20 Android Platform Codelab, Bazel 2.2, Maps Android Utility Library v1.0, & more!
Android Platform Codelab, Bazel 2.2, Maps Android Utility Library v1.0, & more!
Google for Developers
21 Google for Games Developer Summit: A free, digital experience for game developers
Google for Games Developer Summit: A free, digital experience for game developers
Google for Developers
22 Inspecting Home Graph (Assistant for Developers Pro Tips)
Inspecting Home Graph (Assistant for Developers Pro Tips)
Google for Developers
23 Google for Games Developer Summit Keynote
Google for Games Developer Summit Keynote
Google for Developers
24 Stadia Games & Entertainment presents: Keys to a great game pitch (Google Games Dev Summit)
Stadia Games & Entertainment presents: Keys to a great game pitch (Google Games Dev Summit)
Google for Developers
25 Empowering game developers with Stadia R&D (Google Games Dev Summit)
Empowering game developers with Stadia R&D (Google Games Dev Summit)
Google for Developers
26 Supercharging discoverability with Stadia (Google Games Dev Summit)
Supercharging discoverability with Stadia (Google Games Dev Summit)
Google for Developers
27 Stadia Games & Entertainment presents: Creating for content creators (Google Games Dev Summit)
Stadia Games & Entertainment presents: Creating for content creators (Google Games Dev Summit)
Google for Developers
28 Bringing Destiny to Stadia: A postmortem (Google Games Dev Summit)
Bringing Destiny to Stadia: A postmortem (Google Games Dev Summit)
Google for Developers
29 Live Captioning in Google Slides
Live Captioning in Google Slides
Google for Developers
30 [DEPRECATING]  User engagement for the Google Assistant
[DEPRECATING] User engagement for the Google Assistant
Google for Developers
31 TensorFlow Dev Summit ‘20, Google for Games Dev Summit, Cloud AI Platform Pipelines, & much more!
TensorFlow Dev Summit ‘20, Google for Games Dev Summit, Cloud AI Platform Pipelines, & much more!
Google for Developers
32 Top 5 from the TensorFlow Dev Summit 2020
Top 5 from the TensorFlow Dev Summit 2020
Google for Developers
33 Developer Student Clubs 2019 Turkey Leads Summit
Developer Student Clubs 2019 Turkey Leads Summit
Google for Developers
34 Building simpler payment experiences | Google Pay Plugin for Magento 2
Building simpler payment experiences | Google Pay Plugin for Magento 2
Google for Developers
35 Become A Developer Student Club Lead
Become A Developer Student Club Lead
Google for Developers
36 Firebase Kotlin Extensions, ARM apps on the Android Emulator, Angular v9.1, & more!
Firebase Kotlin Extensions, ARM apps on the Android Emulator, Angular v9.1, & more!
Google for Developers
37 Test suite for Smart Home (Assistant for Developers Pro Tips)
Test suite for Smart Home (Assistant for Developers Pro Tips)
Google for Developers
38 Google Play updates, Bazel 3.0, Business Console for Google Pay, & more!
Google Play updates, Bazel 3.0, Business Console for Google Pay, & more!
Google for Developers
39 How to use error logs (Assistant for Developers Pro Tips)
How to use error logs (Assistant for Developers Pro Tips)
Google for Developers
40 Contact Center AI, Android Studio 4.1 Canary 5, TensorFlow QAT API, & more!
Contact Center AI, Android Studio 4.1 Canary 5, TensorFlow QAT API, & more!
Google for Developers
41 WebView DevTools, Kotlin meets gRPC, Flutter CodePen support, & more! (Episode 200)
WebView DevTools, Kotlin meets gRPC, Flutter CodePen support, & more! (Episode 200)
Google for Developers
42 Offline handling for Smart Home (Assistant for Developers Pro Tips)
Offline handling for Smart Home (Assistant for Developers Pro Tips)
Google for Developers
43 Android 11 Dev Preview 3, Google Fonts for Flutter, Shielded VM, & more!
Android 11 Dev Preview 3, Google Fonts for Flutter, Shielded VM, & more!
Google for Developers
44 Machine Learning Foundations: Ep #1 - What is ML?
Machine Learning Foundations: Ep #1 - What is ML?
Google for Developers
45 Flutter web support updates, BigQuery materialized views, Cloud Spanner emulator, & more!
Flutter web support updates, BigQuery materialized views, Cloud Spanner emulator, & more!
Google for Developers
46 Computer vision by building a neural network with TensorFlow | Machine Learning Foundations
Computer vision by building a neural network with TensorFlow | Machine Learning Foundations
Google for Developers
47 Machine Learning Foundations: Ep #3 - Convolutions and pooling
Machine Learning Foundations: Ep #3 - Convolutions and pooling
Google for Developers
48 Android 11 Beta plans, Flutter 1.17, Dart 2.8, & much more!
Android 11 Beta plans, Flutter 1.17, Dart 2.8, & much more!
Google for Developers
49 Machine Learning Foundations: Ep #4 - Coding with Convolutional Neural Networks
Machine Learning Foundations: Ep #4 - Coding with Convolutional Neural Networks
Google for Developers
50 Google Developers ML Summit
Google Developers ML Summit
Google for Developers
51 Real-world image classification using convolutional neural networks | Machine Learning Foundations
Real-world image classification using convolutional neural networks | Machine Learning Foundations
Google for Developers
52 Adobe XD support for Flutter, Architecture Framework, temporary closures with Places API, & more!
Adobe XD support for Flutter, Architecture Framework, temporary closures with Places API, & more!
Google for Developers
53 Machine Learning Foundations: Ep #6 - Convolutional cats and dogs
Machine Learning Foundations: Ep #6 - Convolutional cats and dogs
Google for Developers
54 Machine Learning Foundations: Ep #7 - Image augmentation and overfitting
Machine Learning Foundations: Ep #7 - Image augmentation and overfitting
Google for Developers
55 Announcing Firebase Live, Flutter Day, Java 11 on Google Cloud Functions, & more!
Announcing Firebase Live, Flutter Day, Java 11 on Google Cloud Functions, & more!
Google for Developers
56 Machine Learning Foundations: Ep #8 - Tokenization for Natural Language Processing
Machine Learning Foundations: Ep #8 - Tokenization for Natural Language Processing
Google for Developers
57 Android 11 Beta, Google Play Asset Delivery, Firebase Crashlytics SDK, & much more!
Android 11 Beta, Google Play Asset Delivery, Firebase Crashlytics SDK, & much more!
Google for Developers
58 Natural Language Processing: Using sequencing APIs in TensorFlow | Machine Learning Foundations
Natural Language Processing: Using sequencing APIs in TensorFlow | Machine Learning Foundations
Google for Developers
59 Build a sarcasm classifier using NLP and TensorFlow | Machine Learning Foundations
Build a sarcasm classifier using NLP and TensorFlow | Machine Learning Foundations
Google for Developers
60 AR Realism with the ARCore Depth API
AR Realism with the ARCore Depth API
Google for Developers

Related Reads

📰
Enterprise AI Agency strategy
Learn how to implement a comprehensive Enterprise AI Agency strategy using cloud-native infrastructure and scalable data pipelines to drive AI-driven decision-making
Dev.to AI
📰
Apple Intelligence approved for launch in China with Alibaba’s Qwen AI
Apple's AI services are launching in China through a partnership with Alibaba, bringing Qwen AI models to Apple's operating systems
TechCrunch AI
📰
The AI Jackpot: Why Prompting Feels Like Gambling
Learn why prompting AI models can feel like gambling and how to improve your odds of getting useful results
Dev.to · Brendon O'Neill
📰
Bonsai-27B & Ternary-Bonsai-27B - Updates (on PRs)
Learn about updates to Bonsai-27B and Ternary-Bonsai-27B models, including upstream status and runtime support
Reddit r/LocalLLaMA
Up next
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)
Watch →