Google Lied To Us

Key Takeaways

nerds Google just released Gemini which is a competitor to chat GPT and apparently one of the videos that they demoed it may have been faked let's check it out all right testing Gemini here we go tell me what you see I see you placing a piece of paper on the table I see a squiggly line what about now the contour lines are smooth and flowing with no Sharp angles or Jagged edges it looks like a bird to me so Devon TechCrunch released this article which a lot of the things that I'll be providing this video or from this article problem is that video was created after prompting Gemini and generating all those responses specifically as Pary also noted it was not carried out in real time or in voice the model was shown still images from video footage and human prompts narrated afterwards per a spokesperson and if we go back to that video and and looked inside the description they do have a link to where they say and Show an example of the prompting that it provided so they're not straight out lying about it but they're not exactly being clear so let's look at a few examples that Devon called out I know what you're doing you're playing rock paper scissors the actual prompt provided was of three still images one of a paper rock and scissors and then impr prompted it what do you think I'm doing hint it's a game Gemini then responded you're playing rock paper scissors and they quoted the model as saying I know what you're doing you're playing rock paper scissors which with this title of Hands-On with Gemini leads you to believe that they're interacting with this model in real time as they're displaying all this the other example Deon brought up was regarding this one and the solar system is this the right order no the correct order is sun earth Saturn so really they not only prompted it is this the right order they also had to add consider the distance from the Sun and explain your reasoning and then Gemini responded no the correct order is sun earth Saturn this clip in general made me feel like Gemini was very conversational like you're talking to a friend and now reading it you can actually see that more of talking to a chatbot and the last example is this one you were trying to get me to find the paper ball under the cup I accept the challenge the cup to the left nice so that one was just blatantly misleading because it made it sound like Gemini actually predicted what game it was trying to get it to play so instead they actually had to prompt it explaining the game and then from there it provided still images not in real time of each of the different movements and then finally at the end it says hey tell me where the ball is and summarize how it got there I don't know of all the different examples they showed that one was the cringiest as not only did Gemini not actually guess what the game was but they also Al weren't able to actually play it in a real-time manner as shown now one of the VPS at Deep mine the subsidiary behind Google that developed Gemini released this yesterday after that video and he went on to clarify that we gave Gemini a sequence of difference modalities images and text in this case and he added this damning statement the video illustrates what the multimodal user experience built with gini could look like I'm not going to lie I think Google kind of up in this case and specifically although they have this disclaimer at the beginning it's not exactly clear like even me watching this the first time and that has understanding of large language models thought that this was some sort of demonstration in real time that they were actually able to put together now the other major issue that some of my subscribers have pointed out is that they a lot of the demonstrations that they showed the other day was using the ultra model and this model is not going to be available until early next year as they put it we're currently completing extens of trust and safety Checks In including red teaming by trusted external parties and further refining the model using fine-tuning and reinforcement learning from Human feedback which is great in all that they're doing all these checks but in this example here where they were going over competitive programming they mentioned how it was actually a problem with scaling using Gemini Ultra for actual real world applications so I think there's a lot more going behind the scenes with this anyway as always got value out this video smash that like button with that I'll see you in the next one