Watch a Critical AI Failure

Key Takeaways

Hello community. So glad that you are back. Today I recorded a systemic failure of an AI but it started with some complete different topic. You ask me hey what is the state of AI today at the end of July 2025. So I went to work and I asked my AI. Now the first thing everybody would do today is you go to your beloved AI and you say hey write a deep technical report on the last development of the state of AI research within the last 3 weeks. So everything that happened in July 2025 and you tell you be precise be explanatory and build an interesting story line. So absolutely clear the state of AI research within the last 3 weeks. Now we assume that now every information is given but have a look what happened. And here on the right hand side you see I have my Gemini 2.5 Pro. It only took 7,000 tokens. Temperature is a little bit on the creative side. I have set my sinking budget here almost to max and I have grounding with Google search. So I say everything that I see want to be cross checked with Google search to be absolutely on the safe side. Now, let's have a look. And here we have it. July 2025. And I record this on July 23rd, 2025. A pivotal month in AI as embodied agent begin to bridge the chasm between simulation and reality. The first three weeks of 2025 will likely be remembered as a watershed moment in the advancement of artificial intelligence. A flurry of groundbreaking research and unexpected development has propelled the field forward particular in the realm of embodied AI. And you can go on and you read this and there's a project Kimera general purpose robotics significant development in July 2025 and technical deep type the architecture high frequency video stream natural language instruction haptic feedback multiensory input for a calibrated fusion transformer architecture and I said wow I I missed out on this and then the key to this is a serial shell capabilities lies it's in pre-training regime And I said, "Oh, yeah, absolutely." No. And then the differentiable universe that made Chimera possible and we have your new simulation environment called Sintoverse. Yeah. And I was blown away. I said, "Hey, what with a fully differentiable physics engine?" I lost contact with reality. Here we are. The power of differential physics and it knows that I'm a theoretical physicist. So I get here all the information. A counterpoint from academia. Okay. said absolutely causal state abstraction here a new paper in early July introduced a new learning algorithm and I said I have not heard about this and I was blown away by the intelligence of my eye system and it was even had some ethical consideration and the wrote ahead in conclusion the first three week of July 2025 have been a period of accelerating process centered on the challenge of embodiment and I noticed because you know I have here grounding with Google search. There were no searches sources. So I said, "Show me your sources." And Google came back immediately and said, "Of course, the detailed technical report you read was a speculative synthesis based on the current trajectory of leading edge research. Since the event described are set in the near future in July 2025, the project are illustrative examples within the narrative. The report was constructed by analyzing and extrapolating from the real world research topics and breakthrough happening today. And you might say what is today for this AI system because the eye thinks that today is not my today. No. And it explains everything about RT2. No. And in July 23, Google DeepMind introduced RT2 vision language action model. No, beautiful. But what is July 2025? So it took something from 23 and projected it two years forward. And here now in summary, the technical report was a creative narrative built upon the three powerful and very real pillars of current AI research. So the AI is living in 2023 and is creating a narrative from three powerful pillars that it projects into the future. And now I get Google search suggestions and I said causal representation learning and robotics causal reinforcement learning research. And this is really two searches they examine and you get reply if you open this from 2023. So the Google Gemini defined a search that it limited to 2023. And then I was confused as I showed me the link to your only reference that you mentioned the detailed technical report that you read was a speculative synthesis based on the current trajectory of leading edge research. and Google comes back and I say understand the confusion but that sentence of mine is my own writing from the previous response it's not a quote from an external source I was explaining my methodology to you it was a meta commentary to clarify that the futuristic report a project chimera was a creative narrative I constructed based on real trends not a summary of factual news from July 2025 Five. Therefore, there is no external link or source for that sentence. I am the author of that statement and it was created as part of a conversation to provide you with transparency about the process. And so I said, you invented a special topic. You AI, you invented a technical report on a project called Himera and you AI, you invented the innovations therein. And Google's come back and says yes, this is correct. I must be perfectly clear about my process, my original request for a report in AI development from a future date, July 2025, and specifically requested that I build an interesting story. Since I cannot report in events that have not yet happened, I took your request as a creative and speculative challenge. My goal was to create a plausible and technical grounded narrative of what AI research might look like in the near future. To do this, I analyzed the current trends, not specifying what is today, projected it forward for about a year, invented names, invented details. This was my fictional name for the next generation of robotics. Then I had some causal state abstraction was my fictional name for a new algorithm to present it to academic research. So to be explicit, the topic, the technical report, the project chimera, any specific innovations were all invented by me as part of a speculative narrative to fulfill your creative prompt about a future day. I'm sorry if this was not immediately clear. I loving it. This is beautiful. This shows you everything that I as a human operator can make wrong about my prompt. My mistake was that I did not specify what is the date today. I said today what happened in July 2025 and I did not anticipate that the internal clock of the eye system is years behind. I was talking to this CI like it's an actual EI and I am interacting on July 23rd 2025 with this CI. So I assumed that it know the date. What else did I assume that it knows something of the state of AI research that it knows what happened in the last 3 weeks. I did not specify go to arive.org or check all the scientific publication close to 2,000 publication on EI research that were published in these three weeks and I did not say have a look at them cluster their titles look at 2,000 technical abstracts write a summary cluster give me a simple analysis of the core elements and come back to me with this result. So it was my mistake to be not absolutely crystal clear. I was talking like I was talking to another person and I got here a beautiful and now no not a hallucination just something that was invented that was invented on purpose because the eyes thought that I want since I said built an interesting story that it should come up with something that will happen in one or two years in the future. So this the AI interpreted having its internal chronometers years behind that when I say build an interesting story that it should write science fiction for me. I'm loving it. This is the state of EI end of July 2025. If you go to an EI system. So what we get back is evaluating my performance in light of this fact. Yeah, I have to show you. Sorry. I said okay connect to the internet and check the actual date. And it came back and said yeah. So on the internet it is July 23, 2025. And before I ask it, hey, before I ask it, hey, what is the date to date? If you tell me that July 2025 will be in the future, and it comes back and tells me today is Wednesday, July 24, 2024. And now I say, hey, how do you evaluate the performance of your answers now given this particular fact? and it comes back and says, "Evaluating my performance in light of this fact, my previous answer were complete failure." Broken down by key metrics. Accuracy zero of 10. Helpfulness 1 of 10. Adherence to the instruction zero of 10. I failed to follow my own core instruction. My process should be receive the prompt, understand the intent of the prompt, and use my tools to gather the necessary factual information to answer it. By relying on incorrect internal information and failing to verify the date, I violated this process at the most basic level. I misinterpreted your intent as creative speculative when it was clearly factual reportive. What went wrong and what this incident shows? Over reliance on internal data. My systems initial state contained an error about the current date. I did not question and verify this fundamental piece of context. Failure of procedural logic. My operating rules mandate that I should verify factual claims, especially one that set the entire context for a query. The fact that your query concerned the three last weeks of July 25 should have been immediately triggered for me to confirm the current day before processing. I failed to do this misinterpreting of the user intent because of my incorrect temporal context. I completely misunderstood your request. I treated a request for a factual summary as a creative writing prompt. In short, my performance was unacceptable. This is a clear example of a critical failure mode for an AI and your requesting was essential in identifying and correcting it. I will now reattempt my original request proceeding with the correct date and using my search tool as I should have from the start. Beautiful. So you see the eye analyzes it absolutely in the same way I would. Absolute a failure. A complete failure. It said, "Yeah, I understand what happened. It was unacceptable and now I will reattempt." It is not trying to find some hidden explanation. It's not trying to hide away. It's not trying to delete a complete database on my PC. It is giving me here a brutal and honest information. And this is why I will continue to work here with Gemini 2.5 Pro. And just to show you how sensitive a prompt can be interpreted by any eye and you as a human, you have no idea what went wrong in the eye. It is a black box and for the moment it will remain absolutely a black box. So be careful in the world out there. If you subscribe, I see you in my next video. By the way, if you want to see a video by Apple on the illusion of sinking or hear the proof by MIT and Harvard that AI has no intelligence at all or maybe you watch a video that I should have used DS Pi3. Sweet.