New ChatGPT: Hallucinations No More!

Key Takeaways

I hear more and more people say that they would love to use a cad GPT like model not only as an assistant but also for searching the internet and open AI just gave a lot of Scholars exactly what they wished for we will talk about what it can do why it is not always sunshine and happiness and we don't stop there no sir they also did something in a new research paper that could make it work better than we think I'll get to it in a moment now now most of the world uses Google for searching the internet and it's great for simple questions when it works but check this out a new icon should appear soon for Chad GPT plus users and there you can ask for something more complex planning a trip no problem but it gets better and yes like a good fellow scholar it cites its sources to make sure you indeed get correct information and not some hallucination that the am just made up but this is not just regular search you can also ask followup questions like where to grab dinner according to your preferences or what the weather is going to be like but on specific days not just a huge table with a bunch of stuff you don't need and all this with more sources you can also search and see whether your Nvidia stocks have made you rich already or not and get a little additional context on what just happened also I have to say I am a bit less impressed about the collection of news you can get that anywhere but you don't have to ask simple questions like that ask something more interesting instead like how to fix up your backyard and get not just copy paste but a nicely synthesized answer with images tailored to you and a guide made on the Fly important part it also leads you straight to the source in a very visible way so I hope this will also reward the creators of this kind of information as well but there is a problem here and that problem is called hallucination this is where the AI makes things up if you ask something you are going to get an answer whether that answer is correct or not well that depends we are seeing less and less of those hallucinations now but they still exist however if you use a search engine that hallucinates that could almost be as bad as just getting a bunch of ads instead of an answer okay so what does it have to do with AI research papers a great deal have a look at this okay I hear you asking caroy what a data set so a bunch of questions and answers what does that have to do with hallucinations how does this help us well yes this is a data set published by open AI but it is no ordinary data set these are challenging questions and answers with high correctness on a diverse set of topics I mean just look at that many of these are a tou more complex than what you can just get through search you see when evaluating an AI model with answers to these questions not even that is without challenges it may be fled out incorrect but being incorrect gets a bit more Insidious here look at this oh yes fellow Scholars this is an AI that is not sure and it is hedging you know if I say that the stock market may go up or down or it might go flat I am completely right but not very informative is that incorrect hm hard to say so what are the results well hold on to your papers fellow Scholars because when they compared their Flagship models to their mini variants oh goodness that is a huge difference they are still wrong a lot and our question now is okay but are they like people do they know that they are likely wrong on many of these things are AI aware of their limits I would say the answer is surprisingly yes when asking them how confident they are in their answers whenever they feel more confident they are more likely to be right and when less confident less likely to be right now if we are using this for search we are hoping that the answers are also consistent are they let's have a look by asking the AI models the same question a 100 times and are they consistent I would say that the flagship models kind of are not even close to perfect so careful there but once again their Flagship reasoning 01 and gp40 models are remarkably consistent much more consistent than some politicians I hear speaking these days and this is where this new data set shines it helps AI models become more confident when they should be confident and more consistent and this will finally lead to less hallucinations and more accurate information for us fellow Scholars it is still a long road but this is one bloody good step in the right direction what a time to be alive so what do you think what would you fellow Scholars use this for let me know in the comments below and if you feel inspired and you wish to run your own experiments on Nvidia gpus check out Lambda Lambda is a cloud service for large-scale AI projects and it is my absolute favorite seriously try it out now at lamb.com slapers or click the link in the description if you use Lambda you can run your own experiments on super powerful hardware and you also support two-minute papers