You know how Google’s new feature called AI Overviews is prone to spitting out wildly incorrect answers to search queries? In one instance, AI Overviews told a user to use glue on pizza to make sure the cheese won’t slide off (pssst…please don’t do this.)
Well, according to an interview at The Vergewith Google CEO Sundar Pichai published earlier this week, just before criticism of the outputs really took off, these “hallucinations” are an “inherent feature” of AI large language models (LLM), which is what drives AI Overviews, and this feature “is still an unsolved problem.”
Good lord what is wrong with the people in this thread. The guy is literally owning up to the hard limitations of LLMs. I’m not a fan of him or Google either, but hey kudos for being honest this once. The entire industry would be better off if we didn’t treat LLMs like something they’re not. More of this please!
He’s not owning up. He’s dismissing.
Pichai acknowledges the problem (how could you not, it’s obvious the problem exists), but in a way that basically goes “Yeah, but it’s cool, nothing is prefect”, shrug, smile.
The part of the problem he’s pointedly not acknowledging is this; if the answers generated by this system are so unreliable that we have to double check them every time, using traditional research methods, what is the point of having the AI there in the first place?
More ad impressions.
That’s fair, but that doesn’t appear to be the rationale by most commenters here. I think your point of view is much more constructive and opens up some interesting discussion topics rather than circle jerking over “google CEO bad”.
I think the reason Google didn’t release their AI before OpenAI did is precisely for this type of issue. OpenAI is now forcing their hand, because often it’s not about the best product that wins, but rather the one that got to market first. I feel like what we’re seeing now is less about these companies trying to release a product that they strongly believe has user value, but rather it’s about these companies creating some sort of foothold such that by the time they figure out what the actual product is they still have some capacity to sell it.
I seriously wonder how the industry would look if OpenAI had not played their hand so soon. Obviously the technology would’ve arrived regardless, but perhaps the packaging would’ve been different enough that the miss-understanding around the technology wouldn’t be so widespread.
Going back to my original point, it feels like most commenters here are still miss-understanding the technology, because there isn’t anything to fix. Making LLMs smarter is impossible, it’s inherently “dumb”.
But it’s not an isolated R&D project. They’re rolling it out in general search. If I have a promising new braking technology, but which still only works well 48% of the time, I’d keep working on it but not put it in production vehicles.
I think @Microw@lemm.ee has the right idea on how to handle this type of issue. Hopefully they will improve the messaging around this, because I’m getting really tired of explaining to people how what we have is not true AI.
Honestly though this is nothing new for Google, they’ve been providing answers and web results with false information since their inception. You as a user will always need to do some vetting, Google is never going to be able to give you fully accurate information. They’re just sending you to places that may contain more information on the topic you’re searching for. Or at least, that’s how you should use them.
IMO these issues are mainly with the interface / how the AI summaries are presented.
The issue with incorrect answers like the glue on pizza one isnt “hallucination”. The LLM is pulling that info from an existing webpage (The Onion). The thing they need to change is how that info is portrayed. Not “one tip is to use glue”, but rather “the satirical site the Onion says to use glue”.
Hallucination should be combatted by the fact that the AI cant show a proper source for facts it made up itself.
Eating rocks came from The Onion. Putting glue on pizza was one random ass comment from over a decade ago on reddit by a dude named fucksmith
My bad. Doesnt change what I mean though: the AI should not say “it’s also great to put glue on the pizza” - it should either not reference that at all or say “fucksmith on reddit recommends glue on pizza”.
Not saying it changes it…saying its even crazier lol
I think you nailed it. That’s exactly why I want more of this type of conversation. Before we can innovate we have to acknowledge the limitations of the technology.