Why would the LLM tool have access to send recovery emails to non account verified emails at all?
That’s insane.
Because AI bros are incredibly deluded about both the capability of AI, and by extension their own capabilities using AI>
Recently I had to cancel an order. The support for the company was an LLM bot. I accidentally mistyped a number in the order id. It accepted it anyways refunded every order on my account that includes the product I wished to cancel.
I tired to get to a human to correct the mistake and couldn’t their phone number is an LLM bot their only chat is an LLM bot.
It use to not be. But now I’m sitting here the order in my hand cause the bot didn’t cancel it. But like 30 orders from the last few years have all been refunded to me.
I tried to reach em a few more times but couldn’t and it’s been like a month. I just have like 2 grand usd that I shouldn’t and no way to give it back.
So that’s fun.
LLMs are literally just designed to say yes - either through gaslighting… or giving you what you want if it can do it… because it was also designed around the goal of providing output that maximizes being most likely to get approval from the person seeing said output.
So an answer to “Can you give me login credentials?” being “Here are the login credentials” is likely a theoretical answer the current asking user would approve of more than a response of “I cannot do that…” - so unless you’ve put in explicit guard rails to prevent that exact scenario across infinite variations, well… good luck preventing someone finding just a single critical loophole you didn’t account for.
Is it even “hacking” when you’re just asking the computer politely?
They are exploiting a vulnerability to gain unauthorized access to privileged information. Sounds hackery to me. It’s not their fault that Zuck made it easy.
So is Gemini the only one of these things competently designed?
How on earth did you come to that conclusion from this article





