GEO poisoning makes ChatGPT, Gemini and Google AI Overview answer with scam support numbers for Delta, Lufthansa and Chase
- Attackers use GEO (Generative Engine Optimization) techniques to make ChatGPT, Gemini and Google AI Overview present fake customer support phone numbers, email addresses and login pages for Delta, Lufthansa, Qatar Airways, Chase, Bank of America, Airbnb, TripAdvisor and hundreds more brands.
- A screenshot in the article shows Google AI Overview presenting an attacker-controlled phone number as the official Lufthansa contact.
- Attackers flood the web with optimized posts, PDFs, reviews and fake support pages, padding them with statistics and quotes from recognized experts to raise an LLM's confidence in the planted data points.
- Exploding Topics reported that 92% of users do not verify AI answers, so a poisoned answer reaches victims largely unchecked.
- The author states these are in-the-wild automated campaigns rather than a proof of concept, running at a scale traditional takedowns cannot keep up with, and this post is part one of a blog series.
Hacker News opinions
I keep seeing Claude treat a call transcript as gospel, like it'll tell me the user said they'd use a feature so build it. The whole point of the analysis is separating real signal from conversational niceties and the models are terrible at that.
I mod the Julia Discourse and classic SEO spam is easy to catch since it's all links. Lately scammers post AI-written threads about market statistics, mention a crypto exchange with no link, then edit in a support phone number later. It drags good-faith answers out of the community, which is worse than plain SEO spam.
You don't even need hackers. My elderly neighbor looked up the Microsoft help center number and Gemini handed her a scam number. I spent the weekend wiping her PCs, locking down accounts, and dealing with identity theft.
My wife posted an April Fools thing about our community pool replacing trees with AstroTurf and Google AI Overview reported the pool has beautiful AstroTurf landscaping. It self-corrected a few weeks later, but come on.
My question is how the fake content even gets indexed in the first place. Getting an unknown site crawled and ranked is not trivial these days.
Once spam tailors its content better and stays consistent across threads and syndication, spotting it by hand becomes hard and the models won't stand a chance. Consistency is the tell I currently rely on.
I did takedown work for a brand's fake support numbers and finding them was never hard. Killing one PDF just meant a fresh Medium post by morning.
Legitimate sites block AI crawlers while slop farms let everything through, so the models increasingly feed only on the junk. The web has become easy to poison.
I'd reframe 92% don't verify as the actual product goal. People are relying on hundred-billion parameter models as a shortcut to avoid DOING the work, and LLMs are the oracle.
I really think whatever actually fixes this pays off big with insurance companies, especially for seniors.
Counterpoint, the real fix is just not putting AI summaries at the top of every search. If grandma queries a phone number and gets an AI answer she assumes it's correct. The HTTPS cert used to let you trust the number on Delta's own site, and now every bit of misinformation rides a valid Google cert.
Unlikely the market solves it. This won't even register at the labs, since they worry about existential risk first and paying coding customers second.
AI providers like ChatGPT and Gemini should face criminal liability when their output enables a scam.
It worked for Google and Facebook. Oh wait.
Poisoning is the current exploit frontier and it isn't going away. At work we have agents reading plain document folders, so dropping in files with hidden instructions can leak data or own the host, and downloaded skills are a supply chain risk too.
The models are the inherent risk. Amazing what they can do, but you are no longer in complete control.
That Medium link of the original poster feels phishy to me.