TL;DR
I checked 34 websites from around the world through the eyes of AI crawlers. Instead of thirty-four different stories I found the same seven mistakes. The biggest one: nearly 1 in 5 sites is completely shut to AI — and the owner has no idea. AI invisibility is almost never about content quality. It's plumbing: access, rendering, markup. Half of it is fixable in an evening.
I posted a simple offer on Reddit: drop your website in the comments and I'll check it through the eyes of AI — ChatGPT, Perplexity, Google's AI Overviews. For free. Can they read you, or are you a blank space to them?
I got 34 sites. From everywhere: Germany, Brazil, India, Switzerland, Australia, Kazakhstan, Singapore. E-commerce stores, marketplaces, SaaS tools, news portals, apps, clinics.
I expected a zoo — thirty-four sites, thirty-four different problems. I found the same seven mistakes. On repeat. From Brazil to Singapore. And in nearly one out of five cases, the site was completely shut to AI crawlers — with the owner having no idea.
Here's what I found. No "what is GEO" theory — there are a thousand of those articles already. This is thirty-four live websites, checked by hand.
The method
Nothing fancy: I fetched every site the way AI crawlers do — as GPTBot, ClaudeBot, PerplexityBot — three times each, so a random error wouldn't count as a block. Then I read the raw HTML the bot receives (not what you see in a browser), parsed the structured data, checked robots.txt and sitemaps. Five to ten minutes per site. Enough to find the hole.
The locked door
The most common finding, and the most painful one. Six sites out of 34 don't let AI bots in at all. Hard 403. A regular browser gets in, Bing gets in, Google gets in — GPTBot and PerplexityBot get turned away at the door.
Here's the kicker: almost none of the owners did this on purpose. It's Cloudflare. Since late 2024 it ships a "Block AI Bots" setting that's enabled by default on many plans. Someone builds a good site, writes content, adds schema, even creates an llms.txt — "here, AI, read me" — and the firewall quietly turns away the exact bots that file was written for.
One site had literally negotiated against itself: robots.txt said citation was welcome (use=reference) while the firewall was blocking OAI-SearchBot and PerplexityBot — the two bots that do the citing. The policy says come in. The bouncer says get out.
Five of those six sites were genuinely good — content, markup, 4.9-star reviews, ten thousand customers. None of it matters. The door is locked. Special mention: one of the blocked sites is a tech news portal that writes about AI. A website about artificial intelligence, invisible to artificial intelligence. Curtain.
Takes one minute to check. If you're on Cloudflare — open the bot settings and look. The block is probably there.
The empty storefront
The second-biggest killer: sites that assemble themselves in the browser. An AI crawler is not a person. It doesn't wait for your JavaScript to load the pretty cards. It takes whatever is in the initial HTML. If your content arrives later, via scripts — the bot sees an empty box.
A real-estate site served the bot 55 characters. Not a paragraph — a title, and that's it. A catalog of five hundred AI tools: instead of tools, the bot got 78 chunks of unrendered template code. A database of thousands of clinical trials: the bot sees "0 results. Loading…"
The record-setter was a renovation marketplace in Singapore. Contractors, deals, prices — a living business. The bot gets 41 characters. Forty-one. Every URL on the site — a real page, a made-up one, doesn't matter — returns the same empty shell.
Now the cherry on top. The owner has an external marketing team. With a dedicated technical SEO specialist. Nobody saw the hole. And honestly, you can't blame them: Googlebot does execute JavaScript, so every SEO report was green. They were measuring the wrong bot.
Expect more of these. Vibe-coding platforms — the ones half the market builds on now — produce exactly this by default: beautiful in the browser, blank for the bot. A person sees a rich, living site. The bot sees a skeleton. And there's nothing to cite — because there's nothing there.
GEO theater
My favorite specimen. An antivirus site. Rich structured data. A pompous llms.txt that proudly calls itself the "Ultimate AI SEO Version." The owner clearly read about GEO and ticked every box.
Except every one of the site's 32 pages serves the bot the same 89-character skeleton. One title, duplicated thirty-two times.
That's theater. The set is built, the stage is empty. Markup describes content — it doesn't replace it. You can shout "I'm about antivirus, here's my schema" all you want; if the page has no antivirus text on it, AI has nothing to take. The shop window is dressed. There's nothing in the shop.
Schema without body, body without schema
Two mirror-image sites that explain everything together.
The first: a developer tool with excellent content — docs, blog, honest writing. Everything you need to get cited. Structured data: zero. On every page. The bot reads the text but sees no structure.
The second, the opposite: a services marketplace in India with textbook LocalBusiness and Service markup — and nearly empty pages, because the content loads client-side. Rich schema with nothing to describe.
The moral is simple. GEO walks on two legs: markup and readable text. Nobody walks on one. The first site lacks schema. The second lacks content. Both limp.
The leaked template
Small but telling — because AI shows this stuff verbatim.
A fitness app. I open its JSON-LD — the markup machines read — and instead of a description there's raw engine code: the template never filled in the value, and the placeholder shipped straight to production. To any parser, that markup doesn't exist.
A construction-equipment marketplace had its product heading glued together with no spaces — name, brand and price mashed into one string. The bot reads it exactly as written. As mush.
You only see this in the page source. The browser view is fine. The bot lives in the page source.
No rating in the markup
This one is a disease of good sites specifically. Half the sites — 17 of 34 — expose no rating whatsoever in their structured data. Most of them have reviews. They're just not marked up.
A German store literally says on the page: "4.9 on Google, 10,000+ customers." Great. None of it is in the schema. When AI answers "which service should I pick," it loves to cite with a number: "this one, rated 4.7 by a thousand users." No number in your markup — you get cited without one. Or someone else gets cited instead.
The cheapest fix on this list. The reviews already exist. Mark them up.
Broken basics
And finally — the boring classics that quietly ruin everything. An empty sitemap. A zero-byte robots.txt. A sitemap that returns an HTML page instead of XML, or points to a 404 — promising the bot pages that don't exist. A title tag that reads "Finance operations illustration" — the name of a random image that leaked into the tag. That's what AI and search show as the page's name. Verbatim.
A fresher subspecies: the site where every URL — including deliberate garbage — answers "everything's fine, 200." Its llms.txt technically "exists," too. I opened it: the same app shell inside. A business card for AI with cardboard where the card should be. I found two of those.
Multilingual sites without hreflang belong here too. One API product ships in eleven languages. Eleven! Zero hreflang tags connecting the versions. A clinic in Kazakhstan stepped on the same rake from three sides: the site runs in Russian, Kazakh and English, with hreflang present only on the English version. The languages exist; which one to serve whom, the AI has to guess.
And the quietest one: frozen dates. A consulting firm — perfect access, exemplary rendering, seven thousand characters of text served to the bot. And every date in the sitemap is from May of last year. Articles with no authors and no dates. When AI recommends, it prefers fresh and attributed — and silently picks someone else. The site isn't broken. It just stopped.
To be fair — there was good news
So this doesn't read like "34 idiots." They're not.
One site had near-perfect markup: a full schema graph, comparison pages with tables, a glossary, llms.txt, a robots.txt that practically invites AI in. One dev portal had the best article markup of the batch — author, date, Q&A. One sports-predictions service had an exemplary robots policy: citation bots welcome, training-data scrapers blocked. Two stores are already preparing for AI shopping agents — that's playing ahead of the market.
The market isn't stupid. The market trips over the same seven spots.
What to do about it
One list. The market's checklist, as of today.
Let the bots in. If you're on Cloudflare, check whether it's blocking AI crawlers. That's the first thing to do, and often the only one.
Serve content in the initial HTML instead of assembling it with scripts. The bot won't wait for JavaScript.
Mark up your rating. You already have the reviews — show them to the machines.
Give every page a clean heading and a human title tag. Not "illustration."
Schema and text travel together. One without the other doesn't work.
Check your policy for contradictions: if you want citations, let the citation bots in — not just the training ones.
On speed, since every owner asks "how soon does it work": one practitioner from that same thread unblocked GPTBot and OAI-SearchBot on his docs site — a week later Perplexity was citing 3–5 of his URLs, up from zero. Days, not months. His framing stuck with me: treat an AI-bot block like an outage, not a nice-to-fix.
The part that stuck with me
I did all these audits for free, in the comments. Four owners publicly said thanks; two went straight off to fix things. Here's what I took away.
People assume: "AI doesn't mention me — I need more content, more articles, more SEO." In most cases the content was fine. Good, even. Invisibility isn't about the quality of your site. It's about plumbing: a locked door, an empty storefront, a forgotten checkbox.
I rebuilt my own site for AI the same way. Two weeks. Citability went from 25 to 64. Myself, by hand, without an agency "running a set of activities." I'm not smarter than the people whose sites I checked. I just looked, and I measured.
Half of this list you can fix in an evening. Start with the door.
Want to know which of the seven your site trips on?
I'll send you the same breakdown for your website: what the AI bots actually see, where the hole is and what to fix first. Same method as these 34 audits. No hard sell.
All sites in this article are anonymized — niche and country, no domains: the audits were given publicly in the comments, but collecting them into one article with names attached wouldn't be fair.
