Retrievable, extractable, citable, reusable

Content that works for AI answers has to clear four separate hurdles, and the reason it's worth naming them separately is that failing any one produces the same symptom — you don't appear — while requiring a completely different fix. A crawler that can't reach the page. A crawler that reaches it and can't parse it. A parsed page with nothing specific enough to quote. A quote that stops making sense once it's lifted away from the paragraph that gave it meaning. Four failures, four owners, four afternoons of work that have nothing to do with each other.

Retrievable

The most common expensive mistake in this category is pulling the wrong lever, because the AI companies ship two kinds of switch and they look almost identical. Google-Extended is the one everybody has heard of. Google's documentation is unambiguous about what it does: it manages whether crawled content may be used to train future Gemini models, and it "does not impact a site's inclusion in Google Search nor is it used as a ranking signal." Blocking it does nothing whatsoever to your AI Overview eligibility. The control for that is Googlebot itself, plus the snippet directives: nosnippet, data-nosnippet, max-snippet and noindex. If your page can't be shown with a snippet, it can't be shown in an AI Overview. Apple built the same split. Applebot-Extended, per Apple's June 2026 documentation, "does not crawl webpages" at all — it only governs how already-crawled data gets used. The control for Apple's AI answers is the nosnippet meta tag.

Training opt-outSafe

Google-Extended

Applebot-Extended

GPTBot

ClaudeBot

Blocking these has no documented effect on answer-surface visibility.

Answer-surface opt-outCostly

Googlebot / nosnippet

OAI-SearchBot

Claude-SearchBot

PerplexityBot

Blocking these removes you from the answers themselves.

OpenAI documents the separation explicitly: a site can allow OAI-SearchBot in order to appear in search results while disallowing GPTBot, and sites opted out of OAI-SearchBot "will not be shown in ChatGPT search answers." Anthropic's Claude-SearchBot documentation warns in the same terms that disabling it "may reduce your site's visibility and accuracy in user search results." Perplexity is the outlier worth knowing about. Its help centre states that PerplexityBot won't index full or partial text of a site that disallows it — "however, if a page is blocked, we may still index the domain, headline, and a brief factual summary." A robots.txt block does not fully suppress you there. For scale: Cloudflare's network data shows crawler traffic up 18% between May 2024 and May 2025, with GPTBot up 305% and Googlebot up 96%, and 14% of top domains using robots.txt rules to manage them. Most sites aren't managing this at all, which means most sites don't know which lever they've pulled.

Extractable

This is the gate with the hardest single constraint, and it's the one most likely to be quietly failing on a modern site. Vercel and MERJ instrumented their network over roughly a month and checked whether AI crawlers execute JavaScript. The result: none of the major ones do. Not OpenAI's OAI-SearchBot, ChatGPT-User or GPTBot. Not Anthropic's ClaudeBot. Not Meta's crawler, not Bytespider, not PerplexityBot. Two exceptions — Gemini, because it runs on Googlebot's infrastructure, and Applebot, which renders through a browser. They fetch JavaScript files. ChatGPT's crawler spent 11.5% of its requests on them, Claude's 23.8%. They just don't run them. So a page whose content assembles client-side is, to most of the systems answering questions about your industry, a blank document with a stylesheet. The same study found ChatGPT's crawler spending 34.8% of its fetches on 404s, against Googlebot's 8.2%. These crawlers are less efficient and less forgiving than the one you've spent fifteen years optimizing for.

Citable

Reaching the page and parsing it still leaves the question of whether there's anything on it worth quoting, and this is where the industry's confident advice performs worst against measurement. Google's guidance is direct on two counts. On markup: "You don't need to create new machine readable files, AI text files, markup, or Markdown to appear in Google Search (including its generative AI capabilities), as Google Search itself doesn't use them," and "there's no special schema.org markup you need to add." On structure: "There's no requirement to break your content into tiny pieces for AI to better understand it… There's no ideal page length." Ahrefs tested the schema claim causally — 1,885 pages that added JSON-LD between August 2025 and March 2026, matched against 4,000 control pages, difference-in-differences. AI Overview citations fell 4.6%. AI Mode and ChatGPT moved by amounts indistinguishable from zero. The eye-catching correlational finding they started with, that cited pages were three times likelier to carry schema, did not survive contact with a control group. And Ahrefs' llms.txt study found 97% of the roughly 38,000 valid llms.txt files it identified were never fetched by anything at all in a month. Slackbot fetched more of them than PerplexityBot did. So if not markup, what? Anthropic's own retrieval research points at the answer, and it's about the passage rather than the page.

Anthropic's worked example of the failure is a chunk reading "The company's revenue grew by 3% over the previous quarter." Which company? Which quarter? The paragraph above it knew. The chunk doesn't. Attaching fifty to a hundred tokens of context to each chunk cut the retrieval failure rate by 35%; combining that with keyword matching took it to 49%; adding a reranking pass reached 67%. Google saying "don't chunk your content" and Anthropic showing that chunk context is worth a 49% reduction in retrieval failure are not in conflict. Google is telling you not to fragment your page into confetti. Anthropic is documenting that the retrieval system fragments it anyway, and that passages carrying their own context survive that process considerably better. Both point at the same practice: write sentences that mean what they mean on their own.

Reusable

The last gate is the least instrumented and, if you care about being represented accurately, the most consequential. The underlying problem has a name and it predates all of this. A 2021 paper in Transactions of the ACL, authored by researchers at Google, formally defined sentence decontextualization: "Taking excerpts of text can be problematic, as key pieces may not be explicit in a local window." The task they defined was rewriting a sentence so that it stays interpretable, and stays true, once removed from its surroundings. That's an unsolved research problem, and it is running live on your content right now. The failure rates are documented and they are not small.

The BBC study amended its robots.txt to grant the assistants access for the duration, so those figures describe performance with permission granted, not despite a block.

That last figure is the one that matters for this gate specifically. Thirteen per cent of the quotes attributed to BBC articles were either changed from the original or not present in the cited piece at all. Not a paraphrase problem — a quotation-integrity problem, at the surface where readers actually meet your words. You cannot fix the models. You can reduce your own exposure, and the mechanism is the same one that helps with extraction: sentences that carry their subject, their timeframe and their qualifier inside themselves are harder to misquote, because there's less for the surrounding context to have been supplying.

Where the four gates leave you

The asymmetry across them is the useful part. Retrievability is well documented and almost entirely within your control, and the main risk is pulling a lever you misread. Extractability has one hard technical constraint that a lot of modern stacks quietly fail. Citability is where the loudest advice performs worst against controlled measurement. Reusability is barely instrumented by anyone and is where the industry will spend the next few years. What they have in common is that none of them is a writing-quality problem, and only one of them belongs to a writer. That's the argument for keeping them separate on a scorecard rather than averaging them into a number that tells you something is wrong without telling you who to ask.

Sources

Related reading

• What actually gets a page quoted in an AI Overview — the four mechanics tested against real citation data.
• A single content score can't tell you what to fix — why these mechanics are scored separately instead of averaged into one number.
• The citation is the conversion — what all four mechanics are ultimately in service of.
• The Prism Score — where retrievable, extractable, citable and reusable sit among the ten dimensions.