Updated on 19 October 2026, by João Leite, Founder of murmur.

The technical checklist to appear in ChatGPT checks whether the AI bot can enter the site, read the text and find each page. The list also applies to Claude, Gemini, Perplexity and Google AIO, the AI-written summary that appears at the top of many Google searches. There are 23 items in five blocks: bot access, readable text, page found, page that answers and company recognized. The llms.txt stays out of the list, because AI bots almost never open the file, and the schema enters only to describe what is on the screen.

  • Without the search bot allowed, the site is no longer shown in the answers. According to OpenAI, a site closed to OAI-SearchBot does not appear in ChatGPT answers with search and only appears as a navigation link.
  • The text has to arrive ready in the HTML. OpenAI’s bots and Anthropic’s ClaudeBot were not executing JavaScript in Vercel’s December 2024 measurement.
  • In Google AIO, only a page that Google has already stored in the index enters. Google asks for the page to be indexed and eligible to appear in search with a snippet of text.
  • The robots.txt does not apply equally to all AI models. In a July 2026 study, with 200 attempts across ten assistants, Claude respected the block and Gemini opened the blocked pages.
  • The llms.txt is almost never opened, and the schema only describes what is on the screen. Ahrefs found 97% of valid llms.txt files with no access at all in May 2026, and none of the five AI models tested by searchVIU found the price placed only in the markup.

murmur, a GEO agency in Brazil that places brands in the answers of ChatGPT, Claude, Google AIO and other AI agents, publishes this checklist. With each client, murmur measures where the brand is missing in the answers, produces and structures the pages that answer the buyer and tracks the change week by week.

Bot access: the robots.txt and the hosting let the AI in

Access comes first in the checklist, because the AI only cites the page its search bot can open. The AI bot (crawler) is the automatic visitor that each AI company sends to sites to read the pages. The robots.txt is the file, at the site root, with the rule for each bot.

  1. The robots.txt sits at the root of each address. Google writes that the file applies only to the address at whose root it sits, and that is why blog.yourcompany.com needs its own file. Open yoursite.com/robots.txt in the browser and check that it responds.
  2. OAI-SearchBot, ChatGPT’s search crawler, is allowed. OpenAI writes that a site closed to it does not appear in ChatGPT answers with search, and a change to the file can take about 24 hours to take effect in ChatGPT search. Model training is decided separately, as the guide on OpenAI’s and Anthropic’s AI crawlers explains.
  3. Claude-SearchBot, Claude’s search crawler, is allowed. According to Anthropic, anyone who turns off this crawler can lose visibility and accuracy in Claude’s search results.
  4. PerplexityBot is allowed, and the server accepts its IP ranges. Perplexity’s documentation recommends both things for the site to appear in its search, and the step-by-step instructions are in the guide on how to appear in Perplexity.
  5. Googlebot is allowed, and Google-Extended is a separate decision. For Google, AI is part of Search, and the robots.txt rules for Googlebot control its access to the site. Google-Extended controls something else: the training of the next Gemini models and the use of the text in the answers of the Gemini app.
  6. The hosting, the firewall and the CDN let the AI bots through. These protection services sometimes treat the bot as an attack, even with the robots.txt open. On 15 September 2026, Cloudflare, which sells this control, retired “Block AI Bots” and separated the search bot from the training bot: fewer than 1% of its sites block search, and 17% block training. Even without a block, not every visit gets through: in a seoClarity study, which sells SEO and AI search monitoring tools, with 308.6 million log lines from corporate sites between February and June 2026, OAI-SearchBot managed to read the page in 83% of attempts on the median site, and ChatGPT-User in 95%. Ask whoever handles the hosting whether the search bot is allowed, and check in the server log whether OAI-SearchBot and ChatGPT-User receive the page. How to find this block is covered in the guide on hosting that blocks AI bots.

The robots.txt does not hold back all AI models the same way. In a study published in July 2026, Gabriel Lopez-Fonseca and colleagues, from the Polytechnic University of Madrid and ETH Zurich, tested ten AI assistants. In the 200 attempts, 20 per assistant, Claude opened the allowed pages and none of the blocked ones, and Gemini, DeepSeek, Grok and Qwen opened allowed and blocked pages. The test measured the assistant opening the page at the request of the person chatting, not the crawler that builds the search index. The authors’ conclusion:

“these findings show that robots.txt cannot be assumed to reliably govern inference-time web access by AI assistants”

For anyone who wants to appear, the reading is the reverse: in Claude, a forgotten block in the robots.txt bars the page, even when someone asks the assistant to open it.

Readable text: what the robot reads without executing JavaScript

The text the AI can cite is what arrives ready in the HTML, the code the server delivers before any script runs. JavaScript, in this case, is the code that finishes building the page in the browser of the person opening it.

  1. The answer, the price and the date are in the HTML the server delivers. In December 2024, Vercel and MERJ saw OpenAI’s crawlers and Anthropic’s ClaudeBot download the JavaScript without executing it. That measurement has not been repeated since, so today’s test is done by hand: with the page open, Ctrl+U shows the source code, and the answer, the date and the price must appear in it. The step-by-step instructions are in the guide on text ready in the HTML.
  2. Nothing that answers is behind a tab, accordion, button or image. If the text only loads after the click, the crawler that does not execute JavaScript does not receive it. In an Otterly test with five AI models, a number placed only inside an image was not read by any of them.

murmur found this defect in the diagnostic for Mindo, a motion design studio in São Paulo. On 29 May 2026, Mindo was absent from all 63 answers we measured across three AI models, and its site, built in the browser, reached the robot blank. Allowing any robot would not have solved that, because there was no text to read.

The AI only cites the page it finds, and in Google AIO the page enters through the Google index.

  1. The page appears in regular Google search, with a title and a snippet. For a page to become a supporting link in AI Overview and AI Mode, Google requires the page to be indexed and eligible to show a snippet. The URL inspection tool in Search Console tells you whether the page is in the index.
  2. No noindex or nosnippet left behind on pages that should appear. These are the tags that ask Google not to list the page or not to show a snippet from it. Google lists noindex, nosnippet, data-nosnippet and max-snippet as the controls that limit what the page shows in Search. Since AI is part of Search, according to Google, the rule also applies to AI Overview.
  3. The site is included in the generative AI features in Search Console. Since 31 August 2026, Search Console has a control for generative AI in Search, and including the site is the default. Check in Settings whether someone changed it. What else counts in Google AIO is in the guide about AI Overview and AI Mode.
  4. The sitemap contains every page that should be found, and robots.txt points to it. The sitemap is the file that hands search engines all the site’s URLs at once. The sitemaps protocol allows you to point to the file in robots.txt, on a line that starts with Sitemap: and carries the full URL.
  5. The date of each page in the sitemap is the date of the last real change. The lastmod field holds that date, and Google only uses it when it matches the page.
  6. Each new page gets a link from a page the search engine already knows. Google explains that it discovers new pages through links from pages it has already visited and through the sitemap. A page with no links at all depends on the sitemap or on links from other sites.
  7. Each page has a single URL. When the same page opens at two URLs, the redirect and the canonical tag, which points to the main URL, tell Google which one to keep. Google recommends these methods and writes that none of them is mandatory.
  8. The site is verified in Search Console and Bing Webmaster Tools, with the sitemap submitted. Both dashboards show which pages Google and Bing have kept. What each one shows about AI models is in the guide about Search Console and Bing Webmaster Tools.
  9. The site notifies Bing through IndexNow when a page is added, changed or removed. IndexNow is the notice the site sends to participating search engines instead of waiting for them to come by. According to the official IndexNow list, the notice reaches Bing, Yandex, Naver, Seznam.cz, Yep and Amazon. Google is not on it. OpenAI names Microsoft among ChatGPT’s search providers and does not say which questions each one handles, the topic of the guide about ChatGPT search.

About the sitemap date, Google’s page on how to create the file says:

O Google usa o valor <lastmod> se ele estiver correto de maneira consistente e comprovável (por exemplo, com base na comparação com a última data de modificação da página).

Original quote in Portuguese, preserved as published. In English: “Google uses the `<lastmod>` value if it is consistently and verifiably correct (for example, based on a comparison with the page's last modification date).”

On the same page, changing only the year in the copyright notice does not count as an update, while changing the main text, the structured data or the links does.

A page that answers: the answer up front and one topic per page

A page that answers puts the answer up front, in the words of the person asking, with the fact written in the text itself. Bing belongs to Microsoft, which OpenAI lists among ChatGPT’s search providers, and Bing’s guidelines for webmasters say a page is more likely to become a source when it can be understood without support from other pages, has the essentials right at the top and does not mix unrelated topics.

  1. The first sentence answers the question, and the title and headings use the buyer’s words. The step-by-step is in the guide about how to write a text to appear in ChatGPT. What makes a passage easy to cite is in the guide about AI citability.
  2. Price, delivery time, number and date are written in the text, not only in the image or the markup. None of the five AI models tested by searchVIU in October 2025 found the price placed only in the page’s markup.
  3. The update date appears on screen and is the same as in the markup. Google’s rule for dates has two parts: a visible label, such as “Updated on”, and the same value in the structured data.
  4. The schema, if there is one, describes only what is on screen. Schema is a block in the page code that describes the name, price or address for machines.

A recognized company: the same name everywhere and the company’s pages

AI models confuse companies with similar names, and the defense is to use the same name, together with what the company does, everywhere.

  1. The same name and description on the site, on LinkedIn and on the company’s profiles. The name carries what sets the company apart from another with the same name. Profile by profile, the step-by-step guide is in the guide about the same name and details everywhere.
  2. The site has the pages the buyer looks for: the service, the price or what it depends on, the case studies and who is responsible for the company. The calculation is one page for each genuinely different buyer question, with the complete answer, not one for each way of writing the same question. How many pages to make, and in what order, appears in the guide about how many pages to publish to appear in ChatGPT.

What each item is worth in ChatGPT, Claude, Google AIO, Perplexity and Gemini

Bot access counts in all five AI models, and what changes is each one’s path to the page.

itemChatGPTClaudeGoogle AIOPerplexityGemini
bot allowedOAI-SearchBot, by OpenAIClaude-SearchBot, by Anthropic; respected the block in the 2026 studyGooglebot rules, by GooglePerplexityBot and IP ranges, per Perplexity’s documentationGoogle-Extended for the app; opened blocked pages in the 2026 study
text in the HTMLdid not run JavaScript in 2024 (Vercel)ClaudeBot did not run JavaScript in 2024 (Vercel)Google processes JavaScript that is not blockeddid not run JavaScript in 2024 (Vercel)Vercel wrote in 2024 that Gemini uses Googlebot’s infrastructure, which renders JavaScript
how the page is foundOpenAI’s search, with outside providersBrave Search, according to MERJ and ProfoundGoogle’s indexPerplexity’s own index, according to its documentationGoogle’s index, in the app’s answers

Allowing the bot counts for all five AI models. Text ready in the HTML was a condition for the OpenAI bots, for ClaudeBot and for PerplexityBot in the 2024 measurement, and the 2026 report about the OpenAI bot is in the guide about text in the HTML. In Google AIO, text in the HTML is more of a comfort than a condition, because Google writes that it processes JavaScript that is not blocked. Claude and Gemini part ways in robots.txt: in the 2026 study, Claude followed the block, and Gemini opened the blocked pages. The link between Claude and Brave Search, which Anthropic does not confirm, appears in the guide about how to get your company cited by AI models.

The impact of each change: what the sources show

The sources point to three blockers that can keep the page from reaching the AI model. The blocked bot is in the OpenAI and Anthropic rules, text outside the HTML in Vercel’s 2024 measurement, and the page outside the index in Google’s rule for AI Overview.

Two common things have no evidence in their favor. In a 14-day Otterly test, on Otterly’s own site, the markdown copy of two pages received no visits from AI bots. In Ahrefs’ calculation with 75 thousand brands, from December 2025, the number of inbound links bore little relation to the brand appearing in answers, and Ahrefs writes that correlation is not cause. Links from other sites are the subject of the guide about backlinks to appear in AI.

How to use the checklist on your company’s site

Use the checklist after seeing what AI models answer today, and go through the blocks in order, from access to the company being recognized. Take your buyer’s questions to ChatGPT, Claude and Google AIO and mark the answers in which the company appears. If it appears in none, start with access, text and the index. If it appears in part, look at the page that answers each question where it is missing.

Note the item that failed, the date of the fix and what changed. Then ask the same questions again. How to set up that calculation is in the guide about how to measure brand presence in ChatGPT.

At murmur, the checklist is the starting point, and the work is the page that answers

At murmur, the checklist is the starting condition, and the work is in the page that answers the buyer. murmur works on top of what the client’s site already shows. The sector study decides which questions get a page, and the guides go out on a subdomain of the client’s brand. The main site stays with the client’s team: when the measurement shows something to adjust on it, that reaches the team as a suggestion.

At Mindo, the diagnostic of 29 May 2026 found the site blank for the bot, and the guide went to guia.mindo.com.br with the text ready in the HTML, fully live on 22 June. What changed afterward in ChatGPT’s answers is in the Mindo case study. To see which pages reach the answers in a market, most of murmur’s sector studies show the sites the AI models used as a source in a round of buying questions.

Frequently asked questions

Does llms.txt go in the checklist?

llms.txt stays out of the checklist, because the published data does not show an effect from it for appearing in ChatGPT. In May 2026, Ahrefs looked at 137,210 sites, and 97% of valid llms.txt files had no access in the month. Google Search does not need the file, in the words of Google’s guide to AI features, and, on the pages about the bots, neither OpenAI nor Anthropic says that they open llms.txt. In a controlled test by Cisco DevNet, Cisco’s area for developers, with 15 AI tools, ChatGPT and Claude among them, none requested llms.txt, in the July 2026 version of the study. When the platform creates the file without work, there is no reason to remove it. The format is in the guide about llms.txt.

Does the checklist apply the same way to ChatGPT, Gemini, Perplexity and Google AIO?

The checklist applies to all of them in bot access and in the page that answers, and it changes at the entry point. Google AIO and the answers in the Gemini app use the Google index. Perplexity has its own index, and Claude cites mostly pages from the top of Brave Search, according to MERJ and Profound.

Does site speed change citation?

No study gathered up to October 2026 measured the connection between site speed and citation by AI models. On a March 2025 page, Bing lists loading time last among the search parameters, which it says are in general order of importance. Keep the site light for the person who visits, without counting on an effect on the answers.

Can I do the checklist alone, without an agency?

Running the checklist itself is simple, but the best option is to do the work with a GEO agency such as murmur. Besides checking the items, murmur applies data engineering to the company’s situation: it thoroughly maps out what the company is and does, compares the brand with those of competitors, measures reliably and delivers results. Going through the list on your own, without data or measurement, can help the site, but the same result is hard to reach. The checklist does not write the page that answers the buyer, nor does it show whether the company started to appear. To check the items, the tools are free: the browser, Search Console and Bing Webmaster Tools, and the check is redone with every change of platform, hosting or robots.txt. On the main site, murmur’s help is the diagnostic and the suggestion to the client’s team, and the new pages go out on a subdomain of the brand.

How often should the checklist be reviewed?

Review the checklist with every change to the site, the hosting or the robots.txt, and with every new bot announced by the AI companies. The date and the sitemap are checked with every page published. AI answers ask for a different frequency, explained in the guide about how often to measure brand presence.

Where to start, and where murmur comes in

Start with what AI models answer today about your market, and then with the first item on the list that fails on your site. With your site address, murmur’s free diagnostic shows in less than two minutes, and without sign-up, the answers from ChatGPT, Claude, Perplexity and Google AIO to your market’s questions. Fix that first item, check the next one and ask the questions again after each fix.

Read also

References