For a service business, the useful question is straightforward: can someone asking about your services find an accurate explanation of what you do, where you work and why they should trust you?
AI systems can reach that information through search indexes, direct page requests and interactive browser tools. One platform can use more than one route. Checking your initial HTML is a useful starting point, but it cannot show you exactly what every AI product sees or whether it will recommend your business.
Search, fetching and browser interaction are different
Google says a page must be indexed and eligible for a Search snippet to appear as a supporting link in AI Overviews or AI Mode. Both features can issue related searches across subtopics, so the sources returned may differ from the results for your original question. Eligibility gives your page a chance to appear; it does not guarantee selection. Google’s AI features guidance.
Googlebot can also render JavaScript. It processes the initial response and queues rendering, then uses the rendered HTML for indexing. Rendering can take time, and blocked resources can prevent content from appearing. A site built with JavaScript is therefore not automatically invisible to Google. Google’s JavaScript SEO guidance.
OpenAI and Perplexity distinguish search crawlers from fetchers used for user requests:
| Agent | Documented purpose |
|---|---|
| OAI-SearchBot | Helps surface sites in ChatGPT search results. |
| GPTBot | Crawls content that may be used to train OpenAI foundation models. |
| ChatGPT-User | Visits pages for certain user-initiated actions. |
| PerplexityBot | Crawls for Perplexity search results; Perplexity says this is not foundation-model training. |
| Perplexity-User | Fetches pages in response to user requests. |
These are different jobs, not evidence that every answer triggers a fresh fetch of your page. OpenAI’s crawler documentation and Perplexity’s crawler documentation.
Browser agents add another route. OpenAI documents tools that let a model operate a browser through mouse and keyboard actions, using screenshots and other tool results. So the blanket claim that AI cannot click or use an interactive page is wrong. That capability also does not mean every search crawler opens your tabs or fills in your forms. OpenAI’s computer-use documentation.
Initial HTML is a useful foundation
The initial HTML is the response your server sends before browser JavaScript runs. Putting your essential service information there gives more readers a usable starting point. Google itself recommends server-side rendering or pre-rendering where appropriate because not every bot can execute JavaScript. Google’s rendering guidance.
A Vercel and MERJ study published on 17 December 2024 reported no JavaScript rendering by several named AI crawlers in the traffic and sites it examined. Its observations included Next.js and Vercel traffic alongside two job sites. That is useful evidence from a particular dataset and period, not a permanent specification for all current AI products or browser agents. Vercel’s crawler study.
Keep a well-designed accordion if it helps customers. Its closed state does not tell you whether the answer is available in the HTML: server-rendered accordion content can already be present. Content fetched only after a click is a different case. Google says Search does not interact with pages by clicking or scrolling to load content, so important information should not depend on those actions. Google’s lazy-loading guidance.
For a UK service website, start with the service description, areas covered, genuine qualifications, contact route and any published prices or scope limits. Check that these remain understandable when decorative effects and optional interactions are absent.
Choose crawler access deliberately
OpenAI treats its search and training controls independently: you can allow OAI-SearchBot while disallowing GPTBot. Opting out of OAI-SearchBot prevents inclusion in ChatGPT search answers, according to OpenAI, but a site can still appear as a navigational link. ChatGPT-User is separate, and OpenAI says robots.txt rules may not apply to user-initiated requests. OpenAI’s access guidance.
Perplexity likewise separates PerplexityBot from Perplexity-User and says its user-triggered fetcher generally ignores robots.txt. Its documentation recommends checking published IP ranges as well as the user agent when configuring firewall access. A request identifying itself as a bot is not, by that name alone, proof of who sent it. Perplexity’s access guidance.
For Google Search, inspect indexing and snippet controls as well as robots.txt. Settings such as noindex, nosnippet, data-nosnippet and max-snippet can affect eligibility or content use in AI features. Google does not require a special AI schema or a new machine-readable text file to participate. Prioritise accessible text and structured data that agrees with the visible page. Google’s controls and requirements.
Make the content useful without manufacturing freshness
Write service pages so a customer can understand each section without decoding marketing language. An accountant might explain which business types they support, what an accounts package includes, which work costs extra and how to get an accurate quote. A trades business might explain its service area, response arrangements and relevant accreditations. Publish only details the business can substantiate.
Use a case study when you have real evidence: describe the starting position, work completed, measurement period and result, with permission from the client. Give readers enough context to judge whether it applies to them. A number without a baseline or method can mislead.
There is no need to force a statistic or a current date into every paragraph. Google encourages original, useful information and specifically cautions against changing dates merely to make unchanged content seem fresh. Update facts when they change and record substantive revisions honestly. Google’s helpful-content guidance.
Five practical checks for your important pages
- Inspect the initial response. Use View Source or ask your developer to fetch the HTML. Find your main service description, location, contact details and published scope. A request from your machine, even with another bot’s user agent, is a diagnostic of that response, not an exact reproduction of the platform’s access.
- Compare the working page. Check what JavaScript adds, and whether essential information loads only after interaction. Use Search Console’s URL Inspection to examine Google’s rendered view where available. Google’s inspection advice.
- Review access settings. Check robots.txt, indexing and snippet directives, plus hosting or firewall blocks. Make a separate decision about each documented search, training and user-fetch agent.
- Read as a prospective customer. Can you work out who the service is for, what is included, where it is available and how to enquire? Do visible claims and structured data agree?
- Observe actual answers separately. Test a defined set of realistic buyer questions on the platforms you care about. Save the question, date, platform or mode, answer and cited URLs. Repeat under comparable conditions and record variation. Track enquiries separately from mentions and citations; they measure different things.
Our free SEO health check inspects public responses using AIEO’s own scanner. It checks the homepage and up to four linked internal pages without browser rendering. It can identify technical issues to investigate; it cannot reveal every platform’s internal view or establish that your business is being cited. The methodology explains those limits, and an AI visibility audit adds observed platform testing to the technical checks.