Website Ready for AI Agents? A Practical Checklist

Is Your Website Ready for AI Agents? A Practical Checklist

AI assistants now read, compare and act on websites for people. Here is what that means for your business site, what is confirmed, what is still early, and a practical checklist to get ready.

Rachyet teamWeb, 3D & growth studio Oct 27, 2026 8 min read

In short: AI assistants no longer just answer questions. They open websites, compare options and, more and more, fill in forms and buy things for people. To get your website ready for AI agents, you do not need a rebuild. You need clean, honest HTML, forms that make sense without a mouse, clear prices and availability, sensible crawler rules and fast, stable pages. Most of it is good practice anyway; the newer standards are still early.

What the "agentic web" actually means

An AI agent is software that carries out a task for someone rather than just giving them information. Google's own Search guidance describes agents as "autonomous systems that can perform tasks on behalf of people, such as booking a reservation or comparing product specifications" (Google Search Central).

The same guide explains how browser agents work today: they look at screenshots of the page, read its code (the DOM) and interpret the accessibility tree, the structure that screen readers use. In other words, an agent "sees" your site in roughly the same ways a person and a screen reader do at once.

So your visitor may now be an assistant working for a person. If it cannot find your price, understand your booking form or tell which button submits an enquiry, it moves on to a site where it can.

What Google announced at I/O 2026

Google's May 2026 developer conference put agents at the centre of its web announcements. The points that matter to site owners:

Agents in the browser

Chrome's "auto browse" feature, which lets Gemini carry out tasks in the browser, is already available on desktop, and Google announced it for Android for tasks such as booking appointments and finding in-stock items (Chrome for Developers). Google also said auto browse will connect with its Gemini Spark assistant "in the coming months".

WebMCP: a way for sites to talk to agents directly

The headline item for developers was WebMCP, described by Google as "a proposed open web standard" that lets a website expose structured tools, such as JavaScript functions and HTML forms, to browser-based agents (Chrome for Developers). Instead of guessing which fields to fill, the agent is told: here is the "book a table" action and what it needs.

It is important to be clear about its status. Google says the experimental WebMCP origin trial starts in Chrome 149, and that Gemini in Chrome "will soon" support it. Google's own best-practice guide notes that WebMCP is still in active development and may change (web.dev). It is something to plan for, not something every small business must build this month.

Agent shopping and payments

On 19 May 2026 Google also announced Universal Cart, a shopping cart that works across merchants, built on the Universal Commerce Protocol (UCP), which Google calls "a common language for agents" (Google). Alongside it came the Agent Payments Protocol (AP2), which lets an agent pay only within the brands, products and spending limit the user sets. Google says the brand stays the merchant of record. Rollout is staged: the US first, then Canada and Australia, and later the UK, with AP2 arriving in Google products "in the coming months". Outside those markets, treat this as a direction of travel rather than something live for your customers yet.

Other standards you will hear about

MCP (Model Context Protocol)

MCP is an open protocol that lets AI assistants connect to outside tools and data, such as a booking system or a product database. Anthropic created it and in December 2025 donated it to the Agentic AI Foundation, a fund under the Linux Foundation co-founded with Block and OpenAI (Anthropic). It matters when you want assistants to query your own systems, such as stock or appointment slots, through a controlled connection.

llms.txt

llms.txt is a plain Markdown file at the root of your site that gives AI tools a short summary of who you are and links to your most useful pages. It was proposed by Jeremy Howard in September 2024 (llmstxt.org).

Be realistic about it. Google says Google Search does not use llms.txt, and that creating one "neither helps nor harms" your visibility there, though other services may use it (Google Search Central). It is cheap to add and harmless, so it is a reasonable extra, not a ranking trick. As an example, rachyet.com publishes its own /llms.txt with a summary of our services and key pages.

What agents need from your website

Google's web.dev guide to agent-friendly sites is the most practical official advice so far (web.dev). Its main points, translated into plain business terms:

Use real buttons and links. A <button> or <a> is recognised as clickable. A styled box that only looks like a button often is not.
Label every form field. Connect each label to its input so an agent knows which box is "phone" and which is "email".
No hidden layers. Transparent overlays sitting on top of buttons can make agents ignore the button underneath.
Keep layouts stable. Your "Add to cart" or "Book now" button should sit in the same place on every page and not jump around as the page loads.
Say the same thing everywhere. What the page shows, what the code says and what the accessibility tree reports should all describe the same controls.

Much of this is simply good accessibility. A site that works for screen reader users is most of the way there.

What an agent needs	What to add or fix
To understand what you sell and where	Clear headings, plain text and schema.org structured data (Organization, LocalBusiness, Product, Service)
To know the price and whether it is available	Prices, stock and opening hours in visible text and in structured data, kept up to date
To complete a task	Labelled, accessible forms; native buttons; no CAPTCHA on every simple step
To reach the page at all	robots.txt rules that allow the AI crawlers and user agents you want
To trust and reuse a link	Stable, permanent URLs and proper redirects when pages move
To query live data	A product feed, an API or, later, WebMCP or UCP where they fit

Structured data, prices and stable URLs

Google is clear that structured data "isn't required" for its AI search features, but it still recommends using it as part of your overall SEO (Google Search Central). For agents, the value is simple: a machine-readable price, address or opening time removes guesswork. Mark up only what is genuinely on the page.

Show prices and availability where you can. If you sell services with custom quotes, say what affects the price and how to get one.

Keep URLs stable. Agents and AI search tools store and share links, so pages that change address with every redesign lose that trust. Use permanent redirects and a clean sitemap, which is core technical SEO work.

Crawler rules: let the good agents in, keep the bad ones out

Each major AI company uses different bots for different jobs, and robots.txt lets you choose between them. OpenAI, for example, separates OAI-SearchBot (which surfaces sites in ChatGPT search), GPTBot (which collects content that may be used for model training) and ChatGPT-User (visits made because a user asked). OpenAI notes that for user-initiated visits, "robots.txt rules may not apply" (OpenAI). Anthropic similarly separates ClaudeBot (training), Claude-User (fetching pages for a user's question) and Claude-SearchBot (search quality), and warns that blocking the user and search bots may reduce your visibility (Anthropic).

The practical decision for most businesses: you can block training bots if you wish, but blocking the search and user bots means assistants simply cannot see or recommend you. Rachyet's own robots.txt allows the main AI search and assistant crawlers to read the public site, while keeping admin and account areas closed.

Robots.txt is a polite request, not a lock. Bad bots ignore it. Protect forms and logins with rate limiting, bot protection that does not block every automated visitor by default, and monitoring of unusual traffic. Good website security stops scrapers and spam without shutting out the assistants your customers use.

Speed and reliability still decide the outcome

Agents work within time limits, just like people. A page that shows its price only after several scripts run is harder to read and easier to skip. If you want the business case, read our post on website speed and sales, or see what our speed optimisation service covers.

For businesses with live stock, bookings or quotes, the next step is an API or a product feed. It lets an assistant ask your system directly, and it is the groundwork for MCP, WebMCP or UCP later. This is standard API and integrations work, and it often links up with AI integrations you may already be planning.

Checklist: is your website ready for AI agents?

Check your key pages (home, services, products, contact) load their main content in the HTML, without needing heavy scripts.
Replace fake buttons with real <button> and <a> elements.
Give every form field a connected label and a clear error message.
Remove invisible overlays and keep key buttons in a fixed position across pages.
Add accurate schema.org markup for your business, products or services, prices and opening hours.
Show prices, stock or "from" prices in plain text where you can.
Review robots.txt: decide which training bots to allow, and allow the search and user bots you want to be seen by.
Add a short /llms.txt if you like, knowing Google Search does not use it.
Keep URLs stable and redirect anything you move.
Add rate limiting and bot protection to forms and logins.
Test speed and layout stability on a mid-range phone.
If you sell online or take bookings, plan a feed or API, and keep an eye on WebMCP and UCP as they mature.

What is still early

WebMCP is in an experimental trial, Google's agent checkout is rolling out market by market, and llms.txt is a voluntary convention with mixed support. Do not rebuild around a standard that may change. Do the basics well now: they help people, search engines and agents alike.

Getting started

Start by looking at your top five pages through an agent's eyes: can it find what you sell, what it costs and the next step? If you want a second opinion, the Rachyet team can review your site and tell you what to fix first. Get in touch and we will keep it practical.

Sources

Frequently asked questions

What does it mean for a website to be ready for AI agents?

It means an AI assistant can reach your pages, understand what you offer, see prices and availability, and complete simple tasks such as an enquiry or booking. In practice that comes down to clean HTML, real buttons, labelled forms, accurate structured data, fast and stable pages, and robots.txt rules that let the assistants you want read your public content.

Do I need an llms.txt file?

It is optional. llms.txt is a voluntary Markdown summary of your site proposed in 2024. Google says Google Search does not use it and that it neither helps nor harms visibility there, though other AI services may read it. Because it is quick to create and harmless, many sites add one, but it is not a substitute for good content and structure.

Should I block AI crawlers in robots.txt?

It depends on the bot. Companies like OpenAI and Anthropic use separate bots for model training, search and user-requested visits. You can block training bots if you prefer, but blocking search and user bots means assistants may not see or recommend your business. Remember that robots.txt is only a request, so protect forms and logins separately.

What is WebMCP and do I need it now?

WebMCP is a proposed web standard from the Chrome team that lets a website describe its actions, such as booking or checkout, directly to browser-based agents. Google announced at I/O 2026 that an experimental origin trial starts in Chrome 149. It is still changing, so most businesses should focus on accessible forms and structured data first and plan for WebMCP later.

Will AI agents buy from my online shop?

That is starting to happen. Google announced Universal Cart, the Universal Commerce Protocol and the Agent Payments Protocol in May 2026, with a staged rollout starting in the US. To be ready, keep product data, prices and stock accurate in your feed and structured data, make checkout simple and accessible, and consider an API or feed that agents and shopping platforms can use.

Is Your Website Ready for AI Agents? A Practical Checklist

What the "agentic web" actually means