OAI-SearchBot user agent and robots.txt rules

OAI-SearchBot is OpenAI’s crawler for surfacing websites in ChatGPT’s search features. Sites that block it are left out of ChatGPT search answers, though they can still show as navigational links.

AI search Follows robots.txt OpenAI

User agent

As documented by OpenAI. Version numbers change; match on the token.

User agent
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot

robots.txt rules for OAI-SearchBot

Paste one block into the robots.txt at the root of your domain. Changes can take up to a day to be read.

Block OAI-SearchBot

Disallows every path for OAI-SearchBot. That keeps your pages out of OpenAI’s AI search index, so its answers can’t cite them.

robots.txt
User-agent: OAI-SearchBot
Disallow: /

Allow OAI-SearchBot

Lets OAI-SearchBot fetch every path, whatever “User-agent: *” says.

robots.txt
User-agent: OAI-SearchBot
Allow: /

Good to know

OpenAI recommends allowing it if you want to appear in ChatGPT search. Changes take about 24 hours.

OAI-SearchBot FAQ

What is OAI-SearchBot?

OAI-SearchBot is OpenAI’s crawler for surfacing websites in ChatGPT’s search features. Sites that block it are left out of ChatGPT search answers, though they can still show as navigational links.

How do I block OAI-SearchBot?

Add “User-agent: OAI-SearchBot” and “Disallow: /” to your robots.txt. That keeps your pages out of OpenAI’s AI search index, so its answers can’t cite them.

Does OAI-SearchBot respect robots.txt?

Yes. OpenAI says OAI-SearchBot follows robots.txt rules written for it.

What is the OAI-SearchBot user agent?

OpenAI documents it as “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot”. Version numbers change, so match on “OAI-SearchBot”.

How can I verify OAI-SearchBot requests?

Match the request IP against OpenAI’s published list (https://openai.com/searchbot.json). The user agent alone can be faked.

Source: developers.openai.com, checked Nov 17, 2026. AI Crawler Directory

Files are the start. Answers are the result.

Your buyers’ questions on eight AI engines, on your schedule, with history. Free for 7 days.