OAI-SearchBot user agent and robots.txt rules
OAI-SearchBot is OpenAI’s crawler for surfacing websites in ChatGPT’s search features. Sites that block it are left out of ChatGPT search answers, though they can still show as navigational links.
User agent
As documented by OpenAI. Version numbers change; match on the token.
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot
robots.txt rules for OAI-SearchBot
Paste one block into the robots.txt at the root of your domain. Changes can take up to a day to be read.
Block OAI-SearchBot
Disallows every path for OAI-SearchBot. That keeps your pages out of OpenAI’s AI search index, so its answers can’t cite them.
User-agent: OAI-SearchBot Disallow: /
Allow OAI-SearchBot
Lets OAI-SearchBot fetch every path, whatever “User-agent: *” says.
User-agent: OAI-SearchBot Allow: /
Good to know
OpenAI recommends allowing it if you want to appear in ChatGPT search. Changes take about 24 hours.
Other OpenAI crawlers
GPTBot
GPTBot is OpenAI’s crawler for content that may be used to train its generative AI foundation…
User-triggeredChatGPT-User
ChatGPT-User fetches pages for actions people start in ChatGPT and custom GPTs, such as…
OtherOAI-AdsBot
OAI-AdsBot checks the landing pages of ads submitted to ChatGPT, for safety and to decide when…
OAI-SearchBot FAQ
What is OAI-SearchBot?
OAI-SearchBot is OpenAI’s crawler for surfacing websites in ChatGPT’s search features. Sites that block it are left out of ChatGPT search answers, though they can still show as navigational links.
How do I block OAI-SearchBot?
Add “User-agent: OAI-SearchBot” and “Disallow: /” to your robots.txt. That keeps your pages out of OpenAI’s AI search index, so its answers can’t cite them.
Does OAI-SearchBot respect robots.txt?
Yes. OpenAI says OAI-SearchBot follows robots.txt rules written for it.
What is the OAI-SearchBot user agent?
OpenAI documents it as “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36; compatible; OAI-SearchBot/1.4; +https://openai.com/searchbot”. Version numbers change, so match on “OAI-SearchBot”.
How can I verify OAI-SearchBot requests?
Match the request IP against OpenAI’s published list (https://openai.com/searchbot.json). The user agent alone can be faked.
Source: developers.openai.com, checked Nov 17, 2026. AI Crawler Directory
Files are the start. Answers are the result.
Your buyers’ questions on eight AI engines, on your schedule, with history. Free for 7 days.