GPTBot

Often unwantedai

Operated by OpenAI

What is GPTBot?

GPTBot is OpenAI's official web crawler designed to scrape publicly accessible web content for training and improving its generative AI models, including ChatGPT. It operates under a specific user-agent token to identify its requests.

Why it visits websites

GPTBot visits websites to collect data for training AI models, ensuring they can generate accurate and up-to-date responses. It may also be used to ground generative responses by referencing real-time web content.

Is GPTBot good or bad for your site?

GPTBot can consume significant server resources due to its data collection activities, potentially impacting site performance. While it may drive traffic indirectly by improving AI model accuracy, it does not contribute to traditional SEO or user engagement. Some site owners may prefer to block it to conserve bandwidth and control data usage.

How to block GPTBot

To block GPTBot via robots.txt, add the following directives: User-agent: GPTBot Disallow: / To block it by user-agent, configure your server to reject requests containing the string 'GPTBot'. Example for Apache .htaccess: RewriteEngine On RewriteCond %{HTTP_USER_AGENT} GPTBot [NC] RewriteRule .* - [F,L]

Example user-agent

Mozilla/5.0 (compatible; GPTBot/1.0; +https://openai.com/gptbot)

Related bots

OAI-SearchBot

See which bots hit your site

Gabden Analytics separates real people from crawlers and flags AI scrapers — privately, with no cookies and no IPs stored.

Start free