GPTBot

What it is

GPTBot is the crawler OpenAI uses to gather public web content for one purpose: training future models. It reads your raw HTML, follows links, and adds what it finds to the data OpenAI uses to teach its systems. It is not the bot that decides whether you show up in ChatGPT's answers today: that is OAI-SearchBot, the retrieval crawler. A third agent, ChatGPT-User, handles user-initiated fetches and GPT Actions. GPTBot respects robots.txt, so a single rule blocks it cleanly (per OpenAI's bots documentation).

Why it matters

Blocking GPTBot is a training decision, not a visibility one. Training data is frozen at a model's cutoff, so letting GPTBot in does not make you appear in ChatGPT's live answers, and blocking it does not remove you from them. If your goal is to be cited by ChatGPT, the bot to allow is OAI-SearchBot, not this one. One catch: GPTBot does not run JavaScript. If your content only renders client-side, it sees a blank page, so server-render the parts that matter. See how the AI crawlers actually work and the OAI-SearchBot entry for the retrieval side.