AI Crawler Controls

Block or allow AI crawlers like GPTBot, ClaudeBot and CCBot one by one through robots.txt, opt-out headers and an optional 403, and see every visit they make in a crawl log.

AI Crawler Controls puts the AI crawlers that visit your site on one screen, each with its own switch. AI companies send crawlers for different reasons: some collect pages to train their models, some index pages for AI search answers, and some fetch a page when a person asks an assistant to read it. The plugin lists 25 of them, from GPTBot and ClaudeBot to Google-Extended, PerplexityBot and Bytespider, and explains what each one does before you decide.

Blocking a crawler adds a rule for it to your robots.txt, the standard opt-out every major AI company says its crawlers follow. One click blocks AI training while keeping you visible in AI search, so ChatGPT search and Perplexity can still quote and link to you. For crawlers that don't listen, turn on Refuse requests and they get a 403 error instead of your page. Optional noai and TDM signals ask everyone else not to use your content.

A crawl log records every visit from an AI crawler with the page, the result, the status code and the IP address, plus a 30-day chart and a last-seen time for each crawler, so you can see who's crawling and whether your rules are respected. Nothing changes until you choose, your dashboard and login page are never refused, and regular visitors are never logged. Find it under Content Powered > AI Crawler Controls, and get updates through the normal WordPress Updates screen.

Features

  • 25 AI crawlers, one switch each. GPTBot, ClaudeBot, Google-Extended, PerplexityBot, Bytespider and more, each explained.
  • Block training, keep AI search. One click keeps your content out of training data while AI search can still cite you.
  • robots.txt rules for you. A Disallow rule for each blocked crawler, and a live test that confirms they're there.
  • Refuse crawlers that don't listen. Blocked crawlers that ignore robots.txt can get a 403 error instead of your page.
  • AI opt-out signals. Optional noai, noimageai and TDM reservation signals on every page.
  • A log of every visit. Each request from an AI crawler, with the page, result, status code and IP address.
  • 30-day activity chart. See how often AI crawlers visit and how many requests were refused.
  • Rule breakers flagged. Blocked crawlers that ask for pages anyway are marked in the log.
  • No lockouts. Your dashboard and login are never refused, and tokens that match browsers are rejected.
  1. Click Download above to save ai-crawler-controls.zip. Leave it zipped.
  2. In your WordPress dashboard, go to Plugins > Add New Plugin and click Upload Plugin.
  3. Choose the ZIP and click Install Now, then Activate Plugin.

To update later, upload a newer ZIP the same way. WordPress offers to replace the installed version and keeps your settings.

Will Blocking AI Crawlers Hurt My SEO?

No. None of the crawlers in the list is Googlebot or Bingbot, and search engines ignore the noai and TDM signals. Google-Extended only controls whether Gemini may use your pages; it doesn't affect Google Search or AI Overviews.

Which AI Crawlers Should I Block?

Most sites block AI training and allow AI search, which is what the Block AI training button does. Your content stays out of future training data, while ChatGPT search, Perplexity and other AI search tools can still quote your pages and send visitors to them.

Do AI Crawlers Respect robots.txt?

OpenAI, Anthropic, Google, Apple, Meta, Perplexity and Common Crawl say their crawlers do. Some crawlers don't, and some assistants skip robots.txt when a person asked for the page. The crawl log flags any blocked crawler that requests pages anyway, and Refuse requests turns those requests away with a 403.

What's the Difference Between Training and Search Crawlers?

Training crawlers like GPTBot and ClaudeBot collect pages to build AI models. Search crawlers like OAI-SearchBot and PerplexityBot index pages so AI search tools can cite and link to them. Assistant fetchers like ChatGPT-User visit a single page when someone asks an assistant to read it. The plugin groups them this way so you can treat each kind differently.

Can I Block a Crawler That Isn't Listed?

Yes. Add its user-agent token under Other crawlers to block and it's disallowed in robots.txt, and refused too if Refuse requests is on. Tokens that would also match web browsers, search engines or link previews are rejected, so you can't turn real visitors away by mistake.

Does It Work With Caching Plugins?

Yes. The robots.txt rules and the opt-out meta tags work with any cache. WP Rocket is set up automatically so refused crawlers reach the 403. Other page caches and CDNs that answer before WordPress loads may still serve cached pages to blocked crawlers, and those visits won't appear in the log.

How Do I Get Updates?

Through the normal WordPress Updates screen. The plugin checks our GitHub releases twice a day, and you can update with one click or turn on auto-updates from the Plugins screen.

Download

Download Plugin

By downloading, you agree to the .

All Time Downloads
79

Download activity

More Free WordPress Plugins

Let's Grow Your Business

Want some free consulting? Let's hop on a call and talk about what we can do to help.

WordPress Plugin Terms of Use

Last updated October 26, 2026

These terms apply to every free plugin you download from the WordPress Plugins section of contentpowered.com (the "plugins"). By downloading, installing or using a plugin, you agree to them. If you don't agree, please don't download or use the plugins.

1. Provided "as is"

The plugins are free and are provided "as is" and "as available", without warranty of any kind, express or implied. That includes any warranty of merchantability, fitness for a particular purpose, non-infringement or security, and any promise that a plugin will be free of errors, compatible with your site, or kept up to date.

2. You use them at your own risk

You're responsible for deciding whether a plugin is right for your site, for testing it, for keeping backups, and for keeping WordPress, your theme, your plugins (ours included) and your hosting up to date and secure. We recommend trying any new plugin on a staging copy of your site first.

3. No guarantee of updates or support

We may update, change or stop maintaining any plugin at any time, without notice. Older versions may contain bugs or security vulnerabilities, and it's up to you to install updates. We don't promise to fix problems, answer questions or provide support, though you're welcome to report issues from inside a plugin (Content Powered > Report a Bug in your WordPress dashboard).

4. Limitation of liability

To the fullest extent the law allows, Content Powered and its owners, employees and contractors are not liable for any direct, indirect, incidental, special, consequential or punitive damages, or for any loss of data, revenue, profits, search rankings, traffic or business, arising from downloading, installing, using or being unable to use a plugin. This includes hacks, security breaches, malware, downtime, conflicts with other software, and problems caused by outdated versions. If we are found liable despite this, our total liability is limited to the amount you paid us for the plugin.

5. You agree to hold us harmless

You agree to indemnify and hold harmless Content Powered and its owners, employees and contractors from any claims, losses, damages, liabilities and costs, including reasonable legal fees, that arise from your use of a plugin or your breach of these terms.

6. License

Each plugin is released under the license stated in its files, usually the GNU General Public License (GPL) version 2 or later. The GPL lets you use, study, change and share the plugin, and it also says the software comes with no warranty. Nothing in these terms takes away the rights a plugin's license gives you.

7. Third-party code and services

Some plugins include code from other projects or connect to services run by other companies, which have their own licenses and terms. We aren't responsible for them.

8. Changes to these terms

We may update these terms at any time. The version shown here applies from the date above.

9. Our other terms

These terms add to our website Terms and Conditions and Privacy Statement. Where they conflict, these terms apply to the plugins. Questions? Contact us.

Report an Issue

The best way to report an issue is from inside the plugin:

  1. Log in to your WordPress dashboard at yoursite.com/wp-admin/.
  2. In the left-hand menu, go to Content Powered > Report a Bug.

Can't get into your dashboard? Contact us instead.

Share
Tweet
Share
Pin
Email