Free tool — no signup needed

    AI Crawler Checker

    ChatGPT, Claude, Perplexity and Gemini can only cite your site if their crawlers can read it. One line in robots.txt decides. Check yours in seconds.

    3 free checks a day without an account. Sign in free for more.

    Embeddable

    Put AI Crawler Checker on your own website

    Turn this tool into a lead magnet on your site. Your visitors get instant free results — you get engaged prospects.

    1

    Create an embed key

    Open Settings → Embeds, pick this tool, and set your allowed domains and daily visitor limit.

    2

    Paste one script tag

    Copy the generated snippet into any page on your site. No build step, no dependencies.

    3

    Your visitors run it

    The widget auto-resizes, matches your theme, and every run links back to a full report.

    Works on any website

    WordPress, Webflow, Shopify, or plain HTML — if you can paste a script tag, it works.

    You control the throttle

    Set per-visitor daily limits so embeds never burn through your monthly quota unexpectedly.

    Light AeroSearch badge

    Free embeds carry a small 'Powered by AeroSearch' badge. Paid plans can remove it entirely.

    Sign in free to embed

    Embedding is included on every plan, free included.

    The new gatekeepers of your traffic

    For twenty years, one crawler decided whether the world could find you: Googlebot. Today there are dozens — GPTBot and OAI-SearchBot from OpenAI, ClaudeBot from Anthropic, PerplexityBot, Google-Extended, Applebot-Extended, Meta-ExternalAgent, and more. Each one is controlled by your robots.txt file, and a single misplaced line can quietly make your business invisible to every AI assistant your customers ask for recommendations.

    Training bots vs search bots

    The most important distinction: training crawlers (GPTBot, ClaudeBot, CCBot) collect content to train future models, while search crawlers (OAI-SearchBot, PerplexityBot, Claude-SearchBot) fetch pages to answer live questions with citations. Blocking training bots protects your content; blocking search bots costs you customers. This checker separates the two so you can decide deliberately instead of accidentally.

    Frequently asked questions

    What is an AI crawler checker?

    It reads your robots.txt and reports whether AI bots such as GPTBot, ClaudeBot, PerplexityBot and Google-Extended are allowed to read your website.

    Should I block GPTBot?

    Blocking GPTBot stops OpenAI training on your content but does not remove you from ChatGPT search answers — OAI-SearchBot controls that. Many sites block training bots and allow search bots.

    What if I have no robots.txt?

    Without a robots.txt file, every crawler is allowed by default — including every AI bot. That is fine if you want maximum visibility, but you have no control.

    Is this AI crawler checker free?

    Yes. Run checks without an account, sign in free for higher limits, or embed the checker on your own website.