Quick answer: Applebot-Extended is a secondary user agent token in robots.txt, not a crawler. It lets you control whether data crawled by Applebot may be used to train Apple's foundation models behind Apple Intelligence and related features. Blocking it does not stop Applebot from crawling for Siri, Spotlight, and Safari search.
- Operator
- Apple
- Type
- robots.txt control token
- robots.txt token
Applebot-Extended- Follows robots.txt
- Yes, it only exists in robots.txt
- Used for
- Controlling whether Applebot data may be used to train Apple's foundation models
- User agent
No separate user agent. Crawling is done by Applebot.- How to verify
- Nothing to verify in logs: it never appears as a user agent
What is Applebot-Extended?
Apple introduced Applebot-Extended so publishers could separate search inclusion from model training. Applebot still crawls your pages; the token only tells Apple how that data may be used.
Because it is not a crawler, it never shows up in server logs, and it does not fetch anything by itself. It only has meaning inside robots.txt.
What happens if you block Applebot-Extended?
Adding User-agent: Applebot-Extended with Disallow: / tells Apple not to use your content to train its generative models. Your pages can still appear in Siri and Spotlight search features because Applebot keeps crawling.
Should you allow Applebot-Extended?
This is a clean, low-risk way to opt out of Apple model training if that matters to you. It has no known cost to search visibility on Apple devices. If you want Apple's models to learn your brand, leave it allowed.
Check whether Applebot-Extended can reach your site
Enter your domain or a specific page. The checker reads your live robots.txt, picks the group Applebot-Extended would use, and applies the longest matching rule, the same way major crawlers resolve conflicts.
robots.txt rules for Applebot-Extended
Applebot-Extended only exists inside robots.txt, so these rules are the entire control. They do not stop Applebot from crawling; they tell Apple how the crawled content may be used.
User-agent: Applebot-Extended
Allow: /User-agent: Applebot-Extended
Disallow: /The rule most people get wrong: once Applebot-Extended has its own group, it ignores everything under User-agent: *. If your wildcard group disallows paths such as /admin/, repeat those lines inside the Applebot-Extended group too.
Common Apple setup: stay in Siri and Spotlight, opt out of Apple model training
User-agent: Applebot
Allow: /
User-agent: Applebot-Extended
Disallow: /How to verify real Applebot-Extended traffic
There is no traffic to verify. Confirm the rule by reading your live robots.txt, and check that a broader rule, such as a wildcard Disallow, is not also blocking Applebot itself, since that would remove you from Apple search features.
Applebot-Extended compared with other Apple agents
| Agent | Operator | Type | Follows robots.txt |
|---|---|---|---|
| Applebot | Apple | Search crawler | Yes |
| Applebot-Extended | Apple | robots.txt control token | Yes, it only exists in robots.txt |
See all 20 agents side by side in the AI crawler directory.
Applebot-Extended FAQ
See every AI crawler signal at once
The free AI SEO audit checks robots.txt access for 12 AI crawlers, plus llms.txt, schema, content and metadata, in a few seconds.
Run the free AI SEO audit