Bingbot: the quiet key to Copilot and ChatGPT visibility

Operator

Microsoft

Type

Search engine crawler

robots.txt token

Bingbot

Follows robots.txt

Yes

Used for

Crawling for Bing, which grounds Microsoft Copilot and is a search source for other AI products

User agent

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/ Safari/537.36

How to verify

Reverse DNS must end in search.msn.com, confirmed by a forward lookup, or use Bing Webmaster Tools' Verify Bingbot tool

What is Bingbot?

AI assistants that answer with live sources need a search index to query. Microsoft Copilot is grounded in Bing, and Bing has been a documented search provider behind ChatGPT search. That makes Bingbot access, and actual indexing in Bing, a prerequisite for a large share of AI citations.

Bing also supports IndexNow, a protocol that lets you notify it the moment a URL is published or changed. For a new or rarely crawled site, IndexNow shortens the gap between publishing and being eligible to appear in Copilot answers.

What happens if you block Bingbot?

Blocking Bingbot removes your pages from Bing, which also removes them from Copilot answers grounded in Bing and weakens your chances in other AI products that rely on Bing results. Bing honors a Crawl-delay directive, which is a safer way to reduce load than blocking.

Should you allow Bingbot?

Always allow Bingbot. Then go further: verify your site in Bing Webmaster Tools, submit your sitemap, and ping IndexNow when you publish. A site that is indexed in Google but not in Bing is invisible to a surprising number of AI answers.

Check whether Bingbot can reach your site

Enter your domain or a specific page. The checker reads your live robots.txt, picks the group Bingbot would use, and applies the longest matching rule, the same way major crawlers resolve conflicts.

robots.txt rules for Bingbot

Put each group in the robots.txt file at the root of your domain. User agent names are matched without regard to case, and the most specific matching group wins.

User-agent: Bingbot
Allow: /

User-agent: Bingbot
Disallow: /

User-agent: Bingbot
Disallow: /account/
Disallow: /checkout/
Allow: /

The rule most people get wrong: once Bingbot has its own group, it ignores everything under User-agent: *. If your wildcard group disallows paths such as /admin/, repeat those lines inside the Bingbot group too.

How to verify real Bingbot traffic

Genuine Bingbot requests come from IPs whose reverse DNS ends in search.msn.com, and the forward lookup returns the same IP. Bing Webmaster Tools also includes a Verify Bingbot tool that checks an IP for you.

To see how often it visits, count requests by user agent in your access log. On a typical Nginx server:

grep -i "Bingbot" /var/log/nginx/access.log | awk '{print $4}' | cut -d: -f1 | sort | uniq -c

Bingbot compared with similar agents

See all 20 agents side by side in the AI crawler directory.

Bingbot FAQ

Abd Shanti

Co-founder and strategy lead at Outline Technologies, the team behind FreeGPTSEO and AI Citation Monitor. Abd works on how brands get found and cited by search engines and AI assistants.