Cloudflare blocks AI crawlers but keeps them in search results

Cloudflare updates its rules to block crawlers used for AI training.

Published 1sem2 sources✓ ConfirmedImportantupdated 1sem

≈ 17sSpeed

The fact

The measure targets multi-purpose bots while maintaining their indexing in search results.

This decision could impact practices across the tech industry.

2 sources — click a link to read an article on the topic:

Explore this topic

Auto-synthesis from 2 media sources · identified on September 18, 2026