What else we measure here

10.3%
Sites with an AI crawler policy

Sites with an AI crawler policy

Sample
n=107
Coverage
100.0%
96.5%
Agents that can reach the site

Agents that can reach the site

Sample
n=107
Coverage
100.0%
37.9%
Sites serving an llms.txt

Sites serving an llms.txt

Sample
n=107
Coverage
100.0%
15.8%
Sites with a valid llms.txt

Sites with a valid llms.txt

Sample
n=107
Coverage
100.0%

How this was measured