Category
Search & AI
How search engines and AI assistants find, read and cite websites: crawlers, robots.txt, llms.txt and structured data, measured on the Polish web.
Sort by
- Search & AISitemap in robots.txt: a third of Polish files omit it35% of 6,834 .pl sites with a robots.txt name no sitemap in it, and 6% of those that do point to a wrong address: http, the www twin or a relative path.
- Search & AInosnippet on the Polish web: almost nobody limits Google7 of 7,785 .pl homepages limit Google's snippet, while a quarter ask for bigger previews. None of the 1,015 that block AI crawlers in robots.txt use nosnippet.
- Search & AIAI crawlers vs the Polish web: robots.txt on 1,000 .pl sitesWe read robots.txt on the 1,000 top .pl sites: one in five blocks AI crawlers like GPTBot, mostly for training, rarely AI search. How to split the two.
Tell us what we’re building.
We reply within one business day.