Guides
Guides for keeping domains healthy
Field notes on the problems Robotto checks for. Each guide explains what goes wrong, how to spot it on your own sites, and what to change.
-
01
Ten robots.txt mistakes that quietly kill traffic
A leftover staging rule, a blocked CSS folder, a file that returns your home page. Most robots.txt damage comes from a short list of mistakes.
5 min read
-
02
Letting AI crawlers in on purpose: a GEO checklist
Being cited in AI answers starts with letting the right crawlers read your pages. Here's how to choose which ones and check that your CDN agrees.
4 min read
-
03
www vs non-www: picking one and redirecting the rest
It barely matters which host you choose. It matters a lot that every other version redirects to it in one step.
4 min read
-
04
How to track domain expiry across a portfolio
Auto-renew fails more often than people think. A domain portfolio needs one list, one source of truth for dates, and a few deadlines you actually look at.
4 min read
-
05
Finding the pages you lost in a site migration
After a redesign or platform move, the traffic you lose is mostly on URLs nobody remembered. Here's how to rebuild the list and redirect what matters.
4 min read
-
06
How to read a redirect chain
Every hop in a redirect chain is a status code, a Location header and a decision someone made. Once you can read one, most redirect problems are easy to spot.
4 min read