- to prevent pages from appearing in search results
- to tell search engines about which of your site’s pages you’d like them to crawl
- to prevent your site from being overloaded with requests by crawlers
- to control exactly which pages appear in search results