User-agent: Slurp Disallow: * Crawl-delay: 10 User-agent: dotbot Disallow: / Disallow: /article/ Disallow: * User-agent: twiceler Disallow: /