# robots.txt User-agent: twiceler Disallow: / User-agent: Sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm#07) Disallow: / User-agent: MLBot (www.metadatalabs.com/mlbot) Disallow: / User-agent: radian6_linkcheck_(www.radian6.com/crawler) Disallow: / User-agent: Mozilla/5.0 (compatible; mxbot/1.0; +http://www.chainn.com/mxbot.html) Disallow: / User-agent: Mozilla/5.0 (compatible; abby/1.0; +http://www.ellerdale.com/crawler.html) Disallow: / User-agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.7) Gecko/20091221 Firefox/3.5.7 OneRiot/1.0 (http://www.oneriot.com) Disallow: / User-agent: Mozilla/5.0 (compatible; MSIE 6.0b; Windows NT 5.0) Gecko/2009011913 Firefox/3.0.6 TweetmemeBot Disallow: / User-agent: * Disallow: