User-agent: panscient.com Disallow: / User-agent: Mozilla/5.0 (compatible;picmole/1.0 +http://www.picmole.com) Disallow: / User-agent: LexxeBot/1.0 Disallow: / User-agent: NextGenSearchBot Disallow: / User-agent: Mozilla/5.0 (compatible; spbot/2.0; http://www.seoprofiler.com/bot/ ) Disallow: / User-agent: Sosospider Disallow: / User-agent: Sosospider+(+http://help.soso.com/webspider.htm) Disallow: / User-agent: rogerbot Disallow: / User-agent: SiteBot/0.1 Disallow: / User-agent: SiteBot Disallow: / User-agent: MJ12bot Disallow: / User-agent: crystalsemanticsbot Disallow: / User-agent: CrystalSemanticsBot Disallow: / User-agent: NetSeer crawler Disallow: / User-agent: trovitBot Disallow: / User-agent: LexxeBot Disallow: / User-agent: DotBot Disallow: / User-agent: Ezooms Disallow: / User-agent: discobot Disallow: / User-agent: FatBot Disallow: / User-agent: MaxPointCrawler Disallow: / User-agent: Jyxobot Disallow: / User-agent: sistrix Disallow: / User-agent: heritrix Disallow: / User-agent: GarlikCrawler/1.1 (http://garlik.com/, crawler@garlik.com) Disallow: / User-agent: AhrefsBot Disallow: / User-agent: NerdByNature.Bot Disallow: / User-agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows NT; DigExt; DTS Agent Disallow: / User-agent: psbot Disallow: / User-agent: WBSearchBot Disallow: / User-agent: AddThis.com robot tech.support@clearspring.com Disallow: / User-agent: AddThis.com Disallow: / User-agent: ia_archiver Disallow: / User-agent: proximic Disallow: / User-agent: discoverybot Disallow: / User-agent: bl.uk_lddc_bot Disallow: / User-agent: IstellaBot Disallow: / User-agent: seokicks Disallow: / User-agent: SEOkicks-Robot Disallow: / User-agent: UnisterBot Disallow: / User-agent: Bender Disallow: / User-agent: wotbox Disallow: / User-agent: Yasni Disallow: / User-agent: JikeSpider Disallow: / User-agent: Screaming Frog SEO Spider Disallow: / User-agent: netEstate NE Crawler Disallow: / User-agent: Exabot Disallow: / User-agent: Pixray-Seeker Disallow: / User-agent: Linguee Disallow: / User-agent: integromedb Disallow: / User-agent: SearchmetricsBot Disallow: / User-agent: SemrushBot Disallow: / User-agent: BLEXBot Disallow: / User-agent: BDCbot Disallow: / User-agent: GrapeshotCrawler Disallow: / User-agent: WeSEE:Search Disallow: / User-agent: TurnitinBot Disallow: / User-agent: admantx Disallow: / User-agent: spbot Disallow: / User-agent: BUbiNG Disallow: / User-agent: SeznamBot Disallow: / User-agent: MegaIndex.ru/2.0 Disallow: / User-agent: MegaIndex.ru Disallow: / User-agent: MegaIndex Disallow: / User-agent: ltx71 Disallow: / User-agent: SafeSearch Disallow: / User-agent: ICC-Crawler Disallow: / User-agent: Clickagy Intelligence Bot v2 Disallow: / User-agent: HybridBot Disallow: / User-agent: Mail.RU_Bot Disallow: / User-agent: Baiduspider User-agent: Baiduspider-video User-agent: Baiduspider-image User-agent: Baiduspider-render User-agent: Baiduspider-news User-agent: Baiduspider-favo User-agent: Baiduspider-ads User-agent: Baiduspider-cpro User-agent: Baiduspider+ Disallow: / User-agent: * Disallow: /transfer Disallow: /book Disallow: /feed/ Disallow: /ma/ Disallow: /review Disallow: /protected/ Disallow: /api/ Disallow: /location-not-found Disallow: /auth Disallow: /users Disallow: /console Disallow: /consolev2 Disallow: /markercode Disallow: /adminassistants Disallow: /locations Disallow: /bookings Disallow: /clp Disallow: /autocomplete