User-agent: * Disallow: /administrator/ Sitemap: https://www.mbtech.info/sitemap.xml # Allow Facebook crawler User-agent: facebookexternalhit Allow: / # Optional: Allow other social crawlers User-agent: Twitterbot Allow: / User-agent: LinkedInBot Allow: /