# /robots.txt file for gsa-crawler User-agent: * Disallow: / User-agent: gsa-crawler Disallow: