#This script specifies the files and folders on GandaAncestry.com that shouldn't #be indexed by search engine spiders. #User-agent field specifies the robot, for example googlebot. In this case all robots User-agent: * #Disallow prevents the robot from indexing/downloading the specified folders Disallow: /add/ Disallow: /admin/ Disallow: /app/ Disallow: /appadmin/ Disallow: /crm/ Disallow: /members/ Disallow: /store/ Disallow: /temp/ #Disallow prevents the robot from indexing any dynamic urls Disallow: /?