[Aminet-commit] r129 - trunk/phpinclude

[aminet-discuss at aminet.net]

Author: mendoza
Date: 2008-04-09 17:57:27 -0700 (Wed, 09 Apr 2008)
New Revision: 129

Modified:
   trunk/phpinclude/robots.txt
Log:
Be abit stricter on search engines

Modified: trunk/phpinclude/robots.txt
===================================================================
--- trunk/phpinclude/robots.txt	2008-02-02 19:26:25 UTC (rev 128)
+++ trunk/phpinclude/robots.txt	2008-04-10 00:57:27 UTC (rev 129)
@@ -5,7 +5,9 @@
 Crawl-delay: 10 
 
 User-agent: *
+Crawl-delay: 10 
 Disallow: /stats
+Disallow: /search
 
 User-Agent: Wget
 Disallow: /