Skip to content

libai / footseen-large-website

Go to a project
Commit 6a8b25af by libai
Options

测试

parent d2c06f58
Inline Side-by-side
Showing with 8 additions and 7 deletions
robots.txt
User-agent: Baiduspider
Disallow: /
Disallow:/
User-agent: Sosospider
Disallow: /
Disallow:/
User-agent: sogou spider
Disallow: /
Disallow:/
User-agent: YodaoBot
Disallow: /
Disallow:/
User-agent: yisouspider
Disallow:/
User-agent: Googlebot
Disallow:
User-agent: Bingbot
......@@ -14,8 +16,6 @@ User-agent: Slurp
Disallow:
User-agent: Teoma
Disallow:
User-agent: ia_archiver
Disallow:
User-agent: twiceler
Disallow:
User-agent: MSNBot
......@@ -36,5 +36,6 @@ User-agent: yahoo-blogs/v3.9
Disallow:
User-agent: psbot
Disallow:
User-agent: *
Disallow:
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment