Skip to content

libai / footseen-large-website

Go to a project
Commit 6a8b25af by libai
Options

测试

parent d2c06f58
Inline Side-by-side
Showing with 8 additions and 7 deletions
robots.txt
User-agent: Baiduspider User-agent: Baiduspider
Disallow: / Disallow:/
User-agent: Sosospider User-agent: Sosospider
Disallow: / Disallow:/
User-agent: sogou spider User-agent: sogou spider
Disallow: / Disallow:/
User-agent: YodaoBot User-agent: YodaoBot
Disallow: / Disallow:/
User-agent: yisouspider
Disallow:/
User-agent: Googlebot User-agent: Googlebot
Disallow: Disallow:
User-agent: Bingbot User-agent: Bingbot
...@@ -14,8 +16,6 @@ User-agent: Slurp ...@@ -14,8 +16,6 @@ User-agent: Slurp
Disallow: Disallow:
User-agent: Teoma User-agent: Teoma
Disallow: Disallow:
User-agent: ia_archiver
Disallow:
User-agent: twiceler User-agent: twiceler
Disallow: Disallow:
User-agent: MSNBot User-agent: MSNBot
...@@ -36,5 +36,6 @@ User-agent: yahoo-blogs/v3.9 ...@@ -36,5 +36,6 @@ User-agent: yahoo-blogs/v3.9
Disallow: Disallow:
User-agent: psbot User-agent: psbot
Disallow: Disallow:
User-agent: *
Disallow:
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or sign in to comment