Commit 2296b357 authored by Gary Holtz's avatar Gary Holtz

Adding updated scope to robots.text

With some projects falling under the '-' scope, we'll need some updated
robots.txt lines
parent 9eb2d34f
...@@ -35,6 +35,7 @@ Disallow: /snippets/*/raw ...@@ -35,6 +35,7 @@ Disallow: /snippets/*/raw
User-Agent: * User-Agent: *
Disallow: /*/*.git Disallow: /*/*.git
Disallow: /*/*/fork/new Disallow: /*/*/fork/new
Disallow: /*/-/archive/
Disallow: /*/*/repository/archive* Disallow: /*/*/repository/archive*
Disallow: /*/*/activity Disallow: /*/*/activity
Disallow: /*/*/new Disallow: /*/*/new
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment