• Michal Čihař's avatar
    Avoid using wildcards in robots.txt · 38b26f9f
    Michal Čihař authored
    It's supported only by Google and other robots still scan all the pages.
    
    This change required changes in the URLs to have action as the first
    component which can be easily blocked at robots.txt level without
    wildcard or listing all projects in the robots.txt and making it really
    huge.
    
    The old URLs are correctly redirecting to the new ones, though users
    having loaded the page before the switch might see problems due to
    missing POST parameters which will be missed. Unfortunately there is no
    way to avoid this and it is just one time action...
    
    Fixes rb#27
    Signed-off-by: default avatarMichal Čihař <michal@cihar.com>
    38b26f9f
urls.py 20.2 KB