Commit Graph

1 Commits

Author SHA1 Message Date
Girish Ramakrishnan
acd00222e5 Allow per-app configuration of robots.txt
https://developers.google.com/search/reference/robots_txt has
the specification

Part of #344
2017-07-14 15:25:05 -05:00