Free tool

Robots.txt checker

domain: github.com
status: Allowed
fetch_status: ok
robots_url: https://github.com/robots.txt