Better sitemap and robots checker

Currently, the links are just static links. With the updated checker, we will parse the robots.txt, check for references to sitemaps, and, if so, display all sitemaps.

If no reference is found, we check if there is a sitemap.xml.

If there is no reference in robots.txt, it shows a soft error; if there is no reference and no sitemap.xml, it shows an error.

Comments

To leave a comment, please authenticate.

No comments yet
Voters
-
Status
Completed
Board
Feature
Submitted
6 months ago, Hans