Module for processing Sitemaps.
Note: The main purpose of this module is to provide support for the SitemapSpider, its API is subject to change without notice.
| Class | Sitemap |
Class to parse Sitemap (type=urlset) and Sitemap Index (type=sitemapindex) files |
| Function | sitemap_urls_from_robots |
Return an iterator over all sitemap urls contained in the given robots.txt file |