scrapy.downloadermiddlewares.ajaxcrawl

module documentation

Undocumented

Class ​Ajax​Crawl​Middleware Handle 'AJAX crawlable' pages marked as crawlable via meta tag. For more info see https://developers.google.com/webmasters/ajax-crawling/docs/getting-started.
Variable logger Undocumented
Function ​_has​_ajaxcrawlable​_meta No summary
Variable ​_ajax​_crawlable​_re Undocumented
logger =

Undocumented

def _has_ajaxcrawlable_meta(text): 
>>> _has_ajaxcrawlable_meta('<html><head><meta name="fragment"  content="!"/></head><body></body></html>')
True
>>> _has_ajaxcrawlable_meta("<html><head><meta name='fragment' content='!'></head></html>")
True
>>> _has_ajaxcrawlable_meta('<html><head><!--<meta name="fragment"  content="!"/>--></head><body></body></html>')
False
>>> _has_ajaxcrawlable_meta('<html></html>')
False
_ajax_crawlable_re =

Undocumented