scrapy.pipelines.media.MediaPipeline

class documentation

class MediaPipeline:

Known subclasses: scrapy.pipelines.files.FilesPipeline

View In Hierarchy

Undocumented

Class Method from​_crawler Undocumented
Constant LOG​_FAILED​_RESULTS Undocumented
Class ​Spider​Info Undocumented
Method __init__ Undocumented
Method ​_cache​_result​_and​_execute​_waiters Undocumented
Method ​_check​_media​_to​_download Undocumented
Method ​_check​_signature Undocumented
Method ​_compatible Wrapper for overridable methods to allow backwards compatibility
Method ​_handle​_statuses Undocumented
Method ​_key​_for​_pipe No summary
Method ​_make​_compatible Make overridable methods of MediaPipeline and subclasses backwards compatible
Method ​_modify​_media​_request Undocumented
Method ​_process​_request Undocumented
Method file​_path Returns the path where downloaded media should be stored
Method get​_media​_requests Returns the media requests to download
Method item​_completed Called per item when all media requests has been processed
Method media​_downloaded Handler for success downloads
Method media​_failed Handler for failed downloads
Method media​_to​_download Check request before starting download
Method open​_spider Undocumented
Method process​_item Undocumented
Instance Variable ​_expects​_item Undocumented
Instance Variable allow​_redirects Undocumented
Instance Variable download​_func Undocumented
Instance Variable handle​_httpstatus​_list Undocumented
Instance Variable spiderinfo Undocumented
@classmethod
def from_crawler(cls, crawler):

Undocumented

LOG_FAILED_RESULTS: bool =

Undocumented

Value
True
def __init__(self, download_func=None, settings=None):
overridden in scrapy.pipelines.files.FilesPipeline

Undocumented

def _cache_result_and_execute_waiters(self, result, fp, info):

Undocumented

def _check_media_to_download(self, result, request, info, item):

Undocumented

def _check_signature(self, func):

Undocumented

def _compatible(self, func):
Wrapper for overridable methods to allow backwards compatibility
def _handle_statuses(self, allow_redirects):

Undocumented

def _key_for_pipe(self, key, base_class_name=None, settings=None): 
>>> MediaPipeline()._key_for_pipe("IMAGES")
'IMAGES'
>>> class MyPipe(MediaPipeline):
...     pass
>>> MyPipe()._key_for_pipe("IMAGES", base_class_name="MediaPipeline")
'MYPIPE_IMAGES'
def _make_compatible(self):
Make overridable methods of MediaPipeline and subclasses backwards compatible
def _modify_media_request(self, request):

Undocumented

def _process_request(self, request, info, item):

Undocumented

def file_path(self, request, response=None, info=None, *, item=None):
overridden in scrapy.pipelines.files.FilesPipeline
Returns the path where downloaded media should be stored
def get_media_requests(self, item, info):
overridden in scrapy.pipelines.files.FilesPipeline
Returns the media requests to download
def item_completed(self, results, item, info):
overridden in scrapy.pipelines.files.FilesPipeline
Called per item when all media requests has been processed
def media_downloaded(self, response, request, info, *, item=None):
overridden in scrapy.pipelines.files.FilesPipeline
Handler for success downloads
def media_failed(self, failure, request, info):
overridden in scrapy.pipelines.files.FilesPipeline
Handler for failed downloads
def media_to_download(self, request, info, *, item=None):
overridden in scrapy.pipelines.files.FilesPipeline
Check request before starting download
def open_spider(self, spider):

Undocumented

def process_item(self, item, spider):

Undocumented

_expects_item: dict =

Undocumented

allow_redirects =

Undocumented

download_func =

Undocumented

handle_httpstatus_list =

Undocumented

spiderinfo =

Undocumented