scrapy.pqueues.DownloaderAwarePriorityQueue

class documentation

class DownloaderAwarePriorityQueue:

View In Hierarchy

PriorityQueue which takes Downloader activity into account: domains (slots) with the least amount of active downloads are dequeued first.
Class Method from​_crawler Undocumented
Method __contains__ Undocumented
Method __init__ Undocumented
Method __len__ Undocumented
Method close Undocumented
Method pop Undocumented
Method pqfactory Undocumented
Method push Undocumented
Instance Variable ​_downloader​_interface Undocumented
Instance Variable crawler Undocumented
Instance Variable downstream​_queue​_cls Undocumented
Instance Variable key Undocumented
Instance Variable pqueues Undocumented
@classmethod
def from_crawler(cls, crawler, downstream_queue_cls, key, startprios=()):

Undocumented

def __contains__(self, slot):

Undocumented

def __init__(self, crawler, downstream_queue_cls, key, slot_startprios=()):

Undocumented

def __len__(self):

Undocumented

def close(self):

Undocumented

def pop(self):

Undocumented

def pqfactory(self, slot, startprios=()):

Undocumented

def push(self, request):

Undocumented

_downloader_interface =

Undocumented

crawler =

Undocumented

downstream_queue_cls =

Undocumented

key =

Undocumented

pqueues: dict =

Undocumented