org.archive.crawler.frontier
Class BucketQueueAssignmentPolicy

java.lang.Object
  org.archive.crawler.frontier.QueueAssignmentPolicy
      org.archive.crawler.frontier.BucketQueueAssignmentPolicy

public class BucketQueueAssignmentPolicy
extends QueueAssignmentPolicy

Uses the target IPs as basis for queue-assignment, distributing them over a fixed number of sub-queues.

Author:

Christian Kohlschuetter

Constructor Summary

BucketQueueAssignmentPolicy()

Method Summary

java.lang.String	getClassKey(CrawlController controller, CandidateURI curi) Get the String key (name) of the queue to which the CrawlURI should be assigned.
int	maximumNumberOfKeys() Returns the maximum number of different keys this policy can create.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BucketQueueAssignmentPolicy

public BucketQueueAssignmentPolicy()

Method Detail

getClassKey

public java.lang.String getClassKey(CrawlController controller,
                                    CandidateURI curi)

Description copied from class: QueueAssignmentPolicy

Get the String key (name) of the queue to which the CrawlURI should be assigned. Note that changes to the CrawlURI, or its associated components (such as CrawlServer), may change its queue assignment.

Specified by:

getClassKey in class QueueAssignmentPolicy

Parameters:

controller - This crawls' controller.

curi - CandidateURI to calculate class key for.

Returns:

the String key of the queue to assign the CrawlURI

maximumNumberOfKeys

public int maximumNumberOfKeys()

Description copied from class: QueueAssignmentPolicy

Returns the maximum number of different keys this policy can create. If there is no maximum, -1 is returned (default).

Overrides:

maximumNumberOfKeys in class QueueAssignmentPolicy

Returns:

Maximum number of different keys, or -1 if unbounded.