org.archive.crawler.processor.recrawl
Class PersistLoadProcessor

java.lang.Object
  extended by javax.management.Attribute
      extended by org.archive.crawler.settings.Type
          extended by org.archive.crawler.settings.ComplexType
              extended by org.archive.crawler.settings.ModuleType
                  extended by org.archive.crawler.framework.Processor
                      extended by org.archive.crawler.processor.recrawl.PersistProcessor
                          extended by org.archive.crawler.processor.recrawl.PersistOnlineProcessor
                              extended by org.archive.crawler.processor.recrawl.PersistLoadProcessor
All Implemented Interfaces:
java.io.Serializable, javax.management.DynamicMBean

public class PersistLoadProcessor
extends PersistOnlineProcessor

Store CrawlURI attributes from latest fetch to persistent storage for consultation by a later recrawl.

Version:
$Date: 2006-09-25 20:19:54 +0000 (Mon, 25 Sep 2006) $, $Revision: 4654 $
Author:
gojomo
See Also:
Serialized Form

Nested Class Summary
 
Nested classes/interfaces inherited from class org.archive.crawler.settings.ComplexType
ComplexType.MBeanAttributeInfoIterator
 
Field Summary
static java.lang.String ATTR_PRELOAD_SOURCE
          file (log) or directory (state/env) from which to preload history
 
Fields inherited from class org.archive.crawler.processor.recrawl.PersistOnlineProcessor
historyDb, store
 
Fields inherited from class org.archive.crawler.processor.recrawl.PersistProcessor
URI_HISTORY_DBNAME
 
Fields inherited from class org.archive.crawler.framework.Processor
ATTR_DECIDE_RULES, ATTR_ENABLED, attrDecideRules
 
Fields inherited from class org.archive.crawler.settings.ComplexType
definition, definitionMap
 
Constructor Summary
PersistLoadProcessor(java.lang.String name)
          Usual constructor
 
Method Summary
protected  com.sleepycat.collections.StoredSortedMap<java.lang.String,st.ata.util.AList> initStore()
           
protected  void innerProcess(CrawlURI curi)
          Classes subclassing this one should override this method to perform their custom actions on the CrawlURI.
 
Methods inherited from class org.archive.crawler.processor.recrawl.PersistOnlineProcessor
finalTasks, initialTasks
 
Methods inherited from class org.archive.crawler.processor.recrawl.PersistProcessor
copyPersistSourceToHistoryMap, historyDatabaseConfig, main, persistKeyFor, populatePersistEnv, setupCopyEnvironment, setupCopyEnvironment, shouldLoad, shouldStore
 
Methods inherited from class org.archive.crawler.framework.Processor
checkForInterrupt, getController, getDecideRule, getDefaultNextProcessor, innerRejectProcess, isContentToProcess, isEnabled, isExpectedMimeType, isHttpTransactionContentToProcess, kickUpdate, process, report, rulesAccept, rulesAccept, setDefaultNextProcessor, spawn
 
Methods inherited from class org.archive.crawler.settings.ModuleType
addElement, listUsedFiles
 
Methods inherited from class org.archive.crawler.settings.ComplexType
addElementToDefinition, checkValue, earlyInitialize, getAbsoluteName, getAttribute, getAttribute, getAttribute, getAttributeInfo, getAttributeInfo, getAttributeInfoIterator, getAttributes, getDataContainerRecursive, getDataContainerRecursive, getDefaultValue, getDescription, getElementFromDefinition, getLegalValues, getLocalAttribute, getMBeanInfo, getMBeanInfo, getParent, getPreservedFields, getSettingsHandler, getUncheckedAttribute, getValue, globalSettings, invoke, isInitialized, isOverridden, iterator, removeElementFromDefinition, setAsOrder, setAttribute, setAttribute, setAttributes, setDescription, setPreservedFields, toString, unsetAttribute
 
Methods inherited from class org.archive.crawler.settings.Type
addConstraint, equals, getConstraints, getLegalValueType, isExpertSetting, isOverrideable, isTransient, setExpertSetting, setLegalValueType, setOverrideable, setTransient
 
Methods inherited from class javax.management.Attribute
getName, hashCode
 
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
 

Field Detail

ATTR_PRELOAD_SOURCE

public static final java.lang.String ATTR_PRELOAD_SOURCE
file (log) or directory (state/env) from which to preload history

See Also:
Constant Field Values
Constructor Detail

PersistLoadProcessor

public PersistLoadProcessor(java.lang.String name)
Usual constructor

Parameters:
name -
Method Detail

initStore

protected com.sleepycat.collections.StoredSortedMap<java.lang.String,st.ata.util.AList> initStore()
Overrides:
initStore in class PersistOnlineProcessor

innerProcess

protected void innerProcess(CrawlURI curi)
                     throws java.lang.InterruptedException
Description copied from class: Processor
Classes subclassing this one should override this method to perform their custom actions on the CrawlURI.

Overrides:
innerProcess in class Processor
Parameters:
curi - The CrawlURI being processed.
Throws:
java.lang.InterruptedException


Copyright © 2003-2011 Internet Archive. All Rights Reserved.