org.archive.crawler.processor.recrawl
Class PersistOnlineProcessor

java.lang.Object
  javax.management.Attribute
      org.archive.crawler.settings.Type
          org.archive.crawler.settings.ComplexType
              org.archive.crawler.settings.ModuleType
                  org.archive.crawler.framework.Processor
                      org.archive.crawler.processor.recrawl.PersistProcessor
                          org.archive.crawler.processor.recrawl.PersistOnlineProcessor

All Implemented Interfaces:

java.io.Serializable, javax.management.DynamicMBean

Direct Known Subclasses:

PersistLoadProcessor, PersistStoreProcessor

public abstract class PersistOnlineProcessor
extends PersistProcessor

Common superclass for persisting Processors which directly store/load to persistence (as opposed to logging for batch load later).

Author:

gojomo

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.archive.crawler.settings.ComplexType

ComplexType.MBeanAttributeInfoIterator

Field Summary

protected com.sleepycat.je.Database	historyDb
protected com.sleepycat.collections.StoredSortedMap<java.lang.String,st.ata.util.AList>	store

Fields inherited from class org.archive.crawler.processor.recrawl.PersistProcessor

URI_HISTORY_DBNAME

Fields inherited from class org.archive.crawler.framework.Processor

ATTR_DECIDE_RULES, ATTR_ENABLED, attrDecideRules

Fields inherited from class org.archive.crawler.settings.ComplexType

definition, definitionMap

Constructor Summary

PersistOnlineProcessor(java.lang.String name, java.lang.String string)
Usual constructor

Method Summary

protected void	finalTasks() Classes subclassing this one should override this method to perform processor specific actions.
protected void	initialTasks() Classes subclassing this one should override this method to perform processor specific actions.
protected com.sleepycat.collections.StoredSortedMap<java.lang.String,st.ata.util.AList>	initStore()

Methods inherited from class org.archive.crawler.processor.recrawl.PersistProcessor

copyPersistSourceToHistoryMap, historyDatabaseConfig, main, persistKeyFor, populatePersistEnv, setupCopyEnvironment, setupCopyEnvironment, shouldLoad, shouldStore

Methods inherited from class org.archive.crawler.framework.Processor

checkForInterrupt, getController, getDecideRule, getDefaultNextProcessor, innerProcess, innerRejectProcess, isContentToProcess, isEnabled, isExpectedMimeType, isHttpTransactionContentToProcess, kickUpdate, process, report, rulesAccept, rulesAccept, setDefaultNextProcessor, spawn

Methods inherited from class org.archive.crawler.settings.ModuleType

addElement, listUsedFiles

Methods inherited from class org.archive.crawler.settings.ComplexType

addElementToDefinition, checkValue, earlyInitialize, getAbsoluteName, getAttribute, getAttribute, getAttribute, getAttributeInfo, getAttributeInfo, getAttributeInfoIterator, getAttributes, getDataContainerRecursive, getDataContainerRecursive, getDefaultValue, getDescription, getElementFromDefinition, getLegalValues, getLocalAttribute, getMBeanInfo, getMBeanInfo, getParent, getPreservedFields, getSettingsHandler, getUncheckedAttribute, getValue, globalSettings, invoke, isInitialized, isOverridden, iterator, removeElementFromDefinition, setAsOrder, setAttribute, setAttribute, setAttributes, setDescription, setPreservedFields, toString, unsetAttribute

Methods inherited from class org.archive.crawler.settings.Type

addConstraint, equals, getConstraints, getLegalValueType, isExpertSetting, isOverrideable, isTransient, setExpertSetting, setLegalValueType, setOverrideable, setTransient

Methods inherited from class javax.management.Attribute

getName, hashCode

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

store

protected com.sleepycat.collections.StoredSortedMap<java.lang.String,st.ata.util.AList> store

historyDb

protected com.sleepycat.je.Database historyDb

Constructor Detail

PersistOnlineProcessor

public PersistOnlineProcessor(java.lang.String name,
                              java.lang.String string)

Usual constructor

Parameters:

name -

string -

Method Detail

initialTasks

protected void initialTasks()

Description copied from class: Processor

Classes subclassing this one should override this method to perform processor specific actions.

This method is garanteed to be called after the crawl is set up, but before any URI-processing has occured.

Overrides:

initialTasks in class Processor

initStore

protected com.sleepycat.collections.StoredSortedMap<java.lang.String,st.ata.util.AList> initStore()

finalTasks

protected void finalTasks()

Description copied from class: Processor

Classes subclassing this one should override this method to perform processor specific actions.

Overrides:

finalTasks in class Processor