|
||||||||||
| PREV PACKAGE NEXT PACKAGE | FRAMES NO FRAMES | |||||||||
| Class Summary | |
|---|---|
| PreconditionEnforcer | Ensures the preconditions for a fetch -- such as DNS lookup or acquiring and respecting a robots.txt policy -- are satisfied before a URI is passed to subsequent stages. |
| Preselector | If set to recheck the crawl's scope, gives a yes/no on whether a CrawlURI should be processed at all. |
| QuotaEnforcer | A simple quota enforcer. |
| RuntimeLimitEnforcer | A processor to enforce runtime limits on crawls. |
|
||||||||||
| PREV PACKAGE NEXT PACKAGE | FRAMES NO FRAMES | |||||||||