|
||||||||||
| PREV NEXT | FRAMES NO FRAMES | |||||||||
| Packages that use Robotstxt | |
|---|---|
| org.archive.crawler.datamodel | |
| Uses of Robotstxt in org.archive.crawler.datamodel |
|---|
| Constructors in org.archive.crawler.datamodel with parameters of type Robotstxt | |
|---|---|
RobotsExclusionPolicy(CrawlerSettings settings,
Robotstxt robotstxt,
RobotsHonoringPolicy honoringPolicy)
|
|
|
||||||||||
| PREV NEXT | FRAMES NO FRAMES | |||||||||