21. Release 0.6.0 - 2004-03-25

Abstract

Release made in advance of radical frontier changes. Added bandwidth throttle, operator 'diary', settable robots expiration, crawler cookie pre-population, and changing of certain options mid-crawl. Many UI improvements including UI display of critical exceptions, UI desccription of job-order options, and improved reporting. Optimizations. Updated httpclient lib to 2.0 release and jmx libs to 1.2.1. Lots of bug fixes.

21.1. Changes

Table 14. Changes

IDTypeSummary
861861Add861861 Redirects(/refreshes) from seeds should == new seeds
899223Add899223 Special seed-success report should be offered
891986Add891986 Bandwidth throttle function, setting.
877275Add877275 integrated operator 'diary' needed
891983Add891983 IP, Robots expirations should be settable
910152Add910152 Recovery of old jobs on WUI (re)start
781171Add781171 parsing css
912986Add912986 log views should give an idea of file size (where possible)
912989Add912989 Alerts should have 'select all' button...
856593Add856593 [load][save][turn on/off] cookies
912201Add912201 Add levels to alerts
896665Add896665 Split processor chains.
896754Add896754 Show total of disregards
903095Add903095 Show increments of megabytes in ui
896794Add896794 serious errors (eg outofmemory) should show up in UI
900520Add900520 Short description of ComplexTypes in user interface.
899982Add899982 Should be possible to alter filters while crawling.
896672Add896672 Display progress (doc/sec) with more precision
896677Add896677 Highlight the success or failures of each seed
896760Add896760 Prominent notification when seeds have problems
896801Add896801 java regexps (in log view) need help text
896778Add896778 Log viewing enhancements:
896795Add896795 frontier, thread report improvements
876516Add876516 default launch should nohup, save stdout/stderr
896763Fix127.0.0.1 in job report
896767FixFrontier retry-delay should include units (eg -seconds)
898994FixRevisiting admin URIs if not logged in should prompt login
899019FixDeadlock in Andy's 2nd Crawl
767225FixBetter bad-config handling
815357Fixmysterious pause facing network (DNS) problem
896747FixExtractorJS's report overstates it's discovered URIs
896667FixWeb UI does not display correctly in IE
896780Fixconsole clarity/safety
896655FixDoes not respect per settings added after crawl was started.
856555Fix'empty' records in compressed arc files