Softpedia
 
HomepageWindowsGamesDriversMacLinuxScripts buttonMobileHandheldNews


SCRIPTS CATEGORIES:



NEWS ARCHIVE >>
SOFTPEDIA REVIEWS >>
MEET THE EDITORS >>
WEEK'S BEST
  • PeoplePods 0.9
  • Brackets Sprint 8
  • elFinder 2.0 RC1
  • BBClone 0.6.1
  • Twitter Follow Box...
  • Multilingual Press...
  • SimplePie 1.2.1
  • TinyTips 1.2
  • SWFUpload 2.2.0.1 ...
  • Head Cleaner 1.4.2.9
    		•
    Home > Scripts > Search Engines

    WWW-Crawler-Lite 0.003
    Download button


    Downloads: 325  Tell us about an update
    User Rating:
    Rated by:    		 NOT RATED
    0 user(s)
    Developer:

    Website:

    License / Price:

    Platforms:

    Databases:

    Language:

    Last Updated:

    Category:         		John Drago | More scripts
    search.cpan.org
    Perl License 

    Windows / Linux / Mac OS / BSD / Solaris
    N/A
    Perl
    June 12th, 2011, 05:03 GMT [view history]
    C: \ Search Engines

     Read user reviews (0)  Refer to a friend  Subscribe

    WWW-Crawler-Lite description

    This is a single-threaded crawler/spider for the web built in Perl

    It can be used within a mod_perl, CGI or Catalyst-style environment because it does not fork or use threads.

    The callback-based interface is fast and simple, allowing you to focus on simply processing the data that WWW::Crawler::Lite extracts from the target website.

    It will handle URL queues, and can even stop processing it on demand.

    What's New in This Release: [ read full changelog ]

    · Using WWW::RobotRules is great, as long as you remove extra newlines before parsing.



    TAGS:

    		 web crawler | search engine | find data | search | crawl | page



    HTML code for linking to this page:


    Go to top

    WindowsGamesDriversMacLinuxScriptsMobileHandheldNews

    SUBMIT PROGRAM   |   ADVERTISE   |   GET HELP   |   SEND US FEEDBACK   |   RSS FEEDS   |   UPDATE YOUR SOFTWARE   |   ROMANIAN FORUM
    © 2001 - 2012 Softpedia. All rights reserved.
    Softpedia® and the Softpedia® logo are
    registered trademarks of SoftNews NET SRL.    		 Copyright Information | Privacy Policy | Terms of Use