Softpedia
 
HomepageWindowsGamesDriversMacLinuxScripts buttonMobileHandheldNews


SCRIPTS CATEGORIES:



NEWS ARCHIVE >>
SOFTPEDIA REVIEWS >>
MEET THE EDITORS >>
WEEK'S BEST
  • PeoplePods 0.9
  • Brackets Sprint 8
  • elFinder 2.0 RC1
  • BBClone 0.6.1
  • Twitter Follow Box...
  • Multilingual Press...
  • SimplePie 1.2.1
  • TinyTips 1.2
  • SWFUpload 2.2.0.1 ...
  • Head Cleaner 1.4.2.9
    		•
    Home > Scripts > Modules > Other Modules

    robots.js 0.7.2
    Download button


    Downloads: 80  Tell us about an update
    User Rating:
    Rated by:    		 NOT RATED
    0 user(s)
    Developer:

    Website:

    License / Price:

    Platforms:

    Databases:

    Language:

    Last Updated:

    Category:         		Eugene Kalinin | More scripts
    github.com
    MIT License 

    Windows / Linux / Mac OS / BSD / Solaris
    N/A
    JavaScript
    January 19th, 2012, 05:45 GMT [view history]
    C: \ Modules \ Other Modules

     Read user reviews (0)  Refer to a friend  Subscribe

    robots.js description

    This is a Node.js library for parsing robots.txt files

    Can be used to pull out data from the robots.txt file and feed it to other applications or statistics tools.

    Will extract target URL, reason for access and status data.

    Requirements:

    · Node.js

    What's New in This Release: [ read full changelog ]

    · Fixed package.json meta info for bugs.



    TAGS:

    		 robots.txt parser | user agent | file parser | parse | robots.txt | file



    HTML code for linking to this page:


    Go to top

    WindowsGamesDriversMacLinuxScriptsMobileHandheldNews

    SUBMIT PROGRAM   |   ADVERTISE   |   GET HELP   |   SEND US FEEDBACK   |   RSS FEEDS   |   UPDATE YOUR SOFTWARE   |   ROMANIAN FORUM
    © 2001 - 2012 Softpedia. All rights reserved.
    Softpedia® and the Softpedia® logo are
    registered trademarks of SoftNews NET SRL.    		 Copyright Information | Privacy Policy | Terms of Use