The Web Robots Pages

Database of Web Robots, View Type

Alternatively you can View Contact, or see the Overview.

Name Details
Acme.Spider
Purpose: indexing maintenance statistics
Availability: source
Platform: java
ActiveAgent
Purpose: other
Availability: source
Platform: all
Ahoy! The Homepage Finder
Purpose: maintenance
Availability: none
Platform: UNIX
Alkaline
Purpose: indexing
Availability: binary
Platform: unix windows95 windowsNT
Arachnophilia
Purpose:
Availability:
Platform:
ArchitextSpider
Purpose: indexing, statistics
Availability:
Platform:
Aretha
Purpose:
Availability:
Platform: Macintosh
ASpider (Associative Spider)
Purpose: indexing
Availability:
Platform: unix
Atomz.com Search Robot
Purpose: indexing
Availability: service
Platform: unix
AURESYS
Purpose: indexing,statistics
Availability: Protected by Password
Platform: Aix, Unix
BackRub
Purpose: indexing, statistics
Availability:
Platform:
Big Brother
Purpose: maintenance
Availability: binary
Platform: mac
Bjaaland
Purpose: indexing
Availability: none
Platform: unix
BlackWidow
Purpose: indexing, statistics
Availability:
Platform:
Die Blinde Kuh
Purpose: indexing
Availability: none
Platform: unix
bright.net caching robot
Purpose: caching
Availability: none
Platform:
BSpider
Purpose: indexing
Availability: none
Platform: Unix
CACTVS Chemistry Spider
Purpose: indexing.
Availability:
Platform:
Calif
Purpose: indexing
Availability: none
Platform: unix
Cassandra
Purpose: indexing
Availability: none
Platform: crossplatform
Digimarc Marcspider/CGI
Purpose: maintenance
Availability: none
Platform: windowsNT
Checkbot
Purpose: maintenance
Availability: source
Platform: unix,WindowsNT
churl
Purpose: maintenance
Availability:
Platform:
CMC/0.01
Purpose: maintenance
Availability: none
Platform: unix
Combine System
Purpose: indexing
Availability: source
Platform: unix
Conceptbot
Purpose: indexing
Availability: data
Platform: unix
Web Core / Roots
Purpose: indexing, maintenance
Availability:
Platform:
CS-HKUST WISE: WWW Index and Search Engine
Purpose:
Availability:
Platform:
Cusco
Purpose: indexing
Availability: none
Platform: any
CyberSpyder Link Test
Purpose: link validation, some html validation
Availability: binary
Platform: windows 3.1x, windows95, windowsNT
DeWeb(c) Katalog/Index
Purpose: indexing, mirroring, statistics
Availability:
Platform:
DienstSpider
Purpose: indexing
Availability: none
Platform: unix
DNAbot
Purpose: indexing
Availability: data
Platform: unix, windows, windows95, windowsNT, mac
DownLoad Express
Purpose: graphic download
Availability: binary
Platform: win95/98/NT
DragonBot
Purpose: indexing
Availability: none
Platform: windowsNT
DWCP (Dridus' Web Cataloging Project)
Purpose: indexing, statistics
Availability: source, binary, data
Platform: java
EIT Link Verifier Robot
Purpose: maintenance
Availability:
Platform:
Emacs-w3 Search Engine
Purpose: indexing
Availability:
Platform:
ananzi
Purpose: indexing
Availability:
Platform:
Esther
Purpose: indexing
Availability: data
Platform: unix (FreeBSD 2.2.8)
nzexplorer
Purpose: indexing, statistics
Availability: source (commercial)
Platform: UNIX
Felix IDE
Purpose: indexing, statistics
Availability: binary
Platform: windows95, windowsNT
Wild Ferret Web Hopper #1, #2, #3
Purpose: indexing maintenance statistics
Availability:
Platform:
FetchRover
Purpose: maintenance, statistics
Availability: binary, source
Platform: Windows/NT, Windows/95, Solaris SPARC
fido
Purpose: indexing
Availability: none
Platform: Unix
Hämähäkki
Purpose: indexing
Availability: no
Platform: UNIX
KIT-Fireball
Purpose: indexing
Availability: none
Platform: unix
Fish search
Purpose: indexing
Availability: binary
Platform:
Fouineur
Purpose: indexing, statistics
Availability: none
Platform: unix, windows
Robot Francoroute
Purpose: indexing, mirroring, statistics
Availability:
Platform:
Freecrawl
Purpose: indexing
Availability: none
Platform: unix
FunnelWeb
Purpose: indexing, statisitics
Availability:
Platform:
gazz
Purpose: statistics
Availability: none
Platform: unix
GCreep
Purpose: indexing
Availability: none
Platform: linux+mysql
GetBot
Purpose: maintenance
Availability:
Platform:
GetURL
Purpose: maintenance, mirroring
Availability:
Platform:
Golem
Purpose: maintenance
Availability: none
Platform: mac
Googlebot
Purpose: indexing statistics
Availability:
Platform:
Grapnel/0.01 Experiment
Purpose: Indexing
Availability: None, yet
Platform: WinNT
Gromit
Purpose: indexing
Availability: none
Platform: unix
Northern Light Gulliver
Purpose: indexing
Availability: none
Platform: unix
HamBot
Purpose: indexing
Availability: none
Platform: unix, Windows95
Harvest
Purpose: indexing
Availability:
Platform:
havIndex
Purpose: indexing
Availability: binary
Platform: Java VM 1.1
HI (HTML Index) Search
Purpose: indexing
Availability:
Platform:
Wired Digital
Purpose: indexing
Availability: none
Platform: unix
ht://Dig
Purpose: indexing
Availability: source
Platform: unix
HTMLgobble
Purpose: mirror
Availability:
Platform:
Hyper-Decontextualizer
Purpose: indexing
Availability:
Platform:
IBM_Planetwide
Purpose: indexing, maintenance, mirroring
Availability:
Platform:
Popular Iconoclast
Purpose: statistics
Availability: source
Platform: unix (OpenBSD)
Ingrid
Purpose: Indexing
Availability: Commercial as part of search engine package
Platform: UNIX
Imagelock
Purpose: maintenance
Availability: none
Platform: windows95
IncyWincy
Purpose:
Availability:
Platform:
Informant
Purpose: indexing
Availability: none
Platform: unix
InfoSeek Robot 1.0
Purpose: indexing
Availability:
Platform:
Infoseek Sidewinder
Purpose: indexing
Availability:
Platform:
InfoSpiders
Purpose: search
Availability: none
Platform: unix, mac
Inspector Web
Purpose: maintentance: link validation, html validation, image size validation, etc
Availability: free service and more extensive commercial service
Platform: unix
IntelliAgent
Purpose: indexing
Availability:
Platform:
Iron33
Purpose: indexing, statistics
Availability: source
Platform: unix
Israeli-search
Purpose: indexing.
Availability:
Platform:
Digital Integrity Robot
Purpose: indexing maintenance statistics
Availability: none
Platform: unix
JCrawler
Purpose: indexing
Availability: none
Platform: unix
Jeeves
Purpose: indexing maintenance statistics
Availability: none
Platform: UNIX
Jobot
Purpose: standalone
Availability:
Platform:
JoeBot
Purpose:
Availability:
Platform:
The Jubii Indexing Robot
Purpose: indexing, maintainance
Availability:
Platform:
JumpStation
Purpose: indexing
Availability:
Platform:
Katipo
Purpose: maintenance
Availability: binary
Platform: Macintosh
KDD-Explorer
Purpose: indexing
Availability: none
Platform: unix
Kilroy
Purpose: indexing,statistics
Availability: none
Platform: unix,windowsNT
KO_Yappo_Robot
Purpose: indexing
Availability: none
Platform: unix
LabelGrabber
Purpose: Grabs PICS labels from web pages, submits them to a label bueau
Availability: source
Platform: windows, windows95, windowsNT, unix
LinkWalker
Purpose: maintenance, statistics
Availability: none
Platform: windowsNT
Lockon
Purpose: indexing
Availability: none
Platform: UNIX
logo.gif Crawler
Purpose: indexing
Availability: none
Platform: unix
Lycos
Purpose: indexing
Availability:
Platform:
Mac WWWWorm
Purpose: indexing
Availability: none
Platform: Macintosh
Magpie
Purpose: indexing, statistics
Availability:
Platform: unix
MediaFox
Purpose: indexing and maintenance
Availability: none
Platform: (Java)
MerzScope
Purpose: WebMapping
Availability: binary
Platform: (Java Based) unix,windows95,windowsNT,os2,mac etc ..
NEC-MeshExplorer
Purpose: indexing
Availability: none
Platform: unix
MOMspider
Purpose: maintenance, statistics
Availability: source
Platform: UNIX
Monster
Purpose: maintenance, mirroring
Availability: binary
Platform: UNIX (Linux)
Motor
Purpose: indexing
Availability: data
Platform: mac
Muscat Ferret
Purpose: indexing
Availability: none
Platform: unix
Mwd.Search
Purpose: indexing
Availability: none
Platform: unix (Linux)
NetCarta WebMap Engine
Purpose: indexing, maintenance, mirroring, statistics
Availability:
Platform:
NetMechanic
Purpose: Link and HTML validation
Availability: via web page
Platform: UNIX
NetScoop
Purpose: indexing
Availability: none
Platform: UNIX
newscan-online
Purpose: indexing
Availability: binary
Platform: Linux
NHSE Web Forager
Purpose: indexing
Availability:
Platform:
Nomad
Purpose: indexing
Availability:
Platform:
The NorthStar Robot
Purpose: indexing
Availability:
Platform:
Occam
Purpose: indexing
Availability: none
Platform: unix
HKU WWW Octopus
Purpose: indexing
Availability:
Platform:
Orb Search
Purpose: indexing
Availability: data
Platform: unix
Pack Rat
Purpose: both maintenance and mirroring
Availability: at the moment, none...source when developed.
Platform: unix
Patric
Purpose: statistics
Availability: data
Platform: unix
The Peregrinator
Purpose:
Availability:
Platform:
PerlCrawler 1.0
Purpose: indexing
Availability: source
Platform: unix
Phantom
Purpose: indexing
Availability:
Platform: Macintosh
Pioneer
Purpose: indexing, statistics
Availability:
Platform:
html_analyzer
Purpose: maintainance
Availability:
Platform:
Portal Juice Spider
Purpose: indexing, statistics
Availability: none
Platform: unix
PGP Key Agent
Purpose: indexing
Availability: none
Platform: UNIX, Windows NT
PlumtreeWebAccessor
Purpose: indexing for the Plumtree Server
Availability: none
Platform: windowsNT
GetterroboPlus Puu
Purpose: Purpose of the robot. One or more of: - gathering: gather data of original standerd TAG for Puu contains the information of the sites registered my Search Engin. - maintenance: link validation
Availability: none
Platform: unix
The Python Robot
Purpose:
Availability: none
Platform:
RBSE Spider
Purpose: indexing, statistics
Availability:
Platform:
Resume Robot
Purpose: indexing.
Availability:
Platform:
RoadHouse Crawling System
Purpose:
Availability: none
Platform:
Road Runner: The ImageScape Robot
Purpose: indexing
Availability:
Platform: UNIX
Robbie the Robot
Purpose: indexing
Availability: none
Platform: unix, windows95, windowsNT
ComputingSite Robi/1.0
Purpose: indexing,maintenance
Availability:
Platform: UNIX
Roverbot
Purpose: indexing
Availability:
Platform:
SafetyNet Robot
Purpose: indexing.
Availability:
Platform:
Scooter
Purpose: indexing
Availability: none
Platform: unix
Search.Aus-AU.COM
Purpose: - indexing: gather content for an indexing service
Availability: - none
Platform: - mac - unix - windows95 - windowsNT
Senrigan
Purpose: indexing
Availability: none
Platform: Java
SG-Scout
Purpose: indexing
Availability:
Platform:
Shai'Hulud
Purpose: mirroring
Availability: source
Platform: unix
Simmany Robot Ver1.0
Purpose: indexing, maintenance, statistics
Availability: none
Platform: unix
Open Text Index Robot
Purpose: indexing
Availability: inquire to markk@opentext.com (Mark Kraatz)
Platform: UNIX
SiteTech-Rover
Purpose: indexing
Availability:
Platform:
Inktomi Slurp
Purpose: indexing, statistics
Availability: none
Platform: unix
Smart Spider
Purpose: indexing
Availability: data, binary, source
Platform: windows95, windowsNT
Snooper
Purpose:
Availability: none
Platform:
Solbot
Purpose: indexing
Availability: none
Platform: unix
Spanner
Purpose: indexing,maintenance
Availability: source
Platform: unix
SpiderBot
Purpose: indexing, mirroring
Availability: source, binary, data
Platform: unix, windows, windows95, windowsNT
Spry Wizard Robot
Purpose: indexing
Availability:
Platform:
Site Searcher
Purpose: indexing
Availability: binary
Platform: winows95, windows98, windowsNT
Suke
Purpose: indexing
Availability: source
Platform: FreeBSD3.*
Sven
Purpose: indexing
Availability: none
Platform: Windows
TACH Black Widow
Purpose: maintenance: link validation
Availability: none
Platform: UNIX, Linux
Tarantula
Purpose: indexing
Availability: none
Platform: unix
tarspider
Purpose: mirroring
Availability:
Platform:
Tcl W3 Robot
Purpose: maintenance, statistics
Availability:
Platform:
TechBOT
Purpose: statistics, maintenance
Availability: none
Platform: Unix
Templeton
Purpose: mirroring, mapping, automating web applications
Availability: binary
Platform: OS/2, Linux, SunOS, Solaris
TitIn
Purpose: indexing, statistics
Availability: data, source on request
Platform: unix
TITAN
Purpose: indexing
Availability: no
Platform: SunOS 4.1.4
The TkWWW Robot
Purpose: indexing
Availability:
Platform:
UCSD Crawl
Purpose: indexing, statistics
Availability:
Platform:
URL Check
Purpose: maintenance
Availability: binary
Platform: unix
URL Spider Pro
Purpose: indexing
Availability: binary
Platform: Windows9x/NT
Valkyrie
Purpose: indexing
Availability: none
Platform: unix
Victoria
Purpose: maintenance
Availability: none
Platform: unix
vision-search
Purpose: indexing.
Availability:
Platform:
Voyager
Purpose: indexing, maintenance
Availability: none
Platform: unix
VWbot
Purpose: indexing
Availability: source
Platform: unix
The NWI Robot
Purpose: discovery,statistics
Availability: none (at the moment)
Platform: UNIX
W3M2
Purpose: indexing, maintenance, statistics
Availability:
Platform:
the World Wide Web Wanderer
Purpose: statistics
Availability: data
Platform: unix
WebBandit Web Spider
Purpose: Resource Gathering / Server Benchmarking
Availability: source, binary
Platform: Intel - windows95
WebCatcher
Purpose: indexing
Availability: none
Platform: unix, windows, mac
WebCopy
Purpose: mirroring
Availability:
Platform:
webfetcher
Purpose: mirroring
Availability:
Platform:
The Webfoot Robot
Purpose:
Availability:
Platform:
weblayers
Purpose: maintainance
Availability:
Platform:
WebLinker
Purpose: maintenance
Availability:
Platform:
WebMirror
Purpose: mirroring
Availability:
Platform: Windows95
The Web Moose
Purpose: statistics, maintenance
Availability: data
Platform: Windows NT
WebQuest
Purpose: indexing
Availability: none
Platform: unix
Digimarc MarcSpider
Purpose: maintenance
Availability: none
Platform: windowsNT
WebReaper
Purpose: indexing/offline browsing
Availability: binary
Platform: windows95, windowsNT
webs
Purpose: statistics
Availability: none
Platform: unix
Websnarf
Purpose:
Availability:
Platform:
WebSpider
Purpose: maintenance, link diagnostics
Availability:
Platform:
WebVac
Purpose: mirroring
Availability:
Platform:
webwalk
Purpose: indexing, maintentance, mirroring, statistics
Availability:
Platform:
WebWalker
Purpose: maintenance
Availability: source
Platform: unix
WebWatch
Purpose: maintainance, statistics
Availability:
Platform:
Wget
Purpose: mirroring, maintenance
Availability: source
Platform: unix
WhoWhere Robot
Purpose: indexing
Availability: none
Platform: Sun Unix
w3mir
Purpose: mirroring.
Availability:
Platform: UNIX, WindowsNT
WebStolperer
Purpose: indexing
Availability: none
Platform: unix, NT
The Web Wombat
Purpose: indexing, statistics.
Availability:
Platform:
The World Wide Web Worm
Purpose: indexing
Availability:
Platform:
WWWC Ver 0.2.5
Purpose: maintenance
Availability: binary
Platform: windows, windows95, windowsNT
WebZinger
Purpose: indexing
Availability: binary
Platform: windows95, windowsNT 4, mac, solaris, unix
XGET
Purpose: mirroring
Availability: binary
Platform: X68000, X68030
Nederland.zoek
Purpose: indexing
Availability: none
Platform: unix (Linux)

The Web Robots Database