Crawlers Downloads

ETagFix is an ISAPI plugin for Internet Information Server (IIS) that changes the etag produced by IIS so that your etag does not change every time your computer reboots. This can result in web crawlers and user clients wanting to recache your web page when they don't need to.

By placing a Sitemap-formatted file on your webserver, you enable the Google crawlers to find out what pages are present and which have recently changed, and to crawl your site accordingly. Using Sitemaps to inform and direct the Google crawlers, they hope to expand their coverage of the web and improve the time of inclusion in their index.

KeyWord generator is free powerfull tool to generate keywords and save them. Using KeyWord generator lite you can generate more than 100 keywords and use them in your meta tags, titles or content of your web page. Keywords are really powerfull way to make search engines remember about you. Meta keywords can be extremely effective if they feat content properly. But beware of spam keywording. Awoid repeating of the same keyword too often.

Why do you need Fake Referrals? Have you ever seen sites with 'last 10 referrers' area? Just go to Yahoo! and input this query string 'link:http://cialis-online-buy.surf.to'. Then click search. Almost each site from results contains a small area with referrals. Those areas are indexed by search engines too. You found results using search engine? Crawlers find same results too. You do not need any expensive advertising to get your site indexed.

Manage and track visits by robots, spiders and crawlers to your website. Identify which spiders visited, when they visited, and what pages they indexed. Direct spiders to your most important pages, and exclude them from non-public pages. Maximizes effectiveness with search engines by avoiding over/under submitting. Excellent to improve search engine placements for users of automatic submission software such as WebPosition, TopDog, AddWeb, etc.

StatsNow is the ultimate live statistics program giving you a real time view of your visitors currently on your Web sites, what pages they have visited, who referred them and a whole lot more! StatsNow also offers full log analysis giving you a full range of historical data about your visitors. StatsNow provides log file analysis, real time tracking, error reporting, and search engine information. Download it today and try it for free!

Create your robots.txt files in seconds and maintain them easilly and effortlesly with the unique utility Advanced Robots.txt Generator. The software supports 300 robots, works in a WYSWIWYG visual way and comes with full documentation and unlimited customer support. It's features include automatic creation of Robots.txt files, automatic uploading, comparison with previously saved project, online/offline documentation, manual, free upgrades etc.

Robots.txt is a visual editor for Robot Exclusion Files and a log analyzer software. It allows a user to quickly and easily create the robots.txt files required to instruct search engine spiders, which parts of a Web site are not to be indexed and made searchable by the general Web public and then to identify spiders, which do not keep to those instructions

Run your own file upload site like Rapidshare or MegaUpload with XFileSharing script. XFileSharing script is an advanced solution for creating free file hosting websites. It does not require mysql database and can be installed on any virtual/shared or dedicated hosting running Perl (most servers do). You can customize look and feel of your site with just editing HTML files and can place banners, adsense or other type advertisement on its pages.

Email address code scrambler. Fight spam. Your spam problem comes from spam crawler robots. They crawl your site to add your email addresses their spam lists. Make your email addresses invisible to spam crawlers. Hide your mail addresses from robots in plain sight. Visitors to your web pages can read and click on your email addresses the same way they do now. Robots will not find them. Stay off spam lists. Search engine friendly. Easy to use.

The InnerPeace WebMaster Toolkit is a collection of essensial tools assembeled and made by the Webmasters of InnerPeace. Some of these essensial tools are not available anywhere else. Includes a fast, easy 4 hexillion color combo picker, PageJumper web page forwarder, EScrambler to scramble your email addresses and hide them from spamcrawlers, plus some excellent articles. Easy to use. Runs on any O/S: Windows, Mac, Unix, etc. License: GNU GPL.

Build a custom web spider / web crawler using web data extraction / screen scraping technology. Use the web extract for web data mining of contact lists, product catalogs, government databases, real estate listings, or build a custom email extractor. Creating your own web grabber that can Screen Scrape the data to a database or Excel has never been easier. The website spider / crawlers can even sort and filter the webpages while spidering.

Sothink Tree Menu lets your creation of JavaScript tree menu simple! It supports Opera 9 and Firefox 2. And support Windows Vista. You can create DHTML tree menu within Dreamweaver, Expression Web, FrontPage and Golive directly! The search engine finds menu's contents by sitemaps easily. Site Navigation Structure Generator can generate tree menu based on your website. And also, you can control expanding or collapsing the sub tree in web page!

Sothink Tree Menu lets your creation of JavaScript tree menu simple! It supports Opera 9 and Firefox 2. And support Windows Vista. You can create DHTML tree menu within Dreamweaver, Expression Web, FrontPage and Golive directly! The search engine finds menu's contents by sitemaps easily. Site Navigation Structure Generator can generate tree menu based on your website. And also, you can control expanding or collapsing the sub tree in web page!

Let your creation of JavaScript menus simple! The generated JavaScript menus can be compatible with most browsers. Template is provided to start menu without designing; image library is offered to use the image resource from the program. The search engine will find the menu's contents and its link easily by Search Engine Friendly Code Maker. It integrates with Dreamweaver and FrontPage seamlessly; you can edit the menu in web editor directly!