Spiderline
custom search engine solutions
Your Own Search Engine.
Just seconds after registering, your web site can be searchable with the features you want and reliability you need. No software to install or maintenance required. Search results can match your website design seamlessly.

Site Search Knowledge Base

Search  
   
Browse by Category
Site Search Knowledge Base .: Crawl Questions .: Does NOINDEX keep documents from being counted?

Does NOINDEX keep documents from being counted?

Yes, if the NOINDEX and page or directory is in the URL patterns, this is because the document is avoided by the crawler.

No, if the pages are being ommited by Robot Meta-tags, this is becuase the document has already been downloaded before the crawler is informed that the document is not to be indexed.

How helpful was this article to you?

Related Articles

article Learn How to Use NOINDEX & NOFOLLOW
"Crawling" is the process of finding content on your web site. Finding web pages is similar to a web user browsing through a site and clicking on links. Spiderline spiders also browse your...
  2005-01-20    Views: 5776   
article Useing NOINDEX and NOFOLLOW patterns
"Crawling" is the process of finding content on your web site. Finding web pages is similar to a web user browsing through a site and clicking on links. Spiderline spiders also browse your...
(No rating)  2005-05-03    Views: 3845   
article What type of documents will Spiderline index?
Presently, Spiderline supports Microsoft Word, Adobe PDF, HTML, RTF and text. We can add on additional document types on request.
(No rating)  2005-01-19    Views: 4238   


.: Powered by Lore 1.5.3

Spiderline LLC © 2000-2007 | Knowledge Base | Privacy Policy | Terms Of Use | Resellers | Affiliate Program
Powered by Lucene