Spiderline
custom search engine solutions
Your Own Search Engine.
Just seconds after registering, your web site can be searchable with the features you want and reliability you need. No software to install or maintenance required. Search results can match your website design seamlessly.

Site Search Knowledge Base

Search  
   
Browse by Category
Site Search Knowledge Base .: Crawl Questions .: What parts of a document does Spiderline crawl?

What parts of a document does Spiderline crawl?

For HTML documents, the page title, Meta Keywords, Meta Description, and body text will be crawled. Image ALT tags and Robot comments are read but not indexed, you have the ability to use these with a checkbox in the Control pannels. All other HTML comments and remaining tags are removed before the body text is indexed.

How helpful was this article to you?

Related Articles

article Does NOINDEX keep documents from being counted?
Yes, if the NOINDEX and page or directory is in the URL patterns, this is because the document is avoided by the crawler. No, if the pages are being ommited by Robot Meta-tags, this is becuase...
(No rating)  2005-06-27    Views: 4559   
article Why didn't Spiderline crawl documents on other sites that I linked to?
Check your URL Configuration Patterns. In order to make documents on other websites searchable, but only the documents you link to and not the entire other website, enter "/  INDEX  NOFOLLOW" on...
(No rating)  2005-01-20    Views: 3696   
article How do I exclude parts of my site from being crawled?
To exclude areas from being indexed, you will need to put in commands to the URLs section of the Crawl settngs. These Patterns will tell the crawler what to index and what to avoid indexing. Type...
(No rating)  2005-04-27    Views: 4829   


.: Powered by Lore 1.5.3

Spiderline LLC © 2000-2007 | Knowledge Base | Privacy Policy | Terms Of Use | Resellers | Affiliate Program
Powered by Lucene