If duplicate pages are identical in all respects except for their URLs you should get a report of "document content matches previously processed document" next to the page in your crawl report. Only the one indexed document will be returned in your results. Check to see that your duplicate pages are really identical. If they are not, you may have to use robot exclusion tags.
Print
