Scraping our site - for ourselves

After spending some time digging around online, I've noticed a few things.

1) Scraper pages rank well.

2) Our site ranks well.

3) Links to searches on our site tend to make that page rank well.

By taking a look at our search logs, it appears that people search the same way on our site that they search on Google. That's good news to me, because we use Google hardware for our search. But what's even better news is that we can capture what product they go to when they search for an item and create a link back to the search results the item was found in for each and every product on our site.

So let me back up one step here. Since our product pages are dynamically generated, we can take a look at referrer data (HTTP_REFERER environment variable) and get the search terms out of the URL (if any). We can log these to a table with the brand name and the SKU, then take a look when we display the product page for any queries in that table that we'd like to show.

So now, on a product like the Irwin Hammer Holder, we have a link back to the search for hammer holders. But if we didn't have images, add to cart buttons, and prices - go ahead and cover up those portions of the screen with your hands - you'd call that page a scraper. Well, it is. Google's search appliance scraped (crawled) our site and returned those snippets from the index it created. That IS keyword-rich scraped content. But it's ok since it's from our own site, at least in my opinion.

Oh, one other side-effect of this is that items people choose are getting a slight boost in the site search SERPs because of the repetition of the keyword phrase on the page. It'll be very interesting to see the engine's opinion of these relevant links.