PubCon Vegas 2006 - Duplicate Content
This one was pretty strightforward. The story from each search engine? Duplicate content, while not necessarily malicious, isn’t a good thing. Essentially, if you have two pages across the web or on a single site that are similar or even identical, only one copy of that content will make it onto the results page. The moral of the story? Make sure you’re writing original, valuable content. Also, one type of duplication issue that I had never really thought about is the problem of having multiple URLs (for example, dynamic URLs) indexed by the engines that are pointing to the same page. Keep an eye on that. Finally, use robots.txt to keep the bots out of your duplicates (like printer versions), to avoid getting slapped with a dupe content penalty.
Presenters I heard: Tim Converse from Yahoo! and Brian White from Google.
NOTE: I just want to clarify that you are not penalized for duplicate content by the search engines. Rather, if you are using duplicate content, odds are, your page will not rank in the index because it is not the original. Thanks to Brian White for the clarification.


Jazzcat,
Thanks for the writeup–I wanted to urge people not to think of our duplicate content resolution as a penalty. We’re not out to penalize people for general duplicate content (unless you flat out scrape it). We’re simply desiring to represent one document. For example, we’re not interested in “penalizing” the online version of an offline newspaper simply because they syndicate and serve newswire stories.
Comment by Brian White — November 17, 2006 @ 3:15 pm