Bill Slawski writes about search engine patents, particularly Google's, and how they may affect ranking, links and indexing.
HN commenters cite it for explanations of Google patents like the Reasonable Surfer, and for Facebook's patent filings and how crawlers analyze pages.
This is the failure of google. I stopped using google about 6 months ago and started using duckduckgo. But at the time I stopped using google, one of the reasons I stopped was that the quality was so low.
There is a difference between quality and freshness. I agree some SERPS on DDG look higher quality, but then when you dig down in the results you find out why: They are all safe choices. They could be pages from 2005. They could be pages that were once authoritative, but now lack topicality and news.
Hell, the quality of Google is so low that Bing is actually running ads right now with blind taste tests where people preferred bing
This is more marketing than research.
Page rank was really cutting edge, but that was 10 years ago, yet it is still their primary mechanism
It is one of 200 factors. Also there is internal pagerank and world-visible pagerank. Besides Google has been doing a lot with author rank and mentions.
It's been gamed, but they seem uninterested in moving to more sophisticated mechanisms (they use them but the influence of better methods seems to be too low)
Latent semantic indexing, query deserves diversity, query deserves freshness, detecting spam by following links in spam emails etc. There is no shortage of sophisticated methods.
meanwhile they've used their bully pulpit to influence the web to conserve page juice which has backfired in such a way that actual links to authoritative and useful sites are lower ranked than spam links, making it…
I hope this doesn't seem self serving, but seomoz.org has some of the highest quality information you can find on SEO. In addition to that, I'd suggest articles on http://searchengineland.com/ - they offer high quality content and news in the industry. Anything written by Vanessa Fox on SEL is usually great.
Guides like Excel for SEO shows some of the basic stuff we do:
http://www.distilled.net/excel-for-seo/
For high level patent / research based SEO, nobody does it better than http://www.seobythesea.com/ (his URLs are ugly, but his content is great)
For specific beliefs on how the algo works, this is a good resource:
http://www.seomoz.org/article/search-ranking-factors
SEOmoz has webinars and a PRO Q&A. Distilled.net also offers videos of our conferences.
As for books, the best I've read are Art of SEO, which is co-written by Rand, and SEO Secrets by Danny Dover (formerly of SEOmoz, now at AT&T) which talks more about the consulting side. The problem with books is that stuff can go out of date quickly. If you're reading a book that talks about PageRank sculpting, for example, it's dated.
Other great sites:
http://www.seobook.com/ - high level opinions and analysis of industry issues
http://www.seroundtable.com/ - recap of industry news and forums
http://www.searchenginejournal.com/ - lots of tactical articles
Things I've written for…
This article startled me into checking its publication date. The claim is so clearly obvious for the outlined applications (camera purchases, vacation bookings, movie grosses), it seems unreasonable to think nobody was doing this.
Here's an essay on "trendspotting", suggesting the concept came into vogue in the late '90s:
http://www.radosh.net/writing/trends.html
And here's an article citing analysis of AltaVista query logs in 1998 (and other query studies since):
http://www.seobythesea.com/?p=224
Perhaps because that was so hot at the time, folks I worked with in e-commerce in the mid to late 90's assumed this is what search engines were doing with their data -- data-mining it and selling the intelligence. Our reasoning was: "Well, if the grocery stores can predict what to stock on the shelves and what coupons to mail, surely the search engines can predict macro/economic/consumer trends, and that data must be incredibly valuable. That must be where they're getting their real revenue..."
But maybe they weren't. A month ago, PG suggested Yahoo was making too much money from ads to bother improving commerce related results with "Revenue Loop".
http://www.paulgraham.com/yahoo.html
Perhaps that held true of all the search engines? At least the idea was out there before this Yahoo research…
Googlebot and some other search engine crawlers also crawl css stylesheet files, I have seen evidence in my server log files.
There are several patents describing how a search engine might analyse the visual appearance and structure of a website, with a special view to give some parts of the page content (the main content) a higher weight or priority then the rest (like footer text and navigation). I would expect that the algorithms used by Google will give a good impression about the page rendering. What conclusions are drawn from this and how it affects the ranking of individual pages for certain search queries, who knows, ... pretty much about this is kept secret and there is a lot of speculation.
Many search engine patents are explored and explained by Bill Slawski on his blog "SEO by the sea":
http://www.seobythesea.com/
That there are these patents does of course not tell us, that search engines are working exactly in this or another way, but I think it often gives a good impression about what might be going on behind the scenes.
https://searchengineland.com/google-ebay-penalty-cost-197031
This relates to the ebay penalty. I think one of SEland's founders actually left to take a job at Google as an evangelist. You can probably search for "ebay seo penalty" and find more references. Other notable penalties that were well documented include RapGenius and GV backed Thumbtack and Nest, which quickly had their penalties reversed if I recall correctly.
https://seobook.com/blog is the only place I follow any more, since it is more critical of the digital marketing ecosystem. There are so many snake oil salespeople and clueless marketers, so this is a refreshing commentary.
https://seroundtable.com is good for rumors and breaking news.
https://www.seobythesea.com/ a more technical blog that frequently examines Google's patents.
This definitely is important in terms of how much "value" Google assigns to links on the page, if they can identify where they render. They got a patent for varying the amount of page rank that flows to a link based on how likely the "reasonable surfer" is to click it. So in terms of flowing Pagerank to a bunch of high value landing pages via a slew of footer links, it's game over.
My only question is how frequently Googlebot actually renders the full JavaScript. Surely they don't have the computing resources to render it on every crawl.
Link to patent discussion: http://www.seobythesea.com/2010/05/googles-reasonable-surfer-how-the-value-of-a-link-may-differ-based-upon-link-and-document-features-and-user-data/