iPullRank

Last month, there was a significant buzz among those involved with Google Search (which includes almost the entire modern internet and all its developers). "Erfan Azimi," the owner of an SEO firm, suddenly began sharing leaked documents revealing how Google's ranking system works. It's more complicated than just the search itself; it involves various APIs around it. Nonetheless, these APIs reveal a lot. The leak happened when a Google developer wrote a program to convert API calls into his preferred programming language but accidentally published everything (if you're interested, I've included a link to the commit with all these descriptions [1]).

Multiple confirmations from reputable sources, including former and current Googlers, have verified the authenticity of this leak. It's not a hoax or a joke but a genuine breach of information that has piqued the interest of all SEO researchers. Here's a reliable summary of the findings:

– Google has allowlists of manually optimized sites, at least for certain topics, such as the 2020 elections or COVID-19.

– Domain names and subdomains are significant factors (despite Google's previous claims).

– There's a sandbox for new sites, which Google has always denied.

– Google directly uses data from EWOK (a system where paid users rate the quality of search results).

– User behavior on sites is actively used for ranking.

– Click data is collected not only from Google Analytics but also directly from the Chrome browser.

– Sites…

>We are expecting to get the same chance of ranking as normal pages.

Then you should probably adjust this expectation. You say in your article:

>While having this sort of HTML fallback was technically possible, it added a lot of extra work to public-facing single page apps, to the point where many developers dropped the idea...

A JS-driven site with an HTML fallback is a normal page. Then you don't need any tricks or force Google to run your application and hopefully make pages out of them. Start with the fall-back and enhance.

This is a serious mistake with consequences. Tor bundle and Firefox shipped with JavaScript support, because disabling JS broke too much of the current web. It causes accessibility issues (remember when Twitter changed to hash-bang URL's?), if not for Googlebot, then for regular users (From the Webmaster Guidelines):

>Following these guidelines will help Google find, index, and rank your site.

>Use a text browser such as Lynx to examine your site, because most search engine spiders see your site much as Lynx would. If fancy features such as JavaScript, cookies, session IDs, frames, DHTML, or Flash keep you from seeing all of your site in a text browser, then search engine spiders may have trouble crawling your site.

>Make pages primarily for users, not for search engines.

I am still going on the assumption that you created a one-page application without a time-consuming fallback, and you rely on Google to make rankable pages from them. Then…

My personal opinion is that open sourcing googlebot[0] and tensorflow, as well as a the numerous other tools like elastic search and frameworks like hadoop make it pretty easy to make narrow search tools.

So someone like me, a pretty shitty programmer, can use an language built on top of googlebot, like nodejs, and just index the stuff I care about and instead of that information being returned to google, I can just send it back to bots viewport directly and not even see the webpage, just the info if I want.

The problem is discovery (what google solved in 1999), no one knew where to find websites, like not even info, just what sites even were on the web. This is a huge pain point again because while there are a lot of great websites there are a ton of shitty ones. This is pretty obvious if you look at sites like reddit or hacker news. They are developed around the idea of aggregating good resources because there are so many it is overwhelming.

So, I think, that of the 50% of the people that are already running googlebot, some subset will start probably using it to get info they care about. Eventually, someone will write code you can use to make this easier for normal people, and then google whill either sell the silos of info to them instead of advertising, or use it in it's new meta structure to solve big problems.

[0]http://ipullrank.com/googlebot-is-chrome/
[1]http://www.sitepoint.com/browser-trends-august-2015-chrome-exceeds-50/

Long time dev/designer and part-time SEO person here.

The major implications are that a lot of what Google was telling SEO practitioners were a lie or cleverly disguised red herrings wrapped up in semantics.

The article that was posted earlier today went through a lot of the stuff they were lying about with examples of Matt Cutts and Gary Ilyes statements that are now directly contradicted with the recent release of this code.

Here is the article: https://ipullrank.com/google-algo-leak

The premise:

What I’ll do here is contextualize some of the most interesting ranking systems and features (at least, those I was able to find in the first few hours of reviewing this massive leak) based on my extensive research and things that Google has told/lied to us about over the years.

“Lied” is harsh, but it’s the only accurate word to use here. While I don’t necessarily fault Google’s public representatives for protecting their proprietary information, I do take issue with their efforts to actively discredit people in the marketing, tech, and journalism worlds who have presented reproducible discoveries.

The "lies" there are such bullshit. They basically consist of "Google claimed they didn't do X" now it looks like they probably do X and therefore google lied.

They are somewhat circumspect about the fact that each example is actually, 8+ years ago Google made a claim about what they did at the time and now there's evidence that they are currently doing doesn't match that claim. That's not a lie.

For example, they link to this tweet from 2016 as such a claim: https://ipullrank.com/wp-content/uploads/2024/05/image21.png

Or a later example where they link to this 2016 talk: https://www.youtube.com/watch?v=iJPu4vHETXw

Later on, they use claims from 2012: https://www.seroundtable.com/google-chrome-search-usage-15618.html

If you read the original publication on this[1], they mention there’s a stray commit publishing the internal variant of the SDK intended for the actual Google warehouse database. So the code bases probably live close enough together for someone to accidentally pass the wrong folder name or something.

This has been fixed, but the commit and all it’s changes are out there—and tragically, published alongside a copy of the Apache 2.0 license (intended for the document warehouse API SDK), which officially sanctioned freely copying and using the code. So there is really nothing Google can do about it.

[1] https://ipullrank.com/google-algo-leak