How Google Decides Who’s on Top

“If it isn’t on Google, it doesn’t exist.” —Jimmy Wales

The goal of this post is to uncover the simple three-step process that Google uses to find and rank the immense universe of Web pages floating around cyberspace. Much of the process through which Google “finds,” “reads,” and ranks Web pages is fairly straightforward and commonsensical.

Google Bit Off More Than It Could Chew—The Entire Internet!

According to Netcraft, there are around 644,275,754 active websites on the Internet—and billions of individual Web pages, as most sites contain several pages. And this number is growing by 5 percent each month, see here.

Even more impressive is how effective Google is at what they do. Google has the self-appointed role of finding, scanning, and ranking this expanding universe of sites and delivering the intended result within fractions of a second.

Think about it: The next time you follow a whim to jump on your laptop and search for “the flight speed of a European swallow,” Google’s engine will somehow “decide” which of the billion or so Web pages are likely the most helpful for you, and display them in order of relevance within milliseconds.

To support their goal of producing better results more quickly, Google frequently changes their search algorithms, sometimes wiping out thousands of websites, companies, and even industries in the process. Although this may leave most of us small business owners with a feeling of helplessness at the whim of this elusive information-eating organism, the core process has changed very little—Google uses a simple, three-step process to produce those lightning-quick search results:

• Crawling: Google spiders—also known as Googlebots—scour the Web, following all the links on every page, and the links on those, and so on.
• Indexing: These spiders scan the text of each Web page they find, determining the subject matter of the content.
• Ranking: Google determines the order in which Web pages containing similar content are listed in search results, evaluating them based on trust, relevance, and authority.

Let’s dig just a little bit more deeply into each of these three steps.

CRAWLING

Think of the Web as a series of individual documents (Web pages) linked together by strands, like a spider’s web. Although you may often use the term “website,” incorporating all the pages within, search engines see and scan each individual page on the Web independently. A website is simply a collection of related Web pages linked together using hyperlinks; without these links, the Internet would serve no purpose, denying us and search engines any way to find or rank information!

KEY CONCEPT: Understanding “Googlebots”

Google spiders, also known as “Googlebots,” crawl the entire World Wide Web, scanning each Web page (i.e., billions of documents) and exploring its hyperlinks, storing this data in one of several indexes. This process continues until the search engine spider has found, “read,” and indexed virtually every page on the Internet! Therefore, a great way for Google to find your site is for it to notice and explore links on other sites that point to yours.

INDEXING

Once a Googlebot crawls a website, recording every page of the site, every word, image, and link, it then makes a copy of it, files it under the right category, and produces it within milliseconds on command! Google parses out and stores the code from these pages in massive data centers—Google’s index—ensuring that data can be served up instantaneously. Google assigns a unique ID to each Web page, and even indexes the content of each page to identify precisely which terms it contains.

NOTE: Like Santa, Google Knows Whether You’ve Been Naughty or Nice. When you publish an article on your website, Google finds and scans it within days or even sooner, making note of who wrote it, how long (word count) and origional (unique) it is (an indicator of quality ), what it’s about, how many people “Like” it on Facebook and other social sites, and much more.

RANKING

This is where the rubber meets the road. Have you ever wondered why your site shows up on page three of Google for an important search term, while your competitor’s enjoys the number-one spot, when both sites have roughly the same content?

Assuming your site has been found, crawled, and indexed, it’s in the final step, Ranking, that the search engine battle is won or lost.

Upon receiving a search query, Google must first return only those results related to the query, and, second, rank these results in order of relevance and importance. This ranking process is known as a search engine algorithm. Each major search engine company maintains its own algorithm with the goal of producing the “best” (most useful and relevant) results.

Think of the ranking process as a filter: a search engine’s mission in producing a search results page is to start with all the pages of data on the Web, then filter these pages through a series of screening steps, and, finally, narrow the list to what the search engine determines to be the “best” list of results.

Although complex in its entirety, Google’s search engine ranking algorithm boils down to a few simple concepts that govern how they rank Web pages:

• Relevance: The first test a Web page must pass in getting ranking is relevance. For example, if someone searches for the term “vacations,” Google’s first task is to pull all Web pages that include this exact term, or “keyword.” This is why On-Page SEO is the first step in an SEO campaign. Therefore, your ticket to even being considered in search results is relevant content!
• Authority (also referred to as “Importance”): Unlike relevance, authority is determined largely by backlinks, which are those links residing on other Web pages that point to yours. You may be familiar with the terms “Google Page Rank,” or, “PR.” Google’s page rank system is a measure of authority related to a given search term. Authority—and, therefore, page rank—hinges upon links as a gauge of popularity. In simpler terms, Google’s authority algorithm functions by treating the Internet like a massive voting system/popularity contest, treating each link to your site like a vote: if website A’s subject matter is soccer and has 900 other soccer-related sites linking to it, and site B is also about soccer but has just 150 soccer-related links, then site A is judged to be the greater authority related to soccer and therefore ranks higher.
• Trust: The third phase of Google’s ranking algorithm is trust, or, “Trust Rank.” Trust is measured largely by how consistent and reliable a website is in providing accurate information to users, and this places a higher value on more established sites. Time, therefore, is a component of the trust algorithm, meaning websites that have been around longer get more trust points.
• SPAM Filters: One of the primary reasons that SE algorithms are always changing is due to the need to address “Black Hat SEO” practices, also called “search engine spamming.” These are techniques deployed by site owners who try and game the system by gaining search engine traffic without increasing their rankings naturally, i.e., building authority with quality backlinks over time. Google is aggressive about such sites—excluding them from search results maintains quality and user satisfaction. There are several negative ranking factors that may have adverse effects on your search engine rankings without your knowing it. In Chapter 8, we’ll show you how to keep out of hot water, focusing solely on ethical link-building methods.

KEY CONCEPT: SEO GOES BEYOND WHAT’S ON YOUR WEBSITE

One of the biggest mistakes small business owners make when it comes to SEO is focusing all of their energy on their website and not enough on getting high-quality links to their site from others. All other things being equal, usually the site with the most trusted backlinks will rank higher on Google.

In my next post, we’ll dive more deeply into a simple-to-implement small business SEO plan.

BLOG CHECKLIST

Understand the Three-Step Process Google Uses to Rank Web Pages

• Crawl: Googlebots (Web spiders) scour the Web, following all the links on every page.
• Index: These spiders determine and store the subject matter (tops) of the content.
• Rank: Google determines the order in which Web pages containing similar content are listed in search results, evaluating them based on trust, relevance, and authority.

NEXT STEPS:

Sign-up for our Masterclass below. If you’re struggling with the the frustration, tech overwhelm and information overload that so often comes along with online marketing, this workshop is for you.