How Google Crawls (and Indexes) Your Therapy Website

How does a website end up on Google? Do they just magically appear once a site goes live? Of course not! This is the 21st century after all.

No, Google crawls websites. Google uses “spiders” to crawl, but don’t let arachnophobia get ahead of you. The spider is a software program: a string of code that jumps between websites following trails left by URLs. Every time it finds a site it adds it to Google’s enormous library, kind of like a Rover picking up and examining Martian rocks.

What that means is that when you search “Google” (or any search engine), you’re not searching the entirety of the internet—although it can feel like it. What you’re searching is Google’s ever-expanding catalog of the internet.

And the bots do a lot of work to make that catalog as all-encompassing as possible. Back in 2016, Google said they were aware of over 130 trillion individual pages (130,000,000,000,000.) And the web has only grown since then.

But how does a particular therapy website show up on Google?

All searches start with a question. This is true both on and offline. (At the time of this writing there have been over 3 billion searches today, alone!)

When we ask Google a question it’s Google’s job to return the best possible result. So let’s say you want your website to be the answer to a user’s question. How do you do it?

First of all, see if your site is indexed on Google. You can find out easily by heading to Google and searching “site:yourdomainname.com”. This search will show you every page that Google has already indexed.

But if you have a brand new website there are some steps you can take to ping Google’s attention.

  1. Set up Google Analytics and Google Search Console
  2. Ensure that there are internal links between the pages on your site.
  3. Share your articles and posts on social media channels
  4. Submit your website to therapist directories.
  5. Make sure your articles are SEO friendly.

The more relevant your information is to people’s queries, the more likely your website will show up on Google.

The biggest mistake people make, that we’ve seen all too often, is that people fail to link between articles on their own site. Remember what we said about the spider-bots. They crawl a website by jumping between URLs.

Think of each URL as a door: every time you create links between pages you’re creating a door between them so that Google can move from one room to the next. A well-linked website is like a mansion with many doors.

Reasons why Google can’t crawl your page.

Most website owners shouldn’t have too much trouble finding themselves indexed on Google—showing up on the first page might be another story!

But if you do have trouble finding your site indexed by Google it may be because you’ve run into one of the following issues:

  1. A poorly written meta description.
  2. Server issues.
  3. URLs that have been misconfigured
  4. A missing, or poorly compiled, robots.txt. file.
  5. A domain name with questionable history
  6. Poor PageRank.

You should be using Google Search Console to make sure your site is indexing properly.

What if I don’t want Google to index a page?

There are a number of reasons you may want to tell Google to ignore a page: particularly if the page in question was for a limited event, or it’s not longer relevant to you or your website anymore.

If you don’t want Google to index a page you simply have to add a “noindex” tag to that particular page.

Yoast SEO (a free plugin for the basic package) makes adding a noindex tag easy as a click.

Image of Yoast SEO plugin for WordPress showing how to noindex a post so that it doesn't show up on Google.

Simply click the carrot for the option that says “Allow search engines to show this Post in search results” and select No. If you’re choosing to noindex a post that’s already live it may take a few weeks for the results to show up on Google—as the spiders have to re-crawl your website.

While there are several ranking factors that you can take care of easily—including targeted specialty pages, blogging, social media, and many others—the technical ranking factors mentioned above are a little harder to change, without technical knowledge. If you’d like us to take a look at your website’s performance, feel free to contact us today.

And finally, here’s a helpful video from Google that illustrates exactly how search works. It’s well worth watching!

Scroll to Top