Have you ever felt frustrated searching for your site online and not finding it in the search results? Have you wondered why your site isn't showing up in Google search results?

In this connected world, appearing in search engines is vital to your site's success. So let's explore together why your site isn't showing up in search engines, and how we can fix that to increase your site's visibility and online success.

Scarabio's First Page Method

I've put together a number of solutions that will permanently solve the problem of a site disappearing from search engines, and combined them into a single method — the "First Page Method." I've applied it to several projects, and the results were excellent in improving site indexing in search results.

It also helped me improve site rankings in search engines for more than one keyword.

If you want to know why your blog isn't showing up in search engines and how to fix the problem, follow along with the steps below:

The first step you need to take is to make sure of your site's indexing status in search engines — you need to confirm whether search engines have been able to crawl your site and index its pages or not.

There's more than one way to check this, and the two best are (advanced search-engine search, and the inspection feature in Google Search Console), which we'll explain simply below:

The Inspection Feature in Google Search Console

This is a very easy method — all you need to do is check the URL you want to confirm is indexed using the URL Inspection feature:

[caption id="attachment_145" align="aligncenter" width="1520"] Screenshot from GSC, February 2024[/caption]

If the result is like what's shown in the image (URL is on Google), it means that page has been indexed properly in the search engine; if the result is different, it means there's an issue preventing that page from being indexed.

Advanced Search-Engine Search

This is the easiest and best way to check your site's indexing status in a search engine. Advanced search (search operators) is done by searching your site's root domain to find out how many of your site's pages have been indexed by the search engine.

Site: domain.com

You need to put the search operator (site:) before your site's root domain.

[caption id="attachment_146" align="aligncenter" width="1232"] Google search results, February 2024[/caption]

The number of results that appear is the number of pages that have been indexed in the search engine, and you can search the same way if you want to check the indexing status of a specific page.

Site: domain.com/specific-page/

If the result of your indexing check, using either method, shows that the page or site hasn't been indexed, the following steps will help you fix that problem.

And if the result shows that the site has been indexed but isn't appearing on later pages, the following steps will help you rank your site on Google's first page

Meta robots tags are tags that tell search engines whether or not you want a page to be indexed — they're a piece of code found in the <head> section of a web page.

They can also be used to instruct search engines on whether or not to follow the links on that page.

You can find them by searching your page's code for meta name = 'robots'

You should typically find it looking like this:

<meta name='robots' content='index, follow, max-image-preview:large, max-snippet:-1, max-video-preview:-1' />

Or like this:

<meta name='robots' content='index, follow' />

The instructions in the code ('index, follow') tell the search engine to index the page and follow its links.

If you find different instructions like (noindex, nofollow), this means you're instructing the search engine not to index the page and not to follow its links.

So if you want that page indexed, change the instructions to the first code ('index, follow'), either by editing the page's code directly, or, if you're using WordPress, by editing it through the SEO plugin you use on your site (Yoast, Rank Math, etc.)

Note: If you don't find a meta robots tag on your page, search engines treat it as ('index, follow') by default.

The robots.txt file is a text file that tells search engines how to crawl subfolders or pages on your site — through it, you can prevent search engines from visiting a page or an entire subfolder on your site, meaning it won't get indexed.

Creating a robots.txt file is one of the basics of SEO, and it's different from meta robots tags — meta robots tags allow the search engine to visit the page but prevent it from indexing it or following its links, while robots.txt prevents the search engine from visiting the page at all.

You can find it by adding robots.txt after your site's domain.

domain.com/robots.txt

This file contains instructions for search engines, which I'll explain simply below:

User-agent: *

This means the following instructions apply to all search engines (Google, Bing, Yahoo, etc.).

Disallow: /specific-subfolder/

Here, you're telling search engines not to index a specific subfolder, as well as any page within it.

If you find a subfolder here that you actually want indexed, you should go directly to the file and delete this instruction, or, if you're using WordPress, edit it through the SEO plugin you use.

And if you don't find any important file next to a Disallow instruction, that's normal, and search engines will be able to visit and index the other files or pages.

An XML sitemap is like a map containing all your site's links, provided to search engines so they can crawl those links. It also helps search engines understand your site's structure and helps them quickly discover your new content on an ongoing basis.

You can find it in a browser by going to:

yourdomain.com/sitemap.xml

Or yourdomain.com/sitemap_index.xml

And if you don't find one, there are many tools to help you create one, such as XML Sitemaps Generator, or WordPress plugins like Yoast and Rank Math.

It should also be added to Google Search Console through the Sitemaps feature in the sidebar.

[caption id="attachment_148" align="aligncenter" width="1909"] Screenshot from GSC, February 2024[/caption]

So you should pay attention to your XML sitemap, given its major importance in improving your site's indexing.

Google penalties (manual actions) are penalties Google imposes on you for repeatedly making mistakes on your website, and they significantly affect your site's indexing status and appearance in search engines.

You can find out if you have one by going to Google Search Console and checking the Manual Actions report in the sidebar.

[caption id="attachment_149" align="aligncenter" width="1918"] Screenshot from GSC, February 2024[/caption]

If you see "No issues detected," that means everything is fine; if you see "# issues detected," that means you have manual actions, and you need to investigate the causes of the problem and fix them.

Tip: If you have an issue, don't just delete the pages where the issue shows up — you need to fix the root cause of the problem so it doesn't happen again in the future.

"Content is king" is a famous phrase in digital content creation, and it's one of the most important factors in ranking your site on the first page of search engines. If you take care of it, your site's indexing status will improve, and in turn its ranking. Below are the most important points to keep in mind when creating content: What is the main goal behind a user's search for the keyword you're targeting — do they want to know a piece of information, or do they want to buy a product?

You need to answer this question before publishing your page, and think carefully about how to properly meet the user's search intent.

The amount of content on a page is one of the most important factors affecting its indexing status — if a search engine crawls your page and finds the content thin, the page won't be indexed and, as a result, won't appear in search results.

Note: Thin content is content with fewer than 300 words.

Duplicate (repeated) content is one of the worst practices you can have on your site — publishing the same content more than once is a haphazard practice that hurts your site, and it's one of the most important answers to the question of why your blog isn't showing up in search engines. Copied content is content that's been directly copied from another site, and this is one of the most rejected practices by search engines — it's one of the causes of getting a manual action penalty, in addition to being an unprofessional practice that loses users' trust in your site.

So you should focus on your content and make it unique and not copied, avoid repeating it on your site, pay attention to its amount, and make sure it fully satisfies users' search intent.

Internal linking is when a page on a website points to another page on the same site, and it's one of the most important practices for improving a site's structure.

If you're not using it correctly, your site's structure will look poor to search engines, and as a result, your pages won't get indexed in search results.

Its importance lies in allowing search engines and users to discover other pages and articles on your site.

Internal linking is extremely important, so you should focus on it and implement it correctly on your site.

Targeting the same keyword on more than one page is a type of content overlap called "keyword cannibalization" — this means focusing or targeting more than one page on exactly the same keyword.

This confuses the search engine and makes it hard for it to determine which page is most relevant to the keyword, putting those pages in competition with each other, which as a result means none of them will appear well in search results.

So you shouldn't repeat your targeting of the same keyword, since it puts you in competition with yourself, which hurts your site's indexing status and its ranking in search results.

If you're competing for difficult keywords, it will take a lot of time and effort to appear on the first page of search results, while targeting less competitive keywords takes less time.

So you should choose your keywords carefully so you can achieve a good ranking in search results in less time, and in turn get more visits and great sales.

Some website developers, while building a site, block Google from indexing it because the site is still under construction and not finished yet — this is usually done by setting the meta robots tag to noindex, and the most common way this happens is through enabling a setting in WordPress.

If you find that setting enabled, you should turn it off.

So you should check your site's meta robots tags and make sure they're not blocking Google from indexing your site.

Good keyword research alone won't be enough if you don't use the keyword correctly — you might have done excellent keyword research, but your site still isn't showing up in search engines, and this can be due to several reasons, one of the most important being incorrect keyword usage. Here are the correct steps for using a keyword:
  1. Article title: Add the keyword to the article title in an appealing way
  2. Meta tags: Add the keyword to the meta tags, such as the title tag and description tag
  3. Page URL: Include only the keyword in the page's URL, and try to keep the URL short
  4. Alt text: One of the most commonly overlooked practices, but you should pay attention to it — add the keyword to the alt text of the images on the page.
  5. Avoid keyword stuffing: Adding the keyword to the page is good, but overusing it results in what's called keyword stuffing, so be careful not to overuse the keyword.
Paying attention to your site's server is one of the most important factors in site stability — if the server has any issues, this results in severe slowness on the site, and sometimes the site can go down completely.

If the site goes down repeatedly, search engines will crawl it less often, which delays indexing.

So you should pay attention to your server and check it regularly to avoid any issues that could happen now or in the future.

One of the most important factors for ranking well in search engines is a good user experience, and site speed is one of its most important elements, given its effect on user behavior on the site, and in turn its effect on conversion rates and visits.

One of the best tools to help you test speed is Google PageSpeed Insights — just enter your site's URL, and it will tell you your site's speed on mobile and desktop.

[caption id="attachment_150" align="aligncenter" width="1911"] Screenshot from PageSpeed Insights, February 2024[/caption]

The better your site's speed, the better its ranking in search engines, and if your site's speed is poor, that could be one of the strongest reasons your blog isn't showing up in search engines — this shows just how much search engines care about site speed.

When a site doesn't use HTTPS, data is unencrypted and can easily be intercepted by hackers — this means sensitive user information, such as passwords or credit card details, can be at risk.

Google and other search engines place great importance on security — if your site uses HTTPS, it's more likely to get a higher ranking in search results compared to a site that doesn't use HTTPS. Google explicitly states that using HTTPS is one of its ranking factors.

To enable HTTPS on your site, you need to purchase and install an SSL certificate. There are free certificates and paid ones — paid certificates may offer better protection and support.

Mobile usability is one of the most important elements of the user experience — with the evolution of technology, most users now browse the internet through their mobile phones, and if you're not paying attention to your site's mobile usability, you'll lose most of your customers.

So browse your site from your own phone and try it out as a regular user, and note down anything you didn't like as a user and try to fix it.

One of the best tools for testing mobile usability is Bing's Mobile Friendliness Test Tool — just enter the page's URL, and within seconds you'll get the test results.

[caption id="attachment_151" align="aligncenter" width="1275"] Screenshot from Mobile Friendliness Test, February 2024[/caption]

If the result is "This page is mobile friendly," your site works well on mobile. If a different result appears, like "Page is not mobile friendly," there's an issue with your site's mobile usability that you need to fix.

There used to be a great tool from Google called Mobile-Friendly Test that checked mobile usability, but it was discontinued in early 2024.

Some of the best practices for improving mobile usability include increasing font size to suit mobile screens, leaving enough space between links and buttons on your site to make it easier for users to navigate, and adjusting image dimensions to fit small mobile screens.

Backlinks are links from one website to another, and search engines treat them like a vote or recommendation from one site to another — the more backlinks your site earns, the better its ranking in search engines.

But if you get links from harmful sites, this will hurt your site's indexing status and appearance in search engines.

Backlinks usually come through communication and collaboration with other sites, and you should choose those sites based on how relevant their field is to your site's field.

So you need to be very careful when choosing which sites to reach out to, since backlinks can sometimes do more harm than good.

There are many tools that can help you check your site's backlinks, such as Moz or Semrush.

Domain Authority, or DA for short, is a metric that tells us how trustworthy and well-known a site is. This metric is a score from 1 to 100 — the higher the number, the better and stronger your site looks in the eyes of search engines like Google.

How to Improve Domain Authority?

To make your site more trustworthy and well-known, follow these simple steps: Think of Domain Authority as your site's mark on the internet — to improve it, you need to write good content, get other sites talking about you, use social media, make your site easy and fast, and update it regularly. This is the last step you can take after making sure your site is free of all issues and you've applied SEO practices to it — this is done through Google Search Console.

Click URL Inspection, enter the URL of the page you want indexed, then click Request Indexing, and after waiting a few minutes, it will finish processing.

[caption id="attachment_152" align="aligncenter" width="1495"] Screenshot from GSC, February 2024[/caption]

Google may reject this request, and that would be because it found an issue on your site — but if you follow Scarabio's First Page Method, you won't find any issues on your site, and your indexing request will be approved.

Getting Your Site Indexed by Search Engines

Getting your site indexed by search engines is extremely important for your online success — when your site is visible in search results, it increases the chances of target visitors reaching it, which boosts engagement with your content or services.

To get your site indexed by search engines, you need to create an account with webmaster tools (such as Google Search Console and Bing Webmaster Tools) — these tools help you monitor your site's performance and submit it to search engines.

You should also add your XML sitemap to both tools so they can easily access all of your site's pages.

New Blogs and Websites

Run your first GEO analysis

Top pages, real keywords, dual-format AI content — scored for citation before you publish.

Try the GEO tool free
In the SEO process, it can sometimes take some time to see results, and for SEO for new sites in particular, it takes longer than usual for search engines to notice them and start showing them with a good ranking.

But it's not a difficult matter — with time, good practices, and attention to the user experience on your site, you'll get a good ranking in search engines.

Additional resources

Tools for checking your site's Google ranking

SEO Audit

This is a process of analyzing your site and uncovering all its issues, and determining how well your site aligns with SEO best practices — it's one of the most important steps to take when creating a future plan with measurable results.

The purpose of the audit is to identify as many issues affecting your site's performance in search engines as possible, and the analysis should include:

There are tools that can help with this, such as SEO Site Checkup, but tools aren't effective enough on their own, since they don't understand the nature of your site or your specific field.

If you need to work with an SEO expert to run an SEO audit and fix the issues, feel free to reach out to me!

[caption id="attachment_99" align="aligncenter" width="1200"] If you'd like a free consultation, click the image above![/caption]

Sources