Why is my website not in Google

Not indexed or badly ranked: work this out first

Almost everyone who says "my website is not in Google" actually means one of two things. It pays to know which. Google has to find your page first, then fetch it and then take it into the index. Only after that comes the question of what position it appears in for a search.

Not indexed is a technical problem. It has an identifiable cause, you can usually repair it yourself and the effect is sometimes visible within a few days. Badly ranked is not a problem with one cause. There you compete with other pages about the same subject and no setting works like a switch.

The confusion costs time. Anyone who is not indexed and starts tinkering with keywords achieves nothing. Anyone who is indexed and keeps waiting for indexing is waiting for something that has already happened.

The check that takes five seconds

Type this in Google: site:yourdomain.com, without a space after the colon. You get a rough list of what Google holds in the index for your domain.

• No result at all: your site is not in there. Go to the diagnostic tree at the bottom.
• Results, but not the page you are looking for: that one page is the problem, not the site.
• Everything is there: you do not have an indexing problem. You have a ranking problem or a problem with the keywords you are aiming at.

This list is not exact and not complete. It is meant as a first test to tell the two cases apart and for that it is good enough.

Search Console gives the real answer

Google Search Console is free and it is the only place where Google tells you what it makes of your page. You prove once that the domain is yours, usually through a DNS record or a file on the server. After that you have access to the indexing status per URL.

The tool you need is called URL inspection. You paste the full URL at the top and Google shows whether the page is in the index and if not, why not. At the bottom there is a button to request indexing. That button is no guarantee and it does not work faster if you press it more often.

These common messages each point at something different. The exact wording differs per language setting.

The four causes it almost always is

A noindex that stayed in place after the launch. This is the most embarrassing one and the most common one. During the build the site is deliberately set to noindex so that half-finished work does not land in Google. At launch someone forgets to flip that flag. In WordPress it is the checkbox under Settings, Reading, which discourages search engines from indexing the site. In the source code you then see <meta name="robots" content="noindex">. The correct value for a page that has to be found is index, follow, max-snippet:-1, max-image-preview:large.

A robots.txt that blocks too much. That file sits at yourdomain.com/robots.txt and you can simply open it in your browser. If it says Disallow: / under User-agent: *, you are asking every search engine to read nothing. Watch the order of causes: robots.txt governs whether Google may read, the noindex tag governs whether Google may include. Anyone who blocks a page in robots.txt and also puts a noindex in it achieves the opposite of what they want, because Google does not read the page and therefore never sees that noindex. What you want removed from the index, you let it read with a noindex inside.

A canonical that points at something else. The canonical tells Google which version of a page is the real one. That is useful with duplicate content. It goes wrong when a theme or plugin puts the same canonical on every page, usually to the homepage. Google follows that instruction and does not index the rest. On every important page, check that <link rel="canonical"> points at that page itself.

Google does not know your site yet. A new domain without a single link from outside can stay unnoticed for a long time. How long indexing takes cannot be predicted reliably. Days to weeks is normal and there is no promised term.

Submitting a sitemap to Google

A sitemap is an XML file with the URLs you want to see indexed, usually at yourdomain.com/sitemap.xml or yourdomain.com/sitemap_index.xml. Most content management systems and SEO plugins create it automatically.

You submit it in Search Console, under Sitemaps. You paste the path, you click submit and after that you see how many URLs were found. A sitemap speeds up the discovery of new pages. It does not force indexing. Google decides for itself which URLs it takes in.

Three mistakes keep coming back. A sitemap with URLs that are on noindex gives contradictory signals. A sitemap with the old http or www variant points at redirects instead of at the real pages. And on a multilingual site one language version often lacks an entry of its own, which leaves that language quietly invisible. That last one comes back at length in chapter 5.

What indexing does not solve

Being indexed is the floor and not an achievement. A page that sits in the index but reaches nobody has a different problem: there is too little on it, it answers no question anyone asks or ten other pages answer that question better. Our working threshold is at least 300 words of content for a page that has to explain something. That is a floor and not a target.

Google also does not index everything it comes across. Thin pages, near identical variants and automatically generated overviews often stay out of the index without anything being broken. In that case no button helps and only better content helps.

The diagnostic tree in five steps

Work through these five steps in order. Stop as soon as you find something, repair it and start again at step 1.

1. Type site:yourdomain.com in Google. Zero results for the whole domain: go to step 2. Results but your page is missing: go to step 2 for that one URL. Everything is there: you have a ranking problem, continue with chapter 2.
2. Open yourdomain.com/robots.txt in your browser. If it says Disallow: / under User-agent: *, take that line out. If the path of your page is in there explicitly, take it out. If the file does not exist, that is not a problem.
3. Look at the source code of the page with Ctrl+U or Cmd+Option+U and search for noindex. If you find it, set the robots meta to index, follow, max-snippet:-1, max-image-preview:large or remove the tag. Check the settings of your CMS as well.
4. Search that same source code for canonical. If it points at a URL other than the page you are looking at, correct that. Set it to the page itself.
5. Go to Search Console, URL inspection, paste the URL. Read the reason given there, request indexing and submit your sitemap under Sitemaps if that has not happened yet. Then wait. Days to weeks is normal.

If all five steps are clean and the page is still not in the index after a few weeks, it is no longer a technical problem. Then the page is too thin, it is too much of a copy of something else or your domain is still an unknown to Google that justifies nothing. You solve that with content and with mentions elsewhere, not with settings. See chapter 10.

What you can do yourself and what you cannot

To do yourself, without a developer and without budget:

• Run the site: test and read the outcome correctly.
• Create Search Console, verify the domain and use URL inspection.
• Open robots.txt, read it and take out a wrong Disallow rule.
• Switch off the noindex checkbox in your CMS.
• Submit the sitemap and check how many URLs are in it.

What you need help with:

• Verification through DNS if you cannot get at your domain management.
• A canonical that is generated by a theme or a plugin and cannot be changed through a setting.
• A noindex that comes from an HTTP header instead of from the HTML, because you do not see that one in the source code.
• A site that is built entirely through JavaScript, where Google gets an empty page to look at. That is a build decision and not a setting.

Count on half a day for the whole thing, including creating Search Console. It is the only subject in this guide where the result is sometimes visible within a week.

Would you rather not do this yourself: SEMANU carries out the full SEO and GEO audit for 1,450 euro, report and working session included. What it covers is on the guide page.