Before a page can rank, Google has to clear three steps with it: discover it, crawl it, then decide to index it. Technical SEO is the craft of smoothing that path, spotting where it stalls, and fixing it.
This section gathers IndexProbe's cross-cutting guides on crawling and indexing. Where the Google Search Console indexing statuses section breaks down each state in the report one line at a time, these guides take the topic by its mechanics: how Google discovers your URLs, how fast it crawls them, and why some never reach the index.
Diagnosing an Invisible Site
Why Is My Website Not Showing Up on Google? The Full Diagnostic
A site missing from Google falls into one of three cases, never indexed, deindexed, or indexed but ranking too low, and each calls for a different treatment. The full decision tree, from symptom to Search Console status, all the way to verifying the return.
Checking Your Indexing
Google's URL Inspection Tool: The Complete Guide (and Its Limits)
The one place where Google says, URL by URL, what it did with your pages: verdict, coverage state, selected canonical, last crawl, rendered page. Every data block decoded, the two modes (indexed version vs live test), the tool's structural limits, and the official API to go at scale.
The Google Search Console API for SEOs
The two APIs SEOs actually use (Search Analytics and URL Inspection), the real quota mechanics Google doesn't spell out (sliding window, per-property quota, prefix properties), and the point where pulling indexing data by hand stops scaling: for large sites and agencies especially.
Discovery and Submission
How to Submit Your Website to Google
From the indexing request to verification. Why submitting guarantees nothing, how to declare a sitemap properly, the real limits of the URL Inspection tool, and how to confirm what Google actually indexed.
Crawl Budget and Crawl Frequency
Crawl Budget: Is It Actually Your Problem?
A real SEO lever, but not for every site. Who crawl budget truly matters for, why the threshold sits far lower than the million pages Google cites, and how to pinpoint, URL by URL, where the budget drains away.
Crawl Errors and HTTP Codes
How to Fix 404 Errors in Google Search Console
Not all 404s are equal, and not all of them need fixing. How to tell legitimate 404s from broken links worth repairing, and how to handle the volume without losing entire days to it. (This guide also belongs to the GSC indexing statuses section.)
Facets and URL Volumes
Faceted Navigation & Faceted Search SEO: What to Index
A single category page and four filters are enough to generate thousands of crawlable URLs. Which facets deserve the index, which ones to close (and how), and how to read Google's verdict family by family instead of guessing.
Duplicate Content and Canonicals
Why Duplicate Content Is an SEO Issue (and It Is Not a Penalty)
Duplicate content is not a penalty: Google groups your duplicate pages and keeps only one, the canonical. The real issue starts when it keeps a page other than yours. What the 2026 documentation says, the duplicate-family statuses decoded, and how to check at scale which canonical Google kept.
JavaScript Rendering and Frameworks
JavaScript SEO: What Google Actually Indexes After Rendering
Google's JavaScript rendering now works as an asynchronous queue, far from the old "two waves" model. How Googlebot runs your JS, why your crawler's render is not Google's verdict, and the rendering-symptom → GSC-status grid, for SPAs and classic sites alike.
SPA & React SEO: The 4 Indexing Statuses to Watch
A single-page application (React, Vue, Angular) only exposes its content once the JavaScript runs, and a failed render throws no error. The four Search Console statuses that betray a badly indexed SPA, and the honest triage before switching everything to SSR.
Pagination and Crawl Depth
Pagination SEO & Best Practices: Normal Status or Red Flag
Every paginated page gets its own verdict in Search Console, and they're not all equal: "Crawled – currently not indexed" on a page 3 is usually normal, while a canonical status almost always points to a configuration error. The reading grid, the classic mistakes (page-1 canonicals included), and how to diagnose at scale.
Coming to This Section
Technical SEO for crawling and indexing doesn't stop there. Several guides are in the works: duplicate content on e-commerce sites, and a decision guide for choosing between canonical, noindex, robots.txt, or 404 depending on the case.
The indexing pitfalls specific to each CMS have their own section: CMS Indexing Pitfalls.
Why These Guides Matter
A crawling or indexing problem isn't visible to the naked eye: a page can look perfect and stay out of the index for weeks. The only reliable source is Google's official verdict, page by page. Checking it by hand in Search Console, one URL at a time, doesn't hold up at scale. That's exactly what IndexProbe does: the bulk version of the URL Inspection tool, querying the official Search Console API for the whole list of URLs you provide or build from GSC, with the indexing status, the non-indexation reason, and the last crawl date for each one.
The technical layer is never an end in itself: it serves concrete situations — migrations, monitoring, large catalogs, traffic drops. IndexProbe's use cases connect each of those projects to Search Console's official data.
Frequently Asked Questions
What's the difference between crawling and indexing?
Crawling is Googlebot's visit to read the page. Indexing is Google's decision to keep it in its index. A page can be crawled without being indexed: Google read it but didn't judge it worth keeping. The two steps are distinct, and a problem can arise at either one.
Where should a technical SEO audit start?
With the "Page indexing" report in Google Search Console, which shows how many of your URLs are indexed and why the rest aren't. That's the starting point: it steers you to the right guide, whether the issue is discovery, crawl budget, HTTP errors, or duplicates.
Are these guides meant for small sites?
Partly. Some topics, like submission or 404 errors, concern every site. Others, like crawl budget or faceted navigation, only become real issues past a certain URL volume. Each guide states clearly who the topic is relevant for, without needless drama.