Catch the deploy that hides your site from search engines
The most expensive SEO mistakes are made in a deploy, not in a content plan. The staging configuration reaches production and every page carries noindex. A template change points every canonical tag at the home page. robots.txt from the test system blocks everything. The sitemap is regenerated with no URLs in it. The pages all load, so uptime monitoring stays green — and the damage shows up weeks later as traffic that slowly disappears.
Pingsured checks for all of these on every run, so a mistake is caught within minutes of the release that made it.
The page itself
An HTML check on your most important pages — the home page, a category, a product:
| CSS selector | Operator | Value |
|---|---|---|
meta[name=robots][content*=noindex] | does not exist | |
link[rel=canonical] (attribute href) | equals | https://www.example.com/ |
title | matches | \S |
h1 | count equals | 1 |
link[rel=alternate][hreflang] | count at least | 5 |
The first line is the one worth adding to every ping you have. The last catches a multilingual site whose language links went missing.
A noindex can also arrive as an HTTP header. A header check on X-Robots-Tag with "does not exist" covers that.
robots.txt
robots.txt is plain text, but the HTML check reads it all the same. One check catches a file that blocks the whole site, without tripping over ordinary rules such as Disallow: /admin:
| XPath | Operator |
|---|---|
//body[matches(., '(?m)^Disallow:\s*/\s*$')] | does not exist |
The sitemap
An XML check on /sitemap.xml, as described on our page on feeds and sitemaps: the number of /urlset/url entries stays above what you expect, and no loc points at a staging host.
Why check SEO with Pingsured
- Minutes, not weeks. A search console tells you once the damage is visible. A check that runs every few minutes tells you while the release can still be rolled back.
- Your real pages, from outside. The checks read what a crawler gets, not what your CMS says it publishes.
- From where search engines crawl. A firewall rule or CDN setting that blocks visitors from the US is invisible from Europe; with a check from our US location, it fails there.
- One ping, several checks. Status code, robots tag, canonical and headline in one request, and the ping passes only when all of them do.
Limits worth knowing
- Checks do not run JavaScript, so tags a single-page application sets in the browser are not seen. Most search engines prefer them in the HTML the server sends anyway.
- Each ping checks one URL. Pingsured does not crawl your site; choose the pages whose loss you would notice first.