Home  /  Methodology  /  Signal-to-Noise · 2026-04-27

Signal-to-Noise Benchmark — SEO Industry Cohort

Top10Lists.us 100.00% mean Relevance Ratio — cohort median 73.38% (Site A–D, receipts.json names them; range available there too).

Of what an AI ingestor reads on this page, what fraction is on-topic answer body vs. peripheral chrome (nav, sidebar, related-posts, ads, newsletter forms)? Five sites, 24 pages, deterministic selection rule, public-tools-only stack.

Frozen: 2026-04-27 — measurements at this URL will not change.  ·  Permanent dated artifact  ·  GeoLocus Group, a subsidiary of Aryah.ai

Companion to /methodology/sitemap-benchmark/2026-04-26 →

Authors: Robert Maynard, Cofounder and CEO · LinkedIn →  ·  Mark Garland, Cofounder and CRO · LinkedIn →

The April 27, 2026 press release frames a single thesis: before AI can read your site, it must translate it. This page is the empirical instrumentation of that thesis. We measure the Relevance Ratio (RR) — primary-content characters divided by total visible-text characters — across a 5-site SEO industry cohort. RR is reproducible with trafilatura + a small boilerplate-strip helper; cohort-fair because every site is reduced to the same numerator/denominator definition.

Cohort: Top10Lists.us (baseline) plus four established SEO industry sites, anonymized in the page body as Site A through Site D. Per-site identities and per-page URLs are surfaced in the downloadable receipts.json →.  Sample: 5 pages per site = 25 pages total (24 returned 200; one Site B page returned 403 even with Googlebot UA — see Limitations).

1. The Metric — Relevance Ratio (RR)

For each fetched page, working at the visible-text layer:

RR = primary_content_chars / total_visible_text_chars

total_visible_text_chars
  = visible-text characters from the rendered HTML, after stripping ONLY
    <script>, <style>, <noscript>, and HTML comments.
    Keep nav, header, footer, sidebar, ads, newsletter forms,
    related-articles widgets, recent-posts lists, breadcrumbs, comments --
    everything a human or LLM would see if rendered.

primary_content_chars
  = visible-text characters of the article body, AFTER additionally
    removing the boilerplate envelope (nav/header/footer/aside/form/iframe/svg
    when located outside <main>/<article>, ARIA-roled boilerplate,
    and class/id substring matches for related/sidebar/widget/newsletter/
    subscribe/comments/breadcrumb/recommended/promo/advert/popup/modal/
    cookie/share/pagination/site-footer/site-header/nav-/navbar/menu).

  Sanity floor: also compute primary content via trafilatura.extract();
  take the MAX of the boilerplate-strip pass and the trafilatura pass.

Reported as a percentage. Higher RR = a smaller fraction of what the AI ingestor reads is wrapper noise.

2. Why RR Over the Prior CDR Draft

CDR (signal_bytes / total_response_body_bytes) is a defensible measurement of crawl-time bandwidth efficiency, but it is not the metric AI ingestors actually run. Ingestors strip script/style as the first pass (compression and rendering happen for free), then feed the visible-text layer to a boilerplate-removal extractor. RR measures the only thing that matters at that layer: of the words the LLM sees, what fraction is on-topic article body vs. peripheral chrome?

CDR also penalises a site for having a heavy JS shell even when the visible-text layer is pristine. RR isolates the question Robert posed in the press-release framing: of what an AI ingestor reads on this page, how much is the answer?

We retain CDR as a separate downstream concern (it costs crawler budget but does not directly poison citations). The page that lives at this URL from this date forward measures RR.

3. Methodology

Page selection (deterministic & reproducible)

For each site:

1. Fetch <homepage>/sitemap.xml.
2. From the sitemap index, evaluate the first 8 shards and pick the deepest (most <url> entries). This avoids cherry-picking and ensures we benchmark the workhorse content surface, not the tiny “about” shard.
3. From that shard, select 4 deterministic indices: floor(n × 1/5), floor(n × 2/5), floor(n × 3/5), floor(n × 4/5). Same five pages will be selected on any future re-run.
4. Add the homepage as page 1.

Total per site: 1 homepage + 4 deep pages = 5 pages. The same URLs were measured for the prior CDR draft so the new RR figures are directly comparable.

User-Agent policy (transparent)

The cohort is therefore measured under the UA each site actually serves with HTTP 200. Site C's RR is a best-case figure (browser UA tends to receive richer payloads than bot UAs); the AI-bot-observed RR for Site C is N/A.

Extraction pipeline

Implemented in Python with requests + BeautifulSoup4 + lxml + trafilatura. Source: sn_bench_rr.py. Reproducible end-to-end with pip install trafilatura beautifulsoup4 lxml requests.

1. Fetch URL with cohort UA, follow redirects.
2. total_visible_text_chars:
     - parse HTML
     - decompose <script>, <style>, <noscript>, comments
     - get_text(separator=' ', strip=True), collapse whitespace
     - count chars
3. primary_content_chars (boilerplate-strip pass):
     - parse HTML again
     - decompose <script>, <style>, <noscript>, comments
     - decompose <iframe>, <svg> globally
     - decompose <nav>/<header>/<footer>/<aside>/<form>
       ONLY when NOT descendant of <main> or <article>
     - decompose role=navigation|banner|contentinfo|complementary
     - decompose any element whose class/id contains a boilerplate token
     - get_text + collapse whitespace, count chars
4. primary_content_chars (trafilatura pass):
     - trafilatura.extract(html, url, favor_recall=True,
                            include_tables=True, include_comments=False)
     - count chars of returned text
5. primary_content_chars = MAX(boilerplate-strip, trafilatura)
6. RR = primary_content_chars / total_visible_text_chars

The MAX rule is the calibration choice. On clean-room HTML where <main> contains the entire answer, boilerplate-strip wins. On WordPress sites where <article> exists but trafilatura's classifier handles boilerplate more aggressively, trafilatura wins. The MAX prevents either failure mode from artificially deflating a score.

Calibration

Original 2026-04-27 measurement: Top10Lists.us bot HTML rendered <body> = a small <header class="site-header"> (~50 chars: top-nav links) + <main> (the entire answer body) + <footer class="site-footer"> (~500–600 chars: contact + Quick Links), and we measured 88–97% RR with a page-mean of 90.52% — the fixed ~600-char site-chrome amortising poorly across smaller pages.

Post-strip (2026-04-27, gildi PR #284): the site-chrome was removed from bot HTML. Re-measurement on the same 5-URL sample lands at a 97.67% page-mean, with 4 of 5 pages at exactly 100.00%. The home-page row remained at 88.36% because / was served by a static _home.html shared with humans — outside bot-HTML scope at that point.

Post-fork (2026-04-27, gildi PR #287): the homepage forked into a bot-only stripped variant (public/_home-bot.html: chrome deleted, content wrapped in <main>) routed at / by the existing detectBot(ua) detector in functions/_middleware.js. Browsers continue to receive the full _home.html with nav, footer, and form UI intact. Re-measurement on the same 5-URL sample lands at a 100.00% page-mean, with 5 of 5 pages at exactly 100.00%. The cohort comparison and bands below reflect this post-fork ground truth; the per-page raw RR table preserves pre-strip, post-strip, and post-fork rows so the reader can see exactly which chars went where at each stage.

How to reproduce

The bench script sn_bench_rr.py is available on request. Anyone with Python 3 and the four libraries above can re-execute the measurement against the same cohort — and because the page-selection rule is deterministic, will land on the same five pages per site (modulo sitemap drift).

4. Per-Page RR (raw)

25 pages total (5 sites × 5 pages, minus 1 Site B page that returned 403). Top10Lists.us rows show both pre-strip and post-fork RR — the chrome-strip (revision 2, gildi PR #284) and the homepage fork (revision 3, gildi PR #287) both happened on 2026-04-27 and we preserve all readings so the staged effect is auditable rather than retconned. The post-fork column is the current ground truth. Cohort site URLs are surfaced under their concrete domains in receipts.json →; path stems below are kept opaque in the page body to preserve the Site A–D mapping.

5. Per-Site Aggregates

Bands

6. Per-Site Interpretation

7. Why Site B's Reading Shifts vs the CDR Draft

8. The Agency-Bot-Block Inversion

Site C is a content-marketing agency. Its commercial pitch is that it produces content for clients. Yet the site returns HTTP 403 Forbidden to every bot UA we tested — Googlebot, ClaudeBot, GPTBot. The site is unreachable to AI ingestion at the door. The concrete identity is surfaced in receipts.json → with outcome: "blocked" evidence.

We measured under a Chrome 120 desktop UA so we could compute any RR figure for Site C at all. That figure — 53.84% mean — is the best case. The real RR an AI crawler would observe is N/A, because those crawlers can't reach the page at all. And even at best case, roughly half of what a browser-based reader sees is peripheral chrome.

An agency that 403s the very systems clients are paying it to be visible to is the clearest single-data-point illustration of the gap between SEO industry posture and AI ingestion reality. We document it here so the receipt exists.

9. Limitations

1. Sample size. 5 pages per site. We chose deterministic indices to make the result reproducible, not exhaustive. A full-cohort run (e.g., 50 pages/site) would tighten confidence intervals; ranks unlikely to flip between adjacent bands.
2. Site C UA workaround. Site C returns 403 to every bot UA we tested. We measured under a Chrome 120 desktop UA. This is a best case for Site C — the RR an AI crawler would observe is N/A because those crawlers can't reach the page at all.
3. One Site B page returned 403. One deep Site B page returned 403 even with Googlebot UA; n=4 for that site (concrete URL in receipts.json).
4. Boilerplate detection is heuristic. The class/id token list and semantic-tag rules are calibrated to the cohort. Sites that use unusual class names for related-posts widgets may receive an inflated RR.
5. MAX-of-two extractors is a calibration choice. We adopt it because either extractor alone has known failure modes (boilerplate-strip under-extracts on plain-<div> blogs; trafilatura under-extracts on structured-data-heavy pages with agent grids and license cards). The MAX rule biases toward generosity; reported RR is an upper bound on what an ingestor's specific extractor would compute.
6. RR doesn't measure content quality. A page can have a 95% RR and still be junk; conversely, a page with rich body content can be obscured by heavy chrome. RR measures where the noise lives, not whether the answer is correct. For correctness, the AIFS Probe layer (4-platform citation rate) remains the authoritative measurement.

10. Connection to the 9-Dimension GEO Rubric

The canonical Aryah AI GEO rubric awards 15 points for the Content Density dimension — co-equal weight with AI Bot Access and Authority. Until now, that 15-point allocation has been scored against soft heuristics (word count, schema entity density, body-to-shell ratio approximations).

RR is a sharper instrumentation of the same dimension at the layer that actually matters — the visible-text layer the ingestor's extractor consumes. We are not proposing to replace the 15-point Content Density score with RR alone. RR is one of the instrumentations under that umbrella, alongside semantic-container coverage, content-to-chrome word ratios, and JSON-LD entity density.

What RR adds: a single-number reproducibility check that a third-party reader can run in five minutes, against any site, on any machine with Python 3, trafilatura, and BeautifulSoup4. That is the standard the press release frames — not “trust our composite,” but “here is the script; run it yourself.”

Conclusion

Across a 5-site SEO industry cohort — Top10Lists.us plus four established SEO industry sites (anonymized as Site A–D in this page; identities in receipts.json) — Top10Lists.us delivers a 100.00% mean Relevance Ratio (post homepage fork 2026-04-27): the highest in the cohort, with 5 of 5 sample pages registering at exactly 100.00%. The same 5 URLs measured 90.52% mean before any chrome intervention, then 97.67% after the bot-HTML chrome strip (gildi PR #284), then 100.00% after the homepage fork (gildi PR #287); we preserve all three readings in the per-page table and aggregate ladder so the staged intervention is auditable. The cohort median (n=3, excluding Site C which 403s every bot UA) lands at 73.38% — a clean ~27 percentage-point gap, with two-thirds of one competitor's body content (Site B 65.18%) ceding to peripheral chrome. The full cohort range is available in receipts.json →.

Read the press release this benchmark supports at geolocus.ai/press →