// Frozen edition · taken 15 October 2026
The State of Broken Online Stores.
This is the September 2026 edition, frozen on 15 October 2026. Nothing on this page recomputes: it is what the study said when it was taken, which is what a citation needs. The live report has today’s figures and they will differ.
Of 1,483 public Shopify storefronts visited in a real browser through to the payment step, 37.7% had a broken or degraded buying path on the night they were visited.
1,483 visits of 1,483 distinct stores, one night each, between 1 September 2026 and 2 September 2026 (UTC). Frozen edition: September 2026 edition, taken 15 October 2026. The live figures may have moved since. Our own demo store and test hosts are excluded from every figure.
The sample
public Shopify storefronts, found from public sources and reachable without a login — so the study is about Shopify stores and says so rather than generalising to every platform. Each store was visited once, at night in its own time zone, from the region closest to its market. A store is in the sample if its walk completed and was scored; a walk that fell over measured nothing and is not counted as a healthy store.
Markets are the store’s own currency, recorded when the sample was built — not the region the browser ran from. A cut by region once put 212 US-market stores in a Canadian row, which is the kind of error that makes a study worthless, so the two are kept separate. A market’s row is only shown once it passes 50 stores.
The method, and what it cannot tell you
A real Chromium browser at 1440×900 and 390×844 walks home → collection → product → add-to-cart → cart → checkout, stopping at the payment step. Deterministic checks run first; a language model judges each page from its screenshot under a fixed set of rules, and is never asked about load time — that is measured twice in code, so a judged finding about a slow page cannot be invented. robots.txt is respected. Nothing is ever ordered.
- One night each. A store that was broken only on the night we visited looks broken; one that broke the next day looks fine. This is a cross-section, not a monitoring record.
- We never complete a purchase, so a purchase event cannot be verified and no finding here claims it was.
- A bot-challenged store is not counted as broken. Being walled is a fact about automated browsers, not about a shopper, so those walks are excluded rather than scored.
- Shopify only. Nothing here says anything about WooCommerce, BigCommerce or a custom build.
- Aggregates only. No store is named, no URL is published and no screenshot of a third-party store appears anywhere in this report. That is the condition of visiting other people’s stores at all.
The findings
Six, each with the number it is a share of and the query that produced it. If you think a number is wrong, the query is there to argue with.
Finding 1
One store in four had a broken or degraded buying path on the night it was visited
A P1 or P2 finding means a step of home → collection → product → add-to-cart → cart → checkout failed, or something on it was broken. Not a slow page: a broken one.
The query this number came from
select coalesce(t.market, case s.region when 'syd' then 'AU' when 'iad' then 'US' when 'lax' then 'US' when 'yyz' then 'CA' when 'lhr' then 'UK' when 'fra' then 'EU' else 'other' end) as market, count(*)::int as n,
count(*) filter (where exists (select 1 from findings f where f.scan_id = s.id and f.severity in ('P1','P2')))::int as broken
from scans s left join corpus_targets t on t.domain = s.domain
where s.status = 'complete' and s.score is not null and not (coalesce(s.domain, '') ~* '(^|\.)(navaal\.ai|fly\.dev|localhost)$' or coalesce(s.domain, '') ~* '^(www\.)?(fernwick|.*bilby-testsite)') and s.ip = 'corpus' group by 1 having count(*) >= 50 order by 2 descFinding 2
The median mobile homepage took longer to show its main content than Google's threshold allows
Google's good threshold for largest-contentful-paint is 2.5 s and its poor threshold is 4 s. This is a real phone viewport in a real browser, not a simulation.
The query this number came from
select
count(*)::int as n,
percentile_cont(0.5) within group (order by lcp)::int as p50,
percentile_cont(0.75) within group (order by lcp)::int as p75,
count(*) filter (where lcp > 2500)::int as over_2500,
count(*) filter (where lcp > 4000)::int as over_4000
from (select (select (p->>'lcp')::numeric from jsonb_array_elements(coalesce(s.summary->'speed'->'pages','[]'::jsonb)) p where p->>'viewport' = 'mobile' and p->>'page' = 'home' and (p->>'lcp') is not null limit 1) as lcp from scans s where s.status = 'complete' and s.score is not null and not (coalesce(s.domain, '') ~* '(^|\.)(navaal\.ai|fly\.dev|localhost)$' or coalesce(s.domain, '') ~* '^(www\.)?(fernwick|.*bilby-testsite)') and s.ip = 'corpus') x where lcp is not null and lcp > 0Finding 3
Purchase-path tracking was missing or broken on more than one store in ten
A Money finding means an event that should have fired on add-to-cart or at the start of checkout did not reach the pixel. Purchase itself cannot be verified without placing an order, and we never place one.
The query this number came from
select
count(distinct s.id)::int as visits,
count(distinct s.id) filter (where exists (select 1 from findings f where f.scan_id = s.id and f.lens = 'money'))::int as any_money,
count(distinct s.id) filter (where exists (select 1 from findings f where f.scan_id = s.id and f.lens = 'money' and lower(f.title) like '%add-to-cart%'))::int as atc,
count(distinct s.id) filter (where exists (select 1 from findings f where f.scan_id = s.id and f.lens = 'money' and lower(f.title) like '%checkout%'))::int as checkout
from scans s where s.status = 'complete' and s.score is not null and not (coalesce(s.domain, '') ~* '(^|\.)(navaal\.ai|fly\.dev|localhost)$' or coalesce(s.domain, '') ~* '^(www\.)?(fernwick|.*bilby-testsite)') and s.ip = 'corpus'Finding 4
The same eight faults account for most of what we found
The most common fault is an analytics event that never fires — invisible on the storefront, and it quietly misprices every ad campaign the store runs.
The query this number came from
select lower(f.title) as title, f.lens, f.severity, count(distinct f.scan_id)::int as scans from findings f join scans s on s.id = f.scan_id where s.status = 'complete' and s.score is not null and not (coalesce(s.domain, '') ~* '(^|\.)(navaal\.ai|fly\.dev|localhost)$' or coalesce(s.domain, '') ~* '^(www\.)?(fernwick|.*bilby-testsite)') and s.ip = 'corpus' group by 1,2,3 order by 4 desc limit 8
Finding 5
Most stores score well; the tail is where the money is
The score is P1 35 · P2 20 · P3 10 · P4 5 off a hundred. A store under 50 generally has a failed buy-path step and a tracking fault at once.
The query this number came from
select
count(*) filter (where s.score >= 90)::int as b90,
count(*) filter (where s.score >= 70 and s.score < 90)::int as b70,
count(*) filter (where s.score >= 50 and s.score < 70)::int as b50,
count(*) filter (where s.score < 50)::int as b0,
count(*)::int as n
from scans s where s.status = 'complete' and s.score is not null and not (coalesce(s.domain, '') ~* '(^|\.)(navaal\.ai|fly\.dev|localhost)$' or coalesce(s.domain, '') ~* '^(www\.)?(fernwick|.*bilby-testsite)') and s.ip = 'corpus'Finding 6
Severity, as the product itself grades it
P1 stops a sale, P2 damages one, P3 and P4 are worth fixing and will not cost you tonight. Only P1 and P2 count towards the headline.
The query this number came from
select f.severity, count(distinct f.scan_id)::int as scans from findings f join scans s on s.id = f.scan_id where s.status = 'complete' and s.score is not null and not (coalesce(s.domain, '') ~* '(^|\.)(navaal\.ai|fly\.dev|localhost)$' or coalesce(s.domain, '') ~* '^(www\.)?(fernwick|.*bilby-testsite)') and s.ip = 'corpus' group by 1 order by 1
A second study · September 2026
Product data on 409 live Shopify storefronts
A different method on a different sample, which is why it is a section rather than a row above: a public-web audit of product data on 409 storefronts, with no browser walk and no logins. Published 14 September 2026, updated 24 September 2026. 71.9% had at least one actionable product-data finding — and that is the wrong number to plan around.
| How you define “actionable finding” | Share of stores | Of |
|---|---|---|
| All nine checks | 71.9% | 294/409 |
| Excluding missing product_type and thin descriptions | 36.2% | 148/409 |
| Only attributes AI shopping feeds require, plus policy pages | 22.7% | 93/409 |
| Only structured data (missing or duplicated) | 15.9% | 65/409 |
| Only crawler access | 0.5% | 2/409 |
- Median findings per store: 1 (mean 1.28, maximum 6).
- Stores with zero findings: 28.1%.
- 35.7% had no finding except a missing product_type and/or a short description.
The sensitivity table is the point: the headline is carried by two content-quality checks, and the figure that matters for AI shopping feeds is 22.7%. The full study has the nine checks, their denominators and the method.
Cite this
Published under CC BY 4.0: use the figures, with attribution. The aggregates are downloadable as CSV so you do not have to retype them.
Navaal AI. The State of Broken Online Stores — September 2026 edition. 1 September 2026–2 September 2026. https://navaal.ai/bilby/data/2026-09