Does my store actually work tonight?
Bilby's Watch check opens your store in a real Chromium browser every night — on desktop and mobile — and visits the path a shopper takes: home, a collection, a product, add-to-cart, the cart, and checkout up to the payment step. It also tries your site search, checks the newsletter form validates, applies a discount code if you gave one, fetches the top ten pages from your sitemap, and reads your SSL and domain expiry. A step that fails becomes a finding with a screenshot; a step Bilby can't map is never a finding.
What the Watch check catches
- Add-to-cart that fails — the store's own cart endpoint refused the item (HTTP 4xx) or showed an error
- A disabled add-to-cart with no sold-out message and no variant chooser
- Checkout that shows an error instead of the information step
- Collection or product pages that return HTTP 4xx/5xx from your own navigation
- Site search that returns nothing for a product you sell
- A newsletter form that accepts an invalid address silently
- A discount code you gave us that the cart rejects
- Top-10 sitemap pages returning errors, and unexpected redirects
- SSL certificate or domain registration expiring within 14 days
- Mixed content (http:// resources on an https page) and images that fail to render
- Expired promotions still showing (a promo phrase with an end date in the past)
- Pages that look broken, stale or confusing to a first-time customer — judged by Claude from the screenshot, with a literal evidence quote or the finding is discarded
Severity — the rule, not a feeling
Questions people ask
Does Bilby place test orders?
No. Bilby visits to the checkout's information step and stops. It never enters payment details, never logs in and never submits a real form. Purchase tracking therefore can't be verified by Bilby — the Money check prints the two-minute owner check instead.
What happens when a step can't be mapped on my theme?
It is recorded as "couldn't map this step" — not a defect, not a score penalty. Bilby only reports what it saw fail with evidence; a visit it couldn't complete says so plainly.
How do you avoid false alarms?
Deterministic checks first (HTTP status, the cart endpoint's answer, the certificate's dates). Claude judges screenshots only for what a human eye needs, under thirteen rules with a mandatory evidence quote, and a code backstop suppresses known bot-vs-customer artefacts (cookie banners, geo modals, empty ad slots). Every check passed a 10-store zero-false-positive gate before shipping.
Read next: Why is my add-to-cart button not working? · How often should you test your checkout? · 7 silent problems that kill sales overnight · Is Shopify down, or is it just my store? · Shopify discount code not working
// SEE IT ON YOUR STORE
The free scan runs the Watch check tonight — the whole report, nothing locked.
Scan my store free