Why most hosting benchmarks are worthless

Most hosting speed comparisons do not measure the host. They install an empty WordPress site on each candidate, fire requests at the homepage from one machine, run the test once, and publish a winner to three significant figures. Under those conditions the differences between hosts are mostly noise, geography, and setup choices wearing a lab coat. The failures below are the recurring ones, and the standard at the end is what a comparison has to meet before it earns your click, let alone your migration.

The empty install

A fresh WordPress with the default theme, no plugins, and no content asks the host to print one cached file, a job every competent platform does indistinguishably well. Real sites are what you have: dozens of plugins, a WooCommerce cart, logged-in users, search, and uncached paths that must run PHP and touch the database on every request. A benchmark that removes the workload also removes the differences you were shopping for, and then measures how quickly each host can do nothing. A light test has one defensible use, checking that a platform's baseline is sane; it cannot say how the same hosts carry a real shop, and the charts pretend otherwise.

One run, one moment

A single test at a single hour from a single machine is one sample of a noisy process. Neighbouring load on the node, a backup job, a cron burst, network weather between datacentres: any of these reorder the ranking, and nobody publishes the reordering. Run the same test again five minutes later and you will often get a different winner. A chart with no repeats and no spread is a screenshot of luck. Spreads belong in the chart itself: a host whose results swing between runs is telling you something a median alone hides, and an honest reviewer prints both.

Cache states nobody declares

Host A gets tested with a warm cache, host B takes its first cold hit, and the result is an accident reported as a finding. The same thing happens with CDNs: leave one enabled on a single host and the chart ranks content delivery networks, a layer every host can switch on, while the host's own work stays invisible. Meanwhile the homepage, the one page every benchmark loves, is the page most likely to be cached on every platform on earth. The uncached pages, the admin, the cart, the search, never appear.

What you cannot see

The environment is missing. Which plan tier, trial or production, since the resources often differ. Which region. Which PHP version, which plugin set, what traffic pattern, tested when. Without those facts a number cannot be checked, reused, or argued with; it can only be believed, which is a strange thing to do with a purchasing decision. Region alone can reorder a latency comparison without the hosts differing at all, which is why an undisclosed vantage point invalidates the ranking outright.

The affiliate behind the chart

Many comparison sites earn a commission per referral, and the host paying the best commission has a striking record of winning the speed tests. Affiliate money rarely fakes the numbers outright; it removes the incentive to doubt them, and the disclosure, when it exists, sits below the table. This site runs on affiliate economics too, which is exactly why we publish the method before the numbers and attach raw data when they land. A benchmark you can argue with is the only kind worth being paid for.

The sin of precision

Milliseconds quoted to three significant figures, from one run, on shared infrastructure, across the internet. Real measurements carry spread, and honest reports show the spread: median and range across repeated runs. Precision with no repeats is a costume the number wears to look like knowledge. When a host is declared the winner by a margin smaller than the noise between runs, you are reading fiction with decimal places.

What a fair comparison looks like

The same real site, cloned identically onto every host. The same recorded traffic pattern replayed against each, public pages plus admin and cart activity in realistic shares. Both cache states, tested separately and labelled. Repeated runs across different hours, with medians and spread. The environment published alongside: plan, region, generator, plugin list, dates, raw data attached. Numbers produced that way can be argued with, which is the point. Numbers produced any other way can only be believed.

The checklist for the next benchmark you read

What was installed. How many runs. Which cache state, and was the CDN on. From where, and on which plan. Who pays the site when you click. Most published comparisons fail two of those before you reach the chart. The method we hold ourselves to is written out in the throughput benchmark playbook, and the load testing guide has the version you can run yourself against your own site.