Ranked lists are the most readable format on the web. Ten things, numbered, with the winner at the top. The format promises that work was done, that alternatives were compared, and that someone reached a conclusion you can borrow.
Sometimes all of that is true. Often the numbers arrived last, after every other decision had already been made. This is a look at how a list of ten is typically assembled, in the order the decisions really happen, so you can tell the two cases apart.
What this is, and who it is for
This is for anyone who has finished reading a best-of page and felt uncertain whether it reflected any assessment at all. It is also for operators, who tend to encounter these lists from the other side and want to know what being on one, or missing from one, actually signifies.
Two things to set out first. Plenty of ranked lists are made in good faith by people who genuinely tested things and genuinely have a view. Those exist, they are worth reading, and the checks below are designed to find them rather than to dismiss the format. And nothing here describes any particular list. This is a general account of a common assembly pattern, and the point is to give you a procedure you can run yourself rather than a verdict to accept from us.
If you have not read the five ways listing sites earn, it is the mechanical background to everything on this page.
The order the decisions actually happen in
A reader assumes a list was built in this order: gather the candidates, assess them, order them by the assessment, write it up.
A list is often built like this instead.
- The topic is chosen because people search for it. Not because the writer had a view about it. The phrase comes first and the subject matter is derived from the phrase.
- The length is chosen next. Ten, usually. See the next section for why.
- The candidate pool is assembled from what is available. Often that means the businesses with an affiliate programme, the ones already covered elsewhere, or the ones that responded to an email. Availability, not merit, sets the boundary of the pool.
- Constraints are applied. Commercial relationships, existing partnerships, and the practical need to have something to say about each entry.
- The order is set. This is where money enters, if it is going to. It can also be set by which entry the writer knows best, which is a different problem with the same result.
- The assessment is written. The reasons are composed after the order exists, to fit it.
- The methodology paragraph is added, if there is one, usually last.
The important part is step six. In a genuine assessment, reasons produce the order. In this pattern, the order produces the reasons. The finished page looks identical either way, because reasons written to justify a decision read exactly like reasons that caused one.
This is why "they gave detailed reasons" is such a weak signal. Detail is cheap once the conclusion is fixed.
Where the number ten comes from
Ten is a formatting decision, not a finding.
It survives because it works on every axis except accuracy. It fits a page. It fits a headline. It is long enough to look thorough and short enough to finish. It gives a writer a fixed word budget. It also means the list can be updated by swapping entries in and out without changing the shape of anything.
The consequence is that the number sets the answer. If a category genuinely has three serious options for your situation, a list of ten contains seven entries that exist to reach ten. If it genuinely has forty, the list omits thirty of them and the omission is invisible. Either way, the reader cannot tell which case they are in, because the format looks the same.
Round numbers in a headline are a sign that the container was chosen before the contents. That is not damning by itself. It is a reason to read the entries individually rather than reading the ranking.
What "we assessed" usually means in practice
Most ranked lists include a sentence about how the ranking was produced. The sentence ranges from a genuine, repeatable method to a decorative claim that cannot be checked, and the two are written in similar language.
A stated methodology can be entirely real. What separates a real one from a decorative one is whether it could be repeated by someone else and produce a similar result. These questions expose the difference.
- What exactly was examined? "We evaluated each product" is decorative. "We ran the free tier for a week and completed the same three tasks in each" is a method. A method names the thing that was done.
- Who did it, and what did they already know? Assessment requires expertise, and expertise is specific. A stated author with relevant experience is a much stronger signal than an unattributed house voice.
- When? An assessment is a claim with an expiry date. A page describing software or services that names no date has told you nothing about whether the comparison still holds.
- What did not make it, and why? Real assessment produces rejections. A list that mentions nothing it looked at and declined has either not looked or is not telling you.
- Would the result change if the commercial relationships changed? The most useful question and the one almost never answered on the page.
Note what these questions do not do. None of them accuses anyone of anything. They just distinguish a claim that can be checked from a claim that cannot.
Where the disclosure sits
If a list carries a disclosure, its position is informative in itself.
A disclosure above the list, in the same size as the body text, in plain words, is a publisher telling you something before you use the page. A disclosure below the tenth entry, or inside a collapsed section, or in a footer link, or in grey type smaller than the text around it, meets a formal obligation while defeating its purpose. The disclosure exists so that you can factor it into your reading. That only works if you read it before you read the list.
So look for the disclosure early, and note where you found it. Both facts are data.
Four checks to run before you trust a ranked list
These take about a minute in total and none of them needs a tool.
1. Find the money before you read the entries. Look for an "advertise with us" page, a disclosure line, or a commission notice. Then look at the outbound links: hover one and read the real destination. You are not looking for a reason to reject the page, you are establishing which of the five models is running so you know how to read the order. Spotting a bought listing is the longer version of this check.
2. Test whether the order is load-bearing. Read the write-ups for the first and the seventh entry. If the reasons given for the top spot would apply equally well to the seventh, the ordering is decoration on top of a set. That is worth knowing, because the set may still be a useful set.
3. Look for the rejections and the date. A list that names what it excluded, and says when it was compiled, is showing you the shape of the work. A list with neither may still be good, but you have nothing to check it against.
4. Check the entry you already know. Almost everyone reading a ranked list knows something about at least one entry, or about something obviously missing from it. Compare the write-up against what you know. This is the single most informative check available and it takes ten seconds. If the description of the thing you know is thin, generic or wrong, treat the other nine descriptions as being of the same quality.
When a ranked list is genuinely useful
Ranked lists have real uses, and being sceptical of the ordering does not mean discarding the page.
A list is a good way to discover that a category exists and to learn its vocabulary. It is a good way to find candidate names when you have none. It is often a decent survey of what is commonly used, which is a genuine fact about a market even when it is not a fact about quality.
A list is also useful when the criterion is genuinely objective and stated. An ordering by a published, checkable number is a real ordering. The problem arises when a subjective judgement borrows the visual language of a measurement.
The practical move is to keep the set and drop the sequence. Take the ten names, ignore the order, and then do your own comparison on the two or three that fit your situation. Choosing between suppliers without ratings sets out a way to do that when there is nothing to rank on.
What we publish instead of an order
We publish no ordering of any kind. Nothing here is ranked, scored, starred, tiered or labelled as a pick. There is no top of a list because there is no list with a top.
What exists instead is a bar. An entry is either in the register or it is not, judged against our entry bar, which is published and versioned so it can be read before the entries are. Within any view, entries are not sequenced by merit. The reasoning behind that decision, and the trade-offs it carries, is in why we refuse to rank anything.
You should know what this costs you, because it is a real cost and it falls on you rather than on us.
- We will not tell you which of two entries is better for your purpose. You will have to work that out.
- We give you no shortcut. A ranked list is genuinely quicker to act on, and quicker matters.
- We are less useful when you know nothing about a category and want somewhere to start.
- We are harder to skim. A bar produces a set, and a set has no obvious first thing to read.
We accept those costs because an ordering that we could not defend for every pair of entries would be a decoration presented as a judgement, and because the moment an ordering exists it becomes the most saleable thing on the site. What we will not do is listed at the line.
In Plain English
What it means. The number in a ranked list is usually a formatting decision, and the order is often set before the reasons for it are written. Some ranked lists are the product of real assessment. Four quick checks separate them.
Who it helps. Anyone about to act on a best-of page, and any operator wondering what being on one is worth.
When to use it. In the first minute on a ranked list, before you read the entries.
What to do next. Run the four checks. Then keep the names and throw away the order, and compare the two or three that actually fit your situation. How directories work covers the rest of the reading skills this depends on.
See why we don't rank anyone
Refusing to publish an order costs a reader something real. Why we don't rate anyone sets out exactly what that cost is and why we take it anyway.