Skip to content
All insights
Article2 min read

Why every case study here carries an evidence label

Numbers on this site now say where they come from — measured, estimated or illustrative — and whether anyone will vouch for them.

A before/after number is the most persuasive thing on a consulting website and the easiest thing to fake. Anyone can write "61% → 89.2%". So from today every case study on this site carries an evidence record, and the build fails if one is missing.

What the label says

Each record has three parts.

Source. Client project means built for a paying client. Internal system means built and run by us, on our own data or product. Reference design means an architecture we would build, described with typical values rather than observed ones.

Numbers. Measured means the before/after was observed on a live system, and the record says how: which holdout, which window, which traffic split. Estimated means derived from partial measurements or benchmarks. Illustrative means representative values for a reference design — useful for understanding the shape of the work, not evidence that it happened.

Reference. Whether the client will take a reference call under NDA. Most will not, and the label says so instead of implying otherwise.

Why not just show the measured ones

Because a reference design is still useful to a reader deciding whether a problem is solvable, as long as it is not dressed up as a result. Removing them would make the site thinner without making it more honest. Labelling them makes the difference visible in one glance.

What enforces it

The record lives next to the copy in the content files, and a validation step in the build refuses any case study without one, any "reference design" that claims measured numbers, and any homepage figure that does not link to the case it was taken from. The rule is mechanical on purpose: it does not depend on anyone remembering to be careful.

If you are comparing vendors, ask each of them the same three questions — source, numbers, reference — and see how long the answer takes.

Have a process that costs too much?

Tell us about it. The first call is free, and if AI is not the answer we say so.

Discuss your task