Reading The Evidence
The Two Questions That Beat Every ADHD Product You Will Ever Be Sold
The Short Answer
Ask two things of any ADHD claim. How big is the effect? And did the person rating the outcome know who got the treatment? Almost everything marketed to adults with ADHD survives the first question and fails the second.
Nobody tells you which is which. So you spend money, you spend hope, and you spend the far scarcer resource of your willingness to try one more thing — and when it does not work you conclude, one more time, that the failure was you.
It was not you. It was almost certainly an unblinded trial with fourteen participants and a press release.
What follows is not a debunking. Several things on this page genuinely work. The point is that you are entitled to know how much they work, and that the two questions below will tell you, permanently, without a subscription.
Question one: how big is the effect?
Research reports the size of a treatment effect as a standardized mean difference, usually written as SMD or Cohen’s d.
It is a single number, and the convention is loose but useful: around 0.2 is small, 0.5 is medium, and 0.8 is large. An effect of 0.2 is real but you would struggle to notice it in your own life. An effect of 0.8 is the kind of change other people comment on.
Here is what that looks like for the treatment with the strongest evidence base. In the largest network meta-analysis of
ADHD medications, using clinician ratings, the numbers break down by drug family and by age:
1.02
Amphetamines, children and adolescents
CLINICIAN-RATED SMD
0.78
Methylphenidate, children and adolescents
CLINICIAN-RATED SMD
0.79
Amphetamines, adults
CLINICIAN-RATED SMD
0.49
Methylphenidate, adults
CLINICIAN-RATED SMD
SURVIVING ADHD · THE TWO QUESTIONS 1
Read that last figure again, because it is exactly the kind of detail that gets flattened in a headline. Methylphenidate in adults is a medium effect, not a large one — around half the size of the same drug in children, and noticeably smaller than amphetamines in the same adult population. That is not an argument against it. It is an argument for knowing which number applies to you, and for not concluding that medication has failed when what actually happened is that you were given a medium-sized effect and promised a large one.
The Useful Habit
When a study is quoted at you, ask what population it was run in. An effect measured in eight-year-olds is not automatically your effect. Most of the ADHD evidence base was built in children, and the adult numbers, where they exist, are frequently smaller.
Question two: did the rater know?
This is the one that empties the shelf.
In 2013 a group led by Edmund Sonuga-Barke did something unusual to the non-drug ADHD literature. They took every randomized trial of dietary and psychological treatments, and then analysed the results twice: once using the ratings everyone normally reports — typically from a parent or a teacher who knew perfectly well which arm the child was in
— and once using only ratings from someone who was probably blinded to the assignment.
The gap between those two analyses is the single most useful thing published about ADHD treatment in the last twenty years.
0.21 → 0.16
Free fatty acids / omega-3
UNBLINDED → BLINDED
0.64 → ns
Cognitive training
UNBLINDED → BLINDED
0.59 → ns
Neurofeedback
UNBLINDED → BLINDED
Cognitive training and neurofeedback both looked convincing when the rater knew. Both lost statistical significance when the rater did not. Omega-3 held on — its blinded estimate of 0.16 still just clears zero — but 0.16 is a very small effect, and it is not the effect implied by the bottle.
This is not fraud and it is not stupidity. It is what happens when a person who has driven their child to forty neurofeedback appointments is asked whether the child seems better. They want it to have worked, they have paid for it to have worked, and they are answering honestly. Blinding exists because good faith is not the same as accuracy.
Good faith is not the same as accuracy. That is the entire reason blinding exists.
SURVIVING ADHD · THE TWO QUESTIONS 2
Where honesty cuts the other way
An honest version of this argument has to include the part that complicates it. A larger and more recent meta-analysis of cognitive training, restricted to blinded and objective outcomes, found no significant effect on total ADHD symptoms or on hyperactivity — but did find a small, statistically significant effect specifically on inattention, at 0.17.
So the correct summary is not cognitive training does nothing. It is: cognitive training produces a very small effect on one symptom domain, roughly a fifth the size of what medication does, at considerably greater cost in time. That may still be worth it to you. It is a different sentence from the one on the website, and you are entitled to be the person who decides.
What survives both questions
What to do with this on an ordinary Tuesday