Why therapy is harder to research than a tablet — and what that means for the evidence

If you want to test whether a drug works, you have an elegant procedure available. Half the people receive the active compound, half an identical-looking tablet without it, nobody knows who is in which group, and at the end you compare. That is the double-blind randomised trial, and it is the reason we can trust much of medicine.

Now try the same with psychotherapy. What is a placebo conversation? How do you blind a therapist so they do not know whether they are delivering therapy? How do you measure the outcome when the goal is “we argue less” or “I stopped being afraid of my own anger”?

This piece is about why psychotherapy is harder to research than a tablet — and what that means when you read the phrase “research shows”. This is not an excuse. It is about the fact that the standard we know from medicine is harder to apply here, and that you therefore have to read differently.

Four problems a tablet does not have

First: there is no true placebo. With a drug, an inactive tablet looks the same. With therapy, every substitute — conversation without method, an educational group, a waiting list — either contains active ingredients or is obviously something else. This is precisely why comparisons are often against a waiting list, which overstates the effect: the comparison is not with “almost the same” but with “nothing”.

Second: nobody is blinded. The client knows they are attending therapy. The therapist knows what they are doing. So expectation enters on both sides, which with a tablet we at least partly neutralise.

Third: the procedure is not standardised. Ten milligrams is always ten milligrams. Cognitive behavioural therapy delivered by two therapists is not the same thing — differences between individual therapists are documented in research and are not small. The method is more a recipe than a molecule.

Fourth: the outcome is hard to measure. Blood pressure is a number. “Things are going better in my relationships” is not. Questionnaires completed by the person themselves are used — useful, but sensitive to what someone wants to convey and how they felt that day.

What we nonetheless know quite reliably

Now the other side, so this does not read as though we know nothing. We know a good deal — obtained differently.

Psychotherapy works better than nothing. This is the best-supported claim in the field. Meta-analyses consistently show medium to large effects against control groups, and the finding repeats across decades and across conditions.

Differences between established approaches are small. This result appears so persistently that it acquired a name from the Dodo bird in Alice in Wonderland: “everybody has won and all must have prizes”. It is not universal — for some specific conditions a particular approach has a clear advantage — but as a general rule it holds more than the schools would like.

The relationship is one of the most reliable predictors of outcome. A meta-analysis of more than three hundred studies shows a persistent association between the quality of the working alliance and the result.

The effect is retained after the end better than with medication. This is one of the few differences where therapy has a clear advantage, and it follows from the fact that a person takes what they learned with them.

Some people get worse. We know this too, and it is fair to say — research on negative outcomes exists, though it is cited less.

How the field handles these limitations

Rather than pretending the problems do not exist, research works around them in several ways.

Comparing two active treatments. Instead of “therapy versus nothing”, two approaches are compared. This removes the placebo problem, though it obscures how much of the effect is general.

Blinded assessors. The client and therapist cannot be blinded, but the outcome assessor can. If they do not know which group someone was in, one source of bias falls away.

Manuals and fidelity checks. Therapists work to a written protocol, and sessions are recorded and rated. This increases comparability while moving the study further from how therapy actually happens in a real room.

Practice-based data. There is increasing research tracking outcomes in ordinary clinics rather than research settings. Less control, more reality.

What follows when you read “research shows”

The practical part. Five questions worth putting to any such sentence.

What was it compared with? If with a waiting list, the effect is almost certainly overstated. If with another active treatment, the claim is stronger.

Who measured the outcome? Participant self-report is weaker than a blinded assessor.

How many dropped out? If a third left and only those who stayed are reported, the number says less than it appears to.

How long was the follow-up? The effect at the end of treatment and the effect after a year can be very different.

Has it been replicated? One study is not a finding. The replication crisis in psychology showed that many a headline result does not hold up on repetition.

Why this is not an excuse

Someone might say: if measurement is this hard, then claims about effectiveness mean nothing. That does not follow, and it is worth explaining why.

A methodological difficulty is not the same as an absence of evidence. Surgery cannot be studied double-blind either — a patient cannot be blinded to whether they were operated on — yet nobody claims we know nothing about surgery. Other designs are used, conclusions are drawn somewhat more cautiously, and knowledge accumulates all the same.

The same holds for psychotherapy. We know it works; we know less precisely which part works most. That is an honest and sufficiently useful picture.

What would be dishonest is doing both at once: invoking research when it supports my approach, and invoking methodological limits when it does not. That happens in the field more often than one would like.

What this means when choosing a therapist

Some conclusions that follow.

Do not choose by approach, except for specific conditions. If you have panic disorder, obsessive-compulsive disorder or insomnia, some approaches are clearly better supported. Otherwise the approach matters less than the person.

Be wary of promised percentages. “Eighty per cent of my clients improve” is not a figure anyone has independently verified.

Ask about supervision and continuing training. Because the provider is part of the effect, their own work on themselves matters more than in professions where the provider does not influence the result.

Measure it yourself. The most useful study is your own: watch whether anything is changing in daily life. If after several months it is not, that is important information — I have written separately about the small signs of progress.

What to remember

Psychotherapy cannot be researched the way a drug can, because there is no convincing placebo, nobody is blinded, delivery differs between therapists and the outcome is hard to measure.

Even so, we know it works better than nothing, that differences between established approaches are small, and that the quality of the relationship is one of the most reliable predictors of outcome. That is less precise than we would like — and considerably more than we have for most of the things we choose in life.