The Myth of Mental Illness

[Jules]

Smudge, you keep referring to evidence ("where is it?", "more would be useful"), but whose responsibility is it to provide that evidence?

I don't accept that my analogy is flawed. Actually, the parallels with religion go further than I suggest; the therapist also has to be a believer for the therapy to be successful.

As has been said by myself others, his article is biased, and unhelpful.

"Bias is an inclination to present or hold a partial perspective at the expense of (possibly equally valid) alternatives."

So what are the equally valid alternatives which Lutus is ignoring? and he has given supporting evidence throughout to back up his statements.

If Lutus had titled his article 'improving mental health care based on evidence' it would have been a start. But no. We get a tabloid level 'damn them all'.

I think you're placing too much emphasis on the title, and the last comment is a straw man.

[Pebble]

Have now tracked down the meta-analysis, as expected the results do not support the conclusions drawn.

Journal of Affective Disorders 68 (2002) 159–165
www.elsevier.com/ locate / jad
Review
A meta-(re)analysis of the effects of cognitive therapy versus
‘other therapies’ for depression
Bruce E. Wampold*, Takuya Minami, Thomas W. Baskin, Sandra Callen Tierney
Department of Counseling Psychology, University of Wisconsin–Madison, 321 Education Building–1000 Bascom Mall, Madison,
WI 53562, USA
Received 6 January 2000; accepted 14 June 2000

The objective of the meta-analysis was to determine whether CT was superior to other recognised psychiological therapies, and to confirm that it was superior to placebo therapies.

The apriori definition of a bona fida therapy was that:
1. It was administered by a therapist trained to at least masters degree level in the relevant field
2. The therapy was administered face to face and individualised for the patient
3. The trreatment was psychologically valid - including evidence that the active ingredients of the therapy were identified and referenced.

Treatments classified as non-bonafida included:

group psychotherapy; individual supportive therapy, relaxation therapy.

CT was significantly superior to 'placebo' therapy.

CT was superior to bona fida therapy (p<0.01), but one large study was then retrospectively excluded from the analysis, because it was a statistical outlier.

Problems - only pre 1994 studies included, and outlier study suggesting that focused effective therapy was superior to CT was not also excluded; the number of patients in the bona fida comparison studies left to analyse was 265 - too small to exclude other than large effects; the definition of placebo therapy is subjective.

Conclusion CT is superior to talking to a friend. The studies comparing it to other 'psychological' therapies are too small to reach valid conclusions.

[Jules]

Ok, thanks. I have let Paul know about this. Maybe he HAS overstepped the mark. If so, I hope he is big enough to amend the article.

CT was significantly superior to 'placebo' therapy.

Do you have any info on what a placebo therapy consists of?

Edit:
Actually, he doesn't say that talking to a friend is equally effective, only that it may be:

Because of increasing pressure from mainstream medicine and society to adopt evidence-based practices, psychology has begun to test its own assumptions and therapies. But by opening their assumptions up to critical evaluation, psychologists are discovering there is no evidentiary basis for asserting the superiority of one treatment over another. This in turn allows the explanation that psychological treatments may be demonstrations of the Placebo Effect24, not of targeted, unique therapies of proven efficacy, and people may derive the same benefit from conversations with a sympathetic friend.

[smudge]

Have now tracked down the meta-analysis, as expected the results do not support the conclusions drawn.

Thanks Pebble.
Very interesting but not surprising.

Jules; I think that answers your question regarding bias.

I think we'd all agree more evidence would be useful in the area of mental health.
But the Lutus article is unhelpful crap.

[Jules]

But the Lutus article is unhelpful crap.

I disagree, and you are dismissing the entire article based on what, exactly?
The article may be written in a somewhat polemical style, but the points are valid and there are no inaccuracies, although there may be some disagreement over interpretation; hardly surprising in this field where there are so many controversial issues.

Anyway, for those interested in more data, this looks helpful.

[Pebble]

Ok, thanks. I have let Paul know about this. Maybe he HAS overstepped the mark. If so, I hope he is big enough to amend the article.

Do you have any info on what a placebo therapy consists of?

Edit:
Actually, he doesn't say that talking to a friend is equally effective, only that it may be:

"But by opening their assumptions up to critical evaluation, psychologists are discovering there is no evidentiary basis for asserting the superiority of one treatment over another. This in turn allows the explanation that psychological treatments may be demonstrations of the Placebo Effect24,"


How should one interpret this statement?
Of course once one accepts the need for properly conducted trials, the first observation is that the trials have not been done - so there is no evidence! Having accepted that trials are necessary and setting them up then next observation is that your initial trials are flawed, too small and give conflicting results. Next you note that your trials are now regarded as proof of inefficacy by those who disagree with your theories, even before there is sufficient evidence to come to any conclusion. Fourth by the nature of trial construction, one approach is compared with another, if no therapy can be shown to be superior to any other, then it is reasonable to ask if any are better than nothing - that is not evidence that they are no better than nothing, just a reasonable question.

But as I have pointed out it would be premature to assert that non-inferiority has been demonatrated among all psychological therapies.

So what do we really know? Most of the theories that have been promoted over the past century have not been rigorously tested, the suggested therapies (talking therapies) have not in the main undergone large scale adequately powered, rigorously constructed trials and therefore no specific therapy can assert clear evidence of superiority. There is however substantial (if somewhat imperfect evidence) that one to one therapies are superior to group therapies, and that professionally administered therapies are superior to less well trained therapists.
Could all this still be a placebo effect? Of course - creating a placebo one to one session would be difficult - but not impossible. For example prayer therapy or spiritual healing done one to one might be accepted as purely placebo, but potentially believed in both by the administrator and recipient. But remember the trial size must be large enough, and the construction flawless, (so very expensive) to prevent the results being rejected by the losing side.

[smudge]

I disagree, and you are dismissing the entire article based on what, exactly?

Lack of balance. Mangling terms. Tabloid headline. Assertions not backed up by evidence or decent argument.

Pebble gives a far more accurate view of the situation regarding talking therapies in post number 36.

[Jules]

Conclusion CT is superior to talking to a friend. The studies comparing it to other 'psychological' therapies are too small to reach valid conclusions.

Can't see how I missed it, but this is what I received from Paul by email:

First, if the statement is taken at face value -- "The studies
comparing it to other 'psychological' therapies are too small to reach
valid conclusions." -- that must necessarily exclude the conclusion
(that CBT is superior to talking to a friend) as well. It takes a
particular kind of scientific sensibility to compose a paragraph that
explicitly refutes its own conclusion.

and further:

Second, the statement that the meta-analysis is not sufficient to
support its conclusions is a subjective claim by an individual, not one
a statistician would reach by reading the original work. Were this not
true, the analysis wouldn't have created the stir that it did.

I emphasize that all such studies are open to dispute. This is what
separates psychiatry and psychology from the non-soft sciences. But the
study's probability of being valid is about equal to that of the studies
it criticizes, something the commenter failed to note.

This is not the only study drawing that same conclusion, by the way.
There have been many studies meant to compare therapeutic methods, and
they reliably favor the methods preferred by the researchers, something
that should give any scientist pause.

The above systematic bias explains why a meta-analysis tends to cancel
out various biases in favor of particular therapeutic methods, and end
up supporting either the null hypothesis or the claim that speaking with
a sympathetic friend is a reasonable option.

Here is a reference that makes the same point, based on multiple arguments:

http://en.wikipedia.org/wiki/Dodo_bird_verdict

[Jules]

Could all this still be a placebo effect? Of course - creating a placebo one to one session would be difficult -

You're not kidding.

For example prayer therapy or spiritual healing done one to one might be accepted as purely placebo, but potentially believed in both by the administrator and recipient.

Now that would be quite something.

Since the placebo effect and psychotherapy itself are both based on psychological functions, one may wonder whether there is in fact any difference between them at all. In fact there is strong evidence that they are, for all intents and purposes, indistinguishable. Ironically, perhaps this validates psychotherapy after all.

[Pebble]

Can't see how I missed it, but this is what I received from Paul by email:

and further:

Sophistry, not worthy of anyone dispassionate. In context I had specifically distinguished between that which was taken by the authors of the meta-anlaysis as bona-fida psychological interventions, and contrasted them with therapies which though psychological were treated as non-bona fida and equated by said authors to placebo therapy. If Paul had read the original paper rahter than simply accepted the abstract as supporting his own bias this point would have been abundantly clear to him.

Unlike him however, I do go one stage further in the analysis, pointing out that even these supposed placebo therapies might be deficient and the one to one component delviered by a zealot may mean that these supposed palecbo therapies may not be suffieient to prove the point in respect of superiority to placebo. They do however, unquestionably prove superiority to 'talking to a friend' which though not what is stated in his article, this is clearly implied to be so and that is why you had taken this message from his diatribe.

As to the deficits in the meta-analysis being just my opinion - bollox. If he consulted the Cochrane database as I had suggested he would have known that restricting the analysis to pre 1994 is predjudicial. Second it is nonsense to suggest that since something created a stink it must have merit - as evidenced by the measles vaccine furore within the last decade. Thirdly the issue of retrospecitvely excluding one study because it did not support your initial contention has been noted by others (from pubmed, this is identified under 'comment in - Evid Based Ment Health' which he might have taken the trouble to look up. Finally, meta-analysis are also subjective undertakings, hence the conflicting conclusions reached in serial meta-analyses in all field, and the requirement that when analysing data NICE & Cochrane redo the meta-analyses themselves to exclude the bias of the previous authors.

The amuzing thing in all of this is his inability to see that I agree with the general gist of his article, but profoundly disagree with the slap dash presentation giving false impressions to the reader, simply to create a stink not supported by the available evidence.

[Pebble]

Second, the statement that the meta-analysis is not sufficient to
support its conclusions is a subjective claim by an individual, not one
a statistician would reach by reading the original work. Were this not
true, the analysis wouldn't have created the stir that it did.

This is a ridiculous statement - it implies that all journals check for statistical rigour of all studies accepted for publication.

Percept Mot Skills. 2003 Dec;97(3 Pt 2):1085-8.
Problems in synthesis (meta-analytic) studies: an example from the communication disorders literature.
Meline T.

The University of Texas-Pan American, Communication Sciences and Disorders, 1201 West University Drive, Edinburg, TX 78541-2999, USA. TM2776@AOL.COM
Abstract
This research note examined contemporary issues for meta-analysis with an example from the communication disorders literature. A significant proportion of experimental and quasi-experimental research results are not published in the mainstream literature, which suggests that publication bias is a potential problem for synthesis studies. Nonetheless, 50% of synthesis studies (meta-analyses) in communication disorders do not include statistical or visual procedures, e.g., fail-safe Ns or funnel plots, to reinforce the integrity of results. Tests for publication bias are recommended as routine procedures for meta-analyses.

I emphasize that all such studies are open to dispute. This is what
separates psychiatry and psychology from the non-soft sciences. But the
study's probability of being valid is about equal to that of the studies
it criticizes, something the commenter failed to note.

This is a shot in the dark. Sure having read neither study, such a generalisation is possible. However, I have read the meta(re)-analysis and it has been performed incompetently. The study it excluded may have also been performed poorly, but so may the studies it retained - but without examining each of them objectively this unlike the problem with the meta(re) analysis is an unknown.
The problem as I have clearly identified is the decision to exclude one of the studies despite the fact that the method of choosing studies specifically required inclusion of this study.
Had they apriori agreed to perform a funnel plot and exclude outliers, then if this stduy was the only one meeting those criteria, its exlcusion would be justified.

This is not the only study drawing that same conclusion, by the way.
There have been many studies meant to compare therapeutic methods, and
they reliably favor the methods preferred by the researchers, something
that should give any scientist pause.

So research is biased - welcome to the real world. That is the reason for peer review and indepedent validation of findings.

The above systematic bias explains why a meta-analysis tends to cancel
out various biases in favor of particular therapeutic methods, and end
up supporting either the null hypothesis or the claim that speaking with
a sympathetic friend is a reasonable option.
:

But here is the nub. This meta(re) analysis was undertaken to refute the findings of previous meta-analyses. Proving my point that met-analysis are themselves subjective undertakings - and really have no business being accepted as class A evidence.

Cochrane Database Syst Rev. 2007 Apr 18;(2):MR000010.
Grey literature in meta-analyses of randomized trials of health care interventions.
Hopewell S, McDonald S, Clarke M, Egger M.

UK Cochrane Centre, NHS R&D Programme, Summertown Pavilion, Middle Way, Oxford, Oxfordshire, UK, OX2 7LG. shopewell@cochrane.co.uk
Abstract
BACKGROUND: The inclusion of grey literature (i.e. literature that has not been formally published) in systematic reviews may help to overcome some of the problems of publication bias, which can arise due to the selective availability of data.

OBJECTIVES: To review systematically research studies, which have investigated the impact of grey literature in meta-analyses of randomized trials of health care interventions.

SEARCH STRATEGY: We searched the Cochrane Methodology Register (The Cochrane Library Issue 3, 2005), MEDLINE (1966 to 20 May 2005), the Science Citation Index (June 2005) and contacted researchers who may have carried out relevant studies.

SELECTION CRITERIA: A study was considered eligible for this review if it compared the effect of the inclusion and exclusion of grey literature on the results of a cohort of meta-analyses of randomized trials.

DATA COLLECTION AND ANALYSIS: Data were extracted from each report independently by two reviewers. The main outcome measure was an estimate of the impact of trials from the grey literature on the pooled effect estimates of the meta-analyses. Information was also collected on the area of health care, the number of meta-analyses, the number of trials, the number of trial participants, the year of publication of the trials, the language and country of publication of the trials, the number and type of grey and published literature, and methodological quality.

MAIN RESULTS: Five studies met the inclusion criteria. All five studies showed that published trials showed an overall greater treatment effect than grey trials. This difference was statistically significant in one of the five studies. Data could be combined for three of the five studies. This showed that, on average, published trials showed a 9% greater treatment effect than grey trials (ratio of odds ratios for grey versus published trials 1.09; 95% CI 1.03-1.16). Overall there were more published trials included in the meta-analyses than grey trials (median 224 (IQR 108-365) versus 45(IQR 40-102)). Published trials had more participants on average. The most common types of grey literature were abstracts (55%) and unpublished data (30%). There is limited evidence to show whether grey trials are of poorer methodological quality than published trials.

AUTHORS' CONCLUSIONS: This review shows that published trials tend to be larger and show an overall greater treatment effect than grey trials. This has important implications for reviewers who need to ensure they identify grey trials, in order to minimise the risk of introducing bias into their review.

[Jules]

But here is the nub. This meta(re) analysis was undertaken to refute the findings of previous meta-analyses. Proving my point that met-analysis are themselves subjective undertakings - and really have no business being accepted as class A evidence.

And yet in reply #32 you assert, on the basis of the meta-analysis, that:

1. CT was significantly superior to 'placebo' therapy.
2. CT is superior to talking to a friend.

We could go on and on with this, but let's back up a little and return to basics. In reply #36, you say (your response in bold):

"But by opening their assumptions up to critical evaluation, psychologists are discovering there is no evidentiary basis for asserting the superiority of one treatment over another. This in turn allows the explanation that psychological treatments may be demonstrations of the Placebo Effect,"

How should one interpret this statement?
Of course once one accepts the need for properly conducted trials, the first observation is that the trials have not been done - so there is no evidence!

But what about the null hypothesis? You seem to be suggesting that the efficacy of psychotherapy should be given the benefit of the doubt until it's been proven false. That isn't science!

[Pebble]

And yet in reply #32 you assert, on the basis of the meta-analysis, that:

1. CT was significantly superior to 'placebo' therapy.
2. CT is superior to talking to a friend.

We could go on and on with this, but let's back up a little and return to basics. In reply #36, you say (your response in bold):

But what about the null hypothesis? You seem to be suggesting that the efficacy of psychotherapy should be given the benefit of the doubt until it's been proven false. That isn't science!

The issue here is what is an adequate placebo.

The trials are large enough and sufficiently robust to demonstrate that CBT is superior to some forms of therapy. The question then arises as to the mechanism of benefit. If this is still within the realms of placebo benefit, then it may have nothing to do with the supposed active ingredients of the administered therapy.

Thus it is supposed that the nature of the placebo effect is largely the conviction of the care giver and the recipient that something of value is being done. It is clear that the placebo beneift of a surgical procedure is substantially greater than tablet administration - hence where an intervention is compared to medical treatment, the intervention is found to be sureprior, but when compared to a sham procedure, the effect may vanish - as for example with PFO closure for migraine or medical therapy for angina compared with intramyocardial implantation of the mammary artery.

It is believed but not proven that much of the apparent benefit with CAM is the greater 'care' and 'interest' shown by CAM practitioners when compared with for example GP administered treatments. So it is logical to question whether there are different levels of placebo.

In the bulk of meta-analysis of CBT, it is found to be superior to comparators, but many of the comparators could be regarded as having less placebo benefit (group therapy, relaxation therapy etc) so the effect could be largely due to to the one to one relationship developed by the therapist and the patient reinforced by the conviction on both sides that something effective is being done. This has neither been proven nor disproven adequately. It is however, obviously distinct and of different efficacy to 'talking to a friend' who clearly cannot reliably produce the level of conviction that would be comparable.

If you wish to contend that talking to a friend is superior to group psychotherapy, then the shoe is on the other foot - you now need to start providing the evidence for your conviction of superiority as opposed to non-inferiority. Just to be clear, what you are asserting is that group psychotherapy or relaxtion therapy are demonstrably less effective than talking to a friend.

[Jules]

From an abstract "What is the placebo in psychotherapy?":

The placebo effect is an important component and perhaps the entire basis for the existence, popularly, and effectiveness of numerous methods of psychotherapy." It perhaps should be noted here that the placebo as an inert substance does not exist in psychotherapy. All the variables in the psychotherapy relationship are psychological and
all are active, having some direct or specific effects on the client or patient.

It has been suggested that:

the placebo effect constitutes the most parsimonious explanation that would account for the apparently equal success achieved by each of the diverse collection of therapies practiced.

And:

Wolpe, (1973, p. 9) for example, claims that his method of reciprocal inhibition, as well as other behavioristic techniques, increase the improvement rate over that of the relationship alone, stating that "the procedures of behavior therapy have effects additional to those relational effects that are common to all forms of psychotherapy." Such claims have been disputed, and do not seem to be supported; indeed, it appears that many, if not most, of the specific techniques in the various approaches to psychotherapy, including behavior therapy, operate through the placebo effect--that is they are themselves placebos.

In the summary, it says:

There are other variables in the psychotherapy relationship which have received considerable support from extensive research not involving analogue situations. Three of these variables are empathic understanding, respect or warmth, and therapeutic genuineness. It is proposed that these are specific conditions for certain desirable outcomes in counseling or psychotherapy.

In other words, although the placebo elements aren't so likely to be present in the relationship between a "client" and his/her friend (although they may be!) as opposed to a client-therapist, those qualities I highlighted will be, so it seems plausible, to say the least, that "talking to a sympathetic friend" could be just as effective as psychotherapy.

[smudge]

those qualities I highlighted will be.

Jules, that is a massive assumption.
I made the point earlier in the thread.
You cannot assume all 'friends' are equally selfless, empathic, honest, respectful or genuine. You cannot expect everyone to be able to choose their friends equally well. You cannot assume those most in need of perfect, empathic, genuine friends actually have any friends at all.