Bias Detection: Study

NIH clinical trials and publication bias

J Can Acad Child Adolesc Psychiatry. Feb 2014; 23(1): 7.

Abel Ickowicz, Co-Editor

Recently, a hallway conversation with an esteemed colleague caught my attention. Being pondered was the fact that relative to the literature on psychotherapy for adult persons, less information on the efficacy of these treatments for children and adolescents is available. The conversation turned quickly into an admission of difficulties encountered by authors when trying to publish negative results (i.e., no statistical difference between the active treatment and the control groups). Rather boldly, my response was: that the Journal of the Canadian Academy of Child and Adolescent Psychiatry is decidedly interested in publishing reports of negative clinical trials, as long as they are methodologically and ethically sound.

Critical examination of the scientific literature addressing therapeutic interventions – either medication or psychosocial treatments – reveals that the number of reports of clinical trials rendering positive results is considerably larger than the number of negative reports. The tendency to focus on studies that show significant effects of a particular treatment results in publication bias and constitutes a real threat to the validity of the evidence used to inform clinical practice.

Why are we so fond of positive results? Cultural pressures demand treatment innovation; our society has high expectations towards finding “new and improved cures” that will ideally heal or at least alleviate all our personal ailments. As a consequence, consciously or unconsciously, researchers and clinicians are inclined to show progress in drug discovery and advancement in non-pharmacologic therapies; implicit is the need for these new treatments to show superior efficacy and greater safety. However, when we think about publication bias, it is not unusual to focus exclusively on strategies used by the pharmaceutical industry, such as suppression of negative results, and production of papers produced by ghost writers. Arguably similar biases are encountered in psychotherapy research and knowledge dissemination. Pharmaceutical companies have profound economic incentives to make the medications they develop and eventually commercialize shine in the most affirmative light. Although psychotherapies and other psychosocial treatments are not supported by large economic interests, they are also at considerable risk of publication bias. In addition to the potential for modest financial gain, bias in psychotherapy research literature may be mediated by academic pressures (i.e., publish or perish), ideological allegiance to a treatment modality, as well as the potential for personal fame and recognition. An example on how publication bias might influence our perception of the benefit of psychotherapy for depression is found in a recent review by Cuijpers and colleagues. This study demonstrates that the active treatment’s mean effect size of 0.67 goes down to an effect size of 0.42 when adjusted for publication bias. This study also examined a subsample of studies focusing exclusively on cognitive behavioural therapy (CBT) and found similar effects for publication bias in this modality of treatment. The mechanisms resulting in bias in psychotherapeutic research often resemble those present on reports of clinical trials on medication; these include: comparing the preferred treatment against a less effective intervention; impartial selection of data in support of the favoured treatment; disregard for negative effects; and nonappearance of publications reporting lack of significant effects.

Call to step up the pace of TB-HIV collaborative activities  — Weekly Blitz
Discovery by biomedical research of new and improved interventions can get into the bucket of implementation.

Related Posts