In particular, an article by Prof Sander Greenland offers a fresh discussion based on perspectives from epidemiology, statistics, and cognitive psychology of the effects of financial conflicts of interest (COI) on (clinical, epidemiological, and health services) research (Greenland S. Accounting for uncertainty about investigator bias: disclosure is informative. J Epidemiol Community Health 2009; 63: 593-598. Link here, requires subscription.)
Since only subscribers can easily get this article, and because of its importance, let me summarize its main points and provide appropriate quotations.
Financial COI as Causing One Type of Investigator Bias
Greenland views financial conflicts of interest as a source of one kind of investigator bias, defined as "the biasing of study results towards results expected or desired a priori by the investigators." He believes that this is a particularly important kind of bias, "the pool of reported study results may be affected seriously by prejudices and vested interests of investigators and sponsors. Such incidents suggest that investigator bias can be larger than any other bias, and may often encourage adoption of dubious or even deadly treatments."
Focus on Effects of the Bias, Rather than Intent
Greenland posited that the issue here is not intent,
Investigator bias may be intentional or subconscious, may be of venal or noble origin, and may manifest in outright fraud or in subtle fallacies held true by the investigator. It may reflect no more than the influence of sincere wishes or beliefs of the investigator or their research community on the way findings are interpreted, or it may represent premeditated attempts to mislead.
The danger is that raising the issue of intent may lead to emotional arguments rather than sober thinking about what effect conflicts of interest should have on interpreting study results,
here we encounter an emotional minefield of indignation driven by fears of prejudice in and abuse of such priors [that is, beliefs about the effects of investigator bias due to conflicts of interest.] Similar objections meet proposals to down-weight studies that produce results favourable to sponsors or other parties with financial ties to the investigators.
Rather, the issue is that conflicts of interest may indicate a high risk that a study is biased, and obviously, biased in a certain direction, "we expect investigator bias, if present, to fall in the direction favouring their or their sponsor’s financial interest."
An Approach to Evaluating the Effects of Investigator Bias due to Financial COIs
Greenland pointed out that it may be difficult to account for the effects of investigator bias due to COIs by just being careful in the evaluation of the report of a given research study, "incomplete documentation of ad hoc design and analysis decisions makes investigator bias especially difficult to detect and deal with." Furthermore, "one cannot detect unreported biasing judgements without having the data and the time to reanalyse it."
Thus, the challenge is to get accurate information about COIs, and then use it to accurately evaluate a particular study. The realities of life in the often conflicted world of medical academia means that conflicts of interest may generate a large amount of bias. Greenland suggested that this
reflects a harsh reality. In research funded or conducted by parties with a direct financial stake in the results, there may be an implicit threat of funding withdrawal (or worse) if results presented do not favour the sponsor’s interest or conform to their perceived needs for marketing and litigation.
Furthermore, he noted,
Sponsored investigators have been sued by their corporate sponsors in efforts to block publication of undesirable results. Investigators with no corporate ties have faced interference from corporations upset with their findings. Not all investigators choose to fight such pressure, which leaves them the option to either not publish unfavourable results or else not produce such results. Some go so far as to guarantee their sponsor that the results of their proposed study will be helpful.10 Meanwhile, unfavourable results from research conducted within a company may languish as in-house memos that can be obtained only via litigation, and then only if the court orders their release.
However, information provided about COIs may not be complete or accurate.
Once we accept the reality that disclosure data contain relevant statistical information, we can turn to the quality of information ascertainable from available reports.
There are reports of investigators disclosing only a small fraction of their financial support from industry, even when required to give a full account by university and granting-agency rules. There is little reason to expect more thorough disclosure in journal articles. Full disclosure is often evaded via technicalities, such as payment of non-descript consulting fees that are not explicitly earmarked for the article and thus not deemed reportable, even when they may be for time spent on the database from which the article is generated. If reported at all, such funds may be described only by phrasing such as 'author X has served as a consultant (or expert) for company Y', not that the payments for such services were made with the understanding that they would support time spent on the study being reported.
The frequency of under-reporting and deceptive reporting is far from known. Most incidents come to light only when investigation or litigation occurs, and those discoveries are not always reported in public venues. Other devices for undermining disclosure requirements include funds channelled through innocuously titled foundations or institutes set up specifically to conceal the ultimate funding source. Detecting such events and devices may demand investigative effort beyond the resources of almost all readers.
However, while there may be considerable false negative information about COIs, false positive are unlikely, "In contrast, it is hard to imagine that many articles report funding from a financial stakeholder when in fact there was none, so that misreporting may be limited to under-reporting."
Conclusions and Future Directions
Greenland concluded that investigator bias is very important.
Investigator bias has the potential to overwhelm all other biases. It can become the dominant force in contexts (such as expert reports and testimony for litigation) in which the restraints imposed by editors and peer review are absent. Thus, because of its importance, investigator bias should not be dismissed as unapproachable, any more than we should give up research on what seem to be hopelessly mysterious diseases.
He reminded us that not all investigator bias is due to financial COIs, although that should not be used to deny the importance of bias due to financial COIs,
Although I have focused on distortion from financial input, I have no doubt that ideological commitment can be just as distortive. Indeed, dealing with ideological and other psychological sources of bias (such as commitment to earlier methods and conclusions) deserves to be on the research agenda for bias analysis.
He suggested some directions for the future,
In approaching these problems scientifically, I suspect aid will be needed from those branches of psychology that deal with distorted thinking among normal individuals and groups.
Openness of journals to unpopular research and strenuous debate remains the primary line of defence against information abuse and distortion. Such openness is not universal, and thus it seems likely that proposals to deal with investigator bias will raise a hornet’s nest of objections and questions. Perhaps the essential first step (if not major hurdle) is getting used to the idea that there may be good arguments for including the possibility of investigator bias in an uncertainty assessment. The demand for thorough financial disclosure may then seem no different from demanding hard ('objective') details of design and analysis to aid assessments. Like other psychological constructs, ideological biases will be far more difficult to evaluate but may nonetheless have hard correlates that signal their presence if not magnitude, such as attempts to suppress or boycott opposing views.
I highly recommend that those with access to this journal read Prof Greenland's article in full, and see the other related articles in this issue. In particular, Prof Greenland's article, and the article from the German chapter of Transparency International provide some clear thinking as a counter to the illogic that is now appearing as the backlash by the conflicted against the "pharmascolds" intensifies.