Original Article| Volume 15, ISSUE 7, P1170-1176, July 2020

Ok
• PDF [497 KB]PDF [497 KB]
• Top

# Evidence Strength of Pharmaceutical Industry-Funded Clinical Trials in Metastatic NSCLC: A Comparison With Other Sources of Funding

Open ArchivePublished:March 23, 2020

## Abstract

### Introduction

Clinical trials are expensive and often require funding from the pharmaceutical industry (PI). We aimed to compare studies funded by the PI with those funded by other sources in terms of costs, reported results, and strength of evidence.

### Methods

We searched PubMed for clinical trial reports on metastatic NSCLC published between 2012 and 2017. We divided all the studies into two groups: studies funded by the PI and those funded by other sources. The primary end point was to compare the evidence strength of each group. The secondary end points were to compare the number of patients included, the number and costs of innovative drugs studied, whether there was preferential reporting of positive results in the experimental arm, and the risk of bias.

### Results

We found 3004 studies, and of these, we analyzed 477 studies (275 sponsored by the PI and 202 funded by other sources). A total of 85,328 patients overall were included (64,434 in studies sponsored by the PI and 20,894 in studies with other funding sources; p < 0.001). The studies funded by the PI had stronger evidence (p < 0.001), evaluated more innovative therapies (72% versus 36%; p < 0.001), and resulted in a higher proportion of open-access manuscripts (63% versus 47%; p < 0.001). There was no considerable difference regarding the reporting of experimental arm superiority or the risk of bias between the two groups.

### Conclusions

Compared with studies from other sources of funding, those funded by the PI in the lung cancer field collected stronger evidence, assessed more expensive and innovative therapies, and seemed to equally emphasize positive and negative results.

## Introduction

The practice of evidence-based medicine relies on having the best available information to make better clinical decisions and produce improved patient outcomes.
• Sackett D.L.
• Rosenberg W.M.
• Gray J.A.
• Haynes R.B.
• Richardson W.S.
Evidence based medicine: what it is and what it isn’t.
Stronger levels of evidence usually arise from well-designed, large, randomized clinical trials (RCTs), which are very expensive and often only affordable by the pharmaceutical industry (PI).
• Mullard A.
How much do phase III trials cost?.
There are several sources of bias that can contaminate scientific reports from RCT, such as publication bias,
• Dwan K.
• Gamble C.
• Williamson P.R.
• Kirkham J.J.
Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review.
,
• Djulbegovic B.
• Lacevic M.
• Cantor A.
• et al.
The uncertainty principle and industry-sponsored research.
violation of the uncertainty principle in the design of the study, and financial interest in specific study outcomes.
• Lexchin J.
• Bero L.A.
• Djulbegovic B.
• Clark O.
Pharmaceutical industry sponsorship and research outcome and quality: systematic review.
These biases may specifically affect the reporting of PI-sponsored RCT results because of the financial interests vested in such studies.
• Lexchin J.
• Bero L.A.
• Djulbegovic B.
• Clark O.
Pharmaceutical industry sponsorship and research outcome and quality: systematic review.
,
• Smith R.
Medical journals are an extension of the marketing arm of pharmaceutical companies.
We aimed to determine whether stronger evidence in the lung cancer area would come from the pharmaceutical company–sponsored RCTs than those sponsored with other sources of funding. As a secondary end point, we aimed to evaluate whether there would be an emphasis in the reporting and dissemination of positive results over negative ones among studies sponsored by the PI. Other secondary end points were to compare the number of patients included and the number and costs of innovative drugs studied in reports from each type of funding source.

## Materials and Methods

### Literature Review

We searched the PubMed database for reports published between November 2012 and November 2017 using free text filters and Medical Subject Heading terms, such as “nonsmall cell lung cancer,” “clinical trial,” “advanced,” “metastatic,” and “randomized.” In addition, we filtered the results to clinical trial articles.
We assessed the following data from each study retrieved in the search: (1) general information, including the title, and (2) design and methods, including the type of study and interventions.

### Study Selection

Two reviewers independently evaluated every study retrieved for inclusion. The inclusion criteria were as follows: (1) the included studies were clinical trials with at least one experimental treatment (even if the experimental treatment was previously approved for NSCLC) and (2) all the trials should have included only patients with advanced or metastatic NSCLC.
The authors used the Preferred Reporting Items for Systematic Reviews and Meta-Analysis guidelines for study selection. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis results are available in the Supplementary Figure 1.

### Data Extraction

We extracted the following information from individual studies: source of funding, level of evidence, registration at ClinicalTrials.gov, number of patients included, line of treatment assessed, presence or absence of a biomarker as a trigger for the experimental treatment, innovation of the experimental treatment, monthly cost of the experimental treatment, superiority of the experimental treatment over the control arm, and open-access availability.
The authors defined an innovative treatment as a drug that was not previously approved for lung cancer.
We interpreted the results of the meta-analysis in terms of the findings considering the risk of bias defined in the Cochrane Handbook for Systematic Reviews of Interventions.
We assessed the risk of the following biases: randomization, allocation, performance, detection, attrition, reporting, and others. The authors of this review did not request individual patient data.
We estimated treatment costs considering an average weight (70 kg) and body surface area (1.8 m2). We considered standard costs per milligram available on the up-to-date website (www.uptodate.com).

### Outcome Definition

We assessed the evidence level as follows: (1) RCTs, (2) prospective single-arm studies, and (3) case-control studies.

### Statistical Analysis

The authors divided all the included studies into the following two groups: studies funded by the PI and studies funded by other sources. The primary end point was to compare the evidence strength of each group. The secondary end points were to compare the number of patients included and the number and costs of innovative drugs studied by each group of studies. We used the Mann-Whitney test (when comparing two groups) or the Kruskal-Wallis test (when comparing four subgroups) to compare ordinal or nonparametric continuous data and the Fisher’s exact test to compare nominal data. In addition, we estimated odds ratios in logistic regression models. In this review, p values less than 0.050 were considered statistically significant.
The number and costs of innovative drugs and the presence or absence of a biomarker as a trigger for the experimental treatment were assessed for other subgroups, such as studies funded by the cooperative groups, foundations, or single institutions. Finally, we assessed the proportion of studies funded by the PI and the mean monthly cost of studied treatments over time.

## Results

### Studies Inclusion and Exclusion

We identified 3004 entries from the initial database search. The reviewers excluded 2527 studies after the first analysis, primarily because they were not clinical trials, included other primary sites, included patients with nonmetastatic NSCLC, or were only a report of ongoing trials. We included 477 studies (275 sponsored by the PI and 202 funded by other sources) with 85,328 patients enrolled (64,434 in studies sponsored by the PI and 20,894 in studies with other funding sources; p < 0.001). Table 1 summarizes the study characteristics.
Table 1Study Characteristics
Yes 275 (100%)No 202 (100%)
Number of patients64,43420,894<0.001NA
Mean number of patients231103
Mean monthly cost16,0729330<0.001NA
Evidence level
Randomized clinical trial152 (55%)67 (33%)0.0311.67 (1.05–2.66)
Prospective single-arm122 (44%)126 (62%)
Case-control1 (1%)9 (5%)
Line
First157 (57%)120 (59%)0.2540.75 (0.46–1.23)
Second and further118 (43%)82 (41%)
Biomarker
Yes70 (25%)46 (23%)0.7921.07 (0.67–1.50)
No204 (75%)156 (77%)
Innovative treatment
Yes199 (72%)73 (36%)<0.0012.81 (1.73–4.56)
No76 (28%)129 (64%)
Experimental superiority
Yes67 (44%)31 (46%)0.7640.90 (0.44–1.83)
No85 (56%)37 (54%)
Open-access article
Yes174 (63%)95 (47%)0.0041.98 (1.24–3.14)
No101 (37%)107 (53%)
Boldface indicates parameters with statistically significant differences between both arms.
CI, confidence interval; OR, odds ratio; NA, not applicable.

### Evidence Strength

Compared with studies funded by other sources, those funded by the PI had stronger evidence (RCT, 55% versus 44%; single-arm, 33% versus 62%; and retrospective, 1% versus 5%; p = 0.031).

### Secondary End Points

Studies sponsored by the PI enrolled a higher number of patients on average (mean 231 versus 103; p < 0.001), evaluated more innovative treatments (72% versus 36%; p < 0.001), and evaluated more expensive treatments (mean monthly cost $16,072 versus$9330; p < 0.001). Studies funded by the PI were also more available for open access than those funded by other sources (63% versus 47%; p = 0.004).
There were no differences among studies funded by the PI or those funded by other sources in terms of the line of treatment assessed, presence of trigger biomarkers in the study, or experimental arm superiority in the clinical trial.

### Analysis of Studies Funded by Other Sources

The evidence of studies funded by the PI remains the strongest even when we assessed the subgroups of other sources of funding, such as cooperative groups, foundations, and single institutions (Fig. 1). The weakest evidence level was among studies funded by authors or single institutions.
Studies funded by the PI had not only the strongest evidence level but also the highest proportion of innovative treatments and the highest costs of assessed treatments (Table 2).
Table 2Study Characteristics by Subgroups of Funding
IndustryCooperativeFoundationAuthors’ Center
Mean monthly cost16,072.1410,440.6111,048.688556.36<0.001
Biomarker
Yes70 (25%)8 (26%)12 (29%)26 (20%)0.596
No204 (75%)23 (74%)30 (71%)103 (80%)
Innovative treatment
Yes199 (72%)9 (29%)24 (57%)40 (31%)<0.001
No76 (28%)22 (71%)18 (43%)89 (69%)
Boldface indicates parameters with statistically significant differences between both arms.

### ClinicalTrials.gov Registration and Risk of Bias Assessment

Most of the studies sponsored by the PI were registered in the ClinicalTrials.gov (78%), whereas only 37% of those funded by other sources had the same register (p < 0.001).
In terms of the risk of bias, both groups had similar results for almost all analyses, except for performance bias (Fig. 2). The performance bias assesses the double-blinded design of studies. The proportion of studies classified with low risk of performance bias was 31% of studies funded by the PI versus 8% of studies funded by other sources (p < 0.001). The proportion of studies classified as having unknown and high risks were 18% and 52% for studies funded by the PI and 58% and 34% in those funded by other sources, respectively.
There was no considerable difference regarding the reporting of outcomes (reporting bias). Almost 90% of all studies was classified as having a low risk of reporting bias in both groups (p = 0.829).

### Analysis Over Time

During the 5 years considered in this review, the proportion of studies funded by the PI increased from 49% in 2012 to 65% in 2017 (Fig. 3A). Consequently, the proportion of studies assessing an innovative treatment also increased (from 42% in 2012 to 69% in 2017; Fig. 3B).
The mean monthly cost of treatments assessed also increased over time, although the difference was not statistically significant (p = 0.229; Fig. 4).

## Discussion

Clinical trials are the basis for the practice of evidence-based medicine.
• Sackett D.L.
• Rosenberg W.M.
• Gray J.A.
• Haynes R.B.
• Richardson W.S.
Evidence based medicine: what it is and what it isn’t.
Unfortunately, RCTs are expensive because of the extensive resources required to conduct them and the long time to find their results.
• Mullard A.
How much do phase III trials cost?.
The funding for these studies, therefore, becomes a crucial step in their execution. In fact, the usual amount of money to fund large RCTs are many times beyond the reach of cooperative groups and research institutions, requiring the participation of the PI. As an example, in the United States, the amount of money available for the National Institutes of Health to invest in 1 year is approximately 39 billion, whereas the amount spent by the PI per year is approximately 100 billion dollars. It is thus conceivable that industry interests may determine the types of research questions to be answered by RCTs and their experimental design. Previous studies also suggest that even the results of some trials may be affected by industry-vested interests through several types of systematic errors, such as reporting bias
• Dwan K.
• Gamble C.
• Williamson P.R.
• Kirkham J.J.
Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review.
and violation of the uncertainty principle.
• Djulbegovic B.
• Lacevic M.
• Cantor A.
• et al.
The uncertainty principle and industry-sponsored research.
The emphasis in the reporting and dissemination of positive results may favor a higher rate of reporting in comparison with negative ones.
• Dwan K.
• Gamble C.
• Williamson P.R.
• Kirkham J.J.
Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review.
Furthermore, to obtain results favoring a certain type of drug or device, the uncertainty principle, which presupposes an equipoise of outcomes of all experimental arms before the conduction of an RCT, is often violated.
• Djulbegovic B.
• Lacevic M.
• Cantor A.
• et al.
The uncertainty principle and industry-sponsored research.
In these cases, experimental arms of RCTs may be spuriously favored by using knowingly weaker comparator arms. In the lung cancer field, development of newer therapeutic strategies and introduction of new drugs are increasing; therefore, we wanted to determine whether PI-funded RCTs would differ significantly from studies funded by other sources. We evaluated specifically if there were differences in the strength of evidence and the costs of new drugs employed, number of patients enrolled, and the reporting of positive results in the experimental arms.
As expected, PI-sponsored studies included more patients, tested drugs that are more expensive, and produced results supported by higher levels of evidence. Funding of clinical studies is responsible for a significant part of the costs involved in drug development by the PI. The higher availability of financial resources for the development of new drugs may explain why PI-funded studies can include more patients and therefore generate higher levels of evidence. Likewise, larger availability of financial resources can explain the higher costs of the drugs that can be tested in PI-sponsored RCTs than those with other sources of funding. Furthermore, PI-funded trials may be published more easily in free-access journals, which charge fees to publish accepted articles. Free-access journals can increase the visibility of these trials by freely providing readers with the full version of these published articles.
Bhandari et al.
• Bhandari M.
• Busse J.W.
• Jackowski D.
• et al.
Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials.
and Lexchin et al.
• Lexchin J.
• Bero L.A.
• Djulbegovic B.
• Clark O.
Pharmaceutical industry sponsorship and research outcome and quality: systematic review.
reported that pharmaceutical company-sponsored studies tended to report better results for the experimental arms.
Interestingly, in our study, we noted no such finding. Herein, the results of PI-sponsored trials did not favor the experimental arms more than those with other sources of funding and the reporting bias was considered low for almost 90% of all the included studies. In the context of studying lung cancer, which often has unequivocal end points such as overall survival, it may be difficult to report results inaccurately. In addition, we considered that a high proportion of studies registered in ClinicalTrials.gov helps to improve reporting transparency.
We conclude that studies funded by the PI had stronger evidence, tested more innovative therapies, were more accessible to the readers, and did not have significantly higher risk of bias or report higher experimental arm superiority than those developed with other sources of funding.

## Acknowledgments

This study was funded by the authors.

## References

• Sackett D.L.
• Rosenberg W.M.
• Gray J.A.
• Haynes R.B.
• Richardson W.S.
Evidence based medicine: what it is and what it isn’t.
BMJ. 1996; 312: 71-72
• Mullard A.
How much do phase III trials cost?.
Nat Rev Drug Discov. 2018; 17: 777
• Dwan K.
• Gamble C.
• Williamson P.R.
• Kirkham J.J.
Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias—an updated review.
PLoS One. 2013; 8e66844
• Djulbegovic B.
• Lacevic M.
• Cantor A.
• et al.
The uncertainty principle and industry-sponsored research.
Lancet. 2000; 356: 635-638
• Lexchin J.
• Bero L.A.
• Djulbegovic B.
• Clark O.
Pharmaceutical industry sponsorship and research outcome and quality: systematic review.
BMJ. 2003; 326: 1167-1170
• Smith R.
Medical journals are an extension of the marketing arm of pharmaceutical companies.
PLoS Med. 2005; 2: e138
1. Higgins J.P.T. Green S. Cochrane Handbook for Systematic Reviews of Interventions. Wiley, Hoboken, NJ2008
• National Institutes of Health
Budget.
(Accessed June 23, 2019)
• ABPI
Worldwide Pharmaceutical Company R&D Expenditure by Country.
(Accessed June 23, 2019)
• Bhandari M.
• Busse J.W.
• Jackowski D.
• et al.
Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials.
CMAJ. 2004; 170: 477-480