Accumulating Evidence: When Publication Bias Persists
Author(s)
Arthur Gougeon1, Audrey Lajoinie, PharmD, PhD2.
1Lyon, France, 2RCTs, Lyon, France.
1Lyon, France, 2RCTs, Lyon, France.
OBJECTIVES: The first study investigating the association between bisphosphonates (BP) and osteonecrosis of the jaw (ONJ) was published in 2006, and a recent meta-epidemiologic analysis (2024) identified the presence of publication bias. However, its temporal evolution has not been evaluated. The aim of this study was to investigate the persistence and magnitude of publication bias over time.
METHODS: A systematic review was conducted to identify studies assessing the BP-ONJ association. A cumulative year-by-year meta-analysis was performed to estimate the crude odds ratio (OR) over time. Publication bias was evaluated using Egger’s test and visual inspection of funnel plots. If bias was detected, the trim-and-fill method was applied to obtain an adjusted OR for publication bias. The impact of publication bias was quantified using the ratio of odds ratios (ROR) (adjusted OR / crude OR).
RESULTS: A total of 44 studies assessing the association between bisphosphonates and osteonecrosis of the jaw (ONJ) between 2006 and 2025 were collected. Publication bias was detected from 2008 and remained present until 2025, a period of 17 years. The ROR showed a gradual decrease in publication bias over time. In 2008, the ROR was 0.42, corresponding to a 58% overestimation of the BP-ONJ association. By 2025, the ROR had increased to 0.66, indicating a 24% overestimation (Figure 1). The magnitude of bias was higher in observational studies (40%) compared with randomized controlled trials (11%).
CONCLUSIONS: Despite the accumulation of studies over time, publication bias persisted for 17 years, suggesting that a substantial proportion of studies remain unpublished. Publication bias continues to distort the estimated association between BP and ONJ, potentially leading to a flawed benefit-risk assessment. These findings highlight the need for stronger measures, such as the systematic inclusion of observational studies in research registries, to promote transparency and ensure a more accurate assessment of drug safety.
METHODS: A systematic review was conducted to identify studies assessing the BP-ONJ association. A cumulative year-by-year meta-analysis was performed to estimate the crude odds ratio (OR) over time. Publication bias was evaluated using Egger’s test and visual inspection of funnel plots. If bias was detected, the trim-and-fill method was applied to obtain an adjusted OR for publication bias. The impact of publication bias was quantified using the ratio of odds ratios (ROR) (adjusted OR / crude OR).
RESULTS: A total of 44 studies assessing the association between bisphosphonates and osteonecrosis of the jaw (ONJ) between 2006 and 2025 were collected. Publication bias was detected from 2008 and remained present until 2025, a period of 17 years. The ROR showed a gradual decrease in publication bias over time. In 2008, the ROR was 0.42, corresponding to a 58% overestimation of the BP-ONJ association. By 2025, the ROR had increased to 0.66, indicating a 24% overestimation (Figure 1). The magnitude of bias was higher in observational studies (40%) compared with randomized controlled trials (11%).
CONCLUSIONS: Despite the accumulation of studies over time, publication bias persisted for 17 years, suggesting that a substantial proportion of studies remain unpublished. Publication bias continues to distort the estimated association between BP and ONJ, potentially leading to a flawed benefit-risk assessment. These findings highlight the need for stronger measures, such as the systematic inclusion of observational studies in research registries, to promote transparency and ensure a more accurate assessment of drug safety.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
MSR13
Topic
Clinical Outcomes, Epidemiology & Public Health, Methodological & Statistical Research
Topic Subcategory
Confounding, Selection Bias Correction, Causal Inference
Disease
Diabetes/Endocrine/Metabolic Disorders (including obesity)