Navigating the Changing HEOR Publishing Landscape
An increasing number of stakeholders rely on health economics and outcomes research (HEOR) evidence to assess relative value, to weigh treatment choices, and to help them target possible patient outcomes under finite budgets. Clinicians, payers, patients, governments, health ministries, and other stakeholders are utilizing HEOR research to inform their understanding of the therapeutic and economic value of a given product in a real-world clinical practice environment and to help clarify their decision making. These parties need access to up-to-date HEOR information, such as real-world patient outcomes data, quality-of-life surveys, opportunity costs of various treatment mixes, budget impact studies, and cost-effectiveness models.
In this article, Johan Rooryck, PhD; Michael F. Drummond, MCom, DPhil; Rick Anderson, MLIS; and Richard White, PhD, shared their thoughts on changes to scholarly publishing, including preprint portals, open access, and their impact on the effective dissemination of HEOR.
The HEOR publishing environment is evolving to help meet the demand for more timely access to HEOR evidence with new journals, open access publications, and preprint servers. The explosion of preprint servers and different open access models have changed scientific publishing, recreating the standard for disseminating time-critical research and expanding access to researchers, practitioners, policy makers, patients, and the public. Coupled with newer HEOR journals and HEOR-cognizant reviewers, HEOR researchers face expanding options to share their findings.
Many policy makers are supporting these changes in scholarly publishing. While the United States still lags behind Europe and other countries, such as South Africa, in requiring open access to research derived from government-funded research, this may soon change. In United States, the Trump administration considered enacting an executive order in early 2020 that would mandate open access to federally funded research. As the critical importance of broad and timely access to scientific evidence was supported by the lightning speed of COVID vaccine development, the new Biden administration may soon implement a similar order despite continued pushback from the publishing community, further solidifying these important changes to scholarly publishing.
Changing a Centuries-Old Publishing System
For roughly 300 years, researchers have relied on publishers to disseminate literature through a rigorous peer-review process funded by subscriptions. While this model provides the structure to identify rigorous research, it also slows the dissemination of critical research and limits access to libraries, subscribers, and readers covered by a site license. To ensure free and ready access to timely research, scientific publishers are being pushed to change their publishing models from the traditional subscription model to models that promote “open science,” like preprint servers and open access publishing.
Preprints Expedite Research Dissemination
Preprints (see related article by Cohen, et al, in this issue) are the original versions of publications submitted for review. They may be accessed online by anyone at no cost before (and sometimes after) the work has been published. They are released without peer review, thus cutting months from time to dissemination. Preprints can also help researchers disseminate their findings quickly. Like the open access platforms, preprints have become a rapidly growing option, spurred in part by COVID-19–related materials.
Richard White, MA PhD, Chief Operating Officer, Oxford PharmaGenesis, Oxford, Oxfordshire, UK, argues that preprints could be an attractive option for some types of studies funded by the pharmaceutical industry, sometimes in lieu of a press release as the first point of data dissemination and as an antecedent to a manuscript. The flexibility of preprints may better accommodate the dynamic nature of HEOR. “I think they’re potentially valuable. The challenge with HEOR studies is that flex in the methodology and evolution of results.” He continued, “When you do a randomized controlled trial, it’s all very clear what the protocol is. If you’re doing an economic model, there’s some scope for latitude based on what inputs are actually available.” He notes how some data in economic models may be derived from expert opinion, which may be prone to changes, and many HEOR studies derive results from dynamic, real-world datasets. “There could be merit in preprints having that ability to shape the research a little bit as we’ve seen in the physical sciences.”
Michael Drummond, MCom, DPhil, Centre for Health Economics, York University, York, Yorkshire, UK (and Editor-in-Chief of Value in Health), supports preprints for promotion. However, he warns that he would not recommend the preprint route for junior academic colleagues. For younger academics, a robust publication record in highly reputable journals still drives tenure decisions; preprints provide limited value to these researchers.
Risks of Preprints and Predatory Journals
Rick Anderson, MLIS, University Librarian at Brigham Young University, Salt Lake City, UT, USA, and contributor to the Scholarly Kitchen, provided some critical insight into the challenges surrounding preprints, article processing charges, and open access publishing.
Given the lack of peer review, Anderson warns that preprints can be targeted by bad actors pushing poor research onto preprint servers to tout results as having been published. He noted this problem is often compounded by a credulous journalistic community that may not fully understand the subtleties of scholarly publishing, citing preprint findings as if they were from a peer-reviewed journal. The journal Nature has warned the scientific community that measures must be taken to keep preprints from distorting the public’s view of scientific research, highlighting the problematic dissemination of a (now retracted) paper reporting that genetically modified corn caused cancer in rats.1,2 In some cases, challenges may exceed poor scientific methodology and encroach on fraud, including false affiliations and fake authors.3 Anderson himself participated in sting operations, successfully publishing a fraudulent paper with fake authors, which is still online.4,5
Anderson warned that while preprints accelerate dissemination of results by bypassing the peer-review process, this increases the risk of faulty or erroneous results in the published record. However, he argued that preprints could also accelerate the filtering of bad data from the published record as preprints invite the entire research community to review work within hours of posting. These studies can be quickly critiqued by a wide audience and quickly removed if deemed not credible. For example, Cortegiani et al identified dozens of COVID-related papers retracted from preprint servers.6 Anderson proposes that organizations running preprint servers adopt a policy of actual retraction when posted articles are found to be fraudulent or fundamentally unsound rather than resorting to a simple banner warning. “I think if preprint servers would genuinely retract—which is to say flag and remove—fraudulent and fundamentally unsound science, it would go a long way towards helping to ameliorate the kinds of public health dangers that we’re seeing right now.” He continued, “It’s so important to get the science out there quickly and publicly, but on the other hand, there’s so much danger in putting bad science out quickly and publicly. All of the benefits are exactly mirrored by the dangers.”
“Preprint servers are a sword that cuts both ways,” he warned. “They are both a benefit and a danger to the public. They are both a benefit and a danger to pharma.”
"The lay public is very often educated enough to make up their own minds on the research that’s out there. Most of the time, it’s their tax dollars that are paying for this research. They should be the first to benefit." — Johan Rooryck, PhD
Anderson also warned of predatory journals, such as the Journal of Health Economics and Outcomes Research or the British Journal of Medical and Health Research. These predatory journals and dozens of others are included in both the Beall’s List of Potential Predatory Journals and Publishers and Stop Predatory Journals.7,8 In addition to being open access, these journals are invariably funded by article processing charges. According to Anderson, as long as a manuscript is accompanied by an article processing charge, these predatory journals will likely publish it as peer-reviewed science.
Anderson reiterated that by no means are all open access journals predatory. “There are people in the scholarly community who assume that anytime they’re asked to pay an article processing charge, that means it’s a predatory journal, which is completely false.” He continued, “Predatory journals came into existence exactly as a result of the article processing charge model, not as the result of open access generally.”
“The fundamental problem with these journals is not that they’re low quality; it’s that they’re misrepresenting what they do, defrauding authors and readers alike.”
Open Access
Even traditional journals are evolving to facilitate broader access to research findings by expanding open access to many publications. The term “open access” encompasses a wide mix of publishing models. All of these models provide broad access to published scholarly literature made freely available on the internet where users may read, download, copy, print, or search articles (although not necessarily for unrestricted reuse)—using them for any other lawful purpose, without financial, legal, or technical barriers.9,10 The copyright holder (usually the author) must consent in advance to let users “copy, use, distribute, transmit, and display the work publicly and to make and distribute derivative works, in any digital medium for any responsible purpose, subject to proper attribution of authorship.”
These models differ by where and when material is released, who bears the publication cost, and who retains copyright. The 2 most common open access models—gold and green—differ on every attribute. Table 1 compares 4 open access models.
Table 1. Comparison of Open Access Models.
Benefits of Open Access Models
Johan Rooryck, PhD, Executive Director of cOAlition S, (and Visiting Professor at Leiden University), Leiden, The Netherlands, the organization championing open access publishing models, views timely access to critical research as a key factor in expediting development of COVID-19 vaccines and treatments this past year. He views publishing embargoes (the time publishers keep research behind paywalls, limiting free access) as particularly burdensome. “If you have to wait 6 months or 12 months because of embargoes, that research is 6 or 12 months old. Imagine if COVID-19 vaccine research had been subject to a 6- or 12-month embargo,” said Rooryck.
Rooryck thinks timely access to new research would not only benefit researchers, but all stakeholders. It could provide practitioners with the up-to-date clinical data needed to improve treatment choices. “Once you open the Pandora’s box [to allow free access] for COVID-19 [research], why not do it for everything? Why not do it for deadly cancers?”
Wider Collaboration, Participation
Open access publishing models can also facilitate engagement of the scientific community in research initiatives. Citation rates of open access articles have been found to be 18% higher than those behind paywalls, although the extent of this citation advantage continues to be debated.11,12 Open access can facilitate further research, advance collaboration, and prevent duplicated research. And by reducing duplication of existing research, scientific progress accelerates.
Proponents of open access argue that publicly funded research should be available to the public. This is especially true for research participants. A recent survey of US-based study participants found that the overwhelming majority (91%) wanted to be informed of study results; two-thirds indicated that their participation in future trials would be predicated on receiving such feedback.13
And patients have voiced a desire to access research beyond that in which they were a participant. In the United States, nearly 60% of adults search for medical information within a given year, with 25% finding their search impeded by paywalls. Only 2% of these patients who encounter paywalls pay for further access; 83% search elsewhere, while 13% give up their search.14
Rooryck believes that not only could open access help educate patients about their treatment, it could also help ensure their participation in future clinical trials. “The lay public is very often educated enough to make up their own minds on the research that’s out there. We want those citizens to be informed,” said Rooryck. “Most of the time, it’s their tax dollars that are paying for this research. They should be the first to benefit. It’s not just about the researchers, it’s about the general audience as well.”
Recognizing Null, Inconclusive, or Validating Research
Highly selective journals seek to publish impactful, often newsworthy, research. Such selectivity often leads to riveting issues filled with thought-provoking and headline-grabbing research. However, Rooryck argues, there are significant concerns surrounding this approach. “Do you really want to say that the 90% of research that you reject should not be published?” he said. “While it is not necessarily front-page news for The New York Times, it may be good, solid research.”
This “good, solid research” may include null studies, validation studies, or inconclusive studies. These findings that are critical to scientific advancement. Richard White, PhD, Chief Operating Officer at Oxford PharmaGenesis, sees open access as providing an important venue for these less headline-grabbing studies. “I think there’s tremendous wastage out there when a negative study is just put in a drawer and someone else then gets a grant application funded, does the same study, and that could have been money saved and time saved for everyone.
“We need to go much more into a place where there is collaboration instead of competition, where there is validation instead of prestige—a place that is more tolerant of good research rather than just excellent research,” said Rooryck.
The open access models may also help change the perception of HEOR research as potentially biased or less rigorous. Drummond stressed that building trust in HEOR results is more pressing given that regulatory agencies like the US Food and Drug Association (FDA) and European Medicines Agency (EMA) require less evidence for approval as was required 10 years ago. “You have drugs approved on less mature clinical evidence. Health technology assessments are picking up the consequences (of less preapproval clinical evidence) of the FDA and EMA.” With greater transparency, particularly around real-world evidence, trust in HEOR data and the credibility of the research may improve.
"We need to go much more into a place where there is collaboration instead of competition, where there is validation instead of prestige—a place that is more tolerant of good research rather than just excellent research."— Johan Rooryck, PhD
However, Drummond argued that peer review would be critical to bolstering trust. “The industry is very concerned about their message being believable. Peer review is one kind of verification of that message.” He highlighted 2 ways to build credibility—getting published in a highly regarded, peer-reviewed journal or getting FDA approval. “I think industry wants external verification of their message. They are very well aware that people are suspicious of what industry tells them. Therefore, some kind of external verification, of which peer review is one, is important to them.”
Concerns Surrounding Article Processing Charges
Open access may be the future for Value in Health, according to Drummond. “I think we’d like to publish open access. The main problem is the article processing charges and having the funds to pay those.”
He continued, “I think it’s going to lead to some discrimination between the ‘haves’ and the ‘have-nots’ in terms of who can publish open access.” Drummond notes that there is a good chance industry will pay for open access publication of their industry-funded research. Perhaps not all results, but certainly pay for the key result to be published. However, Drummond notes these open access models may be problematic for patient organizations. “They would find $3000 a bit of a challenge, so it could be discriminatory.”
Defining Open Access Principles: cOAlition S and Plan S
cOAlition S launched in late 2018 with the goal of making full and immediate open access a standard reality in scholarly publishing. The coalition currently consists of more than 2 dozen organizations, including national organizations in Europe and South Africa and charitable funders based in the United Kingdom and the United States.
Table 2. Ten publishing principles for open access.
The Plan S principles have now been endorsed by research funders, including the Wellcome Trust and the Bill & Melinda Gates Foundation. Other notable funders who have or who are developing open access policies include the UK Medical Research Council, the European Commission, the US National Institutes of Health, and the National Science Foundation. Early adopters from the pharmaceutical industry include Shire (now part of Takeda) and Ipsen.
All Plan S funders require that articles be published with an open license, for example Creative Commons BY (CC BY). CC BY is recognized as the most permissive license type in that it allows readers to distribute, adapt, and build upon work, assuming they cite the original author.
Rooryck emphasized that cOAlition S recognizes that pharmaceutical companies may not want to make their research available through open access. However, he notes that the CC BY license would allow these companies to reuse results and perhaps even profit from them. “We have no problem with pharmaceutical companies using research for commercial purposes.”
Changing Publishing Landscape for HEOR Publications
In addition to changes in how scientific data may be disseminated (open access, preprint servers), the medical publishing landscape is expanding to facilitate the dissemination of HEOR evidence.
"The industry is very concerned about their message being believable. Peer review is one kind of verification of that message." — Michael Drummond, MCom, DPhil
White emphasized the importance of new HEOR journals. As he explained, many influential journals in the past lacked reviewers who were familiar with HEOR methodology. “They weren’t trained, weren’t familiar with it. They couldn’t really review it adequately. Definitely proliferation of HEOR has changed publishing hugely.” However, the growing role of groups such as ICER in informing medical decision making means there is now much stronger interest among mainstream clinical and health policy journals in publishing HEOR studies. Coupled with a greater understanding of HEOR concepts among journal peer reviewers, researchers have a wider choice of potential target journals for HEOR studies than ever before.
With this range of options, White recommends a selective approach to targeting HEOR publications. While specialist health economics journals are appropriate for placing more technical or methods-driven studies, he noted that “for potentially practice-changing research, mainstream clinical journals are more receptive to HEOR studies then they have ever been. The leading HEOR and health policy journals, such as Journal of Managed Care & Specialty Pharmacy and Value in Health, also genuinely reach healthcare decision makers.”
Drummond added that the category of journal (ie, clinical journal, health economics journal, or health policy journal) researchers choose to publish in depends on the character of the paper and its core main message. “My main distinction would be as follows: if you are making a point about a particular treatment or healthcare intervention (eg, an orphan drug), go for a clinical journal; if you are making a point about a particular policy issue (eg, the price of orphan drugs or access to orphan drugs), go for a policy journal; if you are making a methodological point (eg, how to evaluate orphan drugs or how to model treatment effect when the data are limited), go for a health economics journal.”
Conclusion
HEOR researchers face a growing and increasingly varied environment in which to share their findings. Many of the recent changes to the medical publishing landscape will help provide more open and timely access to more HEOR research. As the rapid-fire development of COVID-19 vaccines has demonstrated, access to timely scientific data is critically important. And as more stakeholders become involved in decision making, “access to all” may need to encompass more patient-facing plain language summaries of clinical research. As Rooryck emphasized, “Research is not a luxury product. It’s a basic necessity.”
But even as the HEOR publishing landscape continues to evolve, many challenges remain. The most pressing question remains who should pay for this new publishing model—authors, funders, readers, professional associations? And who will pay for effective and careful peer review?
Academic libraries lack the subscription budget to provide open access to all. Funders have also seen budgets cut while organizations divert funds to tackle the COVID pandemic. Even if funds were available and funders or researchers were to carry this cost, will this slow the progress of critical research?
While free and timely access to scholarly work is highly laudable, the financing structure must ensure this new publishing environment remains a trustworthy source of the highest quality research.
Resources:
• Open Access Reference Site. https://www.mpip-initiative.org/transparencymatters/openaccess.html. Accessed March 5, 2021.
• Enhanced Publications Options Navigator. https://www.mpip-initiative.org/transparencymatters/epon.html. Accessed March 5, 2021.
• Plan S. https://www.coalition-s.org. Accessed March 24, 2021.
• Guide to Creative Commons for Scholarly Publications and Educational Resources (Version final). https://zenodo.org/record/4090923/files/Creative%20Commons%20guide_final.pdf?download=1. Accessed March 24, 2021.
References
1. Sheldon T. Preprints could promote confusion and distortion. Nature. 2018;559(7715):445. https://doi.org/10.1038/d41586-018-05789-4. PMID: 30042547.
2. Séralini GE, Clair E, Mesnage R, et al. Long term toxicity of a Roundup herbicide and a Roundup-tolerant genetically modified maize. Food Chem Toxicol. 2012;50(11):4221-4231. doi: 10.1016/j.fct.2012.08.005. Retraction in: Food Chem Toxicol. 2014 Jan;63:244. PMID: 22999595.
3. Bik E. False affiliations and fake authors. Science Integrity Digest. Published June 4, 2019. https://scienceintegritydigest.com/2019/06/04/false-affiliations-and-fake-authors/. Accessed March 19, 2021.
4. Pollock J, Luxemburg RQ, Poirot HCM, Engles FX. Non-invasive laser modalities in treatment of posterior ventral infarction. J Cardiothoracic Surg Ther. 2020;4(1). https://scholars.direct/Articles/cardiothoracic-surgery/jcst-4-012.php. Accessed March 19, 2021.
5. Anderson R. Why should we worry about predatory journals? Here’s one reason. Cabells. Published March 3, 2020. https://blog.cabells.com/2020/03/03/guest-post-why-should-we-worry-about-predatory-journals-heres-one-reason/. Accessed March 19, 2021.
6. Cortegiani A, Catalisano G, Ippolito M, Giarratano A, Absalom AR, Einav S. Retracted papers on SARS-CoV-2 and COVID-19. Br J Anaesth. 2021;126(4):e155-e156. https://doi.org/10.1016/j.bja.2021.01.008.
7. Beall’s List of Potential Predatory Journals & Publishers. https://beallslist.net/. Accessed March 19, 2021.
8. Stop Predatory Journals. https://predatoryjournals.com/journals/. Accessed March 19, 2021.
9. Bethesda Statement on Open Access Publishing. Published June 20, 2003. legacy.earlham.edu/~peters/fos/bethesda.htm. Accessed March 5, 2021.
10. Budapest Open Access Initiative. Ten years on From the Budapest Open Access Initiative: Setting the Default to Open. Published 2012. https://www.budapestopenaccessinitiative.org/boai-10-recommendations. Accessed March 5, 2021.
11. Piwowar H, Priem J, Larivière V, et al. The state of OA: a large-scale analysis of the prevalence and impact of open access articles. Peer J. 2018.6:e4375. https://doi.org/10.7717/peerj.4375.
12. Davis P. Does a Citation Advantage Exist for Mandated Open Access Articles? Scholarly Kitchen. Published January 7, 2010. https://scholarlykitchen.sspnet.org/2010/01/07/citation-advantage-for-mandated-open-access-articles/. Accessed March 7, 2021.
13. Sood A, Prasad K, Chhatwani L, et al. Patients’ attitudes and preferences about participation and recruitment strategies in clinical trials. Mayo Clin Proc 2009.84(3):243-247.
14. Fox S, Duggan M. Health Online 2013. Pew Research Center for Internet and Technology. Published January 15, 2013. http://www.pewinternet.org/2013/01/15/health-online-2013. Accessed March 2021.
About the Author
Michele Cleary is a HEOR writer in Minneapolis, MN.
