Abstract: Scientific integrity is at the forefront of the scientific research enterprise. This paper provides an overview of key existing efforts on scientific integrity by federal agencies, foundations, nonprofit organizations, professional societies, and academia from 1989 to April 2016. It serves as a resource for the scientific community on scientific integrity work and helps to identify areas in which more action is needed. Overall, there is tremendous activity in this area and there are clear linkages among the efforts of the five sectors. All the same, scientific integrity needs to remain visible in the scientific community and evolve along with new research paradigms. High priority in instilling these values falls upon all stakeholders.
Updates to the Scientific Integrity Resource Guide
As part of the publication of this work, ILSI North America committed to keeping the “Resource Guide” a living document by creating a page on the ILSI North America website to post additional and new work on scientific integrity in the 5 different sectors twice a year.
December 2018 Update:
National Science Foundation (NSF)
The NSF has developed a new Strategic Plan Building the Future Investing in Discovery and Innovation for Fiscal Years (FY) 2018-2022 with an objective to advance the practice of research. NSF will implement review processes tailored to interdisciplinary research and to enhance efforts to identify and recruit reviewers with experience in cross-cutting research. Working with the research community, NSF promotes the use of best practices to ensure that research is reproducible, including emphasizing the open availability of results and the data that support them. NSF will promote a research culture that is broadly inclusive in its demography and range of intellectual ideas, has access to cutting-edge infrastructure, and is globally engaged, with increased opportunities for exchanging ideas and collaborating on an international scale.
In October 2018, the National Science Board (NSB) issued a Statement on Security and Science. As partners in the scientific enterprise, U.S. universities and colleges must help promote scientific openness and integrity and safeguard information that impacts national security and economic competitiveness. The NSB recommends that all institutions conducting fundamental research supported by the National Science Foundation embrace transparency and rigorously adhere to conflict of interest and conflict of commitment policies. The Board also encourages those institutions to educate their communities about how to protect the integrity of research.
The Office of Advanced Cyberinfrastructure (OAC) hosted a Cyberinfrastructure (CI) webinar series. One of the webinars titled “Cybersecurity to Enable Science: Hindsight & Vision from the NSF Cybersecurity Center of Excellence” addressed “how cybersecurity can play an enabling role in scientific research and how cybersecurity supports scientific integrity, reproducibility, and productivity. The webinar was given by Von Welch, Director and PI for the NSF Cybersecurity Center of Excellence.
US Department of Health and Human Services (HHS)/Office of Research Integrity (ORI)
New Research Misconduct Case Summaries have been included in the HHS ORI Spring 2018 Newsletter. These cases highlight research misconduct in which administrative actions were imposed.
ORI supported multiple grants at Universities for scientific integrity work which are cited under Academic Updates.
National Institutes of Health (NIH)
In August 2018 NIH released a Statement on Protecting the Integrity of U.S. Biomedical Research as a response to increasing risks to the security of intellectual property and the integrity of peer review. NIH will work with other government agencies, NIH-funded academic institutions, U.S. professional organizations, and a working group of the Advisory Committee to the (NIH) Director to identify robust methods to: 1) improve accurate reporting of all sources of research support, financial interests, and affiliations; 2) mitigate the risk to intellectual property security while continuing NIH’s long tradition of collaborations, including with foreign scientists and institutions; and 3) explore additional steps to protect the integrity of peer review.
US Environmental Protection Agency (EPA)
In August 2018 the EPA released a memorandum on the Implementation of the EPA’s Scientific Integrity Policy. The Office of Inspector General (OIG) for the agency has responsibility for the review. The objective is to examine the following areas:
- Extent and type of employee concerns, if any, with scientific integrity at the EPA.
- Employee awareness of the EPA’s Scientific Integrity Policy, including the process for reporting potential violations.
- Reasons potential violations may not be reported.
- Adjudication process in general and any concerns (e.g., satisfaction with complaint resolution, timeliness of resolution and other process-related issues).
The anticipated benefits of this project are to identify and address any barriers to achieving the Scientific Integrity Policy’s stated purpose, which is “to ensure scientific integrity throughout the EPA and promote scientific and ethical standards, including quality standards; communications with the public; the use of peer review and advisory committees; and professional development.”
US Department of Agriculture (USDA)
In February 2018 the OIG released the updated final report of the survey on USDA Scientists Regarding Scientific Integrity. The purpose of the survey was to assess whether USDA scientists perceive they can perform and communicate all aspects of their research assignments or projects without inappropriate influence or political interference. The survey was sent to 2,212 USDA research-grade scientists in 4 USDA agencies: the Agricultural Research Service, the Economic Research Service, the Forest Service, and the Natural Resources Conservation Service.
November 2017 Update:
National Science Foundation (NSF)
In August 2017, the NSF released Important Notice No. 140, Training in Responsible Conduct of Research – A Reminder of the NSF Requirement. The NSF recognizes the importance of research integrity and the responsible and ethical conduct of research. As such, NSF’s Responsible Conduct of Research (RCR) requirement applies to the breadth of research disciplines the Foundation funds and the different educational levels of the students and post-doctoral researchers the agency supports. RCR training should be effective and appropriately tailored to the specific needs and circumstances at each university, and it is the responsibility of each institution to determine both the focus and delivery method for appropriate training.
US Department of Health and Human Services, Office of Research Integrity (HHS ORI)
In 2017, HHS ORI has added several new case summaries of cases in which administrative actions were imposed due to findings of research misconduct to their ongoing list.
The Office of Research Integrity, The George Washington University, and Public Responsibility in Medicine and Research co-sponsored the 2017 Quest for Research Excellence Conference in August 2017. The goal of the Quest for Research Excellence conference series is to fuel knowledge sharing among all the parties involved in promoting the responsible conduct of research and scientific integrity, from scientists to educators, administrators, government officials, journal editors, science publishers and attorneys. The theme of the 2017 conference was “breaking down the silos” and the conference focused on five topics from the perspectives of various stakeholders. These topics included research misconduct, the responsible conduct of research, the legal implications of research misconduct, scientific publications and open science.
HHS ORI co-sponsored a Responsible Conduct of Research Instruction Workshop (RCRIW) with University of California - San Diego in September 2017. The two-day train-the-trainers workshop helped new or inexperienced Responsible Conduct of Research (RCR) instructors to develop and implement best practices in RCR instruction. The course presenters were experienced RCR instructors and researchers who presented distinct topics, led active discussions and exercises, and identifed useful resources (case studies, short writing assignments, etc.) for a foundational understanding of RCR and the tools needed for successful RCR instruction. Four topic sessions (data; misconduct; collaboration; and publication and authorship) demonstrated effective teaching methods and illustrated positive and negative practices in the conduct of research.
US Department of Health and Human Services, National Institutes of Health (NIH)
NIH has compiled a website on research integrity, including links to its policies on research integrity, resources on research misconduct, and RCR training: https://grants.nih.gov/policy/research_integrity/index.htm
NIH also has a website page on Peer Review Policies and Practices: https://grants.nih.gov/policy/peer/index.htm
May 2017 Update:
Office of Science and Technology Policy (OSTP)
Following the release of the OSTP Memorandum on Scientific Integrity in 2010 that provides guidance for federal agencies to develop scientific integrity policies, OSTP tasked the IDA Science and Technology Policy Institute (STPI) to review the scientific integrity policies of the 24 Federal agencies that complied with the Memorandum. OSTP also asked STPI to identify potential good practices for meeting or exceeding the principles identified in the OSTP Memorandum and to suggest ways of strengthening the policies to reflect current interests and developments. The report of STPI’s work, Review of Federal Agency Policies on Scientific Integrity, was released in January 2017. https://www.ida.org/idamedia/Corporate/Files/Publications/STPIPubs/2016/D-8305.ashx
US Department of Health and Human Services Office of Research Integrity (HHS ORI)
HHS ORI includes a list of new work on their website. Since the Resource Guide was submitted for publication, ORI has updated its list of research misconduct findings, updated their Learning Module on Plagiarism, and updated the Basic Research Concepts Module. ORI's Extramural Program awarded five research grants and five conference grants in FY2016. ORI released 12 new infographics on topics related to the responsible conduct of research (RCR) and the handling of research misconduct and released new video case studies on research misconduct. For a full list of recent work, visit: https://ori.hhs.gov/ori-updates
US Department of Agriculture (USDA)
In November 2016, USDA released its new Scientific Integrity policies by releasing Departmental Regulation 1074-001 ("Scientific Integrity"). The DR supersedes previous guidelines on scientific integrity released by USDA and, “establishes the U.S. Department of Agriculture’s (USDA) Scientific Integrity Policy and provides instruction and guidance to Departmental leadership and employees to ensure the highest level of integrity in all aspects of the Department’s and USDA agencies’ involvement with scientific and technological processes, research, analyses, and products. This DR includes instructions and guidance for decision makers as they develop public policies that are informed by science relevant to food, agriculture, natural resources, rural development, and related issues. This DR is intended to instill public confidence in USDA research and science-based public policymaking by articulating the principles of scientific integrity and the roles and responsibilities of all USDA employees, including career staff and political appointees, in upholding these principles.” https://www.ocio.usda.gov/sites/default/files/docs/2012/Final%20-%20DR%201074-001%20Scientific%20Integrity.pdf
USDA also issued USDA Departmental Manual 1074-001 ("Procedures for Responding to Allegations of Compromised Scientific Integrity"). The new DR and DM, along with other resources on scientific integrity, research misconduct, policy, and reporting of allegations of compromised scientific integrity and research misconduct involving USDA conducted or supported scientific and technological activities are now available in one place on the USDA website: https://www.usda.gov/our-agency/staff-offices/office-chief-scientist-ocs/scientific-integrity-and-research-misconduct
USDA has also created a website of Scientific Integrity Resources that includes links to the White House Memoranda and Executive Orders and the many Federal Department and Agency Scientific Integrity Policies, as well as links to the General Services Administration Guidance on Scientific Integrity and the Whistleblower Protection Enhancement Act of 2012.
December 2018 Update:
John Templeton Foundation
The John Templeton Foundation, Templeton Religion Trust, and Templeton World Charity Foundation has awarded three, multi-year grants to the Center for Open Science (COS). These grants will support the development and improvement of the Open Science Framework (OSF), COS's free and open cloud-based project collaboration system for managing, registering, and archiving the process and outputs of research. Learn more.
November 2017 Update:
Laura and John Arnold Foundation (LJAF)
LJAF has established Guidelines for Investments in Research that outline their expectations for grantees and consultants, including information on pre-registration and openness and sharing. http://www.arnoldfoundation.org/wp-content/uploads/Guidelines-for-Investments-in-Research.pdf
LJAF is the lead funder of the Meta-Research Innovation Center at Stanford University (METRICS) (see METRICS under Academia). METRICS is bringing researchers together from diverse disciplines to improve the way scientists conduct, report, and share their research. It is uncovering weaknesses in scientific investigation, identifying ways to improve efficiency and reliability, and working to reduce bias in study design and outcomes reporting. http://www.arnoldfoundation.org/initiative/research-integrity/center-for-open-science/
LJAF also supports the AllTrials initiative and its campaign to require that all clinical trials—past and present—be registered with full disclosure of study methods and results. The foundation is funding the creation of Open Trials, an open, online database that will aggregate information from a wide variety of sources to provide a comprehensive picture of the data and documents associated with all trials of medicines and other treatments from around the world.
LJAF seeded the Nutrition Science Initiative (NuSI), an organization that is addressing the urgent need for more rigorous research in nutrition science. http://www.arnoldfoundation.org/initiative/research-integrity/center-for-open-science/
In addition, an LJAF grant to the American Association for the Advancement of Science is supporting a partnership between Science and other major journals and government funders. The collaboration is focused on developing new guidelines for transparency that outline the elements editors and reviewers should consider when deciding whether to publish a study.
In an effort to encourage high-quality reporting on health and medical news, LJAF funds Health News Review, an initiative of the University of Minnesota. The group regularly critiques the press releases that universities and scientific journals issue about health studies. LJAF believes Health News Review will help to improve transparency and allow individuals to make more informed decisions about their health. http://www.arnoldfoundation.org/initiative/research-integrity/guidelines-for-investment-research/
December 2018 Update:
National Academies of Sciences, Engineering, and Medicine (NAS)
In March 2018 NAS published the article “Transparency in authors’ contributions and responsibilities to promote integrity in scientific publication” in the Proceedings of the National Academy of Sciences (PNAS). The article outlines proposed changes to journal authorship policies and procedures to provide insight into which author is responsible for which contributions. NAS recommends having journals adopt common and transparent standards for authorship, outline responsibilities for corresponding authors, adopt the Contributor Roles Taxonomy (CRediT) (docs.casrai.org/CRediT) methodology for attributing contributions, include this information in article metadata, and require authors to use the ORCID persistent digital identifier (https://orcid.org). Universities and research institutions should articulate expectations about author roles and responsibilities to provide a point of common understanding for discussion of authorship across research teams. NAS proposes funding agencies adopt the ORCID identifier and accept the CRediT taxonomy to further authorship transparency.
In November 2018 NAS held a session on “Transparency and Accountability and the National Science Foundation” organized by the Committee on Science, Engineering, Medicine, and Public Policy.
November 2017 Update:
National Academies of Sciences, Engineering, and Medicine
The National Academy of Sciences’ Arthur M. Sackler Colloquia on the Science of Science Communication began in 2012 with an effort to survey the state of the art of empirical social science research in science communication and focused on the communication dynamics surrounding issues in science, engineering, and medicine. The second colloquium, held in 2015, highlighted the particular challenges with communicating about science that involves controversy, and was an important impetus for the consensus study report “Communicating Science Effectively: A Research Agenda.” The third colloquium, The Science of Science Communication III: Inspiring Novel Collaborations and Building Capacity, was held in November 2017. It focused on the consensus study as a framework for advancing both research and practice in science communication. It explored ways to build capacity for and foster the use of evidence-based strategies for engaging the public with science and ensuring its appropriate use. http://www.nasonline.org/programs/sackler-colloquia/upcoming-colloquia/Science_Communication_III.html
Center for Open Science (COS)
In 2017, COS has helped to launch multiple preprint services in different areas of science, including MarXiv, for the ocean conservation and marine climate sciences, EarthArXiv, provides free, open access, open source archives for the earth sciences, INA-Rxiv, the preprint server of Indonesia; LISSA, an open scholarly platform for library and information science; MindRxiv, a service for research on mind and contemplative practices; NutriXiv, a preprint service for the nutritional sciences; paleorXiv, a digital archive for Paleontology; and SportRxiv, an open archive for sport and exercise-related research.
COS has 16 community preprint services built on COS’s flagship platform, the Open Science Framework (OSF), which helps researchers design and manage their project workflow, data storage, DOIs, and collaboration. COS has leveraged that platform to help research communities in many disciplines discover new research as it happens and to receive quick feedback on their own research prior to publication. The community preprints platform from COS provides organizations that want to launch their own preprints service an easy, robust, and stable solution. https://cos.io/about/news/center-open-science-and-marxiv-launch-branded-preprint-service/
COS has received a $165,591 grant from the James S. McDonnell Foundation to undertake two studies evaluating the impact of Registered Reports (RRs) on research quality and outcomes. RRs were introduced in 2013 as an innovative method for improving reproducibility. RRs clarify the distinction between confirmatory and exploratory research, and eliminate a variety of questionable research practices, including low statistical power, selective reporting of results, and publication bias. Simultaneously, RRs preserve complete flexibility to report serendipitous findings as exploratory findings. These two studies began in September 2017, will occur concurrently and will be completed over 16 months. They will be published as Registered Reports. https://cos.io/about/news/center-open-science-receives-grant-james-s-mcdonnell-foundation-study-impact-registered-reports/
In August 2017, COS announce the release of Thesis Commons, a free, cloud-based, open-source platform for the submission, dissemination, and discovery of graduate and undergraduate theses and dissertations from any discipline. Authors can share their electronic theses and dissertations (ETDs) with a quick and easy submission workflow. Readers can search, discover, and download with a simple interface. Institutions can sign-up for a branded version of the service for their institutional community for hosting ETDs, preprints, or other scholarship. Thesis Commons is available on the OSF. https://cos.io/about/news/center-open-science-launches-thesis-commons-open-source-platform-theses-and-dissertations/
COS promotes the adoption of the Transparency and Openness Promotion (TOP) Guidelines by journals, funders, and scholarly societies. The TOP Guidelines can increase reproducibility of research by adopting and helping journals, funders, and scholarly societies evolve to meet the needs of researchers and publishers while pursuing the most transparent practices. Published in Science in 2015 (OA), the TOP guidelines include eight modular standards, each with three levels of increasing stringency. Journals select which of the eight transparency standards they wish to adopt for their journal, and select a level of implementation for each standard. These features provide flexibility for adoption depending on disciplinary variation, but simultaneously establish community standards. Over 5,000 journals and organizations have already become signatories of the TOP Guidelines. https://cos.io/our-services/top-guidelines/
May 2017 Update:
National Academies of Sciences, Engineering, and Medicine
The National Academies of Sciences, Engineering, and Medicine published a report, Fostering Integrity in Research, in April 2017. The report was published by the Committee on Responsible Science and the Committee on Science, Engineering, Medicine, and Public Policy. It examines challenges to scientific integrity facing the research enterprise and recommends steps that individual scientists, research sponsors, research institutions, journal publishers, and professional societies should take to meet these challenges and better protect integrity in research.
The National Academies of Sciences, Engineering, and Medicine’s Government-University-Industry Research Roundtable held a meeting in early 2017 on, “Examining the Mistrust of Science.” The meeting was convened with the intent to consider the trends in public opinion of science, examine potential sources of mistrust both internal and external to the science community, and explore the ways in which cross-sector collaboration between government, universities, and industry may improve public trust in science and scientific institutions in the future. http://sites.nationalacademies.org/PGA/guirr/PGA_177700#
The Sackler Colloquium on Reproducibility of Research: Issues and Proposed Remedies was held in March 2017. The goal of this colloquium was to bring together scientists and researchers from multiple disciplines to lay out the scope of the problem of reproducibility in a more tactical way that permits each problematic aspect to be measured, assessed for baseline levels, targeted with proposed interventions to reduce the occurrence, and monitored for improvement.
Union of Concerned Scientists (UCS)
As stated on its website, the Union of Concerned Scientists, “puts rigorous, independent science to work to solve our planet’s most pressing problems. Joining with citizens across the country, we combine technical analysis and effective advocacy to create innovative, practical solutions for a healthy, safe, and sustainable future.” (http://www.ucsusa.org/about-us#.WPDkYtLys2w)
In 2004, UCS issued a report, Scientific Integrity in Federal Policy Making, and a statement signed by 62 leading U.S. scientists (more than 15,000 would eventually add their names) calling for an end to political interference in science. Over the following years, the UCS Scientific Integrity Program documented the problem extensively with in-depth reports, case studies, and surveys of federal scientists, including a compilation of Abuse of Science: Case Studies and Surveys of Scientists at Federal Agencies.
Most recently, UCS has released a report on, Preserving Scientific Integrity in Federal Policymaking, in 2017.
December 2018 Update:
Committee on Publication Ethics (COPE)
On July 2018 COPE held its third educational webinar for members titled “Creating and implementing data research policies” on the practicalities and challenges of choosing data sharing policies for journals and ensuring authors and reviewers adhere to the policies.
In the article “Research integrity and how to buy a Persian carpet: TOP Guidelines, part 2” COPE is collecting comments on the draft article from the Center for Open Science’s Transparency and Openness (TOP) Guidelines Workshop that the Committee attended in September 2017. The extended workshop was designed to make the original TOP guidelines easy for researchers to understand and adopt and implement so research readers can build-upon a piece of work.
Council of Science Editors (CSE)
CSE updated their White Paper on Promoting Integrity in Scientific Journal Publications in May 2018 and will continue to update it on a rolling basis as needed. This paper serves as a basis for developing and improving effective practices to promote integrity in scientific publishing. Through this White Paper and other activities, the Editorial Policy Committee aims to open dialog about ethical publishing practices, inform those involved in the editorial process, and foster informed decision making by editors.
Federation of American Societies for Experimental Biology (FASEB)
In collaboration with the National Institutes of Health (NIH), the 2018 FASEB Science Research Conference (SRC) series is piloting a Transparency and Rigor Initiative to enhance the reproducibility of research findings in research. This pilot will use badges to test whether key information regarding experimental methods can be conveyed effectively using transparency and rigor at the 2018 SRC. This initiative aligns with the broader efforts of FASEB to promote rigor and reproducibility in biological and biomedical research.
American Chemical Society (ACS)
ACS updated their website to share their policy priorities to support scientific integrity. ACS supports the efforts to:
- Promote a strong, non-governmental, scientific publishing enterprise that assures access to information and exchange of scientific ideas and information among all parties with legitimate uses while appropriately protecting copyright and security-related information.
- Assure the quality of science and technological advancement through open, rigorous and inclusive peer review.
- Promote institutions and guidelines to assure that governments make appropriate and open use of scientific and technological information in making policy decisions.
- Assure the most open interactions possible among scientists, engineers, and students from across the globe.
ACS also lists policy priorities in scientific integrity and other areas.
November 2017 Update:
American Chemical Society (ACS)
ACS released its 2017 advocacy agenda in January. The Society’s highest priorities fall into three broad categories, one of which is to ensure scientific integrity and the role of science in the public policy arena.
American Society for Microbiology (ASM)
The American Society for Microbiology (ASM) has compiled resources for authors on the ASM ethics portal; including a particularly useful module on avoiding image manipulation. The ASM website includes a Compendium of Resources highlighting ASM policies/procedures, ASM publications on ethics, and resources on both publishing and research ethics.
Molecular and Cellular Biology published an editorial that discusses how Molecular and Cellular Biology (and other ASM journals) respond to post-publication review as well as ASM’s screening post-acceptance analysis for potential data manipulation and/or plagiarism. This effort to ‘clean up image archives’ was noted in The Scientist.
American Sociological Association (ASA)
ASA has a Code of Ethics that sets forth the principles and ethical standards that underlie sociologists' professional responsibilities and conduct. These principles and standards should be used as guidelines when examining everyday professional activities. ASA's Code of Ethics consists of an Introduction, a Preamble, five General Principles, and specific Ethical Standards. This Code is also accompanied by the Rules and Procedures of the ASA Committee on Professional Ethics which describe the procedures for filing, investigating, and resolving complaints of unethical conduct. ASA is proposing revisions to the Code of Ethics in 2018.
Committee on Publication Ethics (COPE)
Since 2013, COPE has released a monthly newsletter, the COPE Digest: Publication Ethics in Practice.
COPE has developed flowcharts that are designed to help editors follow COPE’s Core Practices and implement its advice when faced with cases of suspected misconduct. The flowcharts have been translated into a number of different languages.
Council of Science Editors (CSE)
The CSE website includes many documents describing CSE policies that are to be used as resources by CSE members, including:
Approved by the CSE Board of Directors
- Predatory or Deceptive Publishers – Recommendations for Caution
- CSE Recommendations for Group-Author Articles in Scientific Journals and Bibliometric Databases
- Who’s the Author? Problems with Biomedical Authorship, and Some Possible Solutions
Retreat and Task Force Papers
Other Supported Statements
- ICMJE’s Statement on Clinical Trial Registration
- WAME Policy Statement on Geopolitical Intrusion on Editorial Decisions
CSE also has compilation of retraction resources that includes primary guidance documents and other materials from COPE, CSE, ICMJE, and others, that provide information related to handling retractions of scientific publications. The materials provided also address publication corrections, errata, partial retractions, and editorial expressions of concern, and include information on how HHS ORI handles suspected scientific misconduct in the research funded by agencies under its governance.
CSE has an annual Award for Meritorious Achievement, which is given to a person or institution that embraces the purposes of CSE–the improvement of scientific communication through the pursuit of high standards in all activities connected with editing. Recipients do not need to be a member of CSE. https://www.councilscienceeditors.org/about/awards/
May 2017 Update:
American Society for Microbiology (ASM)
The American Academy of Microbiology, as part of the American Society for Microbiology, published, A Framework for Improving the Quality of Research in the Biological Sciences, in July/August 2016. The publication outlines a colloquium convened by the American Academy of Microbiology, “to discuss problems in the biological sciences, with emphasis on identifying mechanisms to improve the quality of research. Participants from various disciplines made six recommendations: (i) design rigorous and comprehensive evaluation criteria to recognize and reward high-quality scientific research; (ii) require universal training in good scientific practices, appropriate statistical usage, and responsible research practices for scientists at all levels, with training content regularly updated and presented by qualified scientists; (iii) establish open data at the timing of publication as the standard operating procedure throughout the scientific enterprise; (iv) encourage scientific journals to publish negative data that meet methodologic standards of quality; (v) agree upon common criteria among scientific journals for retraction of published papers, to provide consistency and transparency; and (vi) strengthen research integrity oversight and training. These recommendations constitute an actionable framework that, in combination, could improve the quality of biological research.” http://mbio.asm.org/content/7/4/e01256-16.abstract
December 2018 Update:
George Mason University
George Mason University held a workshop in the summer of 2018 titled “Promoting Research Integrity in Multidisciplinary and Multi-team Based Science Initiatives”. The workshop focused on “National Institutes of Health (NIH) principal investigators and will be aimed at clarifying the nature of lapses in the operation of complex multi-team systems when it comes to research integrity, using the insights of participants from both their direct and indirect multi-teams experience. Funded by HHS ORI grant ORIIR170033.
Ohio State University, Office of Research
In September 2018, Ohio State University’s Office of Research sponsored a one-day summit titled “Seeking Solutions in Research Integrity: A View from All Perspectives.” The conference brought together researchers, institutions, publishers, funders, scientific societies, the press, and federal agencies to discuss concerns in research integrity and current efforts to discuss those concerns, laying the foundation to detail next steps in fostering changes to enhance integrity in research.
Purdue University and Indiana University
In April 2018, Purdue University and Indiana University hosted “Plagiarism: A Conference on the Identification, Processing, Prevention and Cultural Context of Plagiarism,” funded by HHS ORI grant ORIIR170031-01-00. The conference brought together Research Integrity Officers (RIO), Responsible Conduct of Research (RCR) instructors, faculty, and federal partners to explore the breadth of plagiarism issues and their subtleties in the modern, multicultural research environment.”
Stanford University, Center for Advanced Study in Behavioral Sciences (CASBS)
In March 2018 CASBS launched the “Impact Evaluation Design Lab” to build an evidence-informed decision framework to help policy makers assess large-scale public policy programs and engage researchers in advancing social scientific methodology through real-world evaluation design challenges. King County, WA and Stockton, CA are the first two partners in the design lab.
University of California Berkeley, Berkeley Initiative for Transparency in the Social Sciences (BITSS)
BITSS is providing funds for Texas A&M University to develop a graduate health online course in research integrity, transparency, and reproducibility to respond to the growing recognition within the scientific community that flexibility in study design, data analysis, and the reporting of research findings is increasingly leading to the publication of misleading results that capitalize on chance and cannot be replicated. It has been suggested that the use of such practices, if not made apparent in a manuscript describing the results of a study, is a form of research misconduct. This course will examine various threats to the integrity of research, the professional and organizational factors that produce these threats, and the solutions that have been suggested to improve research quality (such as registered reports, open data, and team of rivals). Upon completion of the course, students should have the ability to differentiate research that is conducted with integrity and capable of producing valid and reproducible findings from research that is conducted without integrity and produces chance results that are trivial and non-reproducible. Students should also have the ability to incorporate practices into their own research that will increase its transparency and ensure it is conducted with integrity.
Through the BITSS Catalyst Program, Soazic Elise Wang Sonne has trained more than 150 African researchers via workshops in South Africa, Cameroon, the UK, the US, and the Netherlands on research transparency and reproducibility. At these trainings, participants reflect on how research transparency norms and practices can be better entrenched in African academic higher institutions. Building the capacity of African researchers to conduct reproducible and transparent research will help ensure findings used by policy makers to make key decisions are based on credible evidence. It will also strengthen trust and partnerships between policy makers and the academic community. Further information is found here.
University of Minnesota
In March 2018, University of Minnesota held a conference titled “Research Integrity and Trustworthy Science: Challenges & Solutions.” The conference was part of Research Ethics Week, during which the University focused on professional development and best practices to ensure safety and integrity in research. The conference brought together leading thinkers from multiple disciplines – biomedicine, the social sciences, law, ethics, and others – to analyze the challenges for researchers, universities, journals, and the community and map a way forward.
University of Utah, Spencer S. Eccles Health Sciences Library
The Spencer S. Eccles Health Sciences Library at the University of Utah provides several resources including classes and workshops on the reproducibility of research. In June 2018 the University offered a five-day course that “explored issues of research reproducibility in-depth in a seminar-type setting, followed by hands-on sessions to learn actionable, practical solutions to make research work more reproducible.” On the last day a conference on Building Research Integrity Through Reproducibility funded in part by HHS ORI grant ORIIR170034 was held.
Vanderbilt University, Vanderbilt Linkages Program
In September 2018, Vanderbilt Linkages held the third Growing Research Integrity Together (GRIT) Conference. The 2-day online workshop focused on the role of researcher administrators in promoting research integrity. The event featured U.S. experts from various fields of research integrity and research administration. The GRIT series was developed in part under a grant from HHS ORI ORIIR160029.
November 2017 Update:
Berkeley Initiative for Transparency in the Social Sciences (BITSS)
BITSS has created the Catalyst Program. Catalysts are graduate students, academic faculty, and other researchers in the social sciences who are committed to changing norms by educating the current and next generation of economists, psychologists, political scientists, and other social scientists on transparency tools and practices. The Catalyst program formalizes a network of professionals to advance the teaching, practice, funding, and publishing of transparent social science research. The overarching goal of this program is to connect and empower individuals to make changes in their own classrooms, in their universities, and across their network, thereby achieving far-reaching change. BITSS is supporting 100 academics and research practitioners working in 75 institutions in 28 countries (across five continents) as leaders in the open science movement. Catalysts:
- Educate: Catalysts deliver at least one training per academic year on research transparency. Catalysts can identify the appropriate delivery mechanisms for integrating training on research transparency, including integrating into existing curricula, seminars, workshops, or boot camps.
- Develop, share, and strengthen educational materials: Catalysts share existing resources and new materials that complement and extend the BITSS library, as well as provide feedback on the existing BITSS educational materials library. All materials will be shared through OSF, GitHub, or other sharing mechanisms for the benefit of other Catalysts and the broader community.
- Advocate: Catalysts lead advocacy efforts that will affect policy change at the department or academic senate levels (e.g. among university governing bodies). This may include writing blogs, OpEd pieces, meeting with administrators and other decision makers to discuss and develop additions and or revisions to curriculum.
- Catalyst Training Grants: BITSS offers grants to Catalysts to develop and deliver trainings in the form of workshops, conferences, short-courses, or university-level curriculum development.
BITSS funds Social Science Meta-Analysis and Research Transparency (SSMART) Grants. SSMART aims to improve the quality of research in economics, political science, psychology, and related disciplines by funding research related to transparency and reproducibility issues. With catalytic funding from the LJAF, SSMART is also partially funded by the William and Flora Hewlett Foundation. There are three core categories of projects funded through SSMART:
- Develop new methodology: These projects develop innovative methods to improve the transparency and credibility of research findings.
- Produce new meta-analysis: These projects develop new tools and approaches for meta-analysis.
- Study research culture and the adoption of new methods: These projects study researcher norms and strategies to promote the practice of open science.
BITSS has awarded their 2017 Leamer-Rosenthal Prizes for Open Social Science. There are two types o of awards: Leaders in Education and Emerging Researchers.
Center for Clinical Research Ethics (CCRE)
CCRE, located at Washington University School of Medicine in St. Louis, offers a Professionalism & Integrity (P.I.) in Research Program called “Helping Researchers Become More Effective Professionals.” Using a career-coaching model, the P.I. Program offers personalized assessments, a group workshop, and post-workshop coaching calls to help researchers operate professionally in today’s complex environments. A variety of individuals have profited from the P.I. Program, including researchers who: (1) Find it challenging to balance scientific and compliance demands in today’s complex regulatory environments, (2) Have been investigated for noncompliance or misconduct and wish to move forward constructively, (3) Have staff who have been investigated for noncompliance or misconduct, and want to learn how to provide strong professional leadership and oversight. http://integrityprogram.org/ The P.I. Program offers webinars and courses.
The HHS ORI (see HHS ORI under Federal Agencies) awarded a grant to the P.I. Program to expand its recruitment and assessment efforts. New assessments will examine the work habits of participants. The project builds on work from the recent NIH K01 Award received by Dr. Alison Antes, the P.I. Program’s most recent faculty member, which includes studying the work habits of research exemplars. Drs. Antes and DuBois recently interviewed 52 researchers who conduct high impact research and enjoy a reputation for great leadership and integrity in research. Data from these projects will inform recommendations on best practices for lab leadership and management.
The CCRE Dr. Daniel Bisno Ethics in Medicine Fund was established in 2001 to provide experiences designed to raise awareness of ethical issues in medical practice and to improve ethical decision making. The fund is used to support diverse activities, including lectures, curriculum development, small group meetings, publications, and other activities sponsored by the CCRE.
Meta-Research Innovation Center at Stanford (METRICS)
Launched in April 2014 with a founding grant from the Laura and John Arnold Foundation, METRICS is a research to action center focused on transforming research practices to improve the quality of scientific studies in biomedicine and beyond. METRICS fosters multi-disciplinary research collaborations to help produce solutions that increase the effectiveness and value of scientific investigation. By serving as a lens that both focuses and helps magnify the impact of scholars, policymakers and others working on meta-research issues, METRICS hopes to enhance the public discourse and advance the development of policies and practices that maximize the use of best research practices. METRICS has three main goals:
- Build the meta-research field and catalyze solutions-focused research to develop best scientific practices
- Provide leaders with the skills and knowledge needed to support the development and implementation of high-quality research
- Transform research practices to strengthen the evidence base for informed decision-making
METRICS promotes research and drives innovation in a broad domain of subjects, which are divided into five focal areas corresponding to critical phases of the research cycle:
- Methods: The phase of designing and conducting research
- Rigorous research methods are at the heart of good science. These methods include everything from question formation to study design, conduct, statistical analysis and interpretation. Meta-researchers use both theoretical and empirical approaches to identify flaws and biases in the research enterprise, then develop and test methods to minimize them and define best practices.
- Reporting: The phase of communicating research
- For research to be useful it needs to be communicated properly to scientists and others who use the results. This reporting is done through research papers, written by scientists, or by various forms of media that convey scientific results to broader audiences. Meta-researchers are developing standards for reporting results and experimenting with new means to improve communication to scientists, policy makers, patients, and the general public.
- Evaluation: The phase of evaluating research
- Scientific quality is constantly evaluated in journal publications, funding decisions, academic promotions, industry investment, medical practice guidelines, standard setting, and policy decisions. Peer review has long served as the cornerstone of such evaluations, but it is widely acknowledged to be imperfect and inefficient. Meta-researchers across multiple disciplines are assessing its effectiveness and experimenting with new approaches to improve the assessment of scientific quality.
- Reproducibility: The phase of verifying research
- Replicating research results is central to scientific credibility. Meta-research has repeatedly shown that accurate reporting and sound peer-review do not by themselves guarantee the reproducibility of a scientific claim and that active replications of previous research are too rarely done. Meta-researchers are developing new ways to assess the reproducibility of findings, make experimental data available, and promote policies that can make verifying research more routine and more effective.
- Incentives: The phase of rewarding research
- Research is a rewarding activity in itself, but funding, recognition, and career advancement are critical to a successful scientific career. If the scientific reward system is not aligned with the adoption of best scientific practices, such as study replication and publishing negative results, such practices will happen rarely. Changing the criteria and processes by which rewards are distributed in science can help ensure that the most reliable research is encouraged.
METRICS’ publications database provides the latest in meta-research across METRICS five focal areas. Publications can be filtered by author, publication year, region, and METRICS faculty and affiliates.
To help meet METRICS’ goal of establishing meta-research as a new field of study and to improve how researchers conduct, report and disseminate their findings, METRICS supports a variety of education and training opportunities. Through the development of curricula to be delivered in the classroom and online, METRICS intends to produce a new generation of scholars focused on transforming research practices and improving the quality of scientific studies. https://metrics.stanford.edu/education
Beginning in 2017, METRICS will offer a limited number of research fellowships for doctoral students and postdoctoral scholars to help advance the Center’s research agenda.
National Center for Professional and Research Ethics (NCPRE)
NCPRE is located within the Coordinated Science Lab at the University of Illinois at Urbana-Champaign. NCPRE creates and shares resources to support the development of better ethics and leadership practices. The Center focuses on leadership in a variety of institutional settings, from academia to business. NCPRE is part of the Coordinated Science Laboratory in the College of Engineering at the University of Illinois at Urbana-Champaign.
The NCPRE project began with a $1.5 million grant from the National Science Foundation. Current work includes studying and promoting improved research practice in university and private settings. One of NCPRE’s flagship initiatives is the SOURCE project, which provides universities with tools to assess their research ethics climates and benchmark themselves against institutional peers. Another important current activity is a multi-year leadership development institute, in collaboration with Nanyang Technological University in Singapore.
In all its activities, NCPRE is committed to forming, developing, and supporting communities of responsible professional practice. NCPRE:
- Recognizes that no one seeks disaster; small choices are important in personal and ethical pathways.
- Creates tools, systems, and resources to help universities and others educate about ethics, the responsible conduct of research, and leadership.
- Educates about ethics as the smart choice, not just the right choice.
- Helps institutions assess the integrity of their environments.
- Develops leaders using evidence-based, practical, applied strategies.
- When things go wrong, NCPRE provides strategies for salvage and for returning to business as usual.
- Helps emerging professionals, from high school on, understand the importance of ethical and professional choices, and to lead effectively.
NCPRE offers online instruction in professional development and ethical decision-making. The 4-course series, “Professional IQ: Preventing and Solving Problems at Work,” is targeted towards people seeking or starting new jobs, who are facing dilemmas in their current job, who have just been promoted or who want to be more marketable for a promotion, or who possesses strong technical skills and want to develop leadership skills. The course series is available now as a Massive Open Online Course (MOOC) through Coursera.
Professional and Social Issues Lab (PSI)
The PSI Lab is housed in the Division of General Medical Sciences in the Department of Medicine at the Washington University School of Medicine in St. Louis. With funding from the Institute of Clinical and Translational Sciences (ICTS) and Washington University, PSI is able to provide a limited number of hours of consultation on study designs for projects that explore issues surround ethics, regulations, and professionalism. In general, PSI seeks projects in which they can become co-investigators, providing the support of faculty and staff throughout the life of a project. PSI is affiliated with CCRE (see CCRE under Academia) and the Professionalism & Integrity (P.I.) in Research Program. http://professionalandsocial.org/research-services/
PSI is currently working on projects in the following areas:
- A Recruitment and Assessment Project for the Professionalism and Integrity in Research Program
- Preventing Ethical Disasters in the Practice of Medicine
- How Do Clinical Research Coordinators Learn About Good Clinical Practice?
- Sharing Qualitative Research Data: Identifying and Addressing Ethical and Practical Barriers
May 2017 Update:
Center for Clinical Research Ethics (CCRE), Washington University in St. Louis and Saint Louis University
“Dr. Alison Antes, Assistant Director of CCRE and Assistant Professor in the Division of General Medical Sciences at Washington University School of Medicine, received a K01 career development grant from the National Human Genome Research Institute that will allow Dr. Antes to, ‘develop a program that focuses on the management and leadership practices of researchers as they relate to integrity and societal impact in genomic research. The purpose of this research is to understand the challenges and needs of genomic researchers with regard to these issues focusing in particular on management practices and leadership practices employed by researchers in navigating issues of research integrity and the societal impact of their work. In the first phase of this research, we will conduct semi-structured interviews with a national sample of federally-funded researchers nominated as exemplars of professionalism and integrity to identify the management and leadership practices they employ to foster integrity and impact in their research. In a second phase, we will survey a national sample of genomic scientists regarding the management and leadership practices identified in the interviews. We will ask researchers to report on the management and leadership practices they utilize, their confidence in performing each practice, and their interest in training or support materials. In the third phase of this research, we will conduct focus groups with researchers to understand their preferences regarding types of programs and messages about tailored management and leadership programs for scientists. Next, we will develop messages and test their appeal among with researchers using a survey. These findings will facilitate effective messaging about future initiatives to encourage the participation of researchers. This research will lay the groundwork for future research and practical resources and tools to assist researchers with meeting the various ethical, legal, and social demands of their work, thereby contributing to the quality, integrity, and social impact of genomic science.” http://ethicsresearchcore.org/home/spotlight/
Please help keep this document current by pointing out areas that need to be expanded or updated or additional organizations that should be included. Please send comments or suggestions to email@example.com with the subject line “Scientific Integrity.”
Last updated December 2018
Citation: Kretser, A., Murphy, D., & Dwyer, J. (2017). Scientific integrity resource guide: Efforts by federal agencies, foundations, nonprofit organizations, professional societies, and academia in the United States. Critical Reviews in Food Science and Nutrition, 57(1), 163–180. http://doi.org/10.1080/10408398.2016.1221794
This publication is part of ILSI North America’s work on Scientific Integrity.