ChatGPT is an artificial intelligence (AI) chatbot that uses a natural language processing (NLP) technique to answer user questions. Use of ChatGPT has become a widely debated topic among researchers, educators, and now, journal publishers.

Since the public release of ChatGPT last November, ethicists and editors have weighed boundaries around its use for writing scientific manuscripts. Information accuracy, sourcing, and the use of AI-generated text are some of the concerns that have been raised. Responses so far have varied from outright bans to insisting upon clear notations to distinguish human-generated text from AI-generated text.

At the forefront of these discussions has been NIEHS bioethicist David Resnik, J.D., Ph.D., who is an associate editor at the Accountability in Research journal. Resnik and others from the journal’s team published an editorial that proposes a new policy regarding use of AI-generated text (see sidebar for proposed guidelines).

“We started having conversations about this in December, before anybody had published any papers naming ChatGPT as an author, which has since happened,” Resnik said. “We were thinking, ‘This is a big issue. We want to get in front of it.’”

Transparency, accountability are key

Honesty, transparency, openness, rigor, accountability, and objectivity are values important in research, according to Resnik. One problem with using AI-generated text is that AI cannot be held accountable or responsible for the work. “Some human person has to take accountability for the text, so we emphasize transparency, honesty, and full disclosure of use of these types of capabilities.”

Several other journals have now proposed or established new policies regarding co-authorship with AI programs or the use of AI-generated text in scientific manuscripts. For example, the family of Science journals announced in an editorial that they have updated policies regarding AI authorship and the use of AI-generated text in manuscripts submitted to their journals. The policy reads as follows.

“Text generated from AI, machine learning, or similar algorithmic tools cannot be used in papers published in Science journals.... In addition, an AI program cannot be an author of a Science journal paper. A violation of this policy constitutes scientific misconduct.”

Nature’s new policy also prohibits AI co-authorship across its journals, and the JAMA Network of journals discourages use and labels AI-generated text as reproduced material. JAMA requires authors properly report and cite use of such technologies by clearly describing any AI-generated content, and citing the model or tool used and manufacturer name. The policy also states that use of AI-generated text does not qualify the model or tool as a co-author and that all human authors are responsible for the integrity of the content generated and used.

Resnik suggests scientists regularly check for updated policies at journals where they submit their work.

Predictive, but not dependable

But authorship is just one of many concerns as this technology continues to emerge for widespread public use. For example, ChatGPT—which is an acronym for generative pre-trained transformer—is an NLP program that computationally understands language. It works by predicting a cogent response to prompts.

“AI are very good at manipulating natural human language to predict what an appropriate response would be,” Resnik explained. “They are not really connected to the human social world like we are though, so it is all just a manipulation, a game, or a task. There is no connection to reality or facts at all. It can do some amazing stuff, but it can also get things really wrong.”

“Companies warn us that these tools can hallucinate,” he added. “I have seen them make up stuff, even made-up references. So, humans must provide close oversight and ensure those values in research remain intact when using these new technologies. They will get better with time, but we are always going to have this problem.”

New age or old news?

Maintaining core values in research while also adapting to changes in technology is not anything new. “I have been in this field for a long time, and I am old enough to have seen some big changes,” Resnik said. “I was conducting research before the Internet became a big deal, and all the technology that has emerged since.”

Mentorship, oversight, and clear communication still will be needed, Resnik reiterated.

“It is just really important to understand that ChatGPT is a tool like any other, and it can have good and bad uses. At the end of the day, we still have to have a human being in charge because the real danger here would be putting the AI in charge,” Resnik said.

According to NIEHS Director of Environmental Science Cyberinfrastructure David Fargo, Ph.D., the federal information technology security compliance framework for these typically cloud-hosted resources remains an area of active development.

Citation: Hosseini M, Rasmussen LM, Resnik DB. 2023. Using AI to write scholarly publications. Account Res 1-9.