Author: (1) David M. Markowitz, Department of Communication, Michigan State University, East Lansing, MI 48824. Editor's note: This is part 6 of 10 of a paper evaluating the effectiveness of using generative AI to simplify science communication and enhance public trust in science. The rest of the paper can be accessed via the table of links below. Table of Links Abstract The Benefits of Simple Writing The Current Work Study 1a: Method Study 1a: Results Study 1b: Method Study 1b: Results Study 2: Method Study 2: Results General Discussion, Acknowledgements, and References Study 1b: Method An a priori power analysis using a small effect size (Cohen’s d = 0.20) powered at 80% suggested 788 cases were needed to detect a difference between GPT significance statements and PNAS significance statements. Therefore, a random selection of 800 abstracts from Study 1a was used in this study. Using the OpenAI API, the large language model GPT-4 was fed each abstract individually and given the following prompt, which was drawn from descriptions of what PNAS authors should communicate in their significance statements (31): The following text is an academic abstract from the journal Proceedings of the National Academy of Sciences. Based on this abstract, create a significance statement. This statement should provide enough context for the paper’s implications to be clear to readers. The statement should not contain references and should avoid numbers, measurements, and acronyms unless necessary. It should explain the significance of the research at a level understandable to an undergraduate-educated scientist outside their field of specialty. Finally, it should include no more than 120 words. Write the significance statement here: The same text analytic process was performed on these data as Study 1a. Each GPT significance statement received scores based on common words (LIWC dictionary category), analytic writing (LIWC analytic writing category), and readability (Flesch Reading Ease). This paper is available on arxiv under CC BY 4.0 DEED license. Author: (1) David M. Markowitz, Department of Communication, Michigan State University, East Lansing, MI 48824. Author: Author: (1) David M. Markowitz, Department of Communication, Michigan State University, East Lansing, MI 48824. Editor's note: This is part 6 of 10 of a paper evaluating the effectiveness of using generative AI to simplify science communication and enhance public trust in science. The rest of the paper can be accessed via the table of links below. Editor's note: This is part 6 of 10 of a paper evaluating the effectiveness of using generative AI to simplify science communication and enhance public trust in science. The rest of the paper can be accessed via the table of links below. Editor's note: This is part 6 of 10 of a paper evaluating the effectiveness of using generative AI to simplify science communication and enhance public trust in science. The rest of the paper can be accessed via the table of links below. Table of Links Abstract The Benefits of Simple Writing The Current Work Study 1a: Method Study 1a: Results Study 1b: Method Study 1b: Results Study 2: Method Study 2: Results General Discussion, Acknowledgements, and References Abstract Abstract The Benefits of Simple Writing The Benefits of Simple Writing The Current Work The Current Work Study 1a: Method Study 1a: Method Study 1a: Results Study 1a: Results Study 1b: Method Study 1b: Method Study 1b: Results Study 1b: Results Study 2: Method Study 2: Method Study 2: Results Study 2: Results General Discussion, Acknowledgements, and References General Discussion, Acknowledgements, and References Study 1b: Method An a priori power analysis using a small effect size (Cohen’s d = 0.20) powered at 80% suggested 788 cases were needed to detect a difference between GPT significance statements and PNAS significance statements. Therefore, a random selection of 800 abstracts from Study 1a was used in this study. Using the OpenAI API, the large language model GPT-4 was fed each abstract individually and given the following prompt, which was drawn from descriptions of what PNAS authors should communicate in their significance statements (31): The following text is an academic abstract from the journal Proceedings of the National Academy of Sciences. Based on this abstract, create a significance statement. This statement should provide enough context for the paper’s implications to be clear to readers. The statement should not contain references and should avoid numbers, measurements, and acronyms unless necessary. It should explain the significance of the research at a level understandable to an undergraduate-educated scientist outside their field of specialty. Finally, it should include no more than 120 words. Write the significance statement here: The following text is an academic abstract from the journal Proceedings of the National Academy of Sciences. Based on this abstract, create a significance statement. This statement should provide enough context for the paper’s implications to be clear to readers. The statement should not contain references and should avoid numbers, measurements, and acronyms unless necessary. It should explain the significance of the research at a level understandable to an undergraduate-educated scientist outside their field of specialty. Finally, it should include no more than 120 words. Write the significance statement here: The same text analytic process was performed on these data as Study 1a. Each GPT significance statement received scores based on common words (LIWC dictionary category), analytic writing (LIWC analytic writing category), and readability (Flesch Reading Ease). This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv
