Authors: (1) Renato P. dos Santos, CIAGE – Centre for Generative Artificial Intelligence in Cognition and Education. Table of Links Abstract and Introduction Materials And Methods Results and Analyses Prompts and generated texts Conceptualizing chemical reactions Deepening on understanding of chemical reactions Question about combustion Question about a graph of gases turning into water over time Question about the difference between atoms, molecules, and moles Deepening on the concept of mole Question about changing of state Question about an animated representation of water molecules undergoing phase changes Question about plasma, a state of matter Question about chemical bondings Question about illustration of chemical bonds Question about the essence of the type of chemical bonding Further analysis Conclusions Limitations of the study and possible future studies Author Contributions, Conflicts of interest, Acknowledgements, and References Limitations of the study and possible future studies Despite its inherent limitations, including its single case design and the potential for bias, the exploratory depth of the study uncovered hidden potential within these systems, even amid serious concerns about generalizability. Future research could include: • Refining the crafting of prompts. • Exploring new features of these and other GenAIbots being introduced with increasing frequency. • Establishing standardised benchmarks to evaluate and compare chatbots and AI systems' performance, accuracy, and reliability. • Conducting long-term studies to observe the evolution of chatbots' capabilities and their impact on user interactions over time. • Conducting research with real students in classroom settings and beyond to assess these AI systems' practical educational applications and challenges. • Investigating chatbots' learning and adaptation capabilities to individual user needs and preferences over time. • Researching the integration of multimodal inputs (e.g., text, voice, image) to enhance chatbot capabilities and user interaction experiences. When implementing GenAIbots in Chemistry learning, it is crucial to evaluate benefits and drawbacks judiciously, ensuring accurate information delivery and considering the implications of reduced human interaction. These concerns may be alleviated by integrating GenAIbots with other educational tools or activities promoting collaborative dialogue among learners. Author Contributions The author confirms sole responsibility for the following: study conception and design, data collection, analysis and interpretation of results, and manuscript preparation. Conflicts of interest There are no conflicts to declare. Acknowledgements The authors warmly acknowledge Melanie Swan for her invaluable suggestion, which led to the transition from using the term 'objects-to-think-with' to 'agents-to-thinkwith'. References Adiguzel, T., Kaya, M. H., & Cansu, F. K. (2023). Revolutionising education with AI: Exploring the transformative potential of ChatGPT. Contemporary Educational Technology, 15(3), ep429. https://doi.org/10.30935/cedtech/13152 Baidoo-Anu, D., & Owusu Ansah, L. (2023). Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.4337484 Bardin, L. (1977). L’analyse de contenu. PUF - Presses Universitaires de France. Bitzenbauer, P. (2023). ChatGPT in physics education: A pilot study on easy-to-implement activities. Contemporary Educational Technology, 15(3), ep430. https://doi.org/10.30935/cedtech/13176 Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., … Amodei, D. (2020). Language Models are Few-Shot Learners. ArXiv, 2005.14165. http://arxiv.org/abs/2005.14165 Castro Nascimento, C. M., & Pimentel, A. S. (2023). Do Large Language Models Understand Chemistry? A Conversation with ChatGPT. Journal of Chemical Information and Modeling, 63(6), 1649–1655. https://doi.org/10.1021/acs.jcim.3c00285 David, E. (2023, September 21). Microsoft to add DALL-E 3 to Bing Chat. The Verge; Vox Media. https://www.theverge.com/2023/9/21/23873690/microsoft-new-ai-features-bingsearch-shopping-dall-e-3 Dewi, C. A., Pahriah, P., & Purmadi, A. (2021). The Urgency of Digital Literacy for Generation Z Students in Chemistry Learning. International Journal of Emerging Technologies in Learning, 16(11), 88–103. https://doi.org/10.3991/ijet.v16i11.19871 Dunlop, L., Hodgson, A., & Stubbs, J. E. (2020). Building capabilities in chemistry education: happiness and discomfort through philosophical dialogue in chemistry. Chemistry Education Research and Practice, 21(1), 438–451. https://doi.org/10.1039/C9RP00141G Flavell, J. H. (1976). Metacognitive aspects of problem-solving. In L. B. Resnick (Ed.), The Nature of Intelligence (pp. 231–236). Lawrence Erlbaum. Floridi, L., & Chiriatti, M. (2020). GPT-3: Its Nature, Scope, Limits, and Consequences. Minds and Machines, 30(4), 681–694. https://doi.org/10.1007/s11023-020-09548-1 Franciscu, S. (2023). ChatGPT: A Natural Language Generation Model for Chatbots. https://doi.org/10.13140/RG.2.2.24777.83044 Gregorcic, B., & Pendrill, A.-M. (2023). ChatGPT and the frustrated Socrates. Physics Education, 58(3), 035021. https://doi.org/10.1088/1361-6552/acc299 Latour, B. (1991). Nous n’avons jamais été modernes: Essai d’anthropologie symétrique. La Découverte. Leon, A. J., & Vidhani, D. (2023). ChatGPT Needs a Chemistry Tutor, Too. Journal of Chemical Education. https://doi.org/10.1021/acs.jchemed.3c00288 Liu, F., Budiu, R., Zhang, A., & Cionca, E. (2023, October 1). ChatGPT, Bard, or Bing Chat? Differences Among 3 Generative-AI Bots. https://www.nngroup.com/articles/ai-botcomparison/ Metz, C., & Hsu, T. (2023, September 20). ChatGPT Can Now Generate Images, Too. New York Times - Technology. https://www.nytimes.com/2023/09/20/technology/chatgpt-dalle3-images-openai.html Mishra, A., Soni, U., Arunkumar, A., Huang, J., Kwon, B. C., & Bryan, C. (2023). PromptAid: Prompt Exploration, Perturbation, Testing and Iteration using Visual Analytics for Large Language Models. ArXiv, 2304.01964. https://doi.org/10.48550/arXiv.2304.01964 Mollick, E. R. (2023, April 26). A guide to prompting AI (for what it is worth): A little bit of magic,but mostly just practice. One Useful Thing Blog. https://www.oneusefulthing.org/p/a-guidetoprompting-ai-for-what Okonkwo, C. W., & Ade-Ibijola, A. (2021). Chatbots applications in education: A systematic review. Computers and Education: Artificial Intelligence, 2, 100033. https://doi.org/10.1016/j.caeai.2021.100033 OpenAI. (2023). GPT-4 Technical Report. OpenAI. https://cdn.openai.com/papers/gpt-4.pdf Papert, S. A. (1980). Mindstorms - Children, Computers and Powerful Ideas. Basic Books. http://www.arvindguptatoys.com/arvindgupta/mindstorms.pdf Permatasari, M. B., Rahayu, S., & Dasna, W. (2022). Chemistry Learning Using Multiple Representations: A Systematic Literature Review. J.Sci.Learn.2022, 5(2), 334–341. https://doi.org/10.17509/jsl.v5i2.42656 Pimentel, A., Wagener, A., Silveira, E. F. da, Picciani, P., Salles, B., Follmer, C., & Jr., O. N. O. (2023). Challenging ChatGPT with Chemistry-Related Subjects. ChemRxiv. https://doi.org/10.26434/chemrxiv-2023-xl6w3 Schlosser, M. (2019). Agency. In The Stanford Encyclopedia of Philosophy. https://plato.stanford.edu/archives/win2019/entries/agency/ Swan, M. (2015, January 21). We Should Consider the Future World as one of Multi-Species Intelligence. Response to The Edge Question 2015: What Do You Think About Machines That Think?; Edge.org. http://edge.org/response-detail/26070 Taylor, C. A., Hogarth, H., Hacking, E. B., & Bastos, E. (2022). Posthuman Object Pedagogies: Thinking with Things to Think with Theory for Innovative Educational Research. Cultural and Pedagogical Inquiry, 14(1), 206–221. https://doi.org/10.18733/cpi29662 Timilsena, N. P., Maharjan, K. B., & Devkota, K. M. (2022). Teachers’ And Students’ Experiences In Chemistry Learning Difficulties. Journal of Positive School Psychology, 6(10), 2856–2867. https://www.journalppw.com/index.php/jpsp/article/view/13764 Tümay, H. (2016). Reconsidering learning difficulties and misconceptions in chemistry: emergence in chemistry and its implications for chemical education. Chemistry Education Research and Practice, 17(2), 229–245. https://doi.org/10.1039/C6RP00008H Turkle, S. (1984). The Second Self: Computers and the Human Spirit. Simon and Schuster. Yin, R. K. (2011). Applications of Case Study Research. SAGE Publications, Inc. This paper is under CC BY-SA 4.0 DEED license. available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Overcoming Limitations in AI Chatbot Research for Future Educational Impact

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

102 Languages, One Model: The Multimodal AI Breakthrough You Need to Know

Enhancing Chemistry Learning with ChatGPT, Bing Chat, Bard, and Claude as Agents-to-Think-With

Analyzing GenAIbots' Efficacy in Chemistry Education

GenAIbot Conversations in British vs. American English

Initiating Socratic Dialogues with GenAIbots in Chemistry Education

102 Languages, One Model: The Multimodal AI Breakthrough You Need to Know

Enhancing Chemistry Learning with ChatGPT, Bing Chat, Bard, and Claude as Agents-to-Think-With

Analyzing GenAIbots' Efficacy in Chemistry Education

GenAIbot Conversations in British vs. American English

Initiating Socratic Dialogues with GenAIbots in Chemistry Education

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps