Analyzing the Role of AI Chatbots in Active and Personalized Education

Table of Links

Abstract and Introduction

Materials And Methods

Results and Analyses

Prompts and generated texts

Conceptualizing chemical reactions

Deepening on understanding of chemical reactions

Question about combustion

Question about a graph of gases turning into water over time

Question about the difference between atoms, molecules, and moles

Deepening on the concept of mole

Question about changing of state

Question about an animated representation of water molecules undergoing phase changes

Question about plasma, a state of matter

Question about chemical bondings

Question about illustration of chemical bonds

Question about the essence of the type of chemical bonding

Further analysis

Conclusions

Limitations of the study and possible future studies

Author Contributions, Conflicts of interest, Acknowledgements, and References

Further analysis

This study's findings diverge from those of Gregorcic and Pendrill (2023), who contended that despite rectifying errors and contradictions via Socratic dialogue, ChatGPT did not meet the necessary criteria to function as a physics tutor. This disparity could stem from using an older, less advanced iteration of ChatGPT, specifically the GPT-3 model introduced by OpenAI in 2020 (Brown et al., 2020).

Our research indicates that the current versions of ChatGPT, Bing Chat, Bard, and Claude can act as effective objects-to-think-with, as per Papert's concept (1980, p. 11), cultivating an engaging, interactive, and inclusive learning environment. These AI tools can support creativity, collaboration, exploration, critical thinking, and problem-solving, potentially deepening understanding of the subject matter. This relates to the comparison between ChatGPT, Bard, and Bing Chat in the study of Liu et al. (2023).

The overall success of this educational journey, with GenAIbots serving as agents to think with, highlights a few effective teaching strategies that can facilitate a profound comprehension of the involved concepts. A significant part of the process involved positive reinforcement, as GenAIbots acknowledged students' observations as enjoyable, validating their experiences and stimulating their curiosity. Furthermore, following Gregorcic and Pendrill (2023), we adopted what could be seen as a Socratic-like dialogue strategy, prompting students to think critically and actively engage in their learning rather than merely providing them with answers. GenAIbots also capitalised on the student’s existing knowledge by inquiring about their understanding of topics like weight and gravity, thereby establishing a meaningful bridge between new and previously learned information.

Building upon Bitzenbauer's preliminary study (2023) that highlighted the potential of expansive language model-based chatbots, including ChatGPT, to enrich learning experiences, and Adiguzel et al.'s (2023) assertion of ChatGPT's transformative potential in education via personalised instruction and real-time feedback, our research emphasises the benefits of AI integration in learning environments. AI-driven chatbots like ChatGPT, Bing Chat, Bard, Claude, and other generative models might serve as advanced agents for addressing challenging learning scenarios, expanding students' knowledge incrementally, and providing relevant explanations, examples, and analogies to support their learning. This study highlights the potential of AI-driven tools in promoting active learning, personalised instruction, and conceptual understanding in Chemistry. It underscores the value of AI in enhancing educational experiences, concluding that GenAIbots can serve as innovative platforms for exploring challenging educational scenarios.

Nevertheless, our analysis reveals that ChatGPT and Claude consistently provided more comprehensive, detailed, and accurate answers than Bing Chat, which tended to give shorter and less informative responses. ChatGPT-4 was able to adapt responses to students' questions, address misconceptions, and offer customised explanations. ChatGPT was also more adept at addressing the nuances and subtleties of the questions, recognising the context, and engaging with the user's unique perspective or interpretation. On the other hand, Bing Chat often failed to address these nuances and provided less contextual information. Overall, ChatGPT and Claude demonstrated a superior understanding of the subject matter and a more remarkable ability to convey complex scientific concepts in an accessible manner.

The study also highlighted the need for comprehensive educator training before integrating these tools into classrooms. Adequate training empowers teachers to guide students effectively in articulating their thoughts and constructing solid arguments. This preparation would allow educators to assess GenAIbots' performance when faced with compelling counterexamples, fostering productive discussions among colleagues.

Finally, it is essential to mention that the results of this experience revealed significant variance in the performances of ChatGPT, Bing Chat, Bard, and Claude. This result agrees with Franciscu (2023), in which ChatGPT, a cutting-edge natural language generation model, excels at producing human-like responses suitable for chatbots. Based on the detailed analysis above, despite some fundamental similarities, ChatGPT notably outperformed BingChat, with Bard and Claude closely shadowing ChatGPT's performance and displaying comparable proficiency levels between them, as described below and summarised in Table 2.

1. ChatGPT stands out for its consistent Personalization & Personable Touch through positive reinforcement and feedback mechanisms. It employs effective Instructional Strategies, often breaking down explanations systematically, and frequently engages users with Engagement & Interactivity by soliciting feedback. With a flair for the Use of Analogy & Comparative Illustrations, ChatGPT translates abstract concepts into relatable scenarios. Its Detail Depth & Content Recommendation is tailored and comprehensive, and it occasionally ventures into Meta-discussion & Perspective Framing to provide broader contexts.

2. Claude's strengths lie predominantly in Personalization & Personable Touch and Engagement & Interactivity, making learning feel like an interactive dialogue. Offering straightforward explanations, it leans heavily on the Use of Analogy & Comparative Illustrations, especially when building upon user insights. While its Instructional Strategies provide clarity, the depth can vary, catering to concise and elaborate queries. Claude's penchant for Meta-discussion & Perspective Framing emphasises the broader significance of concepts, enhancing the overall learning experience.

3. Bard's responses resemble a classic Instructional Strategies approach, akin to a conventional classroom style. Focusing on factual and structured presentations, it often employs the Use of Analogy & Comparative Illustrations to elucidate concepts. Personalization & Personable Touch is evident, especially when recognising user inputs and providing deeper insights, fostering Engagement & Interactivity. Bard occasionally integrates Meta-discussion & Perspective Framing to highlight broader applications.

4. BingChat emphasises Detail Depth & Content Recommendation by often directing users to external resources. Adopting a more correctional Reinforcement & Feedback approach, it points out inaccuracies in user inputs, which can sometimes overshadow its Personalization & Personable Touch. While BingChat provides structured Instructional Strategies, its reliance on external links might diminish direct Engagement & Interactivity. Its Use of Analogy & Comparative Illustrations is present, albeit less pronounced.

This ranking is based on the analysis presented earlier. It provides a comparative grade for each AI across different categories, but the specific grading might be subjective and could vary with different questions, contexts or even across sessions. The preference might also be subjective depending on individual learning styles and needs.

Using GenAIbots as educational tools, teachers might create more engaging and personalised learning experiences. This approach could potentially support a deeper understanding of complex concepts in Chemistry and perhaps other subjects.