Table of Links Abstract and 1 Introduction 2. Background 2.1 Effective Tutoring Practice 2.2 Feedback for Tutor Training 2.3 Sequence Labeling for Feedback Generation 2.4 Large Language Models in Education 3. Method 3.1 Dataset and 3.2 Sequence Labeling 3.3 GPT Facilitated Sequence Labeling 3.4 Metrics 4. Results 4.1 Results on RQ1 4.2 Results on RQ2 5. Discussion 6. Limitation and Future Works 7. Conclusion 8. Acknowledgments 9. References APPENDIX A. Lesson Principles B. Input for Fine-Tunning GPT-3.5 C. Scatter Matric of the Correlation on the Outcome-based Praise D. Detailed Results of Fine-Tuned GPT-3.5 Model's Performance 2.3 Sequence Labeling for Feedback Generation Sequence labeling, a fundamental task in natural language processing (NLP), plays a pivotal role in identifying and categorizing key segments of text according to predefined labels [26]. To elucidate the mechanism of sequence labeling, we consider Named Entity Recognition (NER) as a representative subtask, which is closed to the task in our study. NER seeks to automatically detect and classify named entities—words or phrases with specific attributes—into categories such as person, organization, and location [26, 36]. For instance, in the sentence “John said that Pittsburgh is wonderful in the winter,” the terms “John”, “Pittsburgh”, and “winter” would be labeled as Person, Location, and Time, respectively, showcasing how NER distinguishes and categorizes entities within a textual context. Our study extends the application of sequence labeling to identify and highlight components related to different types of praise within tutor responses. This process involves discerning specific words or phrases that signify the kind of praise being used, thereby offering tutors insight into their feedback practices. For example, “You are doing great.”, the phrase “doing great” in this context is identified as an outcome-based praise. Leveraging sequence labeling allows our AI model to spotlight such instances of praise, enabling the provision of nuanced, explanatory feedback. An example of such feedback might be, “Saying “doing great” is praising the student for the outcome. You should focus on praising the students for their effort and process towards learning. Do you want to try responding again?” This approach facilitates the generation of targeted, template-based feedback for tutors. Notably, while previous research has explored sequence labeling techniques for similar purposes [40], the accuracy of their proposed models in precisely identifying and categorizing feedback elements remains a challenge. This limitation underscores the need for leveraging more advanced models to provide accurate, informative feedback to tutors. This paper is available on arxiv under CC BY 4.0 DEED license. Authors:
(1) Jionghao Lin, Carnegie Mellon University (jionghal@cs.cmu.edu);
(2) Eason Chen, Carnegie Mellon University (easonc13@cmu.edu);
(3) Zeifei Han, University of Toronto (feifei.han@mail.utoronto.ca);
(4) Ashish Gurung, Carnegie Mellon University (agurung@andrew.cmu.edu);
(5) Danielle R. Thomas, Carnegie Mellon University (drthomas@cmu.edu);
(6) Wei Tan, Monash University (wei.tan2@monash.edu);
(7) Ngoc Dang Nguyen, Monash University (dan.nguyen2@monash.edu);
(8) Kenneth R. Koedinger, Carnegie Mellon University (koedinger@cmu.edu). Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2. Background 2.1 Effective Tutoring Practice 2.1 Effective Tutoring Practice 2.2 Feedback for Tutor Training 2.2 Feedback for Tutor Training 2.3 Sequence Labeling for Feedback Generation 2.3 Sequence Labeling for Feedback Generation 2.4 Large Language Models in Education 2.4 Large Language Models in Education 3. Method 3.1 Dataset and 3.2 Sequence Labeling 3.1 Dataset and 3.2 Sequence Labeling 3.3 GPT Facilitated Sequence Labeling 3.3 GPT Facilitated Sequence Labeling 3.4 Metrics 3.4 Metrics 4. Results 4.1 Results on RQ1 4.1 Results on RQ1 4.2 Results on RQ2 4.2 Results on RQ2 5. Discussion 5. Discussion 6. Limitation and Future Works 6. Limitation and Future Works 7. Conclusion 7. Conclusion 8. Acknowledgments 8. Acknowledgments 9. References 9. References APPENDIX APPENDIX A. Lesson Principles A. Lesson Principles B. Input for Fine-Tunning GPT-3.5 B. Input for Fine-Tunning GPT-3.5 C. Scatter Matric of the Correlation on the Outcome-based Praise C. Scatter Matric of the Correlation on the Outcome-based Praise D. Detailed Results of Fine-Tuned GPT-3.5 Model's Performance D. Detailed Results of Fine-Tuned GPT-3.5 Model's Performance 2.3 Sequence Labeling for Feedback Generation Sequence labeling, a fundamental task in natural language processing (NLP), plays a pivotal role in identifying and categorizing key segments of text according to predefined labels [26]. To elucidate the mechanism of sequence labeling, we consider Named Entity Recognition (NER) as a representative subtask, which is closed to the task in our study. NER seeks to automatically detect and classify named entities—words or phrases with specific attributes—into categories such as person, organization, and location [26, 36]. For instance, in the sentence “ John said that Pittsburgh is wonderful in the winter ,” the terms “ John ”, “ Pittsburgh ”, and “ winter ” would be labeled as Person, Location, and Time, respectively, showcasing how NER distinguishes and categorizes entities within a textual context. John said that Pittsburgh is wonderful in the winter John Pittsburgh winter Our study extends the application of sequence labeling to identify and highlight components related to different types of praise within tutor responses. This process involves discerning specific words or phrases that signify the kind of praise being used, thereby offering tutors insight into their feedback practices. For example, “You are doing great.”, the phrase “ doing great ” in this context is identified as an outcome-based praise. Leveraging sequence labeling allows our AI model to spotlight such instances of praise, enabling the provision of nuanced, explanatory feedback. An example of such feedback might be, “Saying “doing great” is praising the student for the outcome. You should focus on praising the students for their effort and process towards learning. Do you want to try responding again? ” This approach facilitates the generation of targeted, template-based feedback for tutors. doing great “Saying “doing great” is praising the student for the outcome. You should focus on praising the students for their effort and process towards learning. Do you want to try responding again? Notably, while previous research has explored sequence labeling techniques for similar purposes [40], the accuracy of their proposed models in precisely identifying and categorizing feedback elements remains a challenge. This limitation underscores the need for leveraging more advanced models to provide accurate, informative feedback to tutors. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv Authors: (1) Jionghao Lin, Carnegie Mellon University (jionghal@cs.cmu.edu); (2) Eason Chen, Carnegie Mellon University (easonc13@cmu.edu); (3) Zeifei Han, University of Toronto (feifei.han@mail.utoronto.ca); (4) Ashish Gurung, Carnegie Mellon University (agurung@andrew.cmu.edu); (5) Danielle R. Thomas, Carnegie Mellon University (drthomas@cmu.edu); (6) Wei Tan, Monash University (wei.tan2@monash.edu); (7) Ngoc Dang Nguyen, Monash University (dan.nguyen2@monash.edu); (8) Kenneth R. Koedinger, Carnegie Mellon University (koedinger@cmu.edu). Authors: Authors: (1) Jionghao Lin, Carnegie Mellon University (jionghal@cs.cmu.edu); (2) Eason Chen, Carnegie Mellon University (easonc13@cmu.edu); (3) Zeifei Han, University of Toronto (feifei.han@mail.utoronto.ca); (4) Ashish Gurung, Carnegie Mellon University (agurung@andrew.cmu.edu); (5) Danielle R. Thomas, Carnegie Mellon University (drthomas@cmu.edu); (6) Wei Tan, Monash University (wei.tan2@monash.edu); (7) Ngoc Dang Nguyen, Monash University (dan.nguyen2@monash.edu); (8) Kenneth R. Koedinger, Carnegie Mellon University (koedinger@cmu.edu).

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Sequence Labeling for Automated Feedback in Tutor Training

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Acknowledgments: Funding and Support for Explanatory Feedback Research

Artificial Parenting: A Chance to Feel for Real

Is Artificial Intelligence Turning Human Learning a Thing of the Past?

AI, Absurdity, and Learning Faster

Automated Feedback Systems for Enhanced Tutor Training

GPT Models for Sequence Labeling: Prompt Engineering & Fine-tuning

Acknowledgments: Funding and Support for Explanatory Feedback Research

Artificial Parenting: A Chance to Feel for Real

Is Artificial Intelligence Turning Human Learning a Thing of the Past?

AI, Absurdity, and Learning Faster

Automated Feedback Systems for Enhanced Tutor Training

GPT Models for Sequence Labeling: Prompt Engineering & Fine-tuning

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps