Authors:
(1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland;
(2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland;
(7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland;
(8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland;
(9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA;
(10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland;
(12) EngD. Table of Links Abstract and Introduction Related Work Methods Results and Discussion Limitations Conclusion, Acknowledgment, and References Related work Cognitive Load Detection. To obtain a good and unobtrusive cognitive load detector, which could facilitate both development of timely context-aware reminders and the evaluation of application use effort in the wild[8], researchers considered the utilization of wearable devices and machine learning. Morkova et al.[12] used a Shimmer[3] GSR+ device attached to the participant's two fingers and an ear lobe to capture galvanic skin responses (GSR), electrocardiograms (ECG) and PPG signal of individuals during cognitively demanding tasks and neutral condition (CLAS dataset). The authors trained a person-specific Support Vector Machine (SVM) model on features extracted from GSR either paired with features extracted from ECG or PPG recordings to predict subjects' concentration. They reported average classification performance of 78% (ECG+GSR) and 74% (PPG+GSR) and commented that these performance differences were not significant and do not justify the use of ECG. Consumer-grade wearables, employed in just-in-time interventions, due to high cost and low practicality rarely are equipped with ECG and GSR sensors commonly available in devices used in experimental conditions[15]. Therefore, we focus on developing a cognitive load detector that utilizes PPG signal only, that can be captured by smartwatches. Lisowska et al.[11] use a PPG signal from CLAS dataset[12] to detect cognitive load. The authors compared the performance of SVM with a different number of features vs. simple 1D Convolutional Neural Network (1D CNN). The latter performed better, however, the achieved average cognitive load classification accuracy reached only 60%. The authors tried to improve cognitive load detection through semi-supervised model training leveraging additional unlabeled signal, however, no notable performance gains were reached. We base our solution on this approach and first investigate the limitation of the simple 1D CNN and later suggest a potential change to the training approach. We consider between-task transfer learning. Our investigation is based on the following observations: Even though deep learning approaches have been shown to outperform traditional methods utilizing handcrafted features such as SVMs in a range of applications[14], including cognitive load detection from PPG signal[8], CNN-based models are known to be data hungry and in low data regimes, they tend to be pre-trained to offer better performance[20].


Cognitive load annotations are not easy to gather and there are few datasets utilizing physiological signals from wrist-worn devices designed to capture that state. On the other hand, stress classification is a commonly tackled problem and high-quality publicly available physiological signal datasets, such as WESAD[16] (Wearable Stress and Affect Detection), provide signal paired with labels convenient for ML training. Well-being Survey Gamification. Gamification refers to the use of game elements such as progress tracking, avatars, points, levels and challenges in a non-game context. Carlier et al.[2] conducted a comparative study of user experience with gamified and non-gamified surveys and reported that they found no difference between conditions in terms of data quality gathered but respondents of gamified surveys perceived the time taken to respond to questions as shorter than respondents of non-gamified surveys. We consider real time taken to answer the question and time spent in high cognitive load. In terms of game elements, Carlier et al. used progress tracking in gamified version and the remaining game elements were personalised to each individual. In our previous work, we investigated the personalisation of gamification design of mHealth surveys and found that the selected preferred game elements in that context do not vary strongly between individuals1 . Therefore in this work, we use a uniform version of gamified survey app across all participants selecting game elements associated with the Player archetype such as progress tracking, points, rewards and levels. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland; (2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland; (7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland; (8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland; (9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; (10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland; (12) EngD. Authors: Authors: (1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland; (2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland; (7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland; (8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland; (9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; (10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland; (12) EngD. Table of Links Abstract and Introduction Abstract and Introduction Related Work Related Work Methods Methods Results and Discussion Results and Discussion Limitations Limitations Conclusion, Acknowledgment, and References Conclusion, Acknowledgment, and References Related work Cognitive Load Detection. To obtain a good and unobtrusive cognitive load detector, which could facilitate both development of timely context-aware reminders and the evaluation of application use effort in the wild[8], researchers considered the utilization of wearable devices and machine learning. Morkova et al.[12] used a Shimmer[3] GSR+ device attached to the participant's two fingers and an ear lobe to capture galvanic skin responses (GSR), electrocardiograms (ECG) and PPG signal of individuals during cognitively demanding tasks and neutral condition (CLAS dataset). The authors trained a person-specific Support Vector Machine (SVM) model on features extracted from GSR either paired with features extracted from ECG or PPG recordings to predict subjects' concentration. They reported average classification performance of 78% (ECG+GSR) and 74% (PPG+GSR) and commented that these performance differences were not significant and do not justify the use of ECG. Consumer-grade wearables, employed in just-in-time interventions, due to high cost and low practicality rarely are equipped with ECG and GSR sensors commonly available in devices used in experimental conditions[15]. Therefore, we focus on developing a cognitive load detector that utilizes PPG signal only, that can be captured by smartwatches. Lisowska et al.[11] use a PPG signal from CLAS dataset[12] to detect cognitive load. The authors compared the performance of SVM with a different number of features vs. simple 1D Convolutional Neural Network (1D CNN). The latter performed better, however, the achieved average cognitive load classification accuracy reached only 60%. The authors tried to improve cognitive load detection through semi-supervised model training leveraging additional unlabeled signal, however, no notable performance gains were reached. We base our solution on this approach and first investigate the limitation of the simple 1D CNN and later suggest a potential change to the training approach. We consider between-task transfer learning. Our investigation is based on the following observations: Even though deep learning approaches have been shown to outperform traditional methods utilizing handcrafted features such as SVMs in a range of applications[14], including cognitive load detection from PPG signal[8], CNN-based models are known to be data hungry and in low data regimes, they tend to be pre-trained to offer better performance[20]. Cognitive load annotations are not easy to gather and there are few datasets utilizing physiological signals from wrist-worn devices designed to capture that state. On the other hand, stress classification is a commonly tackled problem and high-quality publicly available physiological signal datasets, such as WESAD[16] (Wearable Stress and Affect Detection), provide signal paired with labels convenient for ML training. Even though deep learning approaches have been shown to outperform traditional methods utilizing handcrafted features such as SVMs in a range of applications[14], including cognitive load detection from PPG signal[8], CNN-based models are known to be data hungry and in low data regimes, they tend to be pre-trained to offer better performance[20]. Even though deep learning approaches have been shown to outperform traditional methods utilizing handcrafted features such as SVMs in a range of applications[14], including cognitive load detection from PPG signal[8], CNN-based models are known to be data hungry and in low data regimes, they tend to be pre-trained to offer better performance[20]. Cognitive load annotations are not easy to gather and there are few datasets utilizing physiological signals from wrist-worn devices designed to capture that state. On the other hand, stress classification is a commonly tackled problem and high-quality publicly available physiological signal datasets, such as WESAD[16] (Wearable Stress and Affect Detection), provide signal paired with labels convenient for ML training. Cognitive load annotations are not easy to gather and there are few datasets utilizing physiological signals from wrist-worn devices designed to capture that state. On the other hand, stress classification is a commonly tackled problem and high-quality publicly available physiological signal datasets, such as WESAD[16] (Wearable Stress and Affect Detection), provide signal paired with labels convenient for ML training. Well-being Survey Gamification. Gamification refers to the use of game elements such as progress tracking, avatars, points, levels and challenges in a non-game context. Carlier et al.[2] conducted a comparative study of user experience with gamified and non-gamified surveys and reported that they found no difference between conditions in terms of data quality gathered but respondents of gamified surveys perceived the time taken to respond to questions as shorter than respondents of non-gamified surveys. We consider real time taken to answer the question and time spent in high cognitive load. Well-being Survey Gamification. Well-being Survey Gamification. In terms of game elements, Carlier et al. used progress tracking in gamified version and the remaining game elements were personalised to each individual. In our previous work, we investigated the personalisation of gamification design of mHealth surveys and found that the selected preferred game elements in that context do not vary strongly between individuals1 . Therefore in this work, we use a uniform version of gamified survey app across all participants selecting game elements associated with the Player archetype such as progress tracking, points, rewards and levels. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Gamified Surveys and Cognitive Load Detection in mHealth: Related Work

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

A Case Study on Gamifying Software Engineering Workplaces

How to Make UX Exponentially Better

Reducing Cognitive Load on Startup Engineers

Gamified Surveys and Cognitive Load Detection in mHealth: Abstract and Introduction

Gamified Surveys and Cognitive Load Detection in mHealth: Methods

Gamified Surveys and Cognitive Load Detection in mHealth: Results and Discussion

A Case Study on Gamifying Software Engineering Workplaces

How to Make UX Exponentially Better

Reducing Cognitive Load on Startup Engineers

Gamified Surveys and Cognitive Load Detection in mHealth: Abstract and Introduction

Gamified Surveys and Cognitive Load Detection in mHealth: Methods

Gamified Surveys and Cognitive Load Detection in mHealth: Results and Discussion

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps