Gamified Surveys and Cognitive Load Detection in mHealth: Related Work

Table of Links

Abstract and Introduction

Related Work

Methods

Results and Discussion

Limitations

Conclusion, Acknowledgment, and References

Related work

Cognitive Load Detection. To obtain a good and unobtrusive cognitive load detector, which could facilitate both development of timely context-aware reminders and the evaluation of application use effort in the wild[8], researchers considered the utilization of wearable devices and machine learning. Morkova et al.[12] used a Shimmer[3] GSR+ device attached to the participant's two fingers and an ear lobe to capture galvanic skin responses (GSR), electrocardiograms (ECG) and PPG signal of individuals during cognitively demanding tasks and neutral condition (CLAS dataset). The authors trained a person-specific Support Vector Machine (SVM) model on features extracted from GSR either paired with features extracted from ECG or PPG recordings to predict subjects' concentration. They reported average classification performance of 78% (ECG+GSR) and 74% (PPG+GSR) and commented that these performance differences were not significant and do not justify the use of ECG. Consumer-grade wearables, employed in just-in-time interventions, due to high cost and low practicality rarely are equipped with ECG and GSR sensors commonly available in devices used in experimental conditions[15]. Therefore, we focus on developing a cognitive load detector that utilizes PPG signal only, that can be captured by smartwatches.

Lisowska et al.[11] use a PPG signal from CLAS dataset[12] to detect cognitive load. The authors compared the performance of SVM with a different number of features vs. simple 1D Convolutional Neural Network (1D CNN). The latter performed better, however, the achieved average cognitive load classification accuracy reached only 60%. The authors tried to improve cognitive load detection through semi-supervised model training leveraging additional unlabeled signal, however, no notable performance gains were reached. We base our solution on this approach and first investigate the limitation of the simple 1D CNN and later suggest a potential change to the training approach. We consider between-task transfer learning. Our investigation is based on the following observations:

1. Even though deep learning approaches have been shown to outperform traditional methods utilizing handcrafted features such as SVMs in a range of applications[14], including cognitive load detection from PPG signal[8], CNN-based models are known to be data hungry and in low data regimes, they tend to be pre-trained to offer better performance[20].

   
   

2. Cognitive load annotations are not easy to gather and there are few datasets utilizing physiological signals from wrist-worn devices designed to capture that state. On the other hand, stress classification is a commonly tackled problem and high-quality publicly available physiological signal datasets, such as WESAD[16] (Wearable Stress and Affect Detection), provide signal paired with labels convenient for ML training.

Well-being Survey Gamification. Gamification refers to the use of game elements such as progress tracking, avatars, points, levels and challenges in a non-game context. Carlier et al.[2] conducted a comparative study of user experience with gamified and non-gamified surveys and reported that they found no difference between conditions in terms of data quality gathered but respondents of gamified surveys perceived the time taken to respond to questions as shorter than respondents of non-gamified surveys. We consider real time taken to answer the question and time spent in high cognitive load.

In terms of game elements, Carlier et al. used progress tracking in gamified version and the remaining game elements were personalised to each individual. In our previous work, we investigated the personalisation of gamification design of mHealth surveys and found that the selected preferred game elements in that context do not vary strongly between individuals1 . Therefore in this work, we use a uniform version of gamified survey app across all participants selecting game elements associated with the Player archetype such as progress tracking, points, rewards and levels.