Loady

loady

Everything You Need to Know About Testcontainers-Based Load Testing Bench

টেস্ট কনটেইনার-ভিত্তিক লোড টেস্টিং বেঞ্চ সম্পর্কে আপনার যা কিছু জানা দরকার

Alles, was Sie über den Lasttest-Bench auf Basis von Testcontainern wissen müssen

Todo lo que necesita saber sobre el banco de pruebas de carga basado en contenedores de prueba

Tout ce que vous devez savoir sur le banc de test de charge basé sur des conteneurs de test

टेस्टकंटेनर-आधारित लोड परीक्षण बेंच के बारे में आपको जो कुछ भी जानना चाहिए

テストコンテナベースの負荷テストベンチについて知っておくべきことすべて

테스트 컨테이너 기반 부하 테스트 벤치에 대해 알아야 할 모든 것

Tudo o que você precisa saber sobre o banco de testes de carga baseado em Testcontainers

Все, что вам нужно знать о стенде нагрузочного тестирования на основе тест-контейнеров

Test Konteyner Tabanlı Yük Test Cihazı Hakkında Bilmeniz Gereken Her Şey

Mọi điều bạn cần biết về Bàn thử nghiệm tải dựa trên Testcontainers

您需要了解的有关基于 Testcontainers 的负载测试平台的一切

Hi there! This is a continuation of my for stress testing your services. The documentation mentions that the main engine for load is Phantom, but it can be replaced with Pandora if you need to create unique test scenarios or work with other types of requests (such as grpc or any other binaries). In this article, we will explore how to integrate Yandex Pandora for two cases: first article on how to use Yandex Tank GRPC flatbuffer (over http) As I mentioned before, our main goal is to use a simple tool with straightforward documentation that doesn’t require hiring a bunch of people to set up the test. Pandora, like Tank, has very simple and understandable documentation, does exactly what we need, and has a low entry threshold. In my case, the primary stack is Golang, and I choose solutions around it. https://hackernoon.com/turbocharge-load-testing-yandextank-ghz-combo-for-lightning-fast-code-checks?embedable=true Let’s start by referring to the documentation and see what Pandora is all about. “Pandora is a high-performance load generator in the Go language. It has built-in HTTP(S) and HTTP/2 support, and you can write your own load scenarios in Go, compiling them just before your test.” Since we are interested in scenarios, let’s take a look at the documentation: https://yandex.github.io/pandora/eng/custom.html?source=post_page-----12c250f2bff2--------------------------------&embedable=true As we can see from the documentation, there is an excellent example of how to write our “gun” to create our load engine. Great, let’s take this example and copy-paste it into our repository. Next, let’s look at our service contracts: syntax = "proto3";

option go_package = "/docs";

service DocumentService {
 rpc GetAllByLimitAndOffset(GetAllRequest) returns (GetAllResponse) {}
 rpc Save(SaveRequest) returns (SaveResponse) {}
 rpc Validate(ValidateRequest) returns (ValidateResponse) {}
} Our task involves covering two methods ( ): yes, we’ll skip save method for current case Save Validate Create a Yandex-tank folder, create a GRPC folder inside it, move our PROTO file there, and generate contracts with the command: protoc - go_out=./docs - go_opt=paths=source_relative \
 - go-grpc_out=./docs - go-grpc_opt=paths=source_relative docs.proto After that, we will get files as shown in the screenshot. And now, let’s start to write our own gun. We have to add a huge number of imports and create a struct for Custom params: package main

import (
 "context"
 "log"
 "strconv"
 "time"

 _ "github.com/golang/protobuf/ptypes/timestamp"
 "github.com/spf13/afero"
 "google.golang.org/grpc"
 pb "grpc-gun/docs"

 "github.com/yandex/pandora/cli"
 "github.com/yandex/pandora/components/phttp/import"
 "github.com/yandex/pandora/core"
 "github.com/yandex/pandora/core/aggregator/netsample"
 "github.com/yandex/pandora/core/import"
 "github.com/yandex/pandora/core/register"
)

type Ammo struct {
 Tag    string
 Param1 string
 Param2 string
 Param3 string
} Now, we should add and where we can add our grpc client (just copy and paste from official docs) GunConfig Gun, type GunConfig struct {
 Target string `validate:"required"` // Configuration will fail, without target defined
}

type Gun struct {
 // Configured on construction.
 client grpc.ClientConn
 conf   GunConfig
 // Configured on Bind, before shooting
 aggr core.Aggregator // Maybe your custom Aggregator.
 core.GunDeps
} And add new func for Gun, almost done! func NewGun(conf GunConfig) *Gun {
   return &Gun{conf: conf}
} We can see, that we should add a little bit more for running: – Bind method. We just bind our client and args: – override Shoot method func (g *Gun) Bind(aggr core.Aggregator, deps core.GunDeps) error {
 // create gRPC stub at gun initialization
 conn, err := grpc.Dial(
  g.conf.Target,
  grpc.WithInsecure(),
  grpc.WithTimeout(time.Second),
  grpc.WithUserAgent("load test, pandora custom shooter"))
 if err != nil {
  log.Fatalf("FATAL: %s", err)
 }
 g.client = *conn
 g.aggr = aggr
 g.GunDeps = deps
 return nil
}

func (g *Gun) Shoot(ammo core.Ammo) {
 customAmmo := ammo.(*Ammo)
 g.shoot(customAmmo)
} It’s time to add switch method for SaveCase and ValidateCase requests and to add methods, as example for ValidateCase: func (g *Gun) caseValidate(client pb.DocumentServiceClient, ammo *Ammo) int {
 code := 0
 docName := ammo.Param1

 out, err := client.Validate(
  context.TODO(), &pb.ValidateRequest{Document: createDoc(docName)},
 )

 if err != nil {
  log.Printf("FATAL: %s", err)
  code = 500
 }

 if out != nil {
  code = 200
 }
 return code
} Full code here: https://github.com/IliaEre/serialisation-contest/tree/main/common/loadtest/yandex-tank/grpc?source=post_page-----12c250f2bff2--------------------------------&embedable=true Alright, pay close attention! func main() {
     // Standard imports.
     fs := afero.NewOsFs()
     coreimport.Import(fs)
     // May not be imported, if you don't need http guns, etc.
     phttp.Import(fs)
    // Custom imports. Integrate your custom types into configuration system.
     coreimport.RegisterCustomJSONProvider("grpc_provider", func() core.Ammo { return &Ammo{} })
    register.Gun("grpc_gun", NewGun, func() GunConfig {
     return GunConfig{
     Target: "localhost:84",
     }
     })
    cli.Run()
} When creating our gun, we must specify everything to compile and run it. We create a provider, specifying the name “grpc_provider” We create a gun and specify the name “grpc_gun” In the target, we specify**“localhost:84”** the location where our service is accessible. Okay, let’s continue. We have successfully copied and adjusted the code for the gun. Now, how do we run it, and what do we need? First, let’s create ammo again. This time, we’ll simply describe them like this: {"tag": "/ValidateCase", "Param1": "validate_doc_grpc_pandora"} This is necessary so that, in the method, we can retrieve the parameters needed for the test. In our case, we will retrieve the document name specified in Param1. Our code will automatically pass this parameter to the ValidateCase method, and we’ll be able to retrieve it. Great, the ammo is ready. Now, let’s configure the test itself: pools:
  - id: HTTP pool
    gun:
      type: grpc_gun   # custom gun name (specified at `register.Gun("my_custom_gun_name", ...`)
      target: "localhost:84"
    ammo:
      type: grpc_provider
      source:
        type: file
        path: validate-json.ammo
    result:
      type: phout
      destination: ./phout.log
    rps: { duration: 60s, type: const, ops: 1000 }
    startup:
      type: once
      times: 10 Here, we specify our ammo as “validate-json.ammo” Their type is “grpc_provider” Our gun is “grpc_gun” The load is 1000 rps for 60 seconds: rps: { duration: 60s, type: const, ops: 1000 } And the output goes to the file “./phout.log” — Yandex Tank understands this. The finishing straight — let’s now perform three simple actions: Build our gun. Assemble a new configurator for the tank. Run tests and check the results. Building it is very straightforward — just like a simple Go application: GOOS=linux GOARCH=amd64 go build It’s important to specify `GOOS=linux GOARCH=amd64`; otherwise, you might encounter errors when launching the tank. That’s it; the binary file is ready. Let’s update another configurator: phantom:
  enabled: false
pandora:
  enabled: true
  package: yandextank.plugins.Pandora
  pandora_cmd: ./grpc-gun # Pandora executable path
  config_file: ./validate-load.yml # Pandora config path
overload:
  enabled: true
  package: yandextank.plugins.DataUploader
  job_name: "grpc validate report"
  token_file: "env/token.txt"
telegraf:
  enabled: false
autostop:
  autostop:
    - time(1s,10s) # if request average > 1s
    - http(5xx,100%,1s) # if 500 errors > 1s
    - http(4xx,25%,10s) # if 400 > 25%
    - net(xx,25,10) # if amount of non-zero net-codes in every second of last 10s period is more than 25 Specify where our gun and its settings are located. Leave everything else as before and run it all with this script: echo "run tank"
docker run \
 -v $(pwd):/var/loadtest \
 - net="host" \
 -it direvius/yandex-tank -c validate-tank-load.yml After that, we will see our results. Special gun for flatbuffer! For flatbuffer, the story repeats. All you need to do is adjust your client code from grpc to http. Here’s an example: type Gun struct {
   // Configured on construction.
   client http.Client
   conf GunConfig
   // Configured on Bind, before shooting.
   aggr core.Aggregator // Maybe your custom Aggregator.
   core.GunDeps
   files [][]byte
}

func NewGun(conf GunConfig) *Gun {
   return &Gun{conf: conf}
}

func (g *Gun) Bind(aggr core.Aggregator, deps core.GunDeps) error {
   c := http.Client{Timeout: time.Duration(1) * time.Second}
   g.client = c
   g.GunDeps = deps
   g.aggr = aggr
} And then create similar methods and describe their behavior: func (g *Gun) caseValidate(client *http.Client) int {
  host := g.conf.Target
  response, err := client.Post(host+"/report/validate", "application/octet-stream", bytes.NewBuffer(g.files[2]))
  if err != nil {
   log.Printf("FATAL: %s", err)
   return 500
  }
  return response.StatusCode
} That’s it! In conclusion We’ve explored a new tool for performance testing called Yandex Pandora. You can run it as a standalone load engine if you don’t need fancy metrics. Alternatively, you can run it based on Yandex Tank, and in this case, you’ll get beautiful graphs and all the features of this cool tool! That’s all, I hope it was helpful! Thank you. Full code here: https://github.com/IliaEre/serialisation-contest/tree/main/common/loadtest/yandex-tank?embedable=true Also published . here

Leveraging Yandex Pandora: Stress Testing GRPC and Flatbuffer Services

A basic understanding of Swift and Node.js is needed to follow this tutorial. One of the most common elements on applications (web or otherwise) is counters. YouTube, for instance, uses counters to see how many people have viewed a particular video. Facebook also does the same for videos on their platform. Most of the counters on these sites, however, only update the count when you have refreshed the page. This leaves a lot to be desired, as sometimes you just want to see the number increase in realtime. This gives you the impression that the item is being viewed by many people at the moment. In this article, we are going to explore how we can leverage the realtime nature of Pusher to create a counter that updates in realtime. We will be creating a video viewer iOS application with a realtime counter showing how many people have viewed the video. To follow along, you will need basic knowledge of Swift, Xcode and command line. You will also need to set up a Pusher account and create an application. You can do so . here Basic setup To get started you will need Cocoapods installed on your machine. Cocoapods is a package manager and we will be using this to manage the dependencies on the application. To install Cocoapods, type this in your command line: $ gem install cocoapods After you are done installing that, launch Xcode and create a new single page application project. Follow the setup wizard, and then once the Xcode project editor is open, close Xcode. to the root directory of your project and run the command: cd $ pod init This should create a in the root of your project. Open this file in your favorite editor and then update the contents of the file to the following: Podfile # Uncomment the next line to define a global platform for your projectplatform :ios, '9.0' target 'counter' do Comment the next line if you're not using Swift and don't want to use dynamic frameworks use_frameworks! Pods for counter pod 'Alamofire'pod 'PusherSwift' end In the file above we have specified two dependencies: and . These will be useful later in the project. Now install these dependencies by running this command from your terminal: PusherSwift Alamofire $ pod install Once this is complete, you should have a file in the root directory of your project. Open this file and it should launch Xcode. (Make sure you don't have any instance of Xcode running for this project before opening the file or you will get an error.) .xcworkspace Creating the views for our realtime application Now that the project is open, we will create some views for our application. Open the file and in there we will create the views. Main.storyboard We want to create a navigation controller that will have a as the root controller of the navigation controller. Then in the new view controller, we will add a webview; this is where we will be embedding the video we want people to view. We will also add two labels, one for the counter and the other will just be a plain immutable message. ViewController After we are done, this is what we have so far: Adding our realtime feature to our application Now that we have created the application, we can now add the code that will interact with the views and add the realtime support and the video also. Create a new class and link it to the view controller we created above. Then create a split view in Xcode and ctrl+drag from the to the view controller. This should create an in the controller; do the same for the counter label so it creates another . Our controller should now have two s one for the webview and one for the counter label. Great. MainViewController webview @IBOutlet @IBOutlet @IBOutlet Now we are going to add the code to load our video. We are going to be using a YouTube video of a Pusher tutorial for this exercise. In the method add the following: viewDidLoad func viewDidLoad() {super.viewDidLoad()loadYoutube(videoID:"xDQ8vzD0lzw")} Now let's create the method and the other dependent methods: loadYoutube func loadYoutube(videoID:String) {self.automaticallyAdjustsScrollViewInsets = falsewebview.allowsInlineMediaPlayback = truewebview.mediaPlaybackRequiresUserAction = false let embedHTML = getEmbedHTML(id:videoID);

let url: NSURL = NSURL(string: "[https://www.youtube.com/embed/\\(videoID)](https://www.youtube.com/embed/%5C%28videoID%29)")!

webview.loadHTMLString(embedHTML as String, baseURL:url as URL ) } private func getEmbedHTML(id: String) -> String {return "<html><head><style type=\"text/css\">body {background-color: transparent;color: white;}</style></head><body style=\"margin:0\"> <iframe webkit-playsinline width=\"100%\" height=\"100%\" src=\" " frameborder=\"0\"></iframe>";} https://www.youtube.com/embed/\(id)?feature=player_detailpage&playsinline=1\ Now we have instructed the application to load a YouTube video automatically. However, the counter functionality does not yet work. Let’s fix that. Import the library and add a new method to update the counter using Pusher: PusherSwift updateViewCount() {let options = PusherClientOptions(host: .cluster("PUSHER_CLUSTER")) pusher = Pusher(key: "PUSHER\_KEY", options: options)

let channel = pusher.subscribe("counter")  
let \_ = channel.bind(eventName: "new\_user", callback: { (data: Any?) -> Void in  
    if let data = data as? \[String: AnyObject\] {  
        let viewCount = data\["count"\] as! NSNumber  
        self.count.text = "\\(viewCount)" as String!  
    }  
})

pusher.connect() } Note: Where it says _PUSHER_CLUSTER_ and _PUSHER_KEY_ , you should replace with your actual Pusher cluster and key. You'll also need to import Now you can just call the from the method so it is called when the view is loaded. updateViewCount viewDidLoad One final thing we will do is use to send a request to a backend so the counter can be updated and saved, so we do not lose the count of people who have viewed the video. Import and add the following: Alamofire Alamofire func sendViewCount() {Alamofire.request(endpoint, method: .post).validate().responseJSON { response inswitch response.result {case .success:if let result = response.result.value {let data = result as! NSDictionarylet viewCount = data["count"] as! NSNumberself.count.text = "\(viewCount)" as String!}case .failure(let error):print(error)}}} Now that we are done with that, the should now look a little like this: MainViewController import UIKitimport Alamofireimport PusherSwift class MainViewController: UIViewController { [@IBOutlet](http://twitter.com/IBOutlet "Twitter profile for @IBOutlet") weak var count: UILabel!  
[@IBOutlet](http://twitter.com/IBOutlet "Twitter profile for @IBOutlet") weak var webview: UIWebView!

var endpoint: String = "[http://localhost:4000/update\_counter](http://localhost:4000/update_counter)"

var pusher : Pusher!

override func viewDidLoad() {  
    super.viewDidLoad()  
    loadYoutube(videoID:"xDQ8vzD0lzw")  
    sendViewCount()  
    updateViewCount()  
}

func sendViewCount() {  
    Alamofire.request(endpoint, method: .post).validate().responseJSON { response in  
        switch response.result {

        case .success:  
            if let result = response.result.value {  
                let data = result as! NSDictionary  
                let viewCount = data\["count"\] as! NSNumber  
                self.count.text = "\\(viewCount)" as String!  
            }  
        case .failure(let error):  
            print(error)  
        }  
    }  
}

func updateViewCount() {  
    let options = PusherClientOptions(  
        host: .cluster("PUSHER\_CLUSTER")  
    )

    pusher = Pusher(key: "PUSHER\_KEY", options: options)

    let channel = pusher.subscribe("counter")  
    let \_ = channel.bind(eventName: "new\_user", callback: { (data: Any?) -> Void in  
        if let data = data as? \[String: AnyObject\] {  
            let viewCount = data\["count"\] as! NSNumber  
            self.count.text = "\\(viewCount)" as String!  
        }  
    })

    pusher.connect()  
}

func loadYoutube(videoID:String) {  
    self.automaticallyAdjustsScrollViewInsets = false  
    webview.allowsInlineMediaPlayback = true  
    webview.mediaPlaybackRequiresUserAction = false  
    let embedHTML = getEmbedHTML(id:videoID);

    let url: NSURL = NSURL(string: "[https://www.youtube.com/embed/\\(videoID)](https://www.youtube.com/embed/%5C%28videoID%29)")!  
    webview.loadHTMLString(embedHTML as String, baseURL:url as URL )  
}

private func getEmbedHTML(id: String) -> String {  
    return "<html><head><style type=\\"text/css\\">body {background-color: transparent;color: white;}</style></head><body style=\\"margin:0\\"> <iframe webkit-playsinline width=\\"100%\\" height=\\"100%\\" src=\\"[https://www.youtube.com/embed/\\(id)?feature=player\_detailpage&playsinline=1\\](https://www.youtube.com/embed/%5C%28id%29?feature=player_detailpage&playsinline=1%5C)" frameborder=\\"0\\"></iframe>";  
} } If we load the application now, it would load the video but the counter will not work. This is because we have not yet set up a backend logic. Creating the backend for our realtime counter iOS application For the backend, we will be creating a very basic Node.js application. This application will simply have one endpoint that saves the counter state and sends a trigger to Pusher so other listeners subscribed to the channel event can pick it up and update in realtime. To start, create a new directory for your application. In the application create two files: File: package.json {"main": "index.js","dependencies": {"body-parser": "^1.16.0","express": "^4.14.1","pusher": "^1.5.1"}} File: index.js var Pusher = require('pusher');let express = require('express');let bodyParser = require('body-parser');let fs = require('fs'); let app = express(); let pusher = new Pusher({appId: 'PUSHER_ID',key: 'PUSHER_KEY',secret: 'PUSHER_SECRET',cluster: 'PUSHER_CLUSTER',encrypted: true}); app.use(bodyParser.json());app.use(bodyParser.urlencoded({ extended: false })); app.post('/update_counter', function(req, res) {let counterFile = './count.txt'; fs.readFile(counterFile, 'utf-8', function(err, count) {count = parseInt(count) + 1;fs.writeFile(counterFile, count, function (err) {pusher.trigger('counter', 'new_user', {count:count});res.json({count:count});});});}); app.use(function(req, res, next) {let err = new Error('Not Found');err.status = 404;next(err);}); module.exports = app; app.listen(4000, function(){console.log('App listening on port 4000!')}) Finally, create the file in the same directory and chmod it to be writable. counter.txt $ echo "0" > count.txt$ chmod 0644 count.txt Now run to install all the dependencies that node needs to make the backend application work. When the dependencies are done installing, run to start your application. npm install node index.js One last change we would need to make to allow our application interact with the localhost application is in our projects file. Make the changes below before launching the application: info.plist Now, when you launch the application you can see the counter increase, and if there is another instance of the application, they can see the counter increase in realtime. Neat! Conclusion In this article, we explored how you can create realtime counters using Pusher in your iOS applications. We are curious to see how you would incorporate this into your iOS application. If you have any other use cases you would like to see, or have any feedback, leave a comment below. This post first appeared on the . Pusher blog

Build a realtime counter using Swift

Reverse engineering visual novels 101

Gamified Surveys and Cognitive Load Detection in mHealth: Conclusion, Acknowledgment, and References

Authors:
(1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland;
(2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland;
(7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland;
(8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland;
(9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA;
(10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland;
(12) EngD. Table of Links Abstract and Introduction Related Work Methods Results and Discussion Limitations Conclusion, Acknowledgment, and References Limitations The main limitation of this work is the small sample size of participants used for both developing the cognitive load detector and evaluating the cognitive effort required to complete the two survey versions. To mitigate this issue, we pre-trained the detector on a similar task, but further improvements in its performance may be possible with a larger sample size. The generalizability of our method can also be impacted by the homogeneous nature of the study participants. The volunteers were healthy individuals from a similar age group. To ensure the robustness of our method, future studies should include a more diverse and bigger sample size. Finally, we conducted experiments in controlled laboratory settings. In the future, we plan to conduct similar studies in real-world conditions[7]. In this feasibility study, we focused solely on developing automatic detection of high cognitive load, which is the first step towards estimating cognitive effort in the real world. This may not be sufficient to distinguish more subtle differences in cognitive demand caused by the mobile application. To identify the game element or combination of such elements that is the most helpful in reducing cognitive burden we would like to conduct an ablation study in the context of well-being questionnaires in our future work. We would also like to include the self-reported cognitive load after survey completion for a better understanding of survey difficulty. Moreover, it would be interesting to collect data during semi-demanding tasks, such as text reading, in order to train a more finely-graded classifier. Finally, we would like to include other tasks requiring a high cognitive load to ensure the efficiency of our method across various cognitively demanding activities. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland; (2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland; (7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland; (8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland; (9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; (10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland; (12) EngD. Authors: Authors: (1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland; (2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland; (7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland; (8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland; (9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; (10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland; (12) EngD. Table of Links Abstract and Introduction Abstract and Introduction Related Work Related Work Methods Methods Results and Discussion Results and Discussion Limitations Limitations Conclusion, Acknowledgment, and References Conclusion, Acknowledgment, and References Limitations The main limitation of this work is the small sample size of participants used for both developing the cognitive load detector and evaluating the cognitive effort required to complete the two survey versions. To mitigate this issue, we pre-trained the detector on a similar task, but further improvements in its performance may be possible with a larger sample size. The generalizability of our method can also be impacted by the homogeneous nature of the study participants. The volunteers were healthy individuals from a similar age group. To ensure the robustness of our method, future studies should include a more diverse and bigger sample size. Finally, we conducted experiments in controlled laboratory settings. In the future, we plan to conduct similar studies in real-world conditions[7]. In this feasibility study, we focused solely on developing automatic detection of high cognitive load, which is the first step towards estimating cognitive effort in the real world. This may not be sufficient to distinguish more subtle differences in cognitive demand caused by the mobile application. To identify the game element or combination of such elements that is the most helpful in reducing cognitive burden we would like to conduct an ablation study in the context of well-being questionnaires in our future work. We would also like to include the self-reported cognitive load after survey completion for a better understanding of survey difficulty. Moreover, it would be interesting to collect data during semi-demanding tasks, such as text reading, in order to train a more finely-graded classifier. Finally, we would like to include other tasks requiring a high cognitive load to ensure the efficiency of our method across various cognitively demanding activities. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Gamified Surveys and Cognitive Load Detection in mHealth: Limitations

Authors:
(1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland;
(2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland;
(7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland;
(8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland;
(9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA;
(10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland;
(12) EngD. Table of Links Abstract and Introduction Related Work Methods Results and Discussion Limitations Conclusion, Acknowledgment, and References Results and Discussion Cognitive Load Detection. To understand how to obtain a reliable cognitive load detection model we investigate different approaches to training protocols including varying lengths of the signal input and model pretraining as described above. Due to a small number of data, the model performance was varying between runs and therefore we run each training session 40 times and report the mean and standard deviation of all runs. We find that the number of data samples was insufficient to train the model from scratch on the pilot cognitive load dataset only (see Vanilla in Table 1). Model pre-training on WESAD dataset has shown to be helpful for cognitive load detection (see WESAD pre-trained in Table 1). The length of the signal is also important, in general, the 10 seconds window seems to be insufficient to obtain promising cognitive load detection performance even when the model is pretrained. We expected that the better the performance of the stress detector on the pretraining task the better will be the performance on the fine-tuning task. However, the link between source and target task performance is not that apparent. There is a weak correlation between source training data stress detection and fine-tuned cognitive load detector performance (r=0.18 p=0.04), a slightly stronger correlation between source validation data stress detection and target cognitive load detection (r=0.25, p=0.005), and no correlation between the test subject stress detection and target dataset cognitive load score (see Figure 3). Further, to check if our cognitive load detector is not just detecting stress, we run the best-performing stress detector with a 30s window on all participants from the pilot dataset and check % of samples for each participant from each condition (baseline and cognitive load) that are classified as stressed. We find that signal from participants 1, 3, 4, 7 and 8 had 0 samples classified as stressed, participants 0, 6, 9 and 10 had 9-22% samples classified as stressed and only participants 2 and 5 showed higher levels of stress with 59% and 55% of signal classified as stressed respectively. This suggests that improvements obtained via pre-training are not due to the physiological signal similarity in both conditions (cognitive load is not classified as stress) but rather due to learning of general feature extraction from the 1D signal. Response Time. On average participants spent less time responding to questions in gamified condition than in Non-gamified condition, with 5.5s and 6s per question respectively. However, interestingly, in gamified condition participants spent more time at the start and end questions which potentially could be due to the introduction of the gamified components which could capture respondents' attention (see Figure 4). Note that time spent on the very last question in each survey has been not included in this analysis as they mostly capture the time needed to move between questionnaires. Cognitive Burden. To ensure that the cognitive load detector can distinguish between high and low cognitive load for each participant in the survey dataset we first applied all 440 models (from 40 runs and 11 folds) trained with a 30s window on the pilot dataset to every participant in the survey dataset in cognitive load and baseline conditions and for each participant, we selected a model showing the best performance for this participant. In other words, we matched the models to participants using a small calibration dataset obtained from participants prior to filling the survey application. The performance of the selected best model is reported in Table 2 for each session. We have excluded the prediction of cognitive load during survey completion for those participants for whom we could not get a model with performance above 0.7 F1 score (marked with X in Table 2). Encouragingly, for 10 out of 13 participants, we were able to find a model which could reliably distinguish between their high and low cognitive load. The time spent in the high cognitive load and stress does not differ between gamified and non-gamified survey conditions. Potentially to understand if there is a difference between cognitive burden we would need a classifier trained on at least three different tasks with cognitive load levels: high (demanding task), medium (e.g. reading task) and low. Participants Experience. Only half of the participants noticed the difference in survey application appearance between the sessions. Participants who noted the difference reported preferring the gamified version more. Potentially the gamification was too simplistic to modulate the level of engagement or enjoyment. It is nontrivial to introduce game components in the surveys to reduce the burden but not affect the responses. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland; (2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland; (7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland; (8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland; (9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; (10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland; (12) EngD. Authors: Authors: (1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland; (2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland; (7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland; (8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland; (9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; (10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland; (12) EngD. Table of Links Abstract and Introduction Abstract and Introduction Related Work Related Work Methods Methods Results and Discussion Results and Discussion Limitations Limitations Conclusion, Acknowledgment, and References Conclusion, Acknowledgment, and References Results and Discussion Cognitive Load Detection. To understand how to obtain a reliable cognitive load detection model we investigate different approaches to training protocols including varying lengths of the signal input and model pretraining as described above. Due to a small number of data, the model performance was varying between runs and therefore we run each training session 40 times and report the mean and standard deviation of all runs. We find that the number of data samples was insufficient to train the model from scratch on the pilot cognitive load dataset only (see Vanilla in Table 1). Model pre-training on WESAD dataset has shown to be helpful for cognitive load detection (see WESAD pre-trained in Table 1). The length of the signal is also important, in general, the 10 seconds window seems to be insufficient to obtain promising cognitive load detection performance even when the model is pretrained. Cognitive Load Detection. Cognitive Load Detection. Vanilla WESAD pre-trained We expected that the better the performance of the stress detector on the pretraining task the better will be the performance on the fine-tuning task. However, the link between source and target task performance is not that apparent. There is a weak correlation between source training data stress detection and fine-tuned cognitive load detector performance (r=0.18 p=0.04), a slightly stronger correlation between source validation data stress detection and target cognitive load detection (r=0.25, p=0.005), and no correlation between the test subject stress detection and target dataset cognitive load score (see Figure 3). Further, to check if our cognitive load detector is not just detecting stress, we run the best-performing stress detector with a 30s window on all participants from the pilot dataset and check % of samples for each participant from each condition (baseline and cognitive load) that are classified as stressed. We find that signal from participants 1, 3, 4, 7 and 8 had 0 samples classified as stressed, participants 0, 6, 9 and 10 had 9-22% samples classified as stressed and only participants 2 and 5 showed higher levels of stress with 59% and 55% of signal classified as stressed respectively. This suggests that improvements obtained via pre-training are not due to the physiological signal similarity in both conditions (cognitive load is not classified as stress) but rather due to learning of general feature extraction from the 1D signal. Response Time. On average participants spent less time responding to questions in gamified condition than in Non-gamified condition, with 5.5s and 6s per question respectively. However, interestingly, in gamified condition participants spent more time at the start and end questions which potentially could be due to the introduction of the gamified components which could capture respondents' attention (see Figure 4). Note that time spent on the very last question in each survey has been not included in this analysis as they mostly capture the time needed to move between questionnaires. Response Time. Response Time. Cognitive Burden. To ensure that the cognitive load detector can distinguish between high and low cognitive load for each participant in the survey dataset we first applied all 440 models (from 40 runs and 11 folds) trained with a 30s window on the pilot dataset to every participant in the survey dataset in cognitive load and baseline conditions and for each participant, we selected a model showing the best performance for this participant. In other words, we matched the models to participants using a small calibration dataset obtained from participants prior to filling the survey application. The performance of the selected best model is reported in Table 2 for each session. We have excluded the prediction of cognitive load during survey completion for those participants for whom we could not get a model with performance above 0.7 F1 score (marked with X in Table 2). Encouragingly, for 10 out of 13 participants, we were able to find a model which could reliably distinguish between their high and low cognitive load. The time spent in the high cognitive load and stress does not differ between gamified and non-gamified survey conditions. Potentially to understand if there is a difference between cognitive burden we would need a classifier trained on at least three different tasks with cognitive load levels: high (demanding task), medium (e.g. reading task) and low. Cognitive Burden. Cognitive Burden. Participants Experience. Only half of the participants noticed the difference in survey application appearance between the sessions. Participants who noted the difference reported preferring the gamified version more. Potentially the gamification was too simplistic to modulate the level of engagement or enjoyment. It is nontrivial to introduce game components in the surveys to reduce the burden but not affect the responses. Participants Experience. Participants Experience. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Gamified Surveys and Cognitive Load Detection in mHealth: Results and Discussion

Gamified Surveys and Cognitive Load Detection in mHealth: Methods

Authors:
(1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland;
(2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland;
(7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland;
(8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland;
(9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA;
(10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland;
(12) EngD. Table of Links Abstract and Introduction Related Work Methods Results and Discussion Limitations Conclusion, Acknowledgment, and References Related work Cognitive Load Detection. To obtain a good and unobtrusive cognitive load detector, which could facilitate both development of timely context-aware reminders and the evaluation of application use effort in the wild[8], researchers considered the utilization of wearable devices and machine learning. Morkova et al.[12] used a Shimmer[3] GSR+ device attached to the participant's two fingers and an ear lobe to capture galvanic skin responses (GSR), electrocardiograms (ECG) and PPG signal of individuals during cognitively demanding tasks and neutral condition (CLAS dataset). The authors trained a person-specific Support Vector Machine (SVM) model on features extracted from GSR either paired with features extracted from ECG or PPG recordings to predict subjects' concentration. They reported average classification performance of 78% (ECG+GSR) and 74% (PPG+GSR) and commented that these performance differences were not significant and do not justify the use of ECG. Consumer-grade wearables, employed in just-in-time interventions, due to high cost and low practicality rarely are equipped with ECG and GSR sensors commonly available in devices used in experimental conditions[15]. Therefore, we focus on developing a cognitive load detector that utilizes PPG signal only, that can be captured by smartwatches. Lisowska et al.[11] use a PPG signal from CLAS dataset[12] to detect cognitive load. The authors compared the performance of SVM with a different number of features vs. simple 1D Convolutional Neural Network (1D CNN). The latter performed better, however, the achieved average cognitive load classification accuracy reached only 60%. The authors tried to improve cognitive load detection through semi-supervised model training leveraging additional unlabeled signal, however, no notable performance gains were reached. We base our solution on this approach and first investigate the limitation of the simple 1D CNN and later suggest a potential change to the training approach. We consider between-task transfer learning. Our investigation is based on the following observations: Even though deep learning approaches have been shown to outperform traditional methods utilizing handcrafted features such as SVMs in a range of applications[14], including cognitive load detection from PPG signal[8], CNN-based models are known to be data hungry and in low data regimes, they tend to be pre-trained to offer better performance[20].


Cognitive load annotations are not easy to gather and there are few datasets utilizing physiological signals from wrist-worn devices designed to capture that state. On the other hand, stress classification is a commonly tackled problem and high-quality publicly available physiological signal datasets, such as WESAD[16] (Wearable Stress and Affect Detection), provide signal paired with labels convenient for ML training. Well-being Survey Gamification. Gamification refers to the use of game elements such as progress tracking, avatars, points, levels and challenges in a non-game context. Carlier et al.[2] conducted a comparative study of user experience with gamified and non-gamified surveys and reported that they found no difference between conditions in terms of data quality gathered but respondents of gamified surveys perceived the time taken to respond to questions as shorter than respondents of non-gamified surveys. We consider real time taken to answer the question and time spent in high cognitive load. In terms of game elements, Carlier et al. used progress tracking in gamified version and the remaining game elements were personalised to each individual. In our previous work, we investigated the personalisation of gamification design of mHealth surveys and found that the selected preferred game elements in that context do not vary strongly between individuals1 . Therefore in this work, we use a uniform version of gamified survey app across all participants selecting game elements associated with the Player archetype such as progress tracking, points, rewards and levels. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland; (2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland; (7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland; (8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland; (9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; (10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland; (12) EngD. Authors: Authors: (1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland; (2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland; (7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland; (8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland; (9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; (10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland; (12) EngD. Table of Links Abstract and Introduction Abstract and Introduction Related Work Related Work Methods Methods Results and Discussion Results and Discussion Limitations Limitations Conclusion, Acknowledgment, and References Conclusion, Acknowledgment, and References Related work Cognitive Load Detection. To obtain a good and unobtrusive cognitive load detector, which could facilitate both development of timely context-aware reminders and the evaluation of application use effort in the wild[8], researchers considered the utilization of wearable devices and machine learning. Morkova et al.[12] used a Shimmer[3] GSR+ device attached to the participant's two fingers and an ear lobe to capture galvanic skin responses (GSR), electrocardiograms (ECG) and PPG signal of individuals during cognitively demanding tasks and neutral condition (CLAS dataset). The authors trained a person-specific Support Vector Machine (SVM) model on features extracted from GSR either paired with features extracted from ECG or PPG recordings to predict subjects' concentration. They reported average classification performance of 78% (ECG+GSR) and 74% (PPG+GSR) and commented that these performance differences were not significant and do not justify the use of ECG. Consumer-grade wearables, employed in just-in-time interventions, due to high cost and low practicality rarely are equipped with ECG and GSR sensors commonly available in devices used in experimental conditions[15]. Therefore, we focus on developing a cognitive load detector that utilizes PPG signal only, that can be captured by smartwatches. Lisowska et al.[11] use a PPG signal from CLAS dataset[12] to detect cognitive load. The authors compared the performance of SVM with a different number of features vs. simple 1D Convolutional Neural Network (1D CNN). The latter performed better, however, the achieved average cognitive load classification accuracy reached only 60%. The authors tried to improve cognitive load detection through semi-supervised model training leveraging additional unlabeled signal, however, no notable performance gains were reached. We base our solution on this approach and first investigate the limitation of the simple 1D CNN and later suggest a potential change to the training approach. We consider between-task transfer learning. Our investigation is based on the following observations: Even though deep learning approaches have been shown to outperform traditional methods utilizing handcrafted features such as SVMs in a range of applications[14], including cognitive load detection from PPG signal[8], CNN-based models are known to be data hungry and in low data regimes, they tend to be pre-trained to offer better performance[20]. Cognitive load annotations are not easy to gather and there are few datasets utilizing physiological signals from wrist-worn devices designed to capture that state. On the other hand, stress classification is a commonly tackled problem and high-quality publicly available physiological signal datasets, such as WESAD[16] (Wearable Stress and Affect Detection), provide signal paired with labels convenient for ML training. Even though deep learning approaches have been shown to outperform traditional methods utilizing handcrafted features such as SVMs in a range of applications[14], including cognitive load detection from PPG signal[8], CNN-based models are known to be data hungry and in low data regimes, they tend to be pre-trained to offer better performance[20]. Even though deep learning approaches have been shown to outperform traditional methods utilizing handcrafted features such as SVMs in a range of applications[14], including cognitive load detection from PPG signal[8], CNN-based models are known to be data hungry and in low data regimes, they tend to be pre-trained to offer better performance[20]. Cognitive load annotations are not easy to gather and there are few datasets utilizing physiological signals from wrist-worn devices designed to capture that state. On the other hand, stress classification is a commonly tackled problem and high-quality publicly available physiological signal datasets, such as WESAD[16] (Wearable Stress and Affect Detection), provide signal paired with labels convenient for ML training. Cognitive load annotations are not easy to gather and there are few datasets utilizing physiological signals from wrist-worn devices designed to capture that state. On the other hand, stress classification is a commonly tackled problem and high-quality publicly available physiological signal datasets, such as WESAD[16] (Wearable Stress and Affect Detection), provide signal paired with labels convenient for ML training. Well-being Survey Gamification. Gamification refers to the use of game elements such as progress tracking, avatars, points, levels and challenges in a non-game context. Carlier et al.[2] conducted a comparative study of user experience with gamified and non-gamified surveys and reported that they found no difference between conditions in terms of data quality gathered but respondents of gamified surveys perceived the time taken to respond to questions as shorter than respondents of non-gamified surveys. We consider real time taken to answer the question and time spent in high cognitive load. Well-being Survey Gamification. Well-being Survey Gamification. In terms of game elements, Carlier et al. used progress tracking in gamified version and the remaining game elements were personalised to each individual. In our previous work, we investigated the personalisation of gamification design of mHealth surveys and found that the selected preferred game elements in that context do not vary strongly between individuals1 . Therefore in this work, we use a uniform version of gamified survey app across all participants selecting game elements associated with the Player archetype such as progress tracking, points, rewards and levels. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Gamified Surveys and Cognitive Load Detection in mHealth: Related Work

Authors:
(1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland;
(2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland;
(7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland;
(8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland;
(9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA;
(10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland;
(11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland;
(12) EngD. Table of Links Abstract and Introduction Related Work Methods Results and Discussion Limitations Conclusion, Acknowledgment, and References Abstract The effectiveness of digital treatments can be measured by requiring patients to self-report their state through applications, however, it can be overwhelming and causes disengagement. We conduct a study to explore the impact of gamification on self-reporting. Our approach involves the creation of a system to assess cognitive load (CL) through the analysis of photoplethysmography (PPG) signals. The data from 11 participants is utilized to train a machine learning model to detect CL. Subsequently, we create two versions of surveys: a gamified and a traditional one. We estimate the CL experienced by other participants (13) while completing surveys. We find that CL detector performance can be enhanced via pre-training on stress detection tasks. For 10 out of 13 participants, a personalized CL detector can achieve an F1 score above 0.7. We find no difference between the gamified and non-gamified surveys in terms of CL but participants prefer the gamified version. Introduction Digital health interventions (DHI) support patient health monitoring and treatment provision outside of the traditional clinical setting. To assess patient health status and track the effectiveness of remote treatments patients are requested to regularly fill surveys which consist of numerous questions relating to their physical and mental health. However, responding to questionnaires is laborious and might discourage the consistent use of mobile health (mHealth) applications. Recently Oakley-Girvan et al.[13] performed scoping review of mobile interventions and identified application features that impact patient engagement. A high number of surveys were among the features decreasing engagement with the mHealth apps. This is a problem when health intervention outcome is associated with effective engagement with the application[4]. To encourage long-term engagement app developers use gamification[9]. The use of game elements such as progress tracking and rewards has shown to motivate users to be physically active[19] or maintain a healthy diet[3]. However, the introduction of game elements in mHealth surveys context has been less commonly studied. In this work, we investigate the impact of the inclusion of simple game elements in mobile surveys on completion time and the cognitive burden of self-reporting. We hypothesize that gamification can ease the process of survey completion and ultimately facilitate long-term well-being tracking. To test this hypothesis we train a machine learning tool for cognitive load detection. Then, we develop gamified and not-gamified versions of a mobile survey application and conduct a study with human subjects to assess the effort required to complete these surveys using the developed tool. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland; (2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland; (7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland; (8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland; (9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; (10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland; (12) EngD. Authors: Authors: (1) Michal K. Grzeszczyk, Sano Centre for Computational Medicine, Cracow, Poland and Warsaw University of Technology, Warsaw, Poland; (2) M.Sc.; Paulina Adamczyk, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (3) B.Sc.; Sylwia Marek, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (4) B.Sc.; Ryszard Pręcikowski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (5) B.Sc.; Maciej Kuś, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (6) B.Sc.; M. Patrycja Lelujko, Sano Centre for Computational Medicine, Cracow, Poland; (7) B.Sc.; Rosmary Blanco, Sano Centre for Computational Medicine, Cracow, Poland; (8) M.Sc.; Tomasz Trzciński, Warsaw University of Technology, Warsaw, Poland, IDEAS NCBR, Warsaw, Poland andTooploox, Wroclaw, Poland; (9) D.Sc.; Arkadiusz Sitek, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; (10) PhD; Maciej Malawski, Sano Centre for Computational Medicine, Cracow, Poland and AGH University of Science and Technology, Cracow, Poland; (11) D.Sc.; Aneta Lisowska, Sano Centre for Computational Medicine, Cracow, Poland and Poznań University of Technology, Poznań, Poland; (12) EngD. Table of Links Abstract and Introduction Abstract and Introduction Related Work Related Work Methods Methods Results and Discussion Results and Discussion Limitations Limitations Conclusion, Acknowledgment, and References Conclusion, Acknowledgment, and References Abstract The effectiveness of digital treatments can be measured by requiring patients to self-report their state through applications, however, it can be overwhelming and causes disengagement. We conduct a study to explore the impact of gamification on self-reporting. Our approach involves the creation of a system to assess cognitive load (CL) through the analysis of photoplethysmography (PPG) signals. The data from 11 participants is utilized to train a machine learning model to detect CL. Subsequently, we create two versions of surveys: a gamified and a traditional one. We estimate the CL experienced by other participants (13) while completing surveys. We find that CL detector performance can be enhanced via pre-training on stress detection tasks. For 10 out of 13 participants, a personalized CL detector can achieve an F1 score above 0.7. We find no difference between the gamified and non-gamified surveys in terms of CL but participants prefer the gamified version. The effectiveness of digital treatments can be measured by requiring patients to self-report their state through applications, however, it can be overwhelming and causes disengagement. We conduct a study to explore the impact of gamification on self-reporting. Our approach involves the creation of a system to assess cognitive load (CL) through the analysis of photoplethysmography (PPG) signals. The data from 11 participants is utilized to train a machine learning model to detect CL. Subsequently, we create two versions of surveys: a gamified and a traditional one. We estimate the CL experienced by other participants (13) while completing surveys. We find that CL detector performance can be enhanced via pre-training on stress detection tasks. For 10 out of 13 participants, a personalized CL detector can achieve an F1 score above 0.7. We find no difference between the gamified and non-gamified surveys in terms of CL but participants prefer the gamified version. Introduction Digital health interventions (DHI) support patient health monitoring and treatment provision outside of the traditional clinical setting. To assess patient health status and track the effectiveness of remote treatments patients are requested to regularly fill surveys which consist of numerous questions relating to their physical and mental health. However, responding to questionnaires is laborious and might discourage the consistent use of mobile health (mHealth) applications. Recently Oakley-Girvan et al.[13] performed scoping review of mobile interventions and identified application features that impact patient engagement. A high number of surveys were among the features decreasing engagement with the mHealth apps. This is a problem when health intervention outcome is associated with effective engagement with the application[4]. To encourage long-term engagement app developers use gamification[9]. The use of game elements such as progress tracking and rewards has shown to motivate users to be physically active[19] or maintain a healthy diet[3]. However, the introduction of game elements in mHealth surveys context has been less commonly studied. In this work, we investigate the impact of the inclusion of simple game elements in mobile surveys on completion time and the cognitive burden of self-reporting. We hypothesize that gamification can ease the process of survey completion and ultimately facilitate long-term well-being tracking. To test this hypothesis we train a machine learning tool for cognitive load detection. Then, we develop gamified and not-gamified versions of a mobile survey application and conduct a study with human subjects to assess the effort required to complete these surveys using the developed tool. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Gamified Surveys and Cognitive Load Detection in mHealth: Abstract and Introduction

Stories, News & Mentions about LOADY

Loady’s Stories on HackerNoon