Load Testing a Mobile Platform Load testing a mobile platform is especially hard to do, for the following reasons: You have to simulate traffic from multiple public IPsYou have to be able to authenticate as the "machine" deviceYou have to be able to track the requests and create a workflow to support your user interaction Here comes Rungutan! What's JWT? JSON Web Token (JWT) is an open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. This information can be verified and trusted because it is digitally signed. JWTs can be signed using a secret (with the HMAC algorithm) or a public/private key pair using RSA or ECDSA. Although JWTs can be encrypted to also provide secrecy between parties, we will focus on signed tokens. Signed tokens can verify the integrity of the claims contained within it, while encrypted tokens hide those claims from other parties. When tokens are signed using public/private key pairs, the signature also certifies that only the party holding the private key is the one that signed it. Adding JWT Auth to Rungutan { "login": { "path": "/obfuscated/path/that/does/login", "method": "POST", "data": "grant_type=OBFUSCATED&username=OBFUSCATED&password=OBFUSCATED", "headers": { "Content-Type": "application/x-www-form-urlencoded", "Authorization": "Basic OBFUSCATED" }, "get_token": { "from": "response", "key": "access_token", "inject_as_header_key": "Authorization", "inject_prefix_header_value": "Bearer " } } } The structure, as defined in the Documentation, is simple: you add a "login" key to the test case definitionyou specify the "path" and "method" (typically POST) for your API login callyou add the payload to the "data" field (usually "username" and "password")you include any headers required for this call (usually a "Content-Type" is enough)finally, you specify how to fetch the authorization, from either the "response", the "headers" or simply from "cookies" Adding some workflow Well, the hard part was done! After we sorted out the authentication logic, let's add some workflow paths. { "workflow": [ { "path": "/obfuscated/first/url", "method": "GET", "data": "", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json", "User-Agent": "Pago/1.9.12" } }, { "path": "/obfuscated/second/url", "method": "GET", "data": "", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json", "User-Agent": "Pago/1.9.12" } }, { "path": "/obfuscated/third/url", "method": "POST", "data": "{\"buildVersion\": \"1.9.12\",\"model\": \"Google Android SDK built for x86 Os:29\",\"posBuild\": \"NATIVE\",\"posOS\": \"Android\"}", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json", "User-Agent": "Pago/1.9.12" } } ] } Putting it all together Now that you have a workflow and a valid login method, all we have to do is define the rest of the test case: { "team_id": "mobileappdemo", "test_name": "open app-> login -> get data -> post something", "num_clients": 250, "hatch_rate": 30, "run_time": 600, "threads_per_region": 3, "domain_name": "the-mobile-app-api-hostname.com", "protocol": "https", "test_region": [ "eu-central-1" ], "min_wait": 1000, "max_wait": 1000, "login": { "path": "/obfuscated/path/that/does/login", "method": "POST", "data": "grant_type=OBFUSCATED&username=OBFUSCATED&password=OBFUSCATED", "headers": { "Content-Type": "application/x-www-form-urlencoded", "Authorization": "Basic OBFUSCATED" }, "get_token": { "from": "response", "key": "access_token", "inject_as_header_key": "Authorization", "inject_prefix_header_value": "Bearer " } }, "workflow": [ { "path": "/obfuscated/first/url", "method": "GET", "data": "", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json" } }, { "path": "/obfuscated/second/url", "method": "GET", "data": "", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json" } }, { "path": "/obfuscated/third/url", "method": "POST", "data": "{\"buildVersion\": \"1.9.12\",\"model\": \"Google Android SDK built for x86 Os:29\",\"posBuild\": \"NATIVE\",\"posOS\": \"Android\"}", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json" } } ] } Running this at every deployment Now that you have a test, you can include it in your CI/CD process in order to ran it every time you deploy. The CLI tool can help you integrate it easily with any system. Here's a sample GitLab CI/CD integration for instance: image: "python:3.7-alpine" stages: - load_test variables: RUNGUTAN_TEAM_ID: your_team RUNGUTAN_API_KEY: your_api_key before_script: - pip install rungutan load_test: stage: load_test script: - rungutan tests add --test_file test_file.json --wait_to_finish --test_name ${CI_PROJECT_PATH_SLUG}-${CI_PIPELINE_ID} Or, if you're not a fan of running PIP packages locally, you can use a Docker image. Here's a working example of Docker with Jenkins: #!groovy def RUNGUTAN_TEAM_ID=your_team def RUNGUTAN_API_KEY=your_api_key pipeline { agent any stages { stage('LoadTest') { agent { docker { image 'rungutancommunity/rungutan-cli:latest' args '-u root -e ${RUNGUTAN_TEAM_ID} -e ${RUNGUTAN_API_KEY}' reuseNode true } } steps { script { rungutan tests add --test_file test_file.json --wait_to_finish --test_name ${BUILD_TAG} } } } } } And if you're using GitHub, you can just reuse the GitHub Marketplace Integration for Rungutan like this: name: Load test with Rungutan on: release: types: - created jobs: load: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Load test your platform with Rungutan uses: Rungutan/rungutan-actions@1.0.0 env: RUNGUTAN_TEAM_ID: ${{ secrets.RUNGUTAN_TEAM_ID }} RUNGUTAN_API_KEY: ${{ secrets.RUNGUTAN_API_KEY }} RUNGUTAN_TEST_FILE: test_file.json RUNGUTAN_TEST_NAME: ${{ github.repository }}-${{ github.ref }} Load Testing a Mobile Platform Load testing a mobile platform is especially hard to do, for the following reasons: You have to simulate traffic from multiple public IPs You have to be able to authenticate as the "machine" device You have to be able to track the requests and create a workflow to support your user interaction You have to simulate traffic from multiple public IPs You have to be able to authenticate as the "machine" device You have to be able to track the requests and create a workflow to support your user interaction Here comes Rungutan ! Rungutan Rungutan What's JWT? JSON Web Token (JWT) is an open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. This information can be verified and trusted because it is digitally signed. JWTs can be signed using a secret (with the HMAC algorithm) or a public/private key pair using RSA or ECDSA. Although JWTs can be encrypted to also provide secrecy between parties, we will focus on signed tokens. Signed tokens can verify the integrity of the claims contained within it, while encrypted tokens hide those claims from other parties. When tokens are signed using public/private key pairs, the signature also certifies that only the party holding the private key is the one that signed it. Adding JWT Auth to Rungutan { "login": { "path": "/obfuscated/path/that/does/login", "method": "POST", "data": "grant_type=OBFUSCATED&username=OBFUSCATED&password=OBFUSCATED", "headers": { "Content-Type": "application/x-www-form-urlencoded", "Authorization": "Basic OBFUSCATED" }, "get_token": { "from": "response", "key": "access_token", "inject_as_header_key": "Authorization", "inject_prefix_header_value": "Bearer " } } } { "login": { "path": "/obfuscated/path/that/does/login", "method": "POST", "data": "grant_type=OBFUSCATED&username=OBFUSCATED&password=OBFUSCATED", "headers": { "Content-Type": "application/x-www-form-urlencoded", "Authorization": "Basic OBFUSCATED" }, "get_token": { "from": "response", "key": "access_token", "inject_as_header_key": "Authorization", "inject_prefix_header_value": "Bearer " } } } { "login" : { "path" : "/obfuscated/path/that/does/login" , "method" : "POST" , "data" : "grant_type=OBFUSCATED&username=OBFUSCATED&password=OBFUSCATED" , "headers" : { "Content-Type" : "application/x-www-form-urlencoded" , "Authorization" : "Basic OBFUSCATED" }, "get_token" : { "from" : "response" , "key" : "access_token" , "inject_as_header_key" : "Authorization" , "inject_prefix_header_value" : "Bearer " } } } { "login" : { "path" : "/obfuscated/path/that/does/login" , "method" : "POST" , "data" : "grant_type=OBFUSCATED&username=OBFUSCATED&password=OBFUSCATED" , "headers" : { "Content-Type" : "application/x-www-form-urlencoded" , "Authorization" : "Basic OBFUSCATED" "get_token" : { "from" : "response" , "key" : "access_token" , "inject_as_header_key" : "Authorization" , "inject_prefix_header_value" : "Bearer " The structure, as defined in the Documentation , is simple: Documentation you add a "login" key to the test case definition you specify the "path" and "method" (typically POST) for your API login call you add the payload to the "data" field (usually "username" and "password") you include any headers required for this call (usually a "Content-Type" is enough) finally, you specify how to fetch the authorization, from either the "response", the "headers" or simply from "cookies" you add a "login" key to the test case definition you specify the "path" and "method" (typically POST) for your API login call you add the payload to the "data" field (usually "username" and "password") you include any headers required for this call (usually a "Content-Type" is enough) finally, you specify how to fetch the authorization, from either the "response", the "headers" or simply from "cookies" Adding some workflow Well, the hard part was done! After we sorted out the authentication logic, let's add some workflow paths. { "workflow": [ { "path": "/obfuscated/first/url", "method": "GET", "data": "", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json", "User-Agent": "Pago/1.9.12" } }, { "path": "/obfuscated/second/url", "method": "GET", "data": "", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json", "User-Agent": "Pago/1.9.12" } }, { "path": "/obfuscated/third/url", "method": "POST", "data": "{\"buildVersion\": \"1.9.12\",\"model\": \"Google Android SDK built for x86 Os:29\",\"posBuild\": \"NATIVE\",\"posOS\": \"Android\"}", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json", "User-Agent": "Pago/1.9.12" } } ] } { "workflow": [ { "path": "/obfuscated/first/url", "method": "GET", "data": "", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json", "User-Agent": "Pago/1.9.12" } }, { "path": "/obfuscated/second/url", "method": "GET", "data": "", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json", "User-Agent": "Pago/1.9.12" } }, { "path": "/obfuscated/third/url", "method": "POST", "data": "{\"buildVersion\": \"1.9.12\",\"model\": \"Google Android SDK built for x86 Os:29\",\"posBuild\": \"NATIVE\",\"posOS\": \"Android\"}", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json", "User-Agent": "Pago/1.9.12" } } ] } { "workflow" : [ { "path" : "/obfuscated/first/url" , "method" : "GET" , "data" : "" , "headers" : { "Accept-Language" : "ro_RO" , "Content-Type" : "application/json" , "User-Agent" : "Pago/1.9.12" } }, { "path" : "/obfuscated/second/url" , "method" : "GET" , "data" : "" , "headers" : { "Accept-Language" : "ro_RO" , "Content-Type" : "application/json" , "User-Agent" : "Pago/1.9.12" } }, { "path" : "/obfuscated/third/url" , "method" : "POST" , "data" : "{\"buildVersion\": \"1.9.12\",\"model\": \"Google Android SDK built for x86 Os:29\",\"posBuild\": \"NATIVE\",\"posOS\": \"Android\"}" , "headers" : { "Accept-Language" : "ro_RO" , "Content-Type" : "application/json" , "User-Agent" : "Pago/1.9.12" } } ] } { "workflow" : [ "path" : "/obfuscated/first/url" , "method" : "GET" , "data" : "" , "headers" : { "Accept-Language" : "ro_RO" , "Content-Type" : "application/json" , "User-Agent" : "Pago/1.9.12" "path" : "/obfuscated/second/url" , "method" : "GET" , "data" : "" , "headers" : { "Accept-Language" : "ro_RO" , "Content-Type" : "application/json" , "User-Agent" : "Pago/1.9.12" "path" : "/obfuscated/third/url" , "method" : "POST" , "data" : "{\"buildVersion\": \"1.9.12\",\"model\": \"Google Android SDK built for x86 Os:29\",\"posBuild\": \"NATIVE\",\"posOS\": \"Android\"}" , "headers" : { "Accept-Language" : "ro_RO" , "Content-Type" : "application/json" , "User-Agent" : "Pago/1.9.12" Putting it all together Now that you have a workflow and a valid login method, all we have to do is define the rest of the test case: { "team_id": "mobileappdemo", "test_name": "open app-> login -> get data -> post something", "num_clients": 250, "hatch_rate": 30, "run_time": 600, "threads_per_region": 3, "domain_name": "the-mobile-app-api-hostname.com", "protocol": "https", "test_region": [ "eu-central-1" ], "min_wait": 1000, "max_wait": 1000, "login": { "path": "/obfuscated/path/that/does/login", "method": "POST", "data": "grant_type=OBFUSCATED&username=OBFUSCATED&password=OBFUSCATED", "headers": { "Content-Type": "application/x-www-form-urlencoded", "Authorization": "Basic OBFUSCATED" }, "get_token": { "from": "response", "key": "access_token", "inject_as_header_key": "Authorization", "inject_prefix_header_value": "Bearer " } }, "workflow": [ { "path": "/obfuscated/first/url", "method": "GET", "data": "", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json" } }, { "path": "/obfuscated/second/url", "method": "GET", "data": "", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json" } }, { "path": "/obfuscated/third/url", "method": "POST", "data": "{\"buildVersion\": \"1.9.12\",\"model\": \"Google Android SDK built for x86 Os:29\",\"posBuild\": \"NATIVE\",\"posOS\": \"Android\"}", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json" } } ] } { "team_id": "mobileappdemo", "test_name": "open app-> login -> get data -> post something", "num_clients": 250, "hatch_rate": 30, "run_time": 600, "threads_per_region": 3, "domain_name": "the-mobile-app-api-hostname.com", "protocol": "https", "test_region": [ "eu-central-1" ], "min_wait": 1000, "max_wait": 1000, "login": { "path": "/obfuscated/path/that/does/login", "method": "POST", "data": "grant_type=OBFUSCATED&username=OBFUSCATED&password=OBFUSCATED", "headers": { "Content-Type": "application/x-www-form-urlencoded", "Authorization": "Basic OBFUSCATED" }, "get_token": { "from": "response", "key": "access_token", "inject_as_header_key": "Authorization", "inject_prefix_header_value": "Bearer " } }, "workflow": [ { "path": "/obfuscated/first/url", "method": "GET", "data": "", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json" } }, { "path": "/obfuscated/second/url", "method": "GET", "data": "", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json" } }, { "path": "/obfuscated/third/url", "method": "POST", "data": "{\"buildVersion\": \"1.9.12\",\"model\": \"Google Android SDK built for x86 Os:29\",\"posBuild\": \"NATIVE\",\"posOS\": \"Android\"}", "headers": { "Accept-Language": "ro_RO", "Content-Type": "application/json" } } ] } { "team_id" : "mobileappdemo" , "test_name" : "open app-> login -> get data -> post something" , "num_clients" : 250 , "hatch_rate" : 30 , "run_time" : 600 , "threads_per_region" : 3 , "domain_name" : "the-mobile-app-api-hostname.com" , "protocol" : "https" , "test_region" : [ "eu-central-1" ], "min_wait" : 1000 , "max_wait" : 1000 , "login" : { "path" : "/obfuscated/path/that/does/login" , "method" : "POST" , "data" : "grant_type=OBFUSCATED&username=OBFUSCATED&password=OBFUSCATED" , "headers" : { "Content-Type" : "application/x-www-form-urlencoded" , "Authorization" : "Basic OBFUSCATED" }, "get_token" : { "from" : "response" , "key" : "access_token" , "inject_as_header_key" : "Authorization" , "inject_prefix_header_value" : "Bearer " } }, "workflow" : [ { "path" : "/obfuscated/first/url" , "method" : "GET" , "data" : "" , "headers" : { "Accept-Language" : "ro_RO" , "Content-Type" : "application/json" } }, { "path" : "/obfuscated/second/url" , "method" : "GET" , "data" : "" , "headers" : { "Accept-Language" : "ro_RO" , "Content-Type" : "application/json" } }, { "path" : "/obfuscated/third/url" , "method" : "POST" , "data" : "{\"buildVersion\": \"1.9.12\",\"model\": \"Google Android SDK built for x86 Os:29\",\"posBuild\": \"NATIVE\",\"posOS\": \"Android\"}" , "headers" : { "Accept-Language" : "ro_RO" , "Content-Type" : "application/json" } } ] } { "team_id" : "mobileappdemo" , "test_name" : "open app-> login -> get data -> post something" , "num_clients" : 250 , "hatch_rate" : 30 , "run_time" : 600 , "threads_per_region" : 3 , "domain_name" : "the-mobile-app-api-hostname.com" , "protocol" : "https" , "test_region" : [ "eu-central-1" "min_wait" : 1000 , "max_wait" : 1000 , "login" : { "path" : "/obfuscated/path/that/does/login" , "method" : "POST" , "data" : "grant_type=OBFUSCATED&username=OBFUSCATED&password=OBFUSCATED" , "headers" : { "Content-Type" : "application/x-www-form-urlencoded" , "Authorization" : "Basic OBFUSCATED" "get_token" : { "from" : "response" , "key" : "access_token" , "inject_as_header_key" : "Authorization" , "inject_prefix_header_value" : "Bearer " "workflow" : [ "path" : "/obfuscated/first/url" , "method" : "GET" , "data" : "" , "headers" : { "Accept-Language" : "ro_RO" , "Content-Type" : "application/json" "path" : "/obfuscated/second/url" , "method" : "GET" , "data" : "" , "headers" : { "Accept-Language" : "ro_RO" , "Content-Type" : "application/json" "path" : "/obfuscated/third/url" , "method" : "POST" , "data" : "{\"buildVersion\": \"1.9.12\",\"model\": \"Google Android SDK built for x86 Os:29\",\"posBuild\": \"NATIVE\",\"posOS\": \"Android\"}" , "headers" : { "Accept-Language" : "ro_RO" , "Content-Type" : "application/json" Running this at every deployment Now that you have a test, you can include it in your CI/CD process in order to ran it every time you deploy. The CLI tool can help you integrate it easily with any system. CLI Here's a sample GitLab CI/CD integration for instance: image: "python:3.7-alpine" stages: - load_test variables: RUNGUTAN_TEAM_ID: your_team RUNGUTAN_API_KEY: your_api_key before_script: - pip install rungutan load_test: stage: load_test script: - rungutan tests add --test_file test_file.json --wait_to_finish --test_name ${CI_PROJECT_PATH_SLUG}-${CI_PIPELINE_ID} image: "python:3.7-alpine" stages: - load_test variables: RUNGUTAN_TEAM_ID: your_team RUNGUTAN_API_KEY: your_api_key before_script: - pip install rungutan load_test: stage: load_test script: - rungutan tests add --test_file test_file.json --wait_to_finish --test_name ${CI_PROJECT_PATH_SLUG}-${CI_PIPELINE_ID} image: "python:3.7-alpine" stages : - load_test variables : RUNGUTAN_TEAM_ID: your_team RUNGUTAN_API_KEY : your_api_key before_script : - pip install rungutan load_test : stage: load_test script : - rungutan tests add --test_file test_file.json --wait_to_finish --test_name ${CI_PROJECT_PATH_SLUG}-${CI_PIPELINE_ID} image: "python:3.7-alpine" stages : variables : RUNGUTAN_API_KEY : your_api_key before_script : load_test : script : Or, if you're not a fan of running PIP packages locally, you can use a Docker image. Here's a working example of Docker with Jenkins: #!groovy def RUNGUTAN_TEAM_ID=your_team def RUNGUTAN_API_KEY=your_api_key pipeline { agent any stages { stage('LoadTest') { agent { docker { image 'rungutancommunity/rungutan-cli:latest' args '-u root -e ${RUNGUTAN_TEAM_ID} -e ${RUNGUTAN_API_KEY}' reuseNode true } } steps { script { rungutan tests add --test_file test_file.json --wait_to_finish --test_name ${BUILD_TAG} } } } } } #!groovy def RUNGUTAN_TEAM_ID=your_team def RUNGUTAN_API_KEY=your_api_key pipeline { agent any stages { stage('LoadTest') { agent { docker { image 'rungutancommunity/rungutan-cli:latest' args '-u root -e ${RUNGUTAN_TEAM_ID} -e ${RUNGUTAN_API_KEY}' reuseNode true } } steps { script { rungutan tests add --test_file test_file.json --wait_to_finish --test_name ${BUILD_TAG} } } } } } #!groovy def RUNGUTAN_TEAM_ID=your_team def RUNGUTAN_API_KEY=your_api_key pipeline { agent any stages { stage ( 'LoadTest' ) { agent { docker { image 'rungutancommunity/rungutan-cli:latest' args '-u root -e ${RUNGUTAN_TEAM_ID} -e ${RUNGUTAN_API_KEY}' reuseNode true } } steps { script { rungutan tests add --test_file test_file.json --wait_to_finish --test_name ${BUILD_TAG} } } } } } #!groovy stage ( 'LoadTest' ) { image 'rungutancommunity/rungutan-cli:latest' args '-u root -e ${RUNGUTAN_TEAM_ID} -e ${RUNGUTAN_API_KEY}' reuseNode true And if you're using GitHub, you can just reuse the GitHub Marketplace Integration for Rungutan like this: GitHub Marketplace Integration for Rungutan name: Load test with Rungutan on: release: types: - created jobs: load: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Load test your platform with Rungutan uses: Rungutan/rungutan-actions@1.0.0 env: RUNGUTAN_TEAM_ID: ${{ secrets.RUNGUTAN_TEAM_ID }} RUNGUTAN_API_KEY: ${{ secrets.RUNGUTAN_API_KEY }} RUNGUTAN_TEST_FILE: test_file.json RUNGUTAN_TEST_NAME: ${{ github.repository }}-${{ github.ref }} name: Load test with Rungutan on: release: types: - created jobs: load: runs-on: ubuntu-latest steps: - uses: actions/checkout@v2 - name: Load test your platform with Rungutan uses: Rungutan/rungutan-actions@1.0.0 env: RUNGUTAN_TEAM_ID: ${{ secrets.RUNGUTAN_TEAM_ID }} RUNGUTAN_API_KEY: ${{ secrets.RUNGUTAN_API_KEY }} RUNGUTAN_TEST_FILE: test_file.json RUNGUTAN_TEST_NAME: ${{ github.repository }}-${{ github.ref }} name: Load test with Rungutan on : release: types: - created jobs : load: runs-on: ubuntu-latest steps : - uses: actions/checkout@v2 - name: Load test your platform with Rungutan uses : Rungutan/rungutan-actions@ 1.0 .0 env : RUNGUTAN_TEAM_ID: ${{ secrets.RUNGUTAN_TEAM_ID }} RUNGUTAN_API_KEY : ${{ secrets.RUNGUTAN_API_KEY }} RUNGUTAN_TEST_FILE : test_file.json RUNGUTAN_TEST_NAME : ${{ github.repository }}-${{ github.ref }} name: Load test with Rungutan on : jobs : steps : - name: Load test your platform with Rungutan uses : Rungutan/rungutan-actions@ 1.0 .0 env : RUNGUTAN_API_KEY : ${{ secrets.RUNGUTAN_API_KEY }} RUNGUTAN_TEST_FILE : test_file.json RUNGUTAN_TEST_NAME : ${{ github.repository }}-${{ github.ref }}
