You got an amazing idea… You nailed the UX… You got a beautiful UI… And you are all set to start developing your app. The fun, exciting and explorative phases are done and you are moving into execution mode. You want an incredible user experience to make your app the next big thing. What does it take? Testing, testing, testing… Things to consider upfront (before you even start to develop) Testing and Quality Assurance (QA) is a time consuming and sometimes dull task. But it’s critical to do in-depth testing in order to deliver an app your users will love 💖. Before you start developing your app consider the following: Which does the application need to support? Web, mobile, tablet, desktop? platforms Which browsers need to be supported? Chrome, Firefox, Safari, Internet Explorer, Edge etc.? Web: iPhone and/or Android? iPad and/or other tablets? Mobile & Tablets: Windows app and/or Mac app? Desktop: Are you only going to support the latest released OS and browser versions, or does it need to be compatible with older versions? Compatibility: Assume you need to support 5 browsers + iPhone + Android, you need backwards compatibility on all platforms and to support different devices; then you’ll see how the testing effort quickly scales up. Overall Test Approach Follow these 5 tips to speed up testing and quickly find critical bugs: Carry out cross-platform testing: Login with different users simultaneously on all supported platforms. Use both simulators and physical devices. You can now start testing. Check the most important functionality, the overall flows etc. Variations, combinations, details and edge cases can be checked later. Do everything that you think you shouldn’t be allowed to do! Try to make the system crash!! And of course check spelling, UI, design, alignment etc. You can of course test a lot with simulators. But remember to test on physical devices as well. They sometimes provide slightly different results 😉 In-depth QA Tests Here’s a detailed list of things to look out for in your current or next project. It’s a checklist for all members involved in your app development. Developers (backend + frontend), Designers (UX + UI), QA, QC, Project Managers, Product Owners and Support. Create account with email. Sign up: Sign up: Activate account via link in email. Sign up: Create password. Which rules apply to the password? Small and capital letters, numbers and special characters, minimum length etc. Successful login. Login: Login: Auto-login when opening the app. Login: Failed login using incorrect username and/or incorrect password. Login: Lock account on too many incorrect login attempts. Login: Reactivate account if it has been locked. Login: Test social logins like Facebook, Google, LinkedIn etc. Login: Test single-sign login. Login: Test two factor authentication. Log out from the application. Log out: Log out: Log out of all devices. Change password. Profile: Verify that you receive a password reset email or SMS code and that the updated password works as expected. Forgot password: Test all variations of the flow. E.g. if the onboarding splits up into different routes depending on a choice, if different templates/groups can be selected etc. Onboarding: Onboarding: Check that you can’t proceed if the user needs to select a minimum number of topics to follow. Onboarding: What happens if the user stops in the middle of the flow? Onboarding: Test the email onboarding sequence after sign up. Maybe you use services like , or ? Drip Mailchimp ActiveCampaign Does the application look properly (and good) when no content is shown? What happens when the first item is added? Empty state: Empty state: Delete all content and check the empty state. Is each field in the application the correct type? Single line of text, multiple lines of text, numeric, yes/no, dropdown, choice, lookup, date, time, person, calculated value, link etc. General: General: Can the field be edited or is it a view-only field (locked for editing)? General: Does the field have a minimum or maximum length? Does it require an exact number of characters (like credit card details where you need to provide an exact number of characters)? General: Highlight selected field. General: Display hint text in fields. Hide after the first character is added. Show hint text again if all characters in the field are deleted. General: Auto-capitalize text fields. General: Test if the field is case sensitive. General: Use special characters like &, ‘, %, æ, etc. General: Try to add spaces in all fields. General: Is it okay to have blank/empty fields? What if you insert a space and try to save? General: Type-ahead, are suggestions provided? General: Copy/paste text, hyperlinks and email addresses. General: Try to add emojis (e.g. by copy/paste from ). www.getemoji.com General: Try to type text in dropdown and time picker fields. General: Can you select multiple items when you should only be able to select a single value and vice versa. General: Set date in the past. Set date in the future e.g. next year. General: Show help text underneath fields. General: Show validation rules (e.g. when creating passwords). General: Are items updated in realtime? General: When should long text be truncated? General: Verify dynamic calculations are correct like storage used, number of episodes available etc. General: Test navigation. E.g. cancel, go back one step, tap outside of popup etc. General: Check error messages. General: Pick up where you left (remember last state). E.g. which page you last had open, which document you last viewed, how far into the podcast you had listened etc. General: Check that online/offline status appears correctly. Try to be online on several devices and shut down one after the other until you are no longer connected. General: Check if sharing works as expected. General: Can follows/likes be unfollowed/unliked? Is it dynamically reflected for other users. General: Check business rules if you have fields which are dependent on each other. E.g. if values filled in one field narrow the options available in the next field. Go back and change a previous field. Verify that dependent fields are updated/cleared. General: Check business rules where a field is hidden until a specific value has been filled in in the previous step. Create a new item. New: New: Quickly add new items one after the other. New: Copy/duplicate an existing item. Update an item and save it. Update/edit: Update/edit: Try to add incorrect input in a field (e.g. letters in a numeric field). Update/edit: Verify data is correctly saved — and displayed when you open the item again. Update/edit: Modify multiple fields (add content in some fields, delete in other fields, modify in some fields) and verify that data is saved correctly. Update/edit: Drag and drop to update items. Update/edit: Change icons. Update/edit: Check simultaneous editing with two users logged in and editing the same item at the same time. Update/edit: Can you archive an item? Update/edit: Bulk update items (e.g. if the name of your organization changes and you need to change all metadata in the system). Update/edit: If you have a locked item (e.g. an approved document), are some metadata “less important” allowing you to edit them? Example: Assume again that the documents have been tagged with your organization’s name and you want to update this. Delete an item (or leave a group). Delete: Delete: Verify deleted items are removed in real-time. Delete: Recover deleted items. Delete: Permanently delete items. Upload multiple items/files/images. Show spinner, progress bar or upload in background to indicate progress to the user. Multiple items: Multiple items: Select multiple items. Multiple items: Move multiple items. Multiple items: Delete multiple items. Add text in required fields → Create/save button becomes active. Popups (modals): Popups (modals): Delete text in a required field → Create/save button becomes inactive. Popups (modals): When clicking outside the popup, should the popup be closed, content saved or should it be kept open? Try to save/send if no text is entered. Text & Messages: Text & Messages: Send a single line of text. Text & Messages: Send long messages (e.g. +256 characters). Text & Messages: Use SHIFT+Enter for line breaks. Text & Messages: Send emojis. Text & Messages: Expand field if the user adds multiple lines? How much should the field be able to expand? Text & Messages: Add cursor in the beginning, middle and end of existing text. Then add some new text. The existing text must remain. Text & Messages: Add cursor in the beginning, middle and end of existing text. Then paste some text. The existing text must remain. Text & Messages: Add emoji in the middle of text — existing text must remain. Text & Messages: Indicate when other users are typing. Text & Messages: Indicate if a message is read. Text & Messages: Correct rendering of emojis across platforms. Text & Messages: Unicode conversion for emojis. Add image. Images: Images**:** Preview images including different resolutions and image ratios. Images: Zoom in/out. Images: Swipe quickly between images. Images: Draw, manipulate, rotate and save changes. Images: Delete image. Upload all types of files (doc, docx, xls, xlsx, jpg, png, pdf, mp4, zip etc.). Files: Files: Upload small and large files. Files: Upload files with different URL encoding. Files: Open and preview all file types (or a defined subset) and with different URL encodings. Files: Download all file types. Files: Rename files. Test file names which include special characters like ‘. Files: Upload as new version. Files: Upload and replace existing document — or avoid current documents with the same name to be overwritten. Files: Add folders and subfolders. Files: Navigate (expand/collapse) folders and subfolders. Files: Re-arrange folders and nested folders. Files: Check document versioning (minor and major versions). Files: Create PDF versions from other file types. Files: Print files or lock generated PDF copies for printing. Received links are clickable. Links: Links: Check all links work as expected. Record and add video. Video: Video: Play/preview video. Check features like voice, video, screen sharing and take over control. Calls: Calls: Check for issues where the microphone cannot be unmuted. We’ve experienced this often in the past when using e.g. Skype for Business where clients could hear us but we couldn’t hear them. Forcing the guests to leave and reconnect; and ultimately call by phone. Test the processes from the first to the last step. Forms: Forms: Test all loopbacks in the flow. Forms: Should previous steps be locked for editing? When to reopen for editing? Test the primary flow where you only select positive outcomes. Workflows: Workflows: Test all major flows with all loopbacks. Workflows: Start a workflow while another workflow is running. Workflows: Terminate workflow. Does it require username and password? Electronic signatures: Electronic signatures: What happens on rejection? Electronic signatures: What happens on approval? View audit trails. Audit trail: Audit trail: Verify dates, events, usernames/ID, old value, new value etc. are correctly captured in the audit trail. Audit trail: Try to modify audit trails. Audit trail: Try to delete audit trails. Audit trail: Export audit trails. Check the permission model. Security: Security: Try to access content you are not allowed to. Security: Check if you can see fields which should be hidden for you Security: Try to edit content you are not allowed to edit. Security: Try to start workflows you are not allowed to. Security: Verify only available workflows are present. Security: Who can participate in workflows? Security: Who can sign off? Security: What’s allowed to do on minor/major versions of a document? Security: Try to inject malicious code into the fields in your application. Security: Check that only admin account/users have access to the admin console and that ordinary users can’t access it. Security: Verify that different organizations can’t accidentally see/access each others data. Check latency from different countries (globally). Performance: Performance: Check what happens if the connection is interrupted. Performance: Should data be cached? Performance: Do you used local dbs? Check data is stored correctly, what happens if an item is deleted etc? Performance: Load old content dynamically, in background, upon scroll etc.? Performance: Test number of users logged in at the same time. Performance: Carry out load testing. Test of backup (full backup + incremental backups). Backup: Backup: Test of restore from backup. Should content be accessible offline? Offline mode: Offline mode: Sync when wifi/carrier signal is back. Offline mode: Check for sync/merge issues when multiple users try to modify the same item in online/offline mode. Validate email address on Enter. Invitations: Invitations: Validate email address on “Send/Add” without pressing Enter. Invitations: Invite multiple people. Try to use a long string, without separators, with space, with a comma, with semicolon etc. Invitations: Incorrect email addresses (without @, with spelling mistakes like @gamil.com instead of gmail.com etc). Invitations: Verify that invitation emails are sent and received. Invitations: Can you access contact details from your email, device or the friends’ list in the application? Does the referral program work? Do users e.g. get the expected credit when the people they invited signs up (ala Dropbox and Uber)? Referral program: Does the payment module work? Premium: Premium: Do users have access to premium features after they upgrade? Premium: Downgrade account and check that access to Premium features are removed (immediately or on the expiration of the subscription period). Reorder items and verify that the ordering is kept and displayed correctly through all platforms. Reorder: Reorder: Have a list with more than 10 items. Reorder items in the range 1–10 and +10 and check that the ordering is saved correctly. Mark/star/favorite/pin an item. Verify that the item is not affected when reordering items or applying sorting. Favorite: Check sorting (Alphabetic, Deadline, Categories, Create date, Tags etc.). Sorting: Sorting: Check sorting is not case sensitive. Sorting: Check date sorting using different years, months and days. Sorting: Check numeric sorting. The classic 1, 10, 11, 12… 2, 21, 22. Sorting: Have the app open with two different users using different sortings. Let one user edit an item. Verify that the correct item is updated on the second user’s screen. Check filters work as expected. Can multiple filters be applied simultaneously? Filters: Filters: Assume you want to filter on a specific user with the initials “NJ” and there’s another user with the initials “NJI”. Does the filter only show items which exactly matches “NJ”? Or does it also include results for “NJI”? Are suggestions provided when you start typing? Search: Search: Are the expected results returned? Search: Do you only see content/results which you are allowed to see? Check different views and that data is displayed correctly. E.g. list view, tile view, thumbnail view, tabular view etc. Views: Views: Toggle items on/off (like in calendars) and verify that the correct data is displayed in the view. Views: Expand/collapse views, hide/show columns etc. : Verify that data displayed in dashboards are correct. Reports & Dashboards Reports & Dashboards: Verify that data are correctly pulled into reports. Change team settings. Verify it applies for all users in the group. Settings: Settings: Change individual settings. Verify that it doesn’t affect any other user in the group. Settings: Change the time zone and verify that timestamps are correctly reflected for all users. Settings: Should the language in your app be shown according to the default detected language in the browser? Can it be manually changed? Check if relevant notifications appear in your notification center. Notifications: Notifications: Can you clear notifications? Notifications: Are notification cleared after an item has been viewed? Notifications: Turn-off email notifications and verify that they are not sent. Notifications: Check that unsubscribe to emails work as expected. Notifications: Verify that alarms/reminders go on off on the scheduled time. Notifications: Are people notified when mentioning them? What happens if they have disabled notifications? Notifications: When the user follows a link in an email notification where should it open? Should it re-direct to the mobile app or web app? Copy/paste. Shortcuts: Shortcuts: Use of TAB between field. Shortcuts: Use of other keyboard shortcuts. Check import tools. Tools: Tools: Check export tools including printing. Tools: Check migration tools (export then import). Check functionality in Chrome, Firefox, Safari, Internet Explorer, Edge etc. Browsers: Browsers: Check compatibility with previous browser versions. Browsers: Check scroll (vertical + horizontal). Browsers: Check that content is still shown correctly if you resize the browser. Check the browser size with e.g. to see how it looks on e.g. a MacBook with a resolution of 1366 x 768 pixels. browsersize.com Browsers: If you use a hellobar (e.g from ) check that it doesn’t impact the UI of your application. Hellobar.com Update content in the navigation bar dependent on which screen the user has selected. Mobile: Mobile: Keyboard appears when a field is tapped. Mobile: Keyboard is dismissed on scroll or tapping elsewhere on the screen. Mobile: Check swipe gestures. Mobile: Tap quickly anywhere on the screen (also outside of fields, near the border of the screen etc.). Do you receive push notifications in killed state? Push notifications: Push notifications: Do you receive push notifications when app is running in background? Push notifications: Do you receive push notifications when app is in foreground? Push notifications: Do you receive push notifications when the app returns from offline mode to online mode? Push notifications: Turn-off push notifications and verify that you don’t receive push notifications. Turn it back on and verify that push notifications are received. Test backwards compatibility with iOS12, iOS11, iOS10… E.g. iPhone5 only supports iOS9. iPhone: iPhone: Clear badge count. When? iPhone: Display push notifications in the notification center and when to clear them? Backwards compatibility (Oreo, Nougat, Marshmallow etc.). Android: Android: Are push notifications grouped correctly (if you group push notifications)? Check functionality in Window desktop app. Desktop app: Desktop app: Check functionality in Mac app. Can external apps be added inside your application? Integrations: Integrations: Do integrations to other apps work as expected? Are data exported, imported and sync’ed correctly? Integrations: Have you tested your API? Do your bots provide relevant information and correct answers? Bots: Upgrade your data model. Check if anything breaks for users on a previous version. If things break, check that mechanisms to auto-upgrade them to the latest stable version works as expected. Upgrades: Upgrades: If you deliver an on-premise solution (like SharePoint), test different upgrade paths from one version to another (direct from an exact version to the latest version, skip minor intermediate versions etc.). Ultimately, check that you can delete your account. Delete account: Speed vs. Quality If you ship often and your app is feature-rich you’ll probably need one or more full-time QA engineers to ensure that you continouosly deliver a high quality app. This was Facebook’s original motto. It may be a fine approach for early startups without too much traction but you (and your customers) need stability as you mature. Move fast and break things. This is why Facebook changed their motto to since fixing bugs was slowing down development. Move fast with stable infrastructure Facebook’s original and new motto. How to help your engineers? There’s nothing more frustrating for your developers if you say “it doesn’t work” without providing any details. Provide the following information to make your developers happy. It will of course also speed up the time to fix the observed issue 😉 Record the time the issue happened. Make sure you can reproduce the issue consistently. Describe the exact steps you follow. Open your browser and press F12. Then reproduce the issue. Open the Console and Network tabs and provide the results to the developers. N.B. Clearing the data in the Console and Network before you reproduce the issue is a good idea. Manual or Automated Testing? If you build a cross-platform solution you’ll have a huge amount of test combinations. So you need to develop your test strategy wisely. Do the right amount of tests — not too little — but also do not test all combinations. If you are in QA but doesn’t have any coding experience, it might be worth talking to the engineers since some of the things you think is important to test might be irrelevant. You can setup automated test tools — or carry out manual test. What’s the best choice depends on the maturity of the product and the size of your company. Automated test tools sounds great, but you need a really mature product before it makes sense. You need to know that core features, UI and UX is not going to change for a long period. Otherwise, you’d constantly have to update the test script. Automated test tools also doesn’t spot things which are not right. It can be softer things like UI, spelling, alignment etc. which is important for the user experience. Final Words We built (a collaboration app for teams) based on lots of the principles and tests listed in this article. SquidHub We’re curious to hear you best tips. Please comment below and we’ll update this post (to the benefit of all app developers, product managers, test engineers etc). If you enjoyed reading this article or found it helpful, please clap 👏 and share it with your network. It would mean the world to us. Thanks. has worked with IT project management and quality testing for +10 years and has originally a background from working with SharePoint. Andreas Overbeck He is Founder & CEO of ; A team collaboration app designed to make teams move faster and collaborate efficiently. Without switching apps (or tabs) you can access your team’s tasks, files and messages. SquidHub SquidHub explained in 1 min 39 sec. Thank you for reading — and have a productive workday!
