TL;DR: To what extent do digital designers and user researchers in industry make use of AI-powered systems to support their work? The answer seems to be “not much,” as we found in a survey with 34 practitioners. In general, there seems to be little awareness of the specific advantages AI can bring to a design process and which tools are already out there, despite significant issues with existing processes that could be well mitigated with the help of AI. However, designers and researchers are very open to the topic and would appreciate AI-powered systems to support them in both ideation and evaluation. Providing practitioners with such systems that are easily accessible and demonstrate added value holds great opportunities. This article describes research that has been conducted in collaboration with , , and . The [2] was published at the (ICWE). Maxim Bakaev Johanna Jagow Sebastian Heil research paper 2022 International Conference on Web Engineering Design and research are a thing. (AI) is a thing. But they’re not quite a thing together yet. There have been massive advances in AI over the past decades. Yet, designers and user researchers in the digital industry still often execute human-centered design processes manually and tediously — from clicking together prototypes to conducting fiddly usability tests to adjusting a design to different screen sizes. (If they have the time and resources to do it properly, that is.) The specific software designers and researchers are using has changed, and there are advances in, e.g., remote asynchronous user testing. But apart from that, the way design is being done, by and large, still doesn’t look much different from what Alan Cooper described in the first edition of in the 1990s (except for design thinking, but that’s ). While that’s not a bad thing at all (you should really listen to Alan Cooper when you wanna design properly), in today’s fast-paced, dynamic environments with their ever-growing amounts of devices, user segments and personalization strategies, tight budgets, and even tighter timelines, we could all use some help. And in the case of digital design and user research, that is where AI comes into play. Artificial intelligence that About Face a different story Or so we thought. There’s a variety of tools out there that leverage the power of AI and aim at supporting designers and researchers in their processes. Some strive to progress the automatic creation of wireframes and interfaces (e.g., [3] and [4]). Some aim at being able to replace traditional approaches to usability testing (e.g., [5] and [7]). Still others automatically generate interface-related performance metrics (e.g., [1] and the [6]). Yet, none of these seem to be overly popular among practitioners. There’s a clear gap between what’s already out there in terms of AI-powered tools and what designers and researchers rely on in their daily work. Is this a marketing problem, since most of these tools stem from academic projects? Or do practitioners see no added value in them? Paper2Wire Chen et al. ’s work USF WaPPU Bakaev et al. ’s work Aalto Interface Metrics To get to the bottom of this, my colleagues Maxim Bakaev, Johanna Jagow, and Sebastian Heil and I initiated a research project in which we set out to answer two research questions: and, more importantly Do practitioners indeed not have the time and resources to follow proper design processes? Why don’t they leverage AI-powered tools to mitigate that issue? What did we find? Based on an online survey we conducted (we’ll get to the details later), first of all, designers’ and researchers’ processes in industry indeed seem to be pretty “standard,” as mentioned in the introduction above. That is, we mostly heard about some kind of human-centered design done along some variation of the double-diamond model. There was no mention of AI when asking about existing processes. Second, the major obstacle to a proper design process seems to be (easy) access to the right users for research; and if there is access and proper user research could be done, conflicting corporate timelines and limited resources. Third, a considerable amount of designers and researchers seem to know and use validation and checking tools such as or the . Also, tools for automatically generating UX-/UI-related metrics are relatively well known. However, the opposite hold for tools that enable automated (i.e., “user-less”) usability evaluation and the like. Overall, the AI-based kinds of tools we were after in our survey seem to be virtually unknown to designers and researchers in industry. Test.ai W3C Validator Fourth, when asking about the reasons for not working with AI-powered tools, the top three responses were a lack of familiarity (cf. point 3 above), cost considerations, and a perceived lack of added value. Based on these findings (please refer to for a more detailed analysis), we formulated three challenges and three opportunities w.r.t. existing design processes and how to potentially support them with AI (taken from [ ]): the actual research paper 2 “– CHALLENGE 1. Finding and recruiting relevant participants for user studies, especially for specialized products. Availability of users. “– CHALLENGE 2. Tight deadlines and timeframes limited by stakeholders and processes, and a lack of user research resources Lack of time and resources. . “– CHALLENGE 3. There seems to be little awareness of what exactly ML-/AI-based systems constitute and in which ways they can support design processes. Designers’ unfamiliarity with ML/AI. “– OPPORTUNITY 1. In principle, digital designers and user researchers welcome tools that support their processes and they do follow the topic (e.g., ). Openness to new things. “Not a part of our current process, but open to trying them if they can mitigate our issues finding qualified users.” “– OPPORTUNITY 2. New AI-/ML-based tools do have a chance to gain traction as currently, there seem to be no established players. Open playing field. “– OPPORTUNITY 3. Specifically ideation (in the sense of creating wireframes, mockups, and prototypes) and evaluation (of design artifacts) are mentioned as the activities in which practitioners would appreciate support.” Support for ideation & evaluation. How did we find it? Our findings are based on a survey we conducted with 34 digital designers and user researchers from industry, a majority of which came from Germany or the U.S. While this is not exactly the largest sample size we’ve ever worked with, our participants had a lot of experience in their field (more than 8 years on average), with several reporting to work for major companies, including IBM, Microsoft, Meta, and Bosch. The survey itself comprised 28 questions that could be completed in 20–40 minutes, hence the relatively small sample size (boy, it’s really difficult to get people to fill out surveys that take longer than 5 minutes!). It inquired into participants’ design processes, their familiarity with common design tools like InVision and Figma, their familiarity with a set of “user-less” tools, and demographics and open-ended feedback. Of these, the third part — familiarity with “user-less” tools — was of particular interest to us since it contained all of the AI-powered tools for supporting design processes. It’s no exaggeration to say it was the core of the whole exercise. Overall, we identified a set of 61 “user-less” tools and platforms — both academic and commercial — that supported at least semi-automation and were further classified into categories such as “generative tools based on AI/ML,” “automated usability validation/evaluation,” “behavior simulation/GUI testing automation,” etc. For each of the 61 tools, we asked the survey participants whether they know them and whether they actively use them, with the sobering result described above. For a complete list of the tools and categories, please check out our . online appendix What comes next? We’re well aware that we didn’t run a representative study and that there is, among other things, still a need to gain deeper qualitative insights about designers’ and researchers’ processes and (potential) interactions with AI-based systems. Yet, our research has already yielded some valuable indications. Mainly that restricted timelines and resources in industry are a major and very real obstacle to the proper execution of design processes (a confirmation of what everyone knows already); access to appropriate users for research is often a pain point; and AI could help with the above two, e.g., by automating tedious, mechanical parts of a design process and providing access to “user-less” evaluation. Yet, there seems to be a lot of room for improvement when it comes to leveraging this support to alleviate strain from practitioners. As next steps in our research agenda, we plan to extend the set of 61 “user-less” tools and platforms into a fully-fledged taxonomy, gather the above mentioned deeper qualitative insights, and work towards a specific set of requirements that practitioners have for efficient and effective AI support of their everyday work. With all of these, we want to work towards a deeper understanding of what the specific problems with existing AI-powered systems are and what needs to change so that they gain more traction among designers and developers in industry. So, stay tuned, . there’s more of this research to come In the meantime, you can head over to ResearchGate and give our paper “ ” [2] a thorough read for more detailed information on our research and its results. We Don’t Need No Real Users?! Surveying the Adoption of User-less Automation Tools by UI Design Practitioners ☕ I love coffee, and if you want to support my work, you can always , or subscribe to . 🗞️ spend me one my newsletter References Bakaev, Maxim, Sebastian Heil, Vladimir Khvorostov, and Martin Gaedke. “ .” 17, no. 6 (2018): 561–590. Auto-extraction and integration of metrics for web user interfaces Journal of Web Engineering Bakaev, Maxim, Maximilian Speicher, Johanna Jagow, Sebastian Heil, and Martin Gaedke. “ .” In , pp. 406–414. Springer, Cham, 2022. We Don’t Need No Real Users?! Surveying the Adoption of User-less Automation Tools by UI Design Practitioners International Conference on Web Engineering Buschek, Daniel, Charlotte Anlauff, and Florian Lachner. “ .” In , pp. 33–41. 2020. Paper2Wire: a case study of user-centred development of machine learning tools for UX designers Proceedings of the Conference on Mensch und Computer Chen, Chunyang, Ting Su, Guozhu Meng, Zhenchang Xing, and Yang Liu. “ .” In , pp. 665–676. 2018. From UI design image to GUI skeleton: a neural machine translator to bootstrap mobile GUI implementation Proceedings of the 40th International Conference on Software Engineering Grigera, Julián, Alejandra Garrido, José Matías Rivero, and Gustavo Rossi. “ .” 97 (2017): 129–148. Automatic detection of usability smells in web applications International Journal of Human-Computer Studies Oulasvirta, Antti, Samuli De Pascale, Janin Koch, Thomas Langerak, Jussi Jokinen, Kashyap Todi, Markku Laine et al. “ .” In , pp. 16–19. 2018. Aalto Interface Metrics (AIM) A Service and Codebase for Computational GUI Evaluation The 31st Annual ACM Symposium on User Interface Software and Technology Adjunct Proceedings Speicher, Maximilian, Andreas Both, and Martin Gaedke. “ .” In , pp. 93–110. Springer, Cham, 2014. Ensuring web interface quality through usability-based split testing International Conference on Web Engineering Originally published here.

This story contains new, firsthand information uncovered by the writer.

The writer has included referral links in this story.

Figma

Google

Microsoft

What Needs to Happen Before AI Becomes a Thing in Design and Research

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

A Smidge of Behavioral Science at the End of the Year

The Noonification: Use This 7-Step McKinsey Framework to Solve Any Problem (1/10/2023)

The Noonification: A Taxonomy of Inclusiveness (1/11/2024)

The Noonification: What is the InfiniteNature-Zero AI Model? (11/19/2022)

10 Ways AI Has Changed Our Lives

100 Days of AI, Day 8: Experimenting With Microsoft's Semantic Kernel Using GPT-4

A Smidge of Behavioral Science at the End of the Year

The Noonification: Use This 7-Step McKinsey Framework to Solve Any Problem (1/10/2023)

The Noonification: A Taxonomy of Inclusiveness (1/11/2024)

The Noonification: What is the InfiniteNature-Zero AI Model? (11/19/2022)

10 Ways AI Has Changed Our Lives

100 Days of AI, Day 8: Experimenting With Microsoft's Semantic Kernel Using GPT-4

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps