No Testing Is (Sometimes) Better Than Some Testing

Dear readers & friends,

So, you might have noticed by now that “research done wrong in industry” is kind of my pet peeve, and I’m sorry you have to endure that. However, I’m not quite done with the topic yet. My new article in The UX Collective, “No research is (often) better than ‘some’ research,” takes a closer look at the two popular mantras, “if it’s not tested it’s just another opinion” and “some research is better than no research” and concludes that both are B.S.

I’ve already made the case that truly innovative designs can’t be reliably evaluated by the methods commonly applied in industry nowadays (especially A/B tests) in “Listen to users, but only 85% of the time”* and this new article is kind of an extension to that. In addition to reïnforcing the point that “some” research might kill innovation more than it furthers it, it highlights a second case in which it does more harm than good: designs that are soundly based on established principles.

However, I’m not just complaining. I also propose how we can do a better job *and* empower others to do so as well. Have fun reading!

Also, as usual, I wanna recommend some other great resources related to the above:

Reading Recommendations

Since a lot of companies (and very badly informed individuals) are now also trying to achieve research “democratization” and “empowerment” through ChatGPT and the likes, despite being a bit older already these two reads are still highly relevant:

📖 ChatGPT cannot do user research by Jason Godesky explains why you can’t train ChatGPT on your own user data and, therefore, expect any kind of correct inferences for your user research.

📖 And **Testing ChatGPT-4 for ‘UX Audits’ Shows an 80% Error Rate & 14–26% Discoverability Rate **by Christian Holst of Baymard Institute has a pretty self-explanatory title, I guess.

Tool Recommendations

🛠️ When to Use Which User-Experience Research Methods by Christian Rohrer is an article, but also a powerful tool to help you learn about and choose the right mix of methods for your research questions.

Enjoy reading and until next time. I really appreciate your being interested in my content. As always, I welcome your feedback and comments on my articles and this newsletter. Simply reply to this email to share your thoughts. 🙏🏻

Cheers,
Max

* My articles are usually behind the Medium paywall. You can get the friend links by subscribing to my newsletter.