Writing tons of headlines, descriptions and error messages for a mobile app or a web service could be really time-consuming, especially for non-native speakers, so using an AI tool like ChatGPT (or ChatGPT itself) for something like this is a really great opportunity.
My past experience, documented in this article on ChatGPT for Product Managers, was less than stellar. However, I may have been wrong and based my approach on just one small and highly specific scenario. Let’s do a deep dive into what ChatGPT can and cannot do for user interface copy.
Does telling ChatGPT that it’s a skilled copywriter help?
I’ve seen this “hack” countless times online. People write a prompt in the gist of “for all future prompts assume you are a skilled copywriter with microcopy experience” to get better results. Does this work? (Spoiler: it does for basic tasks, but can deliver horrible results for the more advanced stuff)
I’ll be testing all the scenarios below with two ChatGPT personas: regular ChatGPT and a “skilled copywriter”. The results are different, but it’s up to you to decide whether one is better than the other.
The scope
In this article, I want to focus on the most boring stuff that one has to write: error messages and empty states. If ChatGPT can do these, I’m pretty sure it’s actually capable when it comes to micro-copy and UI strings.
Error messages
No internet connection
Garden variety ChatGPT is needlessly verbose, but the “skilled copywriter” delivers a good result. It’s a win for AI today (until there’s really no internet connection).
Sign in failed
Another win for the “skilled copywriter” persona. A very good error message.
Session expired
You should start seeing the trend: basic ChatGPT is really, really verbose with these.
Incorrect email format
This is getting obvious by now: all the basic error messages will be really well-done if you use the “skilled copywriter” persona.
Specific AR use cases
This is what made me very critical of ChatGPT’s writing ability in the past. Can the new persona help my old prompt? Can a different prompt save the situation?
Original prompt
In my previous article, I used this prompt:
Write a short UI String that would tell a user of an AR try-on experience to step away for the experience to work better. 4 words max.
Back then, the answer was Step back for AR.
Since then, regular ChatGPT has gotten better at this and suggests Step back for accuracy, which makes a lot more sense.
However, for some reason, the “skilled copywriter” persona ignores the prompt’s limitations and gives a really useless response:
Title: Step Back for AR
Description: Distance from camera required.
Maybe I’m demanding too much here or I am being too controlling? What if ChatGPT can deliver good results with extra details?
Standing too close
Prompt: write a one-line error message for when the user is standing too close to the camera for the experience to work
Surprise! Regular ChatGPT performs this INCREDIBLY. Here’s its response:
"Step back for better experience."
It actually makes the impressive jump from simply stating the error to actually giving the user a very useful action to perform. 10/10, something I would have done (and have done).
Whereas the “skilled copywriter” persona continues to ignore extra requirements and provides an extended version of the same thing. Perhaps, being a skilled copywriter means you can ignore minute stuff like that?
Title: Move further from camera
Description: Please step back for better experience.
Standing too far
The results are exactly the same: regular ChatGPT performs better in unusual scenarios.
Empty states
Generic
Prompt: write text for a generic empty screen state
The “skilled copywriter” is winning again! But is this really a generic empty state? Looks more like it’s meant for an empty search result.
Web bookmarks
Prompt: write text for an empty state of a web bookmarks screen
Regular ChatGPT provides an arguably better headline, but the actual message is better when using the “skilled copywriter” trick.
Favorites in an e-grocery app
Prompt: write text for an empty state of an e-grocery app favourites screen
While it’s more verbose, the regular ChatGPT’s answer gives a better user experience, since it explains what Favourites are for. Like many human writers, it just needs a helping hand of an editor.
Messages
Prompt: write text for an empty state of a conversation in a messaging app
Both varieties fail here for some reason. AI generation definitely has its limitations and looks like discerning between a conversation and a list of conversations is one of them. Still, regular ChatGPT wrote a nice header.
Conclusion
I definitely was wrong when it comes to ChatGPT’s UI writing capabilities. It can absolutely deliver some excellent results when it comes to error messages, including coming up with the very clever “Please step back” instead of “You are too close”, something even humans can struggle with at times.
ChatGPT has its limitations, but if you are in any way involved in writing monotonous error messages, empty state copy and more, then it will definitely be able to make your life better.
But don’t rely on the “skilled copywriter” trick too much, as it can backfire and deliver worse results than the regular state. Try using several approaches, and it’s still going to be easier than writing all the copy yourself.