Sometimes when I read a book I like to make a list of the words to learn more about. For the last book I read here is that list in order of appearance: ElephantineErsatzHeterochromaEchtBanach–Tarski paradoxChiaroscuroPlosiveObeliskReedyTautologySatyrPalimpsestPlumeriaAuteurTreaclyMiasmaUmlautsDenouementVerisimilitudePortmanteausToriiSomnolentCiceroneVerisimilituBromidePompadourPriggishShtetlCopaceticDilettanteAmelioratedHirsuteStygianAkimboOnyxSepulchralPenumbralTalismanicVerbotenSusurrusImpedimentaEcht Elephantine Ersatz Heterochroma Echt Banach–Tarski paradox Chiaroscuro Plosive Obelisk Reedy Tautology Satyr Palimpsest Plumeria Auteur Treacly Miasma Umlauts Denouement Verisimilitude Portmanteaus Torii Somnolent Cicerone Verisimilitu Bromide Pompadour Priggish Shtetl Copacetic Dilettante Ameliorated Hirsute Stygian Akimbo Onyx Sepulchral Penumbral Talismanic Verboten Susurrus Impedimenta Echt Then I asked Claude (Anthropic), ChatGPT (OpenAI), and Gemini (Google), based solely on this list of words by order of appearance, could they guess the book? Now if an AI could answer that, it would be well versed in my current interests. And maybe has too much training data? Here are their responses in full. Claude: ChatGPT: Gemini: One of the three AIs correctly guessed the book. The other two did not. I replied "guess again" 10x and they each guessed a lot of interesting books (to name a few repeated ones: The Dictionary of Lost Words, Permutation City, Foucault’s Pendulum, The Secret History, Infinite Jest, The Name of the Rose, The Overstory, and Gödel, Escher, Bach: An Eternal Golden Braid) but not the one I just read. Now, which book is it? Let me just say this book is bound by the intimacy of gaming. It challenges the idea of the "NPC," revealing that tomorrow is a Shakespearean reference and a promise of digital respawns.  Also it's in America, so there's a guy with a gun who changes the story. No more spoilers or thoughts but if you have a book recommendation, leave a comment. The book I read was: Tomorrow and Tomorrow and Tomorrow by Gabrielle Zevin. Tomorrow and Tomorrow and Tomorrow Gabrielle Zevin Why is the The Words of Interest a Good Benchmark Test for Which AI You Should Use? If someone hasn't read the book, do you want them at bookclub? Maybe if they're your friend or they bring quality food. But mostly, no, people who go to book club want to talk to those who read the book. By picking individual words instead phrases or paraphrases or passages, this test bypasses plot summaries (which are everywhere regurgitating themselves online) and focuses on the author's words. It reveals whether an AI has truly "absorbed" the specific texture of a book or is simply echoing the general internet consensus. I asked Gemini if Google got it right because it had access to more information: Anyways, the next day we switched Proof of Usefulness over to Gemini's APIs :-) Proof of Usefulness

The writer is smart, but don't just like, take their word for it. #DoYourOwnResearch before making any investment decisions or decisions regarding your health or security. (Do not regard any of this content as professional investment advice, or health advice)

The is an opinion piece based on the author’s POV and does not necessarily reflect the views of HackerNoon.

The Words of Interest Benchmark Test For Matching an LLM to Your Interests

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

PB & J: WHAT IS IT GOOD FOR ??

Why Being Likable At Work Matters

Windows Sticky Keys Exploit: The War Veteran That Never Dies

Zero People Charged With Online Pirating, Swedish Prosecutor's Office Reports

Is ‘bias for action’ making product managers lazier?

What No One Told Me About Being a Product Manager at an Early Stage Startup

PB & J: WHAT IS IT GOOD FOR ??

Why Being Likable At Work Matters

Windows Sticky Keys Exploit: The War Veteran That Never Dies

Zero People Charged With Online Pirating, Swedish Prosecutor's Office Reports

Is ‘bias for action’ making product managers lazier?

What No One Told Me About Being a Product Manager at an Early Stage Startup

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps