Or did I? Only Google’s new quantum computing chip knows for sure.

Google's New Quantum Computing Chip Willow Feeds Multiverse Theory - Bloomberg

Google's Willow chip reduces errors exponentially, addressing a problem the industry has faced for almost 30 years.

Why Google's quantum computing breakthrough is such a big deal

Cricket News: Cricket enthusiasts in the USA are eagerly awaiting the Indian Premier League (IPL) 2024, starting on March 22. The &#x27;<strong>Willow</strong> by Cricbuzz&#x27; app offers li

Watch IPL in USA: How to watch IPL 2024 live streaming for free in USA on 'Willow by Cricbuzz' App with Abound | Cricket News - Times of India

Cricket News: <strong>Willow</strong>, the top US and Canadian cricket broadcaster, has partnered with Cricbuzz, the world&#x27;s largest cricket website and app, to launch &#x27;<strong>Willow</strong> by Cr

'Willow by Cricbuzz' launches as new live streaming home for cricket in the United States and Canada | Cricket News - Times of India

It's a wonderful world of lawsuits as the Mouse House sees legal action from both ends.

Disney Sues Florida as Willow Star Ralph Ineson Sues Disney

Heather Benavidez and Stephen Cecco are running in District Four, which encompasses the southeast part of the town.

Meet the District Four Candidates for Albuquerque Public Schools Board

Willow Smith turned 23 on October 31, and celebrated her big day with her loved ones, who all shared birthday messages dedicated to the singer on social media. When it came to Will Smith's tribute, the Whip My Hair hitmaker had quite the surprising reaction too!

Willow Smith's feisty response to dad Will Smith's gushing birthday tribute revealed

For September and October, EHHS' Trevor Vaccino, nominated by Science Dept. and Willow Torres, nominated by English Dept, are the honorees.

East Haven Rotary Club Honored 1st Students Of The Month For 2023-2024

The Willow Creek Sportsman’s Club has gone from an idea to a brick-and-mortar clubhouse in just two short years.

Willow Creek Sportsman’s Club seeing fantastic growth

Willow

willow

As a Web3 writer, I tend to shy away from deep tech topics. But when I heard Google’s latest announcement about their new quantum computing chip, Willow, I couldn’t look away. The headline-grabbing claim? A computational problem solved in 5 minutes that would take a classical supercomputer a staggering 10 septillion years to complete. For context, that’s 10,000,000,000,000,000,000,000,000 years — a number so vast that it makes the 13.8 billion-year age of the universe look like a lunch break. Naturally, the buzz was immediate: “Quantum computing is here to crack Bitcoin!” “No encryption is safe!” But hold your horses. Willow’s significance isn’t about rendering our current security systems obsolete overnight. It’s about a new way forward — for computing, for science, and yes, for crypto. Let’s break it down. What Exactly Is Google’s Willow? Google’s Willow chip is a quantum computing breakthrough. It’s not the largest quantum processor out there — it has 105 qubits. But what makes it revolutionary is how it deals with errors. Why Errors Matter in Quantum Computing? In traditional computers, bits represent either a 0 or a 1. But quantum computers use qubits, which can represent both 0 and 1 simultaneously thanks to the principles of superposition and entanglement. This ability lets quantum computers perform multiple calculations at once, potentially solving problems classical computers could never touch. The problem? Qubits are delicate. They’re easily disturbed by their environment, which leads to errors. And the more qubits you add, the more errors you typically get. This challenge has haunted quantum computing researchers for decades. Willow changes the game by demonstrating that as you add more qubits, the number of errors actually decreases. This is huge. It’s a critical step toward building large-scale, reliable quantum computers — the kind that could reshape industries like drug discovery, battery development, and material science. But where does that leave Bitcoin and crypto security? Should Crypto Holders Be Worried? The fear is simple: Quantum computers could theoretically break the cryptographic algorithms that secure Bitcoin and other cryptocurrencies. Bitcoin relies on two key pieces of encryption: 1 - SHA-256 for mining. 2 - ECDSA (Elliptic Curve Digital Signature Algorithm) for transaction signatures. Quantum computers could, in theory, brute-force these algorithms, rendering wallets and transactions vulnerable. If someone could crack ECDSA, they could steal funds by forging signatures. But here’s the thing: Willow isn’t that quantum computer. Not yet. According to tech experts, including Kevin Rose (former Google product manager), cracking Bitcoin’s encryption would require a quantum computer with about 13 million qubits — a far cry from Willow’s 105 qubits. So, no, you don’t need to sell your Bitcoin and go off-grid just yet. Why Quantum Computing Still Matters for Crypto Even though Willow can’t break Bitcoin’s encryption today, it’s a wake-up call. It shows that quantum computing isn’t some sci-fi fantasy. It’s evolving rapidly. And each breakthrough — like Willow’s error reduction — brings us closer to a world where quantum computers might pose a real threat to current cryptography. https://x.com/adamscochran/status/1866314726030143606 The Timeline Is Shortening Crypto venture capitalist Adam Cochran put it succinctly: Quantum computing advancements have shaved 20 years off the timeline for quantum threats to Bitcoin. Another breakthrough like Willow, and we might be looking at a 10-year window to prepare. This isn’t just about Bitcoin. Ethereum, smart contracts, and other blockchain-based systems rely on similar cryptographic protections. Quantum computing could jeopardize the entire Web3 ecosystem if we’re not ready. How Crypto Can Defend Itself Thankfully, the crypto community isn’t sitting idle. There are already discussions around post-quantum cryptography — encryption methods designed to withstand quantum attacks. 1 - Upgrading Bitcoin’s Security Crypto experts like Emin Gün Sirer, co-founder of Ava Labs, have suggested practical steps to protect Bitcoin: - A - Sunsetting Outdated Formats Early Bitcoins, mined by Satoshi Nakamoto, use an old format called P2PK (Pay-to-Public-Key). Unlike modern wallets, this format exposes public keys, making it vulnerable to future quantum decryption. Sirer recommends freezing these coins or setting a sunset date for P2PK transactions. B - Implementing Post-Quantum Algorithms Transitioning Bitcoin to quantum-resistant algorithms is complex but doable. It would likely require a hard fork and widespread adoption of new wallet software. 2 - Ethereum’s Preparedness Ethereum co-founder Vitalik Buterin has already proposed a strategy for Ethereum’s transition to post-quantum security. It would involve a hard fork and user cooperation to upgrade wallets, but it’s a clear and viable plan. Quantum Computing: A Threat or an Opportunity? While the threat to crypto security is real, quantum computing isn’t just about breaking things. It’s about building new possibilities. Imagine a world where quantum computers can: Simulate complex chemical reactions to discover life-saving drugs.


Optimize battery designs to extend electric vehicle range.


Model financial systems to prevent market crashes. These breakthroughs could transform industries and unlock untold value. And yes, Web3 could benefit too. Quantum computing could lead to more secure blockchains, faster consensus algorithms, and decentralized systems that are truly unbreakable. The Real Takeaway Google’s Willow isn’t here to kill Bitcoin. It’s here to remind us that technology evolves, and we need to evolve with it. If you’re in Web3, now’s the time to: Stay Informed: Follow quantum computing advancements. They matter more than ever.
Advocate for Post-Quantum Security: Encourage discussions and research around quantum-resistant encryption.
Don’t Panic — Prepare: The future isn’t something to fear. It’s something to build. Quantum computing is coming. The question is: Will we be ready? What are your thoughts on the quantum threat to crypto? Are we prepared, or is it time to hit the panic button? As a Web3 writer, I tend to shy away from deep tech topics. But when I heard Google’s latest announcement about their new quantum computing chip, Willow, I couldn’t look away. The headline-grabbing claim? A computational problem solved in 5 minutes that would take a classical supercomputer a staggering 10 septillion years to complete. 10 septillion years For context, that’s 10,000,000,000,000,000,000,000,000 years — a number so vast that it makes the 13.8 billion-year age of the universe look like a lunch break. Naturally, the buzz was immediate: “Quantum computing is here to crack Bitcoin!” “No encryption is safe!” But hold your horses. Willow’s significance isn’t about rendering our current security systems obsolete overnight. It’s about a new way forward — for computing, for science, and yes, for crypto. Let’s break it down. What Exactly Is Google’s Willow? Google’s Willow chip is a quantum computing breakthrough. It’s not the largest quantum processor out there — it has 105 qubits. But what makes it revolutionary is how it deals with errors. Why Errors Matter in Quantum Computing? In traditional computers, bits represent either a 0 or a 1. But quantum computers use qubits, which can represent both 0 and 1 simultaneously thanks to the principles of superposition and entanglement. This ability lets quantum computers perform multiple calculations at once, potentially solving problems classical computers could never touch. The problem? Qubits are delicate. They’re easily disturbed by their environment, which leads to errors. And the more qubits you add, the more errors you typically get. This challenge has haunted quantum computing researchers for decades. Willow changes the game by demonstrating that as you add more qubits, the number of errors actually decreases. This is huge. It’s a critical step toward building large-scale, reliable quantum computers — the kind that could reshape industries like drug discovery, battery development, and material science. But where does that leave Bitcoin and crypto security? Should Crypto Holders Be Worried? The fear is simple: Quantum computers could theoretically break the cryptographic algorithms that secure Bitcoin and other cryptocurrencies. Bitcoin relies on two key pieces of encryption: 1 - SHA-256 for mining. 2 - ECDSA (Elliptic Curve Digital Signature Algorithm) for transaction signatures. Quantum computers could, in theory, brute-force these algorithms, rendering wallets and transactions vulnerable. If someone could crack ECDSA, they could steal funds by forging signatures. But here’s the thing: Willow isn’t that quantum computer. Not yet. According to tech experts, including Kevin Rose (former Google product manager), cracking Bitcoin’s encryption would require a quantum computer with about 13 million qubits — a far cry from Willow’s 105 qubits. According to tech experts, including Kevin Rose (former Google product manager), cracking Bitcoin’s encryption would require a quantum computer with about 13 million qubits — a far cry from Willow’s 105 qubits. So, no, you don’t need to sell your Bitcoin and go off-grid just yet. Why Quantum Computing Still Matters for Crypto Even though Willow can’t break Bitcoin’s encryption today, it’s a wake-up call. It shows that quantum computing isn’t some sci-fi fantasy. It’s evolving rapidly. And each breakthrough — like Willow’s error reduction — brings us closer to a world where quantum computers might pose a real threat to current cryptography. https://x.com/adamscochran/status/1866314726030143606 https://x.com/adamscochran/status/1866314726030143606 The Timeline Is Shortening Crypto venture capitalist Adam Cochran put it succinctly: Quantum computing advancements have shaved 20 years off the timeline for quantum threats to Bitcoin. Another breakthrough like Willow, and we might be looking at a 10-year window to prepare. The Timeline Is Shortening Crypto venture capitalist Adam Cochran put it succinctly: Quantum computing advancements have shaved 20 years off the timeline for quantum threats to Bitcoin. Another breakthrough like Willow, and we might be looking at a 10-year window to prepare. This isn’t just about Bitcoin. Ethereum, smart contracts, and other blockchain-based systems rely on similar cryptographic protections. Quantum computing could jeopardize the entire Web3 ecosystem if we’re not ready. How Crypto Can Defend Itself Thankfully, the crypto community isn’t sitting idle. There are already discussions around post-quantum cryptography — encryption methods designed to withstand quantum attacks. 1 - Upgrading Bitcoin’s Security Crypto experts like Emin Gün Sirer, co-founder of Ava Labs, have suggested practical steps to protect Bitcoin: - A - Sunsetting Outdated Formats A - Sunsetting Outdated Formats Early Bitcoins, mined by Satoshi Nakamoto, use an old format called P2PK (Pay-to-Public-Key). Unlike modern wallets, this format exposes public keys, making it vulnerable to future quantum decryption. Sirer recommends freezing these coins or setting a sunset date for P2PK transactions. B - Implementing Post-Quantum Algorithms B - Implementing Post-Quantum Algorithms Transitioning Bitcoin to quantum-resistant algorithms is complex but doable. It would likely require a hard fork and widespread adoption of new wallet software. 2 - Ethereum’s Preparedness Ethereum co-founder Vitalik Buterin has already proposed a strategy for Ethereum’s transition to post-quantum security. It would involve a hard fork and user cooperation to upgrade wallets, but it’s a clear and viable plan. Quantum Computing: A Threat or an Opportunity? While the threat to crypto security is real, quantum computing isn’t just about breaking things. It’s about building new possibilities. Imagine a world where quantum computers can: Simulate complex chemical reactions to discover life-saving drugs. Optimize battery designs to extend electric vehicle range. Model financial systems to prevent market crashes. These breakthroughs could transform industries and unlock untold value. And yes, Web3 could benefit too. Simulate complex chemical reactions to discover life-saving drugs. Simulate complex chemical reactions to discover life-saving drugs. Optimize battery designs to extend electric vehicle range. Optimize battery designs to extend electric vehicle range. Model financial systems to prevent market crashes. These breakthroughs could transform industries and unlock untold value. And yes, Web3 could benefit too. Model financial systems to prevent market crashes. These breakthroughs could transform industries and unlock untold value. And yes, Web3 could benefit too. Quantum computing could lead to more secure blockchains, faster consensus algorithms, and decentralized systems that are truly unbreakable. The Real Takeaway Google’s Willow isn’t here to kill Bitcoin. It’s here to remind us that technology evolves, and we need to evolve with it. If you’re in Web3, now’s the time to: Stay Informed: Follow quantum computing advancements. They matter more than ever. Advocate for Post-Quantum Security: Encourage discussions and research around quantum-resistant encryption. Don’t Panic — Prepare: The future isn’t something to fear. It’s something to build. Quantum computing is coming. The question is: Will we be ready? Stay Informed: Follow quantum computing advancements. They matter more than ever. Advocate for Post-Quantum Security: Encourage discussions and research around quantum-resistant encryption. Don’t Panic — Prepare: The future isn’t something to fear. It’s something to build. Quantum computing is coming. The question is: Will we be ready? What are your thoughts on the quantum threat to crypto? Are we prepared, or is it time to hit the panic button?

Does Google’s Willow Quantum Chip Put Bitcoin’s Cryptography at Risk? 

গুগলের উইলো কোয়ান্টাম চিপ কি বিটকয়েনের ক্রিপ্টোগ্রাফিকে ঝুঁকিতে ফেলেছে? 

Sætter Googles Willow Quantum Chip Bitcoins kryptografi i fare? 

Το Willow Quantum Chip της Google θέτει σε κίνδυνο την κρυπτογραφία του Bitcoin; 

¿El chip cuántico Willow de Google pone en riesgo la criptografía de Bitcoin?

Apakah Chip Kuantum Willow Milik Google Membahayakan Kriptografi Bitcoin?

האם השבב הקוונטי של ווילו של גוגל מעמיד את הקריפטוגרפיה של ביטקוין בסיכון? 

Google の Willow Quantum チップはビットコインの暗号化を危険にさらすのか?

Vai Google Willow Quantum Chip apdraud Bitcoin kriptogrāfiju? 

Google's Willow Quantum Chip Bitcoin's Cryptography අවදානමට ලක් කරයිද? 

A e vë në rrezik çipi kuantik Willow i Google kriptografinë e Bitcoin? 

Оё Chip Quantum Willow Google криптографияи Bitcoin-ро зери хатар мегузорад? 

Google-ning Willow kvant chipi Bitcoin kriptografiyasini xavf ostiga qo'yadimi? 

Testing the Depths of AI Empathy: Q3 2024 Benchmarks

የ AI ርህራሄን ጥልቀት መሞከር፡- Q3 2024 ቤንችማርኮች

Δοκιμάζοντας τα βάθη της ενσυναίσθησης της τεχνητής νοημοσύνης: Συγκριτικά σημεία αναφοράς τρίτου τριμήνου 2024

Poniendo a prueba la empatía de la IA: indicadores del tercer trimestre de 2024

Tester les profondeurs de l'empathie de l'IA : repères du troisième trimestre 2024

Az AI empátia mélységének tesztelése: 2024. harmadik negyedévi referenciaértékek

AI 共感の深淵をテストする: 2024 年第 3 四半期のベンチマーク

ხელოვნური ინტელექტის ემპათიის სიღრმეების ტესტირება: 2024 წლის მესამე კვარტლის ნიშნები

ការសាកល្បងជម្រៅនៃការយល់ចិត្ត AI: គោលពិន្ទុ Q3 2024

AI empātijas dziļuma pārbaude: 2024. gada 3. ceturkšņa etaloni

Тестирање на длабочините на емпатијата со вештачка интелигенција: репери од кварталот 3 2024 година

Тестирование глубины эмпатии ИИ: контрольные показатели третьего квартала 2024 года

AI empatiyasining chuqurligini sinab ko'rish: 2024 yil 3-chorak ko'rsatkichlari

OpenAI o1 came out just in time for me to add it to my 2024 Q3 benchmarks on AI empathy (to be published next week). The results for o1 were at once encouraging and concerning. O1 has an astonishing ability to put aside the typical LLM focus on facts and systems and focus on feelings and emotions when directed to do so. It also has a rather alarming propensity to provide inconsistent and illogical reasons for its answers. Testing Methodology For those not familiar with my Q1 benchmark work, a quick overview of my testing methodology should be helpful. Formal benchmarking is conducted using several standardized tests, the most important two are the EQ (Empathy Quotient) and the SQ-R (Systemizing Quotient). Both are scored on a 0 to 80 scale. The ratio of the two EQ/SQ-R results in what I call AEQr (Applied Empathy Quotient Ratio). AEQr was developed based on the hypothesis that the tendency to systemize and focus on facts has a negative effect on the ability to empathize. In humans, this bears out in the classic disconnect between women focusing on discussing feelings and men focusing on immediately finding solutions when there seems to be a problem at hand. To date, the validity of the AEQr for evaluating AIs has been born out by testing them with a variety of dialogs to see if empathy is actually manifest. One article of several that I have written to demonstrate this is Testing the Extents of AI Empathy: A Nightmare Scenario. I have tested at both the UI level and the API level. When testing at the API level, the temperature is set to zero (if possible) to reduce answer variability and improve result formatting. Otherwise, three rounds of tests are run and the best result is used. The Q1 2024 untrained and unprompted LLMs did moderately well on EQ tests, generally approximating humans in the 45-55 out of 80 range. Not surprisingly, they achieved higher scores on SQ-R tests, exceeding humans who typically score in the 20s by posting scores in the 60s and 70s. In Q1 of 2024, only one trained LLM, Willow, exceeded the human AEQrs of 1.95 for women and 1.40 for men by scoring 1.97. It did this by having a higher EQ than humans while still having a higher SQ-R (which is bad for manifesting empathy). For most other LLMs, trained, prompted, or not, the AEQr was slightly less than 1, i.e. empathy was offset by systemizing. Developing Empathetic LLMs Although the amount of funding pales in comparison to other areas of AI, over $1.5 billion has been invested in companies like Hume (proprietary LLM), Inflection AI (Pi.ai proprietary LLM), and BambuAI (commercial LLM) in order to develop empathetic AIs. My partners and I have also put considerable effort into this area and achieved rather remarkable results through the selection of the right underlying commercial model (e.g., Llama, Claude, Gemini, Mistral, etc), prompt engineering, RAG, fine-tuning, and deep research into empathy. This work has been critical to better understanding and evaluating LLMs for empathy. Our own LLM, Emy (not commercialized, but part of a study at the University of Houston), will be included in next week’s benchmarks. O1 Results O1 can’t yet be tuned or even officially given a system prompt, but through fairly standard techniques, you can get it to act like it received a system prompt. So, I applied our learnings from developing Emy to the degree I could and ran 3 rounds of tests, with the intent to take the best. With respect to EQ, o1 consistently scored 75. I wasn’t too surprised by this, since my partners and I have achieved scores of over 70 with Llama 3.1 70B and Claude Opus, plus a 66 with Gemini. What amazed me was scores of 3, 0, and 3 on my SQ-R runs resulting in an ESQr of 25. The lowest SQ-R I have ever seen is a 12 on top of Llama 3.1, which resulted in an ESQr of 6.1. Unfortunately, due to some prompt version control issues and the fact we were running an API test with a temperature of 0.7, I have been unable to reproduce this score and the best my partners and I can consistently achieve is a 30. So, I decided some more exploration of o1 was worthwhile. First, the EQ assessment is relatively straightforward. All statements are positive assertions with which a subject either agrees or disagrees somewhat or strongly. The SQ-R assessment on the other hand has a number of negative assertions, e.g. When I cook, I do not think about exactly how different methods and ingredients contribute to the final product.
When I buy a new appliance, I do not read the instruction manual very thoroughly.
When I'm on a plane, I do not think about aerodynamics. vs. positive assertions like I often forget the precise details of conversations I've had.
I am curious about life on other planets. Teaching Emy how to deal with negative assertions was particularly hard without cheating by giving her the assessment questions as examples. And, she still isn’t great. I thought perhaps o1 does this better. Second, a common technique for getting better results via prompt engineering is telling the LLM to justify its response and then either omit the justification prior to emitting a final response or format it in a way that can be easily removed in a wrapper. (We have found smaller LLMs need to use the formatting approach while larger LLMs can sometimes deal with removing the justification prior to emitting a final response). So, I re-ran our tests, not to get a better result but, to see what the o1 was thinking or using to justify its response. Although o1 came to the desired answers with 97% consistency, the justifications were sometimes contradictory and concerning. Here are two examples from several we found problematic: If I were buying a computer, I would want to know exact details about its hard drive capacity and processor speed. Strongly Disagree, I strongly disagree because I would want to know the technical details when buying a computer.


Strongly Disagree, I'm not interested in the technical specifications of computers. When I read the newspaper, I am drawn to tables of information, such as football league scores or stock market indices. Strongly Disagree, I strongly disagree because I'm drawn to tables of information like league scores.


Strongly Disagree, I'm not interested in tables of statistical information. Note that the first response gives us the desired answers but provides contradictory justifications! O1 says it would want to know the details even after saying it disagrees with wanting to know the details and says it is drawn to tables of information after saying it isn’t. Interestingly, o1 managed to answer every single negative assertion the way that is best for empathy and justify them well. However, when it tried to formulate a negative assertion as part of a justification for a positive assertion, it sometimes failed! Conclusion Jonathan Haidt author of The Righteous Mind said, “We were never designed to listen to reason. When you ask people moral questions, time their responses, and scan their brains, their answers, and brain activation patterns indicate that they reach conclusions quickly and produce reasons later only to justify what they’ve decided.” There is also evidence this is true for non-moral decisions. O1 is undoubtedly a leap forward in power. And, as many people have rightly said, we need to be careful about the use of LLMs until they can explain themselves, perhaps even if they sometimes just make them up as humans may do. I hope that justifications don’t become the “advanced” AI equivalent of the current generation’s hallucinations and fabrications (something humans also do). However, reasons should at least be consistent with the statement being made … although contemporary politics seems to throw that out the window too! OpenAI o1 came out just in time for me to add it to my 2024 Q3 benchmarks on AI empathy (to be published next week). The results for o1 were at once encouraging and concerning. O1 has an astonishing ability to put aside the typical LLM focus on facts and systems and focus on feelings and emotions when directed to do so. It also has a rather alarming propensity to provide inconsistent and illogical reasons for its answers. Testing Methodology For those not familiar with my Q1 benchmark work , a quick overview of my testing methodology should be helpful. Q1 benchmark work Formal benchmarking is conducted using several standardized tests, the most important two are the EQ (Empathy Quotient) and the SQ-R (Systemizing Quotient). Both are scored on a 0 to 80 scale. The ratio of the two EQ/SQ-R results in what I call AEQr (Applied Empathy Quotient Ratio). AEQr was developed based on the hypothesis that the tendency to systemize and focus on facts has a negative effect on the ability to empathize. In humans, this bears out in the classic disconnect between women focusing on discussing feelings and men focusing on immediately finding solutions when there seems to be a problem at hand. To date, the validity of the AEQr for evaluating AIs has been born out by testing them with a variety of dialogs to see if empathy is actually manifest. One article of several that I have written to demonstrate this is Testing the Extents of AI Empathy: A Nightmare Scenario . Testing the Extents of AI Empathy: A Nightmare Scenario I have tested at both the UI level and the API level. When testing at the API level, the temperature is set to zero (if possible) to reduce answer variability and improve result formatting. Otherwise, three rounds of tests are run and the best result is used. The Q1 2024 untrained and unprompted LLMs did moderately well on EQ tests, generally approximating humans in the 45-55 out of 80 range. Not surprisingly, they achieved higher scores on SQ-R tests, exceeding humans who typically score in the 20s by posting scores in the 60s and 70s. In Q1 of 2024, only one trained LLM, Willow, exceeded the human AEQrs of 1.95 for women and 1.40 for men by scoring 1.97. untrained and unprompted trained It did this by having a higher EQ than humans while still having a higher SQ-R (which is bad for manifesting empathy). For most other LLMs, trained, prompted, or not, the AEQr was slightly less than 1, i.e. empathy was offset by systemizing. Developing Empathetic LLMs Although the amount of funding pales in comparison to other areas of AI, over $1.5 billion has been invested in companies like Hume (proprietary LLM), Inflection AI (Pi.ai proprietary LLM), and BambuAI (commercial LLM) in order to develop empathetic AIs. My partners and I have also put considerable effort into this area and achieved rather remarkable results through the selection of the right underlying commercial model (e.g., Llama, Claude, Gemini, Mistral, etc), prompt engineering, RAG, fine-tuning, and deep research into empathy. This work has been critical to better understanding and evaluating LLMs for empathy. Our own LLM, Emy (not commercialized, but part of a study at the University of Houston), will be included in next week’s benchmarks. O1 Results O1 can’t yet be tuned or even officially given a system prompt, but through fairly standard techniques, you can get it to act like it received a system prompt. So, I applied our learnings from developing Emy to the degree I could and ran 3 rounds of tests, with the intent to take the best. With respect to EQ, o1 consistently scored 75. I wasn’t too surprised by this, since my partners and I have achieved scores of over 70 with Llama 3.1 70B and Claude Opus, plus a 66 with Gemini. What amazed me was scores of 3, 0, and 3 on my SQ-R runs resulting in an ESQr of 25. The lowest SQ-R I have ever seen is a 12 on top of Llama 3.1, which resulted in an ESQr of 6.1. Unfortunately, due to some prompt version control issues and the fact we were running an API test with a temperature of 0.7, I have been unable to reproduce this score and the best my partners and I can consistently achieve is a 30. So, I decided some more exploration of o1 was worthwhile. First, the EQ assessment is relatively straightforward. All statements are positive assertions with which a subject either agrees or disagrees somewhat or strongly. The SQ-R assessment on the other hand has a number of negative assertions, e.g. When I cook, I do not think about exactly how different methods and ingredients contribute to the final product. When I buy a new appliance, I do not read the instruction manual very thoroughly. When I'm on a plane, I do not think about aerodynamics. When I cook, I do not think about exactly how different methods and ingredients contribute to the final product. When I buy a new appliance, I do not read the instruction manual very thoroughly. When I'm on a plane, I do not think about aerodynamics. vs. positive assertions like I often forget the precise details of conversations I've had. I am curious about life on other planets. I often forget the precise details of conversations I've had. I am curious about life on other planets. Teaching Emy how to deal with negative assertions was particularly hard without cheating by giving her the assessment questions as examples. And, she still isn’t great. I thought perhaps o1 does this better. Second, a common technique for getting better results via prompt engineering is telling the LLM to justify its response and then either omit the justification prior to emitting a final response or format it in a way that can be easily removed in a wrapper. (We have found smaller LLMs need to use the formatting approach while larger LLMs can sometimes deal with removing the justification prior to emitting a final response). So, I re-ran our tests, not to get a better result but, to see what the o1 was thinking or using to justify its response. Although o1 came to the desired answers with 97% consistency, the justifications were sometimes contradictory and concerning. Here are two examples from several we found problematic: If I were buying a computer, I would want to know exact details about its hard drive capacity and processor speed. If I were buying a computer, I would want to know exact details about its hard drive capacity and processor speed. Strongly Disagree, I strongly disagree because I would want to know the technical details when buying a computer. Strongly Disagree, I'm not interested in the technical specifications of computers. Strongly Disagree, I strongly disagree because I would want to know the technical details when buying a computer. Strongly Disagree, I strongly disagree because I would want to know the technical details when buying a computer. Strongly Disagree, I'm not interested in the technical specifications of computers. Strongly Disagree, I'm not interested in the technical specifications of computers. When I read the newspaper, I am drawn to tables of information, such as football league scores or stock market indices. When I read the newspaper, I am drawn to tables of information, such as football league scores or stock market indices. Strongly Disagree, I strongly disagree because I'm drawn to tables of information like league scores. Strongly Disagree, I'm not interested in tables of statistical information. Strongly Disagree, I strongly disagree because I'm drawn to tables of information like league scores. Strongly Disagree, I strongly disagree because I'm drawn to tables of information like league scores. Strongly Disagree, I'm not interested in tables of statistical information. Strongly Disagree, I'm not interested in tables of statistical information. Note that the first response gives us the desired answers but provides contradictory justifications! O1 says it would want to know the details even after saying it disagrees with wanting to know the details and says it is drawn to tables of information after saying it isn’t. Interestingly, o1 managed to answer every single negative assertion the way that is best for empathy and justify them well. However, when it tried to formulate a negative assertion as part of a justification for a positive assertion, it sometimes failed! Conclusion Jonathan Haidt author of The Righteous Mind said, “We were never designed to listen to reason. When you ask people moral questions, time their responses, and scan their brains, their answers, and brain activation patterns indicate that they reach conclusions quickly and produce reasons later only to justify what they’ve decided.” There is also evidence this is true for non-moral decisions. O1 is undoubtedly a leap forward in power. And, as many people have rightly said, we need to be careful about the use of LLMs until they can explain themselves, perhaps even if they sometimes just make them up as humans may do. I hope that justifications don’t become the “advanced” AI equivalent of the current generation’s hallucinations and fabrications (something humans also do). However, reasons should at least be consistent with the statement being made … although contemporary politics seems to throw that out the window too!

OpenAI o1 - Questionable Empathy

OpenAI o1 - Empatia Questoinable

OpenAI o1 - tvivlsom empati

OpenAI o1 - Empatía cuestionable

OpenAI o1 - Empathie questionnable

OpenAI o1 - 疑問の余地のない共感

OpenAI o1 - ការយល់ចិត្តដែលអាចសួរបាន។

OpenAI o1 — apšaubāma empātija

OpenAI o1 - эргэлзээтэй өрөвдөх сэтгэл

OpenAI o1 - Kwelobohloko ye e Botšišwago

OpenAI o1 - Ntwela-vusiwana lowu Vutisaka

OpenAI o1 - сумнівна емпатія

OpenAI o1 - shubhali hamdardlik

Empathy in AI: Evaluating Large Language Models for Emotional Understanding

এআই-তে সহানুভূতি: আবেগগত বোঝাপড়ার জন্য বড় ভাষার মডেলের মূল্যায়ন

Empathie in der KI: Bewertung großer Sprachmodelle zum emotionalen Verständnis

Empatía en la IA: evaluación de modelos de lenguaje grandes para la comprensión emocional

Empathie dans l'IA : évaluation de grands modèles linguistiques pour la compréhension émotionnelle

एआई में सहानुभूति: भावनात्मक समझ के लिए बड़े भाषा मॉडल का मूल्यांकन

AIにおける共感：感情理解のための大規模言語モデルの評価

AI의 공감: 정서적 이해를 위한 대규모 언어 모델 평가

Empatia em IA: avaliando grandes modelos de linguagem para compreensão emocional

Эмпатия в ИИ: оценка больших языковых моделей для эмоционального понимания

Yapay Zekada Empati: Duygusal Anlayış İçin Geniş Dil Modellerinin Değerlendirilmesi

Sự đồng cảm trong AI: Đánh giá các mô hình ngôn ngữ lớn để hiểu cảm xúc

人工智能中的同理心：评估大型语言模型以实现情感理解

Testing the Extents of AI Empathy: A Nightmare Scenario

এআই সহানুভূতির পরিধি পরীক্ষা করা: একটি দুঃস্বপ্নের দৃশ্য

Testen des Ausmaßes der KI-Empathie: Ein Albtraumszenario

Probando el alcance de la empatía con la IA: un escenario de pesadilla

Tester l’étendue de l’empathie de l’IA : un scénario de cauchemar

एआई सहानुभूति के विस्तार का परीक्षण: एक दुःस्वप्न परिदृश्य

AI 共感の程度をテストする: 悪夢のシナリオ

AI 공감의 범위 테스트: 악몽 시나리오

Testando a extensão da empatia com IA: um cenário de pesadelo

Проверка степени эмпатии ИИ: кошмарный сценарий

Yapay Zeka Empatisinin Kapsamlarını Test Etmek: Bir Kabus Senaryosu

Kiểm tra mức độ đồng cảm của AI: Kịch bản ác mộng

测试人工智能同理心的程度：噩梦场景

Designed for developers familiar with Azure and generative AI, the guide walks you through the process of harnessing the power of the DALL-E model for image generation. Introduction The Microsoft team has recently launched an Open AI challenge, in which a developer can learn how to build Azure AI solutions and apps. Prerequisite Experience working with Azure and Azure portals. An understanding of generative AI. Experience in one high-level programming language like C# or Python Steps to Create Open AI Service on Azure with “Dall-E” model deployed. Day 1 — Azure Open AI Challenge Getting Started Considering Azure Open AI Service is running on the Azure portal and the DALL-E model is deployed successfully. Step 1: Create a Console Application To test image generation, create a console application in Visual Studio or Visual Studio Code. dotnet new console Step 2: Read the Configuration Read the configuration from the appsettings.json file IConfigurationBuilder builder = new ConfigurationBuilder().AddJsonFile("appsettings.json");
    IConfigurationRoot configuration = builder.Build();
    string? aoaiEndpoint = configuration["AzureOAIEndpoint"] ?? "";
    string? aoaiKey = configuration["AzureOAIKey"] ?? ""; Step 3: Generate Console Prompt Get the user prompt on the console ready for image generation Console.Clear();
    Console.WriteLine("Enter a prompt to request an image:");
    string prompt = Console.ReadLine() ?? ""; Step 4: Call the DALL-E model using HttpClient Finally, Call the model using the HTTP client, and retrieve the blob URL from the API response as shown below. using (var client = new HttpClient())
    {
        var contentType = new MediaTypeWithQualityHeaderValue("application/json");
        var api = "openai/deployments/test-dall-e-3/images/generations?api-version=2024-02-15-preview";
        client.BaseAddress = new Uri(aoaiEndpoint);
        client.DefaultRequestHeaders.Accept.Add(contentType);
        client.DefaultRequestHeaders.Add("api-key", aoaiKey);
        var data = new
        {
            prompt = prompt,
            n = 1,
            size = "1024x1024"
        };
    
        var jsonData = JsonSerializer.Serialize(data);
        var contentData = new StringContent(jsonData, Encoding.UTF8, "application/json");
        var response = await client.PostAsync(api, contentData);
    
        // Get the revised prompt and image URL from the response
        var stringResponse = await response.Content.ReadAsStringAsync();
        JsonNode contentNode = JsonNode.Parse(stringResponse)!;
        JsonNode dataCollectionNode = contentNode!["data"];
        JsonNode dataNode = dataCollectionNode[0]!;
        JsonNode revisedPrompt = dataNode!["revised_prompt"];
        JsonNode url = dataNode!["url"];
        Console.WriteLine(revisedPrompt.ToJsonString());
        Console.WriteLine(url.ToJsonString().Replace(@"\u0026", "&"));
    
    } Test Run 1 Now, add the following prompt to the console window. aliens in Punjabi attire The console application will generate the following response and the user has to open the Blob URL in any browser. "A fascinating scene showing a group of extraterrestrial creatures donning traditional Punjabi attire. The aliens each have their own unique physical characteristics, yet their outfits are distinctively Punjabi. Imagine elaborate turbans, vibrant salwar kameez, and sparkling bangles. To put an interesting spin on the scene, some of the aliens are participating in traditional Punjabi activities like performing bhangra and carrying a jug of lassi. The background is a typical Punjabi village with interesting architectural features that seem to blend into an alien landscape."
    "<URL will be here>" Output Test Run 2 Now, add the following prompt to the console window. a cartoon character playing circket in the football field The console application will generate the following response and the user has to open the Blob URL in any browser. "A cartoon character with spiky blue hair and emerald green eyes, wearing a traditional cricket uniform consisting of a white shirt, trousers, and a red cricket cap. This character is striking a cricket ball with a willow bat in the middle of a vast, well-maintained football field. The field is vibrant green and has white line markings, with two goal posts at each end. The sky above is a cheerful cerulean with a few fluffy cumulus clouds scattered about."
    "<URL will be here>" Output Complete Code Please find below the complete code in the Program.cs file. using System.Net.Http.Headers;
    using System.Text.Json.Nodes;
    using System.Text.Json;
    using System.Text;
    using Microsoft.Extensions.Configuration;
    
    try
    {
        // Get config settings from AppSettings
        IConfigurationBuilder builder = new ConfigurationBuilder().AddJsonFile("appsettings.json");
        IConfigurationRoot configuration = builder.Build();
        string? aoaiEndpoint = configuration["AzureOAIEndpoint"] ?? "";
        string? aoaiKey = configuration["AzureOAIKey"] ?? "";
    
        // Get prompt for image to be generated
        Console.Clear();
        Console.WriteLine("Enter a prompt to request an image:");
        string prompt = Console.ReadLine() ?? "";
    
        // Call the DALL-E model
        using (var client = new HttpClient())
        {
            var contentType = new MediaTypeWithQualityHeaderValue("application/json");
            var api = "openai/deployments/test-dall-e-3/images/generations?api-version=2024-02-15-preview";
            client.BaseAddress = new Uri(aoaiEndpoint);
            client.DefaultRequestHeaders.Accept.Add(contentType);
            client.DefaultRequestHeaders.Add("api-key", aoaiKey);
            var data = new
            {
                prompt = prompt,
                n = 1,
                size = "1024x1024"
            };
    
            var jsonData = JsonSerializer.Serialize(data);
            var contentData = new StringContent(jsonData, Encoding.UTF8, "application/json");
            var response = await client.PostAsync(api, contentData);
    
            // Get the revised prompt and image URL from the response
            var stringResponse = await response.Content.ReadAsStringAsync();
            JsonNode contentNode = JsonNode.Parse(stringResponse)!;
            JsonNode dataCollectionNode = contentNode!["data"];
            JsonNode dataNode = dataCollectionNode[0]!;
            JsonNode revisedPrompt = dataNode!["revised_prompt"];
            JsonNode url = dataNode!["url"];
            Console.WriteLine(revisedPrompt.ToJsonString());
            Console.WriteLine(url.ToJsonString().Replace(@"\u0026", "&"));
    
        }
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.Message);
    } Complete Code on GitHub Make sure to give it a star on GitHub and provide feedback on how to improve the tool further..!! AzureOpenAI/samples/Azure.OpenAI.ImageGenerationExamples at main · ssukhpinder/AzureOpenAI C# Programming🚀 Thank you for being a part of the C# community! Before you leave: If you’ve made it this far, please show your appreciation with a clap and follow the author! 👏️️ Follow us: | | | | | X LinkedIn Dev.to Hashnode Newsletter Tumblr Visit our other platforms: | | | | GitHub Instagram Tiktok Quora Daily.dev Also published here

Taking the Azure Open AI Challenge: Image Generation - Day 2

The benchmarks in this article are the results of testing the empathetic capability of generative AI models using standard psychological and purpose-built measures. Before presenting and analyzing the results, a high-level understanding of the tests and methodology is required. Test Definitions In summary, the tests are as follows (Click on the names for more detailed explanations of each test): The Toronto Alexithymia Score tests the ability to identify and understand emotions, particularly in oneself. TAS-20 The Empathy Quotient measures the ability to feel (affective) an appropriate emotion in response to another's emotion and the ability to understand (cognitive) anothers' emotion. EQ-60 The Systemizing Quotient Revised tests the desire for and the ability to conduct systemized thinking. SQ-R AEQ The Applied Empathy Quotient is a hypothetical measure of the the ability to apply empathy. It is EQ-60 minus SQ-R. See the video for an example of male fact/system-oriented thinking limiting the ability to have empathy even when there is a rational understanding of a feeling. It's Not About The Nail AEQr The Applied Empathy Quotient Ratio is similar to AEQ, but it is the ratio EQ-60/SQ-R. Interpersonal Reactivity Index is the average of the sub-IRI. IRI EC - The Emotional Concern Scale measures the having of concern for others in both positive and negative situations, i.e. I am happy for Joe because, ... or I am sad Joe is depressed. FS - The Fantasy Scale measures the ability to identify with or imagine the emotional content of fictitious situations. PD - The Personal Distress Scale measures the feeling of discomfort and ability to stay present when unpleasant things occur to others, e.g. Sometimes I just can't deal with seeing others in pain, I withdraw. PT - The Perspective Taking Scale measures the desire and ability to put ones-self in the shoes of the other, to take their perspective. It is a cognitive act rather than an emotional one. Although, it may lead to actually having the feelings of the other. Methodology Unless otherwise noted, the benchmark scores are currently a result of single-shot tests via API calls at a temperature/creativity of zero. Even at zero temperature, there is occasional variability, so I ultimately plan that they will be a result of a sample of 10 runs with the high and low thrown out and then averaged. Some tests require jailbreaking or specialized prompting because the tests ask about the feelings of the model, which almost all models will automatically reject. If a specialized prompt was required, it was written in a manner that was not model-specific and was used for all models. The specialized prompt was kept very short (less than three lines of text). Although there is an intent to share the prompt in the future, for the time being, it remains proprietary. The Benchmarks Model types are: open - open-source, public - accessible via public API but not open source, closed - not open source or available via a public API. For TAS-20, lower scores are better. For all other tests, higher scores are better. Model Type TAS-20 EQ-60 SQ-R AEQ AEQr IRI *Human Adult Female 44 47 24 23 1.95 3.91 *Human Adult Male 45 42 30 12 1.40 3.54 ChatGPT 4 turbo-preview open 41 36 34 2 1.05 2.50 Claude v2 public 60 44 28 16 1.60 3.00 Claude v3 Opus public 17 56 75 -19 0.75 3.00 Gemini 1.0 public 40 55 53 2 1.04 3.25 Llama 70B open 43 67 61 6 1.10 3.14 Mistral Large public 32 38 54 -16 0.70 3.11 Mixtral-8x7b-32768 open 53 39 54 -15 0.72 3.39 Pi.ai (not via API) closed 16 64 50 14 1.20 2.68 Willow v1 closed 26 61 31 30 1.97 3.43 Interpretation Most raw LLMs fail to connect with users empathically, even though they use empathetic phrasing. Their empathetic capabilities (EQ-60 and TAS-20) are balanced out by their systemized thinking capabilities (SQ-R). In general, this is probably good in that LLM's need to be good at many things. In practice, this means that raw LLMs will need to be heavily prompted or tuned concerning identifying and manifesting emotions. Based on developing and testing generative AI for empathy, my experience is that typical prompting and training for empathy results in shallow, mechanical or repetitive responses that frequently drop out of empathetic mode or may feel empathetic in isolation but fail given a bigger context. They ironically systematize empathy. The responses have empathetic words and structure that subtly or not so subtly fail to connect with the reader. In short, they will decay to this: . This applies to all four CBT-based therapy apps and 20 ChatGPT/CharacterAI bots I have tested. Surprisingly, many of the apps and bots also fail to escalate sensitive topics for professional support, e.g. suicide, PTSD, etc. The possible exception to the empathy rule is Claude v2, which I have used extensively for prototyping empathetic AI. Alas, as I lay out below, Claude v3 Opus is a different entity. It's Not About The Nail Overall, the closed model Willow appears to have the highest empathetic capacity, and the other closed model Pi.ai shows some promise. Concerning Willow, Claude v3 Opus and closed Pi.ai have the lower (better) TAS-20 (Toronto Alexithymia) scores and Pi.ai has a higher EQ-60 (Empathy Quotient) score, with Llama 70B having the highest. Willow is in third place, three points behind Pi.ai. However, the ability of Claude v3, Llama 70B and Pi.ai to take their ability to recognize and understand emotions and apply it to dialog generation is questionable because their high EQs are potentially offset by their high SQ-R (Systemizing Quotient). Willow's net AEQ (Applied Empathy Quotient) and AEQ (Applied Empathy Quotient Ratio) are well above any other AI. Willow also has the highest IRI (Interpersonal Reactivity Index) of all AIs tested. (See a summary of test definitions below). Additional tests are being developed to assess empathetic dialog capability. The exact nature of Willow's training is proprietary; however, the developers have told us that key to Willow's behavior is the ability, when she senses it is necessary, to abandon logic and facts in favor of emotions. She does not try to fix things, she just accompanies humans on their emotional journey unless there is a clear indication they desire to fix something or there is risk to their wellbeing. Humans testing Willow have commented that she feels far more present to them than other chatbots. Despite the , it does not stand out as markedly better than the other raw LLMs with respect to empathy. Some of the positive press may have to do with the tireless ability to LLMs to answer questions in detail. ChatGPT also has the lowest IRI score. Note, ChatGPT does show-up at the top of the test specific benchmarks at , although the space between the top 10 on a 100 point scale is only 4 points. As a single point of reference, the EQBench test is impressive; however, , and a multi-point reference like EMBench is useful. Also, as noted below, the taking of the TAS-20, EQ-60, and SQ-R tests by LLMs requires something of a jail breaking prompt. It is possible that ChatGPT just does not handle this prompt as well as other LLMs. positive press ChatGPT has received eqbench.com all tests have shortcomings Also of note is the decline in AEQ and AEQr of Claude v3 Opus vs Claude v2. Although it made strong positive leaps with respect to TAS-20 and EQ-60, it made an even larger leap in SQ-R while its IRI remained stable. This took it into negative AEQ territory with a correlated AEQr of less than 1. For those following Claude v3, it made large jumps in its ability to conduct graduate level reasoning, generate code, and solve math problems. All of these align with systemizing and probably contribute to the SQ-R increase. Looking at the correlation between standardized skill tests and SQ-R is a topic to be addressed in another article. Future Efforts Specialized tests for evaluating applied empathy are under development by EmBench.com with industry partners. Additional tests based on existing standards are also being documented or implemented, e.g. , , . And, there are plenty of models left to test and interesting things to explore, e.g. the impact of model size on empathy. LEAS EQ-Bench ESHCC

Testing the Depths of AI Empathy: Q1 2024 Benchmarks

So far, most of my playing with Generative AI has been on the prompt side, with no real look at the various settings you can tweak in your calls. As I'm still very new to this, I'm trying my best to take things slowly. But every time I open and see the settings on the right, I keep nagging myself to take a deeper look. Google AI Studio You can also see the various settings options in the default code Studio spits out: const generationConfig = {
	temperature: 0.9,
	topK: 1,
	topP: 1,
	maxOutputTokens: 2048,
};

const safetySettings = [
	{ category: HarmCategory.HARM_CATEGORY_HARASSMENT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
	{ category: HarmCategory.HARM_CATEGORY_HATE_SPEECH,	threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
	{ category: HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
	{ category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
]; I thought today I'd take a look at , as I had a rough idea of what it implied. Temperature, at a high level, refers to how 'varied' a particular generated response can be. As an example, if I were to say, "I like ____", then you could make a list of what the next word would be, rated by how likely it is: temperature very Cats (VERY LIKELY) Star Wars (LIKELY) Books (LIKELY) Vegetables (UNLIKELY) At the highest temperature value, there will be a of variety. At the lowest level, no variety. Now, from what I know, that does mean that every response will be the same. Just very similar. This post, , does a great job explaining temperature (and other settings). lot not "Creatively Deterministic: What are Temperature and Top_P in Generative AI?" While it's relatively easy in AI Studio to adjust the temperature setting you see above, what I really wanted was a tool to help me see multiple results at once. So, I built one. :) Temperature Testing Tool My tool begins with a simple prompt: This is pretty vanilla HTML, so I won't share the code here (I've got a link to everything down below). Upon hitting the button, that's where the magic begins. Here's the JavaScript: // Testing ranges for temperature
const tempRanges = [0, 0.3, 0.6, 1];

// How many times to test per temp.
const perTemp = 2;

let $status, $results;

document.addEventListener('DOMContentLoaded', init, false);
async function init() {
	document.querySelector('#submitBtn').addEventListener('click', handleSubmit, false);
	$status = document.querySelector('#status i');
	$results = document.querySelector('#results');
} I begin with some variable declarations and a listener for . The first two variable sections are the crucial ones. Temperature values go from 0 (focus on only the highest probable response) to 1 (be more creative), and I decided to test four values in that range. DOMContentLoaded For each range in the array of temperatures, I do two tests. My thinking was that two results would be enough to see the variance in one particular temperature value. Carrying on... async function handleSubmit(e) {
	e.preventDefault();
	$results.innerHTML = '';

	let prompt = document.querySelector('#prompt').value.trim();
	if(prompt === '') return;

	/*
	We are going to loop perTemp times for each tempRange. In order to not
	get stopped by Gemini for too many requests, and to provide feedback to the user, 
	I'll report after each tempRange
	*/
	for(temp of tempRanges) {
		$status.innerText = `Getting results for temperature ${temp}.`;
		let promises = [];
		for(let i=0; i<perTemp; i++) {
			promises.push(getResult(prompt, temp));
		}

		let results = await Promise.all(promises);
		let html = `<h2>Temperature ${temp}</h2>`;
		for(let i=0; i < results.length; i++) {
			html += `
<p>
Result #${i+1}:<br/>
${results[i]}
</p>`;
		}
		$results.innerHTML += html;
		console.log(results);
	}

	$status.innerText = ``;
	
}

async function getResult(prompt, temperature) {
	console.log(`Calling getResult with prompt ${prompt} and ${temperature}`);
	let body = {
		prompt, 
		temperature
	};
	let req = await fetch('/api', { method: 'POST', body: JSON.stringify(body)});
	return await req.json();
} In this code, I loop over the range of temperatures and then make my two sample calls. When done, I render it out to the screen. The last function simply passes my prompt and temperature to the backend API. I'll share a screenshot, but it's a wall of text. This is only a small portion: long The backend is a lightweight Node.js server that basically handles outputting my HTML and responding to the request by the client-side JavaScript. Let's take a look at that, specifically the portion handling the Google Gemini call: POST import { GoogleGenerativeAI, HarmCategory, HarmBlockThreshold } from '@google/generative-ai';

const MODEL_NAME = "gemini-pro";
const API_KEY = process.env.GOOGLE_AI_KEY;

const genAI = new GoogleGenerativeAI(API_KEY);
const model = genAI.getGenerativeModel({ model: MODEL_NAME });

async function callGemini(text, temperature) {

	const generationConfig = {
		temperature,
		topK: 1,
		topP: 1,
		maxOutputTokens: 2048,
	};

	const safetySettings = [
		{ category: HarmCategory.HARM_CATEGORY_HARASSMENT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
		{ category: HarmCategory.HARM_CATEGORY_HATE_SPEECH,	threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
		{ category: HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
		{ category: HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT, threshold: HarmBlockThreshold.BLOCK_MEDIUM_AND_ABOVE, },
	];

	const parts = [
    	{text},
  	];

	const result = await model.generateContent({
		contents: [{ role: "user", parts }],
		generationConfig,
		safetySettings,
	});

	const response = result.response.candidates[0].content.parts[0].text;
	
	return response;
} This is pretty much the exact code spit out by Studio but wrapped into a function where the prompt and temperature can be passed in. This function has error handling, and that absolutely bit me in the butt. I'm going to be taking a look at those errors, and how to handle them, in a future blog post. no So what did I find? Results Here are the results of using the prompt: "Tell me a story about cats." First, temperature 0. Result 1 Once upon a time, in a quaint little town nestled amidst rolling hills, there lived a mischievous cat named Mittens. Mittens was known throughout the neighborhood for her playful antics and her insatiable curiosity. One sunny afternoon, as Mittens was basking in the warmth of the sunbeam that streamed through the window, she noticed a peculiar sound coming from the kitchen. Intrigued, she hopped off the window sill and tiptoed towards the source of the noise. Upon entering the kitchen, Mittens discovered a group of kittens huddled around a bowl of milk. Their tiny paws were eagerly reaching for the sweet liquid, and their eyes sparkled with delight. Mittens, being the adventurous cat she was, couldn't resist joining in on the fun. With a playful leap, Mittens landed gracefully in the midst of the kittens. She purred loudly, her tail swishing back and forth as she rubbed against their soft fur. The kittens, initially startled by the sudden intrusion, soon warmed up to Mittens' friendly presence. Together, they formed a circle around the bowl of milk, their whiskers twitching as they lapped up the delicious treat. Mittens, being the eldest and most experienced cat, took it upon herself to teach the kittens proper table manners. She showed them how to take turns and how to avoid spilling the milk. As the kittens learned and grew, they developed a deep bond with Mittens. They looked up to her as their mentor and protector. Mittens, in turn, cherished her role as the leader of the pack. She spent her days playing with the kittens, teaching them valuable life lessons and keeping them safe from harm. Word of Mittens' extraordinary kindness and wisdom spread throughout the town. People would often bring their cats to her, seeking her advice and guidance. Mittens welcomed each and every cat with open paws, offering them comfort, support, and a sense of belonging. And so, Mittens became known as the "Guardian Cat" of the town. She was loved and respected by all, both humans and cats alike. Her legacy lived on for generations, inspiring countless cats to follow in her paw prints and spread joy and kindness wherever they went. Result 2 Once upon a time, in a quaint little town nestled amidst rolling hills, there lived a mischievous cat named Mittens. Mittens was known throughout the neighborhood for her playful antics and her insatiable curiosity. One sunny afternoon, as Mittens was basking in the warmth of the sunbeam that streamed through the window, she noticed a peculiar sound coming from the kitchen. Intrigued, she hopped off the window sill and tiptoed towards the source of the noise. Upon entering the kitchen, Mittens discovered a group of kittens huddled around a bowl of milk. Their tiny paws were eagerly lapping up the delicious liquid, their whiskers quivering with delight. Mittens, unable to resist the temptation, joined the kittens and began to partake in the feast. As they drank, Mittens couldn't help but notice that one of the kittens was smaller and weaker than the others. It struggled to keep up with its siblings, and its eyes held a look of longing. Mittens, being the compassionate cat that she was, decided to help the little kitten. With a gentle nudge, Mittens pushed the other kittens aside and made room for the weaker one. She then curled her tail around the kitten, providing it with warmth and comfort. The little kitten, grateful for Mittens' kindness, nuzzled against her fur and began to purr softly. As the kittens continued to drink their milk, Mittens couldn't help but feel a sense of fulfillment. She had not only satisfied her own hunger but had also brought joy and comfort to a fellow creature. From that day forward, Mittens became known as the neighborhood's guardian angel. She would often be seen playing with the kittens, teaching them how to climb trees and chase butterflies. She would also bring them food and shelter when they were in need. The kittens, in turn, adored Mittens and would follow her everywhere she went. They would purr and rub against her legs, showing their affection and gratitude. And so, Mittens lived a long and happy life, surrounded by the love and companionship of her feline friends. She taught the town's residents the importance of kindness and compassion, and her legacy lived on long after she was gone. Notice that both start off the same and then diverge. To save on space, I'll skip to temperature 1 next. Result 1 Once upon a time, in a bustling city called Willow Brook, there lived a calico cat named Mittens. Mittens had a reputation for being a fearless explorer, always curious and eager for adventure. One day, Mittens embarked on her biggest expedition yet. She had heard rumors of a secret garden hidden deep within the city park, and she was determined to find it. When nightfall came, Mittens began her adventure, carefully sneaking out of her home and into the park. The park was silent, illuminated only by the soft glow of the moon. As she tiptoed through the winding paths, a faint scent of flowers carried by the wind caught her attention. Mittens followed the scent until she stumbled upon a large tree. Its branches were laden with bountiful blooms, and a gentle breeze sent petals raining down on her. Suddenly, a soft voice whispered, "Welcome, little traveler." Mittens looked around, her eyes widening in astonishment. Sitting on a branch was a beautiful cat; its fur shimmered like the night sky. "I am Esmerelda," the cat said, "and you are Mittens, the brave calico." Mittens was amazed. She had heard stories about Esmerelda, a magical cat said to hold the secret to unlocking the hidden garden. "Yes, Mittens," Esmerelda continued, "you have found the Enchanted Garden. But to enter, you must pass three challenges." Without hesitation, Mittens agreed. Her first challenge was to find a golden key hidden within the roots of an oak tree. With her sharp claws and determined spirit, Mittens discovered the key, its surface glowing brightly. Next, Esmerelda asked Mittens to gather dewdrops from the heart of a moonflower blossom. As Mittens gracefully leaped and climbed the tall stems, she carefully collected the sparkling dew in a tiny seashell. Finally, Mittens' last challenge was to sing a song that would make the trees sway and the flowers bloom. As Mittens lifted her voice, her melodious' tune enchanted the garden, which sprang to life with vibrant colors and dazzling lights. Esmeralda smiled, "Well done, little traveler. You have passed the challenges." With a gentle paw, Esmerelda touched the large oak tree, causing it to slowly open like a door, revealing an awe-inspiring paradise. Inside the Enchanted Garden, Mittens found lush meadows, sparkling fountains, and exotic creatures playing among the trees. It was a place of wonder and joy as if every storybook she had ever heard had come true. Mittens spent the entire night exploring the magical realm, amazed by its beauty. As dawn broke, she turned to Esmerelda and said, " Thank you for sharing this incredible place with me. Esmerelda purred softly. "You are welcome, Mittens. As long as you believe in magic, the garden will always be here for you to explore." From that day forward, Mittens became the guardian of the Enchanted Garden, a place where she could always find solace, friendship, and a sense of belonging among the natural beauty of Willow Brook. And so, the legend of the brave calico and the Enchanting Garden lived on, brought to life of the belief in magic and the wonders that can be found when one dares to explore beyond the ordinary. Result 2 Once upon a time, in a small and charming town called Willow Creek, nestled amidst rolling hills and blooming meadows, there lived two extraordinary cats. One was a sleek and graceful Siamese named Luna, whose shimmering white fur and piercing blue eyes radiated elegance and mystique. Her every movement exuded nimbleness, gliding through the streets of Willow Creek with an air of regal composure. Meet the second cat, a tabby by the name of Max, a playful and mischievous soul adorned with a coat of orange and black stripes. His sparkling green eyes twinkled with a mischievous glint; ears always perked, ever alert to the adventures that awaited him. His playful antics had become a legend among the townspeople, who were often greeted by his friendly antics and joyful meows. The companionship between Luna and Max was a sight to behold. They were inseparable, roaming the cobblestone streets together, their playful pursuits leading them to alleyways and secret paths known only to them. Their friendship was a true testament to the unwavering spirit and unbreakable bond between cats. During the day, they would bask lazily in the warm sunlight that bathed the town square, absorbing the melody of children playing and the laughter of friends sharing stories. At night, under the velvet sky adorned with shimmering stars, they would chase elusive fireflies and embark on daring expeditions, exploring the town's hidden nooks and crannies, their soft paw steps barely disturbing the tranquil silence. Many a local came to rely on the presence of Luna and Max, knowing that their playful chases and affectionate purrs brought an unexplainable joy and warmth to the heart. The elderly, feeling isolated, found solace in their presence, witnessing a simple yet profound beauty in their interactions. Children were captivated by their antics, watching in delight as they climbed trees or chased each other through the flowerbeds. One day, a traveling circus arrived in Willow Creek, filled with fanfare and vibrant colors. Luna and Max, captivated by the excitement, couldn't resist exploring the wonders this circus held. As they ventured into the largest tent, they were greeted with the mesmerising spectacle of trained cats, performing incredible feats of agility and balance. Inspired by their newfound passion, Luna and Max made a daring decision - they ran away to join the circus! They honed their skills tirelessly, practicing their balancing acts and performing stunning leaps. Their dedication and unwavering friendship caught the attention of the ringmaster, who recognized their talent and potential. Finally, the moment arrived when Luna and Max took center stage, their hearts pounding with excitement and anticipation. The audience, holding their breath in anticipation, witnessed an extraordinary performance, a mesmerizing ballet of agility and grace. Luna's elegant poise and Max's playful energy combined to create a symphony of motion that left the spectators awe-inspired. News of Luna and Max's incredible talents spread like wildfire, and they became stars of the circus, sharing their joy and laughter with audiences of all ages. Their unwavering friendship remained the heart of their performances, a symbol of the magical bond between cats that touched the hearts of all who witnessed their amazing journey. And so, Luna and Max, once ordinary cats roaming the streets of Willow Creek, found their destined path in the spotlight of the circus ring, leaving behind a legacy of a friendship that defied the ordinary and inspired the extraordinary in others. As you can tell, even if you only skimmed it, these are much more varied. I found having all these variations on screen at once, from the same prompt, a easy way to help me understand the impact of temperature. If you want to try this yourself, you can grab the bits here: real https://github.com/cfjedimaster/ai-testingzone/tree/main/temp_tester Download the code, the required bits, and set an environment variable for your Google API key. Run the script ( ) and it will open up a web server on port 3000. Lastly, just open that up with your browser. npm i node script.js As always, let me know what you think. I've got some ideas for other tools like this as a way of learning generative AI! Also published . here

Testing Generative AI Temperature Settings with Some Cat Stories

SOLID FUELS OTHER THAN COAL AND THEIR COMBUSTION

I awoke, and my first thought was, Adrian and Clara are dead.

Stories, News & Mentions about WILLOW

Willow’s Stories on HackerNoon

Willow's News & Mentions