Both US and allied officials have been similarly concerned that the app, owned by Beijing-based ByteDance, could be forced to hand over personal information of its users to the Chinese government. In fact, back in 2020, former US President Donald Trump ...

WeChat and Kaspersky Hit with Fresh Government Ban in Canada

Shares of Nvidia dropped by about 5 per cent on Tuesday following a report that the AI giant may be forced to cancel up to US$5 billion worth of chip orders to China in compliance with new US government restrictions.

US chip maker Nvidia’s shares drop after report of cancelled China orders

The state’s rationale behind the ban was its belief that the Chinese government might access user data through TikTok, given that its parent company, ByteDance, is based in Beijing. Under this law, TikTok downloads would be prohibited in Montana ...

Tik Tok Ban! Are the Government Parenting Again or Is This Long Overdue?

Alibaba Group Holding Limited (NYSE: BABA) stock is trading lower Wednesday in tandem with Chinese tech stocks, with Hong Kong's benchmark stock index reaching an 11-month low, as a sudden drop in manufacturing fueled concerns about the strength of China's economic recovery.

What's Going On With Alibaba Stock Wednesday?

DIY distributor DistroKid has expanded its licensing partnership with TikTok so that it now also covers the new TikTok Music service DistroKid first entered into a licensing deal with TikTok in 2019 covering the use of music on the social media firm's main ...

DistroKid expands its licensing partnership with TikTok

Enter CapCut’s Creative Suite, a versatile and robust video editing tool developed by ByteDance, the innovative force behind TikTok. In this article, we’ll dive deep into the world of CapCut’s Creative Suite, exploring its advanced video editing ...

Unlocking the Full Potential of CapCut’s Creative Suite

INDONESIA will exclude books, films, software and music from its ban on imported products below US$100 on e-commerce platforms, a senior minister said on Wednesday (Nov 1). Read more at The Business Times.

Indonesia to exclude books, films from ban on imported products in e-commerce: minister

Nvidia shares drop after report of cancelled China orders

A ByteDance app called Feishu that holds nearly all of TikTok’s internal communications was subject to a “wide-ranging inspection” before the CCP’s 20th National Congress last fall, documents show.

A Platform Storing TikTok Corporate Secrets Was Inspected By The Chinese Government

According to the report, China’s big tech firms like Alibaba, Bytedance, and Baidu had made large orders for 2024 and the amount exceeded $5 billion. Nvidia has been a beneficiary of the recent boom in AI-related products and the company’s value ...

Nvidia wobbles after report says U.S. export rules could force it to cancel over $5 billion worth of advanced chip orders to China

Get the most recent info and news about ByteDance on HackerNoon, where 10k+ technologists publish stories for 4M+ monthly readers.

Inspire creativity, enrich life.

ByteDance is a technology company operating a range of content platforms that inform, educate, entertain and inspire people across languages, cultures, and geographies.

Dedicated to building global platforms of creation and interaction, ByteDance now has a portfolio of applications available in over 150 markets and 75 languages. For example, TikTok, Helo, Vigo Video, Douyin, and Huoshan.

ByteDance's platforms aim to help users explore and discover the world's creativity, knowledge and moments that matter in everyday life while empowering everyone to be a creator directly from their smartphones. We are committed to building a safe, healthy and positive online environment for all our users.

We welcome people from all backgrounds to join us. If tackling challenges in a fast-changing industry does not scare you, work with us and challenge yourself to move the world!

ByteDance

GRAND CAYMAN, Cayman Islands, May 3rd, 2024/Chainwire/--Protocol launches, growth trajectory, and industry-leading technology point toward more success to come. On the first anniversary of Sui’s Mainnet launch, the Sui community is celebrating a landmark year that saw it rise from a nascent ecosystem to the top tier of Layer 1 blockchains, amassing household name partners and shipping multiple technology breakthroughs in the process. In the build-up to its launch in 2023, the chatter around Sui reached a level of excitement that has not been matched by any chain that has launched since. The first anniversary of Sui represents a culmination of the remarkable milestones achieved by the network in its first year. While centralized institutions face growing public distrust due to decades of anti-competitive and anti-consumer behavior and legacy blockchains lack the speed and technology to solve the problem, in a single year, Sui has emerged as the decentralized solution most capable of disrupting the status quo at scale. The innovations of Sui begin with Sui’s novel programming language, Move, which was created by Mysten Labs Co-founder and CTO Sam Blackshear. Move introduced a new architecture centered around objects, enabling performance and functionality that was simply not available on existing blockchains. The result is a blockchain that is singular in the industry — the universal coordination layer for intelligent assets. “Like many transformative innovations before us, we knew the problem we had to solve and built the technology to solve it without applying labels, but a year after launch, it is clear that Sui is a vibrant developer ecosystem,” said Evan Cheng, Co-founder and CEO of Mysten Labs. “Developers are taking advantage of Sui’s performance, and its scalable, composable, on-chain storage, and native accessibility features to build sustainable business models with a consumer ownership-first approach. While the Network’s achievements to date are remarkable, the road to onboarding the next billion users to web3 has just started.” Technology Innovations The results Sui has achieved have validated the team’s approach. In addition to not experiencing a single minute of downtime or instance of degraded performance since Mainnet launch, in the last 12 months, Sui has achieved: Lightning-fast transaction speeds — time to finality clocked at just 400 milliseconds
Massive scale — 297,000 peak TPS in a controlled environment
Record-setting Mainnet performance — 65.8M transactions executed in a single day, the most of any blockchain ever
Extremely low fees — predictable, stable transaction fees even during high usage Yet technology improvements are a constant at Sui. In recent weeks, including at its first global in-person conference, Sui Basecamp, Sui unveiled Mysticeti, which significantly shortens Sui’s end-to-end time to finality, and Pilotfish, which enables nearly unlimited horizontal scaling by enabling individual validators to use multiple machines to extend their capacity. Sui has also introduced unique implementations of technologies that make adoption easier and make the blockchain accessible to mainstream users, both at the enterprise and retail levels. zkLogin is on-chain authorization with traditional OAuth providers like Google and Twitter, allowing all users to directly operate on-chain with the single sign-on process they have become accustomed to, removing the hurdle of managing wallet addresses and seed phrases. zkSend is an application exclusive to Sui that utilizes zkLogin to enable users to send and claim tokens simply by sharing or clicking a link. Sponsored transactions, enabled by Sui’s extremely low fees, empower builders to remove a final hurdle for engagement. Finally, Enoki, which was announced at Sui Basecamp, removes requirements for enterprises looking to incorporate blockchain technology with a turnkey solution that gives them access to build seamlessly on Sui. “The rate at which the ecosystem’s deep and talented developer community has shipped powerful protocols and industry-first technology breakthroughs has been breathtaking, and it’s only a year in,” said Adeniyi Abiodun, Co-founder and CPO at Mysten Labs. “Sui’s purpose is to redefine how individuals and businesses collaborate to create, grow, and share value in a digital-first economy and since its mainnet launch, Sui has demonstrated important and unique capabilities in that regard.” Network Momentum – Sui is a destination for DeFi, Gaming and Commerce Also owing to the strength of the network and especially notable for an ecosystem so early in its development, Sui has quickly become one of the preeminent destinations for DeFi activity. Within 9 months of its Mainnet launch, Sui ranked in the Top 10 of all blockchains in TVL. A month later, Sui’s DEX volume also achieved top 10 status. Throughout, Wormhole stats show Sui as a top destination for bridging from Ethereum, including over a 30-day period where Sui saw more inflows than all other blockchains combined, and more than twice as much as the next closest blockchain. In each of these cases, Sui eclipsed numerous well-known networks that have existed for far longer. Builders and enterprises integrating with Sui are also adding to the growing momentum. Bluefin, a derivatives exchange that had already executed billions in transaction volume in its first iteration on another chain, shut that integration down to build on Sui. Solend, a leading lending protocol on Solana, chose Sui for its first integration beyond its initial network, launching the Suilend protocol which has already amassed over $30M TVL. Additionally, First Digital Labs, creator of FDUSD, the fastest-growing stablecoin in crypto with over $4B in market cap, chose Sui for its first expansion since its launch on Ethereum and BNB. “As we mark the first anniversary of Sui's launch, the growth and innovation within the Sui ecosystem have been sensational and Sui has quickly ascended to the forefront of Layer 1 blockchains,” said Greg Siourounis, Managing Director of the Sui Foundation. “Sui’s global footprint has continued to expand at a remarkable rate as more and more builders see Sui as the best platform for enabling real world solutions aimed at addressing the world’s most pressing challenges.” Gaming is another focus for Sui that will continue in the coming months and years. With low, predictable fees that allow game developers to build with confidence and dynamic objects that make games built on Sui more expressive, Sui is the optimal blockchain for gaming studios. Dozens of development teams are currently building games on Sui, including established studios such as NHN (Pebble City), ONBUFF x SNK (Samurai Shodown), NDUS Interactive (Xociety), Orange Comet (The Walking Dead: Lands), and Ambrus Studio (E4C: Final Salvation). Leading professional esports team, Team Liquid, will leverage Sui for the relaunch of its fan loyalty program, and at Sui Basecamp in April, Sui joined gaming infrastructure developer Playtron in announcing SuiPlay0X1, the first handheld gaming device with native blockchain capabilities to wide acclaim. Other notable ecosystem partnerships achieved in Sui’s first year include those with Alibaba Cloud and Google Cloud, which focused on enhancing security, scalability, developer tools and user experiences across a range of Web 3.0 and AI-powered applications. BytePlus, the web3 arm of ByteDance, partnered with Sui ecosystem partner Mysten Labs to explore collaboration on data warehousing, AI recommendation algorithms, and AI visual algorithms in web3 game platforms and socialFi projects on Sui. The crowds that gathered for Sui Basecamp 2024 represented the culmination of Sui’s arrival, as over 1,000 projects, partners, investors and enthusiasts from 65 countries around the world came to Paris in April to celebrate Sui at the vaunted Layer 1’s inaugural global conference. Contact Sui Foundation media@sui.io This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program. Learn more about the program here. GRAND CAYMAN, Cayman Islands, May 3rd, 2024/Chainwire/--Protocol launches, growth trajectory, and industry-leading technology point toward more success to come. On the first anniversary of Sui’s Mainnet launch, the Sui community is celebrating a landmark year that saw it rise from a nascent ecosystem to the top tier of Layer 1 blockchains, amassing household name partners and shipping multiple technology breakthroughs in the process. In the build-up to its launch in 2023, the chatter around Sui reached a level of excitement that has not been matched by any chain that has launched since. The first anniversary of Sui represents a culmination of the remarkable milestones achieved by the network in its first year. While centralized institutions face growing public distrust due to decades of anti-competitive and anti-consumer behavior and legacy blockchains lack the speed and technology to solve the problem, in a single year, Sui has emerged as the decentralized solution most capable of disrupting the status quo at scale. The innovations of Sui begin with Sui’s novel programming language, Move, which was created by Mysten Labs Co-founder and CTO Sam Blackshear. Move introduced a new architecture centered around objects, enabling performance and functionality that was simply not available on existing blockchains. The result is a blockchain that is singular in the industry — the universal coordination layer for intelligent assets. “Like many transformative innovations before us, we knew the problem we had to solve and built the technology to solve it without applying labels, but a year after launch, it is clear that Sui is a vibrant developer ecosystem,” said Evan Cheng, Co-founder and CEO of Mysten Labs. “Like many transformative innovations before us, we knew the problem we had to solve and built the technology to solve it without applying labels, but a year after launch, it is clear that Sui is a vibrant developer ecosystem,” said Evan Cheng, Co-founder and CEO of Mysten Labs. “Developers are taking advantage of Sui’s performance, and its scalable, composable, on-chain storage, and native accessibility features to build sustainable business models with a consumer ownership-first approach. While the Network’s achievements to date are remarkable, the road to onboarding the next billion users to web3 has just started.” “Developers are taking advantage of Sui’s performance, and its scalable, composable, on-chain storage, and native accessibility features to build sustainable business models with a consumer ownership-first approach. While the Network’s achievements to date are remarkable, the road to onboarding the next billion users to web3 has just started.” Technology Innovations The results Sui has achieved have validated the team’s approach. In addition to not experiencing a single minute of downtime or instance of degraded performance since Mainnet launch, in the last 12 months, Sui has achieved: Lightning-fast transaction speeds — time to finality clocked at just 400 milliseconds Massive scale — 297,000 peak TPS in a controlled environment Record-setting Mainnet performance — 65.8M transactions executed in a single day, the most of any blockchain ever Extremely low fees — predictable, stable transaction fees even during high usage Lightning-fast transaction speeds — time to finality clocked at just 400 milliseconds Massive scale — 297,000 peak TPS in a controlled environment Record-setting Mainnet performance — 65.8M transactions executed in a single day, the most of any blockchain ever Extremely low fees — predictable, stable transaction fees even during high usage Yet technology improvements are a constant at Sui. In recent weeks, including at its first global in-person conference, Sui Basecamp, Sui unveiled Mysticeti, which significantly shortens Sui’s end-to-end time to finality, and Pilotfish, which enables nearly unlimited horizontal scaling by enabling individual validators to use multiple machines to extend their capacity. Sui has also introduced unique implementations of technologies that make adoption easier and make the blockchain accessible to mainstream users, both at the enterprise and retail levels. zkLogin is on-chain authorization with traditional OAuth providers like Google and Twitter, allowing all users to directly operate on-chain with the single sign-on process they have become accustomed to, removing the hurdle of managing wallet addresses and seed phrases. zkSend is an application exclusive to Sui that utilizes zkLogin to enable users to send and claim tokens simply by sharing or clicking a link. Sponsored transactions, enabled by Sui’s extremely low fees, empower builders to remove a final hurdle for engagement. Finally, Enoki, which was announced at Sui Basecamp, removes requirements for enterprises looking to incorporate blockchain technology with a turnkey solution that gives them access to build seamlessly on Sui. “The rate at which the ecosystem’s deep and talented developer community has shipped powerful protocols and industry-first technology breakthroughs has been breathtaking, and it’s only a year in,” said Adeniyi Abiodun, Co-founder and CPO at Mysten Labs. “Sui’s purpose is to redefine how individuals and businesses collaborate to create, grow, and share value in a digital-first economy and since its mainnet launch, Sui has demonstrated important and unique capabilities in that regard.” Network Momentum – Sui is a destination for DeFi, Gaming and Commerce Also owing to the strength of the network and especially notable for an ecosystem so early in its development, Sui has quickly become one of the preeminent destinations for DeFi activity. Within 9 months of its Mainnet launch, Sui ranked in the Top 10 of all blockchains in TVL. A month later, Sui’s DEX volume also achieved top 10 status. Throughout, Wormhole stats show Sui as a top destination for bridging from Ethereum, including over a 30-day period where Sui saw more inflows than all other blockchains combined, and more than twice as much as the next closest blockchain. In each of these cases, Sui eclipsed numerous well-known networks that have existed for far longer. Builders and enterprises integrating with Sui are also adding to the growing momentum. Bluefin, a derivatives exchange that had already executed billions in transaction volume in its first iteration on another chain, shut that integration down to build on Sui. Solend, a leading lending protocol on Solana, chose Sui for its first integration beyond its initial network, launching the Suilend protocol which has already amassed over $30M TVL. Additionally, First Digital Labs, creator of FDUSD, the fastest-growing stablecoin in crypto with over $4B in market cap, chose Sui for its first expansion since its launch on Ethereum and BNB. “As we mark the first anniversary of Sui's launch, the growth and innovation within the Sui ecosystem have been sensational and Sui has quickly ascended to the forefront of Layer 1 blockchains,” said Greg Siourounis, Managing Director of the Sui Foundation. “Sui’s global footprint has continued to expand at a remarkable rate as more and more builders see Sui as the best platform for enabling real world solutions aimed at addressing the world’s most pressing challenges.” “As we mark the first anniversary of Sui's launch, the growth and innovation within the Sui ecosystem have been sensational and Sui has quickly ascended to the forefront of Layer 1 blockchains,” said Greg Siourounis, Managing Director of the Sui Foundation. “Sui’s global footprint has continued to expand at a remarkable rate as more and more builders see Sui as the best platform for enabling real world solutions aimed at addressing the world’s most pressing challenges.” Gaming is another focus for Sui that will continue in the coming months and years. With low, predictable fees that allow game developers to build with confidence and dynamic objects that make games built on Sui more expressive, Sui is the optimal blockchain for gaming studios. Dozens of development teams are currently building games on Sui, including established studios such as NHN (Pebble City), ONBUFF x SNK (Samurai Shodown), NDUS Interactive (Xociety), Orange Comet (The Walking Dead: Lands), and Ambrus Studio (E4C: Final Salvation). Leading professional esports team, Team Liquid, will leverage Sui for the relaunch of its fan loyalty program, and at Sui Basecamp in April, Sui joined gaming infrastructure developer Playtron in announcing SuiPlay0X1, the first handheld gaming device with native blockchain capabilities to wide acclaim. Other notable ecosystem partnerships achieved in Sui’s first year include those with Alibaba Cloud and Google Cloud, which focused on enhancing security, scalability, developer tools and user experiences across a range of Web 3.0 and AI-powered applications. BytePlus, the web3 arm of ByteDance, partnered with Sui ecosystem partner Mysten Labs to explore collaboration on data warehousing, AI recommendation algorithms, and AI visual algorithms in web3 game platforms and socialFi projects on Sui. The crowds that gathered for Sui Basecamp 2024 represented the culmination of Sui’s arrival, as over 1,000 projects, partners, investors and enthusiasts from 65 countries around the world came to Paris in April to celebrate Sui at the vaunted Layer 1’s inaugural global conference. Contact Sui Foundation media@sui.io This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program. Learn more about the program here. This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program. Learn more about the program here . here here

Sui Turns One: Debut Year Of Growth and Tech Breakthroughs Puts Sui At Forefront Of Web3

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Xiaofei Sun, Zhejiang University & xiaofei_sun@zju.edu.cn;
(2) Xiaoya Li, Shannon.AI, Bytedance & xiaoya_li@shannonai.com;
(3) Shengyu Zhang, Zhejiang University & sy_zhang@zju.edu.cn;
(4) Shuhe Wang, Peking University & wangshuhe@stu.pku.edu.cn;
(5) Fei Wu, Zhejiang University & wufei@zju.edu.cn;
(6) Jiwei Li, Zhejiang University &  jiwei_li@zju.edu.cn;
(7) Tianwei Zhang, Nanyang Technological University & tianwei.zhang@ntu.edu.sg;
(8) Guoyin Wang, Bytedance & guoyin.wang@bytedance.com. Table of Links Abstract and Introduction
Related Work
LLM Negotiation for Sentiment Analysis
Experiments
Ablation Studies
Conclusion and References 5 Ablation Studies In this section, we perform ablation studies on the Twitter dataset to better understand the mechanism behind the negotiation framework. 5.1 Who takes which role matters In the negotiation framework, there are two roles, the generator and the discriminator, which two separate LLMs take. Table 2 shows the performance for setups where GPT-3.5 and GPT-4 take different roles. As can be seen, when GPT-3.5 acts as the generator, and GPT-4 acts as the discriminator (G3.5-D4 for short), the performance (68.8) is better than single GPT-3.5 without negotiation (65.2), but worse than single GPT-4 without negotiation (69.5). In contrast, negotiation-based configurations with GPT-4 acting as the generator (G4-D3.5 and G4-D4) consistently outperforms standalone GPT-4 or GPT-3.5 models without negotiation. These results underscore the pivotal role that the generator plays in influencing the negotiation outcome. Furthermore, we observe G4- D3.5 can beat G4-D4. We attribute such advantage to the hypothesis that utilizing heterogeneous LLMs for distinct roles could optimize the negotiation’s performance. 5.2 Consensus Percentage Table 3 consensus percentage for different setups. As can be seen, when GPT-4 acts as the generator, the negotiation is more likely to reach a consensus, or reach a consensus in fewer turns. The explanation is intuitive: for the twitter task, we can see from table 1 that GPT-4 obtains better performances that GPT-3.5, which means the reasoning process for GPT-4 is more sensible than 3.5, making the decision of the former more likely to be agreed on. 5.3 Effect of the Reasoning Process In the negotiation process, LLMs are asked to articulate the reason process, a strategy akin to CoT(Wei et al., 2022b). We examine the importance for listing reasons in negotiation by removing the reasoning process and asking LLMs to only output decisions. Results are shown in Table 4. As can be seen, for the three setups, single GPT-3.5, where only GPT-3.5 is used without negotiation, single GPT-4, where only GPT-4 is used without negotiation, and GPT-3.5+GPT-4 where negotiation is employed, performances all degrade when the reasoning process is removed. But interestingly, we see a greater degrade (-2.3) for the negotiation than the single model setup (-1.2 for single-GPT-3.5 and -0.9 for single-GPT-4). This is in accord with our expectation as the reasoning process is of greater significance in the negotiation setup. This paper is available on arxiv under CC 4.0 license. Authors: (1) Xiaofei Sun, Zhejiang University & xiaofei_sun@zju.edu.cn; (2) Xiaoya Li, Shannon.AI, Bytedance & xiaoya_li@shannonai.com; (3) Shengyu Zhang, Zhejiang University & sy_zhang@zju.edu.cn; (4) Shuhe Wang, Peking University & wangshuhe@stu.pku.edu.cn; (5) Fei Wu, Zhejiang University & wufei@zju.edu.cn; (6) Jiwei Li, Zhejiang University &  jiwei_li@zju.edu.cn; (7) Tianwei Zhang, Nanyang Technological University & tianwei.zhang@ntu.edu.sg; (8) Guoyin Wang, Bytedance & guoyin.wang@bytedance.com. This paper is available on arxiv under CC 4.0 license. Authors: (1) Xiaofei Sun, Zhejiang University & xiaofei_sun@zju.edu.cn; (2) Xiaoya Li, Shannon.AI, Bytedance & xiaoya_li@shannonai.com; (3) Shengyu Zhang, Zhejiang University & sy_zhang@zju.edu.cn; (4) Shuhe Wang, Peking University & wangshuhe@stu.pku.edu.cn; (5) Fei Wu, Zhejiang University & wufei@zju.edu.cn; (6) Jiwei Li, Zhejiang University &  jiwei_li@zju.edu.cn; (7) Tianwei Zhang, Nanyang Technological University & tianwei.zhang@ntu.edu.sg; (8) Guoyin Wang, Bytedance & guoyin.wang@bytedance.com. Table of Links Abstract and Introduction Related Work LLM Negotiation for Sentiment Analysis Experiments Ablation Studies Conclusion and References Abstract and Introduction Abstract and Introduction Related Work Related Work LLM Negotiation for Sentiment Analysis LLM Negotiation for Sentiment Analysis Experiments Experiments Ablation Studies Ablation Studies Conclusion and References Conclusion and References 5 Ablation Studies In this section, we perform ablation studies on the Twitter dataset to better understand the mechanism behind the negotiation framework. 5.1 Who takes which role matters In the negotiation framework, there are two roles, the generator and the discriminator, which two separate LLMs take. Table 2 shows the performance for setups where GPT-3.5 and GPT-4 take different roles. As can be seen, when GPT-3.5 acts as the generator, and GPT-4 acts as the discriminator (G3.5-D4 for short), the performance (68.8) is better than single GPT-3.5 without negotiation (65.2), but worse than single GPT-4 without negotiation (69.5). In contrast, negotiation-based configurations with GPT-4 acting as the generator (G4-D3.5 and G4-D4) consistently outperforms standalone GPT-4 or GPT-3.5 models without negotiation. These results underscore the pivotal role that the generator plays in influencing the negotiation outcome. Furthermore, we observe G4- D3.5 can beat G4-D4. We attribute such advantage to the hypothesis that utilizing heterogeneous LLMs for distinct roles could optimize the negotiation’s performance. 5.2 Consensus Percentage Table 3 consensus percentage for different setups. As can be seen, when GPT-4 acts as the generator, the negotiation is more likely to reach a consensus, or reach a consensus in fewer turns. The explanation is intuitive: for the twitter task, we can see from table 1 that GPT-4 obtains better performances that GPT-3.5, which means the reasoning process for GPT-4 is more sensible than 3.5, making the decision of the former more likely to be agreed on. 5.3 Effect of the Reasoning Process In the negotiation process, LLMs are asked to articulate the reason process, a strategy akin to CoT(Wei et al., 2022b). We examine the importance for listing reasons in negotiation by removing the reasoning process and asking LLMs to only output decisions. Results are shown in Table 4. As can be seen, for the three setups, single GPT-3.5, where only GPT-3.5 is used without negotiation, single GPT-4, where only GPT-4 is used without negotiation, and GPT-3.5+GPT-4 where negotiation is employed, performances all degrade when the reasoning process is removed. But interestingly, we see a greater degrade (-2.3) for the negotiation than the single model setup (-1.2 for single-GPT-3.5 and -0.9 for single-GPT-4). This is in accord with our expectation as the reasoning process is of greater significance in the negotiation setup.

Sentiment Analysis through LLM Negotiations: Ablation Studies

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Xiaofei Sun, Zhejiang University & xiaofei_sun@zju.edu.cn;
(2) Xiaoya Li, Shannon.AI, Bytedance & xiaoya_li@shannonai.com;
(3) Shengyu Zhang, Zhejiang University & sy_zhang@zju.edu.cn;
(4) Shuhe Wang, Peking University & wangshuhe@stu.pku.edu.cn;
(5) Fei Wu, Zhejiang University & wufei@zju.edu.cn;
(6) Jiwei Li, Zhejiang University &  jiwei_li@zju.edu.cn;
(7) Tianwei Zhang, Nanyang Technological University & tianwei.zhang@ntu.edu.sg;
(8) Guoyin Wang, Bytedance & guoyin.wang@bytedance.com. Table of Links Abstract and Introduction
Related Work
LLM Negotiation for Sentiment Analysis
Experiments
Ablation Studies
Conclusion and References 3 LLM Negotiation for Sentiment Analysis 3.1 Overview In this section, we detail the multi-LLM negotiation framework for sentiment analysis: Two LLMs perform as the answer generator and discriminator. We refer to the interaction between the generator and the discriminator as a negotiation. The negotiation will repeat until a consensus is reached or the maximum number of negotiation turns is exceeded. Illustrations are shown in Figures 1 and 2. 3.2 Reasoning-infused generator The generator is backboned by a large language model. We ask the answer generator based on the ICL paradigm through prompts, aiming to generate a step-by-step reasoning chain and a decision towards the sentiment polarity of the test input. Prompts are composed of three elements: a task description, demonstrations, and a test input.  The task description is a description of the task in natural language (e.g., "Please determine the overall sentiment of test input."); the test input is the textual input in the test set (e.g., "The sky is blue."); demonstrations are from the train set of the task. Each consists of three elements: input, reasoning chains, and sentimental decision. For each test input, we first retrieve K nearest neighbors (input, sentiment decision) from the train set as demonstrations. Then, we transform demonstrations to (input, reasoning process, sentiment decision) triplets by prompting the generator to produce a reasoning chain. After concatenating the task description, demonstrations, and the test input, we forward the prompt to the generator, which will respond with a step-by-step reasoning chain and a sentimental decision. 3.3 Explanation-deriving discriminator The discriminator is backboned by another LLM. After finishing the answer generating process, the answer discriminator is used to judge whether the decision made by the generator is correct and provide a reasonable explanation. To accomplish this goal, we first construct prompts for the answer discriminator. The prompt is composed of four elements: a task description, demonstrations, a test input, and the response from the answer generator. The task description is a piece of text that describes the task in natural language (e.g., "Please determine whether the decision is correct."). Each demonstration is composed of six elements: (input text, a reasoning chain, sentiment decision, discriminator attitude, discriminator explanations, discriminator decision) and constructed by prompting the answer discriminator to provide explanations of why the sentiment decision is correct for the input text. Then we ask the discriminator with the construct prompt. The answer discriminator will respond with a text string, containing an attitude (i.e., yes, no) that denotes whether the discriminator agrees with the generator, explanations that explain why the discriminator agrees/disagrees with the generator, and a discriminator decision that determines the sentiment of the test input. Why Two LLMs but Not One? There are two reasons for using two different LLMs separately for the generator and the discriminator rather than using a single LLM to act as two roles: (1) If an LLM makes a mistake as a generator due to incorrect reasoning, it is more likely that it will also make the same mistake as the discriminator as since generator and the discriminator from the same model are very likely to make similar rationales; (2) by using two separate models, we are able to take the advantage of the complementary abilities of the two models. 3.4 Role-flipped Negotiation After two LLMs end with a negotiation, we ask them flip roles and initiate a new negotiation, where the second LLM acts as the generator, and the first LLM acts as the discriminator. We refer the interaction of two LLMs with flipped roles as role-flipped negotiation. Likewise, the role flipped negotiation is ended until a consensus is reached or the maximum number of negotiation turns is exceeded. When both negotiations result in an agreement and their decisions are the same, we can choose either decision as the final one since they are the same. If one of the negotiations fails to reach a consensus while the other reaches a decision, we choose the decision from the negotiation  that reached a consensus as the final decision. However, if both negotiations reach a consensus but their decisions do not align, we will require the assistance of an additional Language Model (LLM), as will be explained in more detail below." Introducing a third LLM If the decision from the two negotiations do not align, we introduce a third LLM and conduct the negotiation and role-flipped negotiation with each of the two aforementioned LLMs. Subsequently, we will get 6 negotiation results and vote on these results: the decision that appears most frequently is taken as the sentiment polarity of the input test. This paper is available on arxiv under CC 4.0 license. Authors: (1) Xiaofei Sun, Zhejiang University & xiaofei_sun@zju.edu.cn; (2) Xiaoya Li, Shannon.AI, Bytedance & xiaoya_li@shannonai.com; (3) Shengyu Zhang, Zhejiang University & sy_zhang@zju.edu.cn; (4) Shuhe Wang, Peking University & wangshuhe@stu.pku.edu.cn; (5) Fei Wu, Zhejiang University & wufei@zju.edu.cn; (6) Jiwei Li, Zhejiang University &  jiwei_li@zju.edu.cn; (7) Tianwei Zhang, Nanyang Technological University & tianwei.zhang@ntu.edu.sg; (8) Guoyin Wang, Bytedance & guoyin.wang@bytedance.com. This paper is available on arxiv under CC 4.0 license. Authors: (1) Xiaofei Sun, Zhejiang University & xiaofei_sun@zju.edu.cn; (2) Xiaoya Li, Shannon.AI, Bytedance & xiaoya_li@shannonai.com; (3) Shengyu Zhang, Zhejiang University & sy_zhang@zju.edu.cn; (4) Shuhe Wang, Peking University & wangshuhe@stu.pku.edu.cn; (5) Fei Wu, Zhejiang University & wufei@zju.edu.cn; (6) Jiwei Li, Zhejiang University &  jiwei_li@zju.edu.cn; (7) Tianwei Zhang, Nanyang Technological University & tianwei.zhang@ntu.edu.sg; (8) Guoyin Wang, Bytedance & guoyin.wang@bytedance.com. Table of Links Abstract and Introduction Related Work LLM Negotiation for Sentiment Analysis Experiments Ablation Studies Conclusion and References Abstract and Introduction Abstract and Introduction Related Work Related Work LLM Negotiation for Sentiment Analysis LLM Negotiation for Sentiment Analysis Experiments Experiments Ablation Studies Ablation Studies Conclusion and References Conclusion and References 3 LLM Negotiation for Sentiment Analysis 3.1 Overview In this section, we detail the multi-LLM negotiation framework for sentiment analysis: Two LLMs perform as the answer generator and discriminator. We refer to the interaction between the generator and the discriminator as a negotiation. The negotiation will repeat until a consensus is reached or the maximum number of negotiation turns is exceeded. Illustrations are shown in Figures 1 and 2. 3.2 Reasoning-infused generator The generator is backboned by a large language model. We ask the answer generator based on the ICL paradigm through prompts, aiming to generate a step-by-step reasoning chain and a decision towards the sentiment polarity of the test input. Prompts are composed of three elements: a task description, demonstrations, and a test input.  The task description is a description of the task in natural language (e.g., "Please determine the overall sentiment of test input."); the test input is the textual input in the test set (e.g., "The sky is blue."); demonstrations are from the train set of the task. Each consists of three elements: input, reasoning chains, and sentimental decision. For each test input, we first retrieve K nearest neighbors (input, sentiment decision) from the train set as demonstrations. Then, we transform demonstrations to (input, reasoning process, sentiment decision) triplets by prompting the generator to produce a reasoning chain. After concatenating the task description, demonstrations, and the test input, we forward the prompt to the generator, which will respond with a step-by-step reasoning chain and a sentimental decision. 3.3 Explanation-deriving discriminator The discriminator is backboned by another LLM. After finishing the answer generating process, the answer discriminator is used to judge whether the decision made by the generator is correct and provide a reasonable explanation. To accomplish this goal, we first construct prompts for the answer discriminator. The prompt is composed of four elements: a task description, demonstrations, a test input, and the response from the answer generator. The task description is a piece of text that describes the task in natural language (e.g., "Please determine whether the decision is correct."). Each demonstration is composed of six elements: (input text, a reasoning chain, sentiment decision, discriminator attitude, discriminator explanations, discriminator decision) and constructed by prompting the answer discriminator to provide explanations of why the sentiment decision is correct for the input text. Then we ask the discriminator with the construct prompt. The answer discriminator will respond with a text string, containing an attitude (i.e., yes, no) that denotes whether the discriminator agrees with the generator, explanations that explain why the discriminator agrees/disagrees with the generator, and a discriminator decision that determines the sentiment of the test input. Why Two LLMs but Not One? There are two reasons for using two different LLMs separately for the generator and the discriminator rather than using a single LLM to act as two roles: (1) If an LLM makes a mistake as a generator due to incorrect reasoning, it is more likely that it will also make the same mistake as the discriminator as since generator and the discriminator from the same model are very likely to make similar rationales; (2) by using two separate models, we are able to take the advantage of the complementary abilities of the two models. Why Two LLMs but Not One? 3.4 Role-flipped Negotiation After two LLMs end with a negotiation, we ask them flip roles and initiate a new negotiation, where the second LLM acts as the generator, and the first LLM acts as the discriminator. We refer the interaction of two LLMs with flipped roles as role-flipped negotiation. Likewise, the role flipped negotiation is ended until a consensus is reached or the maximum number of negotiation turns is exceeded. When both negotiations result in an agreement and their decisions are the same, we can choose either decision as the final one since they are the same. If one of the negotiations fails to reach a consensus while the other reaches a decision, we choose the decision from the negotiation  that reached a consensus as the final decision. However, if both negotiations reach a consensus but their decisions do not align, we will require the assistance of an additional Language Model (LLM), as will be explained in more detail below." Introducing a third LLM If the decision from the two negotiations do not align, we introduce a third LLM and conduct the negotiation and role-flipped negotiation with each of the two aforementioned LLMs. Subsequently, we will get 6 negotiation results and vote on these results: the decision that appears most frequently is taken as the sentiment polarity of the input test. Introducing a third LLM

Sentiment Analysis through LLM Negotiations: LLM Negotiation for Sentiment Analysis

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Xiaofei Sun, Zhejiang University & xiaofei_sun@zju.edu.cn;
(2) Xiaoya Li, Shannon.AI, Bytedance & xiaoya_li@shannonai.com;
(3) Shengyu Zhang, Zhejiang University & sy_zhang@zju.edu.cn;
(4) Shuhe Wang, Peking University & wangshuhe@stu.pku.edu.cn;
(5) Fei Wu, Zhejiang University & wufei@zju.edu.cn;
(6) Jiwei Li, Zhejiang University &  jiwei_li@zju.edu.cn;
(7) Tianwei Zhang, Nanyang Technological University & tianwei.zhang@ntu.edu.sg;
(8) Guoyin Wang, Bytedance & guoyin.wang@bytedance.com. Table of Links Abstract and Introduction
Related Work
LLM Negotiation for Sentiment Analysis
Experiments
Ablation Studies
Conclusion and References 2 Related Work 2.1 Sentiment Analysis Sentiment analysis (Pang and Lee, 2008; Go et al., 2009; Maas et al., 2011a; Zhang and Liu, 2012; Baccianella et al., 2010; Medhat et al., 2014; Bakshi et al., 2016; Zhang et al., 2018) is a task that aims to determine the overall sentiment polarity (e.g., positive, negative, neutral) of a given text. Earlier work often formalized the task as a two-step problem: (1) extract features using RNNs (Socher et al., 2013; Qian et al., 2016; Peled and Reichart, 2017; Wang et al., 2016b; Guggilla et al., 2016; Vo and Zhang, 2015), CNNs (Kalchbrenner et al., 2014; Wang et al., 2016a; Guan et al., 2016; Yu and Jiang, 2016; Mishra et al., 2017), pretrained language models (Lin et al., 2021; Sun et al., 2021; Phan and Ogunbona, 2020; Dai et al., 2021), etc; and (2) feed extracted features into a classifier for obtaining a pre-defined sentimental label. In recent years, in-context learning (ICL) has achieved great success and changed the paradigm of NLP tasks. Many works adapt ICL to the sentiment analysis task: Qin et al. (2023b); Sun et al.(2023a) propose a series of strategies to improve ChatGPT’s performance on the sentiment analysis task; Fei et al. (2023) propose a threehop reasoning framework, which induces the implicit aspect, opinion, and finally the sentiment polarity for the implicit sentiment analysis task;ƒ Zhang et al. (2023d) find that LLMs can achieve satisfactory performance on the binary sentiment classification task, but they underperform to the supervised baseline on more complex tasks (e.g., fine-grained sentiment analysis) that require deeper understanding or structured sentiment information. 2.2 Large Language Models and In-context Learning Large language models (LLMs) (Wang et al., 2022a; Zhang et al., 2023b) are models trained on massive unlabeled text corpora with self supervised learning techniques. Based on the model architecture, LLMs can be categorized into three types: (1) encoder-only models, which contain a text encoder and generate the input representations, such as BERT (Devlin et al., 2018) and its variants (Lan et al., 2019; Liu et al., 2019; Sun et al., 2020; Clark et al., 2020; Feng et al., 2020; Joshi et al., 2020; Sun et al., 2020, 2021); (2) decoder-only models, which have a decoder and generate text conditioned on the input text like GPT-series models (Radford et al., 2019; Brown et al., 2020; Keskar et al., 2019; Radford et al., 2019; Chowdhery et al., 2022; Ouyang et al., 2022; Zhang et al., 2022a; Scao et al., 2022; Zeng et al., 2022b; Touvron et al., 2023a; Peng et al., 2023; OpenAI, 2023); and (3) encoder-decoder models, which have a pair of encoder-decoder and generate text conditioned on the input representation, such as T5 (Raffel et al., 2020) and its variants (Lewis et al., 2019; Xue et al., 2020). Starting with GPT-3 (Brown et al., 2020), LLMs have shown emerging capabilities (Wei et al., 2022a) and completed NLP tasks through incontext learning (ICL), where LLMs generate labelintensive text conditioned on a few annotated examples without gradient updates. Many studies in the literature propose strategies for improving ICL performances on NLP tasks. Li and Liang (2021); Chevalier et al. (2023); Mu et al. (2023) optimize prompts in the continuous space. Liu et al. (2021a); Wan et al. (2023); Zhang et al. (2023a) search through the train set to retrieve k nearest neighbors of a test input as demonstrations. Zhang et al. (2022b); Sun et al. (2023b); Yao et al (2023) decompose a task into a few subtasks and solve them step-by-step towards the final answer conditioned on LLM-generated reasoning chains. Sun et al. (2023a); Wang et al. (2023) propose to verify LLMs’ results by conducting a new round of prompting; Liu et al. (2021b); Feng et al. (2023) use LLMs to generate natural language knowledge statements and integrate external knowledge statements into prompts. 2.3 The LLM collaboration The LLM collaboration involves multiple LLMs working together to solve a given task. Specifically, the task is decomposed to several intermediate tasks, and each LLM is assigned to complete one intermediate task independently. The given task is solved after integrating or summarizing these intermediate results. The LLM collaboration approach can exploit the capabilities of LLMs, improve performances on complex tasks and enable to build complicated systems. Shinn et al. (2023); Sun et al. (2023a); Gero et al. (2023); Wang and Li (2023); Chen et al. (2023b) construct auxiliary tasks (e.g., reflection, verification tasks) and revise the response to the original task referring to the result of the auxiliary task. Talebirad and Nadiri (2023); Hong et al. (2023); Qian et al. (2023) assign characterize profiles (e.g., project manager, software engineer) to LLMs and gain performance boosts on character-specific tasks through behavior animations. Li et al. (2022); Zeng et al. (2022a); Chen et al. (2023a); Du et al. (2023); Liang et al. (2023) use a debate strategy in which multiple different LLMs propose their own responses to the given task and debate over multiple turns until getting a common final answer. Besides, Shen et al. (2023); Gao et al. (2023); Ge et al. (2023); Zhang et al. (2023c); Hao et al. (2023) employone LLM as the task controller, which devises a  plan for the given task, selects expert models for implementation and summarizes the responses of intermediate planned tasks. Other LLMs serve as task executors, completing intermediate tasks in their areas of expertise. This paper is available on arxiv under CC 4.0 license. Authors: (1) Xiaofei Sun, Zhejiang University & xiaofei_sun@zju.edu.cn; (2) Xiaoya Li, Shannon.AI, Bytedance & xiaoya_li@shannonai.com; (3) Shengyu Zhang, Zhejiang University & sy_zhang@zju.edu.cn; (4) Shuhe Wang, Peking University & wangshuhe@stu.pku.edu.cn; (5) Fei Wu, Zhejiang University & wufei@zju.edu.cn; (6) Jiwei Li, Zhejiang University &  jiwei_li@zju.edu.cn; (7) Tianwei Zhang, Nanyang Technological University & tianwei.zhang@ntu.edu.sg; (8) Guoyin Wang, Bytedance & guoyin.wang@bytedance.com. This paper is available on arxiv under CC 4.0 license. Authors: (1) Xiaofei Sun, Zhejiang University & xiaofei_sun@zju.edu.cn; (2) Xiaoya Li, Shannon.AI, Bytedance & xiaoya_li@shannonai.com; (3) Shengyu Zhang, Zhejiang University & sy_zhang@zju.edu.cn; (4) Shuhe Wang, Peking University & wangshuhe@stu.pku.edu.cn; (5) Fei Wu, Zhejiang University & wufei@zju.edu.cn; (6) Jiwei Li, Zhejiang University &  jiwei_li@zju.edu.cn; (7) Tianwei Zhang, Nanyang Technological University & tianwei.zhang@ntu.edu.sg; (8) Guoyin Wang, Bytedance & guoyin.wang@bytedance.com. Table of Links Abstract and Introduction Related Work LLM Negotiation for Sentiment Analysis Experiments Ablation Studies Conclusion and References Abstract and Introduction Abstract and Introduction Related Work Related Work LLM Negotiation for Sentiment Analysis LLM Negotiation for Sentiment Analysis Experiments Experiments Ablation Studies Ablation Studies Conclusion and References Conclusion and References 2 Related Work 2.1 Sentiment Analysis Sentiment analysis (Pang and Lee, 2008; Go et al., 2009; Maas et al., 2011a; Zhang and Liu, 2012; Baccianella et al., 2010; Medhat et al., 2014; Bakshi et al., 2016; Zhang et al., 2018) is a task that aims to determine the overall sentiment polarity (e.g., positive, negative, neutral) of a given text. Earlier work often formalized the task as a two-step problem: (1) extract features using RNNs (Socher et al., 2013; Qian et al., 2016; Peled and Reichart, 2017; Wang et al., 2016b; Guggilla et al., 2016; Vo and Zhang, 2015), CNNs (Kalchbrenner et al., 2014; Wang et al., 2016a; Guan et al., 2016; Yu and Jiang, 2016; Mishra et al., 2017), pretrained language models (Lin et al., 2021; Sun et al., 2021; Phan and Ogunbona, 2020; Dai et al., 2021), etc; and (2) feed extracted features into a classifier for obtaining a pre-defined sentimental label. In recent years, in-context learning (ICL) has achieved great success and changed the paradigm of NLP tasks. Many works adapt ICL to the sentiment analysis task: Qin et al. (2023b); Sun et al.(2023a) propose a series of strategies to improve ChatGPT’s performance on the sentiment analysis task; Fei et al. (2023) propose a threehop reasoning framework, which induces the implicit aspect, opinion, and finally the sentiment polarity for the implicit sentiment analysis task;ƒ Zhang et al. (2023d) find that LLMs can achieve satisfactory performance on the binary sentiment classification task, but they underperform to the supervised baseline on more complex tasks (e.g., fine-grained sentiment analysis) that require deeper understanding or structured sentiment information. 2.2 Large Language Models and In-context Learning Large language models (LLMs) (Wang et al., 2022a; Zhang et al., 2023b) are models trained on massive unlabeled text corpora with self supervised learning techniques. Based on the model architecture, LLMs can be categorized into three types: (1) encoder-only models, which contain a text encoder and generate the input representations, such as BERT (Devlin et al., 2018) and its variants (Lan et al., 2019; Liu et al., 2019; Sun et al., 2020; Clark et al., 2020; Feng et al., 2020; Joshi et al., 2020; Sun et al., 2020, 2021); (2) decoder-only models, which have a decoder and generate text conditioned on the input text like GPT-series models (Radford et al., 2019; Brown et al., 2020; Keskar et al., 2019; Radford et al., 2019; Chowdhery et al., 2022; Ouyang et al., 2022; Zhang et al., 2022a; Scao et al., 2022; Zeng et al., 2022b; Touvron et al., 2023a; Peng et al., 2023; OpenAI, 2023); and (3) encoder-decoder models, which have a pair of encoder-decoder and generate text conditioned on the input representation, such as T5 (Raffel et al., 2020) and its variants (Lewis et al., 2019; Xue et al., 2020). Starting with GPT-3 (Brown et al., 2020), LLMs have shown emerging capabilities (Wei et al., 2022a) and completed NLP tasks through incontext learning (ICL), where LLMs generate labelintensive text conditioned on a few annotated examples without gradient updates. Many studies in the literature propose strategies for improving ICL performances on NLP tasks. Li and Liang (2021); Chevalier et al. (2023); Mu et al. (2023) optimize prompts in the continuous space. Liu et al. (2021a); Wan et al. (2023); Zhang et al. (2023a) search through the train set to retrieve k nearest neighbors of a test input as demonstrations. Zhang et al. (2022b); Sun et al. (2023b); Yao et al (2023) decompose a task into a few subtasks and solve them step-by-step towards the final answer conditioned on LLM-generated reasoning chains. Sun et al. (2023a); Wang et al. (2023) propose to verify LLMs’ results by conducting a new round of prompting; Liu et al. (2021b); Feng et al. (2023) use LLMs to generate natural language knowledge statements and integrate external knowledge statements into prompts. 2.3 The LLM collaboration The LLM collaboration involves multiple LLMs working together to solve a given task. Specifically, the task is decomposed to several intermediate tasks, and each LLM is assigned to complete one intermediate task independently. The given task is solved after integrating or summarizing these intermediate results. The LLM collaboration approach can exploit the capabilities of LLMs, improve performances on complex tasks and enable to build complicated systems. Shinn et al. (2023); Sun et al. (2023a); Gero et al. (2023); Wang and Li (2023); Chen et al. (2023b) construct auxiliary tasks (e.g., reflection, verification tasks) and revise the response to the original task referring to the result of the auxiliary task. Talebirad and Nadiri (2023); Hong et al. (2023); Qian et al. (2023) assign characterize profiles (e.g., project manager, software engineer) to LLMs and gain performance boosts on character-specific tasks through behavior animations. Li et al. (2022); Zeng et al. (2022a); Chen et al. (2023a); Du et al. (2023); Liang et al. (2023) use a debate strategy in which multiple different LLMs propose their own responses to the given task and debate over multiple turns until getting a common final answer. Besides, Shen et al. (2023); Gao et al. (2023); Ge et al. (2023); Zhang et al. (2023c); Hao et al. (2023) employone LLM as the task controller, which devises a  plan for the given task, selects expert models for implementation and summarizes the responses of intermediate planned tasks. Other LLMs serve as task executors, completing intermediate tasks in their areas of expertise.

Sentiment Analysis through LLM Negotiations: Related Work

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Xiaofei Sun, Zhejiang University & xiaofei_sun@zju.edu.cn;
(2) Xiaoya Li, Shannon.AI, Bytedance & xiaoya_li@shannonai.com;
(3) Shengyu Zhang, Zhejiang University & sy_zhang@zju.edu.cn;
(4) Shuhe Wang, Peking University & wangshuhe@stu.pku.edu.cn;
(5) Fei Wu, Zhejiang University & wufei@zju.edu.cn;
(6) Jiwei Li, Zhejiang University &  jiwei_li@zju.edu.cn;
(7) Tianwei Zhang, Nanyang Technological University & tianwei.zhang@ntu.edu.sg;
(8) Guoyin Wang, Bytedance & guoyin.wang@bytedance.com. Table of Links Abstract and Introduction
Related Work
LLM Negotiation for Sentiment Analysis
Experiments
Ablation Studies
Conclusion and References 4 Experiments To evaluate the effectiveness of the proposed method, we use GPT-3.5, GPT-4 (OpenAI, 2023) and InstructGPT3.5 (Ouyang et al., 2022) [1] as backbones for the multi-model negotiation method. In this process, we use the fine-tuned RoBERTa-Large (Liu et al., 2019) as the similarity function for retrieving k nearest neighbors as demonstrations. In the empirical study, we investigate the following three distinct ICL approaches, offering insights of integrating such methods for sentiment analysis. • Vanilla ICL: the sentiment analysis task is finished by asking a LLM with a prompt to generate sentiment-intensive text without gradient updates. In practice, we conduct two sets of experiments under this setting with GPT3.5 and GPT-4, respectively. • Self-Negotiation: the task is finished by using one LLM to discriminate and correct the answer generated by itself. We conduct two experiments with GPT3.5 and GPT-4 and get two results. • Negotiation with two LLMs: the task is completed by employing two different LLMs to take turns performing as the answer generator and discriminator. Specifically, we conduct one set of experiment with GPT3.5 and GPT-4. 4.1 Datasets We conduct experiments on six sentiment analysis datasets, including SST-2 (Socher et al., 2013), Movie Review (Zhang et al., 2015), Twitter (Rosenthal et al., 2019), Yelp-Binary (Zhang et al., 2015), AmazonBinary (Zhang et al., 2015), and IMDB (Maas et al., 2011b). More details of the datasets are shown as follows: • SST-2 (Socher et al., 2013): SST-2 is a binary (i.e., positive, negative) sentiment classification dataset and contains movie review snippets from the Rotton Tomato. We follow Socher et al. (2013) and use the train, valid, test splits with the number of examples of 67,349, 872, 1,821, respectively. • Movie Review (MR) (Zhang et al., 2015): Movie Reviews is a dataset for use in sentiment-analysis experiments. Available are collections of movie-review documents labeled with respect to their overall sentiment polarity (i.e., positive or negative). • Twitter (Rosenthal et al., 2019): Twitter is a three-class (i.e., positive, negative, neutral) sentiment analysis dataset, aiming to detecting whether a piece of text expresses a sentiment polarity in respect to a specific topic, such as a person, a product, or an event. The dataset is origin a shared task at SemEval 2017, containing 50,333 examples in the train set and 12,284 examples in the test set. • Yelp-Binary (Zhang et al., 2015): Yelp is a binary (i.e., positive, negative) sentiment analysis dataset, containing product reviews from Yelp. The dataset has 560,000 trainig samples and 38,000 testing samples. • Amazon-Binary (Zhang et al., 2015): Amazon is a binary sentiment classification task, containing product reviews from Amazon with 3,600,000 examples in the train set and 400,000 examples in the test set. • IMDB (Maas et al., 2011b): The IMDB dataset contains movie reviews along with their associated binary sentiment polarity labels. The dataset contains 50,000 reviews split evenly into 25k train and 25k test sets. The overall distribution of labels is balanced (25k positive and 25k negative). We use accuracy as the evaluation metric. 4.2 Baselines We use supervised neural network models and ICL approaches with LLMs as baselines for comparisons. For supervised methods, we choose the following four models: • DRNN (Wang, 2018): incorporates positioninvariance into RNN and CNN models by limiting the distance of information flow in neural networks. • RoBERTa (Liu et al., 2019): is a reimplementation of BERT (Devlin et al., 2018) aiming to improve performances on NLP downstream tasks. In this paper, we report results achieved by fine-tuned RoBERTa-Large. • XLNet (Yang et al., 2019): is a pretrained autoregressive LM that integrates Transformer-XL (Dai et al., 2019) and enables to learn bidirectional contexts by maximizing the expected likelihood over all permutations of the factorization order. • UDA (Xie et al., 2020): is short for Unsupervised Data Augmentation, which is a data augmentation strategy that employs a consistency loss function for unsupervised and supervised training stages. Performances in Table 1 are obtained by BERT-Large with UDA. • BERTweet (Nguyen et al., 2020): is a pretrained language model for English Tweets. The BERTweet has the same number of parameters as RoBERTa-Base. • EFL (Wang et al., 2021): is backboned by RoBERTa-Large and fine-tuned on natural language entailment examples. For ICL approaches, we report experimental results with LLMs from the following studies: • Zhang et al. (2023d): presents a comprehensive study for applying LLMs (i.e., FLan-UL2, T5 and ChatGPT) on sentiment analysis tasks. Experimental results in the Table 1 are obtained in few-shot(k = 5) settings. • InstructGPT-3.5 (Ouyang et al., 2022): is a large language model trained to follow human instructions. Experimental results in the Table 1 are achieved by the text-davinci-003 model. • IDS (Qin et al., 2023a): propose an Iterative Demonstration Selection (IDS) strategy to select demonstrations from diversity, similarity, and task-specific perspectives. Results shown in Table 1 are obtained by using GPT-3.5 (gpt-3.5-turbo). • GPT-4 (OpenAI, 2023): is a large multimodal model, achieving human-level performance on various NLP benchmarks. • Self-negotiation: The same LLM acts as both the roles of the generator and the discriminator. 4.3 Results and analysis Experiment results are shown in Table 1. As can be seen in the table, compared to vanilla ICL, following the generate-discriminate paradigm with one LLM (self-negotiation) receives performance gains on six sentiment analysis datasets: GPT-3.5 gains +0.9 on average; GPT-4 receives +1.0 acc on average. This phenomenon illustrates that the LLM, performing as the answer discriminator, can correct a portion of errors caused by the task generator. We also observe that using two different LLMs as the task generator and task discriminator in turn introduces significant performance improvements compared to merely using one model. Negotiations with two LLMs outperform the self-negotiation method by +1.7, +2.1, and +2.3 in terms of accuracy on MR, Twitter, IMDB datasets, respectively. The reason for this phenomenon is that using two different LLMs finish the sentiment analysis task through negotiations can take the advantage of different understandings of the given input and unleash the power of two LLMs, leading to more accurate decisions. We also find that when introduce a third LLM to resolve the disagreement between the flippled-roled negotiations, additional performance boost can be obtained. This demonstrates that the third LLM can resolve conflicts between two LLMs through multiple negotiations and improve performances on the sentiment analysis task. It is noteworthy that the multi-model negotiation method outperforms the supervised method RoBERTa-Large by +0.9 on the MR dataset, and bridges the gap between vanilla ICL and the supervised method: achieving 94.1 (+1.4) accuracy on SST-2; 92.1 (+2.7) on Twitter; 96.3 (+2.5) on Yelp-2; 87.2 (+3.7) on Amazon-2; and 94.5 (+6.0) on IMDB dataset. [1] text-davinci-003 This paper is available on arxiv under CC 4.0 license. Authors: (1) Xiaofei Sun, Zhejiang University & xiaofei_sun@zju.edu.cn; (2) Xiaoya Li, Shannon.AI, Bytedance & xiaoya_li@shannonai.com; (3) Shengyu Zhang, Zhejiang University & sy_zhang@zju.edu.cn; (4) Shuhe Wang, Peking University & wangshuhe@stu.pku.edu.cn; (5) Fei Wu, Zhejiang University & wufei@zju.edu.cn; (6) Jiwei Li, Zhejiang University &  jiwei_li@zju.edu.cn; (7) Tianwei Zhang, Nanyang Technological University & tianwei.zhang@ntu.edu.sg; (8) Guoyin Wang, Bytedance & guoyin.wang@bytedance.com. This paper is available on arxiv under CC 4.0 license. Authors: (1) Xiaofei Sun, Zhejiang University & xiaofei_sun@zju.edu.cn; (2) Xiaoya Li, Shannon.AI, Bytedance & xiaoya_li@shannonai.com; (3) Shengyu Zhang, Zhejiang University & sy_zhang@zju.edu.cn; (4) Shuhe Wang, Peking University & wangshuhe@stu.pku.edu.cn; (5) Fei Wu, Zhejiang University & wufei@zju.edu.cn; (6) Jiwei Li, Zhejiang University &  jiwei_li@zju.edu.cn; (7) Tianwei Zhang, Nanyang Technological University & tianwei.zhang@ntu.edu.sg; (8) Guoyin Wang, Bytedance & guoyin.wang@bytedance.com. Table of Links Abstract and Introduction Related Work LLM Negotiation for Sentiment Analysis Experiments Ablation Studies Conclusion and References Abstract and Introduction Abstract and Introduction Related Work Related Work LLM Negotiation for Sentiment Analysis LLM Negotiation for Sentiment Analysis Experiments Experiments Ablation Studies Ablation Studies Conclusion and References Conclusion and References 4 Experiments To evaluate the effectiveness of the proposed method, we use GPT-3.5, GPT-4 (OpenAI, 2023) and InstructGPT3.5 (Ouyang et al., 2022) [1] as backbones for the multi-model negotiation method. In this process, we use the fine-tuned RoBERTa-Large (Liu et al., 2019) as the similarity function for retrieving k nearest neighbors as demonstrations. In the empirical study, we investigate the following three distinct ICL approaches, offering insights of integrating such methods for sentiment analysis. • Vanilla ICL: the sentiment analysis task is finished by asking a LLM with a prompt • Vanilla ICL: to generate sentiment-intensive text without gradient updates. In practice, we conduct two sets of experiments under this setting with GPT3.5 and GPT-4, respectively. • Self-Negotiation: the task is finished by using one LLM to discriminate and correct the • Self-Negotiation: answer generated by itself. We conduct two experiments with GPT3.5 and GPT-4 and get two results. • Negotiation with two LLMs: the task is completed by employing two different • Negotiation with two LLMs: LLMs to take turns performing as the answer generator and discriminator. Specifically, we conduct one set of experiment with GPT3.5 and GPT-4. 4.1 Datasets We conduct experiments on six sentiment analysis datasets, including SST-2 (Socher et al., 2013), Movie Review (Zhang et al., 2015), Twitter (Rosenthal et al., 2019), Yelp-Binary (Zhang et al., 2015), AmazonBinary (Zhang et al., 2015), and IMDB (Maas et al., 2011b). More details of the datasets are shown as follows: • SST-2 (Socher et al., 2013): SST-2 is a binary (i.e., positive, negative) sentiment • SST-2 classification dataset and contains movie review snippets from the Rotton Tomato. We follow Socher et al. (2013) and use the train, valid, test splits with the number of examples of 67,349, 872, 1,821, respectively. • Movie Review (MR) (Zhang et al., 2015): Movie Reviews is a dataset for use in sentiment-analysis experiments. Available are collections of movie-review documents labeled with respect to their overall sentiment polarity (i.e., positive or negative). • Movie Review (MR) • Twitter (Rosenthal et al., 2019): Twitter is a three-class (i.e., positive, negative, neutral) sentiment analysis dataset, aiming to detecting whether a piece of text expresses a sentiment polarity in respect to a specific topic, such as a person, a product, or an event. The dataset is origin a shared task at SemEval 2017, containing 50,333 examples in the train set and 12,284 examples in the test set. • Twitter • Yelp-Binary (Zhang et al., 2015): Yelp is a binary (i.e., positive, negative) sentiment analysis dataset, containing product reviews from Yelp. The dataset has 560,000 trainig samples and 38,000 testing samples. • Yelp-Binary • Amazon-Binary (Zhang et al., 2015): Amazon is a binary sentiment classification task, containing product reviews from Amazon with 3,600,000 examples in the train set and 400,000 examples in the test set. • Amazon-Binary • IMDB (Maas et al., 2011b): The IMDB dataset contains movie reviews along with their associated binary sentiment polarity labels. The dataset contains 50,000 reviews split evenly into 25k train and 25k test sets. The overall distribution of labels is balanced (25k positive and 25k negative). • IMDB We use accuracy as the evaluation metric. 4.2 Baselines We use supervised neural network models and ICL approaches with LLMs as baselines for comparisons. For supervised methods, we choose the following four models: • DRNN (Wang, 2018): incorporates positioninvariance into RNN and CNN models by limiting the distance of information flow in neural networks. • DRNN • RoBERTa (Liu et al., 2019): is a reimplementation of BERT (Devlin et al., 2018) aiming to improve performances on NLP downstream tasks. In this paper, we report results achieved by fine-tuned RoBERTa-Large. RoBERTa • XLNet (Yang et al., 2019): is a pretrained autoregressive LM that integrates Transformer-XL (Dai et al., 2019) and enables to learn bidirectional contexts by maximizing the expected likelihood over all permutations of the factorization order. XLNet • UDA (Xie et al., 2020): is short for Unsupervised Data Augmentation, which is a data augmentation strategy that employs a consistency loss function for unsupervised and supervised training stages. Performances in Table 1 are obtained by BERT-Large with UDA. • UDA • BERTweet (Nguyen et al., 2020): is a pretrained language model for English Tweets. The BERTweet has the same number of parameters as RoBERTa-Base. • BERTweet • EFL (Wang et al., 2021): is backboned by RoBERTa-Large and fine-tuned on natural language entailment examples. EFL For ICL approaches, we report experimental results with LLMs from the following studies: • Zhang et al. (2023d): presents a comprehensive study for applying LLMs (i.e., FLan-UL2, T5 and ChatGPT) on sentiment analysis tasks. Experimental results in the Table 1 are obtained in few-shot(k = 5) settings. • InstructGPT-3.5 (Ouyang et al., 2022): is a large language model trained to follow human instructions. Experimental results in the Table 1 are achieved by the text-davinci-003 model. InstructGPT-3.5 • IDS (Qin et al., 2023a): propose an Iterative Demonstration Selection (IDS) strategy to select demonstrations from diversity, similarity, and task-specific perspectives. Results shown in Table 1 are obtained by using GPT-3.5 (gpt-3.5-turbo). • IDS • GPT-4 (OpenAI, 2023): is a large multimodal model, achieving human-level performance on various NLP benchmarks. • GPT-4 • Self-negotiation: The same LLM acts as both the roles of the generator and the discriminator. • Self-negotiation: 4.3 Results and analysis Experiment results are shown in Table 1. As can be seen in the table, compared to vanilla ICL, following the generate-discriminate paradigm with one LLM (self-negotiation) receives performance gains on six sentiment analysis datasets: GPT-3.5 gains +0.9 on average; GPT-4 receives +1.0 acc on average. This phenomenon illustrates that the LLM, performing as the answer discriminator, can correct a portion of errors caused by the task generator. We also observe that using two different LLMs as the task generator and task discriminator in turn introduces significant performance improvements compared to merely using one model. Negotiations with two LLMs outperform the self-negotiation method by +1.7, +2.1, and +2.3 in terms of accuracy on MR, Twitter, IMDB datasets, respectively. The reason for this phenomenon is that using two different LLMs finish the sentiment analysis task through negotiations can take the advantage of different understandings of the given input and unleash the power of two LLMs, leading to more accurate decisions. We also find that when introduce a third LLM to resolve the disagreement between the flippled-roled negotiations, additional performance boost can be obtained. This demonstrates that the third LLM can resolve conflicts between two LLMs through multiple negotiations and improve performances on the sentiment analysis task. It is noteworthy that the multi-model negotiation method outperforms the supervised method RoBERTa-Large by +0.9 on the MR dataset, and bridges the gap between vanilla ICL and the supervised method: achieving 94.1 (+1.4) accuracy on SST-2; 92.1 (+2.7) on Twitter; 96.3 (+2.5) on Yelp-2; 87.2 (+3.7) on Amazon-2; and 94.5 (+6.0) on IMDB dataset. [1] text-davinci-003

Sentiment Analysis through LLM Negotiations: Experiments

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Xiaofei Sun, Zhejiang University & xiaofei_sun@zju.edu.cn;
(2) Xiaoya Li, Shannon.AI, Bytedance & xiaoya_li@shannonai.com;
(3) Shengyu Zhang, Zhejiang University & sy_zhang@zju.edu.cn;
(4) Shuhe Wang, Peking University & wangshuhe@stu.pku.edu.cn;
(5) Fei Wu, Zhejiang University & wufei@zju.edu.cn;
(6) Jiwei Li, Zhejiang University &  jiwei_li@zju.edu.cn;
(7) Tianwei Zhang, Nanyang Technological University & tianwei.zhang@ntu.edu.sg;
(8) Guoyin Wang, Bytedance & guoyin.wang@bytedance.com. Table of Links Abstract and Introduction
Related Work
LLM Negotiation for Sentiment Analysis
Experiments
Ablation Studies
Conclusion and References Abstract A standard paradigm for sentiment analysis is to rely on a singular LLM that makes the decision in a single round under the framework of in-context learning. This framework suffers the key disadvantage that the single-turn output generated by a single LLM might not deliver the perfect decision, just as humans sometimes need multiple attempts to get things right. This is especially true for the task of sentiment analysis where deep reasoning is required to address the complex linguistic phenomenon (e.g., clause composition, irony, etc) in the input. To address this issue, this paper introduces a multi-LLM negotiation framework for sentiment analysis. The framework consists of a reasoning-infused generator to provide decision along with rationale, a explanationderiving discriminator to evaluate the credibility of the generator. The generator and the discriminator iterate until a consensus is reached. The proposed framework naturally addressed the aforementioned challenge, as we are able to take the complementary abilities of two LLMs, have them use rationale to persuade each other for correction. Experiments on a wide range of sentiment analysis benchmarks (SST-2, Movie Review, Twitter, yelp, amazon, IMDB) demonstrate the effectiveness of proposed approach: it consistently yields better performances than the ICL baseline across all benchmarks, and even superior performances to supervised baselines on the Twitter and movie review datasets. 1 Introduction Sentiment analysis (Pang and Lee, 2008; Go et al., 2009; Maas et al., 2011a; Zhang and Liu, 2012; Baccianella et al., 2010; Medhat et al., 2014; Bakshi et al., 2016; Zhang et al., 2018) aims to extract opinion polarity expressed by a chunk of text. Recent advances in large language models (LLMs) (Brown et al., 2020; Ouyang et al., 2022; Touvron et al., 2023a,b; Anil et al., 2023; Zeng et al., 2022b; OpenAI, 2023; Bai et al., 2023) open a new door for the resolving the task (Lu et al., 2021; Kojima et al., 2022; Wang et al., 2022b; Wei et al., 2022b; Wan et al., 2023; Wang et al., 2023; Sun et al., 2023b,a; Lightman et al., 2023; Li et al., 2023; Schick et al., 2023): under the paradigm of in-context learning (ICL), LLMs are able to achieve performances comparable to supervised learning strategies (Lin et al., 2021; Sun et al., 2021; Phan and Ogunbona, 2020; Dai et al., 2021) with only a small number of training examples. Existing approaches that harness LLMs for sentiment analysis usually rely on a singular LLM, and make a decision in a single round under ICL. This strategy suffers from the following disadvantage: the single-turn output generated by a single LLM might not deliver the perfect response: Just as humans sometimes need multiple attempts to get things right, it might take multiple rounds before an LLM makes the right decision. This is especially true for the task of sentiment analysis, where LLMs usually need to articulate the reasoning process to address the complex linguistic phenomenon (e.g., clause composition, irony, etc) in the input sentence. To address the this issue, in this paper, we propose a multi-LLM negotiation strategy for sentiment analysis. The core of the proposed strategy is a generator-discriminator framework, where one LLM acts as the generator (G) to produce sentiment decisions, while the other acts as a discriminator (D), tasked with evaluating the credibility of the generated output from the first LLM. The proposed method innovates on three aspects: (1) Reasoning-infused generator (G): an LLM that adheres to a structured reasoning chain, enhancing the ICL of the generator while offering the discriminator the evidence and insights to evaluate its validity; (2) Explanation deriving discriminator (D); other LLM designed to offer post-evaluation rationales for its judgments; (3) Negotiation: two LLMs act as the roles of the generator and the discriminator, and perform the negotiation until a consensus is reached. This strategy harnesses the collective abilities of the two LLMs and provide with the channel for the model to correct imperfect responses, and thus naturally resolves the issue that a single LLM cannot yield the correct decision on its first try. The contributions of this work can be summarized as follows: 1) we provide a novel perspective on how sentiment analysis can benefit from multi-LLM negotiation. 2) we introduce a Generator Discriminator Role-switching DecisionMaking framework that enables multi-LLM collaboration through iteratively generating and validating sentiment categorizations. 3) our empirical findings offer evidence for the efficacy of the proposed approach: experiments on a wide range of sentiment analysis benchmarks (SST-2, Movie Review, Twitter, yelp, amazon, IMDB) demonstrate that the proposed method consistently yields better performances than the ICL baseline across all benchmarks, and even superior performances to supervised baselines on the Twitter and movie review datasets. This paper is available on arxiv under CC 4.0 license. Authors: (1) Xiaofei Sun, Zhejiang University & xiaofei_sun@zju.edu.cn; (2) Xiaoya Li, Shannon.AI, Bytedance & xiaoya_li@shannonai.com; (3) Shengyu Zhang, Zhejiang University & sy_zhang@zju.edu.cn; (4) Shuhe Wang, Peking University & wangshuhe@stu.pku.edu.cn; (5) Fei Wu, Zhejiang University & wufei@zju.edu.cn; (6) Jiwei Li, Zhejiang University &  jiwei_li@zju.edu.cn; (7) Tianwei Zhang, Nanyang Technological University & tianwei.zhang@ntu.edu.sg; (8) Guoyin Wang, Bytedance & guoyin.wang@bytedance.com. This paper is available on arxiv under CC 4.0 license. Authors: (1) Xiaofei Sun, Zhejiang University & xiaofei_sun@zju.edu.cn; (2) Xiaoya Li, Shannon.AI, Bytedance & xiaoya_li@shannonai.com; (3) Shengyu Zhang, Zhejiang University & sy_zhang@zju.edu.cn; (4) Shuhe Wang, Peking University & wangshuhe@stu.pku.edu.cn; (5) Fei Wu, Zhejiang University & wufei@zju.edu.cn; (6) Jiwei Li, Zhejiang University &  jiwei_li@zju.edu.cn; (7) Tianwei Zhang, Nanyang Technological University & tianwei.zhang@ntu.edu.sg; (8) Guoyin Wang, Bytedance & guoyin.wang@bytedance.com. Table of Links Abstract and Introduction Related Work LLM Negotiation for Sentiment Analysis Experiments Ablation Studies Conclusion and References Abstract and Introduction Abstract and Introduction Related Work Related Work LLM Negotiation for Sentiment Analysis LLM Negotiation for Sentiment Analysis Experiments Experiments Ablation Studies Ablation Studies Conclusion and References Conclusion and References Abstract A standard paradigm for sentiment analysis is to rely on a singular LLM that makes the decision in a single round under the framework of in-context learning. This framework suffers the key disadvantage that the single-turn output generated by a single LLM might not deliver the perfect decision, just as humans sometimes need multiple attempts to get things right. This is especially true for the task of sentiment analysis where deep reasoning is required to address the complex linguistic phenomenon (e.g., clause composition, irony, etc) in the input. To address this issue, this paper introduces a multi-LLM negotiation framework for sentiment analysis. The framework consists of a reasoning-infused generator to provide decision along with rationale, a explanationderiving discriminator to evaluate the credibility of the generator. The generator and the discriminator iterate until a consensus is reached. The proposed framework naturally addressed the aforementioned challenge, as we are able to take the complementary abilities of two LLMs, have them use rationale to persuade each other for correction. Experiments on a wide range of sentiment analysis benchmarks (SST-2, Movie Review, Twitter, yelp, amazon, IMDB) demonstrate the effectiveness of proposed approach: it consistently yields better performances than the ICL baseline across all benchmarks, and even superior performances to supervised baselines on the Twitter and movie review datasets. 1 Introduction Sentiment analysis (Pang and Lee, 2008; Go et al., 2009; Maas et al., 2011a; Zhang and Liu, 2012; Baccianella et al., 2010; Medhat et al., 2014; Bakshi et al., 2016; Zhang et al., 2018) aims to extract opinion polarity expressed by a chunk of text. Recent advances in large language models (LLMs) (Brown et al., 2020; Ouyang et al., 2022; Touvron et al., 2023a,b; Anil et al., 2023; Zeng et al., 2022b; OpenAI, 2023; Bai et al., 2023) open a new door for the resolving the task (Lu et al., 2021; Kojima et al., 2022; Wang et al., 2022b; Wei et al., 2022b; Wan et al., 2023; Wang et al., 2023; Sun et al., 2023b,a; Lightman et al., 2023; Li et al., 2023; Schick et al., 2023): under the paradigm of in-context learning (ICL), LLMs are able to achieve performances comparable to supervised learning strategies (Lin et al., 2021; Sun et al., 2021; Phan and Ogunbona, 2020; Dai et al., 2021) with only a small number of training examples. Existing approaches that harness LLMs for sentiment analysis usually rely on a singular LLM, and make a decision in a single round under ICL. This strategy suffers from the following disadvantage: the single-turn output generated by a single LLM might not deliver the perfect response: Just as humans sometimes need multiple attempts to get things right, it might take multiple rounds before an LLM makes the right decision. This is especially true for the task of sentiment analysis, where LLMs usually need to articulate the reasoning process to address the complex linguistic phenomenon (e.g., clause composition, irony, etc) in the input sentence. To address the this issue, in this paper, we propose a multi-LLM negotiation strategy for sentiment analysis. The core of the proposed strategy is a generator-discriminator framework, where one LLM acts as the generator (G) to produce sentiment decisions, while the other acts as a discriminator (D), tasked with evaluating the credibility of the generated output from the first LLM. The proposed method innovates on three aspects: (1) Reasoning-infused generator (G): an LLM that adheres to a structured reasoning chain, enhancing the ICL of the generator while offering the discriminator the evidence and insights to evaluate its validity; (2) Explanation deriving discriminator (D); other LLM designed to offer post-evaluation rationales for its judgments; (3) Negotiation: two LLMs act as the roles of the generator and the discriminator, and perform the negotiation until a consensus is reached. This strategy harnesses the collective abilities of the two LLMs and provide with the channel for the model to correct imperfect responses, and thus naturally resolves the issue that a single LLM cannot yield the correct decision on its first try. The contributions of this work can be summarized as follows: 1) we provide a novel perspective on how sentiment analysis can benefit from multi-LLM negotiation. 2) we introduce a Generator Discriminator Role-switching DecisionMaking framework that enables multi-LLM collaboration through iteratively generating and validating sentiment categorizations. 3) our empirical findings offer evidence for the efficacy of the proposed approach: experiments on a wide range of sentiment analysis benchmarks (SST-2, Movie Review, Twitter, yelp, amazon, IMDB) demonstrate that the proposed method consistently yields better performances than the ICL baseline across all benchmarks, and even superior performances to supervised baselines on the Twitter and movie review datasets.

Sentiment Analysis through LLM Negotiations: Abstract and Introduction 

Sentiment Analysis through LLM Negotiations: Conclusion and References

This paper is under CC 4.0 license. available on arxiv Authors: (1) Zhe Liu, State Key Laboratory of Intelligent Game, Beijing, China Institute of Software Chinese Academy of Sciences, Beijing, China; University of Chinese Academy of Sciences, Beijing, China; (2) Chunyang Chen, Monash University, Melbourne, Australia; (3) Junjie Wang, State Key Laboratory of Intelligent Game, Beijing, China Institute of Software Chinese Academy of Sciences, Beijing, China; University of Chinese Academy of Sciences, Beijing, China & Corresponding author; (4) Mengzhuo Chen, State Key Laboratory of Intelligent Game, Beijing, China Institute of Software Chinese Academy of Sciences, Beijing, China; University of Chinese Academy of Sciences, Beijing, China; (5) Boyu Wu, State Key Laboratory of Intelligent Game, Beijing, China Institute of Software Chinese Academy of Sciences, Beijing, China; University of Chinese Academy of Sciences, Beijing, China; (6) Zhilin Tian, State Key Laboratory of Intelligent Game, Beijing, China Institute of Software Chinese Academy of Sciences, Beijing, China; University of Chinese Academy of Sciences, Beijing, China; (7) Yuekai Huang, State Key Laboratory of Intelligent Game, Beijing, China Institute of Software Chinese Academy of Sciences, Beijing, China; University of Chinese Academy of Sciences, Beijing, China; (8) Jun Hu, State Key Laboratory of Intelligent Game, Beijing, China Institute of Software Chinese Academy of Sciences, Beijing, China; University of Chinese Academy of Sciences, Beijing, China; (9) Qing Wang, State Key Laboratory of Intelligent Game, Beijing, China Institute of Software Chinese Academy of Sciences, Beijing, China; University of Chinese Academy of Sciences, Beijing, China & Corresponding author. Table of Links Abstract and Introduction Motivational Study and Background Approach Experiment Design Results and Analysis Discussion and Threats to Validity Related Work Conclusion and References 8 CONCLUSION Automated testing is crucial for helping improve app quality. Despite the dozens of mobile app GUI testing techniques, how to automatically generate the diversified unusual text inputs for fully testing mobile apps remains a challenge. This paper proposes InputBlaster which leverages the LLM to produce the unusual inputs together with the mutation rules which serve as the reasoning chains. It formulates the unusual inputs generation problem as a task of producing a set of test generators, each of which can yield a batch of unusual text inputs under the same mutation rule. The evaluation is conducted for both effectiveness and usefulness, with 136% higher bug detection rate than the best baselines, and uncovering 37 new crashes. In the future, we plan to further analyze the root causes and repair strategy of these input-related bugs, and design automated bug repair methods. REFERENCES [1] 2022. Crash bug text. https://www.theguardian.com/technology/iphone-crashbug-text imessage-ios. [2] 2022. Crash bug text in ios. https://tech.hindustantimes.com/tech/news/becareful-a-new-text bomb-is-making-whatsapp-crash-and-will-hang-yourphone-71599532897852.html. [3] 2022. F-droid. https://f-droid.org/. [4] Toufique Ahmed and Premkumar Devanbu. 2022. Few-shot training LLMs for project-specific code-summarization. ASE (2022). [5] Nadia Alshahwan and Mark Harman. 2011. Automated web application testing using search based software engineering. In ASE. IEEE, 3–12. [6] Saswat Anand, Mayur Naik, Mary Jean Harrold, and Hongseok Yang. 2012. Automated concolic testing of smartphone apps. In Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering. 1–11. [7] Yauhen Leanidavich Arnatovich, Minh Ngoc Ngo, Tan Hee Beng Kuan, and Charlie Soh. 2016. Achieving high code coverage in android ui testing via automated widget exercising. In 2016 23rd Asia-Pacific Software Engineering Conference (APSEC). IEEE, 193–200. [8] Yauhen Leanidavich Arnatovich, Lipo Wang, Ngoc Minh Ngo, and Charlie Soh. 2018. Mobolic: An automated approach to exercising mobile application GUIs using symbiosis of online testing technique and customated input generation. Software: Practice and Experience 48, 5 (2018), 1107 1142. [9] Tanzirul Azim and Iulian Neamtiu. 2013. Targeted and depth-first exploration for systematic testing of android apps. In Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications. 641–660. [10] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901. [11] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901. [12] Tianqin Cai, Zhao Zhang, and Ping Yang. 2020. Fastbot: A Multi-Agent ModelBased Test Generation System Beijing Bytedance Network Technology Co., Ltd.. In Proceedings of the IEEE/ACM 1st International Conference on Automation of Software Test. 93–96. [13] Chen Chen, Baojiang Cui, Jinxin Ma, Runpu Wu, Jianchao Guo, and Wenqian Liu. 2018. A systematic review of fuzzing techniques. Computers & Security 75 (2018), 118–137. [14] Taolue Chen, Alejandro Flores-Lamas, Matthew Hague, Zhilei Han, Denghang Hu, Shuanglong Kan, Anthony W Lin, Philipp Rümmer, and Zhilin Wu. 2022. Solving string constraints with Regex dependent functions through transducers with priorities and variables. Proceedings of the ACM on Programming Languages 6, POPL (2022), 1–31. [15] Taolue Chen, Matthew Hague, Jinlong He, Denghang Hu, Anthony Widjaja Lin, Philipp Rümmer, and Zhilin Wu. 2020. A decision procedure for path feasibility of string manipulating programs with integer data type. In Automated Technology for Verification and Analysis: 18th International Symposium, ATVA 2020, Hanoi, Vietnam, October 19–23, 2020, Proceedings. Springer, 325–342. [16] Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. 2020. Big self-supervised models are strong semi-supervised learners. Advances in neural information processing systems 33 (2020), 22243–22255. [17] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 (2022). [18] Joel D Day, Thorsten Ehlers, Mitja Kulczynski, Florin Manea, Dirk Nowotka, and Danny Bøgsted Poulsen. 2019. On solving word equations using SAT. In Reachability Problems: 13th International Conference, RP 2019, Brussels, Belgium, September 11–13, 2019, Proceedings 13. Springer, 93–106. [19] Biplab Deka, Zifeng Huang, Chad Franzen, Joshua Hibschman, Daniel Afergan,  Yang Li, Jeffrey Nichols, and Ranjitha Kumar. 2017. Rico: A Mobile App Dataset for Building Data-Driven Design Applications. In UIST. [20] Yinlin Deng, Chunqiu Steven Xia, Chenyuan Yang, Shizhuo Dylan Zhang, Shujing Yang, and Lingming Zhang. 2023. Large Language Models are Edge-Case Fuzzers: Testing Deep Learning Libraries via FuzzGPT. ISSTA (2023). [21] Yinlin Deng, Chenyuan Yang, Anjiang Wei, and Lingming Zhang. 2022. Fuzzing deep-learning libraries via automated relational API inference. In Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 44–56. [22] Android Developers. 2012. Ui/application exerciser monkey. [23] Zhen Dong, Marcel Böhme, Lucia Cojocaru, and Abhik Roychoudhury. 2020. Time-travel testing of android apps. In 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE). IEEE, 481–492. [24] Robert Feldt and Simon Poulding. 2013. Finding test data with specific properties via metaheuristic search. In 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 350–359. [25] Shivam Garg, Dimitris Tsipras, Percy S Liang, and Gregory Valiant. 2022. What can transformers learn in-context? a case study of simple function classes. Advances in Neural Information Processing Systems 35 (2022), 30583–30598. [26] Tianxiao Gu, Chengnian Sun, Xiaoxing Ma, Chun Cao, Chang Xu, Yuan Yao, Qirun Zhang, Jian Lu, and Zhendong Su. 2019. Practical GUI testing of Android applications via model abstraction and refinement. In 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 269–280. [27] Yuyu He, Lei Zhang, Zhemin Yang, Yinzhi Cao, Keke Lian, Shuai Li, Wei Yang, Zhibo Zhang, Min Yang, Yuan Zhang, et al. 2020. TextExerciser: feedback-driven text input exercising for android applications. In 2020 IEEE Symposium on Security and Privacy (SP). IEEE, 1071–1087. [28] Luk Holk, Petr Jank, Anthony W Lin, and Rmmer. 2017. String constraints with concatenation and transducers solved efficiently. Proceedings of the ACM on Programming Languages 2, POPL (2017), 1–32. [29] Yang Hu, Umair Z Ahmed, Sergey Mechtaev, Ben Leong, and Abhik Roychoudhury. 2019. Re factoring based program repair applied to programming assignments. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 388–398. [30] Text input. 2022. Introduction about text input on Android Developer website. https://developer.android.google.cn/reference/android/widget/EditText?hl=en. [31] Nan Jiang, Kevin Liu, Thibaud Lutellier, and Lin Tan. 2023. Impact of Code Language Models on Automated Program Repair. ICSE (2023). [32] Sungmin Kang, Juyeon Yoon, and Shin Yoo. 2023. Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproduction. ICSE (2023). [33] Adam Kiezun, Vijay Ganesh, Shay Artzi, Philip J Guo, Pieter Hooimeijer, and Michael D Ernst. 2013. HAMPI: A solver for word equations over strings, regular expressions, and context-free grammars. ACM Transactions on Software Engineering and Methodology (TOSEM) 21, 4 (2013), 1–28. [34] Sebastian Krings, Joshua Schmidt, Patrick Skowronek, Jannik Dunkelau, and Dierk Ehmke. 2020. Towards constraint logic programming over strings for test data generation. In Declarative Programming and Knowledge Management: Conference on Declarative Programming, DECLARE 2019, Unifying INAP, WLP, and WFLP, Cottbus, Germany, September 9–12, 2019, Revised Selected Papers 22. Springer, 139–159. [35] Misha Laskin, Kimin Lee, Adam Stooke, Lerrel Pinto, Pieter Abbeel, and Aravind Srinivas. 2020. Reinforcement learning with augmented data. Advances in neural information processing systems 33 (2020), 19884–19895. [36] Caroline Lemieux, Jeevana Priya Inala, Shuvendu K Lahiri, and Siddhartha Sen. 2023. CODAMOSA: Escaping Coverage Plateaus in Test Generation with Pretrained Large Language Models. In ICSE. [37] Guodong Li and Indradeep Ghosh. 2013. PASS: String solving with parameterized array and interval automaton. In Hardware and Software: Verification and Testing: 9th International Haifa Verification Conference, HVC 2013, Haifa, Israel, November 5-7, 2013, Proceedings 9. Springer, 15–31. [38] Yuanchun Li, Ziyue Yang, Yao Guo, and Xiangqun Chen. 2017. DroidBot: A Lightweight UI-Guided Test Input Generator for Android (ICSE-C ’17). [39] Yuanchun Li, Ziyue Yang, Yao Guo, and Xiangqun Chen. 2017. Droidbot: a lightweight ui-guided test input generator for android. In 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C). IEEE, 23– 26. [40] Yuanchun Li, Ziyue Yang, Yao Guo, and Xiangqun Chen. 2019. Humanoid: a deep learning-based approach to automated black-box Android app testing. In ASE. IEEE, 1070–1073. [41] Hongliang Liang, Xiaoxiao Pei, Xiaodong Jia, Wuwei Shen, and Jian Zhang. 2018. Fuzzing: State of the art. IEEE Transactions on Reliability 67, 3 (2018), 1199–1218. [42] Tianyi Liang, Andrew Reynolds, Cesare Tinelli, Clark Barrett, and Morgan Deters. 2014. A DPLL (T) theory solver for a theory of strings and regular expressions. In Computer Aided Verification: 26th International Conference, CAV 2014, Held as Part of the Vienna Summer of Logic, VSL 2014, Vienna, Austria, July 18-22, 2014. Proceedings 26. Springer, 646–662. [43] Peng Liu, Xiangyu Zhang, Marco Pistoia, Yunhui Zheng, Manoel Marques, and Lingfei Zeng. 2017. Automatic text input generation for mobile testing. In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). IEEE, 643–653. [44] Zhe Liu, Chunyang Chen, Junjie Wang, Xing Che, Yuekai Huang, Jun Hu, and Qing Wang. 2022. Fill in the Blank: Context-aware Automated Text Input Generation for Mobile GUI Testing. arXiv preprint arXiv:2212.04732 (2022). [45] Zhe Liu, Chunyang Chen, Junjie Wang, Yuekai Huang, Jun Hu, and Qing Wang. 2020. Owl Eyes: Spotting UI Display Issues via Visual Understanding. In ASE. IEEE. https://doi.org/10.1145/3324884.3416547 [46] Zhe Liu, Chunyang Chen, Junjie Wang, Yuekai Huang, Jun Hu, and Qing Wang. 2022. Nighthawk: Fully Automated Localizing UI Display Issues via Visual Understanding. IEEE Transactions on Software Engineering (2022), 1–16. https: //doi.org/10.1109/TSE.2022.3150876 [47] Li Lucy and David Bamman. 2021. Gender and representation bias in GPT-3 generated stories. In Proceedings of the Third Workshop on Narrative Understanding. 48–55. [48] Aravind Machiry, Rohan Tahiliani, and Mayur Naik. 2013. Dynodroid: An input generation system for android apps. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. 224–234. [49] Ke Mao, Mark Harman, and Yue Jia. 2016. Sapienz: Multi-objective automated testing for Android applications. In Proceedings of the 25th International Symposium on Software Testing and Analysis. 94–105. [50] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. Computer Science (2013). [51] Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? arXiv preprint arXiv:2202.12837 (2022). [52] Noor Nashid, Mifta Sintaha, and Ali Mesbah. 2023. Retrieval-Based Prompt Selection for Code Related Few-Shot Learning. In Proceedings of the 45th International Conference on Software Engineering (ICSE’23). [53] Minxue Pan, An Huang, Guoxin Wang, Tian Zhang, and Xuandong Li. 2020. Reinforcement learning based curiosity-driven testing of Android applications. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis. 153–164. [54] Gabriel Poesia, Oleksandr Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Christopher Meek, and Sumit Gulwani. 2022. Synchromesh: Reliable code generation from pre-trained language models. ICLR (2022). [55] Simon Poulding and Robert Feldt. 2017. Generating controllably invalid and atypical inputs for robustness testing. In 2017 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW). IEEE, 81–84. [56] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. J. Mach. Learn. Res. 21 (2020), 140:1–140:67. [57] Vaibhav Rastogi, Yan Chen, and William Enck. 2013. Appsplayground: automatic security analysis of smartphone applications. In Proceedings of the third ACM conference on Data and application security and privacy. 209–220. [58] J Schulman, B Zoph, C Kim, J Hilton, J Menick, J Weng, JFC Uribe, L Fedus, L Metz, M Pokorny, et al. 2022. ChatGPT: Optimizing language models for dialogue. [59] Carolyn B. Seaman. 1999. Qualitative methods in empirical studies of software engineering. IEEE Transactions on software engineering 25, 4 (1999), 557–572. [60] Mike Sharples. 2022. Automated Essay Writing: An AIED Opinion. International Journal of Artificial Intelligence in Education (2022), 1–8. [61] Ting Su, Guozhu Meng, Yuting Chen, Ke Wu, Weiming Yang, Yao Yao, Geguang Pu, Yang Liu, and Zhendong Su. 2017. Guided, stochastic model-based GUI testing of Android apps. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. 245–256. [62] Nezih Sunman, Yiğit Soydan, and Hasan Sözer. 2022. Automated web application testing driven by pre-recorded test cases. Journal of Systems and Software (2022), 111441. [63] Minh-Thai Trinh, Duc-Hiep Chu, and Joxan Jaffar. 2014. S3: A symbolic string solver for vulnerability detection in web applications. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security. 1232–1243. [64] Minh-Thai Trinh, Duc-Hiep Chu, and Joxan Jaffar. 2017. Model counting for recursively-defined strings. In Computer Aided Verification: 29th International Conference, CAV 2017, Heidelberg, Germany, July 24-28, 2017, Proceedings, Part II 30. Springer, 399–418. [65] UIAutomator. 2021. Python wrapper of Android uiautomator test tool. https://github. com/xiaocong/uiautomator. [66] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems (2017). [67] Jue Wang, Yanyan Jiang, Chang Xu, Chun Cao, Xiaoxing Ma, and Jian Lu. 2020. Combodroid: generating high-quality test inputs for android apps via use case combinations. In ICSE. 469–480. [68] Zhengyuan Yang, Zhe Gan, Jianfeng Wang, Xiaowei Hu, Yumao Lu, Zicheng Liu, and Lijuan Wang. 2022. An empirical study of gpt-3 for few-shot knowledgebased vqa. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 3081–3089. [69] Zhengran Zeng, Hanzhuo Tan, Haotian Zhang, Jing Li, Yuqun Zhang, and Lingming Zhang. 2022. An extensive study on pre-trained models for program understanding and generation. In Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis. 39–51. [70] Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui  Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068 (2022).

BYTEDANCE

bytedance.com

10K+ employees

Since 2012

COMPANY RANKING

BYTEDANCE

EVERGREEN INDEX #1646

Representing this company? Click here to claim and customize this page!

ByteDance WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

BYTEDANCE

bytedance.com

10K+ employees

Since 2012

COMPANY RANKING

BYTEDANCE

EVERGREEN INDEX #1646

RELATED COMPANIESRANK

STORIES ABOUT BYTEDANCE

Representing this company? Click here to claim and customize this page!

ByteDance WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!