Newly formed Chinese artificial intelligence (AI) company Baichuan has raised $300m in funding, reported Reuters. The financing round was joined by technology giants Tencent and Alibaba. Baichuan was set up in April 2023 by Wang Xiaochuan, who founded Sogou, an internet search engine.

Chinese AI startup Baichuan raises $300m in funding

To-do, which is available as a WeChat mini program and iOS app, will cease operations on Dec. 20, the Shenzhen-based firm said yesterday.

Tencent to Shutter Scheduling App To-do in December

After a security flaw was found in Chinese keyboard app Sogou, we look at the implications for similar apps in China and how even encrypted platforms like Signal are at risk.

How Chinese keyboard apps could potentially put the online security of hundreds of millions in China at risk

The buzz in China, first ignited by the success of OpenAI's ChatGPT almost a year ago, has given rise to what a senior Tencent executive described this month as "war of a hundred models".

China’s AI ‘war of a hundred models’ heads for a shakeout

Tensions between Beijing and Washington have also weighed on the sector, as US dollar funds invest less in early-stage projects and difficulties obtaining AI chips made by the likes of Nvidia start to bite.

As China's AI 'war of a hundred models' gains pace, investors warn of a shakeout

Opinions on which firms will last vary widely. Yuan Hongwei, chair of Shenzhen-based venture capital Z&Y Capital, said she believed that only two to three general-purpose LLMs will end up dominating the market. That is why her firm looks for experienced founders when deciding on which startups to invest in.

China’s craze over generative artificial intelligence has triggered a flurry of product announcements from startups and tech giants on an almost

China’s craze over generative AI has triggered a flurry of product announcements from startups and tech giants Read more at straitstimes.com.

China's AI 'war of a hundred models' heads for a shakeout

Wang Xiaochuan, the founder and CEO of Baichuan Intelligence, said that the 7 billion-parameter Baichuan2-7B has surpassed Meta's open-source large model Llama2-13B in mainstream tasks in Chinese and English. With the release of Baichuan2's open-source large model in China, the era of using Llama 2 as a shared open-source model has passed.

Baichuan Intelligence Claims Its Product Surpasses Meta’s Llama

Hunyuan is now available for enterprises in China to test and build apps via Tencent Cloud Tencent's AI model has a range of functions, including image creation, copywriting and text recognition, that can be applied in multiple industries Chinese video gaming and social media giant Tencent Holdings has launched its foundation artificial intelligence (AI) model called Hunyuan,

Tencent unveils Hunyuan foundation AI model for enterprises as public debut of internet giant’s chatbot remains on hold

Sogou

sogou

Today, most eCommerce businesses have expanded their reach across the world. You might be doing the same or probably planning for the same. But, expansion does not instantly lead to getting conversions. And, when you expand across multiple countries, the challenge you face is to fight the competition on the search engines. That’s because those countries already have their popular eCommerce websites which their local people rely on. How would they come to know about you and your eCommerce store? Obviously, you can run some . But, don’t forget: social media marketing campaigns As of 2019, around 50% of online product search takes place over search engines, as per a report by Statista. Hence, your most important concern is to work on the SEO strategies for websites across various countries. Sometimes, delivering the expected results become challenging because you lack understanding of different cultures, languages, and search engine dominance across those regions. In this article, I will help you with the end-to-end SEO process to ensure your eCommerce website wins over the region-specific eCommerce leaders: SEO practices followed by the region The search engines that dominates that region A fail-proof SEO strategy to implement straightaway So, let’s get started. Product optimization for eCommerce SEO practices across the world Before you start following the SEO practices around the world, optimize your product pages to ensure you see instant results. Recently, I published an with some Dos and Don’ts so that you can directly focus on what’s important and what’s not while optimizing your product pages. eCommerce SEO guide 11 Dos of eCommerce SEO for Product Pages Design the exceptional UX Work on technical SEO Select the right keywords Create relevant product page URLs Create unique & descriptive product page titles Write effective meta description for each product page Write searchable image ALT attributes Create unique and interactive product descriptions Utilize rich snippets Add FAQs Highlight customer reviews 6 Don’ts of eCommerce Product Pages SEO Don’t stuff keywords Avoid copying the product description from the manufacturers’ websites Don’t create too long URLs Never ignore negative feedback Never stop optimizing your product pages Never hesitate in taking help of SEO experts (not necessarily from your team) Once you’re ready with effective and unbeatable product pages with the above eCommerce SEO Dos and Don’ts, you’re all set to focus on SEO practices for the countries around the world. SEO practices in Europe Okay, so before I move on to talk about the SEO practices to be followed for Europe, I would like to share a few things which every search marketer should know about Europe: Europe has 50 countries There are 23 official languages of the European countries that are part of the European Union only There are many other languages of the European countries except for the European Union More than 100 dialect languages are only spoken in Benelux (Belgium, Netherlands, and Luxembourg) Apart from language differences, there are cultural differences that affect the users’ behavior offline as well as online These things about Europe make it extremely challenging for you to consider all of these countries in one group and target them entirely. Also, an on SearchEngineLand explains how languages, dialects, and countries become a huge issue when optimizing websites for Europe. article Search Engines in Europe When planning for SEO practices for Europe, it becomes mandatory to know which search engines are dominating in Europe. According to an by Matthew Capala, Yandex leads the market share in Russia making Google the 2nd best search engine with 45% market share. article But, as per , here is Google’s search engine dominance: Statista This clearly means that while working on SEO strategy for Europe, Google and Yandex both should be considered. SEO strategy for Europe Here, the following things need to be considered when creating SEO strategy and optimizing sites for Europe: Hreflang remains the most important part while working with multilingual websites. Everything from complexities of multilingual websites to content needs to be carefully worked upon You can check out the guidelines from for handling localized versions Google Here is in-depth by Aleyda Solis on Moz on implementing the Hreflang appropriately and without any errors guidance When working with Wordpress websites, you can also have a look at the hreflang guidance by . Yoast SEO practices in Australia What is the difference between creating an SEO strategy for the U.S, U.K, and Australia? Technically speaking, there is no difference because all of these three regions are English speaking ones. The most important concern is- Australians do not have a particular preference for either British English or American English. Though Australian English emerged from British English, there are certain words or phrases that Australian prefers to be used in their day-to-day lives. So, you need to be utterly careful while picking up the right keywords to optimize a website for the Australian audience. Search Engines in Australia Let’s have a look at the search engine market share in Australia from : StatCounter According to this chart, Google has 94.4% of market share followed by Bing (3.58%), MSN (0.72%), DuckDuckGo (0.54%), Yahoo! (0.52%), and Baidu (0.08%) making Google the most dominant search engine in Australia. SEO strategy for Australia Consider the following things while creating an SEO strategy for the Australian websites or audiences: Make sure you use Hreflang because many businesses have multiple iterations of an Australian website. So, whether the website has ccTLD, ensure you implement Hreflangs. Work on writing effective content and here is a that explains how to write Australian English content. small article For keyword research, use the keyword research tools from SEMrush, Ahrefs, Moz, and Wordstream. Use meta content-language tags and XDefault tags SEO practices in China Creating an SEO strategy for Chinese websites is the most difficult thing to do for the search marketers across the world. The challenges you face while doing SEO for a Chinese website are- Language Search Engine When dealing with the language, you need to use language translators such as Google Translate in order to understand what the website is all about. Coming to another challenge which is the search engine. China’s internet population has grown to more than 800 million, reports . Statista With such a huge Internet population, China has literally blocked Google as a search engine since the year 2010. So, let’s check out which search engines are trending in China. Search Engines in China China’s most popular search engine is Baidu and here is the market share of all the search engines in China by : StatCounter The search engine market share of Baidu is 70.3%, Shenma 15.62%, Sogou 4.74%, Haosou 4.54%, Google 2.57%, and Bing 2.01%. Google is also planning to enter China. But, until then, you need to create an SEO strategy that is effective for Chinese search engines. SEO strategy for China Before I discuss the SEO strategy for China, let’s discuss how optimizing a website for Baidu is different than optimizing for Google. Baidu does not include a new website until a deep examination on the website content while Google is faster. Baidu works on strict censorship rules and only indexes the websites that adhere to those rules while Google shows anything and everything on the world wide web except for the ads that are illegal. Baidu gives high preference for external links that have relevance as well as a huge number of external links while Google checks the relevance between the external links and content. Baidu gives ranking preferences to the top level page in the hierarchy on websites which is why Chinese website homepages have too much content. Google prefers the first level page to rank higher. There is nothing like Hreflang for Baidu and the website should avoid having flash and javascript. Baidu considers meta descriptions and meta keywords as the ranking factors Baidu gives different ranking weight to canonicals, page titles, metadata, and H1s. Here is an SEO strategy for China: Avoid Flash and JavaScript in your website Have a Chinese domain Add keywords naturally in Title, meta description, image ALT attributes Make sure your H1 tag matches with your title tag Keep posting new content on your website and strictly avoid duplicate content Use appropriate keywords while doing internal linking Optimize your website for page loading speed Ensure your website uses guidelines offered by the Baidu Webmaster Tools For detailed technical and on-page SEO guide for Baidu, check out by Simon Lesser. this guide SEO practices in Russia Russia is considered to be the world’s largest nation according to and its official language is Russian. You would be surprised to know that Russian is the eight-most spoken language in the world by a number of native speakers also by . Wikipedia Wikipedia So, just like China, there are no language differences which needs to be taken care of. But, there is definitely a change of how you optimize a website for Russian businesses. What is that change? Let’s figure out now. Search Engines in Russia The change is in the search engine. Yes, just like China, Russia has less of Google. It’s Yandex search engine ruling in the Russian Federation. To learn why Yandex is beating Google in Russia, here is a shared by CNBC. video Also, Yandex filed against Google for giving the search engine pre-installed in the Android phones. Due to this, Google started giving a ‘choice of window’ where the user is given a choice to select their own search engine as default. In short, Yandex is dominating the search engine market in Russia. SEO strategy for Russia When it comes to preparing the SEO strategy for Russian websites, the technical SEO remains similar to Baidu. The only difference is that Yandex gives a huge importance to the good quality content. Yandex also has strict algorithms that ban a website for the over usage of the keywords. Let’s compare Yandex with Google for optimizing websites for SEO: Yandex uses algorithms that are not similar to Google’s standard algorithms This is the reason why it becomes much easier to rank a website for Yandex compared to Google A website can have longer page titles to rank on Yandex Yandex uses meta keywords tag in which you can have around 4 to 5 highly relevant keywords You can put keywords in the URL as Yandex prefers it Yandex takes more time to index a website but once it indexes an appropriately optimized website has huge benefits So, when optimizing a Russian website, make sure you take care of everything from technical SEO to on-page SEO. It is easier to rank on Yandex compared to Google. Here are a few guides that would be helpful for you to optimize a website for Yandex: by Search Engine Journal Yandex SEO Guide by Link Assistant Yandex SEO Tips SEO practices in Japan Japan is an island country in East Asia where Japanese is their national language. So, when considering the language while optimizing the website for SEO, you need to consider the four writing styles- Latin, Kanjo, Hirangana, and Katakana. So, writing a content, meta tags, title tags, heading tags, etc. is not as easy as just translating it to the Japanese language. You need to get it done with the help of a Japanese writer because of the language differences it has compared to English. Now, let’s check out which search engines leads in Japan. Search Engines in Japan The search engine market share in Japan by is as below: StatCounter 74.4% of the search engine market share is with Google followed by Yahoo! with 22.36% which gives Google a good competition. Well, though Google has huge market share, Yahoo! Japan still remains popular among the Japanese. That means you cannot avoid optimizing a website for even Yahoo! Japan. SEO Strategy for Japan So, now while creating an SEO strategy for Japan, you need to make sure you consider the guidelines of Yahoo! Japan as well. Here is an SEO strategy for optimizing a website for both- Google & Yahoo! Japan: Technical SEO remains the same for both the search engines as Yahoo! uses Google’s backend algorithm For on-page SEO you need to consider looking at both- Google search console and Yahoo webmaster tools For on-page SEO you need to consider using more of Latin words while optimizing a website for Japan Here are a few resources that provide good knowledge of doing SEO for Japan: SEO guide for Japan Yahoo SEO optimization SEO around the world Every person looking for information come to the search engines. But, how a particular search engine brings the right information to the user differs across the world is what you need to know. So, before you begin to create an eCommerce SEO strategy for any particular country, you need to know the following three things- The language being spoken The search engine dominance The search behaviors and user expectation So, make sure you keep all these SEO practices in mind while crafting SEO strategies for the respective countries mentioned above and outplay the dominance of the existing eCommerce market leaders.

Ecommerce SEO Practices for Europe, Australia, China, Russia and Japan

On real world data, Deep Learning performance can be shallow I often get pitched with a superior solution for Natural Language Understanding ( ). The plan appears prudent. After all, . A better NLU AI entices many useful advancements, ranging from and to , with an ultimate promise of better language comprehension. deep learning NLU deep learning is the disruptive new force in AI smarter chat bots virtual assistants news categorization State of the Practice Lets assume this superior deep learning (DL) "product” is called . Their pitch deck will invariably have a bar chart that looks something like this — the claim being that the new DL topic classifier/tagger of is better than state of the art methods. "(dot)AI” (Dot)AI In many industries, it is expected that production grade ML classifiers have more than 90% accuracy for quality assurance and a decent user experience. This is expected tolerance level for news categorization or conversational bots The chart presents an interesting proposition, even though performance is only slightly superior than state of the art. In any product, specific to that industry. For example, a model’s best accuracy score might be reasonable for video recommenders or image transcription, but is outside the tolerance limits for news categorization. what constitutes “good enough” depends on tolerance for error In the realm of natural language text classification, do DL techniques significantly outperform shallow methods, e.g or bag of words (BoW) based approaches? You don’t have to be a sceptic to ask the question: TF-IDF The assumption often is a confident Yes — that DL obliterates shallow methods in NLU. Three recent trends underpin this illusion: But does it? In industry AI conferences, , with almost zero talks on production level natural language tasks. Why? deep learning talks overwhelming relate to image/audio/video data The media and others continuously hype deep learning as , . This can lead to confusion for practitioners trying to evaluate DL’s utility in their domain. a silver bullet without perusing the actual results in papers A lot of results just on some artificial benchmark, whereas robustness and applicability matter more. squeeze a few percent of performance While DL has taken the computing world by storm, its impact on certain fundamental NLU tasks remains uncertain and performance is not always superior. To understand why, let me first describe and then the trying to solve it and the NLU task state of the art models how DL underperforms. A Fundamental NLU Task A critical task in natural language understanding is to comprehend the topic of a sentence. The topic could be a tag (such as , , , or ), but usually it merely such as a person’s name or extracting location. politics music gaming immigration adventure-sports isn’t a named entity based task A topic tagger will attempt to tag the first WFTV article to “ ”, while the second WFTV article to “ ” although both mentions `Tiger`. This can get complicated quickly due to things like , as is shown in the . sports animals word sense disambiguation example on the right This type of software is called topic taggers. Their utility cannot be overstated. Topics are key in and Consider — the most common problem bot companies face is the lack of any automated way to capture what their users are messaging about. The only way to estimate user intent from bot messages is either via human eye-balling or whatever matches pre-built regex scripts. Both methods are suboptimal and cannot cover larger topics space. extracting intent formulating automated responses. chat bots In fact, topic tagging at various semantic resolutions is a to NLU for two reasons: (1) Text classification into topics is a precursor to most higher level NLU tasks such as sentiment detection, discourse analysis, episodic memory, and even question answering ( ) (2) Also, NLP pipelines are considerably prone to error propagation, i.e. an error in topic classification can jeopardize future analysis, such as or discourse or even sentiment analysis. Thus, finding the right topic is crucial for NLU. gateway solution approach the quintessential NLU task episodic memory modeling What good is sentiment of a piece of news unless we know what exactly this sentiment is about? Incorrect topic tagging can adversely affect sentiment utility. In reality, topic classification is a hard problem , which has been at times underestimated and overlooked by the AI community. Comprehending the topic is a first step in taking meaningful action. State of the Art Over the years, several technologies have tried to tackle the topic classification problem. There was , and others like , etc. Half of these are either or . The other half has or needs of whatever it outputs. LSA LDA PLSI Explicit Semantic Analysis not production grade don’t scale well with messy real-world data poor interpretability considerable post-processing Today, two main solutions appear overwhelmingly in topic classification performance comparisons. (1) First is the very deep C [ ] model from 2016, which proposes the use of very deep neural network architecture - a “ . (2) Second is the approach (also 2016). Its performance is almost as good as but is orders of magnitude faster in training and evaluation than . Some call FastText the — whatever that means. New world models: onvolutional Neural Net DCNN state of the art in computer vision” [FastText](https://arxiv.org/abs/1607.01759) DCNN DCNN Tesla of NLP Both methods are elegant in their own way. The big difference is that whereas does not fall in the “stereotype” of fancy deep neural nets. Instead it uses word embeddings to solve the tag prediction task. FastText is a shallow network DCNN is 29 layers deep neural net. FastText FastText extends the basic idea to predict a topic label, instead of predicting the middle/missing word (which recall is the original task). This visualizes Word2Vec word embeddings [ ] word embedding Word2Vec link Holding the ground for older/naive models are n-gram/bag of word based models and TFIDF, which still find value in large-scale implementations. Old world models: A final component in examination of state of the art is datasets on which these models are tested. Benchmark datasets are key for reproducibility and comparative analysis. In topic classification tasks, three popular datasets are: , and . They differ in corpus size and number of topics (classes) present in the data. Benchmark Data : [AG news](http://www.di.unipi.it/~gulli/AG_corpus_of_news_articles.html) Sogou news Yahoo! answers The three datasets marked in red rectangles are specifically used for topic classification. Shown with arrow is one instance in the dataset. Task is to predict the label by analyzing the sample. Deep (Learning) Impact? First, lets look at the results from the DL paper and compare it with naive models. The numbers below indicate error rates when running a particular configuration of a model on the topic classification datasets. DCNN This is Table 4 from . Topic classification datasets (mentioned above) are marked with rectangles. The corresponding comparable error values are marked with red ellipses. [DCNN] paper main observations here: Four In 2 /3 topic classification datasets ( + ), the . AG Sogou naive/shallow methods perform better than deep learning In the 3rd dataset ( ), . Yah. Ans DL reduces the error by just **~1.63** The , which is non-trivial and significantly lower than the tolerance level for most quality production systems. accuracy of the best model on ( **Yah. Ans** ) dataset is still at **~73** An important thing to note: all 3 datasets have a topic space less than , which is still somewhat synthetic. In real world natural language data (news streams or conversational messaging), topic spaces could easily exceed 20 or 25 different topics (or intents). This is key, because the next point hints can have huge impact on accuracy. 11 topic space cardinality 4. Notice when the topic space grows from 4 to 10 ( vs. ), While its possible this is caused by spurious factors, such as imbalanced datasets, there is a good chance that a Accuracy Degradation: AG Yah.Ans the error skyrockets to _28.26_ from _7.64_ with the same model. four-fold increase in error is due to complexities involved in generalizing larger topic spaces. Finally, lets look at performance on these datasets and compare it to DL and the naive approaches: FastText DCNN This is Table 1 from showing accuracy values on the three topic classification datasets, comparing FastText with naive methods. FastText paper observations with comparative results : Three further FastText’s 5. Once again, in 2/3 datasets model. In dataset, is only inferior by . FastText performs better than deep learning Yah. Ans FastText ~1.1 6. The in the first two datasets ( and ). DCNN deep learning method actually performs worse than naive models AG Sogou 7. And again, in 2/3 datasets, the naive model’s performance is comparable or better than FastText In addition to these (stunning) results, recall that non-DL models are usually orders of magnitude and faster to train much much more interpretable. Why is this U_nreasonable_? Well, looks like when it comes to topic classifiers — the old world models (naive / shallower) aren’t ready to give up their throne just yet! This It is counterintuitive, given the new world DL models were produced at a company with tons of data, performance should be significantly better. ineffectiveness of deep learning is somewhat unexpected. However, we observe little difference in accuracy. Naive/older models are better or comparable to DL models when classifying text into topics. From “ ”, performance beats older algorithms with sufficient data. What Data Scientists should know about Deep Learning Deep learning might have deep problems in classifying language, but the objective here isn’t to disparage it or have anything to do with a . I think its impact is clear and promising. In computer vision and and , DNN’s have taken us where we have never been before. deep learning conspiracy speech recognition playing games But the reality is your mileage may vary when using deep learning for a basic Natural Language task like text classification. Why this gap in performance between image/video/audio vs. language data? Perhaps it has to do with the patterns of biological signal processing required to “perceive” the former while the patterns of cultural context required to “comprehend” the latter? In any case, there is still much we have to learn about the intricacies of learning itself, especially with different forms of multimedia.

The Unreasonable Ineffectiveness of Deep Learning in NLU

HackerNoon Reporter: Please tell us briefly about your background. I always wanted to start my own business, which was inspired by my grandparents, who were entrepreneurs themselves. Running a business in the technology space would mean impacting millions around the world. That said, I knew that I needed to get some practical experience in a company first before starting my own thing. When I graduated from university, one of the pillars powering some of the most exciting tech companies was advertising. That’s why I started my career in the ad-tech industry when I was 23 years old, and two years later, was given the opportunity to expand and lead that company into Southeast Asia. In 2016, I co-founded AnyMind Group. What's your startup called? And in a sentence or two, what does it do? At AnyMind Group, our mission is to make every business borderless. This means that we create software that forms the next-generation business infrastructure for the digital economy. What is the origin story? When I was in the university more than a decade ago, I traveled to various regions in Asia including India and some parts of Southeast Asia, and saw there was huge potential in the region stemming from two factors: highly enterprising and talented populations in various countries and a region primed for technological disruption. When we started, we were called AdAsia Holdings and took the marketing industry in Asia by storm - providing various software for digital marketing, influencer marketing, and publisher monetization. Today, we’re powering over a thousand publishers and serve over 16 billion impressions a month through the AnyManager platform, and our influencer marketing platform has a daily data update cycle from more than 200,000 influencers, and so much more. However, we realized that we could help our customer base of marketers and businesses, influencers and online publishers, in so many other ways. That’s why we expanded into AnyMind Group in January 2018. Over the years, we have added new software and verticals to our business, including cloud manufacturing, e-commerce enablement, and logistics. We’re also working on more exciting projects. Since the start, we have been constantly innovating, and we are always looking to bring new value to our customers. Today, the business is operating across 13 markets: from Tokyo and Singapore to Dubai and India, and everywhere in between. We have a diverse team of engineers from more than 15 nationalities, with engineering hubs in Tokyo, Bangkok, Ho Chi Minh City, and Mumbai. What do you love about your team, and why are you the ones to solve this problem? I really love that the team is very open-minded and collaborative. This is important because we have a wide variety of products, and we always have very open discussions about new innovations and ideas. There are products in the market now that started as ideas over lunch with the team. This also means that the team has the flexibility to work on products for a specific vertical or across a wider range of verticals and learn new things about specific customer segments and industries. True to our mission to make every business borderless, we are constantly developing new software to add new value to our customers and help them in so many ways, and that means being open-minded and collaborative to work on new products and/or further develop existing products. If you weren’t building your startup, what would you be doing? To be very honest, I never really thought about it, as I wanted to build my own startup from a young age. It is literally in my blood, haha. But hypothetically speaking, I would try my hand at being a professional basketball player. At the moment, how do you measure success? What are your core metrics? I would say that one of our core metrics is the number of businesses, creators, and publishers that we have made an impact on around the world. True to our mission, we’re looking to make every business borderless. It’s an ambitious one, but something that we intend to do. Some of the other core metrics are product ones but would be too vast and detailed to explain quickly. In short, we have achieved considerable scale since we started the business, and we’re looking to drive even more scale-like increasing the number of publishers and monthly impressions, increasing the number of influencers signed up to our platform, increasing the number of e-commerce stores connected to our platforms - so that ultimately, our software can provide more accurate recommendations and actions. What’s most exciting about your traction to date? There are two things of equal excitement, I would say. How we’ve continuously grown our product suite, and the number of businesses, creators, and publishers that we’ve made an impact on so far. At present, our software is used by the most exciting brands, most dynamic creators and influencers, and most forward-thinking publishers, and we are constantly delivering new products that can add value to our customers. What technologies are you currently most excited about, and most worried about? And why? The explosion of people using digital means to access the world, and on top of that, how 5G will further connect the world. We’re seeing people operate an entire business through their mobile phones, purchase digital products with just a unit of data signifying their ownership, digital transformations of supply chains, and so much more. I get excited about essentially new technologies, and that’s why we’re constantly developing new products to add greater value to our customers. What drew you to get published on HackerNoon? What do you like most about our platform? I would have to say, and I’m sure you agree, it’s all about the community. The brightest minds in tech publish on this platform, and just-as-bright minds consume content on here. What advice would you give to the 21-year-old version of yourself? The path you’ll take is the right one, but also don’t ever be satisfied with what you currently have. You’ll make an impact on many people and businesses in the future. What is something surprising you've learned this year that your contemporaries would benefit from knowing? As a founder, the tasks that we get running a company can be overwhelming and it’s easy to just focus solely on work and ignore everything else. However, I’ve also learned that it is super important to allocate some time each week to exercise and stay healthy, as it gives me, even more, focus and energy for my work. Vote for AnyMind Group as . the startup of the year, Singapore

Startup Interview with Kosuke Sogo, CEO of AnyMind Group 

Table of Links Abstract


Introduction
Background
Reinforcement Learning
Similar Work


Methodology
DRLAgent Design


Training Procedures


Testing Procedures


Results


SABR Experiments


Conclusions


Appendix A


References REFERENCES Assa, H., C. Kenyon, and H. Zhang. 2021. “Assessing Reinforcement Delta Hedging.” SSRN SSRN 3918375. http://dx.doi.org/10.2139/ssrn.3918375. Black, F., and M. Scholes. 1973. “The Pricing of Options and Corporate Liabilities” 81 (3): 637– 54. https://doi.org/10.1086/260062. Cao, Jay, Jacky Chen, Soroush Farghadani, John Hull, Zissis Poulos, Zeyu Wang, and Jun Yuan. 2023. “Gamma and Vega Hedging Using Deep Distributional Reinforcement Learning.” Frontiers in Artificial Intelligence 6. https://www.frontiersin.org/articles/10.3389/frai.2023.1129370. Cao, Jay, Jacky Chen, John Hull, and Zissis Poulos. 2021. “Deep Hedging of Derivatives Using Reinforcement Learning.” The Journal of Financial Data Science 3 (1): 10–27. https://doi.org/10.3905/jfds.2020.1.052. Chebyshev, P. 1864. “Sur l’interpolation.” Zapiski Akademii Nauk 1 (4): 539–60. Du, Jiayi, Muyang Jin, Petter N. Kolm, Gordon Ritter, Yixuan Wang, and Bofei Zhang. 2020. “Deep Reinforcement Learning for Option Replication and Hedging.” The Journal of Financial Data Science 2 (4): 44–57. https://doi.org/10.3905/jfds.2020.1.045. Fathi, A., and B. Hientzsch. 2023. “A Comparison of Reinforcement Learning and Deep Trajectory Based Stochastic Control Agents for Stepwise Mean-Variance Hedging.” arXiv arXiv:2302.07996. Florescu, Ionuţ, and Cristian Gabriel Pãsãricã. 2009. “A Study about the Existence of the Leverage Effect in Stochastic Volatility Models.” Physica A: Statistical Mechanics and Its Applications 388 (4): 419–32. Giurca, B., and S. Borovkova. 2021. “Delta Hedging of Derivatives Using Deep Reinforcement Learning.” SSRN SSRN 3847272. https://ssrn.com/abstract=3847272. Glau, Kathrin, Micro Mahlstedt, and Christian Potz. 2018. “A New Approach for American Option Pricing: The Dynamic Chebyshev Method.” arXiv. https://doi.org/10.48550/arXiv.1806.05579. Hagan, Patrick, Deep Kumar, Andrew Lesniewski, and Diana Woodward. 2002. “Managing Smile Risk.” Wilmott Magazine 1 (January): 84–108. Halperin, Igor. 2017. “QLBS: Q-Learner in the Black-Scholes(-Merton) Worlds.” The Journal of Derivatives 28 (1): 99–122. https://doi.org/10.3905/jod.2020.1.108. Heston, Steven L. 1993. “A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options.” The Review of Financial Studies 6 (2): 327–43. Hull, John, 1946-. 2012. Options, Futures, and Other Derivatives. Eighth edition. Boston : Prentice Hall, [2012] ©2012. https://search.library.wisc.edu/catalog/9910112878402121. Kolm, Petter N., and Gordon Ritter. 2019. “Dynamic Replication and Hedging: A Reinforcement Learning Approach.” The Journal of Financial Data Science 1 (1): 159–71. https://doi.org/10.3905/jfds.2019.1.1.159. Lillicrap, Timothy, Johnathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015. “Continuous Control with Deep Reinforcement Learning.” arXiv arXiv:1509.02971. https://doi.org/10.48550/arXiv.1509.02971. Lin, Long-Ji. 1992. “Self-Improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching.” Machine Learning 8 (3): 293–321. https://doi.org/10.1007/BF00992699. Liu, Peng. 2023. “A Review on Derivative Hedging Using Reinforcement Learning.” The Journal of Financial Data Science. https://doi.org/10.3905/jfds.2023.1.124. Longstaff, Francis A., and Eduardo S. Schwartz. 2001. “Valuing American Options by Simulation: A Simple Least-Squares Approach.” UCLA: Finance. https://escholarship.org/uc/item/43n1k4jb. Mikkilä, Oskari, and Juho Kanniainen. 2023. “Empirical Deep Hedging.” Quantitative Finance 23 (1): 111–22. https://doi.org/10.1080/14697688.2022.2136037. Mnih, V., K. Kavukcuoglu, David Silver, A. Graves, I. Antonoglou, Daan Wierstra, and M. Riedmiller. 2013. “Playing Atari with Deep Reinforcement Learning.” arXiv arXiv:1312.5602. https://doi.org/10.48550/arXiv.1312.5602. Murray, Phillip, B. Wood, H. Buehler, Magnus Wiese, and Mikko Pakkanen. 2022. “Deep Hedging: Continuous Reinforcement Learning for Hedging of General Portfolios across Multiple Risk Aversions.” In ICAIF ’22: Proceedings of the Third ACM International Conference on AI in Finance, 361–68. Association for Computing Machinery. https://doi.org/10.1145/3533271.3561731. Pham, Uyen, Quoc Luu, and Hien Tran. 2021. “Multi-Agent Reinforcement Learning Approach for Hedging Portfolio Problem.” Soft Computing 25 (12): 7877–85. https://doi.org/10.1007/s00500-021-05801-6. Pickard, Reilly, and Yuri Lawryshyn. 2023. “Deep Reinforcement Learning for Dynamic Stock Option Hedging: A Review.” Mathematics 11 (24). https://doi.org/10.3390/math11244943. Silver, David, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, et al. 2016. “Mastering the Game of Go with Deep Neural Networks and Tree Search.” Nature 529 (7587): 484–89. https://doi.org/10.1038/nature16961. Smith, Leslie N. 2018. “A Disciplined Approach to Neural Network Hyper-Parameters: Part 1 -- Learning Rate, Batch Size, Momentum, and Weight Decay.” https://doi.org/10.48550/arXiv.1803.09820. Sutton, Richard S., and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. A Bradford Book. Watkins, Christopher John Cornish Hellaby. 1989. “Learning from Delayed Rewards.” King’s College. https://www.cs.rhul.ac.uk/~chrisw/new_thesis.pdf. Xiao, B., W. Yao, and X. Zhou. 2021. “Optimal Option Hedging with Policy Gradient.” In 2021 International Conference on Data Mining Workshops (ICDMW), 1112–19. IEEE. https://doi.org/10.1109/ICDMW53433.2021.00145. Xu, Wei, and Bing Dai. 2022. “Delta-Gamma–Like Hedging with Transaction Cost under Reinforcement Learning Technique.” The Journal of Derivatives 29 (5): 60–82. https://doi.org/10.3905/jod.2022.1.156. Zheng, C., J. He, and C. Yang. 2023. “Option Dynamic Hedging Using Reinforcement Learning.” arXiv arXiv:2306.10743. Authors:
(1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada (reilly.pickard@mail.utoronto.ca);
(2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. Table of Links Abstract Introduction
Background
Reinforcement Learning
Similar Work Methodology
DRLAgent Design Training Procedures Testing Procedures Results SABR Experiments Conclusions Appendix A References Abstract Abstract Abstract Introduction Background Reinforcement Learning Similar Work Introduction Background Background Reinforcement Learning Reinforcement Learning Similar Work Similar Work Methodology DRLAgent Design Methodology Methodology DRLAgent Design DRLAgent Design Training Procedures Training Procedures Training Procedures Testing Procedures Testing Procedures Testing Procedures Results Results Results SABR Experiments SABR Experiments SABR Experiments Conclusions Conclusions Conclusions Appendix A Appendix A Appendix A References References References REFERENCES Assa, H., C. Kenyon, and H. Zhang. 2021. “Assessing Reinforcement Delta Hedging.” SSRN SSRN 3918375. http://dx.doi.org/10.2139/ssrn.3918375. Black, F., and M. Scholes. 1973. “The Pricing of Options and Corporate Liabilities” 81 (3): 637– 54. https://doi.org/10.1086/260062. Cao, Jay, Jacky Chen, Soroush Farghadani, John Hull, Zissis Poulos, Zeyu Wang, and Jun Yuan. 2023. “Gamma and Vega Hedging Using Deep Distributional Reinforcement Learning.” Frontiers in Artificial Intelligence 6. https://www.frontiersin.org/articles/10.3389/frai.2023.1129370. Cao, Jay, Jacky Chen, John Hull, and Zissis Poulos. 2021. “Deep Hedging of Derivatives Using Reinforcement Learning.” The Journal of Financial Data Science 3 (1): 10–27. https://doi.org/10.3905/jfds.2020.1.052. Chebyshev, P. 1864. “Sur l’interpolation.” Zapiski Akademii Nauk 1 (4): 539–60. Du, Jiayi, Muyang Jin, Petter N. Kolm, Gordon Ritter, Yixuan Wang, and Bofei Zhang. 2020. “Deep Reinforcement Learning for Option Replication and Hedging.” The Journal of Financial Data Science 2 (4): 44–57. https://doi.org/10.3905/jfds.2020.1.045. Fathi, A., and B. Hientzsch. 2023. “A Comparison of Reinforcement Learning and Deep Trajectory Based Stochastic Control Agents for Stepwise Mean-Variance Hedging.” arXiv arXiv:2302.07996. Florescu, Ionuţ, and Cristian Gabriel Pãsãricã. 2009. “A Study about the Existence of the Leverage Effect in Stochastic Volatility Models.” Physica A: Statistical Mechanics and Its Applications 388 (4): 419–32. Giurca, B., and S. Borovkova. 2021. “Delta Hedging of Derivatives Using Deep Reinforcement Learning.” SSRN SSRN 3847272. https://ssrn.com/abstract=3847272. Glau, Kathrin, Micro Mahlstedt, and Christian Potz. 2018. “A New Approach for American Option Pricing: The Dynamic Chebyshev Method.” arXiv. https://doi.org/10.48550/arXiv.1806.05579. Hagan, Patrick, Deep Kumar, Andrew Lesniewski, and Diana Woodward. 2002. “Managing Smile Risk.” Wilmott Magazine 1 (January): 84–108. Halperin, Igor. 2017. “QLBS: Q-Learner in the Black-Scholes(-Merton) Worlds.” The Journal of Derivatives 28 (1): 99–122. https://doi.org/10.3905/jod.2020.1.108. Heston, Steven L. 1993. “A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options.” The Review of Financial Studies 6 (2): 327–43. Hull, John, 1946-. 2012. Options, Futures, and Other Derivatives. Eighth edition. Boston : Prentice Hall, [2012] ©2012. https://search.library.wisc.edu/catalog/9910112878402121. Kolm, Petter N., and Gordon Ritter. 2019. “Dynamic Replication and Hedging: A Reinforcement Learning Approach.” The Journal of Financial Data Science 1 (1): 159–71. https://doi.org/10.3905/jfds.2019.1.1.159. Lillicrap, Timothy, Johnathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. 2015. “Continuous Control with Deep Reinforcement Learning.” arXiv arXiv:1509.02971. https://doi.org/10.48550/arXiv.1509.02971. Lin, Long-Ji. 1992. “Self-Improving Reactive Agents Based on Reinforcement Learning, Planning and Teaching.” Machine Learning 8 (3): 293–321. https://doi.org/10.1007/BF00992699. Liu, Peng. 2023. “A Review on Derivative Hedging Using Reinforcement Learning.” The Journal of Financial Data Science. https://doi.org/10.3905/jfds.2023.1.124. Longstaff, Francis A., and Eduardo S. Schwartz. 2001. “Valuing American Options by Simulation: A Simple Least-Squares Approach.” UCLA: Finance. https://escholarship.org/uc/item/43n1k4jb. Mikkilä, Oskari, and Juho Kanniainen. 2023. “Empirical Deep Hedging.” Quantitative Finance 23 (1): 111–22. https://doi.org/10.1080/14697688.2022.2136037. Mnih, V., K. Kavukcuoglu, David Silver, A. Graves, I. Antonoglou, Daan Wierstra, and M. Riedmiller. 2013. “Playing Atari with Deep Reinforcement Learning.” arXiv arXiv:1312.5602. https://doi.org/10.48550/arXiv.1312.5602. Murray, Phillip, B. Wood, H. Buehler, Magnus Wiese, and Mikko Pakkanen. 2022. “Deep Hedging: Continuous Reinforcement Learning for Hedging of General Portfolios across Multiple Risk Aversions.” In ICAIF ’22: Proceedings of the Third ACM International Conference on AI in Finance, 361–68. Association for Computing Machinery. https://doi.org/10.1145/3533271.3561731. Pham, Uyen, Quoc Luu, and Hien Tran. 2021. “Multi-Agent Reinforcement Learning Approach for Hedging Portfolio Problem.” Soft Computing 25 (12): 7877–85. https://doi.org/10.1007/s00500-021-05801-6. Pickard, Reilly, and Yuri Lawryshyn. 2023. “Deep Reinforcement Learning for Dynamic Stock Option Hedging: A Review.” Mathematics 11 (24). https://doi.org/10.3390/math11244943. Silver, David, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, et al. 2016. “Mastering the Game of Go with Deep Neural Networks and Tree Search.” Nature 529 (7587): 484–89. https://doi.org/10.1038/nature16961. Smith, Leslie N. 2018. “A Disciplined Approach to Neural Network Hyper-Parameters: Part 1 -- Learning Rate, Batch Size, Momentum, and Weight Decay.” https://doi.org/10.48550/arXiv.1803.09820. Sutton, Richard S., and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. A Bradford Book. Watkins, Christopher John Cornish Hellaby. 1989. “Learning from Delayed Rewards.” King’s College. https://www.cs.rhul.ac.uk/~chrisw/new_thesis.pdf. Xiao, B., W. Yao, and X. Zhou. 2021. “Optimal Option Hedging with Policy Gradient.” In 2021 International Conference on Data Mining Workshops (ICDMW), 1112–19. IEEE. https://doi.org/10.1109/ICDMW53433.2021.00145. Xu, Wei, and Bing Dai. 2022. “Delta-Gamma–Like Hedging with Transaction Cost under Reinforcement Learning Technique.” The Journal of Derivatives 29 (5): 60–82. https://doi.org/10.3905/jod.2022.1.156. Zheng, C., J. He, and C. Yang. 2023. “Option Dynamic Hedging Using Reinforcement Learning.” arXiv arXiv:2306.10743. Authors: (1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada (reilly.pickard@mail.utoronto.ca); (2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. Authors: Authors: (1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada ( reilly.pickard@mail.utoronto.ca ); reilly.pickard@mail.utoronto.ca (2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. available on arxiv available on arxiv

Hedging American Put Options with Deep Reinforcement Learning: References

A New Framework for Trustworthy AI Deductive Reasoning

This paper is under CC 4.0 license. available on arxiv Authors: (1) Joel Jang, CarperAI,University of Washington & Allen Institute for AI; (2) Seungone Kim, KAIST AI; (3) Yizhong Wang, University of Washington; (4) Jack Hessel, University of Washington; (5) Luke Zettlemoyer, Aleph Alpha; (6) Hannaneh Hajishirzi, University of Washington & Allen Institute for AI; (7) Yejin Choi, UC San Diego. 5 CONCLUSION Previous work has shown that adapting LLMs with RLHF helps them generate outputs that are preferred by humans over the supervised fine-tuned counterpart. However, recent work has also pointed out that simply training LLMs to abide by the preference of the general may result in ignoring individual preferences and values. In this work, we provide the first steps to tackle this issue by proposing Reinforcement Learning from Personalized Human Feedback as a multi-objective problem so that LLMs can be aligned to follow conflicting preferences. We propose a promising method called P-SOUPS that is able to composite models trained on single objectives on the fly during inference. We also highlight the scalability of P-SOUPS by showing that it scales linearly, instead of exponentially like the MORL baseline, with regards to the number of new preferences, which is required to provide true personalization to individual users. ACKNOWLEDGMENTS Thanks to Minyoung Hwang, Sungdong Kim, Tim Dettmers, Yoonjoo Lee, and Margaret Li for helpful feedback and discussion. REFERENCES Xiang Ao, Xiting Wang, Ling Luo, Ying Qiao, Qing He, and Xing Xie. PENS: A dataset and generic framework for personalized news headline generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 82–92, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.acl-long.7. URL https: //aclanthology.org/2021.acl-long.7. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, T. J. Henighan, Nicholas Joseph, Saurav Kadavath, John Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom B. Brown, Jack Clark, Sam McCandlish, Christopher Olah, Benjamin Mann, and Jared Kaplan. Training a helpful and harmless assistant with reinforcement learning from human feedback. ArXiv, abs/2204.05862, 2022a. URL https://api.semanticscholar.org/ CorpusID:248118878. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073, 2022b. Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jer´ emy Scheurer, Javier ´ Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, et al. Open problems and fundamental limitations of reinforcement learning from human feedback. arXiv preprint arXiv:2307.15217, 2023. Deborah Cohen, Moonkyung Ryu, Yinlam Chow, Orgad Keller, Ido Greenberg, Avinatan Hassidim, Michael Fink, Yossi Matias, Idan Szpektor, Craig Boutilier, et al. Dynamic planning in openended dialogue using reinforcement learning. arXiv preprint arXiv:2208.02294, 2022. Shachar Don-Yehiya, Elad Venezian, Colin Raffel, Noam Slonim, Yoav Katz, and Leshem Choshen. Cold fusion: Collaborative descent for distributed multitask finetuning. arXiv preprint arXiv:2212.01378, 2022. Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpacafarm: A simulation framework for methods that learn from human feedback. arXiv preprint arXiv:2305.14387, 2023. Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair nlp models. arXiv preprint arXiv:2305.08283, 2023. Xinyang Geng, Arnav Gudibande, Hao Liu, Eric Wallace, Pieter Abbeel, Sergey Levine, and Dawn Song. Koala: A dialogue model for academic research. Blog post, April, 1, 2023. Conor F Hayes, Roxana Radulescu, Eugenio Bargiacchi, Johan K ˘ allstr ¨ om, Matthew Macfarlane, ¨ Mathieu Reymond, Timothy Verstraeten, Luisa M Zintgraf, Richard Dazeley, Fredrik Heintz, et al. A practical guide to multi-objective reinforcement learning and planning. Autonomous Agents and Multi-Agent Systems, 36(1):26, 2022. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum? id=nZeVKeeFYf9. Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du, and Min Lin. Lorahub: Efficient cross-task generalization via dynamic lora composition. ArXiv, abs/2307.13269, 2023. URL https://api.semanticscholar.org/CorpusID:260155012. Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. ICLR, 2022. Joel Jang, Seungone Kim, Seonghyeon Ye, Doyoung Kim, Lajanugen Logeswaran, Moontae Lee, Kyungjae Lee, and Minjoon Seo. Exploring the benefits of training expert language models over instruction tuning. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 14702–14729. PMLR, 23–29 Jul 2023. URL https://proceedings.mlr.press/v202/jang23a. html. Sungdong Kim, Sanghwan Bae, Jamin Shin, Soyoung Kang, Donghyun Kwak, Kang Min Yoo, and Minjoon Seo. Aligning large language models through synthetic feedback. arXiv preprint arXiv:2305.13735, 2023. Hannah Rose Kirk, Bertie Vidgen, Paul Rottger, and Scott A Hale. Personalisation within bounds: A ¨ risk taxonomy and policy framework for the alignment of large language models with personalised feedback. arXiv preprint arXiv:2303.05453, 2023. Noriyuki Kojima, Alane Suhr, and Yoav Artzi. Continual learning for grounded instruction generation by observing human following behavior. Transactions of the Association for Computational Linguistics, 9:1303–1319, 2021. Margaret Li, Suchin Gururangan, Tim Dettmers, Mike Lewis, Tim Althoff, Noah A Smith, and Luke Zettlemoyer. Branch-train-merge: Embarrassingly parallel training of expert language models. arXiv preprint arXiv:2208.03306, 2022. Pan Li and Alexander Tuzhilin. Towards controllable and personalized review generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3237–3245, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D19-1319. URL https://aclanthology.org/D19-1319. Bodhisattwa Prasad Majumder, Shuyang Li, Jianmo Ni, and Julian McAuley. Generating personalized recipes from historical user preferences. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 5976–5982, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D19-1613. URL https://aclanthology.org/D19-1613. Pierre-Emmanuel Mazare, Samuel Humeau, Martin Raison, and Antoine Bordes. Training mil- ´ lions of personalized dialogue agents. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2775–2779, Brussels, Belgium, OctoberNovember 2018. Association for Computational Linguistics. doi: 10.18653/v1/D18-1298. URL https://aclanthology.org/D18-1298. Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, Francis Song, Martin Chadwick, Mia Glaese, Susannah Young, Lucy Campbell-Gillingham, Geoffrey Irving, et al. Teaching language models to support answers with verified quotes. arXiv preprint arXiv:2203.11147, 2022. Reiichiro Nakano, Jacob Hilton, S. Arun Balaji, Jeff Wu, Ouyang Long, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. Webgpt: Browser-assisted question-answering with human feedback. ArXiv, abs/2112.09332, 2021a. URL https://api.semanticscholar.org/CorpusID:245329531. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021b. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke E. Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Francis Christiano, Jan Leike, and Ryan J. Lowe. Training language models to follow instructions with human feedback. ArXiv, abs/2203.02155, 2022a. URL https://api.semanticscholar.org/ CorpusID:246426909. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 27730–27744, 2022b. Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277, 2023. Jonas Pfeiffer, Naman Goyal, Xi Victoria Lin, Xian Li, James Cross, Sebastian Riedel, and Mikel Artetxe. Lifting the curse of multilinguality by pre-training modular transformers. arXiv preprint arXiv:2205.06266, 2022. Alexandre Rame, Guillaume Couairon, Mustafa Shukor, Corentin Dancette, Jean-Baptiste Gaya, Laure Soulier, and Matthieu Cord. Rewarded soups: towards pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards. arXiv preprint arXiv:2306.04488, 2023. Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. Lamp: When large language models meet personalization. arXiv preprint arXiv:2304.11406, 2023. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose opinions do language models reflect? arXiv preprint arXiv:2303.17548, 2023. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. Umer Siddique, Paul Weng, and Matthieu Zimmer. Learning fair policies in multi-objective (deep) reinforcement learning with average and discounted rewards. In International Conference on Machine Learning, pp. 8905–8915. PMLR, 2020. David Silver, Satinder Singh, Doina Precup, and Richard S Sutton. Reward is enough. Artificial Intelligence, 299:103535, 2021. Prasann Singhal, Tanya Goyal, Jiacheng Xu, and Greg Durrett. A long way to go: Investigating length correlations in rlhf. arXiv preprint arXiv:2310.03716, 2023. Feifan Song, Bowen Yu, Minghao Li, Haiyang Yu, Fei Huang, Yongbin Li, and Houfeng Wang. Preference ranking optimization for human alignment. arXiv preprint arXiv:2306.17492, 2023. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008–3021, 2020. Alane Suhr and Yoav Artzi. Continual learning for instruction following from realtime feedback. arXiv preprint arXiv:2212.09710, 2022. Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239, 2022. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothee´ Lacroix, Baptiste Roziere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and ` efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. Peter Vamplew, Richard Dazeley, Cameron Foale, Sally Firmin, and Jane Mummery. Humanaligned artificial intelligence is a multiobjective problem. Ethics and Information Technology, 20:27–40, 2018. Kristof Van Moffaert and Ann Nowe. Multi-objective reinforcement learning using sets of pareto ´ dominating policies. The Journal of Machine Learning Research, 15(1):3483–3512, 2014. Kristof Van Moffaert, Madalina M Drugan, and Ann Nowe. Scalarized multi-objective reinforce- ´ ment learning: Novel design techniques. In 2013 IEEE symposium on adaptive dynamic programming and reinforcement learning (ADPRL), pp. 191–199. IEEE, 2013. Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A Smith, Iz Beltagy, et al. How far can camels go? exploring the state of instruction tuning on open resources. arXiv preprint arXiv:2306.04751, 2023. Mitchell Wortsman, Suchin Gururangan, Shen Li, Ali Farhadi, Ludwig Schmidt, Michael Rabbat, and Ari S Morcos. lo-fi: distributed fine-tuning without communication. arXiv preprint arXiv:2210.11948, 2022a. Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In International Conference on Machine Learning, pp. 23965–23998. PMLR, 2022b. Jeff Wu, Long Ouyang, Daniel M Ziegler, Nisan Stiennon, Ryan Lowe, Jan Leike, and Paul Christiano. Recursively summarizing books with human feedback. arXiv preprint arXiv:2109.10862, 2021a. Yuwei Wu, Xuezhe Ma, and Diyi Yang. Personalized response generation via generative split memory network. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1956–1970, 2021b. Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A Smith, Mari Ostendorf, and Hannaneh Hajishirzi. Fine-grained human feedback gives better rewards for language model training. arXiv preprint arXiv:2306.01693, 2023. Jie Xu, Yunsheng Tian, Pingchuan Ma, Daniela Rus, Shinjiro Sueda, and Wojciech Matusik. Prediction-guided multi-objective reinforcement learning for continuous robot control. In International conference on machine learning, pp. 10607–10616. PMLR, 2020. Jing Xu, Arthur Szlam, and Jason Weston. Beyond goldfish memory: Long-term open-domain conversation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5180–5197, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.356. URL https: //aclanthology.org/2022.acl-long.356. Runzhe Yang, Xingyuan Sun, and Karthik Narasimhan. A generalized algorithm for multi-objective reinforcement learning and policy adaptation. Advances in neural information processing systems, 32, 2019. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. Personalizing dialogue agents: I have a dog, do you have pets too? arXiv preprint arXiv:1801.07243, 2018. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. arXiv preprint arXiv:2306.05685, 2023. Yinhe Zheng, Guanyi Chen, Minlie Huang, Song Liu, and Xuan Zhu. Personalized dialogue generation with diversified traits. arXiv preprint arXiv:1901.09672, 2019. Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019. A DETAILS OF EVALUATION SETUP We use a modified version of the GPT4 annotation prompt used by Dubois et al. (2023). We modify the criteria to perform the pairwise evaluation from general to a single preference dimension. We also provide 3 demonstrations: one scenario where there is a tie because both responses do not contain any notion of the preference (e.g. both responses do not show any signs of friendliness), one scenario where there is a clear winner, and one scenario where they are both good, but one is better than the other. Simulated pairwise evaluation We recruited 24 crowd workers for our human evaluation. Figure 5 shows the interface used for human evaluation. We consider both the ‘Tie’ and ‘Both are bad’ options to be Ties. B CRITERIA-WISE EVALUATION The criteria-wise win rate (%) across all the methods are shown in Figure 6. The criteria-wise win rate was calculated by getting the average win-rate of the preference combinations that contained the specific preference dimension. For example, when calculating the criteria-wise win rate of ‘Elementary’, we got the average win-rate of the preference combinations that contained the ‘Elementary’ preference, which includes AAA, AAB, ABA, and ABB. C THE FULL LIST OF EVALUATION PROMPTS The full list of evaluation prompts used in our experiments are provided in Table 5 D DETAILED RESULTS FOR HUMAN EVALUATION AND GPT-4 EVALUATION We provide detailed results (win / loss / tie for each of the preference combinations for our main experimental results. Table 6 shows the GPT4 evaluation and Table 7 shows the human evaluation results. E EXAMPLES OF P-SOUPS TEXT GENERATIONS Table 8 shows empirical examples of the text generated from each preference combination of the 16 preference combination experiments for the same prompt.

Personalized Soups: LLM Alignment Via Parameter Merging - Conclusion & References

Dust on Crust Part Deux

ভূত্বক অংশ দুই উপর ধুলো

Staub auf der Kruste Teil Zwei

Polvo sobre la corteza, segunda parte

पपड़ी पर धूल भाग दो

クラスト上のダスト パート 2

지각 위의 먼지 2부

Poeira na Crosta Parte Dois

Пыль на корке, часть вторая

Kabuktaki Toz İkinci Bölüm

Bụi trên lớp vỏ Phần thứ hai

地壳上的灰尘第二部分

Gemini - A Family of Highly Capable Multimodal Models: Discussion and Conclusion, References

240 Stories To Learn About Startups Of The Year

Scientific American Supplement, No. 711, August 17, 1889, by Various, is part of the HackerNoon Books Series. You can jump to any chapter in this book . GIBRALTAR. here GIBRALTAR. The point or rock known as Gibraltar is a promontory two and one-half miles long and from a quarter to three-quarters of a mile wide. It rises abruptly from the sandy shore to a height at its highest point of 1,408 ft. It is composed of gray limestone, honeycombed with caves and subterranean passages, some of which contain most beautiful stalactites in the form of massive pillars. Gibraltar is emphatically a fortress, and in some respects its fortifications are unique. On the eastern side the rock needs no defense beyond its own precipitous cliffs, and in all other directions it has been rendered practically impregnable. Besides a sea wall extending at intervals round the western base of the rock, and strengthened by curtains and bastions and three formidable forts, there are batteries in all available positions from the sea wall up to the summit, 1,350 feet above the sea, and a remarkable series of galleries has been hewn out of the solid face of the rock toward the north and northwest. These galleries have an aggregate length of between two and three miles, and their breadth is sufficient to let a carriage pass. Portholes are cut at intervals of twelve yards, so contrived that the gunners are safe from the shot of any possible assailants. At the end of one of the galleries hollowed out in a prominent part of the cliff is St. George's Hall, 50 feet long by 85 feet wide, in which the governor was accustomed to give fetes. Alterations, extensions, and improvements are continually taking place in the defensive system, and new guns of the most formidable sort are gradually displacing or supplementing the old fashioned ordnance. The whole population of Gibraltar, whether civil or military, is subjected to certain stringent rules. For even a day's sojourn the alien must obtain a pass from the town major, and if he wish to remain longer, a consul or householder must become security for his good behavior. Licenses of residence are granted only for short periods—ten, fifteen, or twenty days—but they can be renewed if occasion require. Military officers may introduce a stranger for thirty days. A special permit is necessary if the visitor wishes to sketch. Though the town of Gibraltar may be said to date from the fourteenth century, it has preserved very little architectural evidence of its antiquity. Rebuilt on an enlarged and improved plan after its almost complete destruction during the great siege, it is still, on the whole, a mean-looking town, with narrow streets and lanes and an incongruous mixture of houses after the English and the Spanish types. As a proprietor may at any moment be called upon to give up his house and ground at the demand of the military authorities, he is naturally deterred from spending his money on substantial or sumptuous erections. The area of the town is about one hundred acres. Gibraltar was known to the Greek and Roman geographers as Calpe or Alybe, the two names being probably corruptions of the same local (perhaps Phenician) word. The eminence on the African coast near Ceuta, which bears the modern English name of Apes' Hill, was then designated Abyla; and Calpe and Abyla, at least according to an ancient and widely current interpretation, formed the renowned pillars of Hercules (Herculis columnæ), which for centuries were the limits of enterprise to the seafaring peoples of the Mediterranean world. The strategic importance of the rock appears to have been first discovered by the Moors, who, when they crossed over from Africa in the eighth century, selected it as the site of a fortress. From their leader, Tarik Ibn Zeyad, it was called Gebel Tarik or Tarik's Hill; and, though the name had a competitor in Gebel af Futah, or Hill of the Entrance, it gradually gained acceptance, and still remains sufficiently recognizable in the corrupted form of the present day. The first siege of the rock was in 1309, when it was taken by Alonzo Perez de Guzman for Ferdinand IV. of Spain, who, in order to attract inhabitants to the spot, offered an asylum to swindlers, thieves, and murderers, and promised to levy no taxes on the import or export of goods. The attack of Ismail Ben Ferez, in 1315 (second siege), was frustrated; but in 1333 Vasco Paez de Meira, having allowed the fortifications and garrison to decay, was obliged to capitulate to Mahomet IV. (third siege). Alphonso's attempts to recover possession (fourth siege) were futile, though pertinacious and heroic, and he was obliged to content himself with a tribute for the rock from Abdul Melek of Granada; but after his successful attack on Algeciras in 1344 he was encouraged to try his fortune again at Gibraltar. In 1349 he invested the rock, but the siege (fifth siege) was brought to an untimely close by his death from the plague in February, 1350. The next or sixth siege resulted simply in the transference of the coveted position from the hands of the King of Morocco to those of Yussef III. of Granada; and the seventh, undertaken by the Spanish Count of Niebla, Enrico de Guzman, proved fatal to the besieger and his forces. In 1462, however, success attended the efforts of Alphonso de Arcos (eighth siege), and in August the rock passed once more under Christian sway. The Duke of Medina Sidonia, a powerful grandee who had assisted in its capture, was anxious to get possession of the fortress, and though Henry IV. at first managed to maintain the claims of the crown, the duke ultimately made good his ambition by force of arms (ninth siege), and in 1469 the king was constrained to declare his son and his heirs perpetual governors of Gibraltar. In 1479 Ferdinand and Isabella made the second duke Marquis of Gibraltar, and in 1492 the third duke, Don Juan, was reluctantly allowed to retain the fortress. At length, in 1501, Garcilaso de la Vega was ordered to take possession of the place in the king's name, and it was formally incorporated with the domains of the crown. After Ferdinand and Isabella were both dead the duke, Don Juan, tried in 1506 to recover possession, and added a tenth to the list of sieges. Thirty-four years afterward the garrison had to defend itself against a much more formidable attack (eleventh siege)—the pirates of Algiers having determined to recover the rock for Mahomet and themselves. The conflict was severe, but resulted in the repulse of the besiegers. After this the Spaniards made great efforts to strengthen the place, and they succeeded so well that throughout Europe Gibraltar was regarded as impregnable. In the course of the war of the Spanish succession, however, it was taken by a combined English and Dutch fleet under Sir George Rooke, assisted by a body of troops under Prince George of Hesse-Darmstadt. The captors had ostensibly fought in the interests of Charles Archduke of Austria (afterward Charles III.), but, though his sovereignty over the rock was proclaimed on July 24, 1704, Sir George Rooke on his own responsibility caused the English flag to be hoisted, and took possession in name of Queen Anne. It is hardly to the honor of England that it was both unprincipled enough to sanction and ratify the occupation and ungrateful enough to leave unrewarded the general to whose unscrupulous patriotism the acquisition was due. The Spaniards keenly felt the injustice done to them, and the inhabitants of the town of Gibraltar in great numbers abandoned their homes rather than recognize the authority of the invaders. In October, 1704, the rock was invested by sea and land; but the Spanish ships were dispersed by Sir John Leake, and the Marquis of Villadarias fared so ill with his forces that he was replaced by Marshal Tesse, who was at length compelled to raise the siege in April, 1705. During the next twenty years there were endless negotiations for the peaceful surrender of the fortress, and in 1726 the Spaniards again appealed to arms. But the Conde de la Torres, who had the chief command, succeeded no better than his predecessors, and the defense of the garrison under General Clayton and the Earl of Portmore was so effectual that the armistice of June 23 practically put a close to the siege, though two years elapsed before the general pacification ensued. The most memorable siege of Gibraltar, indeed one of the most memorable of all sieges, was that which it sustained from the combined land and sea forces of France and Spain during the years 1779-1783. The grand attack on the place was made on the 13th September, 1782, and all the resources of power and science were exhausted by the assailants in the fruitless attempt. On the side of the sea they brought to bear against the fortress forty-six sail of the line and a countless fleet of gun and mortar boats. But their chief hope lay in the floating batteries planned by D'Arcon, an eminent French engineer, and built at the cost of half a million sterling. They were so constructed as to be impenetrable by the red hot shot which it was foreseen the garrison would employ; and such hopes were entertained of their efficiency that they were styled invincible. The Count D'Artois (afterward Charles X.) hastened from Paris to witness the capture of the place. He arrived in time to see the total destruction of the floating batteries and a considerable portion of the combined fleet by the English fire. Despite this disaster, however, the siege continued till brought to a close by the general pacification, February 2, 1783. The history of the four eventful years' siege is fully detailed in the work of Drinkwater, who himself took part in the defense, and in the life of its gallant defender Sir George Augustus Eliott, afterward Lord Heathfield, whose military skill and moral courage place him among the best soldiers and noblest men whom Europe produced during the 18th century. Since 1783 the history of Gibraltar has been comparatively uneventful. In the beginning of 1801 there were rumors of a Spanish and French attack, but the Spanish ships were defeated off Algeciras in June by Admiral Saumarez. Improvements in the fortifications, maintenance of military discipline, and legislation in regard to trade and smuggling are the principal matters of recent interest. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Various (2004). Scientific American Supplement, No. 711, August 17, 1889. Urbana, Illinois: Project Gutenberg. Retrieved https://www.gutenberg.org/cache/epub/16972/pg16972-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

Stories, News & Mentions about SOGOU

Sogou’s Stories on HackerNoon

Sogou's News & Mentions