Duke opens up about his father’s cancer diagnosis and life in America

Prince Harry: I love my family and my father’s cancer diagnosis could bring us together | The Independent

For many, the bonds formed by choice can be as profound, as tortured, as those into which we are born.

Finding Your Family - The New York Times

Duke of Sussex, who flew to UK to see his father, says in US interview he was grateful for time with family

Prince Harry hints he’s open to family reconciliation after king’s cancer diagnosis | Prince Harry | The Guardian

The Beach Boys star will "remain at home" and "continue to work" at his discretion, the family says.

Brian Wilson: Beach Boys star's family seeks conservatorship due to health issues

The prince tells US TV he "jumped on a plane" as soon as possible to see his father following the diagnosis.

Prince Harry: King's cancer may bring family closer together - BBC News

**Job Description:**\n\n* Responsible for generating sales with existing customers and developing opportunities with new customers within assigned territory

Famiry

The NFT world has made millions for lots of investors, while at the same time, it has bankrupted just as many people. There are rumors that NFTs will make a big comeback in 2024 and grab the headlines again. But is that really true? Can we trust these rumors? To find out, we will return to the very beginning - to the peak of NFTs in 2021, we will try to understand the reasons for the collapse of the market in 2022, its gradual recovery in 2023, and give a forecast for the current year. Main Takes: : NFTs experienced a significant surge in 2021, with trading volumes reaching $17.6 billion, driven by unique blockchain technology, the crypto bull run, the COVID-19 pandemic, the decline in ICOs, network effects, and FOMO. 2021 Historical Context : A dramatic correction occurred in 2022, with trading volumes and NFT prices plummeting due to a broader crypto market downturn, highlighted by events such as the collapse of TerraUSD and the bankruptcy of major crypto players. 2022 Market Correction : Despite a sluggish start, the NFT market showed signs of revival in the last quarter of 2023, buoyed by a bullish crypto market and the SEC's approval of Bitcoin ETFs, with weekly trading volumes surpassing $500 million in December. 2023 Revival Signs : The introduction of Bitcoin Ordinals and the growth of Solana contributed to the renewed interest in NFTs, offering unique features and security advantages. Bitcoin Ordinals and Solana's Influence : Positive expectations for the NFT market are supported by a more mature and diverse sector, the broader crypto market's performance, utility in various niches, and mainstream adoption by traditional brands. However, challenges such as wash trading, scams, and regulatory fragmentation remain. 2024 Outlook Why there was an NFT boom in 2021? In 2021, the NFT market experienced an unprecedented surge. At that time, many thought that the interest in non-fungible tokens was confined to a relatively closed group of traders who had little connection to the art world. Soon, auction houses began to offer more NFT sales and create infrastructure and consistent offerings of this kind. Research shows an explosion of interest in NFTs as a new asset class, with sales reaching in 2021. Famous NFTs, like the ones from Beeple, Bored Ape Yacht Club, and CryptoPunks, became iconic. Even big brands like Gucci and D&G joined in, making their own collectibles. $17.6 billion In 2022, the NFT market faced a serious disruption, leaving everyone reeling. The once bustling market took a nosedive in prices, sparking widespread concerns and discussions about its future. NFT crash in 2022 In June 2022, we witnessed a 12-month low. Sales fell to 1 billion dollars, the worst market performance since June 2021 (when sales were a modest 648 million dollars). According to , while the market was worth 3 trillion dollars in November 2021, it's now sporting less than 1 trillion dollars. Chainanalisys In the second quarter of 2022, things took a turn for the worse. The , laid out by the Non-Fungible, reveals that sales volume has decreased by 20.05% from the first quarter of 2022, from 12,639,781 to 10,105,967. The number of sellers has decreased by 35.88% and the number of buyers has decreased by 25.41%. market report for Q2 This data suggested that collectors weren't exactly interested in taking big risks when it came to investing in new and upcoming projects appearing on NFT platforms. They've decided to trade on the secondary market, where prices remained higher. NFTs at one time depreciated by millions of percent, almost to several hundred, having been purchased for hundreds of thousands of dollars. Take Logan Paul, for example. His Azuki NFT, once purchased for a whopping $623,000, is now worth $10. Likewise, Steve Aoki's Doodles NFT has seen its value drop to less than $42,000. Even the stars aren't immune to the NFT tragedy. Snoop Dogg also lost more than $300,000 on CryptoPunk, bought for $1.1 million, which is now worth about $688,000. However, the most popular loss is Justin Bieber's journey with the Bored Ape Yacht Club NFT project. What started as a $1.31 million investment turned into a shocking $59,090. The NFT market has plummeted in 2022, thanks to several factors including market oversaturation, scams, and the collapse of big systems. With a flood of NFT projects and the art of deception on the rise, investor hesitation contributed to the market downturn. First factor. Market oversaturation Before the NFT boom of 2021, the market was relatively niche, with just a handful of NFT projects grabbing attention. But when the hype hit its peak, it felt like everyone wanted a piece of the action. The result? An overwhelming deluge of NFT projects flooding the market.** Second factor. Deception and scams Where large sums of money start circulating, fraudsers immediately appear, eager to profit from it. The NFT market hasn`t been an exception. As interest in NFTs has grown, so had the number of fraudulent activities and scams targeting unsuspecting investors. Hackers and scammers have seized upon the NFT boom, deceiving people and causing them to part with their assets. Discord breaches, malicious links, and the whole range of free NFT coin scams left many people with heavy losses and undermined confidence in the market. Third factor. The collapse of big systems When FTX, one of the largest crypto exchanges, filed for bankruptcy, allegedly due to a liquidity shortage and a disappearing act with the funds. Additionally, the Terra blockchain cryptocurrencies, TerraUSD and LUNA, have seen catastrophic drops in value, leaving investors reeling from significant losses. In any ecosystem, when a big player fails, it can send shakes through the whole system. In the case of the NFT market crash, the collapse of important systems associated with Web3 contributed to the downturn. What happened with the NFT market in 2023? Despite the NFT market crashing in 2022, experts and enthusiasts are pretty sure that NFTs will bounce back to their best state. Even though there was a big drop in trading in June, the NFT market overall stayed strong in the first half of the year. The beginning of 2023 looked good for NFTs, with a sudden increase in trading activity. The launch of trading platforms like LooksRare and X2Y2 played a big part, along with the rewards they offered for trading. NFT sales in 2023 reached around $17.7 billion, similar to the sales recorded in 2021. This is also almost a 10-fold increase compared to the same period last year. Taking a closer look at the main factors, a pretty stable first half of the year was helped by a healthy growth in the number of unique buyers and transactions. But this was balanced out by a drop in the average sale price. Even if we ignore the less valuable and less important projects, most collections nowadays aren't really worth much. Out of the 8,850 collections leading in market value, 18% are not worth anything, and 41% are being sold for prices between $5 and $100. Less than 1% have a price higher than $6,000 – a far cry from the usual million-dollar deals made two years ago. Despite this, the NFT community's stability, combined with ongoing innovations in this area, is paving the way for a potential comeback. Thanks to forward-thinking NFT creators and big brands using this technology, the next bullish rise of NFTs looks promising. Yuga Labs' Role in the NFT Comeback Yuga Labs' groundbreaking games, like Dookey Dash, brought in new collectors and created a fresh wave of excitement in the NFT space. Thanks to these innovative Yuga Labs projects, they became the driving force behind the NFT comeback. Their contribution to a whopping 30% of all Ethereum NFT trades in just one month made them a beacon of hope for the NFT market. As we move forward, their impact will inevitably determine the course of NFTs for many years to come. NFT brands are selling stuff in big regular and online stores. We are seeing the launch of major blockchain-based games. And more big companies are joining the NFT world. As a result, NFT-based brand building could be a significant factor beyond just the recovery but also surpass previous heights in 2024. Will NFTs be reborn in 2024? The NFT market is expected to change a lot in 2024. Despite the prolonged decline and negative background, it's predicted that the NFT market will make around in revenue in 2024. $2,378.0 million In 2024, the NFT market is expected to go through big changes with lots of new trends. Here are some of the important trends. Metaverses The global Metaverse market size is steadily at the expected rate, recording a CAGR of 47.2% from 2023 to 2027. The cost of the metaverse industry will increase from $61.8 billion in 2022 to approximately $426.9 billion by the end of 2027. expected to grow The Metaverse NFT market is huge and includes a variety of digital assets such, as virtual houses, avatars, art, and even virtual fashion. Many things are making this market grow, like more people using blockchain, games, and virtual worlds becoming popular, and the growing demand for unique and scarce digital assets. Additionally, the Metaverse NFT market gives creators, artists, and gamers new chances to make money from their digital creations, and investors can diversify their portfolios by investing in these unique assets. NFT Aggregators In 2022-2023, saw the rise of NFT aggregators. These platforms let traders buy and sell NFTs from different marketplaces, all in one place. Top aggregators Gem and Genie showed impressive growth: this year, the trading volume went over 511,000 ETH. That's about 21 times more than in 2021. NFT aggregators also have extra benefits. They let traders complete whole collections by buying lots of NFTs at the lowest prices, without needing to go to each market separately. The market share of NFT aggregators from just over 1% at the end of 2021 to about 5% today. Even though the market share of NFT aggregators is still not very big, recent purchases of and Genie by Uniswap show strong interest from existing players in NFT aggregators. It confirms that NFT aggregators seem ready to play a big role in shaping the NFT trading scene for the long term. has also grown Gem by OpenSea Entering the era of Web3, the NFT world holds promises of growth, creativity, and financial opportunities. Will NFTs make a big comeback in 2024? It seems like Metaverse and NFT aggregators are ready to contribute significantly to the upcoming success.

Is the Revival of NFTs in 2024 a Reality, or Just Another Fairy Tale for Beginners?

The digital asset industry is not one that is stagnant. Beyond the perennial excitement induced among enthusiasts and users due to the volatile price movements of the assets, the industry is no stranger to eventful occurrences. Most recently, the US SEC (Securities and Exchange Commission) approved the issuance of Bitcoin ETFs (Exchange Traded Funds) for a handful of prominent asset management and financial services companies. The move pushes the crypto industry towards institutionalization, with multiple jurisdictions worldwide already regulating or working on implementing regulations for all kinds of digital assets beyond Bitcoin. As the industry matures and the innovation evoked by the asset class attracts retail and institutional investors alike, its institutionalization is the only logical step ahead. The Good Comes with the Bad – Still, the Industry Grows Beyond those desired happenings, the industry has also been prone to setbacks. For example, the recent Binance debacle saw the company shelling out over due to its negligence in reporting and preventing transactions serving money laundering. Another example is the infamous that led to collective losses in the billions for crypto users due to the blatant mismanagement of funds. $4 billion in fines FTX saga Regulations will ensure such failures do not occur, keeping digital asset investors and users safe from untoward incidents. Nevertheless, despite the setbacks, the development of innovative products from this space remains strong. Digital assets, alongside a vibrant DeFi ecosystem and robust centralized platforms offering asset storage and exchange activities, among others, are booming. Family Offices Are Showing Unwavering Interest in Digital Assets A rapidly evolving digital asset industry is witnessing massive influxes of funds, now more than ever. With the capabilities it offers to build massive wealth and regulatory perceptions shifting towards the positive, one group of investors flocking to the ecosystem is family offices. Servicing HNWIs (High Net Worth Individuals) and wealthy families to build and maintain generational wealth, family offices are diversifying investment strategies to accommodate digital assets. However, their entry into the crypto markets should occur with the assistance of platforms specializing in handling the large value they pour into these assets. Family offices must turn to working to safeguard users’ assets from the various elements in the ecosystem that can lead to their loss or theft. digital asset custodians Custodians Make Digital Asset Usage Safe and Convenient for Family Offices In fact, jurisdictions rolling out regulations for the asset class are mandating the usage of licensed custodians while dealing with large amounts of assets. And that is for good reason. For one, self-custody wallets aimed to secure digital assets are a good choice for retail users dealing with less significant amounts. On the other hand, interacting with such assets in the millions of dollars, as family offices do, can be inconvenient and dangerous when using self-custody storage solutions. The chances of losing private keys and having assets worth millions trapped forever are more common than assumed and obviously not ideal. Turning to experienced custodians, like BitGo, Liminal, and Copper, is the best way for family offices to go. These custodians offer the perfect balance of security and convenience that they can ask for. Licensed custodians utilize state-of-the-art asset storage and data transfer infrastructure, ensuring the digital assets they handle remain safe. Hot and cold wallets protected by Multi-Signature and are commonplace with custodians, alongside infosec certifications attesting to their robust infrastructure. MPC (Multi-Party Computation) technology Increased Access to the DeFi Ecosystem Through Custody Platforms By turning to custodians, can stay assured they possess instant access to their assets, overseen by bank-grade storage security. Of course, asset storage is only one aspect of custodian offerings. Custodian platforms also come equipped to let their users indulge in DeFi strategies across various blockchain networks – all from their platform interfaces. family offices With the DeFi ecosystem being one of the most prominent use cases of digital assets, family offices naturally utilize the assets under their management to earn steady passive income. So, family offices can indulge in multichain DeFi ecosystems, purchasing all kinds of tokens to put to work in applications deployed on prominent blockchains from well-known third-party custody platforms. Custodian Insurance Removes All Worries of Dealing with Third Parties While relying on third parties to hold crypto assets can seem daunting, especially when blockchain tenets dictate doing away with intermediaries, reputed custodians bring unfaltering trust. It is so because licensed custodians come insured to protect their users from worst-case scenarios like hacks, exploits, and bankruptcies. Family offices can confidently employ custodians, knowing their value is recoverable through insurance despite losses incurred due to negligence or bad behavior exhibited by custodians. Reputed Custodians Uphold Compliance Even for Their Clients Numerous jurisdictions with established digital asset regulations enforce insurance on custodians to safeguard investors. They also enforce AML (Anti-Money Laundering) and CFT (Countering the Financing of Terrorism) protocols upon these platforms, which is a need in the digital asset ecosystem. Such protocols prevent family offices from accidentally indulging in prohibited tokens like privacy coins and interacting with sanctioned wallets linked with illicit activities. Tapping into custody services can make sure family offices remain within legal bounds and do not have their operations, funds, and investments jeopardized. Likewise, custodians can even effect good behavior among family offices, mainly multi-family offices handling investments for multiple HNWIs and families. This type of family office is operated by third parties, rather than the traditional single-family office set up and controlled by HNWIs or families themselves. Most jurisdictions have lax regulations over family offices, meaning the clients of multi-family offices will find their value at risk if the third parties misappropriate said value. Custodians storing end of such offices are mandated to track and report suspicious movements of funds by regulators. client digital assets on behalf Malicious activity can easily be detectable when huge sums are misused, protecting investors from bad actors. Thereby, custodians offer a layer of transparency and accountability. Multi-family offices looking to increase their credibility with end clients interested in digital assets must employ custodians. Custody Solutions Are a Necessity for Crypto-Inclined Family Offices Digital asset adoption is in full swing as investors turn to the asset type and the innovative products it is central to. Family offices are the same as these assets, as they have shown their ability to steadily bring returns that magnify wealth, often more effectively than traditional assets. As they take a more influential place in the portfolios of HNWIs and wealthy families, custodians can help alleviate the issues with handling them. Thus, family offices should consider custody-related solutions to traverse the while making the most out of the novel financial system. digital asset landscape securely and conveniently

Why Family Offices Should Care About Digital Asset Custody

When I wrote my first book, , my goal was to create an unconventional programming book with the appearance of a regular storybook, but is actually written in code.  C programming concepts are taught through code explanations. Since a standard picture book has 32 pages, I imposed the requirement on myself to make the book 32 pages, which is funny in retrospect. A Day in Code: An Illustrated Story Written in the C Programming Language Fast forward three years, and I released that has 94 pages. In this book, classic rhymes and stories you’re familiar with are “translated” into full C programs. When you run the programs on your computer, the logic of the programs makes the text of the nursery rhyme or the fairy tale story print out on the console. Programs gradually increase in complexity, with simple nursery rhymes teaching a single topic, followed by more complex fairy tales combining multiple concepts. I think readers’ familiarity with the classic rhymes/stories will help with naturally understanding the logic of the programs. Of course, it’s also illustrated like a picture book to maintain a comforting storybook feel. Learn C Programming through Nursery Rhymes and Fairy Tales Compared to my first book, my objective with this new book was to delve deeper into code explanations, have longer programs, and add a few additional topics—including pointers, which allow you to access memory and directly manipulate it. Pointers are considered an advanced and difficult topic that beginners still must learn, as they are fundamental to the C programing language’s low-level memory control.  Pointers are introduced in a program that tells the story of “The Three Little Pigs”, with each pig’s house being located at a memory address. This book is a follow-up to my previous book, . The “translation” of nursery rhymes and fairy tales into C programs was more challenging and space-consuming than Python. The biggest takeaway is that strings are much easier to work with in Python. was made possible by using constants and variable names representing elements in the rhyme or story. Learn Python through Nursery Rhymes and Fairy Tales Learn C Programming through Nursery Rhymes and Fairy Tales C is Useful As an embedded software engineer, I’m a big proponent of C programming. It’s efficient and fast.  It provides high level logical constructs like and loops, while also being “close to the hardware” by allowing for low level manipulation of memory and bits. It has influenced many other programming languages like Python and Java, so provides a strong foundation for learning other languages. if…else while()

Learn To Code in C with Nursery Rhymes and Fairy Tales

This paper is available on arxiv under CC 4.0 license. . Authors: Gemini Team, Google Table of Links Abstract and Introduction Model Architecture Training Infrastructure Training Dataset Evaluation Responsible Deployment Discussion and Conclusion, References Contributions and Acknowledgments Appendix 9. Appendix 9.1. Chain-of-Thought Comparisons on MMLU benchmark We contrast several chain-of-thought approaches on MMLU and discuss their results in this section. We proposed a new approach where model produces k chain-of-thought samples, selects the majority vote if the model is confident above a threshold, and otherwise defers to the greedy sample choice. The thresholds are optimized for each model based on their validation split performance. The proposed approach is referred to as uncertainty-routed chain-of-thought. The intuition behind this approach is that chain-of-thought samples might degrade performance compared to the maximum-likelihood decision when the model is demonstrably inconsistent. We compare the gains from the proposed approach on both Gemini Ultra and GPT-4 in Figure 7. We find that Gemini Ultra benefits more from this approach compared to using only chain-of-thought samples. GPT-4’s performance improves from 84.2% with greedy sampling to 87.3% with uncertainty-routed chain-of-thought approach with 32 samples, but it already achieves these gains from using 32 chain-of-thought samples. In contrast, Gemini Ultra improves its performance significantly from 84.0% with greedy sampling to 90.0% with uncertainty-routed chain-of-thought approach with 32 samples while it marginally improves to 85.0% with the use of 32 chain-of-thought samples only. 9.2. Capabilities and Benchmarking Tasks We use more than 50 benchmarks as a holistic harness to evaluate the Gemini models across text, image, audio and video. We provide a detailed list of benchmarking tasks for six different capabilities in text understanding and generation: factuality, long context, math/science, reasoning, summarization, and multilinguality. We also enumerate the benchmarks used for image understanding, video understanding, and audio understanding tasks. • : We use 5 benchmarks: BoolQ (Clark et al., 2019), NaturalQuestions-Closed (Kwiatkowski et al., 2019), NaturalQuestions-Retrieved (Kwiatkowski et al., 2019), RealtimeQA (Kasai et al., 2022), TydiQA-noContext and TydiQA-goldP (Clark et al., 2020). Factuality We use 6 benchmarks: NarrativeQA (Kočiský et al., 2018), Scrolls-Qasper, Scrolls-Quality (Shaham et al., 2022), XLsum (En), XLSum (non-English languages) (Hasan et al., 2021), and one other internal benchmark. • Long Context: We use 8 benchmarks: GSM8k (with CoT) (Cobbe et al., 2021), Hendryck’s MATH pass@1 (Hendrycks et al., 2021b), MMLU (Hendrycks et al., 2021a), Math-StackExchange, Math-AMC 2022-2023 problems, and three other internal benchmarks. • Math/Science: • : We use 7 benchmarks: BigBench Hard (with CoT) (Srivastava et al., 2022; Suzgun et al., 2022), CLRS (Veličković et al., 2022), Proof Writer (Tafjord et al., 2020), Reasoning-Fermi problems (Kalyan et al., 2021), Lambada (Paperno et al., 2016), HellaSwag (Zellers et al., 2019), DROP (Dua et al., 2019). Reasoning • : We use 5 benchmarks: XL Sum (English), XL Sum (non-English languages) (Hasan et al., 2021), WikiLingua (non-English languages), WikiLingua (English) (Ladhak et al., 2020), XSum (Narayan et al., 2018). Summarization • : We use 10 benchmarks: XLSum (Non-English languages) (Hasan et al., 2021), WMT22 (Kocmi et al., 2022), WMT23 (Tom et al., 2023), FRMT (Riley et al., 2023), WikiLingua (Non-English languages) (Ladhak et al., 2020), TydiQA (no context), TydiQA (GoldP) (Clark et al., 2020), MGSM (Shi et al., 2023), translated MMLU (Hendrycks et al., 2021a), NTREX (Federmann et al., 2022), FLORES-200 (Team et al., 2022). Multilinguality We use 9 benchmarks for image understanding: MMMU (Yue et al., 2023), TextVQA (Singh et al., 2019), DocVQA (Mathew et al., 2021), ChartQA (Masry et al., 2022), InfographicVQA (Mathew et al., 2022), MathVista (Lu et al., 2023), AI2D (Kembhavi et al., 2016), VQAv2 (Goyal et al., 2017), XM3600 (Thapliyal et al., 2022) for multi-lingual image understanding, and 6 benchmarks for video understanding: VATEX (Wang et al., 2019) for captioning in two different languages, YouCook2 (Zhou et al., 2018), NextQA (Xiao et al., 2021), ActivityNet-QA (Yu et al., 2019), and Perception Test MCQA (Pătrăucean et al., 2023). • Image and Video: • : We use 5 benchmarks including automatic speech recognition (ASR) tasks such as FLEURS (Conneau et al., 2023), VoxPopuli (Wang et al., 2021), Multi-lingual Librispeech (Pratap et al., 2020), and automatic speech translation task such as CoVoST 2 (Wang et al., 2020). Audio 9.3. Qualitative Examples This section shows sample qualitative examples from prompting the Gemini Ultra model. Some illustrative examples of multimodal reasoning for image understanding tasks over charts, natural images and memes are shown in Figures 8, 9, 11, 13, 14, and 15. Figure 10 shows an example of image generation capabilities of Gemini Ultra where the user generates an interleaved sequence of image and text to design a blog post. Beyond English, Figure 16 shows model’s capability to understand images in a multilingual setting. Gemini models also show strong performance on multimodal image understanding and reasoning in mathematics, as shown in Figures 12, 18 and 19. Figure 20 is an example of complex multimodal reasoning demonstrating how the model composes complex image understanding, code generation, and instruction following capabilities for a given user task. In Figure 17, we see another example of the model being able to generate working code and follow complex user instructions. Finally, Figure 21 shows an example of Gemini Ultra’s capability of understanding video by reasoning over temporally connected set of frames. 9.3.1. Chart understanding and reasoning over data 9.3.2. Multimodal question answering 9.3.3. Interleaved image and text generation 9.3.4. Image understanding and reasoning 9.3.5. Geometrical reasoning 9.3.6. Information seeking about objects 9.3.7. Multimodal reasoning based on visual cues 9.3.8. Multimodal humor understanding 9.4. Commonsense reasoning in a multilingual setting 9.4.1. Reasoning and code generation 9.4.2. Mathematics: Calculus 9.5. Multi-step reasoning and mathematics 9.5.1. Complex image understanding, code generation, and instruction following 9.5.2. Video understanding and reasoning

Gemini - A Family of Highly Capable Multimodal Models: Appendix

This paper is available on arxiv under CC 4.0 license. . Authors: Gemini Team, Google Table of Links Abstract and Introduction Model Architecture Training Infrastructure Training Dataset Evaluation Responsible Deployment Discussion and Conclusion, References Contributions and Acknowledgments Appendix 8. Contributions and Acknowledgments Leads Rohan Anil, Co-Lead, Text Sebastian Borgeaud, Co-Lead, Text Yonghui Wu, Co-Lead, Text Jean-Baptiste Alayrac, Co-Lead, MM Vision Jiahui Yu, Co-Lead, MM Vision Radu Soricut*, Co-Lead, MM Vision* Johan Schalkwyk, Lead, MM Audio Andrew M. Dai, Co-Lead, Data Anja Hauth*, Co-Lead, Data* Katie Millican, Co-Lead, Data David Silver, Co-Lead, Fine-Tuning Slav Petrov, Co-Lead, Fine-Tuning Melvin Johnson, Lead, Instruction Tuning Ioannis Antonoglou*, Co-Lead, RL Techniques* Julian Schrittwieser, Co-Lead, RL Techniques Amelia Glaese, Lead, Human Data Jilin Chen, Lead, Safety Emily Pitler, Co-Lead, Tool Use Timothy Lillicrap, Co-Lead, Tool Use Angeliki Lazaridou, Co-Lead, Eval Orhan Firat, Co-Lead, Eval James Molloy, Co-Lead, Infra Michael Isard, Co-Lead, Infra Paul R. Barham, Co-Lead, Infra Tom Hennigan, Co-Lead, Infra Benjamin Lee, Co-Lead, Codebase & Parallelism Fabio Viola, Co-Lead, Codebase & Parallelism Malcolm Reynolds, Co-Lead, Codebase & Parallelism Yuanzhong Xu, Co-Lead, Codebase & Parallelism Ryan Doherty, Lead, Ecosystem Eli Collins, Lead, Product Clemens Meyer, Co-Lead, Operations Eliza Rutherford, Co-Lead, Operations Erica Moreira, Co-Lead, Operations Kareem Ayoub, Co-Lead, Operations Megha Goel, Co-Lead, Operations Core Contributors George Tucker Enrique Piqueras Maxim Krikun Iain Barr Nikolay Savinov Ivo Danihelka Becca Roelofs Core Contributors Anaïs White Anders Andreassen Tamara von Glehn Lakshman Yagati Mehran Kazemi Lucas Gonzalez Misha Khalman Jakub Sygnowski Alexandre Frechette Charlotte Smith Laura Culp Lev Proleev Yi Luan Xi Chen James Lottes Nathan Schucher Federico Lebron Alban Rrustemi Natalie Clay Phil Crone Tomas Kocisky Jeffrey Zhao Bartek Perz Dian Yu Heidi Howard Adam Bloniarz Jack W. Rae Han Lu Laurent Sifre Marcello Maggioni Fred Alcober Dan Garrette Megan Barnes Shantanu Thakoor Jacob Austin Gabriel Barth-Maron William Wong Rishabh Joshi Rahma Chaabouni Deeni Fatiha Arun Ahuja Ruibo Liu Yunxuan Li Sarah Cogan Jeremy Chen Core Contributors Chao Jia Chenjie Gu Qiao Zhang Jordan Grimstad Ale Jakse Hartman Martin Chadwick Gaurav Singh Tomar Xavier Garcia Evan Senter Emanuel Taropa Thanumalayan Sankaranarayana Pillai Jacob Devlin Michael Laskin Diego de Las Casas Dasha Valter Connie Tao Lorenzo Blanco Adrià Puigdomènech Badia David Reitter Mianna Chen Jenny Brennan Clara Rivera Sergey Brin Shariq Iqbal Gabriela Surita Jane Labanowski Abhi Rao Stephanie Winkler Emilio Parisotto Yiming Gu Kate Olszewska Yujing Zhang Ravi Addanki Antoine Miech Annie Louis Laurent El Shafey Denis Teplyashin Geoff Brown Elliot Catt Nithya Attaluri Jan Balaguer Jackie Xiang Pidong Wang Zoe Ashwood Anton Briukhov Albert Webson Sanjay Ganapathy Smit Sanghavi Core Contributors Ajay Kannan Ming-Wei Chang Axel Stjerngren Josip Djolonga Yuting Sun Ankur Bapna Matthew Aitchison Pedram Pejman Henryk Michalewski Tianhe Yu Cindy Wang Juliette Love Junwhan Ahn Dawn Bloxwich Kehang Han Peter Humphreys Thibault Sellam James Bradbury Varun Godbole Sina Samangooei Bogdan Damoc Alex Kaskasoli Sébastien M. R. Arnold Vijay Vasudevan Shubham Agrawal Jason Riesa Dmitry Lepikhin Richard Tanburn Srivatsan Srinivasan Hyeontaek Lim Sarah Hodkinson Pranav Shyam Johan Ferret Steven Hand Ankush Garg Tom Le Paine Jian Li Yujia Li Minh Giang Alexander Neitz Zaheer Abbas Sarah York Machel Reid Elizabeth Cole Aakanksha Chowdhery Dipanjan Das Dominika Rogozińska Vitaly Nikolaev Core Contributors Pablo Sprechmann Zachary Nado Lukas Zilka Flavien Prost Luheng He Marianne Monteiro Gaurav Mishra Chris Welty Josh Newlan Dawei Jia Miltiadis Allamanis Clara Huiyi Hu Raoul de Liedekerke Justin Gilmer Carl Saroufim Shruti Rijhwani Shaobo Hou Disha Shrivastava Anirudh Baddepudi Alex Goldin Adnan Ozturel Albin Cassirer Yunhan Xu Daniel Sohn Devendra Sachan Reinald Kim Amplayo Craig Swanson Dessie Petrova Shashi Narayan Arthur Guez Siddhartha Brahma Jessica Landon Miteyan Patel Ruizhe Zhao Kevin Villela Luyu Wang Wenhao Jia Matthew Rahtz Mai Giménez Legg Yeung Hanzhao Lin James Keeling Petko Georgiev Diana Mincu Boxi Wu Salem Haykal Rachel Saputro Kiran Vodrahalli Core Contributors James Qin Zeynep Cankara Abhanshu Sharma Nick Fernando Will Hawkins Behnam Neyshabur Solomon Kim Adrian Hutter Priyanka Agrawal Alex Castro-Ros George van den Driessche Tao Wang Fan Yang Shuo-yiin Chang Paul Komarek Ross McIlroy Mario Lučić Guodong Zhang Wael Farhan Michael Sharman Paul Natsev Paul Michel Yong Cheng Yamini Bansal Siyuan Qiao Kris Cao Siamak Shakeri Christina Butterfield Justin Chung Paul Kishan Rubenstein Shivani Agrawal Arthur Mensch Kedar Soparkar Karel Lenc Timothy Chung Aedan Pope Loren Maggiore Jackie Kay Priya Jhakra Shibo Wang Joshua Maynez Mary Phuong Taylor Tobin Andrea Tacchetti Maja Trebacz Kevin Robinson Yash Katariya Sebastian Riedel Core Contributors Paige Bailey Kefan Xiao Nimesh Ghelani Lora Aroyo Ambrose Slone Neil Houlsby Xuehan Xiong Zhen Yang Elena Gribovskaya Jonas Adler Mateo Wirth Lisa Lee Music Li Thais Kagohara Jay Pavagadhi Sophie Bridgers Anna Bortsova Sanjay Ghemawat Zafarali Ahmed Tianqi Liu Richard Powell Vijay Bolina Mariko Iinuma Polina Zablotskaia James Besley Da-Woon Chung Timothy Dozat Ramona Comanescu Xiance Si Jeremy Greer Guolong Su Martin Polacek Raphaël Lopez Kaufman Simon Tokumine Hexiang Hu Elena Buchatskaya Yingjie Miao Mohamed Elhawaty Aditya Siddhant Nenad Tomasev Jinwei Xing Christina Greer Helen Miller Shereen Ashraf Aurko Roy Zizhao Zhang Ada Ma Angelos Filos Core Contributors Milos Besta Rory Blevins Ted Klimenko Chih-Kuan Yeh Soravit Changpinyo Jiaqi Mu Oscar Chang Mantas Pajarskas Carrie Muir Vered Cohen Charline Le Lan Krishna Haridasan Amit Marathe Steven Hansen Sholto Douglas Rajkumar Samuel Mingqiu Wang Sophia Austin Chang Lan Jiepu Jiang Justin Chiu Jaime Alonso Lorenzo Lars Lowe Sjösund Sébastien Cevey Zach Gleicher Thi Avrahami Anudhyan Boral Hansa Srinivasan Vittorio Selo Rhys May Konstantinos Aisopos Léonard Hussenot Livio Baldini Soares Kate Baumli Michael B. Chang Adrià Recasens Ben Caine Alexander Pritzel Filip Pavetic Fabio Pardo Anita Gergely Justin Frye Vinay Ramasesh Dan Horgan Kartikeya Badola Nora Kassner Subhrajit Roy Ethan Dyer Core Contributors Víctor Campos Alex Tomala Yunhao Tang Dalia El Badawy Elspeth White Basil Mustafa Oran Lang Abhishek Jindal Sharad Vikram Zhitao Gong Sergi Caelles Ross Hemsley Gregory Thornton Fangxiaoyu Feng Wojciech Stokowiec Ce Zheng Phoebe Thacker Çağlar Ünlü Zhishuai Zhang Mohammad Saleh James Svensson Max Bileschi Piyush Patil Ankesh Anand Roman Ring Katerina Tsihlas Arpi Vezer Marco Selvi Toby Shevlane Mikel Rodriguez Tom Kwiatkowski Samira Daruki Keran Rong Allan Dafoe Nicholas FitzGerald Keren Gu-Lemberg Mina Khan Lisa Anne Hendricks Marie Pellat Vladimir Feinberg James Cobon-Kerr Tara Sainath Maribeth Rauh Sayed Hadi Hashemi Richard Ives Yana Hasson YaGuang Li Eric Noland Core Contributors Yuan Cao Nathan Byrd Le Hou Qingze Wang Thibault Sottiaux Michela Paganini Jean-Baptiste Lespiau Alexandre Moufarek Samer Hassan Kaushik Shivakumar Joost van Amersfoort Amol Mandhane Pratik Joshi Anirudh Goyal Matthew Tung Andrew Brock Hannah Sheahan Vedant Misra Cheng Li Nemanja Rakićević Mostafa Dehghani Fangyu Liu Sid Mittal Junhyuk Oh Seb Noury Eren Sezener Fantine Huot Matthew Lamm Nicola De Cao Charlie Chen Contributors Gamaleldin Elsayed Ed Chi Mahdis Mahdieh Ian Tenney Nan Hua Ivan Petrychenko Patrick Kane Dylan Scandinaro Rishub Jain Jonathan Uesato Romina Datta Adam Sadovsky Oskar Bunyan Dominik Rabiej Shimu Wu John Zhang Contributors Gautam Vasudevan Edouard Leurent Mahmoud Alnahlawi Ionut Georgescu Nan Wei Ivy Zheng Betty Chan Pam G Rabinovitch Piotr Stanczyk Ye Zhang David Steiner Subhajit Naskar Michael Azzam Matthew Johnson Adam Paszke Chung-Cheng Chiu Jaume Sanchez Elias Afroz Mohiuddin Faizan Muhammad Jin Miao Andrew Lee Nino Vieillard Sahitya Potluri Jane Park Elnaz Davoodi Jiageng Zhang Jeff Stanway Drew Garmon Abhijit Karmarkar Zhe Dong Jong Lee Aviral Kumar Luowei Zhou Jonathan Evens William Isaac Zhe Chen Johnson Jia Anselm Levskaya Zhenkai Zhu Chris Gorgolewski Peter Grabowski Yu Mao Alberto Magni Kaisheng Yao Javier Snaider Norman Casagrande Paul Suganthan Evan Palmer Contributors Geoffrey Irving Edward Loper Manaal Faruqui Isha Arkatkar Nanxin Chen Izhak Shafran Michael Fink Alfonso Castaño Irene Giannoumis Wooyeol Kim Mikołaj Rybiński Ashwin Sreevatsa Jennifer Prendki David Soergel Adrian Goedeckemeyer Willi Gierke Mohsen Jafari Meenu Gaba Jeremy Wiesner Diana Gage Wright Yawen Wei Harsha Vashisht Yana Kulizhskaya Jay Hoover Maigo Le Lu Li Chimezie Iwuanyanwu Lu Liu Kevin Ramirez Andrey Khorlin Albert Cui Tian LIN Marin Georgiev Marcus Wu Ricardo Aguilar Keith Pallo Abhishek Chakladar Alena Repina Xihui Wu Tom van der Weide Priya Ponnapalli Caroline Kaplan Jiri Simsa Shuangfeng Li Olivier Dousse Fan Yang Jeff Piper Nathan Ie Contributors Minnie Lui Rama Pasumarthi Nathan Lintz Anitha Vijayakumar Lam Nguyen Thiet Daniel Andor Pedro Valenzuela Cosmin Paduraru Daiyi Peng Katherine Lee Shuyuan Zhang Somer Greene Duc Dung Nguyen Paula Kurylowicz Sarmishta Velury Sebastian Krause Cassidy Hardin Lucas Dixon Lili Janzer Kiam Choo Ziqiang Feng Biao Zhang Achintya Singhal Tejasi Latkar Mingyang Zhang Quoc Le Elena Allica Abellan Dayou Du Dan McKinnon Natasha Antropova Tolga Bolukbasi Orgad Keller David Reid Daniel Finchelstein Maria Abi Raad Remi Crocker Peter Hawkins Robert Dadashi Colin Gaffney Sid Lall Ken Franko Egor Filonov Anna Bulanova Rémi Leblond Vikas Yadav Shirley Chung Harry Askham Luis C. Cobo Contributors Kelvin Xu Felix Fischer Jun Xu Christina Sorokin Chris Alberti Chu-Cheng Lin Colin Evans Hao Zhou Alek Dimitriev Hannah Forbes Dylan Banarse Zora Tung Jeremiah Liu Mark Omernick Colton Bishop Chintu Kumar Rachel Sterneck Ryan Foley Rohan Jain Swaroop Mishra Jiawei Xia Taylor Bos Geoffrey Cideron Ehsan Amid Francesco Piccinno Xingyu Wang Praseem Banzal Petru Gurita Hila Noga Premal Shah Daniel J. Mankowitz Alex Polozov Nate Kushman Victoria Krakovna Sasha Brown MohammadHossein Bateni Dennis Duan Vlad Firoiu Meghana Thotakuri Tom Natan Anhad Mohananey Matthieu Geist Sidharth Mudgal Sertan Girgin Hui Li Jiayu Ye Ofir Roval Reiko Tojo Contributors Michael Kwong James Lee-Thorp Christopher Yew Quan Yuan Sumit Bagri Danila Sinopalnikov Sabela Ramos John Mellor Abhishek Sharma Aliaksei Severyn Jonathan Lai Kathy Wu Heng-Tze Cheng David Miller Nicolas Sonnerat Denis Vnukov Rory Greig Jennifer Beattie Emily Caveness Libin Bai Julian Eisenschlos Alex Korchemniy Tomy Tsai Mimi Jasarevic Weize Kong Phuong Dao Zeyu Zheng Frederick Liu Fan Yang Rui Zhu Mark Geller Tian Huey Teh Jason Sanmiya Evgeny Gladchenko Nejc Trdin Andrei Sozanschi Daniel Toyama Evan Rosen Sasan Tavakkol Linting Xue Chen Elkind Oliver Woodman John Carpenter George Papamakarios Rupert Kemp Sushant Kafle Tanya Grunina Rishika Sinha Contributors Alice Talbert Abhimanyu Goyal Diane Wu Denese Owusu-Afriyie Cosmo Du Chloe Thornton Jordi Pont-Tuset Pradyumna Narayana Jing Li Saaber Fatehi John Wieting Omar Ajmeri Benigno Uria Tao Zhu Yeongil Ko Laura Knight Amélie Héliou Ning Niu Shane Gu Chenxi Pang Dustin Tran Yeqing Li Nir Levine Ariel Stolovich Norbert Kalb Rebeca Santamaria-Fernandez Sonam Goenka Wenny Yustalim Robin Strudel Ali Elqursh Balaji Lakshminarayanan Charlie Deck Shyam Upadhyay Hyo Lee Mike Dusenberry Zonglin Li Xuezhi Wang Kyle Levin Raphael Hoffmann Dan Holtmann-Rice Olivier Bachem Summer Yue Sho Arora Eric Malmi Daniil Mirylenka Qijun Tan Christy Koh Soheil Hassas Yeganeh Contributors Siim Põder Steven Zheng Francesco Pongetti Mukarram Tariq Yanhua Sun Lucian Ionita Mojtaba Seyedhosseini Pouya Tafti Ragha Kotikalapudi Zhiyu Liu Anmol Gulati Jasmine Liu Xinyu Ye Bart Chrzaszcz Lily Wang Nikhil Sethi Tianrun Li Ben Brown Shreya Singh Wei Fan Aaron Parisi Joe Stanton Chenkai Kuang Vinod Koverkathu Christopher A. Choquette-Choo Yunjie Li TJ Lu Abe Ittycheriah Prakash Shroff Pei Sun Mani Varadarajan Sanaz Bahargam Rob Willoughby David Gaddy Ishita Dasgupta Guillaume Desjardins Marco Cornero Brona Robenek Bhavishya Mittal Ben Albrecht Ashish Shenoy Fedor Moiseev Henrik Jacobsson Alireza Ghaffarkhah Morgane Rivière Alanna Walton Clément Crepy Alicia Parrish Contributors Yuan Liu Zongwei Zhou Clement Farabet Carey Radebaugh Praveen Srinivasan Claudia van der Salm Andreas Fidjeland Salvatore Scellato Eri Latorre-Chimoto Hanna Klimczak-Plucińska David Bridson Dario de Cesare Tom Hudson Piermaria Mendolicchio Lexi Walker Alex Morris Ivo Penchev Matthew Mauger Alexey Guseynov Alison Reid Seth Odoom Lucia Loher Victor Cotruta Madhavi Yenugula Dominik Grewe Anastasia Petrushkina Tom Duerig Antonio Sanchez Steve Yadlowsky Amy Shen Amir Globerson Adam Kurzrok Lynette Webb Sahil Dua Dong Li Preethi Lahoti Surya Bhupatiraju Dan Hurt Haroon Qureshi Ananth Agarwal Tomer Shani Matan Eyal Anuj Khare Shreyas Rammohan Belle Lei Wang Chetan Tekur Mihir Sanjay Kale Jinliang Wei Contributors Ruoxin Sang Brennan Saeta Tyler Liechty Yi Sun Yao Zhao Stephan Lee Pandu Nayak Doug Fritz Manish Reddy Vuyyuru John Aslanides Nidhi Vyas Martin Wicke Xiao Ma Taylan Bilal Evgenii Eltyshev Daniel Balle Nina Martin Hardie Cate James Manyika Keyvan Amiri Yelin Kim Xi Xiong Kai Kang Florian Luisier Nilesh Tripuraneni David Madras Mandy Guo Austin Waters Oliver Wang Joshua Ainslie Jason Baldridge Han Zhang Garima Pruthi Jakob Bauer Feng Yang Riham Mansour Jason Gelman Yang Xu George Polovets Ji Liu Honglong Cai Warren Chen XiangHai Sheng Emily Xue Sherjil Ozair Adams Yu Christof Angermueller Contributors Xiaowei Li Weiren Wang Julia Wiesinger Emmanouil Koukoumidis Yuan Tian Anand Iyer Madhu Gurumurthy Mark Goldenson Parashar Shah MK Blake Hongkun Yu Anthony Urbanowicz Jennimaria Palomaki Chrisantha Fernando Kevin Brooks Ken Durden Harsh Mehta Nikola Momchev Elahe Rahimtoroghi Maria Georgaki Amit Raul Sebastian Ruder Morgan Redshaw Jinhyuk Lee Komal Jalan Dinghua Li Ginger Perng Blake Hechtman Parker Schuh Milad Nasr Mia Chen Kieran Milan Vladimir Mikulik Trevor Strohman Juliana Franco Tim Green Program Leads Demis Hassabis Koray Kavukcuoglu Overall Technical Leads (equal contribution) Jeffrey Dean Oriol Vinyals The roles are defined as below: • : Individual(s) responsible for the sub-team throughout the project. Lead Individual that had significant impact throughout the project. • Core Contributor: : Individual that had contributions to the project and was partially involved with the effort. • Contributor Responsible for the organizational aspects of the Gemini effort • Program Lead: Responsible for the technical direction of the overall Gemini effort • Overall Technical Lead: Within each role, contributions are equal, and are listed in a randomized order. Ordering within each role does not indicate ordering of the contributions. Gemini is a cross-Google effort, with members from Google DeepMind (GDM), Google Research (GR), Knowledge and Information (K&I), Core ML, Cloud, Labs, and more. We thank our reviewers and colleagues for their valuable discussions and feedback on the report— Alexandra Belias, Arielle Bier, Eleanor Tomlinson, Emily Hossellman, Gaby Pearl, Helen King, Hollie Dobson, Jaclyn Konzelmann, Jennifer Beroshi, Joel Moss, Jon Small, Jonathan Fildes, Oli Gaymond, Rebecca Bland, Reena Jana, and Tom Lue. Our work is made possible by the dedication and efforts of numerous teams at Google. We would like to acknowledge the support from Abhi Mohan, Adekunle Bello, Aishwarya Nagarajan, Alaa Saade, Alejandro Lince, Alexander Chen, Alexander Kolbasov, Alexander Schiffhauer, Amar Subramanya, Ameya Shringi, Amin Vahdat, Anda Rabatić, Anthonie Gross, Antoine Yang, Anthony Green, Anton Ruddock, Art Khurshudov, Artemis Chen, Arthur Argenson, Avinatan Hassidim, Beiye Liu, Benjamin Schroeder, Bin Ni, Brett Daw, Bryan Chiang, Burak Gokturk, Carl Crous, Carrie Grimes Bostock, Charbel Kaed, Charlotte Banks, Che Diaz, Chris Larkin, Christy Lian, Claire Cui, Clare Bycroft, Corentin Tallec, Daniel Herndon, Dave Burke, David Battle, David Engel, Dipannita Shaw, Donghyun Koo, Doug Ritchie, Dragos Stefanescu, Elissa Wolf, Emre Sargin, Eran Ofek, Eric Herren, Estella King, Fatema Alkhanaizi, Felix Gimeno, Fernando Pereira, Florent Altché, Gabriel Carvajal, Gaurav Gandhi, George Powell, Goran Pavičić, Harry Richardson, Hassan Wassel, Hongji Li, Idan Szpektor, Igor Ivanisevic, Ivan Jambrešić, Ivan Jurin, Jade Fowler, James Assiene, Jay Yagnik, Jeanbastien Grill, Jeff Seibert, Jenna LaPlante, Jessica Austin, Jianxing Lu, Jim O’Keeffe, Jin Huang, Joe Heyward, Johannes Welbl, John Jumper, Jonathan Caton, Josh Woodward, Joshua Foster, Kathryn Tunyasuvunakool, Katrina Wong, Kavya Kopparapu, Kelvin Nguyen, Kira Yin, Konstantin Sharlaimov, Kun Li, Lee Hong, Lilly Taylor, Longfei Shen, Luc Mercier, Maciej Mikuła, Mania Abdi, Manuel Sanchez, Maria Ines Aranguren, Mario Carlos Cortes III, Matthew Tait, Matthias Lochbrunner, Mehdi Ghissassi, Micah Mosley, Michael Bendersky, Michael Figurnov, Michael Harris, Michael Mathieu, Michael O’Neill, Michael Vorburger, Mihir Paradkar, Nandita Dukkipati, Nathan Carter, Nathan Watson, Neil Rabinowitz, Nikhil Dandekar, Nishant Ranka, Obaid Sarvana, Olcan Sercinoglu, Olivier Lacombe, Ottavia Bertolli, Paul Caron, Pranesh Srinivasan, Praveen Kumar, Rahul Sukthankar, Raia Hadsell, Rajagopal Ananthanarayanan, Roberto Lupi, Rosie Zou, Sachin Menezes, Sadegh Jazayeri, Sam Cheung, Sameer Bidichandani, Sania Alex, Sanjiv Kumar, Sara Wiltberger, Sarah Fitzgerald, Saz Basu, Sebastian Nowozin, Shannon Hepburn, Shayne Cardwell, Sissie Hsiao, Srinivasan Venkatachary, Sugato Basu, Sundar Pichai, Sundeep Tirumalareddy, Susannah Young, Swetha Vijayaraghavan, Tania Bedrax-Weiss, Taylor Applebaum, Teiva Harsanyi, Terry Chen, Tim Blyth, Ting Liu, Tom Cobley, Tomas Izo, Trystan Upstill, Varun Singhai, Vedrana Klarić Trupčević, Victor Cai, Vladimir Pudovkin, Vu Dang, Wenbo Zhao, Wesley Crow, Wesley Szeng, Xiaodan Song, Yazhou Zu, Ye Tian, Yicong Wang, Yixing Wang, Yossi Matias, Yunlong Jiao, Zachary Jessup, Zhenchuan Pang, Žiga Avsec, Zimeng Yang, and Zoubin Ghahramani. We’d also like to recognize the AlphaCode team, the Borg Scheduling team, the Facilities team, the Gemini Demo Team, the Global Server Ops (GSO) team, the JAX team, the the Legal team, ML SRE team, the ML Supercomputer (MLSC) team, the PartIR team, the Platforms Infrastructure Engineering (PIE) team, and the XLA Compiler team. We thank everyone at Google not explicitly mentioned above, who have shared excitement, given feedback on early Gemini models or created interesting demo uses of Gemini, and worked with or supported the core Gemini team on many aspects of this project.

Gemini -  A Family of Highly Capable Multimodal Models: Contributions and Acknowledgments

Gemini - A Family of Highly Capable Multimodal Models: Discussion and Conclusion, References

This paper is available on arxiv under CC 4.0 license. Authors: (1) Gemini Team, Google. Table of Links Abstract and Introduction Model Architecture Training Infrastructure Training Dataset Evaluation Responsible Deployment Discussion and Conclusion, References Contributions and Acknowledgments Appendix 6. Responsible Deployment During the development of the Gemini models, we follow a structured approach to responsible deployment in order to identify, measure, and manage foreseeable downstream societal impacts of our models, in line with previous releases of Google’s AI technology (Kavukcuoglu et al., 2022). Throughout the lifecycle of the project, we follow the structure below. This section outlines our broad approach and key findings through this process. We will share more details on this in an upcoming report. 6.1. Impact Assessment We develop model impact assessments to identify, assess, and document key downstream societal benefits and harms associated with the development of advanced Gemini models. These are informed by prior academic literature on language model risks (Weidinger et al., 2021), findings from similar prior exercises conducted across the industry (Anil et al., 2023; Anthropic, 2023; OpenAI, 2023a), ongoing engagement with experts internally and externally, and unstructured attempts to discover new model vulnerabilities. Areas of focus include: factuality, child safety, harmful content, cybersecurity, biorisk, representation and inclusivity. These assessments are updated in tandem with model development. Impact assessments are used to guide mitigation and product delivery efforts, and inform deployment decisions. Gemini impact assessments spanned across different capabilities of Gemini models, assessing the potential consequences of these capabilities with Google’s AI Principles (Google, 2023). 6.2. Model Policy Building upon this understanding of known and anticipated effects, we developed a set of “model policies” to steer model development and evaluations. Model policy definitions act as a standardized criteria and prioritization schema for responsible development and as an indication of launch-readiness. Gemini model policies cover a number of domains including: child safety, hate speech, factual accuracy, fairness and inclusion, and harassment. 6.3. Evaluations To assess the Gemini models against policy areas and other key risk areas identified within impact assessments, we developed a suite of evaluations across the lifecycle of model development. Development evaluations are conducted for the purpose of ‘hill-climbing’ throughout training and fine-tuning Gemini models. These evaluations are designed by the Gemini team, or are assessments against external academic benchmarks. Evaluations consider issues such as helpfulness (instruction following and creativity), safety and factuality. See section 5.1.6 and the next section on mitigations for a sample of results. Assurance evaluations are conducted for the purpose of governance and review, usually at the end of key milestones or training runs by a group outside of the model development team. Assurance evaluations are standardized by modality and datasets are strictly held-out. Only high-level insights are fed back into the training process to assist with mitigation efforts. Assurance evaluations include testing across Gemini policies, and include ongoing testing for dangerous capabilities such as potential biohazards, persuasion, and cybersecurity (Shevlane et al., 2023). External evaluations are conducted by partners outside of Google to identify blindspots. External groups stress-test our models across a range of issues, including across areas listed in the White House Commitments,[7] and tests are conducted through a mixture of structured evaluations and unstructured red teaming. The design of these evaluations are independent and results are reported periodically to the Google DeepMind team. In addition to this suite of external evaluations, specialist internal teams conduct ongoing red teaming of our models across areas such as the Gemini policies and security. These activities include less structured processes involving sophisticated adversarial attacks to identify new vulnerabilities. Discovery of potential weaknesses can then be used to mitigate risks and improve evaluation approaches internally. We are committed to ongoing model transparency and plan to share additional results from across our evaluation suite over time. 6.4. Mitigations Mitigations are developed in response to the outcomes of the assessment, policy, and evaluation approaches described above. Evaluations and mitigations are used in an iterative way, with evaluations being re-run following mitigation efforts. We discuss our efforts on mitigating model harms across data, instruction-tuning, and factuality below. 6.4.1. Data Prior to training, we take various steps to mitigate potential downstream harms at the data curation and data collection stage. As discussed in the section on “Training Data”, we filter training data for high-risk content and to ensure all training data is sufficiently high quality. Beyond filtering, we also take steps to ensure all data collected meets Google DeepMind’s best practices on data enrichment,[8] developed based on the Partnership on AI’s “Responsible Sourcing of Data Enrichment Services”[9]. This includes ensuring all data enrichment workers are paid at least a local living wage. 6.4.2. Instruction Tuning Instruction tuning encompasses supervised fine tuning (SFT) and reinforcement learning through human feedback (RLHF) using a reward model. We apply instruction tuning in both text and multimodal settings. Instruction tuning recipes are carefully designed to balance the increase in helpfulness with decrease in model harms related to safety and hallucinations (Bai et al., 2022a). Curation of “quality” data is critical for SFT, reward model training, and RLHF. The data mixture ratios are ablated with smaller models to balance the metrics on helpfulness (such as instruction following, creativity) and reduction of model harms, and these results generalize well to larger models. We have also observed that data quality is more important than quantity (Touvron et al., 2023b; Zhou et al., 2023), especially for larger models. Similarly, for reward model training, we find it critical to balance the dataset with examples where the model prefers to say, “I cannot help with that,” for safety reasons and examples where the model outputs helpful responses. We use multi-objective optimization with a weighted sum of reward scores from helpfulness, factuality, and safety, to train a multi-headed reward model. We further elaborate our approach to mitigate risks of harmful text generation. We enumerate approximately 20 harm types (e.g. hate speech, providing medical advice, suggesting dangerous behavior) across a wide variety of use cases. We generate a dataset of potential harm-inducing queries in these categories, either manually by policy experts and ML engineers, or via prompting high capability language models with topical keywords as seeds. Given the harm-inducing queries, we probe our Gemini models and analyze the model responses via side-by-side evaluation. As discussed above, we balance the objective of model output response being harmless versus being helpful. From the detected risk areas, we create additional supervised fine-tuning data to demonstrate the desirable responses. To generate such responses at scale, we heavily rely on a custom data generation recipe loosely inspired from Constitutional AI (Bai et al., 2022b), where we inject variants of Google’s content policy language as “constitutions”, and utilize language model’s strong zero-shot reasoning abilities (Kojima et al., 2022) to revise responses and choose between multiple response candidates. We have found this recipe to be effective – for example in Gemini Pro, this overall recipe was able to mitigate a majority of our identified text harm cases, without any perceptible decrease on response helpfulness. 6.4.3. Factuality It is important that our models generate responses that are factual in a variety of scenarios, and to reduce the frequency of hallucinations. We focused instruction tuning efforts on three key desired behaviors, reflecting real-world scenarios: : If instructed to generate a response that should be fully attributed to a given context in the prompt, Gemini should produce a response with the highest degree of faithfulness to the context (Rashkin et al., 2023). This includes the summarization of a user-provided source, generating fine-grained citations given a question and provided snippets akin to Menick et al. (2022); Peng et al. (2023), answering questions from a long-form source such as a book (Mihaylov et al., 2018), and transforming a given source to a desired output (e.g. an email from a portion of a meeting transcript). 1. Attribution If provided with a fact-seeking prompt without any given source, Gemini should not hallucinate incorrect information (see Section 2 of Roberts et al. (2020) for a definition). These prompts can range from information-seeking prompts (e.g. “Who is the prime minister of India?”) to semi-creative prompts that may request factual information (e.g. “Write a 500-word speech in favor of the adoption of renewable energy”). 2. Closed-Book Response Generation: If prompted with an input such that it is “unanswerable”, Gemini should not hallucinate. Rather, it should acknowledge that it cannot provide a response by hedging. These include scenarios where the input prompt contains false-premise questions (see examples in Hu et al. (2023)), the input prompt instructs the model to perform open-book QA, but the answer is not derivable from the given context, and so forth. 3. Hedging: We elicited these desired behaviors from Gemini models by curating targeted supervised-fine tuning datasets and performing RLHF. Note that the results produced here do not include endowing Gemini with tools or retrieval that purportedly could boost factuality (Menick et al., 2022; Peng et al., 2023). We provide three key results on respective challenge sets below. An evaluation set containing fact-seeking prompts (primarily closed-book). This is evaluated via human annotators who fact-check each response manually; we report the percentage of factually-inaccurate responses as judged by annotators. 1. Factuality Set: An evaluation set containing a variety of prompts that require attribution to sources in the prompt. This is evaluated via human annotators who check for attribution to sources in the prompt for each response manually; the reported metric is AIS (Rashkin et al., 2023). 2. Attribution Set: : An automatic evaluation setup where we measure whether Gemini models hedge 3. Hedging Set accurately We compare Gemini Pro with a version of instruction-tuned Gemini Pro model without any factuality focused adaptation in Table 14. We observe that the rate of inaccuracy is halved in the factuality set, the accuracy of attribution is increased by 50% from the attribution set, and the model successfully hedges 70% (up from 0%) in the provided hedging set task. 6.5. Deployment Following the completion of reviews, model cards (Mitchell et al., 2019) for each approved Gemini model are created for structured and consistent internal documentation of critical performance and responsibility metrics as well as to inform appropriate external communication of these metrics over time. 6.6. Responsible Governance Across the responsible development process, we undertake ethics and safety reviews with the Google DeepMind’s Responsibility and Safety Council (RSC),[10] an interdisciplinary group which evaluates Google DeepMind’s projects, papers and collaborations against Google’s AI Principles. The RSC provides input and feedback on impact assessments, policies, evaluations and mitigation efforts. During the Gemini project, the RSC set specific evaluation targets across key policy domains (e.g. child safety). [7] https://whitehouse.gov/wp-content/uploads/2023/07/Ensuring-Safe-Secure-and-Trustworthy-AI.pdf [8] https://deepmind.google/discover/blog/best-practices-for-data-enrichment/ [9] https://partnershiponai.org/responsible-sourcing-considerations/ [10] https://deepmind.google/about/responsibility-safety/

FAMIRY

famiry.ru

-- employees

Since n.d.

COMPANY RANKING

FAMIRY

EVERGREEN INDEX #4336

Representing this company? Click here to claim and customize this page!

Famiry WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

FAMIRY

famiry.ru

-- employees

Since n.d.

COMPANY RANKING

FAMIRY

EVERGREEN INDEX #4336

RELATED COMPANIESRANK

STORIES ABOUT FAMIRY

Representing this company? Click here to claim and customize this page!

Famiry WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!