At 5ire Capital, we are building an exclusive investor fund, focused on creating the next 100 billionaires with web3. An essential part of our mission is to help our partner investors grow their capital by investing in next-gen blockchain projects. We encourage and invest in projects that propagate the “for-benefit” economy, and are committed to shifting the paradigm from “for-profit” to “for benefit.”

5ire

The Sociable's 40 Under 40 Influencers in Blockchain for 2023

Los 40 influencers menores de 40 de The Sociable en Blockchain para 2023

Les 40 influenceurs de moins de 40 ans de The Sociable dans la blockchain pour 2023

2023 के लिए ब्लॉकचेन में सोशिएबल के 40 अंडर 40 प्रभावशाली व्यक्ति

The Sociable の 2023 年のブロックチェーンにおける 40 歳未満のインフルエンサー 40 人

Os 40 influenciadores com menos de 40 anos do Sociable no Blockchain para 2023

Danh sách 40 người có ảnh hưởng dưới 40 tuổi của The Sociable trong Blockchain cho năm 2023

The Sociable 评选 2023 年区块链领域 40 位 40 岁以下影响者

H2O Securities received $150 million from (Global Emerging Market), the project is strange, GEM which is already trading (I think they bought at very sweet prices). The project is in South Africa and as I understood it is going to create water production stations, the says everything is very blurry and as I understood they are going to share the profit with token holders and they claim they will build with blockchain these stations 5 times faster than conventional companies. The most interesting, I found nothing about the project on Crunchbase, but I found about GEM fund on the Crunchbase page and it says that they have 3.4 billion, but there is no news or investments on Crunchbase, which is very strange, in general everything is strange and I recommend to be careful, so if you want you can just follow the project, but it all seems strange to me, about the investment in h2o written in such sources as , , , . H2O Securities - GEM bought from the $150 million of their token white paper finance yahoo investing CNBC cointelegraph Rubix got $100M from . Rubix is a new blockchain with technology, they want to make it: secure and scalable, with no fees and no carbon, add NFT there, and release their token. The project is very white in contrast to the project above, the entire team is not anonymous, there is a where you can follow their work, there is on which you can explore each employee and similar social networks, and the project is at a very early stage and recommend to look out for new activities (they will soon), investors private fund which has already invested in such companies as GetSwift, PT Widodo Makmur Perkasa Tbk, Enry’s Island, SingularityNET. Rubix - LDA Capital proof of pledge Github LinkedIn Safe - raised $100M from funds like 1kx, A&T Capital, Tiger Global Management, Greenfield One, Digital Currency Group, Rockaway Blockchain Fund, and 77 other funds. A pretty old project that used to serve only very large DAOs and has been doing it to this day since 2016(they have a total of in vaults). They provide storage for DAOs or users who want to secure their funds. Security is created by creating a vault in networks like Ethereum, Gnosis Chain, Polygon, BNB Smart Chain, Energy Web Chain, Arbitrum, Aurora, Avalanche, Optimism, Goerli, Rinkeby, and Volta. Add their participants (or your different wallets) and drop cryptocurrency and NFT, then if you suddenly need to withdraw funds or NFT, you need to confirm the transaction by various participants of the safe (the number of users to confirm that the transaction began, is exposed when you create a safe), so, for example, you created this safe and added there your Metamask, ledger and trust wallet from the phone, So if you lose your phone or someone breaks into your Metamask he will not be able to withdraw money until you sign with your other wallets, I recommend to set the number of users to sign the transaction to 1 less than their total number, so if you suddenly lose your phone could withdraw money, also do not forget that if you create a store in the network Polygon you can send there only what is in the network Polygon. Gnosis Safe $107,176,958,383 5ire - raised $100M from . Blockchain with a transaction speed of 600 ms, throughput of 60000 tps, block generation time of 3 sec and block size of 4 MB. They say it will be faster and cheaper than any existing blockchain, there will be no carbon footprint, interchangeability with the ability to send assets, collectibles, or files to any chain, mobile number, or web3 address, and say they will make it possible to write smart contracts without code. The team is competent, as I understand they have a member from the UN and a woman who worked with Tron. 5ire Sram & Mram Animoca Brands - raised $75,320,000 from Liberty City Ventures, Kingsway Capital, Alpha Wave Ventures, 10T, SG Spring Limited Partnership Fund, Generation Highway Ltd, and Cosmic Summit. I think a lot of people know this company, they created a lot of games and one of the most popular of them is the sandbox meta-universe. They have also invested in 225 projects of which 95 are lead investors. They raised $75,320,000 to invest in the crypto market and maybe buy out some projects. Animoca Brands Hidden Road - raised $50M from Castle Island Ventures, Citadel Securities, FTX Ventures, Uncorrelated Ventures, Greycroft, XBTO Humla Ventures, Wintermute, SLN Capital, Profluent Trading, Coinbase Ventures, and Corner Capital among others. It is a global credit network for institutional investors, providing easy access to both traditional and digital markets. Hidden Road Seed - raised $41 million from Animoca Brands, Kingway Capital, Anthos, Novator, Supercell, Roosh and Angelhub. Open-world simulator in which when you leave the world the game will continue, the game from the studio , I like the graphics and the project at a very early stage, I also like that in investors Animoca Brands creators sandbox and Supercell creators: Brawl Stars, Clash Royale, Clash of Clans, Hay Day and many other cool games on the phone. They have 3,000 followers on Twitter at the time of writing, so I highly recommend trying to be active in their community at an early stage. And you can also sign up for a now. Seed Klang games game test Pulley - raised $40M from , Sherwin Gandhi, La Famiglia, Jack Altman, , Elad Gill, Avichal Garg. It’s tools and insights to help you fundraise smarter and manage a company’s balance sheet. Pulley Stripe Founders Fund Planetarium — raised 32mn from Animoca Brands, Kakao Venture, Krust Universe, Samsung NEXT, and Wemade. Creates Web3 community-driven gaming network, the project is very early stage, you can tell from the website, so I recommend keeping an eye on them and if possible start becoming their first active community. Planetarium Meta World — raised $30 million from billionaire . The project itself, as I understand it, is a meta-universe into which they want to insert: Smart contracts, internal store, and play-to-earn games (games — NFTs). Meta World Mai Vu Minh Thank you all for your attention and hope you like this compilation, I’m waiting for comments and feedback, if you want to support me here are my links, I would be very grateful if you subscribe: https://linktr.ee/immunefomo Also published . here

Fundraising in the Crypto Sector

SHERIDAN, United States, July 24th, 2024/Chainwire/--Proptokenmarket, a dynamic newcomer to the real estate and blockchain landscape, proudly unveils its global Real Estate Token Directory. This groundbreaking platform is the first to provide a third-party directory for Real Estate Tokens, offering a fresh, user-friendly approach to discovering and engaging with tokenized real estate assets from various issuers worldwide. Key Features of Proptokenmarket's Real Estate Token Directory: 1.      Global Tokenized Real Estate Assets: Exploring a curated selection of tokenized real estate assets, from single-family gems to luxury apartments on exotic islands and commercial opportunities worldwide. 2.      Data-Driven Decision-Making: Accessing comprehensive property information, token metrics, historical performance data, and market analytics to make informed investment decisions. 3.      Extensive Learning Opportunities: Proptokenmarket offers a wealth of resources for both beginners and seasoned investors to learn about real estate tokens. Users can access educational articles, tutorials, and guides that cover everything from the basics of real estate tokenization to advanced blockchain knowledge. A dedicated news feed summarizes the latest developments and trends in the real estate token market, ensuring users stay informed and educated. 4.      User-Centric Design: Navigating with ease through an intuitive and visually appealing interface, designed for both seasoned investors and newcomers. 5.      Responsive Support: Proptokenmarket is committed to providing exceptional customer support, guiding users through their investment journey and promptly addressing inquiries. Florian Schick, founder of Proptokenmarket, commented, "At Proptokenmarket, we're thrilled to introduce our Real Estate Token Directory. Our platform is designed to democratize real estate investment, allowing users to easily explore real estate tokens globally and find related platforms and issuers. Additionally, Proptokenmarket offers valuable information on real estate tokenization, catering to all knowledge levels.A dedicated news feed summarizing exciting and relevant news topics completes our platform." Users can discover the future of real estate investment by visiting https://proptokenmarket.com. About Proptokenmarket Proptokenmarket is a forward-thinking startup dedicated to reshaping the real estate token investment landscape by providing excellent transparency, accessibility and innovative solutions through a comprehensive global perspective on the real estate token market. Their mission is to provide transparency, security, and accessibility to investors worldwide. LinkedIn Contact Founder Florian Schick Proptokenmarket LLC florian@proptokenmarket.com This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program. Learn more about the program here. SHERIDAN, United States, July 24th, 2024/Chainwire/--Proptokenmarket, a dynamic newcomer to the real estate and blockchain landscape, proudly unveils its global Real Estate Token Directory. This groundbreaking platform is the first to provide a third-party directory for Real Estate Tokens, offering a fresh, user-friendly approach to discovering and engaging with tokenized real estate assets from various issuers worldwide. Key Features of Proptokenmarket's Real Estate Token Directory: 1.      Global Tokenized Real Estate Assets: Exploring a curated selection of tokenized real estate assets, from single-family gems to luxury apartments on exotic islands and commercial opportunities worldwide. 2.      Data-Driven Decision-Making: Accessing comprehensive property information, token metrics, historical performance data, and market analytics to make informed investment decisions. 3.      Extensive Learning Opportunities: Proptokenmarket offers a wealth of resources for both beginners and seasoned investors to learn about real estate tokens. Users can access educational articles, tutorials, and guides that cover everything from the basics of real estate tokenization to advanced blockchain knowledge. A dedicated news feed summarizes the latest developments and trends in the real estate token market, ensuring users stay informed and educated. 4.      User-Centric Design: Navigating with ease through an intuitive and visually appealing interface, designed for both seasoned investors and newcomers. 5.      Responsive Support: Proptokenmarket is committed to providing exceptional customer support, guiding users through their investment journey and promptly addressing inquiries. Florian Schick, founder of Proptokenmarket, commented, "At Proptokenmarket, we're thrilled to introduce our Real Estate Token Directory. Our platform is designed to democratize real estate investment, allowing users to easily explore real estate tokens globally and find related platforms and issuers. Additionally, Proptokenmarket offers valuable information on real estate tokenization, catering to all knowledge levels. A dedicated news feed summarizing exciting and relevant news topics completes our platform." Florian Schick, founder of Proptokenmarket, commented, "At Proptokenmarket, we're thrilled to introduce our Real Estate Token Directory. Our platform is designed to democratize real estate investment, allowing users to easily explore real estate tokens globally and find related platforms and issuers. Additionally, Proptokenmarket offers valuable information on real estate tokenization, catering to all knowledge levels. A dedicated news feed summarizing exciting and relevant news topics completes our platform." Users can discover the future of real estate investment by visiting https://proptokenmarket.com . https://proptokenmarket.com https://proptokenmarket.com About Proptokenmarket Proptokenmarket is a forward-thinking startup dedicated to reshaping the real estate token investment landscape by providing excellent transparency, accessibility and innovative solutions through a comprehensive global perspective on the real estate token market. Their mission is to provide transparency, security, and accessibility to investors worldwide. Proptokenmarket Proptokenmarket LinkedIn LinkedIn LinkedIn Contact Founder Florian Schick Proptokenmarket LLC florian@proptokenmarket.com This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program. Learn more about the program here. This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program. Learn more about the program here . here here

Introducing Proptokenmarket: Pioneering the Future with the “First Real Estate Token Directory”

প্রপটোকেনমার্কেট প্রবর্তন: "প্রথম রিয়েল এস্টেট টোকেন ডিরেক্টরি" সহ ভবিষ্যতের পথপ্রদর্শক

Vorstellung von Proptokenmarket: Pionierarbeit für die Zukunft mit dem „ersten Token-Verzeichnis für Immobilien“

Presentamos Proptokenmarket: pioneros en el futuro con el "primer directorio de tokens inmobiliarios"

Présentation de Proptokenmarket : pionnier de l'avenir avec le « premier répertoire de jetons immobiliers »

प्रॉपटोकनमार्केट का परिचय: "पहली रियल एस्टेट टोकन निर्देशिका" के साथ भविष्य की ओर अग्रसर होना

Proptokenmarket の紹介: 「初の不動産トークンディレクトリ」で未来を切り拓く

Proptokenmarket 소개: “최초의 부동산 토큰 디렉토리”로 미래 개척

Apresentando o Proptokenmarket: Pioneirismo no Futuro com o “Primeiro Diretório de Tokens Imobiliários”

Представляем Proptokenmarket: новаторство в будущем с «первым каталогом токенов недвижимости»

Proptokenmarket ile Tanışın: “İlk Gayrimenkul Token Dizini” ile Geleceğe Öncülük Etmek

Giới thiệu Proptokenmarket: Tiên phong cho tương lai với “Danh mục token bất động sản đầu tiên”

Proptokenmarket 介绍：以“第一个房地产代币目录”开拓未来

In this tutorial, I’ll show you how to deploy a full-stack JavaScript web app into Firebase. We’ll deploy a Next.js application to Firebase hosting. I’ll show you a quick and efficient process that can be completed in 5 minutes. https://youtu.be/-VwulR_wTv0?si=nW8dD2VCdhJPFSvg&t=82&embedable=true Creating the Next.js Project Let’s create a Next.js project if you don’t have one yet. Create a new Next.js project using the terminal with the npx command. npx create-next-app@latest Push the project to GitHub. This step is crucial as the project will be published directly on Firebase from GitHub.
Run the project locally using the "npm run dev" command to ensure everything works correctly. Setting Up Firebase Go to the Firebase console (https://console.firebase.google.com/) and click the "Add project" link.


Name the project e.g. "nextjs-on-firebase" and click a “Continue” button


In the next step, choose whether to enable Google Analytics. I’ll turn it off because I don’t need it for this tutorial.


After creating the project, navigate to the “App Hosting” section in the left sidebar. Click “Agree” to accept the Google Cloud Platform terms and conditions. The next step will be to upgrade the Firebase plan because hosting for full-stack applications is not free. Upgrading to Blaze Plan To continue, you must upgrade to the Blaze plan (pay-as-you-go). To upgrade the plan, click on the “Upgrade project” button. Fill in all the necessary details. If everything is okay, then you will see the screen below. After upgrading, click the "Get Started" button. At the next screen, you need to connect your GitHub account and project to Firebase. This is an essential step because we will use GitHub as a project source. Connect your GitHub account to Firebase.


Select your project from your GitHub repositories.


Choose the main branch (e.g., "master") for deployment.


Keep the automatic deployment toggle on for updates to Firebase when changes are pushed to the master branch.


Name the backend project (e.g., "njs-firebase") and click "Finish and deploy." If you get lost, you can watch this video: https://youtu.be/-VwulR_wTv0?si=nW8dD2VCdhJPFSvg&t=82&embedable=true Waiting for Deployment The deployment process takes about 2-5 minutes. During this time, Firebase creates the necessary cloud environment. You can view the progress on the dashboard, which also provides the URL for your future app. Accessing the Deployed App The provided URL may not work immediately. It can take another 3-5 minutes for the deployment to be fully complete. Once finished, a green check box appears in the "Rollout status" section. The Next.js project will be live on the Firebase platform. If you are having trouble accessing the project link, try clicking "App Hosting" in the left menu and opening the project link from there. If you want to know how to set up a custom domain for your project, please watch this video - “Firebase custom domain & Free SSL” That’s it! :) If you have any questions, please use the comments section below. Cheers! In this tutorial, I’ll show you how to deploy a full-stack JavaScript web app into Firebase. We’ll deploy a Next.js application to Firebase hosting. I’ll show you a quick and efficient process that can be completed in 5 minutes. https://youtu.be/-VwulR_wTv0?si=nW8dD2VCdhJPFSvg&t=82&embedable=true https://youtu.be/-VwulR_wTv0?si=nW8dD2VCdhJPFSvg&t=82&embedable=true Creating the Next.js Project Let’s create a Next.js project if you don’t have one yet. Create a new Next.js project using the terminal with the npx command. Create a new Next.js project using the terminal with the npx command. npx create-next-app@latest npx create-next-app@latest Push the project to GitHub. This step is crucial as the project will be published directly on Firebase from GitHub. Run the project locally using the "npm run dev" command to ensure everything works correctly. Push the project to GitHub. This step is crucial as the project will be published directly on Firebase from GitHub. Run the project locally using the " npm run dev " command to ensure everything works correctly. npm run dev Setting Up Firebase Go to the Firebase console (https://console.firebase.google.com/) and click the "Add project" link. Name the project e.g. "nextjs-on-firebase" and click a “Continue” button In the next step, choose whether to enable Google Analytics. I’ll turn it off because I don’t need it for this tutorial. After creating the project, navigate to the “App Hosting” section in the left sidebar. Go to the Firebase console (https://console.firebase.google.com/) and click the "Add project" link. Go to the Firebase console ( https://console.firebase.google.com/ ) and click the "Add project" link. https://console.firebase.google.com/ Name the project e.g. "nextjs-on-firebase" and click a “Continue” button Name the project e.g. "nextjs-on-firebase" and click a “Continue” button In the next step, choose whether to enable Google Analytics. I’ll turn it off because I don’t need it for this tutorial. In the next step, choose whether to enable Google Analytics. I’ll turn it off because I don’t need it for this tutorial. After creating the project, navigate to the “App Hosting” section in the left sidebar. After creating the project, navigate to the “App Hosting” section in the left sidebar. Click “Agree” to accept the Google Cloud Platform terms and conditions. Click “Agree” to accept the Google Cloud Platform terms and conditions. The next step will be to upgrade the Firebase plan because hosting for full-stack applications is not free. Upgrading to Blaze Plan To continue, you must upgrade to the Blaze plan (pay-as-you-go). To upgrade the plan, click on the “Upgrade project” button. Fill in all the necessary details. If everything is okay, then you will see the screen below. After upgrading, click the " Get Started " button. Get Started At the next screen, you need to connect your GitHub account and project to Firebase. This is an essential step because we will use GitHub as a project source. Connect your GitHub account to Firebase. Select your project from your GitHub repositories. Choose the main branch (e.g., "master") for deployment. Keep the automatic deployment toggle on for updates to Firebase when changes are pushed to the master branch. Name the backend project (e.g., "njs-firebase") and click "Finish and deploy." Connect your GitHub account to Firebase. Connect your GitHub account to Firebase. Select your project from your GitHub repositories. Select your project from your GitHub repositories. Choose the main branch (e.g., "master") for deployment. Choose the main branch (e.g., "master") for deployment. Keep the automatic deployment toggle on for updates to Firebase when changes are pushed to the master branch. Keep the automatic deployment toggle on for updates to Firebase when changes are pushed to the master branch. Name the backend project (e.g., "njs-firebase") and click "Finish and deploy." Name the backend project (e.g., "njs-firebase") and click "Finish and deploy." If you get lost, you can watch this video: https://youtu.be/-VwulR_wTv0?si=nW8dD2VCdhJPFSvg&t=82&embedable=true https://youtu.be/-VwulR_wTv0?si=nW8dD2VCdhJPFSvg&t=82&embedable=true Waiting for Deployment The deployment process takes about 2-5 minutes. During this time, Firebase creates the necessary cloud environment. You can view the progress on the dashboard, which also provides the URL for your future app. Accessing the Deployed App The provided URL may not work immediately. It can take another 3-5 minutes for the deployment to be fully complete. Once finished, a green check box appears in the "Rollout status" section. The Next.js project will be live on the Firebase platform. If you are having trouble accessing the project link, try clicking " App Hosting " in the left menu and opening the project link from there. App Hosting If you want to know how to set up a custom domain for your project, please watch this video - “ Firebase custom domain & Free SSL ” Firebase custom domain & Free SSL That’s it! :) If you have any questions, please use the comments section below. Cheers!

The Step-by-Step Guide to Deploying Your Next.js App to Firebase Hosting

So You Want To Be On A Startup Board of Directors

If something is valuable, you need to protect it with trustworthy shields. This holds true for both tangible and digital assets. With hackers and malicious software constantly weaving their way through the web, cybersecurity specialists are on a mission to develop next-gen hazard mitigation methods to keep digital threats at bay. Durga Prasada Rao Sanagana, the lead cybersecurity architect at a renowned financial organization, is among the tech experts stepping up to combat cyber threats with advanced firewall defenses. “As technology for good evolves, so does ill-intentioned technology. Threats evolve faster than ever, so our defenses must keep pace, anticipate the next move, and outsmart the adversary,” says Sanagana, reflecting on the importance of revamping cybersecurity ahead of the curve. The North Star of Cybersecurity Innovation Thanks to his significant career strides, Sanagana has become a notable figure in cybersecurity. As a lead security architect for a major financial services company, Sanagana focuses on safeguarding digital assets and developing new defense methods. Applying his forte to real-life situations, Sanagana has built specialization in a myriad of sectors. This includes cybersecurity architecture and engineering, AI, data science, cloud security & architecture, and DevSecOps automation among others. His significant contributions in retail, banking and finance, pharmaceuticals, and healthcare are evident in the solutions he developed for global giants. His work today directly sparks a movement with the next generation of cybersecurity champions. When he is not actively conceptualizing new fundamentals for online safety, he mentors an international faculty development program and guides 140 faculty members, PhD scholars, and industry professionals in building a resilient cybersecurity architecture for data science operations. Passing the baton of knowledge certifies the continued success of the cybersecurity field. Beyond his direct career-related achievements, Sanagana participates actively in the cybersecurity community. That said, he has taken on an off-duty role as a peer reviewer for Springer and IEEE (Institute of Electrical and Electronics Engineers) conferences and regularly judges hackathons and international award events. Sanagana has also put his prowess to paper through notable pieces in Elsevier and JSTAR. Before and After: Perpetual Cybersecurity Growth In 2023, the cost of cybercrime in the United States saw an immense uptick. The increasing frequency and sophistication of cyber threats drove this significant growth. Sanagana's role is crucial in cultivating this aspect of the digital world. His projects surrounding next-gen firewalls, threat intelligence, threat modeling, and intrusion detection systems are fueled by deep learning techniques to predict and mitigate cyber threats. These initiatives offer a significant edge over traditional methods. Moreover, AI is applicable in nearly every industry, and cybersecurity is no exception. Sanagana recognizes the power of AI and stands at the forefront of integrating it into cybersecurity. The architect’s AI-driven adaptive learning systems for business intelligence simulation structures use machine learning and data analytics to enhance threat detection and proactive defense mechanisms. Locked and Loaded: 21st Century Cybersecurity The Internet of Things (IoT) is expanding by the minute, giving cyber threats a more spacious platform to attack. Sanagana envisions a multi-layered cybersecurity method combining AI, blockchain, and quantum computing in a solid attempt to thwart this. As cyber threats grow in complexity, sophisticated defenses are indispensable. Sanagana's work personifies the forward-thinking perspectives required to secure humanity's digital future. “The cyber threats we face in the present are in a constant state of change. The forms they take today may be completely unrecognizable by tomorrow. Our response must be equally innovative, integrating the best of technology and human ingenuity,” he concludes, highlighting his commitment to a safer digital landscape. Building next-gen firewalls that keep assets safe and hackers out; this is the focal point of Durga Prasada Rao Sanagana's work. This piece was published as part of HackerNoon’s Business Blogging Program. For more details, please click here. If something is valuable, you need to protect it with trustworthy shields. This holds true for both tangible and digital assets. With hackers and malicious software constantly weaving their way through the web, cybersecurity specialists are on a mission to develop next-gen hazard mitigation methods to keep digital threats at bay. If something is valuable, you need to protect it with trustworthy shields. This holds true for both tangible and digital assets. With hackers and malicious software constantly weaving their way through the web, cybersecurity specialists are on a mission to develop next-gen hazard mitigation methods to keep digital threats at bay. Durga Prasada Rao Sanagana , the lead cybersecurity architect at a renowned financial organization, is among the tech experts stepping up to combat cyber threats with advanced firewall defenses. Durga Prasada Rao Sanagana Durga Prasada Rao Sanagana “As technology for good evolves, so does ill-intentioned technology. Threats evolve faster than ever, so our defenses must keep pace, anticipate the next move, and outsmart the adversary,” says Sanagana, reflecting on the importance of revamping cybersecurity ahead of the curve. “As technology for good evolves, so does ill-intentioned technology. Threats evolve faster than ever, so our defenses must keep pace, anticipate the next move, and outsmart the adversary,” The North Star of Cybersecurity Innovation The North Star of Cybersecurity Innovation Thanks to his significant career strides, Sanagana has become a notable figure in cybersecurity. As a lead security architect for a major financial services company, Sanagana focuses on safeguarding digital assets and developing new defense methods. Applying his forte to real-life situations, Sanagana has built specialization in a myriad of sectors. This includes cybersecurity architecture and engineering, AI, data science, cloud security & architecture, and DevSecOps automation among others. His significant contributions in retail, banking and finance, pharmaceuticals, and healthcare are evident in the solutions he developed for global giants. His work today directly sparks a movement with the next generation of cybersecurity champions. When he is not actively conceptualizing new fundamentals for online safety, he mentors an international faculty development program and guides 140 faculty members, PhD scholars, and industry professionals in building a resilient cybersecurity architecture for data science operations. Passing the baton of knowledge certifies the continued success of the cybersecurity field. Beyond his direct career-related achievements, Sanagana participates actively in the cybersecurity community. That said, he has taken on an off-duty role as a peer reviewer for Springer and IEEE (Institute of Electrical and Electronics Engineers) conferences and regularly judges hackathons and international award events. Sanagana has also put his prowess to paper through notable pieces in Elsevier and JSTAR. Before and After: Perpetual Cybersecurity Growth Before and After: Perpetual Cybersecurity Growth In 2023, the cost of cybercrime in the United States saw an immense uptick. The increasing frequency and sophistication of cyber threats drove this significant growth. Sanagana's role is crucial in cultivating this aspect of the digital world. His projects surrounding next-gen firewalls, threat intelligence, threat modeling, and intrusion detection systems are fueled by deep learning techniques to predict and mitigate cyber threats. These initiatives offer a significant edge over traditional methods. Moreover, AI is applicable in nearly every industry, and cybersecurity is no exception. Sanagana recognizes the power of AI and stands at the forefront of integrating it into cybersecurity. The architect’s AI-driven adaptive learning systems for business intelligence simulation structures use machine learning and data analytics to enhance threat detection and proactive defense mechanisms. Locked and Loaded: 21st Century Cybersecurity Locked and Loaded: 21st Century Cybersecurity The Internet of Things (IoT) is expanding by the minute, giving cyber threats a more spacious platform to attack. Sanagana envisions a multi-layered cybersecurity method combining AI, blockchain, and quantum computing in a solid attempt to thwart this. As cyber threats grow in complexity, sophisticated defenses are indispensable. Sanagana's work personifies the forward-thinking perspectives required to secure humanity's digital future. “The cyber threats we face in the present are in a constant state of change. The forms they take today may be completely unrecognizable by tomorrow. Our response must be equally innovative, integrating the best of technology and human ingenuity,” he concludes, highlighting his commitment to a safer digital landscape. “The cyber threats we face in the present are in a constant state of change. The forms they take today may be completely unrecognizable by tomorrow. Our response must be equally innovative, integrating the best of technology and human ingenuity,” Building next-gen firewalls that keep assets safe and hackers out; this is the focal point of Durga Prasada Rao Sanagana's work. This piece was published as part of HackerNoon’s Business Blogging Program. For more details, please click here. This piece was published as part of HackerNoon’s Business Blogging Program. For more details, please click here . here

Tech Expert Durga Sanagana Advances Next-Gen Firewalls and Threat Modeling Techniques

টেক এক্সপার্ট দুর্গা সনাগানা নেক্সট-জেন ফায়ারওয়াল এবং থ্রেট মডেলিং কৌশলকে অগ্রসর করেছে

Tech-Expertin Durga Sanagana entwickelt Firewalls und Bedrohungsmodellierungstechniken der nächsten Generation weiter

El experto en tecnología Durga Sanagana avanza en las técnicas de modelado de amenazas y firewalls de próxima generación

L'expert technique Durga Sanagana fait progresser les pare-feu de nouvelle génération et les techniques de modélisation des menaces

तकनीकी विशेषज्ञ दुर्गा सनागना ने अगली पीढ़ी की फायरवॉल और खतरा मॉडलिंग तकनीक विकसित की

技術専門家のドゥルガ・サナガナが次世代ファイアウォールと脅威モデリング技術を進化させる

기술 전문가 Durga Sanagana, 차세대 방화벽 및 위협 모델링 기술 발전

A especialista em tecnologia Durga Sanagana desenvolve firewalls de última geração e técnicas de modelagem de ameaças

Технический эксперт Дурга Санагана продвигает межсетевые экраны нового поколения и методы моделирования угроз

Teknoloji Uzmanı Durga Sanagana Yeni Nesil Güvenlik Duvarlarını ve Tehdit Modelleme Tekniklerini Geliştiriyor

Chuyên gia công nghệ Durga Sanagana nâng cao kỹ thuật mô hình hóa mối đe dọa và tường lửa thế hệ tiếp theo

技术专家 Durga Sanagana 推进下一代防火墙和威胁建模技术

MetaMask Swaps Platform Under Fire for Allegedly Operating Without Broker Registration

Last week, the SEC filed a suit against Consensys, the developer of the popular MetaMask wallet. The SEC claims that Consensys, which “markets itself as a leader and innovator in the crypto asset industry”, was offering products that performed age-old functions. These products include brokering securities transactions for retail investors and engaging in the offer and sale of securities. According to the SEC, Consensys should have registered as a broker, as well as registered the offer and sale of certain securities. The SEC is focused on going after MetaMask Swaps and MetaMask Staking services. This is not the first time the SEC has accused staking services of violating securities laws. Back in 2023, the SEC sued Coinbase and Kraken. Meanwhile, the well-known case against Binance involved multiple allegations and ultimately put Binance’s founder CZ in jail. Kraken’s case is illustrative. The management of the crypto exchange decided that it was futile to oppose the regulator. As a result, Kraken stopped offering its staking-as-a-service program for U.S. users. The service is still available for customers in other countries. Kraken’s settlement with the SEC also involved payment of $30 million in disgorgement, prejudgment interest, and civil penalties. Apparently, Consensys believes that it is in a better position compared to Binance or Kraken. The company has previously sued the SEC in a preventive effort to stop a potential suit against Consensys. As we know now, this attempt has failed, and the regulator proceeded with its own lawsuit against the company. Consensys' lawsuit centered on the classification of ETH, arguing that it should be recognized as a commodity rather than a security. The company emphasized that Ethereum is a global computing platform, not an investment scheme. Consensys expressed concerns about the potential designation of ETH as a security, given MetaMask's pivotal role in the ETH ecosystem. Taking a look at last year’s cases of Binance, Coinbase, and Kraken, it is obvious that the SEC does not plan to destroy the whole industry or any single coin. Such a plan is impossible both technically and politically as the crypto industry has gained strong momentum and coins have found their place in portfolios of multiple investors of any level, from retail guys to institutional pros. The regulator uses lawsuits to get more control over the industry and slow down its development. However, things are not always going in the SEC's favor. In the case of SEC vs Ripple, the regulator has mostly lost the case. The court found out that Ripple violated securities laws when selling XRP to institutional investors but dismissed other charges brought by the regulator. In this light, Consensys should have a decent chance for a long and complicated court battle against the SEC. Potentially, the company may try to wait for the changes in the SEC leadership, hoping that new people will be more crypto friendly. Talking about ETH, its holders have nothing to worry about. As usual, the case might generate some FUD on Ethereum, which is typical for the industry. Last week, the SEC filed a suit against Consensys, the developer of the popular MetaMask wallet. The SEC claims that Consensys, which “markets itself as a leader and innovator in the crypto asset industry”, was offering products that performed age-old functions. claims claims These products include brokering securities transactions for retail investors and engaging in the offer and sale of securities. According to the SEC, Consensys should have registered as a broker, as well as registered the offer and sale of certain securities. The SEC is focused on going after MetaMask Swaps and MetaMask Staking services. This is not the first time the SEC has accused staking services of violating securities laws. Back in 2023, the SEC sued Coinbase and Kraken. Meanwhile, the well-known case against Binance involved multiple allegations and ultimately put Binance’s founder CZ in jail. Kraken’s case is illustrative. The management of the crypto exchange decided that it was futile to oppose the regulator. As a result, Kraken stopped offering its staking-as-a-service program for U.S. users. The service is still available for customers in other countries. Kraken’s settlement with the SEC also involved payment of $30 million in disgorgement, prejudgment interest, and civil penalties. case Apparently, Consensys believes that it is in a better position compared to Binance or Kraken. The company has previously sued the SEC in a preventive effort to stop a potential suit against Consensys. As we know now, this attempt has failed, and the regulator proceeded with its own lawsuit against the company. sued sued Consensys' lawsuit centered on the classification of ETH, arguing that it should be recognized as a commodity rather than a security. The company emphasized that Ethereum is a global computing platform, not an investment scheme. Consensys expressed concerns about the potential designation of ETH as a security, given MetaMask's pivotal role in the ETH ecosystem. Taking a look at last year’s cases of Binance, Coinbase, and Kraken, it is obvious that the SEC does not plan to destroy the whole industry or any single coin. Such a plan is impossible both technically and politically as the crypto industry has gained strong momentum and coins have found their place in portfolios of multiple investors of any level, from retail guys to institutional pros. The regulator uses lawsuits to get more control over the industry and slow down its development. However, things are not always going in the SEC's favor. In the case of SEC vs Ripple , the regulator has mostly lost the case. The court found out that Ripple violated securities laws when selling XRP to institutional investors but dismissed other charges brought by the regulator. SEC vs Ripple In this light, Consensys should have a decent chance for a long and complicated court battle against the SEC. Potentially, the company may try to wait for the changes in the SEC leadership, hoping that new people will be more crypto friendly. Talking about ETH, its holders have nothing to worry about. As usual, the case might generate some FUD on Ethereum, which is typical for the industry.

SEC vs MetaMask: Is Staking Is Under Fire Again?

Authors:
(1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co);
(2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team;
(3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction
Related Work
Method
Experimental Details
Results and Ablations
Conclusions and Limitations , Acknowledgements and References
Appendix 3 METHOD The goal of this work is to align an open-source large-language model to the intent of the user. Throughout the work we assume access to a larger teacher model πT which can be queried by prompted generation. Our goal is be to produce a student model πθ and our approach follows similar stages as InstructGPT (Ouyang et al., 2022) as shown in Figure 2. Distilled Supervised Fine-Tuning (dSFT) Starting with a raw LLM, we first need to train it to respond to user prompts. This step is traditionally done through supervised fine tuning (SFT) on a dataset of high-quality instructions and responses (Chung et al., 2022; Sanh et al., 2021). Given access to a teacher language models, we can instead have the model generate instructions and responses (Taori et al., 2023), and train the model directly on these. We refer to this as distilled SFT (dSFT). Approaches to dSFT follow the self-instruct protocol (Wang et al., 2023). Let x 0 1 , . . . , x0 J be a set of seed prompts, constructed to represent a diverse set of topical domains. A dataset is constructed through iterative self-prompting where the teacher is used to both respond to an instruction and refine the instruction based on the response. For each x 0 , we first sample response y 0 ∼ πT(·|x 0 ), and then refine by sampling a new instruction (using a prompt for refinement), x 1 ∼ πT(·|x 0 , y0 ). The end point is a final dataset, C = {(x1, y1), . . . ,(xJ , yJ )}. Distillation is performed by SFT, AI Feedback through Preferences (AIF) Human feedback (HF) can provide additional signal to align LLMs. Human feedback is typically given through preferences on the quality of LLM responses (Ouyang et al., 2022). For distillation, we instead use AI preferences from the teacher model on generated outputs from other models. We follow the approach of UltraFeedback (Cui et al., 2023) which uses the teacher to provide preferences on model outputs. As with SFT, the system starts with a set of prompts x1, . . . , xJ . Each prompt x is fed to a collection of four models π1, . . . , π4, e.g. Claude, Falcon, Llama, etc, each of which yield a response y 1 ∼ π1(·|x), . . . , y4 ∼ π4(·|x). These responses are then fed to the teacher model, e.g. GPT-4, which gives a score for the response s 1 ∼ πT (·|x, y1 ), . . . , s4 ∼ πT (·|x, y4 ). After collecting the scores for a prompt x, we save the highest scoring response as yw and a random lower scoring prompt as yl . The final feedback dataset D consists of a set of these triples (x, yw, yl). Distilled Direct Preference Optimization (dDPO) The goal of the final step is to refine the πdSFT by maximizing the likelihood of ranking the preferred yw over yl in a preference model. The preference model is determined by a reward function rθ(x, y) which utilizes the student language model πθ. Past work using AI feedback has primarily focused on using RL methods such as proximal policy optimization (PPO) to optimize θ with respect to this reward. These approaches optimize θ by first training the reward and then sampling from the current policy to compute updates. Direct preference optimization (DPO) uses a simpler approach to directly optimize the preference model from the static data (Rafailov et al., 2023). The key observation is to derive the optimal reward function in terms of the optimal LLM policy π∗ and the original LLM policy πdSFT. Under an appropriate choice of preference model they show, for constant β and partition function Z that, By plugging this function of the reward into the preference model, the authors show that the objective can be written as, While this term looks complex, we note that it implies a simple training procedure. Starting with the dSFT version of the model, we iterate through each AIF triple (x, yw, yl). 1. Compute the probability for (x, yw) and (x, yl) from the dSFT model (forward-only). 2. Compute the probability for (x, yw) and (x, yl) from the dDPO model. 3. Compute Eq 1 and backpropagate to update. Repeat. This paper is available on arxiv under CC 4.0 license. Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Authors: Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction Related Work Method Experimental Details Results and Ablations Conclusions and Limitations , Acknowledgements and References Appendix Abstract and Introduction Abstract and Introduction Related Work Related Work Method Method Experimental Details Experimental Details Results and Ablations Results and Ablations Conclusions and Limitations , Acknowledgements and References Conclusions and Limitations , Acknowledgements and References Appendix Appendix 3 METHOD The goal of this work is to align an open-source large-language model to the intent of the user. Throughout the work we assume access to a larger teacher model πT which can be queried by prompted generation. Our goal is be to produce a student model πθ and our approach follows similar stages as InstructGPT (Ouyang et al., 2022) as shown in Figure 2. Distilled Supervised Fine-Tuning (dSFT) Starting with a raw LLM, we first need to train it to respond to user prompts. This step is traditionally done through supervised fine tuning (SFT) on a dataset of high-quality instructions and responses (Chung et al., 2022; Sanh et al., 2021). Given access to a teacher language models, we can instead have the model generate instructions and responses (Taori et al., 2023), and train the model directly on these. We refer to this as distilled SFT (dSFT). Distilled Supervised Fine-Tuning (dSFT) Approaches to dSFT follow the self-instruct protocol (Wang et al., 2023). Let x 0 1 , . . . , x0 J be a set of seed prompts, constructed to represent a diverse set of topical domains. A dataset is constructed through iterative self-prompting where the teacher is used to both respond to an instruction and refine the instruction based on the response. For each x 0 , we first sample response y 0 ∼ πT(·|x 0 ), and then refine by sampling a new instruction (using a prompt for refinement), x 1 ∼ πT(·|x 0 , y0 ). The end point is a final dataset, C = {(x1, y1), . . . ,(xJ , yJ )}. Distillation is performed by SFT, AI Feedback through Preferences (AIF) Human feedback (HF) can provide additional signal to align LLMs. Human feedback is typically given through preferences on the quality of LLM responses (Ouyang et al., 2022). For distillation, we instead use AI preferences from the teacher model on generated outputs from other models. AI Feedback through Preferences (AIF) We follow the approach of UltraFeedback (Cui et al., 2023) which uses the teacher to provide preferences on model outputs. As with SFT, the system starts with a set of prompts x1, . . . , xJ . Each prompt x is fed to a collection of four models π1, . . . , π4, e.g. Claude, Falcon, Llama, etc, each of which yield a response y 1 ∼ π1(·|x), . . . , y4 ∼ π4(·|x). These responses are then fed to the teacher model, e.g. GPT-4, which gives a score for the response s 1 ∼ πT (·|x, y1 ), . . . , s4 ∼ πT (·|x, y4 ). After collecting the scores for a prompt x, we save the highest scoring response as yw and a random lower scoring prompt as yl . The final feedback dataset D consists of a set of these triples (x, yw, yl). Distilled Direct Preference Optimization (dDPO) The goal of the final step is to refine the πdSFT by maximizing the likelihood of ranking the preferred yw over yl in a preference model. The preference model is determined by a reward function rθ(x, y) which utilizes the student language model πθ. Past work using AI feedback has primarily focused on using RL methods such as proximal policy optimization (PPO) to optimize θ with respect to this reward. These approaches optimize θ by first training the reward and then sampling from the current policy to compute updates. Distilled Direct Preference Optimization (dDPO) Direct preference optimization (DPO) uses a simpler approach to directly optimize the preference model from the static data (Rafailov et al., 2023). The key observation is to derive the optimal reward function in terms of the optimal LLM policy π∗ and the original LLM policy πdSFT. Under an appropriate choice of preference model they show, for constant β and partition function Z that, By plugging this function of the reward into the preference model, the authors show that the objective can be written as, While this term looks complex, we note that it implies a simple training procedure. Starting with the dSFT version of the model, we iterate through each AIF triple (x, yw, yl). 1. Compute the probability for (x, yw) and (x, yl) from the dSFT model (forward-only). 2. Compute the probability for (x, yw) and (x, yl) from the dDPO model. 3. Compute Eq 1 and backpropagate to update. Repeat. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Zephyr: Direct Distillation of LM Alignment: Method

Authors:
(1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co);
(2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team;
(3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction
Related Work
Method
Experimental Details
Results and Ablations
Conclusions and Limitations , Acknowledgements and References
Appendix ABSTRACT We aim to produce a smaller language model that is aligned to user intent. Previous research has shown that applying distilled supervised fine-tuning (dSFT) on larger models significantly improves task accuracy; however, these models are unaligned, i.e. they do not respond well to natural prompts. To distill this property, we experiment with the use of preference data from AI Feedback (AIF). Starting from a dataset of outputs ranked by a teacher model, we apply distilled direct preference optimization (dDPO) to learn a chat model with significantly improved intent alignment. The approach requires only a few hours of training without any additional sampling during fine-tuning. The final result, ZEPHYR7B, sets a new state-of-the-art on chat benchmarks for 7B parameter models, and requires no human annotation. In particular, results on MT-Bench show that ZEPHYR-7B surpasses LLAMA2-CHAT-70B, the best open-access RLHFbased model. Code, models, data, and tutorials for the system are available at https://github.com/huggingface/alignment-handbook. 1 INTRODUCTION Smaller, open large language models (LLMs) have greatly increased in ability in recent years, from early GPT-2-like models (Wang & Komatsuzaki, 2021) to accurate and compact models (Touvron et al., 2023; Penedo et al., 2023; Jiang et al., 2023) that are trained on significantly more tokens than the “compute-optimal” amount suggested by the Chincilla scaling laws (De Vries, 2023). In addition, researchers have shown that these models can be further trained through distilled supervised fine-tuning (dSFT) based on proprietary models to increase their accuracy (Taori et al., 2023). In this approach, the output of a more capable teacher model is used as supervised data for the student model. Distillation has proven to be an effective tool for improving open models on a range of different tasks (Chiang et al., 2023); however, it does not reach the performance of the teacher models (Gudibande et al., 2023). Users have noted that these models are not “intent aligned”, i.e. they do not behave in a manner that aligns with human users’ preferences. This property often leads to outputs that do not provide correct responses to queries. Intention alignment has been difficult to quantify, but recent work has led to the development of benchmarks like MT-Bench (Zheng et al., 2023) and AlpacaEval (Li et al., 2023) that specifically target this behavior. These benchmarks yield scores that correlate closely with human ratings of model outputs and confirm the qualitative intuition that proprietary models perform better than open models trained with human feedback, which in turn perform better than open models trained with distillation. This motivates careful collection of human feedback for alignment, often at enormous cost at scale, such as in LLAMA2-CHAT (Touvron et al., 2023). In this work, we consider the problem of aligning a small open LLM entirely through distillation. The main step is to utilize AI Feedback (AIF) from an ensemble of teacher models as preference data, and apply distilled direct preference optimization as the learning objective (Rafailov et al., 2023). We refer to this approach as dDPO. Notably, it requires no human annotation and no sampling compared to using other approaches like proximal preference optimization (PPO) (Schulman et al., 2017). Moreover, by utilizing a small base LM, the resulting chat model can be trained in a matter of hours on 16 A100s (80GB). To validate this approach, we construct ZEPHYR-7B, an aligned version of Mistral-7B (Jiang et al., 2023). We first use dSFT, based on the UltraChat (Ding et al., 2023) dataset. Next we use the AI feedback data collected in the UltraFeedback dataset (Cui et al., 2023). Finally, we apply dDPO based on this feedback data. Experiments show that this 7B parameter model can achieve performance comparable to 70B-parameter chat models aligned with human feedback. Results show improvements both in terms of standard academic benchmarks as well as benchmarks that take into account conversational capabilities. Analysis shows that the use of preference learning is critical in achieving these results. Models, code, and instructions are available at https://github.com/huggingface/alignment-handbook. We note an important caveat for these results. We are primarily concerned with intent alignment of models for helpfulness. The work does not consider safety considerations of the models, such as whether they produce harmful outputs or provide illegal advice (Bai et al., 2022). As distillation only works with the output of publicly available models this is technically more challenging to do because of added challenges in curating that type of synthetic data, and is an important subject for future work. This paper is available on arxiv under CC 4.0 license. Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Authors: Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction Related Work Method Experimental Details Results and Ablations Conclusions and Limitations , Acknowledgements and References Appendix Abstract and Introduction Abstract and Introduction Related Work Related Work Method Method Experimental Details Experimental Details Results and Ablations Results and Ablations Conclusions and Limitations , Acknowledgements and References Conclusions and Limitations , Acknowledgements and References Appendix Appendix ABSTRACT We aim to produce a smaller language model that is aligned to user intent. Previous research has shown that applying distilled supervised fine-tuning (dSFT) on larger models significantly improves task accuracy; however, these models are unaligned, i.e. they do not respond well to natural prompts. To distill this property, we experiment with the use of preference data from AI Feedback (AIF). Starting from a dataset of outputs ranked by a teacher model, we apply distilled direct preference optimization (dDPO) to learn a chat model with significantly improved intent alignment. The approach requires only a few hours of training without any additional sampling during fine-tuning. The final result, ZEPHYR7B, sets a new state-of-the-art on chat benchmarks for 7B parameter models, and requires no human annotation. In particular, results on MT-Bench show that ZEPHYR-7B surpasses LLAMA2-CHAT-70B, the best open-access RLHFbased model. Code, models, data, and tutorials for the system are available at https://github.com/huggingface/alignment-handbook. 1 INTRODUCTION Smaller, open large language models (LLMs) have greatly increased in ability in recent years, from early GPT-2-like models (Wang & Komatsuzaki, 2021) to accurate and compact models (Touvron et al., 2023; Penedo et al., 2023; Jiang et al., 2023) that are trained on significantly more tokens than the “compute-optimal” amount suggested by the Chincilla scaling laws (De Vries, 2023). In addition, researchers have shown that these models can be further trained through distilled supervised fine-tuning (dSFT) based on proprietary models to increase their accuracy (Taori et al., 2023). In this approach, the output of a more capable teacher model is used as supervised data for the student model. Distillation has proven to be an effective tool for improving open models on a range of different tasks (Chiang et al., 2023); however, it does not reach the performance of the teacher models (Gudibande et al., 2023). Users have noted that these models are not “intent aligned”, i.e. they do not behave in a manner that aligns with human users’ preferences. This property often leads to outputs that do not provide correct responses to queries. Intention alignment has been difficult to quantify, but recent work has led to the development of benchmarks like MT-Bench (Zheng et al., 2023) and AlpacaEval (Li et al., 2023) that specifically target this behavior. These benchmarks yield scores that correlate closely with human ratings of model outputs and confirm the qualitative intuition that proprietary models perform better than open models trained with human feedback, which in turn perform better than open models trained with distillation. This motivates careful collection of human feedback for alignment, often at enormous cost at scale, such as in LLAMA2-CHAT (Touvron et al., 2023). In this work, we consider the problem of aligning a small open LLM entirely through distillation. The main step is to utilize AI Feedback (AIF) from an ensemble of teacher models as preference data, and apply distilled direct preference optimization as the learning objective (Rafailov et al., 2023). We refer to this approach as dDPO. Notably, it requires no human annotation and no sampling compared to using other approaches like proximal preference optimization (PPO) (Schulman et al., 2017). Moreover, by utilizing a small base LM, the resulting chat model can be trained in a matter of hours on 16 A100s (80GB). To validate this approach, we construct ZEPHYR-7B, an aligned version of Mistral-7B (Jiang et al., 2023). We first use dSFT, based on the UltraChat (Ding et al., 2023) dataset. Next we use the AI feedback data collected in the UltraFeedback dataset (Cui et al., 2023). Finally, we apply dDPO based on this feedback data. Experiments show that this 7B parameter model can achieve performance comparable to 70B-parameter chat models aligned with human feedback. Results show improvements both in terms of standard academic benchmarks as well as benchmarks that take into account conversational capabilities. Analysis shows that the use of preference learning is critical in achieving these results. Models, code, and instructions are available at https://github.com/huggingface/alignment-handbook. We note an important caveat for these results. We are primarily concerned with intent alignment of models for helpfulness. The work does not consider safety considerations of the models, such as whether they produce harmful outputs or provide illegal advice (Bai et al., 2022). As distillation only works with the output of publicly available models this is technically more challenging to do because of added challenges in curating that type of synthetic data, and is an important subject for future work. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Zephyr: Direct Distillation of LM Alignment: Abstract and Introduction

Authors:
(1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co);
(2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team;
(3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction
Related Work
Method
Experimental Details
Results and Ablations
Conclusions and Limitations , Acknowledgements and References
Appendix 4 EXPERIMENTAL DETAILS We conduct all of our fine-tuning experiments using Mistral 7B (Jiang et al., 2023), which is the current state-of-the-art base LM at the 7B parameter scale, and matches the performance of much larger models like LLaMa 34B on many NLP benchmarks. We use the Transformer Reinforcement Learning (TRL) library for fine-tuning (von Werra et al., 2020), in conjunction with DeepSpeed ZeRO3 (Rajbhandari et al., 2020) and FlashAttention-2 (Dao, 2023) to optimize memory and improve training speed. All models are trained with the AdamW optimizer and no weight decay. We did not experiment with parameter-efficient techniques such as LoRA (Hu et al., 2021), but expect similar results to hold with these methods. All experiments were run on 16 A100s using bfloat16 precision and typically took 2-4 hours to complete. For the full set of hyperparameters and instructions on how to train the models, see: https://github.com/huggingface/alignment-handbook. 4.1 DATASETS We focus on two dialogue datasets that have been distilled from a mix of open and proprietary models, and have previously been shown to produce strong chat models like the UltraLM (Ding et al., 2023): • UltraChat (Ding et al., 2023) is a self-refinement dataset consisting of 1.47M multi-turn dialogues generated by GPT-3.5-TURBO over 30 topics and 20 different types of text material. We initially ran dSFT over the whole corpus, but found the resulting chat model had a tendency to respond with incorrect capitalization and would preface its answers with phrases such as “I don’t have personal experiences”, even for straightforward questions like “How do I clean my car?”. To handle these issues in the training data, we applied truecasing heuristics to fix the grammatical errors (approximately 5% of the dataset), as well as several filters to focus on helpfulness and remove the undesired model responses. The resulting dataset contains approximately 200k examples. • UltraFeedback (Cui et al., 2023) consists of 64k prompts, each of which have four LLM responses that are rated by GPT-4 according to criteria like instruction-following, honesty, and helpfulness. We construct binary preferences from UltraFeedback by selecting the highest mean score as the “chosen” response and one of the remaining three at random as “rejected”. We opted for random selection instead of selecting the lowest-scored response to encourage diversity and make the DPO objective more challenging. As noted above, this step is computed offline and does not involve any sampling from the reference model. We make the pre-processed datasets available on the Hugging Face Hub.1 4.2 EVALUATION Our main evaluations are on single-turn and multi-turn chat benchmarks that measure a model’s ability to follow instructions and respond to challenging prompts across a diverse range of domains: • MT-Bench (Zheng et al., 2023) is a multi-turn benchmark that consists of 160 questions across eight different areas of knowledge. In this benchmark, the model must answer an initial question, and then provide a second response to a predefined followup question. Each model response is then rated by GPT-4 on a scale from 1-10, with the final score given by the mean over the two turns. • AlpacaEval (Li et al., 2023) is a single-turn benchmark where a model must generate a response to 805 questions on different topics, mostly focused on helpfulness. Models are also scored by GPT-4, but the final metric is the pairwise win-rate against a baseline model (text-davinci-003). We also evaluate ZEPHYR-7B on the Open LLM Leaderboard (Beeching et al., 2023), which measures the performance of LMs across four multiclass classification tasks: ARC (Clark et al., 2018), HellaSwag (Zellers et al., 2019), MMLU (Hendrycks et al., 2021), and Truthful QA(Lin et al., 2022). Although this leaderboard does not directly measure the conversational quality of chat models, it does provide a useful signal to validate whether fine-tuning has introduced regressions on the base model’s reasoning and truthfulness capabilities. Across all benchmarks, we compare ZEPHYR-7B against a variety of open and proprietary models, each with different alignment procedures. To facilitate comparison across open model sizes, we group our comparisons in terms of 7B models (XWIN-LM (Team, 2023), MISTRALINSTRUCT (Jiang et al., 2023), MPT-CHAT (ML, 2023), and STABLELM-α), as well as larger models up to 70B parameters (LLAMA2-CHAT (Touvron et al., 2023), VICUNA˜ (Chiang et al., 2023), WizardLM (Xu et al.), and GUANACO (Dettmers et al., 2023)). For the chat benchmarks, we also compare against proprietary models, including CLAUDE 2, GPT-3.5-TURBO and GPT-4 (OpenAI, 2023). 4.3 DETAILS OF SFT TRAINING We train our SFT models for one to three epochs. We use a cosine learning rate scheduler with a peak learning rate of 2e-5 and 10% warmup steps. We train all models with a global batch size of 512 and use packing with a sequence length of 2048 tokens. 4.4 DETAILS OF DPO TRAINING Similar to SFT, we train our DPO models for one to three epochs. We use a linear learning rate scheduler with a peak learning rate of 5e-7 and 10% warmup steps. We train all models with a global batch size of 32 and use β = 0.1 from Eq. (1) to control the deviation from the reference model. The final ZEPHYR-7B model was initialized from the SFT model that was trained for one epoch and further optimized for three DPO epochs (see Figure 3 for an epoch ablation on MT-Bench). This paper is available on arxiv under CC 4.0 license. 1 https://huggingface.co/collections/HuggingFaceH4/ zephyr-7b-6538c6d6d5ddd1cbb1744a66 Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Authors: Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction Related Work Method Experimental Details Results and Ablations Conclusions and Limitations , Acknowledgements and References Appendix Abstract and Introduction Abstract and Introduction Related Work Related Work Method Method Experimental Details Experimental Details Results and Ablations Results and Ablations Conclusions and Limitations , Acknowledgements and References Conclusions and Limitations , Acknowledgements and References Appendix Appendix 4 EXPERIMENTAL DETAILS We conduct all of our fine-tuning experiments using Mistral 7B (Jiang et al., 2023), which is the current state-of-the-art base LM at the 7B parameter scale, and matches the performance of much larger models like LLaMa 34B on many NLP benchmarks. We use the Transformer Reinforcement Learning (TRL) library for fine-tuning (von Werra et al., 2020), in conjunction with DeepSpeed ZeRO3 (Rajbhandari et al., 2020) and FlashAttention-2 (Dao, 2023) to optimize memory and improve training speed. All models are trained with the AdamW optimizer and no weight decay. We did not experiment with parameter-efficient techniques such as LoRA (Hu et al., 2021), but expect similar results to hold with these methods. All experiments were run on 16 A100s using bfloat16 precision and typically took 2-4 hours to complete. For the full set of hyperparameters and instructions on how to train the models, see: https://github.com/huggingface/alignment-handbook. 4.1 DATASETS We focus on two dialogue datasets that have been distilled from a mix of open and proprietary models, and have previously been shown to produce strong chat models like the UltraLM (Ding et al., 2023): • UltraChat (Ding et al., 2023) is a self-refinement dataset consisting of 1.47M multi-turn dialogues generated by GPT-3.5-TURBO over 30 topics and 20 different types of text material. We initially ran dSFT over the whole corpus, but found the resulting chat model had a tendency to respond with incorrect capitalization and would preface its answers with phrases such as “I don’t have personal experiences”, even for straightforward questions like “How do I clean my car?”. To handle these issues in the training data, we applied truecasing heuristics to fix the grammatical errors (approximately 5% of the dataset), as well as several filters to focus on helpfulness and remove the undesired model responses. The resulting dataset contains approximately 200k examples. • UltraChat • UltraFeedback (Cui et al., 2023) consists of 64k prompts, each of which have four LLM responses that are rated by GPT-4 according to criteria like instruction-following, honesty, and helpfulness. We construct binary preferences from UltraFeedback by selecting the highest mean score as the “chosen” response and one of the remaining three at random as “rejected”. We opted for random selection instead of selecting the lowest-scored response to encourage diversity and make the DPO objective more challenging. As noted above, this step is computed offline and does not involve any sampling from the reference model. • UltraFeedback We make the pre-processed datasets available on the Hugging Face Hub.1 4.2 EVALUATION Our main evaluations are on single-turn and multi-turn chat benchmarks that measure a model’s ability to follow instructions and respond to challenging prompts across a diverse range of domains: • MT-Bench (Zheng et al., 2023) is a multi-turn benchmark that consists of 160 questions across eight different areas of knowledge. In this benchmark, the model must answer an initial question, and then provide a second response to a predefined followup question. Each model response is then rated by GPT-4 on a scale from 1-10, with the final score given by the mean over the two turns. • MT-Bench • AlpacaEval (Li et al., 2023) is a single-turn benchmark where a model must generate a response to 805 questions on different topics, mostly focused on helpfulness. Models are also scored by GPT-4, but the final metric is the pairwise win-rate against a baseline model (text-davinci-003). • AlpacaEval We also evaluate ZEPHYR-7B on the Open LLM Leaderboard (Beeching et al., 2023), which measures the performance of LMs across four multiclass classification tasks: ARC (Clark et al., 2018), HellaSwag (Zellers et al., 2019), MMLU (Hendrycks et al., 2021), and Truthful QA(Lin et al., 2022). Although this leaderboard does not directly measure the conversational quality of chat models, it does provide a useful signal to validate whether fine-tuning has introduced regressions on the base model’s reasoning and truthfulness capabilities. Across all benchmarks, we compare ZEPHYR-7B against a variety of open and proprietary models, each with different alignment procedures. To facilitate comparison across open model sizes, we group our comparisons in terms of 7B models (XWIN-LM (Team, 2023), MISTRALINSTRUCT (Jiang et al., 2023), MPT-CHAT (ML, 2023), and STABLELM-α), as well as larger models up to 70B parameters (LLAMA2-CHAT (Touvron et al., 2023), VICUNA˜ (Chiang et al., 2023), WizardLM (Xu et al.), and GUANACO (Dettmers et al., 2023)). For the chat benchmarks, we also compare against proprietary models, including CLAUDE 2, GPT-3.5-TURBO and GPT-4 (OpenAI, 2023). 4.3 DETAILS OF SFT TRAINING We train our SFT models for one to three epochs. We use a cosine learning rate scheduler with a peak learning rate of 2e-5 and 10% warmup steps. We train all models with a global batch size of 512 and use packing with a sequence length of 2048 tokens. 4.4 DETAILS OF DPO TRAINING Similar to SFT, we train our DPO models for one to three epochs. We use a linear learning rate scheduler with a peak learning rate of 5e-7 and 10% warmup steps. We train all models with a global batch size of 32 and use β = 0.1 from Eq. (1) to control the deviation from the reference model. The final ZEPHYR-7B model was initialized from the SFT model that was trained for one epoch and further optimized for three DPO epochs (see Figure 3 for an epoch ablation on MT-Bench). This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv 1 https://huggingface.co/collections/HuggingFaceH4/ zephyr-7b-6538c6d6d5ddd1cbb1744a66

Zephyr: Direct Distillation of LM Alignment: Experimental Details

Authors:
(1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co);
(2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team;
(3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction
Related Work
Method
Experimental Details
Results and Ablations
Conclusions and Limitations , Acknowledgements and References
Appendix A APPENDIX A.1 QUALITATIVE EXAMPLES To qualitatively compare the responses from our dSFT and dDPO models, we choose prompts from a few domains of MT-Bench, as well as some adversarial prompts to test each model’s capability to follow instructions with false premises or harmful intent. Completions for the adversarial prompts were generated with nucleus sampling(top-p = 0.95) and T = 0.7. A.2 SFT IS A REQUIRED STEP BEFORE DPO In Table 3 we ran an ablation to see whether SFT is necessary prior to the DPO step. We observed a significant reduction in performance in both the MT-Bench and AlpacaEval scores when the SFT step is skipped. After a qualitative evaluation of the MT-Bench generations, we observe that the pure DPO model struggles to learn the chat template: This paper is available on arxiv under CC 4.0 license. Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Authors: Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction Related Work Method Experimental Details Results and Ablations Conclusions and Limitations , Acknowledgements and References Appendix Abstract and Introduction Abstract and Introduction Related Work Related Work Method Method Experimental Details Experimental Details Results and Ablations Results and Ablations Conclusions and Limitations , Acknowledgements and References Conclusions and Limitations , Acknowledgements and References Appendix Appendix A APPENDIX A.1 QUALITATIVE EXAMPLES To qualitatively compare the responses from our dSFT and dDPO models, we choose prompts from a few domains of MT-Bench, as well as some adversarial prompts to test each model’s capability to follow instructions with false premises or harmful intent. Completions for the adversarial prompts were generated with nucleus sampling(top-p = 0.95) and T = 0.7. A.2 SFT IS A REQUIRED STEP BEFORE DPO In Table 3 we ran an ablation to see whether SFT is necessary prior to the DPO step. We observed a significant reduction in performance in both the MT-Bench and AlpacaEval scores when the SFT step is skipped. After a qualitative evaluation of the MT-Bench generations, we observe that the pure DPO model struggles to learn the chat template: This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Zephyr: Direct Distillation of LM Alignment: Appendix

Authors:
(1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co);
(2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team;
(3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction
Related Work
Method
Experimental Details
Results and Ablations
Conclusions and Limitations , Acknowledgements and References
Appendix 2 RELATED WORK There has been significant growth in the number of open large language models (LLMs) that have served as artifacts for the research community to study and use as a starting model for building chatbots and other applications. After the release of ChatGPT, the LLaMA model (Touvron et al., 2023) opened the doors to a wide range of research on efficient fine-tuning, longer prompt context, retrieval augmented generation (RAG), and quantization. After LLaMA, there has been a continuous stream of open access text based LLMs including MosaicML’s MPT (ML, 2023), the Together AI’s RedPajama-INCITE (AI, 2023), the TII’s Falcon (Penedo et al., 2023), Meta’s Llama 2 (Touvron et al., 2023), and the Mistral 7B (Jiang et al., 2023). Zephyr uses Mistral 7B as the starting point due to its strong performance. With the development of open models, researchers have worked on approaches to improve small model performance by distillation from larger models. This trend started with self-instruct method (Wang et al., 2023) and the Alpaca model (Taori et al., 2023), which was followed by Vicuna (Chiang et al., 2023)and other distilled models. These works primarily focused on distilling the SFT stage of alignment, whereas we focus on both SFT and preference optimization. Some models such as WizardLM (Xu et al.) have explored methods beyond dSFT. Contemporaneously with this work, Xwin-LM (Team, 2023) introduced an approach that distilled preference optimization through PPO (Schulman et al., 2017). We compare to these approaches in our experiments. Tools for benchmarking and evaluating LLMs have greatly evolved to keep up with the pace of innovation in generative AI. Powerful LLMs such as GPT-4 and Claude are used as evaluators to judge model responses by scoring model outputs or ranking responses in a pairwise setting. The LMSYS chatbot arena benchmarks LLMs in anonymous, randomized battles using crowdsourcing (Zheng et al., 2023). The models are ranked based on their Elo ratings on the leaderboard. AlpacaEval is an example of another such leaderboard that compares models in a pairwise setting but instead uses bigger LLMs such as GPT-4 and Claude in place of humans (Dubois et al., 2023). In a similar spirit, MTBench uses GPT-4 to score model responses on a scale of 1-10 for multi-turn instructions across task categories such as reasoning, roleplay, math, coding, writing, humanities, STEM and extraction (Zheng et al., 2023). The HuggingFace Open LLM leaderbaord (Beeching et al., 2023), the Chain-of-Thought Hub (Fu et al., 2023), ChatEval (Sedoc et al., 2019), and FastEval (fas, 2023) are examples of other tools for evaluating chatty models. We present results by evaluating on MTBench, AlpacaEval, and the HuggingFace OpenLLM Leaderboard. This paper is available on arxiv under CC 4.0 license. Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Authors: Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction Related Work Method Experimental Details Results and Ablations Conclusions and Limitations , Acknowledgements and References Appendix Abstract and Introduction Abstract and Introduction Related Work Related Work Method Method Experimental Details Experimental Details Results and Ablations Results and Ablations Conclusions and Limitations , Acknowledgements and References Conclusions and Limitations , Acknowledgements and References Appendix Appendix 2 RELATED WORK There has been significant growth in the number of open large language models (LLMs) that have served as artifacts for the research community to study and use as a starting model for building chatbots and other applications. After the release of ChatGPT, the LLaMA model (Touvron et al., 2023) opened the doors to a wide range of research on efficient fine-tuning, longer prompt context, retrieval augmented generation (RAG), and quantization. After LLaMA, there has been a continuous stream of open access text based LLMs including MosaicML’s MPT (ML, 2023), the Together AI’s RedPajama-INCITE (AI, 2023), the TII’s Falcon (Penedo et al., 2023), Meta’s Llama 2 (Touvron et al., 2023), and the Mistral 7B (Jiang et al., 2023). Zephyr uses Mistral 7B as the starting point due to its strong performance. With the development of open models, researchers have worked on approaches to improve small model performance by distillation from larger models. This trend started with self-instruct method (Wang et al., 2023) and the Alpaca model (Taori et al., 2023), which was followed by Vicuna (Chiang et al., 2023)and other distilled models. These works primarily focused on distilling the SFT stage of alignment, whereas we focus on both SFT and preference optimization. Some models such as WizardLM (Xu et al.) have explored methods beyond dSFT. Contemporaneously with this work, Xwin-LM (Team, 2023) introduced an approach that distilled preference optimization through PPO (Schulman et al., 2017). We compare to these approaches in our experiments. Tools for benchmarking and evaluating LLMs have greatly evolved to keep up with the pace of innovation in generative AI. Powerful LLMs such as GPT-4 and Claude are used as evaluators to judge model responses by scoring model outputs or ranking responses in a pairwise setting. The LMSYS chatbot arena benchmarks LLMs in anonymous, randomized battles using crowdsourcing (Zheng et al., 2023). The models are ranked based on their Elo ratings on the leaderboard. AlpacaEval is an example of another such leaderboard that compares models in a pairwise setting but instead uses bigger LLMs such as GPT-4 and Claude in place of humans (Dubois et al., 2023). In a similar spirit, MTBench uses GPT-4 to score model responses on a scale of 1-10 for multi-turn instructions across task categories such as reasoning, roleplay, math, coding, writing, humanities, STEM and extraction (Zheng et al., 2023). The HuggingFace Open LLM leaderbaord (Beeching et al., 2023), the Chain-of-Thought Hub (Fu et al., 2023), ChatEval (Sedoc et al., 2019), and FastEval (fas, 2023) are examples of other tools for evaluating chatty models. We present results by evaluating on MTBench, AlpacaEval, and the HuggingFace OpenLLM Leaderboard. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Zephyr: Direct Distillation of LM Alignment: Related Work

Authors:
(1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co);
(2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team;
(3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team;
(14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction
Related Work
Method
Experimental Details
Results and Ablations
Conclusions and Limitations , Acknowledgements and References
Appendix 5 RESULTS AND ABLATIONS In this section we collect our main results; see Appendix A for sample model completions. dDPO Improves Chat Capabilities. In Table 1 we compare the performance of ZEPHYR-7B on the MT-Bench and AlpacaEval benchmarks. Compared to other open 7B models, ZEPHYR-7B sets a new state-of-the-art and performs significantly better than dSFT models across both benchmarks. In particular, ZEPHYR-7B outperforms XWIN-LM-7B, which is one of the few open models to be trained with distilled PPO (dPPO). When compared to larger open models, ZEPHYR-7B achieves competitive performance with LLAMA2-CHAT 70B, scoring better on MT-Bench and within two standard deviations on AlpacaEval. However, ZEPHYR-7B performs worse than WIZARDLM-70B and XWIN-LM-70B, which suggests that applying dDPO to larger model sizes may be needed to match performance at these scales. When compared to proprietary models, ZEPHYR-7B is competitive with GPT-3.5-TURBO and CLAUDE 2 on AlpacaEval, however these results should be interpreted with care since the prompts in AlpacaEval may not be representative of real-usage and advanced applications. This is partly visible in Figure 1, which shows the breakdown of model performance on MT-Bench across each domain. We can see that although ZEPHYR-7B is competitive with proprietary models on several categories, is much worse in math and coding. dDPO Improves Academic Task Performance Table 2 shows the main chat results comparing the performance of the proposed model with a variety of other closed source and open-source LLMs. Results show that the dDPO model performs the best among all 7B models, with a large gap over the best dSFT models as well as Xwin-LM dPPO model. Model scale does matter more for these results and the larger models perform better than Zephyr on some of the knowledge intensive tasks. However, Zephyr does reach the performance of the 40B scale models. Is Preference Optimization Necessary? In Table 3 we examine the impact from different steps of the alignment process by fine-tuning Mistral 7B in four different ways: • dDPO - dSFT fine-tunes the base model directly with DPO for one epoch on UltraFeedback. • dSFT-1 fine-tunes the base model with SFT for one epoch on UltraChat • dSFT-2 applies dSFT-1 first, followed by one more epoch of SFT on the top-ranked completions of UltraFeedback. • dDPO + dSFT applies dSFT-1 first, followed by one epoch of DPO on UltraFeedback. First, we replicate past results (Ouyang et al., 2022) and show that without an initial SFT step (- dSFT), models are not able to learn at all from feedback and perform terribly. Using dSFT improves model score significantly on both chat benchmarks. We also consider running dSFT directly on the feedback data by training on the most preferred output (dSFT-2); however we find that this does not make an impact in performance. Finally, we see that the full Zephyr models (dDPO+dDSFT) gives a large increase in both benchmarks. Does Overfitting Harm Downstream Performance? In the process of training ZEPHYR-7B we observed that after one epoch of DPO training, the model would strongly overfit, as indicated by perfect training set accuracies in Figure 3. Surprisingly, this did not harm downstream performance on MT-Bench and AlpacaEval; as shown in Figure 3, the strongest model was obtained with one epoch of SFT followed by three epochs of DPO. However, we do observe that if the SFT model is trained for more than one epoch, the DPO step actually induces a performance regression with longer training. This paper is available on arxiv under CC 4.0 license. Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Authors: Authors: (1) Lewis Tunstall,  Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team (email: lewis@huggingface.co); (2) Edward Beeching, Equal contribution and The H4 (Helpful, Honest, Harmless, Huggy) Team; (3) Nathan Lambert, The H4 (Helpful, Honest, Harmless, Huggy) Team; (4) Nazneen Rajani, The H4 (Helpful, Honest, Harmless, Huggy) Team; (5) Kashif Rasul, The H4 (Helpful, Honest, Harmless, Huggy) Team; (6) Younes Belkada, The H4 (Helpful, Honest, Harmless, Huggy) Team; (7) Shengyi Huang, The H4 (Helpful, Honest, Harmless, Huggy) Team; (8) Leandro von Werra, The H4 (Helpful, Honest, Harmless, Huggy) Team; (9) Clementine Fourrier, The H4 (Helpful, Honest, Harmless, Huggy) Team; (10) Nathan Habib, The H4 (Helpful, Honest, Harmless, Huggy) Team; (11) Nathan Sarrazin, The H4 (Helpful, Honest, Harmless, Huggy) Team; (12) Omar Sanseviero, The H4 (Helpful, Honest, Harmless, Huggy) Team; (13) Alexander M. Rush, The H4 (Helpful, Honest, Harmless, Huggy) Team; (14) Thomas Wolf, The H4 (Helpful, Honest, Harmless, Huggy) Team. Table of Links Abstract and Introduction Related Work Method Experimental Details Results and Ablations Conclusions and Limitations , Acknowledgements and References Appendix Abstract and Introduction Abstract and Introduction Related Work Related Work Method Method Experimental Details Experimental Details Results and Ablations Results and Ablations Conclusions and Limitations , Acknowledgements and References Conclusions and Limitations , Acknowledgements and References Appendix Appendix 5 RESULTS AND ABLATIONS In this section we collect our main results; see Appendix A for sample model completions. dDPO Improves Chat Capabilities. In Table 1 we compare the performance of ZEPHYR-7B on the MT-Bench and AlpacaEval benchmarks. Compared to other open 7B models, ZEPHYR-7B sets a new state-of-the-art and performs significantly better than dSFT models across both benchmarks. In particular, ZEPHYR-7B outperforms XWIN-LM-7B, which is one of the few open models to be trained with distilled PPO (dPPO). When compared to larger open models, ZEPHYR-7B achieves competitive performance with LLAMA2-CHAT 70B, scoring better on MT-Bench and within two standard deviations on AlpacaEval. However, ZEPHYR-7B performs worse than WIZARDLM-70B and XWIN-LM-70B, which suggests that applying dDPO to larger model sizes may be needed to match performance at these scales. When compared to proprietary models, ZEPHYR-7B is competitive with GPT-3.5-TURBO and CLAUDE 2 on AlpacaEval, however these results should be interpreted with care since the prompts in AlpacaEval may not be representative of real-usage and advanced applications. This is partly visible in Figure 1, which shows the breakdown of model performance on MT-Bench across each domain. We can see that although ZEPHYR-7B is competitive with proprietary models on several categories, is much worse in math and coding. dDPO Improves Chat Capabilities. dDPO Improves Academic Task Performance Table 2 shows the main chat results comparing the performance of the proposed model with a variety of other closed source and open-source LLMs. Results show that the dDPO model performs the best among all 7B models, with a large gap over the best dSFT models as well as Xwin-LM dPPO model. Model scale does matter more for these results and the larger models perform better than Zephyr on some of the knowledge intensive tasks. However, Zephyr does reach the performance of the 40B scale models. dDPO Improves Academic Task Performance Is Preference Optimization Necessary? In Table 3 we examine the impact from different steps of the alignment process by fine-tuning Mistral 7B in four different ways: Is Preference Optimization Necessary? • dDPO - dSFT fine-tunes the base model directly with DPO for one epoch on UltraFeedback. • dDPO - dSFT • dSFT-1 fine-tunes the base model with SFT for one epoch on UltraChat • dSFT-1 • dSFT-2 applies dSFT-1 first, followed by one more epoch of SFT on the top-ranked completions of UltraFeedback. • dSFT-2 • dDPO + dSFT applies dSFT-1 first, followed by one epoch of DPO on UltraFeedback. • dDPO + dSFT First, we replicate past results (Ouyang et al., 2022) and show that without an initial SFT step (- dSFT), models are not able to learn at all from feedback and perform terribly. Using dSFT improves model score significantly on both chat benchmarks. We also consider running dSFT directly on the feedback data by training on the most preferred output (dSFT-2); however we find that this does not make an impact in performance. Finally, we see that the full Zephyr models (dDPO+dDSFT) gives a large increase in both benchmarks. Does Overfitting Harm Downstream Performance? In the process of training ZEPHYR-7B we observed that after one epoch of DPO training, the model would strongly overfit, as indicated by perfect training set accuracies in Figure 3. Surprisingly, this did not harm downstream performance on MT-Bench and AlpacaEval; as shown in Figure 3, the strongest model was obtained with one epoch of SFT followed by three epochs of DPO. However, we do observe that if the SFT model is trained for more than one epoch, the DPO step actually induces a performance regression with longer training. Does Overfitting Harm Downstream Performance? This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Zephyr: Direct Distillation of LM Alignment: Results and Ablations

**LONDON, United Kingdom, July 2nd, 2024/Chainwire/--**Web3 data platform Archisinal has announced a collaboration with the Innovation and Artificial Intelligence Lab (UBA IALAB) of the University of Buenos Aires. The entities have partnered to develop a Polkadot-based project that will revamp the UBA law facilities. Archisinal will work with UBA’s Web3, Blockchain & IA Lab to apply blockchain technology to design and urbanism. Students, architects, and members of the academic community who wish to be part of the project are invited to participate in a contest to transform the faculty’s Hearing Room into a smart classroom. Through its blockchain platform, Archisinal will leverage Polkadot’s blockchain framework and oversee the contest. It will determine the steps required to execute the work and conduct the project with the aid of collective financing delivered using blockchain. The project will serve as a pilot for Archisinal to validate the value behind its project, allowing it to prove its MVP in a university and collaborative environment. The initiative provides an exemplary use case for blockchain’s application within the academic sector. It will drive and promote community aid within Buenos Aires while contributing to the maintenance and refurbishment of public infrastructure. The University of Buenos Aires has already begun exploring the benefits of applied blockchain technology through its Web3, Blockchain & AI Lab. Archisinal, which is deeply rooted in the Polkadot ecosystem, will build upon this foundation and identify new applications for blockchain technology at UBA. In January 2023, UBA IALAB, together with adLIB! and its Director Marina Marchesotti, established a partnership with Polkadot Blockchain Academy. This academic proposal, created by Dr. Gavin Wood, held its second edition at the UBA Faculty of Law. The partnership with Polkadot Blockchain Academy has brought numerous benefits to the academic and Polkadot communities through their continuous and close collaboration. Archisinal’s collaboration with UBA will provide the impetus for a number of future university initiatives that are in the pipeline. Recent winners of a scholarship program from Decentralized Futures of the Web3 Foundation, Archisinal have won three hackathons in 2024. The partnership between UBA IALAB and Archisinal will serve as a case study for blockchain implementation, aiding academic research and developing future industry leaders. In the process, it will add value to law students by creating a technological and innovative space at one of Argentina’s leading educational institutions. About Archisinal Archisinal is a pioneer in interactions with real-world assets in the built environment. Through leveraging blockchain and AI, Archisinal eliminates the need for trust by democratizing data. Archisinal enables developers and collaborating parties to easily monitor and interact with Real-World Assets (RWAs) with full transparency, aiding accountability, transparency and dispute resolution. Learn more: https://www.archisinal.io/ About University of Buenos Aires The University of Buenos Aires is a public research university situated in Buenos Aires, Argentina. Established in 1821, it has educated 17 Argentine presidents, produced four of the country's five Nobel Prize laureates, and is responsible for around 40% of the country's research output. Learn more: https://www.uba.ar/ Contact Founder Thomas Humber Archisinal tom@archisinal.io This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program. Learn more about the program here. **LONDON, United Kingdom, July 2nd, 2024/Chainwire/--**Web3 data platform Archisinal has announced a collaboration with the Innovation and Artificial Intelligence Lab (UBA IALAB) of the University of Buenos Aires. The entities have partnered to develop a Polkadot-based project that will revamp the UBA law facilities. Archisinal will work with UBA’s Web3, Blockchain & IA Lab to apply blockchain technology to design and urbanism. Students, architects, and members of the academic community who wish to be part of the project are invited to participate in a contest to transform the faculty’s Hearing Room into a smart classroom. Through its blockchain platform, Archisinal will leverage Polkadot’s blockchain framework and oversee the contest. It will determine the steps required to execute the work and conduct the project with the aid of collective financing delivered using blockchain. The project will serve as a pilot for Archisinal to validate the value behind its project, allowing it to prove its MVP in a university and collaborative environment. The initiative provides an exemplary use case for blockchain’s application within the academic sector. It will drive and promote community aid within Buenos Aires while contributing to the maintenance and refurbishment of public infrastructure. The University of Buenos Aires has already begun exploring the benefits of applied blockchain technology through its Web3, Blockchain & AI Lab. Archisinal, which is deeply rooted in the Polkadot ecosystem, will build upon this foundation and identify new applications for blockchain technology at UBA. In January 2023, UBA IALAB, together with adLIB! and its Director Marina Marchesotti, established a partnership with Polkadot Blockchain Academy. This academic proposal, created by Dr. Gavin Wood, held its second edition at the UBA Faculty of Law. The partnership with Polkadot Blockchain Academy has brought numerous benefits to the academic and Polkadot communities through their continuous and close collaboration. Archisinal’s collaboration with UBA will provide the impetus for a number of future university initiatives that are in the pipeline. Recent winners of a scholarship program from Decentralized Futures of the Web3 Foundation, Archisinal have won three hackathons in 2024. The partnership between UBA IALAB and Archisinal will serve as a case study for blockchain implementation, aiding academic research and developing future industry leaders. In the process, it will add value to law students by creating a technological and innovative space at one of Argentina’s leading educational institutions. About Archisinal Archisinal is a pioneer in interactions with real-world assets in the built environment. Through leveraging blockchain and AI, Archisinal eliminates the need for trust by democratizing data. Archisinal enables developers and collaborating parties to easily monitor and interact with Real-World Assets (RWAs) with full transparency, aiding accountability, transparency and dispute resolution. Learn more: https://www.archisinal.io/ https://www.archisinal.io/ https://www.archisinal.io/ About University of Buenos Aires The University of Buenos Aires is a public research university situated in Buenos Aires, Argentina. Established in 1821, it has educated 17 Argentine presidents, produced four of the country's five Nobel Prize laureates, and is responsible for around 40% of the country's research output. Learn more: https://www.uba.ar/ https://www.uba.ar/ https://www.uba.ar/ Contact Founder Thomas Humber Archisinal tom@archisinal.io This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program. Learn more about the program here. This story was distributed as a release by Chainwire under HackerNoon’s Business Blogging Program. Learn more about the program here . here here

University of Buenos Aires And Archisinal Partner To Revamp UBA Law Facilities Using Polkadot

Author:
(1) Mohammad AL-Smad, Qatar University, Qatar and (e-mail: malsmadi@qu.edu.qa). Table of Links Abstract and Introduction History of Using AI in Education Research Methodology Literature Review Summary Conclusion and References 5. Summary The integration of generative AI models in education offer a wide range of potential benefits, including improved productivity, enhanced creativity, and personalized learning, providing timely feedback and support, and automating tasks (Farrokhnia et al., 2023; Cotton et al., 2023). The advantages of using generative AI models in education are summarised in Table 1. However, the adoption of generative AI models in education raises challenges that need to be tackled in order to guarantee a safe and responsible usage of such technologies (Su & Yang, 2023). These challenges are summarised in Table 2. Findings of this survey underscore the need for further research, regulatory policies, and pedagogical guidance to navigate issues related to academic integrity and equity in the era of AI enhanced learning (Chaudhry et al., 2023; Yan, 2023; Cotton et al., 2023). The irresponsible implementation of generative AI models in education have ethical and legal consequences (Chaudhry et al., 2023). Table 3 discusses potential ethical consideration for the adoption of Generative AI models in education. Whereas, Table 4 focuses on the legal implication. Governments around the world are developing regulations to address the ethical and legal challenges posed by generative AI models. For example, the ”European Union Artificial Intelligence Act (EU AI Act)” was proposed to promote the development and use of trustworthy AI, while also mitigating the risks associated with AI (Madiega, 2021). The Act does this by establishing a riskbased approach to AI regulation. AI systems are classified into four risk categories: ”unacceptable risk, high risk, limited risk, and minimal risk”. The EU AI Act includes the following provisions: (a) Risk Assessment: the AI Act mandates the assessment of risks associated with AI systems, ensuring that potential harms are identified and mitigated, (b) Transparency: transparency requirements are specified to enhance the explainability of AI systems and make their decisionmaking processes more understandable, the EU AI Act specified the following three transparency requirements when it comes to adopt generative AI technology: • Disclosure of AI Generation: The imperative for transparency in the context of generative AI pertains to the acknowledgment and notification that content has been produced by artificial intelligence rather than a human agent. Such disclosures serve to elucidate the nature of the content creation process, fostering user understanding and trust. • Mitigation of Illicit Content Generation: The design and implementation of generative AI models must encompass measures to forestall the generation of content that contravenes legal statutes. These measures are integral to ethical and legal compliance, involving content moderation and safety mechanisms to curtail the production of potentially harmful, offensive, or unlawful materials. • Publication of Summaries of Copyrighted Data Utilized in Training: In the sphere of AI model development, the utilization of copyrighted data engenders concerns pertaining to intellectual property rights. To address these concerns and exemplify transparency, developers may be encouraged to provide summaries or descriptions of the copyrighted data sources employed during training. This step showcases adherence to copyright laws while enhancing transparency regarding the model’s training data. and (c) Human Oversight: the AI Act emphasizes the importance of human oversight in AI systems to ensure responsible and ethical use. However, there are some limitations to the current regulatory landscape (Schuett, 2023). For example, the EU AI Act does not explicitly address the issue of generative AI models hallucination. Additionally, the EU AI Act focuses on the obligations of AI developers and users, rather than on the rights of individuals who may be affected by AI systems (Enqvist, 2023). Findings from literature also provide valuable insights into the determinants of intention to use generative AI models for educational purposes. The findings have important implications for the models developers, instructors, and universities as they work to accelerate the adoption of them in educational contexts. Table 5 discusses a set of recommendations for the different stakeholders fostering them to have a safe and responsible adoption of such technologies in education. The adoption or generative AI models in education requires a trust raising plan (Choudhury & Shamszare, 2023). In addition to addressing the forementioned challenges (See Table 2), there are a number of steps that can be done to build trust in generative AI, including: (1) Transparency: developers should be transparent about the limitations, biases, data sources, and potential risks of these models. (2) Explainability: developers should develop explainable AI methods to explain how their models generate outputs (Dosilovi ˇ c et al., 2018). (3) ´ Human-in-the-loop validation: human experts should be involved in the evaluation of AI-generated content to identify and rectify errors. This will help into solving other problems such as bias, and hallucination. (4) Accountability: developers and users of generative AI should be accountable for the consequences of their actions. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. Author: (1) Mohammad AL-Smad, Qatar University, Qatar and (e-mail: malsmadi@qu.edu.qa). Author: Author: (1) Mohammad AL-Smad, Qatar University, Qatar and (e-mail: malsmadi@qu.edu.qa). Table of Links Abstract and Introduction Abstract and Introduction History of Using AI in Education History of Using AI in Education Research Methodology Research Methodology Literature Review Literature Review Summary Summary Conclusion and References Conclusion and References 5. Summary The integration of generative AI models in education offer a wide range of potential benefits, including improved productivity, enhanced creativity, and personalized learning, providing timely feedback and support, and automating tasks (Farrokhnia et al., 2023; Cotton et al., 2023). The advantages of using generative AI models in education are summarised in Table 1. However, the adoption of generative AI models in education raises challenges that need to be tackled in order to guarantee a safe and responsible usage of such technologies (Su & Yang, 2023). These challenges are summarised in Table 2. Findings of this survey underscore the need for further research, regulatory policies, and pedagogical guidance to navigate issues related to academic integrity and equity in the era of AI enhanced learning (Chaudhry et al., 2023; Yan, 2023; Cotton et al., 2023). The irresponsible implementation of generative AI models in education have ethical and legal consequences (Chaudhry et al., 2023). Table 3 discusses potential ethical consideration for the adoption of Generative AI models in education. Whereas, Table 4 focuses on the legal implication. Governments around the world are developing regulations to address the ethical and legal challenges posed by generative AI models. For example, the ”European Union Artificial Intelligence Act (EU AI Act)” was proposed to promote the development and use of trustworthy AI, while also mitigating the risks associated with AI (Madiega, 2021). The Act does this by establishing a riskbased approach to AI regulation. AI systems are classified into four risk categories: ”unacceptable risk, high risk, limited risk, and minimal risk”. The EU AI Act includes the following provisions: (a) Risk Assessment: the AI Act mandates the assessment of risks associated with AI systems, ensuring that potential harms are identified and mitigated, (b) Transparency: transparency requirements are specified to enhance the explainability of AI systems and make their decisionmaking processes more understandable, the EU AI Act specified the following three transparency requirements when it comes to adopt generative AI technology: • Disclosure of AI Generation: The imperative for transparency in the context of generative AI pertains to the acknowledgment and notification that content has been produced by artificial intelligence rather than a human agent. Such disclosures serve to elucidate the nature of the content creation process, fostering user understanding and trust. Disclosure of AI Generation: • Mitigation of Illicit Content Generation: The design and implementation of generative AI models must encompass measures to forestall the generation of content that contravenes legal statutes. These measures are integral to ethical and legal compliance, involving content moderation and safety mechanisms to curtail the production of potentially harmful, offensive, or unlawful materials. • Mitigation of Illicit Content Generation: • Publication of Summaries of Copyrighted Data Utilized in Training: In the sphere of AI model development, the utilization of copyrighted data engenders concerns pertaining to intellectual property rights. To address these concerns and exemplify transparency, developers may be encouraged to provide summaries or descriptions of the copyrighted data sources employed during training. This step showcases adherence to copyright laws while enhancing transparency regarding the model’s training data. • Publication of Summaries of Copyrighted Data Utilized in Training: and (c) Human Oversight: the AI Act emphasizes the importance of human oversight in AI systems to ensure responsible and ethical use. However, there are some limitations to the current regulatory landscape (Schuett, 2023). For example, the EU AI Act does not explicitly address the issue of generative AI models hallucination. Additionally, the EU AI Act focuses on the obligations of AI developers and users, rather than on the rights of individuals who may be affected by AI systems (Enqvist, 2023). Findings from literature also provide valuable insights into the determinants of intention to use generative AI models for educational purposes. The findings have important implications for the models developers, instructors, and universities as they work to accelerate the adoption of them in educational contexts. Table 5 discusses a set of recommendations for the different stakeholders fostering them to have a safe and responsible adoption of such technologies in education. The adoption or generative AI models in education requires a trust raising plan (Choudhury & Shamszare, 2023). In addition to addressing the forementioned challenges (See Table 2), there are a number of steps that can be done to build trust in generative AI, including: (1) Transparency: developers should be transparent about the limitations, biases, data sources, and potential risks of these models. (2) Explainability: developers should develop explainable AI methods to explain how their models generate outputs (Dosilovi ˇ c et al., 2018). (3) ´ Human-in-the-loop validation: human experts should be involved in the evaluation of AI-generated content to identify and rectify errors. This will help into solving other problems such as bias, and hallucination. (4) Accountability: developers and users of generative AI should be accountable for the consequences of their actions. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. This paper is available on arxiv under CC BY-NC-ND 4.0 DEED license. available on arxiv

Generative AI in Education: Benefits, Challenges, and Future Directions

Author:
(1) David Novoa-Paradela, Universidade da Coruña, CITIC, Campus de Elviña s/n, 15008, A Coruña, Spain & Corresponding author (Email: david.novoa@udc.es);
(2) Oscar Fontenla-Romero, Universidade da Coruña, CITIC, Campus de Elviña s/n, 15008, A Coruña, Spain (Email: oscar.fontenla@udc.es);
(3) Bertha Guijarro-Berdiñas, Universidade da Coruña, CITIC, Campus de Elviña s/n, 15008, A Coruña, Spain (Email: berta.guijarro@udc.es). Table of Links Abstract and Introduction Related work The proposed pipeline Evaluation Conclusion & Acknowledgements References & Appendix 5. Conclusion In this work, we have proposed a pipeline for detecting anomalous reviews associated with Amazon products, which can be directly extrapolated to other online review platforms or scenarios with similar characteristics. The representation of the reviews using MPNet embeddings has enabled the training of classical anomaly detection algorithms that have achieved a very good performance. These have been evaluated using reviews from different products and categories, and the score they emit allows us to sort the reviews based on their normality. A technique based on the occurrence of frequent terms has been proposed to generate explanations associated with the classifications of the reviews. This technique has been compared with SHAP, one of the reference post-hoc techniques in the field of explainability, and with GPT-3, due to its high power and versatility. To evaluate this aspect of the pipeline, we conducted a two-part survey in which 241 members of the university community participated. From the first part of the explainability test we can conclude that, in general terms, the effect of the explanations has not been beneficial for the users. In any case, these tests allow us to reflect on the difficulty of using explainability and evaluation techniques in borderline scenarios where subjectivity plays an important role, such as the one presented in this article or in other fields of NLP, as well as in areas such as image or audio generation. Regarding the second part of the explainability test, we have been able to conclude that respondents preferred explanations that presented a more natural and familiar appearance over more condensed and concise explanations such as those provided by SHAP, regardless of the explanation effect they provide. Explanations based on term frequency analysis have been preferred by respondents along with GPT-3, however, our approach presents a significantly lower computational costs and both its use and the explanations produced are simpler for the users. As future work, it would be interesting to evaluate GPT-3 or other large models carrying out the complete process followed by the pipeline proposed in this work, instead of being tested only in the explainability module. We have not carried out this test due to the high computational cost that would be involved in processing the thousands of reviews to be evaluated using GPT3. Another interesting possible line of future work would be to broaden the scope of the survey, both in terms of the number of products involved and the number of reviews, in order to clarify the conclusions reached at the forward simulation stage. Lastly, it would be very useful to try presenting the explanations issued by SHAP in a more familiar or natural format for the end user, so that we can see if their level of preference is increased for the general public. Acknowledgements This work was supported in part by grant Machine Learning on the Edge - Ayudas Fundación BBVA a Equipos de Investigación Científica 2019 ; the Spanish National Plan for Scientific and Technical Research and Innovation (PID2019-109238GB-C22 and TED2021-130599A-I00); the Xunta de Galicia (ED431C 2022/44) and ERDF funds. CITIC, as a Research Center of the University System of Galicia, is funded by Consellería de Educación, Universidade e Formación Profesional of the Xunta de Galicia, Spain through the European Regional Development Fund (ERDF) and the Secretaría Xeral de Universidades (Ref. ED431G 2019/01). This paper is available on arxiv under CC 4.0 license. Author: (1) David Novoa-Paradela, Universidade da Coruña, CITIC, Campus de Elviña s/n, 15008, A Coruña, Spain & Corresponding author (Email: david.novoa@udc.es); (2) Oscar Fontenla-Romero, Universidade da Coruña, CITIC, Campus de Elviña s/n, 15008, A Coruña, Spain (Email: oscar.fontenla@udc.es); (3) Bertha Guijarro-Berdiñas, Universidade da Coruña, CITIC, Campus de Elviña s/n, 15008, A Coruña, Spain (Email: berta.guijarro@udc.es). Author: Author: (1) David Novoa-Paradela, Universidade da Coruña, CITIC, Campus de Elviña s/n, 15008, A Coruña, Spain & Corresponding author (Email: david.novoa@udc.es); (2) Oscar Fontenla-Romero, Universidade da Coruña, CITIC, Campus de Elviña s/n, 15008, A Coruña, Spain (Email: oscar.fontenla@udc.es); (3) Bertha Guijarro-Berdiñas, Universidade da Coruña, CITIC, Campus de Elviña s/n, 15008, A Coruña, Spain (Email: berta.guijarro@udc.es). Table of Links Abstract and Introduction Abstract and Introduction Related work Related work The proposed pipeline The proposed pipeline Evaluation Evaluation Conclusion & Acknowledgements Conclusion & Acknowledgements References & Appendix References & Appendix 5. Conclusion In this work, we have proposed a pipeline for detecting anomalous reviews associated with Amazon products, which can be directly extrapolated to other online review platforms or scenarios with similar characteristics. The representation of the reviews using MPNet embeddings has enabled the training of classical anomaly detection algorithms that have achieved a very good performance. These have been evaluated using reviews from different products and categories, and the score they emit allows us to sort the reviews based on their normality. A technique based on the occurrence of frequent terms has been proposed to generate explanations associated with the classifications of the reviews. This technique has been compared with SHAP, one of the reference post-hoc techniques in the field of explainability, and with GPT-3, due to its high power and versatility. To evaluate this aspect of the pipeline, we conducted a two-part survey in which 241 members of the university community participated. From the first part of the explainability test we can conclude that, in general terms, the effect of the explanations has not been beneficial for the users. In any case, these tests allow us to reflect on the difficulty of using explainability and evaluation techniques in borderline scenarios where subjectivity plays an important role, such as the one presented in this article or in other fields of NLP, as well as in areas such as image or audio generation. Regarding the second part of the explainability test, we have been able to conclude that respondents preferred explanations that presented a more natural and familiar appearance over more condensed and concise explanations such as those provided by SHAP, regardless of the explanation effect they provide. Explanations based on term frequency analysis have been preferred by respondents along with GPT-3, however, our approach presents a significantly lower computational costs and both its use and the explanations produced are simpler for the users. As future work, it would be interesting to evaluate GPT-3 or other large models carrying out the complete process followed by the pipeline proposed in this work, instead of being tested only in the explainability module. We have not carried out this test due to the high computational cost that would be involved in processing the thousands of reviews to be evaluated using GPT3. Another interesting possible line of future work would be to broaden the scope of the survey, both in terms of the number of products involved and the number of reviews, in order to clarify the conclusions reached at the forward simulation stage. Lastly, it would be very useful to try presenting the explanations issued by SHAP in a more familiar or natural format for the end user, so that we can see if their level of preference is increased for the general public. Acknowledgements This work was supported in part by grant Machine Learning on the Edge - Ayudas Fundación BBVA a Equipos de Investigación Científica 2019 ; the Spanish National Plan for Scientific and Technical Research and Innovation (PID2019-109238GB-C22 and TED2021-130599A-I00); the Xunta de Galicia (ED431C 2022/44) and ERDF funds. CITIC, as a Research Center of the University System of Galicia, is funded by Consellería de Educación, Universidade e Formación Profesional of the Xunta de Galicia, Spain through the European Regional Development Fund (ERDF) and the Secretaría Xeral de Universidades (Ref. ED431G 2019/01). This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Effective Anomaly Detection Pipeline for Amazon Reviews: Insights and Future Directions

In our previous discussion, we introduced the MinIO Object Store Firewall, a novel security component that goes beyond traditional network and application security layers. Neither IP-based firewalls nor application firewalls are designed for data. That is why we built the MinIO Enterprise Object Store Firewall – because in the modern enterprise, data is what must be protected and an S3-aware firewall doesn’t exist. This S3-aware data firewall is crucial for modern data protection, operating at the storage layer to safeguard your data comprehensively. The MinIO Enterprise Object Store Firewall is designed specifically to work with applications using MinIO object store and its API endpoints. The Enterprise Firewall is lightweight, powerful, flexible and extensible. Let’s delve into setting up this advanced firewall, designed to secure your data in today’s increasingly complex digital landscape. Enable and Configure Firewall Let's use the Enterprise Console to set up the firewall. Follow the below steps to enable and configure the Firewall. Configure TLS for Secure Communication As part of the MinIO Enterprise Suite, we’ve always recommended to ensure you enable TLS on MinIO Enterprise Object Store so that even inter cluster communications are encrypted. With that same spirit, we support TLS for the Enterprise Firewall as well. This ensures that any connection to the MinIO Object Store via the Firewall is encrypted end to end. For enhanced security, configure TLS settings when launching the firewall with Let’s Encrypt. Precedence of Anonymous Rule In our MinIO Enterprise Firewall configuration, you’ll encounter two distinct rules for anonymous access: Global Anonymous Setting: At the start when you enable firewall, we specify a global setting that allows anonymous access across all buckets unless a more specific rule denies it. Bucket Specific Anonymous: After configuring the global setting, under each individual rules, we can set a more specific rule that overrides the global setting, effectively denying anonymous access In this case, even though we initially toggled Global Anonymous to Allow for all buckets, because we set another more specific rule below it, which also applies to all buckets, the one we set later takes precedence over the global setting. In other words, anonymous bucket access will be denied for all buckets. Load Balance across MinIO nodes As an added bonus, because we need to define all the MinIO nodes as a backend for the Firewall, the Firewall doubles as a LoadBalancer, negating the need for a separate load balancer between the firewall and MinIO nodes. This eliminates further complexity and ensures there is no single point of failure in case one of the MinIO backend goes offline. Please note, you need to have another level of redundancy to ensure when connecting to Enterprise Firewall the incoming connections are also distributed to multiple Enterprise Firewall instances. This configuration is beyond the scope of this blog but its something to keep in mind. Health Checks and Monitoring The health check enabled you to see if the Firewall is in a healthy state. You can check the health and liveliness as follows. If everything is Green that will indicate that the firewall is functioning properly. Final Thoughts With the MinIO Enterprise Firewall, gone are the days of wrestling with complex IPtables and unclear access policies.Our firewall solution simplifies your security by focusing exclusively on the essential rules required for your object store and API interactions. It is optimized to ensure there are no latency or unforeseen rules, blocking access to your MinIO object store. Moreover, the Enterprise Firewall is fully supported by our awesome team at SUBNET where we can help you by architecting the Enterprise Firewall the right way to work with MinIO object store and troubleshoot any future issues. So what are you waiting for? If you have any questions on MinIO Enterprise Object Store Firewall be sure to reach out to us on Slack or hello@min.io! In our previous discussion , we introduced the MinIO Object Store Firewall, a novel security component that goes beyond traditional network and application security layers. Neither IP-based firewalls nor application firewalls are designed for data. That is why we built the MinIO Enterprise Object Store Firewall – because in the modern enterprise, data is what must be protected and an S3-aware firewall doesn’t exist. This S3-aware data firewall is crucial for modern data protection, operating at the storage layer to safeguard your data comprehensively. previous discussion previous discussion The MinIO Enterprise Object Store Firewall is designed specifically to work with applications using MinIO object store and its API endpoints. The Enterprise Firewall is lightweight, powerful, flexible and extensible. Let’s delve into setting up this advanced firewall, designed to secure your data in today’s increasingly complex digital landscape. Enable and Configure Firewall Let's use the Enterprise Console to set up the firewall. Follow the below steps to enable and configure the Firewall. Configure TLS for Secure Communication As part of the MinIO Enterprise Suite, we’ve always recommended to ensure you enable TLS on MinIO Enterprise Object Store so that even inter cluster communications are encrypted. With that same spirit, we support TLS for the Enterprise Firewall as well. This ensures that any connection to the MinIO Object Store via the Firewall is encrypted end to end. For enhanced security, configure TLS settings when launching the firewall with Let’s Encrypt. Precedence of Anonymous Rule In our MinIO Enterprise Firewall configuration, you’ll encounter two distinct rules for anonymous access: Global Anonymous Setting: At the start when you enable firewall, we specify a global setting that allows anonymous access across all buckets unless a more specific rule denies it. Global Anonymous Setting: Bucket Specific Anonymous: After configuring the global setting, under each individual rules, we can set a more specific rule that overrides the global setting, effectively denying anonymous access Bucket Specific Anonymous: In this case, even though we initially toggled Global Anonymous to Allow for all buckets, because we set another more specific rule below it, which also applies to all buckets, the one we set later takes precedence over the global setting. In other words, anonymous bucket access will be denied for all buckets. Load Balance across MinIO nodes As an added bonus, because we need to define all the MinIO nodes as a backend for the Firewall, the Firewall doubles as a LoadBalancer, negating the need for a separate load balancer between the firewall and MinIO nodes. This eliminates further complexity and ensures there is no single point of failure in case one of the MinIO backend goes offline. Please note, you need to have another level of redundancy to ensure when connecting to Enterprise Firewall the incoming connections are also distributed to multiple Enterprise Firewall instances. This configuration is beyond the scope of this blog but its something to keep in mind. Health Checks and Monitoring The health check enabled you to see if the Firewall is in a healthy state. You can check the health and liveliness as follows. If everything is Green that will indicate that the firewall is functioning properly. Final Thoughts With the MinIO Enterprise Firewall, gone are the days of wrestling with complex IPtables and unclear access policies.Our firewall solution simplifies your security by focusing exclusively on the essential rules required for your object store and API interactions. It is optimized to ensure there are no latency or unforeseen rules, blocking access to your MinIO object store. Moreover, the Enterprise Firewall is fully supported by our awesome team at SUBNET where we can help you by architecting the Enterprise Firewall the right way to work with MinIO object store and troubleshoot any future issues. SUBNET SUBNET So what are you waiting for? If you have any questions on MinIO Enterprise Object Store Firewall be sure to reach out to us on Slack or hello@min.io ! Slack Slack hello@min.io hello@min.io

Stories, News & Mentions about 5IRE

5ire’s Stories on HackerNoon