In the sixth season of Netflix&#x27;s Formula 1 documentary &quot;Drive to Survive&quot;, Mercedes driver Lewis Hamilton addresses situations that may have driven him into the arms of Ferrari · SPEEDWEEK.COM provides this translated article advertising-free

Hamilton ignored by Mercedes, therefore to Ferrari?

Mercanis

mercanis

“8% of U.S. respondents now use ChatGPT as their primary search tool, up from just 1% a few months ago” In the early days of the internet, writing for search engines was all about keywords, backlinks, and meta tags. Then came the rise of conversational AI like ChatGPT, Gemini, Venice, Perplexity, Grok, and Meta AI, reshaping how users interact with content. By 2024, this shift has become undeniable—8% of U.S. respondents now use ChatGPT as their primary search tool, up from just 1% a few months ago, while Google's dominance has slipped from 80% to 74% (Barron's). Organizations are also embracing AI at an unprecedented pace, with 65% regularly using generative AI, nearly double the rate from ten months prior (McKinsey). Today, writing for AI is about direct answers, natural language, and meeting user intent in new ways. If you're interested in future-proofing your content for AI-driven platforms, check out our guide on How to AI-Proof Your Content for 2025. While the fundamentals of content creation remain the same—delivering value and being findable—the methods to optimize for AI and search engines diverge. So, how do you write content that performs well on both AI tools and traditional search engines? The differences may surprise you, but the strategies can work in harmony. Let’s break it down. Similarities Between Writing for AI and Search Engines 1. Focus on User Intent Both AI and search engines prioritize content that is aligned with user intent. Generative AI tools like ChatGPT are designed to provide contextually relevant answers (OpenAI) while search engines like Google use algorithms to match content with the searcher’s needs (Google Search Central). 2. Value-Driven Content High-quality, informative content is non-negotiable. Search engines favor pages with high E-A-T (Expertise, Authoritativeness, Trustworthiness), while AI prefers well-researched, concise answers backed by credible sources (Backlinko). 3. Importance of Structure A clear, logical structure helps both platforms process content efficiently. Using headings, subheadings, and bullet points ensures content is scannable and easy to parse (Microsoft AI Blog). 4. Use of Keywords Keywords help both systems understand your content. While search engines use them for indexing, AI tools rely on them for relevance in natural language processing. 5. Optimization of Metadata Titles, meta descriptions, alt text, image, and video file naming correctly all provide essential context for search engines and AI, influencing how your content is indexed or summarized. Key Differences Between Writing for AI and Search Engines How to Write for AI and Search Engines Simultaneously 1. Answer Questions Directly AI tools often pull direct answers from content. Start your article with a concise response to the query before diving deeper. 2. Use Conversational Language Tools like Bard and ChatGPT thrive on natural language. Write as if you're explaining concepts to a friend or colleague. 3. Optimize for Structure Use clear headings (H2s and H3s) for major topics.
Break down complex ideas into bullet points or numbered lists.
Incorporate FAQs to address specific user queries. 4. Leverage Metadata and Schema For search engines: Add FAQ schema, HowTo schema, and descriptive alt text. For AI: Provide context through concise, well-written metadata to help the AI interpret your content. 5. Blend Long-Tail Keywords With Natural Queries Optimize for specific phrases users might type or say, like “best AI optimization tips for content” instead of generic keywords. 6. Build Trust Signals For AI: Highlight the author's expertise and ethical claims. For search engines: Focus on gaining backlinks and social signals. For a comprehensive guide on adapting to AI-driven search trends, including examples and actionable tips, visit How to AI-Proof Your Content for 2025. FAQs Q: How does AI decide which content to prioritize? AI models prioritize content that is conversational, concise, and aligned with user queries. They often favor transparent, well-cited sources and use metadata to infer relevance (OpenAI). Q: Will AI eventually replace traditional search engines? AI tools like ChatGPT and Gemini are changing how people search, but they complement rather than replace traditional search engines. Search engines still excel at linking to in-depth resources, while AI provides direct answers (Search Engine Journal). Q: Do I Need Different Content Strategies for AI and SEO? Not necessarily. A hybrid strategy works best. Focus on providing clear answers for AI while incorporating traditional SEO elements like keywords and backlinks to serve both audiences. For a detailed breakdown, see How to AI-Proof Your Content for 2025. Q: Is multimedia important for AI-driven content? Yes! AI increasingly integrates multimodal content, so optimizing videos, images, and transcripts is essential for visibility (Microsoft AI Blog). Q: What are examples of AI-friendly content formats? FAQs, how-to guides, and step-by-step instructions are AI-friendly because they align with how AI extracts and delivers information. Conclusion Writing for AI and search engines may differ in approach, but their goals—delivering value and satisfying user intent—are the same. By blending conversational tone, structured answers, and traditional SEO practices, you can future-proof your content strategy for both worlds. Pro Tip: Stay ahead of the curve by experimenting with AI tools like ChatGPT or Bard to see how your content performs in real-time AI interactions. The future of content is here, and it’s hybrid. Stay Connected Want to keep up with the latest trends in AI, SEO, and content marketing? Subscribe to Darragh Grove-White on HackerNoon for expert insights, guides, and actionable strategies. Follow him on X (Twitter) and LinkedIn for real-time updates, tips, and breaking news on the ever-changing landscape of AI-driven content. Stay informed. Stay ahead. “8% of U.S. respondents now use ChatGPT as their primary search tool, up from just 1% a few months ago” “8% of U.S. respondents now use ChatGPT as their primary search tool, up from just 1% a few months ago” up from just 1% a few months ago” In the early days of the internet, writing for search engines was all about keywords, backlinks, and meta tags. Then came the rise of conversational AI like ChatGPT, Gemini, Venice, Perplexity, Grok, and Meta AI, reshaping how users interact with content. By 2024, this shift has become undeniable— 8% of U.S. respondents now use ChatGPT as their primary search tool, up from just 1% a few months ago, while Google's dominance has slipped from 80% to 74% ( Barron's ). 8% of U.S. respondents now use ChatGPT as their primary search tool, up from just 1% a few months ago, Barron's Organizations are also embracing AI at an unprecedented pace, with 65% regularly using generative AI, nearly double the rate from ten months prior ( McKinsey ). McKinsey Today, writing for AI is about direct answers, natural language, and meeting user intent in new ways. If you're interested in future-proofing your content for AI-driven platforms, check out our guide on How to AI-Proof Your Content for 2025 . How to AI-Proof Your Content for 2025 While the fundamentals of content creation remain the same—delivering value and being findable—the methods to optimize for AI and search engines diverge. So, how do you write content that performs well on both AI tools and traditional search engines? The differences may surprise you, but the strategies can work in harmony. Let’s break it down. Similarities Between Writing for AI and Search Engines Similarities Between Writing for AI and Search Engines 1. Focus on User Intent 1. Focus on User Intent Both AI and search engines prioritize content that is aligned with user intent. Generative AI tools like ChatGPT are designed to provide contextually relevant answers ( OpenAI ) while search engines like Google use algorithms to match content with the searcher’s needs ( Google Search Central ). OpenAI Google Search Central 2. Value-Driven Content 2. Value-Driven Content High-quality, informative content is non-negotiable. Search engines favor pages with high E-A-T (Expertise, Authoritativeness, Trustworthiness), while AI prefers well-researched, concise answers backed by credible sources ( Backlinko ). pages with high E-A-T Backlinko 3. Importance of Structure 3. Importance of Structure A clear, logical structure helps both platforms process content efficiently. Using headings, subheadings, and bullet points ensures content is scannable and easy to parse ( Microsoft AI Blog ). Microsoft AI Blog 4. Use of Keywords 4. Use of Keywords Keywords help both systems understand your content. While search engines use them for indexing, AI tools rely on them for relevance in natural language processing. 5. Optimization of Metadata 5. Optimization of Metadata Titles, meta descriptions, alt text, image, and video file naming correctly all provide essential context for search engines and AI, influencing how your content is indexed or summarized. Key Differences Between Writing for AI and Search Engines How to Write for AI and Search Engines Simultaneously 1. Answer Questions Directly AI tools often pull direct answers from content. Start your article with a concise response to the query before diving deeper. 2. Use Conversational Language Tools like Bard and ChatGPT thrive on natural language. Write as if you're explaining concepts to a friend or colleague. 3. Optimize for Structure Use clear headings (H2s and H3s) for major topics. Break down complex ideas into bullet points or numbered lists. Incorporate FAQs to address specific user queries. Use clear headings (H2s and H3s) for major topics. Break down complex ideas into bullet points or numbered lists. Incorporate FAQs to address specific user queries. 4. Leverage Metadata and Schema For search engines: Add FAQ schema, HowTo schema, and descriptive alt text. For AI: Provide context through concise, well-written metadata to help the AI interpret your content. 5. Blend Long-Tail Keywords With Natural Queries Optimize for specific phrases users might type or say, like “best AI optimization tips for content” instead of generic keywords. 6. Build Trust Signals For AI: Highlight the author's expertise and ethical claims. For AI: Highlight the author's expertise and ethical claims. For search engines: Focus on gaining backlinks and social signals. For search engines: Focus on gaining backlinks and social signals. For a comprehensive guide on adapting to AI-driven search trends, including examples and actionable tips, visit How to AI-Proof Your Content for 2025. visit How to AI-Proof Your Content for 2025 . How to AI-Proof Your Content for 2025 FAQs FAQs Q: How does AI decide which content to prioritize? Q: How does AI decide which content to prioritize? AI models prioritize content that is conversational, concise, and aligned with user queries. They often favor transparent, well-cited sources and use metadata to infer relevance ( OpenAI ). OpenAI Q: Will AI eventually replace traditional search engines? Q: Will AI eventually replace traditional search engines? AI tools like ChatGPT and Gemini are changing how people search, but they complement rather than replace traditional search engines. Search engines still excel at linking to in-depth resources, while AI provides direct answers (Search Engine Journal). Q: Do I Need Different Content Strategies for AI and SEO? Q: Do I Need Different Content Strategies for AI and SEO? Not necessarily. A hybrid strategy works best. Focus on providing clear answers for AI while incorporating traditional SEO elements like keywords and backlinks to serve both audiences. For a detailed breakdown, see How to AI-Proof Your Content for 2025. For a detailed breakdown, see How to AI-Proof Your Content for 2025. How to AI-Proof Your Content for 2025. Q: Is multimedia important for AI-driven content? Q: Is multimedia important for AI-driven content? Yes! AI increasingly integrates multimodal content, so optimizing videos, images, and transcripts is essential for visibility ( Microsoft AI Blog ). Microsoft AI Blog Q: What are examples of AI-friendly content formats? Q: What are examples of AI-friendly content formats? FAQs, how-to guides, and step-by-step instructions are AI-friendly because they align with how AI extracts and delivers information. Conclusion Conclusion Writing for AI and search engines may differ in approach, but their goals—delivering value and satisfying user intent—are the same. By blending conversational tone, structured answers, and traditional SEO practices, you can future-proof your content strategy for both worlds. Pro Tip: Stay ahead of the curve by experimenting with AI tools like ChatGPT or Bard to see how your content performs in real-time AI interactions. The future of content is here, and it’s hybrid. Pro Tip: Stay Connected Stay Connected Want to keep up with the latest trends in AI, SEO, and content marketing? Subscribe to Darragh Grove-White on HackerNoon for expert insights, guides, and actionable strategies. Subscribe to Darragh Grove-White on HackerNoon for expert insights, guides, and actionable strategies. Subscribe to Darragh Grove-White on HackerNoon Follow him on X (Twitter) and LinkedIn for real-time updates, tips, and breaking news on the ever-changing landscape of AI-driven content. Follow him on X (Twitter) and LinkedIn for real-time updates, tips, and breaking news on the ever-changing landscape of AI-driven content. Follow him on X (Twitter) and LinkedIn Follow him on X (Twitter) LinkedIn Stay informed. Stay ahead.

Future-proof Your Marketing With This Guide on Writing for AI Search Engines

এআই সার্চ ইঞ্জিনের জন্য লেখার এই নির্দেশিকা দিয়ে আপনার বিপণনকে ভবিষ্যৎ প্রমাণ করুন

Machen Sie Ihr Marketing zukunftssicher mit diesem Leitfaden zum Schreiben für KI-Suchmaschinen

Prepare su marketing para el futuro con esta guía sobre cómo escribir para motores de búsqueda con inteligencia artificial

Assurez l'avenir de votre marketing grâce à ce guide de rédaction pour les moteurs de recherche IA

AI सर्च इंजन के लिए लेखन पर इस गाइड के साथ अपने मार्केटिंग को भविष्य-सुरक्षित बनाएं

AI 検索エンジン向けのライティングに関するガイドで、マーケティングを将来に備えましょう

AI 검색 엔진을 위한 글쓰기 가이드로 마케팅을 미래에 대비하세요

Prepare seu marketing para o futuro com este guia sobre como escrever para mecanismos de busca de IA

Обеспечьте будущее вашему маркетингу с помощью этого руководства по написанию текстов для поисковых систем на базе искусственного интеллекта

AI Arama Motorları İçin Yazma Kılavuzuyla Pazarlamanızı Geleceğe Hazırlayın

Tương lai tiếp thị của bạn với hướng dẫn này về cách viết cho công cụ tìm kiếm AI

这份针对人工智能搜索引擎的写作指南将为您的营销提供未来保障

Are you overwhelmed by Google’s cluttered interface, filled with ads and irrelevant links? Do you find that navigating through numerous options for simple queries detracts from your search experience? If so, it might be time to explore AI search engines that offer a cleaner, more intuitive alternative. AskAI.io is one of the go-to options, and in this review, we will delve into what makes it a popular choice among users looking for a more streamlined search experience. What Is AskAI.io? AskAI.io is an AI-powered search tool designed to enhance traditional search experiences by providing direct answers and personalized results. Unlike conventional search engines that primarily rely on keyword matching, Ask AI, powered by advanced AI algorithms, understands the context and intent behind user queries, delivering more accurate and relevant information. One of its standout features is the ability to access real-time data from the web, ensuring users receive up-to-date information while maintaining a clean, ad-free interface that improves user experience. It aims to rival established search engines like Google by offering a more conversational and efficient way to find information, making it a promising development in the evolving landscape of search technology. The Top Benefits of Ask AI Speed and Precision One of AskAI.io's standout features is the ability to deliver exact answers to complex queries quickly, eliminating the need for users to sift through irrelevant information and ads, which enhances overall efficiency and user satisfaction. Specialized Search Experience Ask AI excels in handling niche and research-intensive queries, making it an ideal tool for professionals who require deep insights rather than surface-level information. Its advanced AI technologies provide tailored results that are context-aware, catering specifically to enterprise needs. User-Friendly Interface Ask AI boasts a clean and intuitive interface that simplifies navigation, allowing users to focus on obtaining information without distractions. This streamlined design contributes to a more enjoyable search experience compared to cluttered alternatives like Google. Accurate Information with Citations Ask AI provides citations for its responses, enabling users to verify sources easily. This feature is particularly valuable for academic research and fact-checking, as it enhances the credibility of the information presented. More Advanced Features of AskAI Mindmap Creation AskAI now offers users the ability to generate visually engaging mindmaps, providing a versatile tool for brainstorming and organizing ideas. With this, users can transform their thoughts into striking visual representations in a matter of seconds. Presentation Slide Generation This innovative platform has also introduced a presentation slide generator, enabling users to create professional-quality slides in just a few clicks. It means that users can instantly generate visually appealing slides with relevant data, streamlining the process of creating captivating presentations. Event Summaries In addition to its already impressive suite of features, AskAI can generate concise summaries of events related to users' queries. This functionality helps users stay informed on key takeaways, ensuring they remain up-to-date with the latest developments in their respective fields. Why Should You Choose Aks AI Instead of Google? Direct Answers: AskAI.io provides direct answers and summaries rather than a list of links, streamlining the information retrieval process. Personalized Search Results: AI search engines like AskAI.io use machine learning algorithms to tailor search results based on individual preferences and past search history. This leads to a more personalized and relevant search experience. Efficiency: Aks AI can process vast amounts of data quickly and efficiently, providing more targeted and up-to-date results. Natural Language Processing: Aks AI can understand queries better thanks to natural language processing (NLP), which helps them interpret context and user intent more accurately. Innovation: As AI technology continues to advance, Ask AI is likely to provide more sophisticated features and improvements over traditional search engines like Google and ChatGPT. Final Thoughts AskAI.io offers a revolutionary AI-driven search experience, addressing the shortcomings of traditional search engines like Google. With its user-friendly interface, tailored results, and advanced features, AskAI.io is a promising development in the evolving landscape of search technology. This story was distributed by Margrowth under HackerNoon’s Brand As An Author Program. Learn more about the program here: https://business.hackernoon.com/brand-as-author Are you overwhelmed by Google’s cluttered interface, filled with ads and irrelevant links? Do you find that navigating through numerous options for simple queries detracts from your search experience? If so, it might be time to explore AI search engines that offer a cleaner, more intuitive alternative. AskAI.io is one of the go-to options, and in this review, we will delve into what makes it a popular choice among users looking for a more streamlined search experience. What Is AskAI.io? AskAI.io is an AI-powered search tool designed to enhance traditional search experiences by providing direct answers and personalized results. Unlike conventional search engines that primarily rely on keyword matching, Ask AI, powered by advanced AI algorithms, understands the context and intent behind user queries, delivering more accurate and relevant information. AskAI.io One of its standout features is the ability to access real-time data from the web, ensuring users receive up-to-date information while maintaining a clean, ad-free interface that improves user experience. It aims to rival established search engines like Google by offering a more conversational and efficient way to find information, making it a promising development in the evolving landscape of search technology. The Top Benefits of Ask AI Speed and Precision One of AskAI.io's standout features is the ability to deliver exact answers to complex queries quickly, eliminating the need for users to sift through irrelevant information and ads, which enhances overall efficiency and user satisfaction. Specialized Search Experience Ask AI excels in handling niche and research-intensive queries, making it an ideal tool for professionals who require deep insights rather than surface-level information. Its advanced AI technologies provide tailored results that are context-aware, catering specifically to enterprise needs. Ask AI User-Friendly Interface Ask AI boasts a clean and intuitive interface that simplifies navigation, allowing users to focus on obtaining information without distractions. This streamlined design contributes to a more enjoyable search experience compared to cluttered alternatives like Google. Accurate Information with Citations Ask AI provides citations for its responses, enabling users to verify sources easily. This feature is particularly valuable for academic research and fact-checking, as it enhances the credibility of the information presented. More Advanced Features of AskAI Mindmap Creation AskAI now offers users the ability to generate visually engaging mindmaps, providing a versatile tool for brainstorming and organizing ideas. With this, users can transform their thoughts into striking visual representations in a matter of seconds. Presentation Slide Generation This innovative platform has also introduced a presentation slide generator, enabling users to create professional-quality slides in just a few clicks. It means that users can instantly generate visually appealing slides with relevant data, streamlining the process of creating captivating presentations. Event Summaries In addition to its already impressive suite of features, AskAI can generate concise summaries of events related to users' queries. This functionality helps users stay informed on key takeaways, ensuring they remain up-to-date with the latest developments in their respective fields. Why Should You Choose Aks AI Instead of Google? Direct Answers: AskAI.io provides direct answers and summaries rather than a list of links, streamlining the information retrieval process. Direct Answers: AskAI.io provides direct answers and summaries rather than a list of links, streamlining the information retrieval process. Personalized Search Results: AI search engines like AskAI.io use machine learning algorithms to tailor search results based on individual preferences and past search history. This leads to a more personalized and relevant search experience. Personalized Search Results: AI search engines like AskAI.io use machine learning algorithms to tailor search results based on individual preferences and past search history. This leads to a more personalized and relevant search experience. Efficiency: Aks AI can process vast amounts of data quickly and efficiently, providing more targeted and up-to-date results. Efficiency: Aks AI can process vast amounts of data quickly and efficiently, providing more targeted and up-to-date results. Natural Language Processing: Aks AI can understand queries better thanks to natural language processing (NLP), which helps them interpret context and user intent more accurately. Natural Language Processing: Aks AI can understand queries better thanks to natural language processing (NLP), which helps them interpret context and user intent more accurately. Innovation: As AI technology continues to advance, Ask AI is likely to provide more sophisticated features and improvements over traditional search engines like Google and ChatGPT. Innovation: As AI technology continues to advance, Ask AI is likely to provide more sophisticated features and improvements over traditional search engines like Google and ChatGPT. Final Thoughts AskAI.io offers a revolutionary AI-driven search experience, addressing the shortcomings of traditional search engines like Google. With its user-friendly interface, tailored results, and advanced features, AskAI.io is a promising development in the evolving landscape of search technology. This story was distributed by Margrowth under HackerNoon’s Brand As An Author Program. Learn more about the program here: https://business.hackernoon.com/brand-as-author This story was distributed by Margrowth under HackerNoon’s Brand As An Author Program. Learn more about the program here: https://business.hackernoon.com/brand-as-author This story was distributed by Margrowth under HackerNoon’s Brand As An Author Program. Learn more about the program here: https://business.hackernoon.com/brand-as-author https://business.hackernoon.com/brand-as-author

AskAI.io Review: Is It the Best AI-Powered Search Engine?

AskAI.io İcmalı: Bu, AI ilə işləyən ən yaxşı axtarış motorudur?

Преглед на AskAI.io: Това ли е най-добрата търсачка, базирана на AI?

Revisión de AskAI.io: ¿Es el mejor motor de búsqueda impulsado por IA?

AskAI.io Iritzia: AI-k bultzatutako bilatzailerik onena al da?

AskAI.io-arvostelu: Onko se paras tekoälyllä toimiva hakukone?

Avis sur AskAI.io : est-ce le meilleur moteur de recherche basé sur l'IA ?

AskAI.io レビュー: 最高の AI 搭載検索エンジンですか?

Преглед на AskAI.io: Дали е најдобриот пребарувач со вештачка интелигенција?

Recenzja AskAI.io: Czy jest to najlepsza wyszukiwarka oparta na sztucznej inteligencji?

Análise do AskAI.io: é o melhor mecanismo de busca com tecnologia de IA?

AskAI.io recension: Är det den bästa AI-drivna sökmotorn?

AskAI.io syn: AI bilen işleýän iň gowy gözleg motorymy?

I Tried Perplexity For a Week, And I Don't Think AI Search Engines Can Replace Google.. Yet

আমি এক সপ্তাহের জন্য বিভ্রান্তির চেষ্টা করেছি, এবং আমি মনে করি না এআই সার্চ ইঞ্জিনগুলি গুগলকে প্রতিস্থাপন করতে পারে.. তবুও

Ich habe Perplexity eine Woche lang ausprobiert und glaube nicht, dass KI-Suchmaschinen Google ersetzen können.

Probé Perplexity durante una semana y no creo que los motores de búsqueda con inteligencia artificial puedan reemplazar a Google... todavía

J'ai essayé Perplexity pendant une semaine et je ne pense pas que les moteurs de recherche IA puissent remplacer Google... pour l'instant

मैंने एक सप्ताह तक पेरप्लेक्सिटी की कोशिश की, और मुझे नहीं लगता कि एआई सर्च इंजन गूगल की जगह ले सकते हैं.. अभी तक

私は1週間Perplexityを試してみましたが、AI検索エンジンがGoogleに取って代わることができるとは思いません...まだ

일주일 동안 Perplexity를 사용해 보았는데, AI 검색 엔진이 아직은 Google을 대체할 수 있다고 생각하지 않습니다.

Eu tentei o Perplexity por uma semana e não acho que os mecanismos de busca de IA possam substituir o Google... ainda

Я пробовал Perplexity в течение недели, и я не думаю, что поисковые системы на основе искусственного интеллекта могут заменить Google... Пока

Perplexity'yi Bir Hafta Boyunca Denedim ve Yapay Zeka Arama Motorlarının Google'ın Yerini Alabileceğini Düşünmüyorum.. Henüz

Tôi đã thử Perplexity trong một tuần và tôi không nghĩ công cụ tìm kiếm AI có thể thay thế Google..

我试用了 Perplexity 一周，我认为人工智能搜索引擎暂时还不能取代谷歌

If an individual is trying to get some sleep, but cannot sleep, what is the reason for that sleepless state? Mind wandering? Pain somewhere? Anxiety? Unconducive environment? Depression? The side effects of medication? Slept some hours before? What is the minimum threshold, in the brain, for sleep to set in? This question means that what components must have what characteristics for the state of sleep to be in effect? Sleep can be characterized as an array, where some components have to align in a certain way for it to be possible. Or, sleep can be said to be a graph where some components have to result in a curve instead of a straight-line, for wakefulness—so to speak. Now, what are the points on this curve? What components have to align and why? What makes them align for sleep? The points on the curve can be described as the interaction of sets of electrical and chemical signals, while the changes that result to make them become a curve are the attributes of the sets of electrical and chemical signals. This means that, conceptually, the two most important components for sleepare the electrical and chemical signals of neurons. So, sleep is a particular alignment or array for some sets of electrical and chemical signals. Adenosine and melatonin are known to be involved in sleep. It is proposed that they are a part of the chemical signals involved, as sets. In nuclei and ganglia [clusters of neurons in the central and peripheral nervous systems], electrical and chemical signals are in sets, conceptually. This means that they provide configurations for functions by interactions. Their states at the time of the interactions define attributes, measures, or grades of those functions. Every smell is a function, like every touch or sight. What makes each distinct is a formation or configuration of sets of signals, respectively. This formation is a combination of interaction + attribute. Simply, functions like memory, emotions, feelings, and regulation of internal senses are a result of the interactions of electrical and chemical signals. Attributes like attention, awareness, intent, and subjectivity are attributes that grade or qualify those interactions. Therefore, in a space, points are the interactions of electrical and chemical signals whose changes are possible by their respective attributes at the time of the interaction. When they align into a curve, because of the combination of interaction + attributes, for some sets, sleep results. Wakefulness When awake, sets of electrical and chemical signals have their interactions + attributes serve exteroception—or external sensory inputs—such as sight, smell, touch, auditory and taste. Sets of signals for internal systems also stay active, like for regulation of digestion, respiration, and so forth, but those for wakefulness often get more attributes. For example, sets of electrical signals for external sensory inputs arrive with higher intensities, resulting in more reaches of attention [or prioritization, an attribute] for several external signals, like sight, smell, or others. Also, because of navigation of the external world and what is possible, intent is used more often, making the attribute [intent] play a role in the arrays or straight-line for wakefulness. While awareness or [pre-prioritization of all other sets] is less than attention [prioritization of a set], there are often more pre-prioritized sets, ready to become prioritized for external sensory inputs, than for internal senses, during wakefulness, conceptually. So, due to these, the shape of wakefulness [say, a straight-line] is different from sleep [a curve]. Sleep Sleep can be described as the interval that sets of electrical and chemical signalshave interactions + attributes of internal senses take precedence, resulting in a curve. This means that sets of signals get prioritized at different intervals, more regularly than they would during the day. However, for sleep to set in, there has to be a drop of some of the interactions + attributes of the components of external senses, which then become openings for those of internal senses to take position. For example, it is possible to feel sleepy when the eyes are open or while walking, but it is easier to fall asleep when the eyes are closed and in a restful state. This is similar to feeling sleepy in a noisy place, but easier to fall asleep in a calmer place. The sleep state means that there is a lack of interactions + attributes for the sets of signals of external senses, so those of internal senses have their attributes jump [including for intent, to stop walking or not to keep the eyes open or to be trying to switch attention away from listening to the sound]. This results in a curve, hence the possibility for sleep. Principally, the attribute that sometimes defines sleep the most is attention or prioritization of a set. There is just one prioritized set of signals at any moment, but there are often fast and numerous interchanges with others that are pre-prioritized. Prioritization mostly means that the set with the most volume of chemical signals in an instance is prioritized. It could also be the set with the highest intensity of electrical signals. If the volume of a set of chemical signals [of an external sensory input] remains high, even if there is no prioritization, it could be tough to fall asleep. Also, if the set for anxiety is prioritized or pre-prioritized but could be prioritized, it would reduce the possibility to result in a curve for sleep. New Sleep Medication Mechanism The problem of sleeplessness that has to do with attention [or prioritization] of a set of signals[of an external sensory input] reducing the possibility for a fall into the curve of sleeplessness to hold can be solved by attention diversion by a medication that makes the set of signals for an internal sense get prioritized. Simply, if there is lack of sleep because attention [an attribute] in on one or more sets of signals for external senses, or even for an internal sense like anxiety, then solving that may require finding a way to make the attention elsewhere. Already, there are active placebos that can produce certain effects in the body. It is possible to explore drug targets for a benign sensation in a part of the body, say the outer ear, towards touching it a bit, to give it prioritization, or towards some effects on the shoulders. The dosage could be useful an hour before trying to fall asleep, and then the reaction to the prioritization could be done, when in the state to fall asleep, to ease sleep and then ensure that the reaction does not remain after waking up. The target here is attention diversion, in a way that does not produce side-effects or have the medication linger longer than necessary, as well as assisted by touching the outer ear or a part of the shoulder to result in sleep. Using the concept towards drug development could become a major solution to some of the sleep problems many people face. The concept also explores the possibility of explaining how the mind works as architecture for some brain disorders. There is a recent book [Doctored: Fraud, Arrogance, and Tragedy in the Quest to Cure Alzheimer’s (February 4, 2025)] excerpted in STAT, How the ‘amyloid mafia’ took over Alzheimer’s research, stating that, “The amyloid mafia is not a real organization, of course. But it aptly characterizes power figures who some say steer grants and career opportunities away from those who suggest alternatives. That influence seemed apparent in the years that followed a famous 2006 experiment in Nature that tied cognitive decline to a specific protein dubbed amyloid-beta*56. When the FDA fully approved Leqembi, the U.S. Centers for Medicare and Medicaid Services approved payments. With pricing set at $26,000 annually, its makers, Eisai and Biogen, hope to earn more than $10 billion a year on the drug by 2031. And many recipients would owe $6,600 in copays, a substantial portion of the $50,290 median annual household income for seniors, making Leqembi unaffordable for many who would ostensibly benefit from it. If widely adopted, it would almost certainly spike drug premiums for Medicare recipients.” If an individual is trying to get some sleep, but cannot sleep, what is the reason for that sleepless state? Mind wandering? Pain somewhere? Anxiety? Unconducive environment? Depression? The side effects of medication? Slept some hours before? What is the minimum threshold, in the brain, for sleep to set in? This question means that what components must have what characteristics for the state of sleep to be in effect? Sleep can be characterized as an array, where some components have to align in a certain way for it to be possible. Or, sleep can be said to be a graph where some components have to result in a curve instead of a straight-line, for wakefulness—so to speak. Now, what are the points on this curve? What components have to align and why? What makes them align for sleep? The points on the curve can be described as the interaction of sets of electrical and chemical signals, while the changes that result to make them become a curve are the attributes of the sets of electrical and chemical signals. This means that, conceptually, the two most important components for sleep are the electrical and chemical signals of neurons. So, sleep is a particular alignment or array for some sets of electrical and chemical signals. Adenosine and melatonin are known to be involved in sleep. It is proposed that they are a part of the chemical signals involved, as sets. conceptually, the two most important components for sleep In nuclei and ganglia [clusters of neurons in the central and peripheral nervous systems], electrical and chemical signals are in sets, conceptually. This means that they provide configurations for functions by interactions. Their states at the time of the interactions define attributes, measures, or grades of those functions. Every smell is a function, like every touch or sight. What makes each distinct is a formation or configuration of sets of signals, respectively. This formation is a combination of interaction + attribute. Simply, functions like memory, emotions, feelings, and regulation of internal senses are a result of the interactions of electrical and chemical signals. Attributes like attention, awareness, intent, and subjectivity are attributes that grade or qualify those interactions. Therefore, in a space, points are the interactions of electrical and chemical signals whose changes are possible by their respective attributes at the time of the interaction. When they align into a curve, because of the combination of interaction + attributes, for some sets, sleep results. Wakefulness Wakefulness When awake, sets of electrical and chemical signals have their interactions + attributes serve exteroception—or external sensory inputs—such as sight, smell, touch, auditory and taste. Sets of signals for internal systems also stay active, like for regulation of digestion, respiration, and so forth, but those for wakefulness often get more attributes. For example, sets of electrical signals for external sensory inputs arrive with higher intensities, resulting in more reaches of attention [or prioritization, an attribute] for several external signals, like sight, smell, or others. Also, because of navigation of the external world and what is possible, intent is used more often, making the attribute [intent] play a role in the arrays or straight-line for wakefulness. While awareness or [pre-prioritization of all other sets] is less than attention [prioritization of a set], there are often more pre-prioritized sets, ready to become prioritized for external sensory inputs, than for internal senses, during wakefulness, conceptually. So, due to these, the shape of wakefulness [say, a straight-line] is different from sleep [a curve]. Sleep Sleep Sleep can be described as the interval that sets of electrical and chemical signals have interactions + attributes of internal senses take precedence, resulting in a curve. This means that sets of signals get prioritized at different intervals, more regularly than they would during the day. described as the interval that sets of electrical and chemical signals However, for sleep to set in, there has to be a drop of some of the interactions + attributes of the components of external senses, which then become openings for those of internal senses to take position. For example, it is possible to feel sleepy when the eyes are open or while walking, but it is easier to fall asleep when the eyes are closed and in a restful state. This is similar to feeling sleepy in a noisy place, but easier to fall asleep in a calmer place. The sleep state means that there is a lack of interactions + attributes for the sets of signals of external senses, so those of internal senses have their attributes jump [including for intent, to stop walking or not to keep the eyes open or to be trying to switch attention away from listening to the sound]. This results in a curve, hence the possibility for sleep. Principally, the attribute that sometimes defines sleep the most is attention or prioritization of a set. There is just one prioritized set of signals at any moment, but there are often fast and numerous interchanges with others that are pre-prioritized. Prioritization mostly means that the set with the most volume of chemical signals in an instance is prioritized. It could also be the set with the highest intensity of electrical signals. If the volume of a set of chemical signals [of an external sensory input] remains high, even if there is no prioritization, it could be tough to fall asleep. Also, if the set for anxiety is prioritized or pre-prioritized but could be prioritized, it would reduce the possibility to result in a curve for sleep. New Sleep Medication Mechanism New Sleep Medication Mechanism The problem of sleeplessness that has to do with attention [or prioritization] of a set of signals [of an external sensory input] reducing the possibility for a fall into the curve of sleeplessness to hold can be solved by attention diversion by a medication that makes the set of signals for an internal sense get prioritized. attention [or prioritization] of a set of signals Simply, if there is lack of sleep because attention [an attribute] in on one or more sets of signals for external senses, or even for an internal sense like anxiety, then solving that may require finding a way to make the attention elsewhere. Already, there are active placebos that can produce certain effects in the body. It is possible to explore drug targets for a benign sensation in a part of the body, say the outer ear, towards touching it a bit, to give it prioritization, or towards some effects on the shoulders. The dosage could be useful an hour before trying to fall asleep, and then the reaction to the prioritization could be done, when in the state to fall asleep, to ease sleep and then ensure that the reaction does not remain after waking up. The target here is attention diversion, in a way that does not produce side-effects or have the medication linger longer than necessary, as well as assisted by touching the outer ear or a part of the shoulder to result in sleep. Using the concept towards drug development could become a major solution to some of the sleep problems many people face. The concept also explores the possibility of explaining how the mind works as architecture for some brain disorders. There is a recent book [Doctored: Fraud, Arrogance, and Tragedy in the Quest to Cure Alzheimer’s (February 4, 2025)] excerpted in STAT , How the ‘amyloid mafia’ took over Alzheimer’s research , stating that, “The amyloid mafia is not a real organization, of course. But it aptly characterizes power figures who some say steer grants and career opportunities away from those who suggest alternatives. That influence seemed apparent in the years that followed a famous 2006 experiment in Nature that tied cognitive decline to a specific protein dubbed amyloid-beta*56. When the FDA fully approved Leqembi, the U.S. Centers for Medicare and Medicaid Services approved payments. With pricing set at $26,000 annually, its makers, Eisai and Biogen, hope to earn more than $10 billion a year on the drug by 2031. And many recipients would owe $6,600 in copays, a substantial portion of the $50,290 median annual household income for seniors, making Leqembi unaffordable for many who would ostensibly benefit from it. If widely adopted, it would almost certainly spike drug premiums for Medicare recipients.” STAT How the ‘amyloid mafia’ took over Alzheimer’s research

Biotech: Mechanism for a New Medication for Sleep?

RYT Layer-1 Blockchain Promises Real-World Solutions with Innovative Proof Mechanism

Security's Moving Parts 01: Linux Access Control Mechanisms

Authors:
(1) Nicholas A. G. Johnson (nagj@mit.edu);
(2) Theo Diamandis (tdiamand@mit.edu);
(3) Alex Evans (aevans@baincapital.com);
(4) Henry de Valence (hdevalence@penumbra.zone);
(5) Guillermo Angeris (gangeris@baincapital.com). Table of Links Abstract and Introduction 1 The concave pro-rata game 1.1 Symmetric pure strict equilibrium 1.2 Uniqueness of equilibrium 1.3 Equilibrium payoff 2 Batched decentralized exchanges 2.1 Arbitrage 3 Conclusion and References A Numerics B Additional Numerics C Relaxing strict concavity D Rosen condition Abstract In this paper, we introduce a family of games called concave pro-rata games. In such a game, players place their assets into a pool, and the pool pays out some concave function of all assets placed into it. Each player then receives a pro-rata share of the payout; i.e., each player receives an amount proportional to how much they placed in the pool. Such games appear in a number of practical scenarios, including as a simplified version of batched decentralized exchanges, such as those proposed by Penumbra. We show that this game has a number of interesting properties, including a symmetric pure equilibrium that is the unique equilibrium of this game, and we prove that its price of anarchy is Ω(n) in the number of players. We also show some numerical results in the iterated setting which suggest that players quickly converge to an equilibrium in iterated play. Introduction Existing blockchain systems come to consensus on transactions in batches, called blocks. Yet the economic mechanisms those transactions interact with are generally designed to process each individual transaction sequentially, making their behavior reliant on the ordering of transactions within the batch. This abstraction mismatch is the primary source of miner extractible value (MEV), defined as economic value that can be captured by the block proposer (originally the miner) who selects and sequences the transactions to be included in the batch [6]. However, rather than trying to blind the block proposer, or choose a “fair” ordering (which is difficult, if not impossible, to construct in any direct sense on current systems) within a batch, we could alternatively attempt to design economic mechanisms which do not depend on the order of transactions within a block, and instead, process each batch of transactions ‘all at once’. These mechanisms would then be aligned with the actual ordering provided by the consensus mechanism, stepping from one batch of transactions to the next in the same discrete time steps in which consensus happens. One such mechanism is a ‘pro-rata mechanism’. In this mechanism, there is some known notion of value: for example, every user might want to trade some asset A for another, say B, and everyone ‘pitches in’ some amount of asset A into a pool. After everyone has placed their amounts, the pool, as a whole, is traded on an exchange for some amount of asset B, and the resulting amount of asset B is distributed back to each player, in proportion to how much of asset A each player placed in the pool. It is not difficult to show that such a mechanism has the desired property: the order in which players placed asset A into the collective pool does not change how much of asset B each player receives. Using some ideas from cryptography, this game can additionally be implemented in a way that does not reveal any one player’s contributions or identity [9], and so may be considered a simultaneous game. On the other hand, mechanisms of this form often lead to interesting phenomena as users must now consider the possible actions of other users when planning their own actions. A natural framework to study these kinds of problems, where players must reason about the strategies of other players, recursively, is via game theory and the study of the equilibria of games [12]. This paper serves to cleanly set up the game resulting from a pro-rata mechanism in a simple mathematical framework and derive a number of useful results for such games. This paper. The paper is organized as follows. We introduce the concave pro-rata game in §1 and show a few interesting properties under mild conditions. Such properties include the existence and uniqueness of a symmetric pure strategy equilibrium and an explicit way of efficiently computing this equilibrium by solving a single variable, unimodal optimization problem. We also show some simple bounds for the price of anarchy. In §2 we then describe how this type of game connects to a recent proposal for a batched decentralized exchange. We run a number of simulations in §A and §B, illustrating the price of anarchy and showing that in the iterated setting agents appear to converge quickly to the specified equilibrium. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Nicholas A. G. Johnson (nagj@mit.edu); (2) Theo Diamandis (tdiamand@mit.edu); (3) Alex Evans (aevans@baincapital.com); (4) Henry de Valence (hdevalence@penumbra.zone); (5) Guillermo Angeris (gangeris@baincapital.com). Authors: Authors: (1) Nicholas A. G. Johnson (nagj@mit.edu); (2) Theo Diamandis (tdiamand@mit.edu); (3) Alex Evans (aevans@baincapital.com); (4) Henry de Valence (hdevalence@penumbra.zone); (5) Guillermo Angeris (gangeris@baincapital.com). Table of Links Abstract and Introduction Abstract and Introduction 1 The concave pro-rata game 1 The concave pro-rata game 1.1 Symmetric pure strict equilibrium 1.1 Symmetric pure strict equilibrium 1.2 Uniqueness of equilibrium 1.2 Uniqueness of equilibrium 1.3 Equilibrium payoff 1.3 Equilibrium payoff 2 Batched decentralized exchanges 2 Batched decentralized exchanges 2.1 Arbitrage 2.1 Arbitrage 3 Conclusion and References 3 Conclusion and References A Numerics A Numerics B Additional Numerics B Additional Numerics C Relaxing strict concavity C Relaxing strict concavity D Rosen condition D Rosen condition Abstract In this paper, we introduce a family of games called concave pro-rata games. In such a game, players place their assets into a pool, and the pool pays out some concave function of all assets placed into it. Each player then receives a pro-rata share of the payout; i.e., each player receives an amount proportional to how much they placed in the pool. Such games appear in a number of practical scenarios, including as a simplified version of batched decentralized exchanges, such as those proposed by Penumbra. We show that this game has a number of interesting properties, including a symmetric pure equilibrium that is the unique equilibrium of this game, and we prove that its price of anarchy is Ω(n) in the number of players. We also show some numerical results in the iterated setting which suggest that players quickly converge to an equilibrium in iterated play. Introduction Existing blockchain systems come to consensus on transactions in batches, called blocks. Yet the economic mechanisms those transactions interact with are generally designed to process each individual transaction sequentially, making their behavior reliant on the ordering of transactions within the batch. This abstraction mismatch is the primary source of miner extractible value (MEV), defined as economic value that can be captured by the block proposer (originally the miner) who selects and sequences the transactions to be included in the batch [6]. However, rather than trying to blind the block proposer, or choose a “fair” ordering (which is difficult, if not impossible, to construct in any direct sense on current systems) within a batch, we could alternatively attempt to design economic mechanisms which do not depend on the order of transactions within a block, and instead, process each batch of transactions ‘all at once’. These mechanisms would then be aligned with the actual ordering provided by the consensus mechanism, stepping from one batch of transactions to the next in the same discrete time steps in which consensus happens. One such mechanism is a ‘pro-rata mechanism’. In this mechanism, there is some known notion of value: for example, every user might want to trade some asset A for another, say B, and everyone ‘pitches in’ some amount of asset A into a pool. After everyone has placed their amounts, the pool, as a whole, is traded on an exchange for some amount of asset B, and the resulting amount of asset B is distributed back to each player, in proportion to how much of asset A each player placed in the pool. It is not difficult to show that such a mechanism has the desired property: the order in which players placed asset A into the collective pool does not change how much of asset B each player receives. Using some ideas from cryptography, this game can additionally be implemented in a way that does not reveal any one player’s contributions or identity [9], and so may be considered a simultaneous game. On the other hand, mechanisms of this form often lead to interesting phenomena as users must now consider the possible actions of other users when planning their own actions. A natural framework to study these kinds of problems, where players must reason about the strategies of other players, recursively, is via game theory and the study of the equilibria of games [12]. This paper serves to cleanly set up the game resulting from a pro-rata mechanism in a simple mathematical framework and derive a number of useful results for such games. This paper. The paper is organized as follows. We introduce the concave pro-rata game in §1 and show a few interesting properties under mild conditions. Such properties include the existence and uniqueness of a symmetric pure strategy equilibrium and an explicit way of efficiently computing this equilibrium by solving a single variable, unimodal optimization problem. We also show some simple bounds for the price of anarchy. In §2 we then describe how this type of game connects to a recent proposal for a batched decentralized exchange. We run a number of simulations in §A and §B, illustrating the price of anarchy and showing that in the iterated setting agents appear to converge quickly to the specified equilibrium. This paper. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Pro-Rata Mechanisms: Game Theory Insights for Decentralized Exchanges

EIP-1559: Separating Mechanisms From Memes

Authors:
(1) Albert Gu, Machine Learning Department, Carnegie Mellon University and with equal contribution;
(2) Tri Dao, Department of Computer Science, Princeton University and with equal contribution. Table of Links Abstract and 1 Introduction 2 State Space Models 3 Selective State Space Models and 3.1 Motivation: Selection as a Means of Compression 3.2 Improving SSMs with Selection 3.3 Efficient Implementation of Selective SSMs 3.4 A Simplified SSM Architecture 3.5 Properties of Selection Mechanisms 3.6 Additional Model Details 4 Empirical Evaluation and 4.1 Synthetic Tasks 4.2 Language Modeling 4.3 DNA Modeling 4.4 Audio Modeling and Generation 4.5 Speed and Memory Benchmarks 4.6 Model Ablations 5 Discussion 6 Conclusion and References A Discussion: Selection Mechanism B Related Work C Mechanics of Selective SSMs D Hardware-aware Algorithm For Selective SSMs E Experimental Details and Additional Results A Discussion: Selection Mechanism Our selection mechanism is inspired by and related to concepts such as gating, hypernetworks, and data-dependence. It can also be viewed as related to “fast weights” (J. Ba et al. 2016), which connects classical RNNs with the mechanism of linear attention (Schlag, Irie, and Schmidhuber 2021). However, we believe that it is a distinct concept that is worth clarifying. Gating. Gating originally referred to the gating mechanisms of RNNs such as the LSTM (Hochreiter and Schmidhuber 1997) and GRU (J. Chung et al. 2014), or the gated equation (5)n Theorem 1. This was interpreted as a particular mechanism for controlling whether to let an input into the hidden state of an RNN. In particular, this affects the propagation of signal through time and causes inputs to interact along the sequence length dimension. However, the concept of gating has since been relaxed in popular usage to simply mean any multiplicative interaction (often with an activation function). For example, elementwise multiplicative components of neural network architectures (that do not interact along sequence length) are now commonly referred to as gated architectures (Hua et al. 2022; Mehta et al. 2023), despite a very different meaning than the original RNN sense. Thus we believe the original concept of RNN gating versus the popular usage of multiplicative gating actually have a very different semantic meaning. Hypernetworks. Hypernetworks refer to neural networks whose parameters are themselves generated by smaller neural networks. The original idea (Ha, Dai, and Quoc V. Le 2017) used it in a narrow sense to define a large RNN whose recurrent parameters are generated by a smaller RNN. Data-dependence. Similar to hypernetworks, data-dependence can refer to any notion where some parameters of the model depend on the data (Poli et al. 2023). This is a rather trivial transformation, yet it technically satisfies the common meanings of gating (since it has a multiplicative “branch”), hypernetworks (since the parameter D is generated by another layer), and data-dependent (since D depends on the data x). However, this in fact simply defines a GLU function, which is so simple that it is often considered just an activation function (Dauphin et al. 2017; Shazeer 2020) instead of a meaningful layer. Selection. Thus, while selection mechanisms could be considered a special case of ideas such as architectural gating, hypernetworks, or data-dependence, so can an enormous range of other constructions—essentially anything with a multiplication, including standard attention mechanisms (Bahdanau, Cho, and Bengio 2015; Vaswani et al. 2017) as well—and we find it uninformative to think of them as such. Instead, we view it as most closely related to the gating mechanism of traditional RNNs, which is a special case (Theorem 1) and also has a deeper history of connections to SSMs through variable (input-dependent) discretization of ∆ (Funahashi and Nakamura 1993; Gu, Dao, et al. 2020; Tallec and Ollivier 2018). We also eschew the term “gating” in favor of selection to clarify the overloaded use of former. More narrowly, we use selection to refer to the mechanistic action of a model to select or ignore inputs and facilitate data interaction along the sequence length (Section 3.1). Beyond selective SSMs and gated RNNs, other examples may include input-dependent convolutions (Kosma, Nikolentzos, and Vazirgiannis 2023; Lioutas and Guo 2020; Lutati, Zimerman, and Wolf 2023; Yang et al. 2019) and even attention. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Albert Gu, Machine Learning Department, Carnegie Mellon University and with equal contribution; (2) Tri Dao, Department of Computer Science, Princeton University and with equal contribution. Authors: Authors: (1) Albert Gu, Machine Learning Department, Carnegie Mellon University and with equal contribution; (2) Tri Dao, Department of Computer Science, Princeton University and with equal contribution. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 State Space Models 2 State Space Models 3 Selective State Space Models and 3.1 Motivation: Selection as a Means of Compression 3 Selective State Space Models and 3.1 Motivation: Selection as a Means of Compression 3.2 Improving SSMs with Selection 3.2 Improving SSMs with Selection 3.3 Efficient Implementation of Selective SSMs 3.3 Efficient Implementation of Selective SSMs 3.4 A Simplified SSM Architecture 3.4 A Simplified SSM Architecture 3.5 Properties of Selection Mechanisms 3.5 Properties of Selection Mechanisms 3.6 Additional Model Details 3.6 Additional Model Details 4 Empirical Evaluation and 4.1 Synthetic Tasks 4 Empirical Evaluation and 4.1 Synthetic Tasks 4.2 Language Modeling 4.2 Language Modeling 4.3 DNA Modeling 4.3 DNA Modeling 4.4 Audio Modeling and Generation 4.4 Audio Modeling and Generation 4.5 Speed and Memory Benchmarks 4.5 Speed and Memory Benchmarks 4.6 Model Ablations 4.6 Model Ablations 5 Discussion 5 Discussion 6 Conclusion and References 6 Conclusion and References A Discussion: Selection Mechanism A Discussion: Selection Mechanism B Related Work B Related Work C Mechanics of Selective SSMs C Mechanics of Selective SSMs D Hardware-aware Algorithm For Selective SSMs D Hardware-aware Algorithm For Selective SSMs E Experimental Details and Additional Results E Experimental Details and Additional Results A Discussion: Selection Mechanism Our selection mechanism is inspired by and related to concepts such as gating, hypernetworks, and data-dependence. It can also be viewed as related to “fast weights” (J. Ba et al. 2016), which connects classical RNNs with the mechanism of linear attention (Schlag, Irie, and Schmidhuber 2021). However, we believe that it is a distinct concept that is worth clarifying. Gating . Gating originally referred to the gating mechanisms of RNNs such as the LSTM (Hochreiter and Schmidhuber 1997) and GRU (J. Chung et al. 2014), or the gated equation (5)n Theorem 1. This was interpreted as a particular mechanism for controlling whether to let an input into the hidden state of an RNN. In particular, this affects the propagation of signal through time and causes inputs to interact along the sequence length dimension. Gating However, the concept of gating has since been relaxed in popular usage to simply mean any multiplicative interaction (often with an activation function). For example, elementwise multiplicative components of neural network architectures (that do not interact along sequence length) are now commonly referred to as gated architectures (Hua et al. 2022; Mehta et al. 2023), despite a very different meaning than the original RNN sense. Thus we believe the original concept of RNN gating versus the popular usage of multiplicative gating actually have a very different semantic meaning. Hypernetworks . Hypernetworks refer to neural networks whose parameters are themselves generated by smaller neural networks. The original idea (Ha, Dai, and Quoc V. Le 2017) used it in a narrow sense to define a large RNN whose recurrent parameters are generated by a smaller RNN. Hypernetworks Data-dependence. Similar to hypernetworks, data-dependence can refer to any notion where some parameters of the model depend on the data (Poli et al. 2023). Data-dependence. This is a rather trivial transformation, yet it technically satisfies the common meanings of gating (since it has a multiplicative “branch”), hypernetworks (since the parameter D is generated by another layer), and data-dependent (since D depends on the data x). However, this in fact simply defines a GLU function, which is so simple that it is often considered just an activation function (Dauphin et al. 2017; Shazeer 2020) instead of a meaningful layer. Selection . Thus, while selection mechanisms could be considered a special case of ideas such as architectural gating, hypernetworks, or data-dependence, so can an enormous range of other constructions—essentially anything with a multiplication, including standard attention mechanisms (Bahdanau, Cho, and Bengio 2015; Vaswani et al. 2017) as well—and we find it uninformative to think of them as such. Selection Instead, we view it as most closely related to the gating mechanism of traditional RNNs, which is a special case (Theorem 1) and also has a deeper history of connections to SSMs through variable (input-dependent) discretization of ∆ (Funahashi and Nakamura 1993; Gu, Dao, et al. 2020; Tallec and Ollivier 2018). We also eschew the term “gating” in favor of selection to clarify the overloaded use of former. More narrowly, we use selection to refer to the mechanistic action of a model to select or ignore inputs and facilitate data interaction along the sequence length (Section 3.1). Beyond selective SSMs and gated RNNs, other examples may include input-dependent convolutions (Kosma, Nikolentzos, and Vazirgiannis 2023; Lioutas and Guo 2020; Lutati, Zimerman, and Wolf 2023; Yang et al. 2019) and even attention. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Why Selection Mechanisms Are Key to the Future of Sequence Modeling

Authors:
(1) Albert Gu, Machine Learning Department, Carnegie Mellon University and with equal contribution;
(2) Tri Dao, Department of Computer Science, Princeton University and with equal contribution. Table of Links Abstract and 1 Introduction 2 State Space Models 3 Selective State Space Models and 3.1 Motivation: Selection as a Means of Compression 3.2 Improving SSMs with Selection 3.3 Efficient Implementation of Selective SSMs 3.4 A Simplified SSM Architecture 3.5 Properties of Selection Mechanisms 3.6 Additional Model Details 4 Empirical Evaluation and 4.1 Synthetic Tasks 4.2 Language Modeling 4.3 DNA Modeling 4.4 Audio Modeling and Generation 4.5 Speed and Memory Benchmarks 4.6 Model Ablations 5 Discussion 6 Conclusion and References A Discussion: Selection Mechanism B Related Work C Mechanics of Selective SSMs D Hardware-aware Algorithm For Selective SSMs E Experimental Details and Additional Results 3.5 Properties of Selection Mechanisms The selection mechanism is a broader concept that can be applied in different ways, such as to more traditional RNNs or CNNs, to different parameters (e.g. A in Algorithm 2), or using different transformations s(x). 3.5.1 Connection to Gating Mechanisms We highlight the most important connection: the classical gating mechanism of RNNs is an instance of our selection mechanism for SSMs. We note that the connection between RNN gating and the discretization of continuous-time systems is well established (Funahashi and Nakamura 1993; Tallec and Ollivier 2018). In fact, Theorem 1 is an improvement of Gu, Johnson, Goel, et al. (2021, Lemma 3.1) generalizing to the ZOH discretization and input-dependent gates (proof in Appendix C). More broadly, ∆ in SSMs can be seen to play a generalized role of the RNN gating mechanism. In line with prior work, we adopt the view that discretization of SSMs is the principled foundation of heuristic gating mechanisms. 3.5.2 Interpretation of Selection Mechanisms We elaborate on two particular mechanistic effects of selection. Filtering Context. It has been empirically observed that many sequence models do not improve with longer context (F. Shi et al. 2023), despite the principle that more context should lead to strictly better performance. An explanation is that many sequence models cannot effectively ignore irrelevant context when necessary; an intuitive example are global convolutions (and general LTI models). On the other hand, selective models can simply reset their state at any time to remove extraneous history, and thus their performance in principle improves monotonicly with context length (e.g. Section 4.3.2). Additionally, we elaborate on effects of each selective parameter. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Albert Gu, Machine Learning Department, Carnegie Mellon University and with equal contribution; (2) Tri Dao, Department of Computer Science, Princeton University and with equal contribution. Authors: Authors: (1) Albert Gu, Machine Learning Department, Carnegie Mellon University and with equal contribution; (2) Tri Dao, Department of Computer Science, Princeton University and with equal contribution. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 State Space Models 2 State Space Models 3 Selective State Space Models and 3.1 Motivation: Selection as a Means of Compression 3 Selective State Space Models and 3.1 Motivation: Selection as a Means of Compression 3.2 Improving SSMs with Selection 3.2 Improving SSMs with Selection 3.3 Efficient Implementation of Selective SSMs 3.3 Efficient Implementation of Selective SSMs 3.4 A Simplified SSM Architecture 3.4 A Simplified SSM Architecture 3.5 Properties of Selection Mechanisms 3.5 Properties of Selection Mechanisms 3.6 Additional Model Details 3.6 Additional Model Details 4 Empirical Evaluation and 4.1 Synthetic Tasks 4 Empirical Evaluation and 4.1 Synthetic Tasks 4.2 Language Modeling 4.2 Language Modeling 4.3 DNA Modeling 4.3 DNA Modeling 4.4 Audio Modeling and Generation 4.4 Audio Modeling and Generation 4.5 Speed and Memory Benchmarks 4.5 Speed and Memory Benchmarks 4.6 Model Ablations 4.6 Model Ablations 5 Discussion 5 Discussion 6 Conclusion and References 6 Conclusion and References A Discussion: Selection Mechanism A Discussion: Selection Mechanism B Related Work B Related Work C Mechanics of Selective SSMs C Mechanics of Selective SSMs D Hardware-aware Algorithm For Selective SSMs D Hardware-aware Algorithm For Selective SSMs E Experimental Details and Additional Results E Experimental Details and Additional Results 3.5 Properties of Selection Mechanisms The selection mechanism is a broader concept that can be applied in different ways, such as to more traditional RNNs or CNNs, to different parameters (e.g. A in Algorithm 2), or using different transformations s(x). 3.5.1 Connection to Gating Mechanisms 3.5.1 Connection to Gating Mechanisms We highlight the most important connection: the classical gating mechanism of RNNs is an instance of our selection mechanism for SSMs. We note that the connection between RNN gating and the discretization of continuous-time systems is well established (Funahashi and Nakamura 1993; Tallec and Ollivier 2018). In fact, Theorem 1 is an improvement of Gu, Johnson, Goel, et al. (2021, Lemma 3.1) generalizing to the ZOH discretization and input-dependent gates (proof in Appendix C). More broadly, ∆ in SSMs can be seen to play a generalized role of the RNN gating mechanism. In line with prior work, we adopt the view that discretization of SSMs is the principled foundation of heuristic gating mechanisms. 3.5.2 Interpretation of Selection Mechanisms 3.5.2 Interpretation of Selection Mechanisms We elaborate on two particular mechanistic effects of selection. Filtering Context. It has been empirically observed that many sequence models do not improve with longer context (F. Shi et al. 2023), despite the principle that more context should lead to strictly better performance. An explanation is that many sequence models cannot effectively ignore irrelevant context when necessary; an intuitive example are global convolutions (and general LTI models). On the other hand, selective models can simply reset their state at any time to remove extraneous history, and thus their performance in principle improves monotonicly with context length (e.g. Section 4.3.2). Filtering Context. Additionally, we elaborate on effects of each selective parameter. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

How Selection Mechanisms Address Long-Context Limitations in AI Sequence Models

Stories, News & Mentions about MERCANIS

Mercanis’s Stories on HackerNoon

Mercanis's News & Mentions