We understand the challenges of scale \u0026 remote work, and provide experienced digital talent - both individuals and full scrums teams that we know can add impact immediately

MVP Match

mvpmatch

Today, I am speaking with Levin Wense, Co-Founder & Managing Director of MVP Match, which connects startups with top remote tech talent. As the world continues to adapt to a more virtual work environment, the ability to find and retain high-quality remote tech talent has become increasingly crucial. We will explore the benefits and challenges of hiring remote tech talent and gain insights from Levin on effectively sourcing, interviewing, and retaining the best remote tech talent. So, without further ado, let's begin the interview and hear what Levin says about this vital topic. Tell Us About Yourself and Why You Created MVP Match I'm Levin Wense, the co-founder and CEO of MVP Match, a top-tier tech talent marketplace launched in 2020. My background is in shaping early-stage tech startups. I held business development roles at Redi (mobile food ordering) and GoButler (AI-powered concierge services) before joining MVP Factory in 2017 as the first employee. Now, it's a well-established Berlin scaleup that designs and builds digital products for clients using agile teams of highly-skilled tech freelancers. Throughout my work, especially as VP for product & talent at MVP Factory, I've witnessed many companies struggle to build successful product teams – first, to identify the correct headcount and skill mix, and second, to find these people. It's a problem when great products don't come to market because the right team isn't there to execute, but it happens daily. On the other side of that coin are the talents themselves. They're trying to sift through and find suitable projects and roles, often landing unfulfilling jobs that don't support their professional growth. And traditional tech recruiting isn't working to close any of these gaps. There was a need for a new model of talent matching led by experts who really understand product building from both a client and talent perspective. So I co-founded MVP Match to do that. What Advantages Can Remote Talent Offer Compared to In-Person Talent? In a nutshell: more and better opportunities for companies and talent. Many companies still need to adapt to the future of work. They are either resisting the change or waiting to see what happens. But the future is obvious: remote-first and agile. Smart companies are quickly moving in this direction or have already arrived. And they are landing the best talent because they are open to recruiting from any location. For instance, our pool of 7,000+ senior-level tech talent is spread over 120 countries. They work in freelance and permanent roles for our clients across the EU and UK. These clients range from small scaleups to global enterprises. As long as the vetting and matching are done well, and projects are managed right, and there's no reason that talent, and even whole teams, can't be distributed anywhere. For talent, working remotely means a quality of life upgrade. It's the ability to grow your career, work on the most exciting and challenging projects, and live where / how you want. These are the conditions that the best tech talent wants, and they don't need to compromise. Can You Tell Us Some Common Mistakes Companies Make When Hiring Decentralized Teams and How They Can Avoid Them? The most common mistake I see is one that affects all companies hiring teams: needing to know how to design a team for your project. By that, I mean not starting with a clear view of the tasks, relationships, and processes you need to successfully get your project off the ground. If you do that first, you should know what roles to hire for. (We often get called in to support the team design piece before the hiring piece.) How Do You Establish and Adjust Work Culture Through Team Augmentations? We work with many companies that manage agile, fluid product teams. They augment their core teams with freelance talent depending on the project stage. Here are a few key things they do to establish and keep a strong work culture. First, they have great project team leads. In addition to day-to-day task oversight, these leads make everyone feel valued and supported (regardless of employment status). Second, they have a solid onboarding process that introduces new team members quickly to people, expectations, tools, and procedures. Third, if they're hiring freelancers, they prefer experienced ones. This typically means talent who has worked at multiple companies on multiple project teams. People who know how to quickly embed in a team, communicate well, document their processes, etc. The ‘Great Resignation’ Indicates Employees Have the Upper Hand. Is the Current Market Downturn Shifting the Power Dynamics Back Towards Employers? I believe we'll see some overall power shifting in 2023 in the job marketplace. This is because workers are less likely to change jobs in times of economic uncertainty. Naturally, that gives employers more leverage over pay, benefits, and requirements, such as where employees work. I also think employers will opt for more liquid workforce options, like freelancers, where they can flex up or down depending on their needs and financial constraints. However, I don't foresee a power shift in the employer's direction when it comes to tech hiring. The recent layoffs in Big Tech grab headlines, but they represent only a tiny percentage of people working in the sector globally. As for senior-level tech talent – developers, engineers, designers, project managers – the demand from companies for both freelance and permanent roles is still fierce. These people will continue to be able to command very competitive wages and work conditions, like remote-first. What’s Your Take on ChatGPT and Its Involvement With R&D? Will It Replace Software Engineers, Make Their Work Better, or Won’t It Have Any Effect? The use of AI in work is inevitable, not just for developers but for all kinds of knowledge workers. My take is that ChatGPT and similar AI models will make the software engineer's work better and more productive. How good you are at promoting an AI chatbot is quickly becoming a critical programming skill. Smart engineers will use it to do repetitive, tedious tasks faster, like writing, debugging, and explaining code snippets. What chatbots can't do – senior engineers know this – is understand the needs of the business and figure out scalable solutions to meet those needs. How Has Developer Demand Evolved Over the Last Year, and What Do You Predict From the Sector in 2023? There have been some notable changes over the last 12 months. For one, the rise in popularity of Web3 is definitely visible and therefore drives the need for talent with experience in blockchain or crypto. Golang is also quickly becoming a standard favorite technology that is being sought after more frequently. The same can be said for Flutter, the go-to framework for mobile. Security is becoming an integral part of development, shifting the requirement for tech talent. Projects that started three to four years ago needed security as a top priority. Now, companies are specifically looking for talent that can plug that gap. Q3 and Q4 of 2022 were unsettling for many industries – it's difficult to predict the outlook for 2023. However, I do think that, given layoffs and economic uncertainty in the near future, the need for remote tech teams will continue to grow and likely at a record pace, especially in expanding sectors like climate tech. Thank you for speaking with me today, Levin!

Hiring Remote Tech Talent: Overcoming the Challenges, Maximizing the Benefits

-𝙷𝚘𝚠 𝚍𝚘 𝚢𝚘𝚞 𝚝𝚎𝚊𝚌𝚑 𝚊 𝚋𝚘𝚢 𝚑𝚘𝚠 𝚝𝚘 𝚍𝚛𝚘𝚠𝚗? There are mismatched socks everywhere you step; it’s like a pandemic of socks. You walk on your toes, careful not to step on any. Today is the day you learn how to swim. You are excited; it shows in your demeanor. “Oga, hurry up, na. We go leave you o.” You hear the voice of your bosom friend and classmate since the beginning of JSS1, the boy who sticks by you like glue. “No dey rush me, I dey come.” You thunder back as he sighs and tells you to hurry it up. Your room is a mess; it always is. You never remember where the ballpoint pens you so dearly love are kept; you never remember if your bag is on the top of a pile of unwashed clothes. You never remember anything, or rather, you choose to forget. “Oboi, this one goes mad o. Shey, you fit swim?” One of your friends asks, and you are tempted to roll your eyes. It is not a question you grace with a response, so you keep mute. The others come to your rescue in a flash. “No dey do like beans na, you know say we wan teach am how to swim. Sometimes this Otis dey do like person wey him brain dey glitch.” Ayo says, looking at Otis with a look of contempt. Ayo is the boy you look up to, the one most girls would kill to talk to. He looks like a cross between an American and a Nigerian, even though he keeps reminding everyone who decides to listen that he’s a full-fledged Yoruba boy. Otis is the guy everybody ignores, even you. He keeps repeating irrelevant details to fill up the silence. “No mind Otis. Na why him dey do like Otu. E no sabi anything. I go teach you how to swim, no worry.” You nod in acceptance, staring at the boy who speaks with such confidence you can’t help being shell-shocked. He’s another guy, you rever, who’s somehow good at whatever he does. Kanye is his name—the boy who does it all. It doesn’t matter what; he’s good at it. The girls used to like him until they got annoyed with him since they consider him a 𝑘𝑛𝑜𝑤-𝑖𝑡-𝑎𝑙𝑙 or rather, 𝐼𝑇𝐾. So, he sticks with the boys now. “I no mind am before na. Na so he dey do. We don know am. No worry, we get him password for hand.” You say with a laugh, as the others chorus. Four boys go to the swimming pool on a blistering hot day. Even the gods of misfortune could see nothing wrong with that idea. Or so you thought. The pool is a large one; even standing at 5’8, you can tell there are parts of the pool you absolutely must not enter. Girls are squealing like ducks out of the water, but you ignore them. It’s how girls are: noisemakers. Even in your school, it’s no different. In a bid to be seen, they often try to outshine the boys who do whatever they do with no regard for who’s watching. You do not like girls. You used to, but now you don’t. You are startled out of your reverie when Otis jostles you forward with a casual smirk on his face. He looks at you and winks while Ayo and Kanye watch you and Otis fool yourselves. The pool looks inviting as some girls leave it, clapping hands and whispering into their ears. You know how they are—gossip, all of them. You cannot remember where your hatred of women stems from, but you do know that it is from a valid place. You hear it’s called misogyny nowadays. Humans have names for everything, even inconsequential things. “So, how we go do am? Who goes in first?” Ayo asks, one of his hands raised. “Who cares? Make everybody enter jare.” Otis enters the conversation, using his body to push you aside from where you were standing. You stand at the pool's edge before you fall, your hands flailing in the air. The thought that you were just 5’8 feet tall in a pool over 7 feet deep strikes you as ironic. Nature does have a sense of humor, a wicked one. You smile, and you plunge into the water. The water cradles your head as you begin to see the beauty in the darkness of the water. The scenery changes; you lie in the arms of the one who broke you, staring into those deep brown eyes that seem to house all the pain in the universe. “I never taught you how to swim, did I?” She says, a sad smile on the edge of her lips. “You? How are you here?” You ask, fighting back tears. She looks as beautiful as you remember from the past, a phantom with no place to call home. “Does it matter? I guess they call this a place between worlds, right? So, you’re dying. How does that make you feel?” She speaks, and you close your eyes momentarily, wondering why she keeps picking you apart. “Why must you be the last thing I see before I die? Haven’t you done enough? You’re not even dead!” You scream at her face, but the only thing that comes out of your mouth are bubbles. “Dead or alive, I’m the one you love above all others. How does it feel to have your last moments with the only girl your family and friends abhor?” She grins at you, and you begin to see how messed up everything is. She isn’t your friend, she isn’t your family. She’s nothing. Yet, she’s everything. “Haven’t you done enough? Why do you keep doing this?” You say, knowing the fight has been drained out of you. “Because I can.” You can hear voices through the depths, but you refuse to heed them. You want to talk to her a little bit longer. “Wake up, dammit!” You hear a voice pulling you out of your private world with her, a voice that seems to break with pain. You don’t ask; you follow. “Thank God. Yes, that’s it. Breathe in, breathe out. Don’t worry about it. You’ll be fine, you’ll be fine…” The voice gets a little more transparent, and you open your eyes groggily to see the panic-stricken faces of Ayo and Kanye hovering over you. They fuss over you as you cough, water coming from your mouth. Your tiredness is overwhelming; your whole being begs you to go back to sleep. But in a haze, you notice that Otis is nowhere to be found. “Where is Otis?” You ask, looking around. “He’s fine. He was the one who went after you.” Ayo replies as though it is obvious. “Okay, where is he?” You ask again, and Ayo and Kanye exchange nervous glances before they try to find Otis. But you don’t say anything because you know more than anyone else. The same way you know you can never be with 𝑉𝑒𝑟𝑎, 𝑡ℎ𝑒 𝑤𝑖𝑡𝑐ℎ as most of your friends call her. You know where Otis is, and your conversation with him once rings loudly in your head. “Omo me, I just dey o. I no sabi swim. I remember when we go bar beach sef. I just dey play with sand. Me wey get that phobia for water. Wetin be the name again, sef?” Otis asks you as you hold back a laugh before replying. “Hydrophobia. The fear of water.” “Yes! Na im be that. Omo, no tell anybody sha. I wan get over am my way.” “No wahala. Nobody go hear.” You remember; you remember it all. You remember the promise you made not to tell anyone. You don’t have to look for Otis; nobody does. You know where he is. He has become one with the water. You do not remember 𝑡ℎ𝑒 𝑤𝑖𝑡𝑐ℎ anymore; everything else flies by in a haze. Did you attend Otis’ funeral? You do not remember. Your friends try to comfort you, but it is a wasted effort. You didn’t exactly love Otis; he was more like a pesky annoying fly that buzzes around and makes a fuss. You do not care about the death of a fly, yet your heart fills up so much that you fear it’s going to burst. You do not cry; crying is for babies. You stare dry-eyed at the future of the boy who never saw 16. They say the state of a person’s room reflects the state of that person’s mind. That’s why yours is squeaky clean, not a pair of socks in sight. Perhaps you do it for Otis, or maybe it’s for yourself. Maybe your room reflects your mental state, a world empty of who you once were. There are no mismatched socks anymore, just memories of the boy who learned to drown in the guise of swimming. “You can not swim. Why, then, did you do it?” You ask the empty air. But no response comes. Not one. You’re alone with the boy you taught how to drown. A fly, that’s all he was. A pesky fly. Why, then, do you cry?

Mismatched Socks

Authors: (1) Nathan Lambert, Allen Institute for AI; (2) Roberto Calandra, TU Dresden. Table of Links Abstract & Introduction Related Work Background Understanding Objective Mismatch Discussions Conclusion Acknowledgments, and References Acknowledgments This work was partly supported by the German Research Foundation (DFG, Deutsche Forschungsgemeinschaft) as part of Germany’s Excellence Strategy – EXC 2050/1 – Project ID 390696704 – Cluster of Excellence “Centre for Tactile Internet with Human-in-the-Loop” (CeTI) of Technische Universität Dresden, and by Bundesministerium für Bildung und Forschung (BMBF) and German Academic Exchange Service (DAAD) in project 57616814 (SECAI, School of Embedded and Composite AI). References Askell, A., Bai, Y., Chen, A., Drain, D., Ganguli, D., Henighan, T., . . . others (2021). A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861. Baheti, A., Lu, X., Brahman, F., Bras, R. L., Sap, M., & Riedl, M. (2023). Improving language models with advantage-based offline policy gradients. arXiv preprint arXiv:2305.14718. Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., . . . others (2022). Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., . . . others (2023). Open problems and fundamental limitations of reinforcement learning from human feedback. arXiv preprint arXiv:2307.15217. Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., & Amodei, D. (2017). Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30. Chua, K., Calandra, R., McAllister, R., & Levine, S. (2018). Deep reinforcement learning in a handful of trials using probabilistic dynamics models. Advances in neural information processing systems, 31. Coste, T., Anwar, U., Kirk, R., & Krueger, D. (2023). Reward model ensembles help mitigate overoptimization. Cui, G., Yuan, L., Ding, N., Yao, G., Zhu, W., Ni, Y., . . . Sun, M. (2023). Ultrafeedback: Boosting language models with high-quality feedback. arXiv preprint arXiv:2310.01377. Deng, H., & Raffel, C. (2023). Reward-augmented decoding: Efficient controlled text generation with a unidirectional reward model. arXiv preprint arXiv:2310.09520. Ethayarajh, K., Choi, Y., & Swayamdipta, S. (2022, 17–23 Jul). Understanding dataset difficulty with V-usable information. In K. Chaudhuri, S. Jegelka, L. Song, C. Szepesvari, G. Niu, & S. Sabato (Eds.), Proceedings of the 39th international conference on machine learning (Vol. 162, pp. 5988–6008). PMLR. Feng, X., Wan, Z., Wen, M., Wen, Y., Zhang, W., & Wang, J. (2023). Alphazero-like tree-search can guide large language model decoding and training. arXiv preprint arXiv:2309.17179. Fernandes, P., Madaan, A., Liu, E., Farinhas, A., Martins, P. H., Bertsch, A., . . . others (2023). Bridging the gap: A survey on integrating (human) feedback for natural language generation. arXiv preprint arXiv:2305.00955. Gao, L., Schulman, J., & Hilton, J. (2022). Scaling laws for reward model overoptimization. arXiv preprint arXiv:2210.10760. Gilbert, T. K., Dean, S., Zick, T., & Lambert, N. (2022). Choices, risks, and reward reports: Charting public policy for reinforcement learning systems. arXiv preprint arXiv:2202.05716. Glaese, A., McAleese, N., Tr˛ebacz, M., Aslanides, J., Firoiu, V., Ewalds, T., . . . others (2022). Improving alignment of dialogue agents via targeted human judgements. arXiv preprint arXiv:2209.14375. Janner, M., Li, Q., & Levine, S. (2021). Offline reinforcement learning as one big sequence modeling problem. Advances in neural information processing systems, 34, 1273–1286. Kiela, D., Thrush, T., Ethayarajh, K., & Singh, A. (2023). Plotting progress in ai. Contextual AI Blog. (https://contextual.ai/blog/plotting-progress) Kirk, R., Mediratta, I., Nalmpantis, C., Luketina, J., Hambro, E., Grefenstette, E., & Raileanu, R. (2023). Understanding the effects of rlhf on llm generalisation and diversity. Knox, W. B., Hatgis-Kessell, S., Adalgeirsson, S. O., Booth, S., Dragan, A., Stone, P., & Niekum, S. (2023). Learning optimal advantage from preferences and mistaking it for reward. Knox, W. B., & Stone, P. (2008). Tamer: Training an agent manually via evaluative reinforcement. In 2008 7th ieee international conference on development and learning (pp. 292–297). Lambert, N., Amos, B., Yadan, O., & Calandra, R. (2020). Objective mismatch in model-based reinforcement learning. In Learning for dynamics and control (pp. 761–770). Lambert, N., Gilbert, T. K., & Zick, T. (2023). Entangled preferences: The history and risks of reinforcement learning and human feedback. Lambert, N., Pister, K., & Calandra, R. (2022). Investigating compounding prediction errors in learned dynamics models. arXiv preprint arXiv:2203.09637. Lambert, N., Tunstall, L., Rajani, N., & Thrush, T. (2023). Huggingface h4 stack exchange preference dataset. Retrieved from https://huggingface.co/datasets/HuggingFaceH4/stack-exchange-preferences Lambert, N., Wilcox, A., Zhang, H., Pister, K. S., & Calandra, R. (2021). Learning accurate long-term dynamics for model-based reinforcement learning. In 2021 60th ieee conference on decision and control (cdc) (pp. 2880–2887). Leike, J., Krueger, D., Everitt, T., Martic, M., Maini, V., & Legg, S. (2018). Scalable agent alignment via reward modeling: a research direction. arXiv preprint arXiv:1811.07871. Li, X., Zhang, T., Dubois, Y., Taori, R., Gulrajani, I., Guestrin, C., . . . Hashimoto, T. B. (2023). Alpacaeval: An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/alpaca_eval. GitHub. Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., . . . others (2022). Holistic evaluation of language models. arXiv preprint arXiv:2211.09110. This paper is under CC 4.0 license. available on arxiv

Objective Mismatch in Reinforcement Learning from Human Feedback: Acknowledgments, and References

Authors: (1) Nathan Lambert, Allen Institute for AI; (2) Roberto Calandra, TU Dresden. Table of Links Abstract & Introduction Related Work Background Understanding Objective Mismatch Discussions Conclusion Acknowledgments, and References 6 Conclusion This paper presents the multiple ways by which objective mismatch limits the accessibility and reliability of RLHF methods. This current disconnect between design a reward model, optimizing it, and the downstream model goals creates a method that is challenging to implement and improve on. Future work mitigating mismatch and the proxy objectives present in RLHF, LLMs and other popular machine learning methods will becomes easier to align with human values and goals, solving many common challenges users encounter with state-of-the-art LLMs. This paper is under CC 4.0 license. available on arxiv

Objective Mismatch in Reinforcement Learning from Human Feedback: Conclusion

Understanding Objective Mismatch

Authors: (1) Nathan Lambert, Allen Institute for AI; (2) Roberto Calandra, TU Dresden. Table of Links Abstract & Introduction Related Work Background Understanding Objective Mismatch Discussions Conclusion Acknowledgments, and References ABSTRACT Reinforcement learning from human feedback (RLHF) has emerged as a powerful technique to make large language models (LLMs) easier to prompt and more capable in complex settings. RLHF at its core is providing a new toolkit to optimize LLMs other than next-token prediction, enabling the integration of qualitative training goals. The attempted match between user preferences and downstream performance, which happens in a learned reward model, results in an optimization landscape where training and evaluation metrics can appear correlated. The apparent correlation can lead to unexpected behaviors and stories of “too much RLHF.” In RLHF, challenges emerge because the following sub-modules are not consistent with each other: the reward model training, the policy model training, and the policy model evaluation. This mismatch results in models that sometimes avoid user requests through false safety flags, are difficult to steer to an intended characteristic, or always answer in a specific style. As chat model evaluation becomes increasingly nuanced, the reliance on a perceived link between reward model score and downstream performance drives the objective mismatch issue. In this paper, we illustrate the cause of this issue, reviewing relevant literature from model-based reinforcement learning, and discuss relevant solutions to encourage further research. By solving objective mismatch in RLHF, the LLMs of the future will be more precisely aligned to user instructions for both safety and helpfulness 1 Introduction Reinforcement learning from human feedback (RLHF) is a powerful tool for integrating qualitative styles and values into large machine learning models (Bai et al., 2022; Christiano et al., 2017; Ouyang et al., 2022). RLHF was popularized with its use to integrated human values into large language models (LLMs) for aligning chat tools (Schulman, Zoph, Kim, & more, 2022). In doing so, RLHF has become an important technique in the process of making models better at responding to user requests, often referred to as instruction-tuned, steerability, chat-models, etc. RLHF methods typically operate in a two step process following the training of a base language model, first they learn a model of human preferences that acts as a reward function, and second they use this model within a reinforcement learning (RL) optimization loop. In the RLHF process, these two steps are often executed independently, with an accurate reward model being trained on human preference data and then the RL optimizer is used to extract maximum information into the chat model. A common challenge of modern LLMs trained with RLHF is difficulties in extracting intended behaviors from the model. Sometimes, the models decline benign requests for safety reasons and other times they need clever prompt tuning to extract full performance. In this paper, we detail a fundamental challenge in modern RLHF learning schemes: the objective mismatch issue. In RLHF, three important parts of training are numerically decoupled: the design of evaluation metrics, the training of a reward model, and the training of the generating model. This mismatch between the reward model and the RL training is visualized in Fig. 1, yet other links exist between the goals of evaluation and simulating human values. Specifically, there are many avenues to better align reward model training to the literature in preference quantification (Lambert, Gilbert, & Zick, 2023) and fundamental optimization challenges need to be solved in RLHF practices (Casper et al., 2023). ChatGPT, the most popular model trained with RLHF, shows signs this limitation through issues such as verbosity, self-doubt and question refusals, repeated phrases, hedging, and more (Schulman, 2023). These traits of overoptimization are results of the subtle proxy objective problem that objective mismatch provides a frame for studying and solving – the reward model attributes excess value to phrases that do not contribute to user benefit, which the RL optimizer exploits, such as safety flags. On the hand, the current training setups are not fully aligned with evaluation tools because the RLHF’d models still need sophisticated prompting techniques such as “thinking step by step” (J. Wei et al., 2022) or “take a deep breath” (Yang et al., 2023) to reach maximum performance. Solving objective mismatch will remove the need for these advanced techniques and reduce the likelihood of out-of-scope refusals from an LLM. The phrase objective mismatch originates from model-based reinforcement learning (MBRL), where an agent iteratively learns a dynamics model that it later uses to solve a control task (Lambert, Amos, Yadan, & Calandra, 2020; R. Wei, Lambert, McDonald, Garcia, & Calandra, 2023). In this context, the mismatch is between learning an accurate dynamics model rather than one that is optimized for high task reward. In RLHF, the problem is related, but with added complexity, as the reward model is optimized for preference data over a closed distribution, which does not match the end users. Second, the task of open-ended language generation is less specific to a notion of reward than that of RL control policies. For these reasons, as we explore in this paper, the objective mismatch issue is more nuanced and critical to RLHF. In this position paper, we make three contributions: • Clearly explain the origins and potential manifestations of objective mismatch in chat-tuned LLMs, • Connect related work from NLP and RL literature around objective mismatch, • Propose directions of study to solve the mismatch and foster better RLHF practices. This paper is under CC 4.0 license. available on arxiv

The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human Feedback

সারিবদ্ধকরণ সিলিং: মানব প্রতিক্রিয়া থেকে শক্তিবৃদ্ধি শিক্ষায় উদ্দেশ্যগত অমিল

Die Ausrichtungsobergrenze: Objektive Diskrepanz beim verstärkenden Lernen aus menschlichem Feedback

El techo de alineación: desajuste objetivo en el aprendizaje por refuerzo a partir de la retroalimentación humana

Le plafond d’alignement : inadéquation des objectifs dans l’apprentissage par renforcement à partir de la rétroaction humaine

संरेखण छत: मानव प्रतिक्रिया से सुदृढीकरण सीखने में उद्देश्य बेमेल

アライメントの天井: 人間のフィードバックからの強化学習における目標の不一致

정렬 천장: 인간 피드백을 통한 강화 학습의 객관적 불일치

O Teto de Alinhamento: Incompatibilidade de Objetivos na Aprendizagem por Reforço a partir do Feedback Humano

Потолок согласования: объективное несоответствие в обучении с подкреплением на основе обратной связи с человеком

Hizalama Tavanı: İnsan Geri Bildiriminden Takviyeli Öğrenimde Hedef Uyuşmazlığı

Mức trần liên kết: Sự không phù hợp khách quan trong việc học tăng cường từ phản hồi của con người

对齐上限：来自人类反馈的强化学习中的目标不匹配

The European Union's proposed Cyber Resilience Act (CRA) aims to bolster c standards for products with digital elements. While the intention is commendable, its current form could inadvertently stifle the open-source community, which forms the backbone of modern software development. It was a main topic at Open Source Summit Europe this year, where the slogan could be found on stickers and slideshows everywhere. ybersecurity #FixTheCRA The Hippocratic Oath and Open Source: A Mismatch Doctors take the Hippocratic Oath, vowing to do no harm and to act in the best interests of their patients. This oath is a commitment to uphold specific ethical standards, ensuring that the patient's well-being is always the priority. The CRA, in its current form, seems to impose a similar oath on open-source developers, holding them accountable for the software they release. However, the comparison between doctors and open-source developers starts to break down when we consider scale and control. A doctor typically has a , allowing them to diagnose, treat, and follow up. In contrast, an open-source maintainer might release software that is downloaded by millions, with no direct relationship or even knowledge of the end-users. Moreover, once the software is out, the developer often has no control over how it's used, modified, or integrated. direct relationship with their patients The CRA's Impact on Open Source Commercial Support of Open Source: The CRA's current text only excludes open-source software (OSS) that has no commercial activity around it. This vague definition could mean that if the main contributors to an OSS project are employed, the project might be considered commercially tainted. Such a stance could discourage commercial entities from supporting open-source projects, leading to a potential decline in the quality and quantity of open-source contributions. Vulnerability Disclosure: The CRA would require vulnerabilities to be disclosed within hours of discovery, even if no fix is available. This could jeopardize coordinated disclosure efforts, where researchers give vendors time to develop security patches before vulnerabilities are disclosed publicly. Such a move could inadvertently expose software to more risks. Zero-Days and the CRA: Zero-day vulnerabilities are software vulnerabilities unknown to those who should be interested in mitigating them. If the CRA mandates immediate disclosure of vulnerabilities without allowing time for a fix, it could lead to an increase in zero-day exploits. Attackers would have information about vulnerabilities before patches are available, giving them a significant advantage. Compliance Overload: The CRA could impose a heavy compliance burden on open-source developers, especially those working on critical products. Many open-source projects operate without the financial or legal resources to navigate complex regulatory landscapes. The potential legal risks could deter developers from contributing to open-source projects. Conclusion While the CRA's intentions are noble, its potential impact on the open-source community could be detrimental. The open-source ecosystem thrives on collaboration, innovation, and shared responsibility. Imposing stringent regulations without considering the unique dynamics of this community could stifle innovation and hinder global collaboration. It's crucial to strike a balance that without compromising the spirit of open source. ensures cybersecurity On a personal note, this article was in part inspired by the NHS Doctors Strike in the UK this week. It’s made me reflect very deeply on how much harm can come from putting too much burden, with too little support, on people in critical services. More Resources to Understand #FixTheCRA: [ ] [ ] [[Digital Strategy EC](https://https://[Digital Strategy EC)] [ ] DevOps Linux Foundation TechCrunch Title Image developed by , inspired by this article’s title words. deepai.org

The Cyber Resilience Act (CRA) and Open Source: A Mismatched Prescription

Effective Communication at Work: Key Practices to Enhance Collaboration and Success

is a popular technology for efficient and scalable data storage and retrieval. However, maintaining its performance and data integrity requires a crucial practice called reindexing. Indexing is the initial process of adding data to Elasticsearch, while reindexing is essential for maintaining data accuracy and optimizing search performance. Elasticsearch Whether you're a seasoned Elasticsearch user or just beginning your journey, understanding reindexing is important for maintaining an efficient Elasticsearch cluster. In this article, we'll delve into the essentials of Elasticsearch reindexing, answering when it's necessary, how to trigger it, and the best practices to get the most out of your Elasticsearch cluster. Understanding Elasticsearch Reindexing In Elasticsearch, reindexing helps maintain data integrity and increase performance. Put simply, it's the process of copying data from one index to another. While this might sound straightforward, if not done correctly it can cause issues such as slow data retrieval or even incorrect results Imagine your Elasticsearch indices as well-organized libraries. Over time, books might need to be updated, rearranged, or even replaced. Reindexing is akin to rearranging the library shelves or updating the books to keep everything in order. Without it, your library can become disorganized, leading to slower searches and potential inaccuracies in your data. This analogy underscores the importance of understanding reindexing in Elasticsearch. It's not just about copying data; it's about maintaining the integrity of your "library" for efficient searching and retrieval. Let’s take a look at and how to keep on top of it. when reindexing is required When is Reindexing Necessary? Reindexing becomes essential when changes occur in your Elasticsearch data models or mappings, or when you're seeking performance enhancements. In this section, we’ll look into these scenarios in more detail to understand the nuances around why reindexing is required. Structural Changes in Data Models Structural changes in data models refer to modifications in how data is structured within Elasticsearch. These changes can include things like adding or removing new fields or altering data types of existing fields. Introducing new fields often requires a reindex to ensure Elasticsearch knows how to efficiently search for data stored in that field. Modifying data types requires a new index altogether as you cannot change data types in place. Once the new mapping has been created for the modified data type then the data needs reindexing. These structural changes require reindexing due to Elasticsearch's schema-on-write approach. Elasticsearch indexes data as it is ingested, and any changes to the data structure can lead to inconsistencies between existing data and data written with the new schema. As a result, without reindexing, search queries may yield unexpected or inaccurate results due to the schema mismatch of data items. This can have an impact on both data accuracy and search performance. Mapping Updates or Changes Mappings serve as the blueprint for how data is indexed and queried in Elasticsearch. When these mappings are modified then reindexing is usually required. Mappings define the data types and properties of fields within Elasticsearch. Any change to these mappings affects how data is indexed, stored, and retrieved. For instance, altering a text field to a date field fundamentally changes how data is processed and queried. Elasticsearch enforces data consistency based on mapping definitions. Changes to mappings can lead to inconsistencies between existing data and the updated schema if the data is not reindexed. When mappings are modified, particularly if it involves changing data types or field properties, backfilling also becomes important. Backfilling is the process of retroactively populating or updating existing data to align it with a new schema or data structure. This means that the existing data can still be queried efficiently and accurately after the mapping change. Performance Enhancements and Index Optimizations Reindexing isn't just a routine maintenance task, it's a powerful tool for optimizing search performance within Elasticsearch. For example, reindexing allows you to modify the number of shards in an index. Adjusting the shard count, or resharding, can distribute data more evenly, preventing uneven workloads on specific nodes to improve search performance. Reindexing can also be used to consolidate indices together. Let's say you have multiple small indices that share the same data structure and are frequently queried together. Reindexing can consolidate them into a single, larger index. This reduces the overhead of managing numerous small indices which can in turn enhance search speed. Finally, reindexing can be used to improve routing. By reindexing and applying routing strategies effectively, you can route queries to specific shards, minimizing the number of shards that need to be searched. This targeted approach can significantly speed up search queries if your data is frequently searched by specific keys such as a user ID. Upgrading Your Cluster When upgrading from Elasticsearch version 6.X to 8.0 (current major version) and beyond, you may need to reindex any indices that were created in version 6. Elasticsearch's data structures and underlying mechanisms changed significantly between these versions requiring reindexing for compatibility and optimal performance. The reindexing process ensures that data aligns with the updated structure and new functionality to ensure you can migrate seamlessly from old to new. Elasticsearch recommends using their upgrade assistant to help with this process. How to Trigger a Reindexing Operation Reindexing in Elasticsearch is made possible through the Elasticsearch Reindex API. The Reindex API serves as the bridge between your existing index and the new index you want to create or modify. Its primary purpose is to enable the efficient transfer of data from one index to another, on top of this, you can also: Selectively copy documents from the source index to the target index. Apply complex data transformations, such as field renaming or type conversions. Filter data based on specific criteria. Control the indexing process with options like throttling and refresh intervals. Before using the Reindex API, ensure that the target index, where you want to move or transform your data, is created and properly configured. To trigger reindexing, you then need to formulate a POST request to the endpoint, specifying the source and target indices, as well as any desired transformations or filters. An example reindex POST request could look as follows. _reindex POST /_reindex
{
  "source": {
    "index": "source_index"
  },
  "dest": {
    "index": "target_index"
  },
  "script": {
    "source": "ctx._source.new_field = 'transformed value'"
  },
  "query": {
    "term": {
      "category.keyword": "example"
    }
  }
} Once your request is built you can send the request to Elasticsearch, initiating the reindexing process. Elasticsearch will start copying data from the source index to the target index, following your defined instructions. Once the reindexing is complete, thoroughly test the data in the target index to ensure it aligns with your expectations. For example, you can compare the field mappings between the source and target indices to confirm that fields were mapped correctly during reindexing. You could also retrieve a sample of documents from both the source and target indices and compare them to verify the data was reindexed accurately. Best Practices for Reindexing When reindexing within Elasticsearch, you should look to follow these best practices to ensure the reindexing procedure is smooth with no data loss and little impact on existing cluster operations. Prioritize Data Backup Before initiating any reindexing activity, it is important to back up your cluster. This precautionary step acts as a safety net, offering a way to revert to the original state should any unexpected issues arise during the reindexing process. The source index should still exist after reindexing, however, it's a fundamental principle to always have a reliable copy of your data before making significant changes. Conduct Reindexing in a Controlled Environment First To mitigate potential risks and challenges during reindexing, it is advisable to perform the operation in a pre-production environment first. By doing so, you can identify and address any unforeseen issues without affecting the production system. Once the procedure has been completed and verified in the pre-production environment, it can then safely be run in production. Monitor Resource Usage It is important to monitor system resources during reindexing to prevent strain on your infrastructure. Reindexing can be resource-intensive, especially for larger datasets. Keeping a close eye on CPU, memory, disk usage, and network activity can help optimize resource allocation, ensuring the process runs efficiently without causing performance bottlenecks. To check resource usage you can use the node stats API. GET /_nodes/stats This will return a response that looks as follows. {
  "_nodes": {
    "total": 2,
    "successful": 2,
    "failed": 0
  },
  "cluster_name": "my_cluster",
  "nodes": {
    "node_id1": {
      "name": "node_name1",
      "process": {
        "cpu": {
          "percent": 30,
        }
      },
      "jvm": {
        "mem": {
          "heap_used_percent": 40.3,
          "heap_used_in_bytes": 123456789,
          "heap_max_in_bytes": 256000000
        }
      }
    },
    "node_id2": {
      "name": "node_name2",
      "process": {
        "cpu": {
          "percent": 50,
        }
      },
      "jvm": {
        "mem": {
          "heap_used_percent": 60.8,
          "heap_used_in_bytes": 210987654,
          "heap_max_in_bytes": 256000000
        }
      }
    }
  }
} If you find reindexing is too intensive, you can throttle the process by setting the parameter when submitting the reindex request. This will add a sleep between batches for the number of seconds set by the parameter, to provide a cooldown period between batches. requests_per_second Verify and Validate Results Once the reindexing is complete you should verify the data in the target index to ensure it looks as expected. This validation process should encompass a variety of tests including document counts, field mappings, and search queries. Alternative Solutions Elasticsearch has undoubtedly established itself as a prominent solution in the NoSQL search and analytics space. However, it's worth exploring alternative solutions that offer unique approaches to data indexing and querying, particularly one like Rockset. Rockset is a to Elasticsearch and offers a different perspective on indexing and querying data. Unlike Elasticsearch's schema-on-write approach, Rockset allows schemaless ingestion. Data can be ingested and queried without the need for upfront schema definition, offering more flexibility in handling ever-evolving datasets without the need for reindexing. cloud-native alternative In the area of index management, Rockset benefits from its model where a row index, a column index, and a search index are all created automatically for the data as it is ingested. This contrasts with Elasticsearch, where indexes are created by users and structural changes often necessitate time-consuming reindexing procedures. converged indexing While Elasticsearch remains a robust solution for various use cases, exploring alternatives like Rockset may be useful, especially if you find reindexing in Elasticsearch becoming a frequent activity. Conclusion Reindexing is a fundamental process in Elasticsearch and is important for maintaining the efficiency and accuracy of search results as data structures evolve. If you find that reindexing is becoming a constant time burden for your team it might be worth exploring alternative solutions like Rockset. Rockset offers a more streamlined index management process that enables developers to concentrate on more value-add activities.

Understanding Elasticsearch Reindexing: When to Reindex, Best Practices and Alternatives

ইলাস্টিকসার্চ রিইন্ডেক্সিং বোঝা: কখন পুনঃসূচীকরণ করতে হবে, সর্বোত্তম অনুশীলন এবং বিকল্প

Neuindizierung von Elasticsearch verstehen: Wann eine Neuindizierung erfolgen sollte, Best Practices und Alternativen

Comprensión de la reindexación de Elasticsearch: cuándo reindexar, mejores prácticas y alternativas

Comprendre la réindexation Elasticsearch : quand réindexer, bonnes pratiques et alternatives

इलास्टिकसर्च रीइंडेक्सिंग को समझना: कब रीइंडेक्स करना है, सर्वोत्तम अभ्यास और विकल्प

Elasticsearch の再インデックスを理解する: 再インデックスを行うタイミング、ベストプラクティス、代替手段

Elasticsearch 재인덱싱 이해: 재인덱싱 시기, 모범 사례 및 대안

Compreendendo a reindexação do Elasticsearch: quando reindexar, práticas recomendadas e alternativas

Понимание переиндексации Elasticsearch: когда проводить переиндексацию, лучшие практики и альтернативы

Elasticsearch Yeniden İndekslemeyi Anlamak: Ne Zaman Yeniden İndekslenmeli, En İyi Uygulamalar ve Alternatifler

Tìm hiểu về lập chỉ mục lại Elaticsearch: Khi nào nên lập chỉ mục lại, các phương pháp hay nhất và giải pháp thay thế

了解 Elasticsearch 重新索引：何时重新索引、最佳实践和替代方案

Why I Switched to Table Driven Testing approach in Go

Authors:
(1) Jingjing Wang, School of Computing Clemson University Clemson, South Carolina, USA;
(2) Joshua Luo, The Westminster Schools Atlanta, Georgia, USA;
(3) Grace Yang, South Windsor High School South Windsor, Connecticut, USA;
(4) Allen Hong, D.W. Daniel High School Clemson, South Carolina, USA;
(5) Feng Luo, School of Computing Clemson University Clemson, South Carolina, USA. Table of Links Abstract & Introduction Related Work Methodology Experiment Result Discussion and References II. RELATED WORK A. Large Language Model Prompting The remarkable success of LLMs, such as GPT and its variants, has also brought prompt-based learning to the fore. Prompt-based learning has found applications in various natural language processing (NLP) tasks, such as sentiment classification and natural language inference. Prompting visuallanguage models for computer vision tasks is another area that has started to gain attention [13], [23]. However, the existing body of work in prompt-based learning has primarily focused on unimodal tasks, with limited research delving into multimodal tasks [22], [30]. A few exceptions like the work [15] attempted multimodal tasks by utilizing the GPT-3 model [18] for science question answering tasks, where the questions have either an image or text context or both. However, these attempts have their own limitations, with models like GPT-3 being less accurate and possessing restrictive input length constraints, thereby limiting their performance and utility. GPT-3.5-Turbo, which performs much better than GPT-3, provides an alternative solution. It presents a practical and efficient route for prompt-based learning in multimodal tasks. Therefore, our research focuses on leveraging the capabilities of the GPT-3.5-Turbo for classifying sentiments in memes, which requires the simultaneous processing of visual and textual data. Additionally, research indicates that when equipped with a range of external NLP tools [16], [17], Large Language Models (LLMs) can serve as effective action planners, selecting and utilizing tools for problem-solving. It suggests that these models can be extended to more complex multimodal scenarios, involving both reasoning and action, thereby enhancing their capabilities. The GPT series has played a pivotal role in promoting prompt-based learning. We follow many of its core concepts in our work, but our focus is different. While most studies are set on applying prompting to extract knowledge from pre-trained models, our goal is to employ these techniques to fine-tune downstream tasks. Specifically, we are interested in exploring the application of these methods to the sentiment classification of memes, a complex and nuanced multimodal task. B. Hateful Meme Detection and Memotion Analysis Navigation Detecting hateful content in memes is a difficult process. The subtle undertones of humor and sarcasm are often interwoven within memes, coupled with the potential mismatch between the images and text, present formidable obstacles to traditional text-based hate speech detection approaches. This challenge has been the focal point of several studies, all in an effort to establish viable solutions [25], [26], [31]. The Hateful Memes Challenge initiated by Facebook stands as a pioneering effort in this sphere, inviting researchers worldwide to develop models capable of identifying hate speech in multimodal meme content. The dataset provided by Facebook, consisting of 10,000 manually annotated memes for hate speech, has proven to be an invaluable resource propelling research in this domain. Mirroring these endeavors, a model that combines multitask learning was proposed to detect hateful and offensive content present in memes. A crucial component of this model is its ability to help GPT model to view the images and do the hateful/non-hateful classification, thereby contributing to a better comprehension of hateful memes. In spite of these extensive efforts, the task of hateful meme detection remains intricate and largely unsolved, attributed to the nuanced interaction of language and imagery inherent to the meme content. The ongoing research and continuous strides towards advancements hint at a necessity for more sophisticated models, ones that can manage these complexities and accurately pinpoint harmful or offensive content within memes. Memotion analysis offers a captivating intersection between the realms of computer vision, natural language processing, and cognitive science. The main aim is to unpack the intricate layers of meaning, sentiment, and emotion encapsulated within memes, an endeavor that has seen many researchers strive to establish computational models to this end [27], [28]. A noteworthy stride in this area was the SemEval 2020 Task 8: Multimodal Memotion Analysis [9]. This project tasked various models with the prediction of sentiment, humor, sarcasm and offensiveness in memes. Taking a similar direction, another researcher embarked on the creation of an automatic meme generator capable of tailoring its output based on the targeted sentiment [19]. Within this project, we proposed a similar framework for hateful meme detection tasks with some more detailed prompts by utilizing multimodal sentiment analysis and sarcasm detection to dissect visual and textual cues present in memes. The insights gathered from our work have shed light on the intricate facets of memotion analysis, showcasing the potential and limitations of AI in this field. This paper is available on arxiv under CC 4.0 license. Authors: (1) Jingjing Wang, School of Computing Clemson University Clemson, South Carolina, USA; (2) Joshua Luo, The Westminster Schools Atlanta, Georgia, USA; (3) Grace Yang, South Windsor High School South Windsor, Connecticut, USA; (4) Allen Hong, D.W. Daniel High School Clemson, South Carolina, USA; (5) Feng Luo, School of Computing Clemson University Clemson, South Carolina, USA. Authors: (1) Jingjing Wang, School of Computing Clemson University Clemson, South Carolina, USA; (2) Joshua Luo, The Westminster Schools Atlanta, Georgia, USA; (3) Grace Yang, South Windsor High School South Windsor, Connecticut, USA; (4) Allen Hong, D.W. Daniel High School Clemson, South Carolina, USA; (5) Feng Luo, School of Computing Clemson University Clemson, South Carolina, USA. Table of Links Abstract & Introduction Abstract & Introduction Related Work Related Work Methodology Methodology Experiment Result Experiment Result Discussion and References Discussion and References II. RELATED WORK A. Large Language Model Prompting A. Large Language Model Prompting The remarkable success of LLMs, such as GPT and its variants, has also brought prompt-based learning to the fore. Prompt-based learning has found applications in various natural language processing (NLP) tasks, such as sentiment classification and natural language inference. Prompting visuallanguage models for computer vision tasks is another area that has started to gain attention [13], [23]. However, the existing body of work in prompt-based learning has primarily focused on unimodal tasks, with limited research delving into multimodal tasks [22], [30]. A few exceptions like the work [15] attempted multimodal tasks by utilizing the GPT-3 model [18] for science question answering tasks, where the questions have either an image or text context or both. However, these attempts have their own limitations, with models like GPT-3 being less accurate and possessing restrictive input length constraints, thereby limiting their performance and utility. GPT-3.5-Turbo, which performs much better than GPT-3, provides an alternative solution. It presents a practical and efficient route for prompt-based learning in multimodal tasks. Therefore, our research focuses on leveraging the capabilities of the GPT-3.5-Turbo for classifying sentiments in memes, which requires the simultaneous processing of visual and textual data. Additionally, research indicates that when equipped with a range of external NLP tools [16], [17], Large Language Models (LLMs) can serve as effective action planners, selecting and utilizing tools for problem-solving. It suggests that these models can be extended to more complex multimodal scenarios, involving both reasoning and action, thereby enhancing their capabilities. The GPT series has played a pivotal role in promoting prompt-based learning. We follow many of its core concepts in our work, but our focus is different. While most studies are set on applying prompting to extract knowledge from pre-trained models, our goal is to employ these techniques to fine-tune downstream tasks. Specifically, we are interested in exploring the application of these methods to the sentiment classification of memes, a complex and nuanced multimodal task. B. Hateful Meme Detection and Memotion Analysis Navigation B. Hateful Meme Detection and Memotion Analysis Navigation Detecting hateful content in memes is a difficult process. The subtle undertones of humor and sarcasm are often interwoven within memes, coupled with the potential mismatch between the images and text, present formidable obstacles to traditional text-based hate speech detection approaches. This challenge has been the focal point of several studies, all in an effort to establish viable solutions [25], [26], [31]. The Hateful Memes Challenge initiated by Facebook stands as a pioneering effort in this sphere, inviting researchers worldwide to develop models capable of identifying hate speech in multimodal meme content. The dataset provided by Facebook, consisting of 10,000 manually annotated memes for hate speech, has proven to be an invaluable resource propelling research in this domain. Mirroring these endeavors, a model that combines multitask learning was proposed to detect hateful and offensive content present in memes. A crucial component of this model is its ability to help GPT model to view the images and do the hateful/non-hateful classification, thereby contributing to a better comprehension of hateful memes. In spite of these extensive efforts, the task of hateful meme detection remains intricate and largely unsolved, attributed to the nuanced interaction of language and imagery inherent to the meme content. The ongoing research and continuous strides towards advancements hint at a necessity for more sophisticated models, ones that can manage these complexities and accurately pinpoint harmful or offensive content within memes. Memotion analysis offers a captivating intersection between the realms of computer vision, natural language processing, and cognitive science. The main aim is to unpack the intricate layers of meaning, sentiment, and emotion encapsulated within memes, an endeavor that has seen many researchers strive to establish computational models to this end [27], [28]. A noteworthy stride in this area was the SemEval 2020 Task 8: Multimodal Memotion Analysis [9]. This project tasked various models with the prediction of sentiment, humor, sarcasm and offensiveness in memes. Taking a similar direction, another researcher embarked on the creation of an automatic meme generator capable of tailoring its output based on the targeted sentiment [19]. Within this project, we proposed a similar framework for hateful meme detection tasks with some more detailed prompts by utilizing multimodal sentiment analysis and sarcasm detection to dissect visual and textual cues present in memes. The insights gathered from our work have shed light on the intricate facets of memotion analysis, showcasing the potential and limitations of AI in this field. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Using GPT-3.5-Turbo for Multimodal Sentiment Analysis

The modern data landscape demands a new breed of infrastructure – one that seamlessly integrates structured and unstructured data, scales effortlessly, and empowers efficient AI/ML workloads. This is where come in, providing a central hub for all your data needs. However, building and managing an effective data lake can be complex. modern datalakes This blog post dives deep into three powerful tools that can optimize your current approach: , and MinIO. The steps below will walk you through how these services seamlessly combine to create a robust, cloud-native data lake architecture specifically optimized for AI/ML workloads. Apache Iceberg Tabular, What is Tabular? Tabular is a data platform created by the original creators of Apache Iceberg. It is designed to provide an independent, universal storage platform that connects to any compute layer, eliminating data vendor lock-in. This feature is critical to the modern data stack, it allows users to pick best-in-class compute and storage tools without being forced into a particular vendor's aged and or mismatched tool set. In an of MinIO and Iceberg and can be enhanced by Tabular. Tabular can be used to manage and query Iceberg data stored in MinIO, allowing for the storage and management of structured data in a scalable, high-performance, and cloud-native manner. These Kubernetes native, components work together smoothly with very little friction and build on each other’s capabilities to perform at scale. architecture Why S3FileIO instead of Hadoop's file-io? This implementation leverages Iceberg's S3FileIO. S3FileIO is considered better than Hadoop's file-io for several reasons. Some of which we’ve already discussed : elsewhere : Iceberg's S3FileIO is designed to work with cloud-native storage. Optimized for Cloud Storage Iceberg uses an to distribute files across multiple prefixes in a MinIO bucket, which helps to minimize throttling and maximize throughput for S3-related IO operations. Improved Throughput and Minimized Throttling: ObjectStoreLocationProvider Iceberg has been to fully leverage strict consistency by eliminating redundant consistency checks that could impact performance. Strict Consistency: updated Iceberg's S3FileIO implements a algorithm, which uploads data files parts in parallel as soon as each part is ready, reducing local disk usage and increasing upload speed. Progressive Multipart Upload: progressive multipart upload Iceberg allows for for S3 API writes to ensure the integrity of uploaded objects, which can be enabled by setting the appropriate catalog property. Checksum Verification: checksum validations Iceberg supports adding to objects during write and delete operations with the S3 API, which can be useful for cost tracking and management. Custom Tags: custom tags The FileIO interface in Iceberg does not require as strict guarantees as a Hadoop-compatible FileSystem, which allows it to that could otherwise degrade performance. Avoidance of Negative Caching: avoid negative caching In contrast, Hadoop's S3A FileSystem, which was used before S3FileIO, does not offer the same level of optimization for cloud storage. All this to say: Don’t hobble your future-facing data lake infrastructure with the trappings of the past. Prerequisites Before you begin, make sure your system meets the following requirements: Docker Docker Compose If you're starting from scratch, you can install both using the installer for your specific platform. It is often easier than downloading Docker and Docker Compose separately. Verify if Docker is installed by running the following command: Docker Desktop docker-compose --version Getting started To get started, clone or copy the YAML file in Tabular’s . You just need the YAML for this tutorial. Feel free to explore the rest of the repository at a later time. git repository Breaking it Down The provided YAML file is a Docker Compose configuration file. It defines a set of services and their configurations for a multi-container Docker application. In this case, there are two services: Spark-Iceberg and MinIO. Let's break down each section: 1. Spark-Iceberg Service: spark-iceberg:
    image: tabulario/spark-iceberg
    container_name: spark-iceberg
    build: spark/
    networks:
      iceberg_net:
    depends_on:
      - rest
      - minio
    volumes:
      - ./warehouse:/home/iceberg/warehouse
      - ./notebooks:/home/iceberg/notebooks/notebooks
    environment:
      - AWS_ACCESS_KEY_ID=admin
      - AWS_SECRET_ACCESS_KEY=password
      - AWS_REGION=us-east-1
    ports:
      - 8888:8888
      - 8080:8080
      - 10000:10000
      - 10001:10001
  rest:
    image: tabulario/iceberg-rest
    container_name: iceberg-rest
    networks:
      iceberg_net:
    ports:
      - 8181:8181
    environment:
      - AWS_ACCESS_KEY_ID=admin
      - AWS_SECRET_ACCESS_KEY=password
      - AWS_REGION=us-east-1
      - CATALOG_WAREHOUSE=s3://warehouse/
      - CATALOG_IO__IMPL=org.apache.iceberg.aws.s3.S3FileIO
      - CATALOG_S3_ENDPOINT=http://minio:9000 Specifies the Docker image to use for the spark-iceberg service. In this case, it uses the tabulario/spark-iceberg:latest image. image: Specifies that the spark-iceberg service depends on the rest and minio services. depends_on: Assigns a specific name (spark-iceberg) to the container. container_name: Sets environment variables for the container, including Spark and AWS credentials. environment: Mounts local directories (./warehouse and ./notebooks) as volumes inside the container. volumes: Maps container ports to host ports for accessing Spark UI and other services. ports: 2. Minio Service: minio:
    image: minio/minio
    container_name: minio
    environment:
      - MINIO_ROOT_USER=admin
      - MINIO_ROOT_PASSWORD=password
      - MINIO_DOMAIN=minio
    networks:
      iceberg_net:
        aliases:
          - warehouse.minio
    ports:
      - 9001:9001
      - 9000:9000
    command: ["server", "/data", "--console-address", ":9001"] Specifies the Docker image for the MinIO service. image: Assigns a specific name (MinIO) to the container. container_name: Sets environment variables for configuring MinIO, including root user credentials. environment: Maps container ports to host ports for accessing MinIO UI. ports: Specifies the command to start the MinIO server with specific parameters. command: Another aspect of the MinIO service is , MinIO’s command line tool. mc mc:
    depends_on:
      - minio
    image: minio/mc
    container_name: mc
    networks:
      iceberg_net:
    environment:
      - AWS_ACCESS_KEY_ID=admin
      - AWS_SECRET_ACCESS_KEY=password
      - AWS_REGION=us-east-1
    entrypoint: >
      /bin/sh -c "
      until (/usr/bin/mc config host add minio http://minio:9000 admin password) do echo '...waiting...' && sleep 1; done;
      /usr/bin/mc rm -r --force minio/warehouse;
      /usr/bin/mc mb minio/warehouse;
      /usr/bin/mc policy set public minio/warehouse;
      tail -f /dev/null
      " Specifies that the mc service depends on the MinIO service. depends_on: Specifies the Docker image for the mc service. image: Assigns a specific name (mc) to the container. container_name: Sets environment variables for configuring the MinIO client. environment: Defines the entry point command for the container, including setup steps for the MinIO client. entrypoint: /usr/bin/mc rm -r --force minio/warehouse;
      /usr/bin/mc mb minio/warehouse;
      /usr/bin/mc policy set public minio/warehouse;
      tail -f /dev/null
      " This sequence of commands essentially performs the following tasks: Removes the existing warehouse directory and its contents from the MinIO server. Creates a new bucket named warehouse. Sets the access policy of the warehouse bucket to public. This Docker Compose file orchestrates a multi-container environment with services for Spark, PostgreSQL, MinIO. It sets up dependencies, environment variables, and commands necessary for running the services together. The services work in tandem to create a development environment for data processing using Spark and Iceberg with MinIO as the object storage backend. Starting Up In a terminal window, cd into the tabular-spark-setup directory in the repository and run the following command: docker-compose up Login into MinIO at with the credentials to see that the warehouse bucket has been created. http://127.0.0.1:9001 admin:password Once all the containers are up and running, you can access your Jupyter Notebook server by navigating to http://localhost:8888 Run one of the sample notebooks and return to MinIO at to see your warehouse populated with data. http://127.0.0.1:9001 Building Your Modern Datalake This tutorial on building a modern datalake with Iceberg, Tabular, and MinIO is just the beginning. This powerful trio opens doors to a world of possibilities. With these tools, you can seamlessly integrate and analyze all your data, structured and unstructured, to uncover hidden patterns and drive data-driven decisions that fuel innovation. Leverage the efficiency and flexibility of this architecture in production to expedite your AI/ML initiatives and unlock the true potential of your machine learning models, accelerating your path to groundbreaking discoveries. Reach out to us at or on our channel if you have any questions as you build. hello@min.io Slack

Building Effective Modern Data Architectures with Iceberg, Tabular and MinIO

আইসবার্গ, ট্যাবুলার এবং মিনিও সহ কার্যকরী আধুনিক ডেটা আর্কিটেকচার তৈরি করা

Erstellen effektiver moderner Datenarchitekturen mit Iceberg, Tabular und MinIO

Creación de arquitecturas de datos modernas y eficaces con Iceberg, Tabular y MinIO

Construire des architectures de données modernes et efficaces avec Iceberg, Tabular et MinIO

आइसबर्ग, टेबुलर और मिनियो के साथ प्रभावी आधुनिक डेटा आर्किटेक्चर का निर्माण

Iceberg、Tabular、MinIO を使用した効果的な最新データ アーキテクチャの構築

Iceberg, Tabular 및 MinIO를 사용하여 효과적인 최신 데이터 아키텍처 구축

Construindo arquiteturas de dados modernas e eficazes com Iceberg, Tabular e MinIO

Создание эффективных современных архитектур данных с помощью Iceberg, Tabular и MinIO

Iceberg, Tabular ve MinIO ile Etkili Modern Veri Mimarileri Oluşturma

Xây dựng kiến trúc dữ liệu hiện đại hiệu quả với Iceberg, Tabular và MinIO

使用 Iceberg、Tabular 和 MinIO 构建有效的现代数据架构

A Guide to Troubleshooting Frequent Compile-Time Errors: C# for Beginners

How to Implement the Idempotency-Key Specification on Apache APISIX

Apache APISIX-এ আইডেমপোটেন্সি-কী স্পেসিফিকেশন কীভাবে প্রয়োগ করবেন

So implementieren Sie die Idempotency-Key-Spezifikation auf Apache APISIX

Cómo implementar la especificación de clave de idempotencia en Apache APISIX

Comment implémenter la spécification Idempotency-Key sur Apache APISIX

Apache APISIX पर Idempotency-Key विनिर्देश को कैसे लागू करें

Apache APISIX で Idempotency-Key 仕様を実装する方法

Apache APISIX에서 멱등성 키 사양을 구현하는 방법

Como implementar a especificação de chave de idempotência no Apache APISIX

Как реализовать спецификацию идемпотентного ключа в Apache APISIX

Idempotency-Key Belirtimi Apache APISIX'te Nasıl Uygulanır?

Cách triển khai Đặc tả khóa Idempotency trên Apache APISIX

如何在 Apache APISIX 上实现幂等密钥规范

The is going through a big change at the moment. Customers want different things, competition is tough, shareholders expect more, and there are lots of new technical advances. As a result, banks have to change the way they do things digitally. It's not that banks don't want to change — it's that there are big hurdles in the way. In this article, we'll take a close look at these hurdles. banking industry We'll start by looking at the top three external challenges that are impacting digital transformation in banking. Limited Ability to Partner with Fintechs The banking sector traditionally operates in a silo mentality, which makes collaboration with fintech companies a challenge. Navigating it requires understanding the ethos of collaboration, aligning goals, and creating symbiotic partnerships that can drive innovation within the banking sector. In many cases, banks find it difficult to establish these partnerships due to concerns about data and financial security. However, fostering an environment of proactive engagement with fintech companies can greatly boost efforts to overcome the challenges of digital transformation. Here’s what a Chief Digital Officer at a U.S. bank anonymously in a survey: shared “When we work with a third party, we expect to hold them to every standard we have, even when it’s onerous to us.” Many banks acknowledge partnering with FinTechs as vital to their success, despite the potential risks, such as the loss of control over new digital services. An example is a large bank in the Asia-Pacific region adopting Apple Pay due to customer demand, despite initial hesitations. That said, forming partnerships with FinTechs is not without challenges. FinTech companies, given their different IT systems and speedy operations, may not have the capacity to integrate seamlessly with the complex systems of traditional financial institutions. Barclays Ventures CEO, Ben Daley, the role of larger organizations in helping FinTech companies navigate these complexities. acknowledged to Deloitte Successful banking-FinTech partnerships are not built overnight but require a shared long-term vision and matching values. Aligning corporate goals is key to overcoming these financial services industry challenges and preventing partnerships from crumbling due to mismatched expectations. Regulations Constraining Large-Scale Digital Initiatives Existing regulatory frameworks struggle to keep up with the speed of technological innovation. Regulators are being urged to adopt more agile and principle-based regulations that can evolve over time, instead of fixed and potentially outdated structures. “PSD2 (European Commission Payment Services Directive 2) is coming into force now and it was designed seven, eight years ago—so you can imagine the changes [in the marketplace] … We need to think about activity-based regulation.” – , Executive Chairman of Grupo Santander Ana Botín Financial institutions are governed by a unique set of rules and regulations compared to start-ups and FinTechs. These regulations are often based on the type of institution rather than its specific activities. This poses a challenge as the landscape of financial services is quickly evolving, with services being repackaged and restructured in new ways. There are a few key regulations that are impacting large-scale digital transformation in banks: . Laws like Europe's GDPR and California's CCPA strictly dictate how banks handle customer data, making it harder to offer personalized digital experiences. Data Privacy Rules . These regulations mandate thorough identity checks and constant transaction scrutiny to prevent illegal activities, complicating the process of digital onboarding and account setup. AML and KYC Rules . Post-2008 financial crisis regulations force banks to maintain a high volume of cash and assets, restricting their ability to invest in novel digital platforms and operations. Capital and Liquidity Requirements . The increasing demand for cybersecurity, breach notification, and resilience calls for significant IT investments from banks, ensuring secure digital channels and protection against cyber threats. Cybersecurity Rules . Many banks rely on outdated systems and cumbersome infrastructure that hampers the creation of agile new digital platforms. Addressing these legacy systems requires a major tech overhaul. Legacy Tech Limitations High Investor Expectations Hindering Digitalization Efforts With soaring investor expectations, banks often find themselves under pressure to deliver immediate returns. This has the potential to stifle long-term digitalization efforts, that require time, patience, and a tolerance for risk. Regional banks such as PNC and US Bank plans for national digital expansion and new digital-only bank brands. Investor enthusiasm has been dampened by economic conditions and challenges in the financial services industry, particularly in balancing digital growth with risk management. Here’s what an anonymous Chief Innovation and Technology Officer at a UK bank said to : have scaled back Deloitte “Shareholder expectations really put a strain on how much money you can put into digital transformation. Everyone knows the customers’ demands, but it is about us creating the financial capacity to invest in it. We know what to improve, just how do we do it? You have to hit ROE numbers or your share price won’t perform.” High investor expectations have a substantial effect on financial services transformation. As stakeholders of the bank, investors naturally demand good short-term returns. However, these expectations can sometimes put banks under pressure, making them wary of investing in digital transformation – an initiative that usually requires time to show a return on investment. Similarly, in Q3 2022, Wells Fargo slower-than-expected progress on its digital transformation initiatives. This led to disappointment from investors and analysts, with the bank's stock price falling over 5%. Wells Fargo had to scale back some of its digital priorities to focus on stabilization and risk/compliance projects. announced Balancing these expectations while investing in long-term digital transformation often becomes a challenge. Let's now shift our focus to the top 3 internal digital banking challenges that hinder the digital transformation initiative within banks. Legacy Systems and Lack of Interoperability Around are still dependent on legacy systems. Updating and adjusting old systems to work with new digital technologies hold bank digital transformation behind. Interoperability issues often stump digital transformation plans. 60% of banks Wells Fargo in March 2023 that left millions of customers unable to access online/mobile banking or ATMs for over 24 hours. The bank blamed the outage on its reliance on outdated infrastructure that couldn't handle a power shutdown. Similarly, Bank of America that it would delay the launch of its real-time payments platform due to technical difficulties interfacing the new tech with the bank's older systems. suffered a systems outage announced in 2023 Even though many banks see the need to move away from these old systems, practical changes often pose a challenge without disturbing existing work. Moreover, not all fintech and digital banking solutions are made to work well with legacy systems, leading to pricy and dragging customizations. Banks need to work around old systems that have been in use for many years. These legacy systems often fail to offer the compatibility needed for successful digital banking transformation. Banks then, face the issue of upgrading these systems without hindering daily operations. This demands careful planning and gradual application to ensure a smooth transition. Ultimately, the obstacle doesn't rest solely on the technology itself, but how it’s integrated into the banks’ current system, procedures, and culture. Adopting digital banking transformation means developing comprehensive strategies that not only incorporate the technology but also delineate how it will impact and improve various processes within the bank. Focus on Regulatory Changes, Instead of Finance Transformation Initiatives Often, internal focus on complying with regulatory changes takes precedence over technological transformation, creating more banking industry challenges. Nearly are concerned about meeting regulatory compliance. While this focus protects banks from potential legal risks, it can divert valuable resources and attention away from much-needed digital transformation. 50% of banks This is yet another internal issue of digital transformation challenges. Although regulatory changes are crucial for controlling risk and improving transparency, they can also shift the focus away from digital transformation initiatives. Institutions need to find a balance between meeting regulatory demands and prioritizing digital initiatives by creating dedicated teams or outsourcing regulatory processes. Digital transformation in banking goes hand in hand with regulatory compliance – a factor that often diverts attention and resources away from innovation. That's because regulatory changes in the banking industry are frequent and substantial, imposing a significant burden on banks' internal structures. The constant flux of rules and norms forces many financial institutions to focus their efforts on ensuring policy adherence rather than driving digital innovation. This mindset constricts the tempo of digital transformation, ultimately slowing progress within the financial services industry. Risk-conscious Culture Preventing Innovation It's no secret embracing the risk involved with innovation could potentially lead to tremendous growth. The future of finance lies in digitalization, from fintech collaborations to the wide-scale implementation of blockchain technologies. Thus, fostering a more balanced culture that values both safety and innovation is one of the biggest challenges of digital transformation in banking. Moving forward, banks should look into integrating their risk-conscious nature with the agility and innovation of fintechs. Incorporating IT for financial services should be considered a key element in any banking transformation plan, with strategies adjusted to accommodate industry regulations without stifling progress. The path towards more customer-focused, efficient, and secure banking services may be filled with challenges, but these obstacles are vital stepping stones toward digital transformation in banking. As banks start to reevaluate their risk-averse attitudes and welcome change, the promise of a fully digital, customer-centric era in finance edges closer to reality. Summary Employing a banking digital transformation demands a well-planned, strategic methodology that considers both external and internal elements. Only have succeeded in digitally transforming at scale, which emphasizes the need to approach it strategically. Overcoming these challenges is tough, but the benefits make it worthwhile. 17% of banks “The riskiest thing we can do is just maintain the status quo.” — Bob Iger

Top 6 Challenges of Digital Transformation in Banking

This is paper is available on arxiv under CC 4.0 DEED license. Authors: (1) Leah Chrestien, Czech Technical University in Prague; (2) Tomá˘s Pevný, Czech Technical University in Prague, and Gen Digital, Inc.; (3) Stefan Edelkamp, Czech Technical University in Prague; (4) Antonín Komenda, Czech Technical University in Prague. Table of Links Abstract & Introduction Preliminaries Conditions on strictly optimally efficient heuristic Loss functions Related Work Experiments Conclusion and References Appendix 7 Conclusion The motivation of this paper stemmed from the observation that even the cost-to-goal, considered to be an optimal heuristic, fails to guarantee a strictly optimally efficient search. Since a large body of existing research optimizes this quantity, we are effectively lost with respect to what should be optimized. To fill this gap, we have stated the necessary and sufficient conditions guaranteeing the forward search to be strictly optimally efficient. These conditions show that the absolute value of the heuristic is not important, but that the ranking of states in the Open list is what controls the efficiency. Ranking can be optimized by minimizing the ranking loss functions, but its concrete implementation needs to correspond to a variant of the forward search. In case of mismatch, the resulting heuristic can perform poorly, which has been demonstrated when the heuristic optimized for BGFS search was used with A* search. The other benefit of ranking losses is that from the point of view of statistical learning theory, they solve a simpler problem than ubiquitous regression in estimating the cost-to-goal. We do not question the existence of a strictly optimally efficient heuristic. Given our experiments, we believe that if the space of heuristic functions over which the loss is optimized is sufficiently rich, the result will be sufficiently close to the optimal for the needs of the search. 7.1 Acknowledgements This work has been supported by project numbers 22-32620S and 22-30043S from Czech Science Foundation and OP VVV project CZ.02.1.01/0.0/0.0/16_019/0000765 "Research Center for Informatics". This work has also received funding from the European Union’s Horizon Europe Research and Innovation program under the grant agreement TUPLES No 101070149. References [1] Forest Agostinelli, Stephen McAleer, Alexander Shmakov, and Pierre Baldi. Solving the rubik’s cube with deep reinforcement learning and search. Nature Machine Intelligence, 1(8):356–363, 2019. [2] Thomas Anthony, Zheng Tian, and David Barber. Thinking fast and slow with deep learning and tree search. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett, editors, Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 5360–5370, 2017. [3] Shahab Jabbari Arfaee, Sandra Zilles, and Robert C Holte. Learning heuristic functions for large state spaces. Artificial Intelligence, 175(16-17):2075–2098, 2011. [4] Jeff Bezanson, Alan Edelman, Stefan Karpinski, and Viral B Shah. Julia: A fresh approach to numerical computing. SIAM review, 59(1):65–98, 2017. [5] Mohak Bhardwaj, Sanjiban Choudhury, and Sebastian Scherer. Learning heuristic search via imitation. In Conference on Robot Learning, pages 271–280. PMLR, 2017. [6] Blai Bonet and Héctor Geffner. Planning as heuristic search. Artificial Intelligence. 2001 Jun; 129 (1-2): 5-33., 2001. [7] Vladimir Cherkassky, Xuhui Shao, Filip M Mulier, and Vladimir N Vapnik. Model complexity control for regression using vc generalization bounds. IEEE transactions on Neural Networks, 10(5):1075–1089, 1999. [8] Leah Chrestien, Tomáš Pevný, Antonín Komenda, and Stefan Edelkamp. Heuristic search planning with deep neural network using imitation, attention and curriculum learning. In PRLIJCAI workshop, 2021. [9] Lukás Chrpa. Combining learning techniques for classical planning: Macro-operators and entanglements. In ICTAI, pages 79–86. IEEE Computer Society, 2010. [10] Rina Dechter and Judea Pearl. The optimality of A* revisited. In Michael R. Genesereth, editor, AAAI, pages 95–99. AAAI Press, 1983. [11] Edsger W. Dijkstra. A note on two problems in connexion with graphs. Numerische Mathematik, 1:269–271, 1959. [12] Stefan Edelkamp. Automated creation of pattern database search heuristics. In Model Checking and Artificial Intelligence, pages 35–50, 2006. [13] Stefan Edelkamp and Stefan Schrödl. Heuristic Search - Theory and Applications. Academic Press, 2012. [14] Ariel Felner, Uzi Zahavi, Robert Holte, Jonathan Schaeffer, Nathan Sturtevant, and Zhifu Zhang. Inconsistent heuristics in theory and practice. Artificial Intelligence, 175(9):1570–1603, 2011. [15] Dieqiao Feng, Carla P. Gomes, and Bart Selman. Solving hard AI planning instances using curriculum-driven deep reinforcement learning. In Christian Bessiere, editor, Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI 2020, pages 2198–2205. ijcai.org, 2020. [16] Patrick Ferber, Malte Helmert, and Jörg Hoffmann. Neural network heuristics for classical planning: A study of hyperparameter space. In ECAI, pages 2346–2353. IOS Press, 2020. [17] Santiago Franco, Álvaro Torralba, Levi H. S. Lelis, and Mike Barley. On creating complementary pattern databases. In IJCAI, pages 4302–4309, 2017. [18] Caelan Reed Garrett, Leslie Pack Kaelbling, and Tomás Lozano-Pérez. Learning to rank for synthesizing planning heuristics. page 3089–3095, 2016. [19] Edward Groshev, Aviv Tamar, Maxwell Goldstein, Siddharth Srivastava, and Pieter Abbeel. Learning generalized reactive policies using deep neural networks. 2018. [20] Arthur Guez, Théophane Weber, Ioannis Antonoglou, Karen Simonyan, Oriol Vinyals, Daan Wierstra, Rémi Munos, and David Silver. Learning to search with mctsnets. In International conference on machine learning, pages 1822–1831. PMLR, 2018. [21] Peter E Hart, Nils J Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE transactions on Systems Science and Cybernetics, 4(2):100–107, 1968. [22] Patrik Haslum, Adi Botea, Malte Helmert, Blai Bonet, and Sven Koenig. Domain-independent construction of pattern database heuristics for cost-optimal planning. In AAAI, pages 1007–1012, 2007. [23] Malte Helmert and Gabriele Röger. How good is almost perfect? In AAAI, pages 944–949. AAAI Press, 2008. [24] Robert C. Holte and Sandra Zilles. On the optimal efficiency of cost-algebraic A. In The Thirty-Third AAAI Conference on Artificial Intelligence, AAAI 2019, pages 2288–2295. AAAI Press, 2019. [25] Michael Katz, Shirin Sohrabi, Horst Samulowitz, and Silvan Sievers. Delfi: Online planner selection for cost-optimal planning. IPC-9 planner abstracts, pages 57–64, 2018. [26] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv:1412.6980, 2014. [27] Donald E Knuth and Ronald W Moore. An analysis of alpha-beta pruning. Artificial intelligence, 6(4):293–326, 1975. [28] Richard E. Korf. Iterative-deepening-A*: An optimal admissible tree search. In IJCAI, pages 1034–1036. Morgan Kaufmann, 1985. [29] Richard E Korf. Macro-operators: A weak method for learning. Artificial intelligence, 26(1):35–77, 1985. [30] Šimon Mandlík, Matej Ra ˇ cinsk ˇ y, Viliam Lis ` y, and Tomáš Pevn ` y. Jsongrinder. jl: automated ` differentiable neural architecture for embedding arbitrary json data. Journal of Machine Learning Research, 23(298):1–5, 2022. [31] Drew V. McDermott. A heuristic estimator for means-ends analysis in planning. In AIPS, pages 142–149. AAAI, 1996. [32] David Ménager, Dongkyu Choi, Mark Roberts, and David W. Aha. Learning planning operators from episodic traces. In 2018 AAAI Spring Symposia. AAAI Press, 2018. [33] Ionut Moraru, Stefan Edelkamp, Santiago Franco, and Moisés Martínez. Simplifying automated pattern selection for planning with symbolic pattern databases. In KI, pages 249–263, 2019. [34] Christian Muise. Collection of pddl files of classical problems. https://github.com/AI-Planning/classical-domains, 2023. [35] XuanLong Nguyen, Martin J Wainwright, and Michael I Jordan. On surrogate loss functions and f-divergences. The Annals of Statistics, pages 876–904, 2009. [36] Laurent Orseau, Levi Lelis, Tor Lattimore, and Théophane Weber. Single-agent policy tree search with guarantees. Advances in Neural Information Processing Systems, 31, 2018. [37] Laurent Orseau and Levi HS Lelis. Implementation of the algorithms described in "policyguided heuristic search with guarantees" by l. orseau and l. lelis, published at aaai’21. https://github.com/levilelis/h-levin, 2021. [38] Laurent Orseau and Levi HS Lelis. Policy-guided heuristic search with guarantees. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 12382–12390, 2021. [39] Arthur L Samuel. Some studies in machine learning using the game of checkers. ii—recent progress. IBM Journal of research and development, 11(6):601–617, 1967. [40] Max-Philipp B. Schrader. gym-sokoban. https://github.com/mpSchrader/gym-sokoban, 2018. [41] Jendrik Seipp, Florian Pommerening, and Malte Helmert. New optimization functions for potential heuristics. In ICAPS, 2015. [42] Jendrik Seipp and Silvan Sievers. Fast downward stone soup 2023. [43] Jendrik Seipp, Álvaro Torralba, and Jörg Hoffmann. PDDL generators. https://doi.org/10.5281/zenodo.6382173, 2022. [44] William Shen. Strips-hgn. https://github.com/williamshen-nz/STRIPS-HGN, 2021. [45] William Shen, Felipe Trevizan, and Sylvie Thiébaux. Learning domain-independent planning heuristics with hypergraph networks. In ICAPS, volume 30, pages 574–584, 2020. [46] David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, 550(7676):354–359, 2017. [47] Kristian Smith. Maze creator / solver. https://github.com/ravenkls/Maze-Generator-and Solver, 2022. [48] Gustav Sourek, Vojtech Aschenbrenner, Filip Zelezny, Steven Schockaert, and Ondrej Kuzelka. Lifted relational neural networks: Efficient learning of latent relational structures. Journal of Artificial Intelligence Research, 62:69–100, 2018. [49] Simon Ståhlberg, Blai Bonet, and Hector Geffner. Learning general optimal policies with graph neural networks: Expressive power, transparency, and limits. In Akshat Kumar, Sylvie Thiébaux, Pradeep Varakantham, and William Yeoh, editors, Proceedings of the Thirty-Second International Conference on Automated Planning and Scheduling, ICAPS 2022, Singapore (virtual), June 13-24, 2022, pages 629–637. AAAI Press, 2022. [50] Ingo Steinwart. Consistency of support vector machines and other regularized kernel classifiers. IEEE transactions on information theory, 51(1):128–142, 2005. [51] Takeshi Takahashi, He Sun, Dong Tian, and Yebin Wang. Learning heuristic functions for mobile robot path planning using deep neural networks. In ICAPS, volume 29, pages 764–772, 2019. [52] Alvaro Torralba, Vidal Alcázar, Daniel Borrajo, Peter Kissmann, and Stefan Edelkamp. SymBA*: A symbolic bidirectional A* planner. In International Planning Competition, pages 105–108, 2014. [53] Sam Toyer, Sylvie Thiébaux, Felipe Trevizan, and Lexing Xie. Asnets: Deep learning for generalised planning. Journal of Artificial Intelligence Research, 68:1–68, 2020. [54] Vladimir Vapnik. Principles of risk minimization for learning theory. Advances in neural information processing systems, 4, 1991. [55] Marin Vlastelica, Michal Rolinek, and Georg Martius. Neuro-algorithmic policies enable fast combinatorial generalization. In International Conference on Machine Learning, pages 10575– 10585. PMLR, 2021. [56] Xuemei Wang. Learning planning operators by observation and practice. In Kristian J. Hammond, editor, AIPS, pages 335–340. AAAI, 1994. [57] Christopher Wilt and Wheeler Ruml. When does weighted A* fail? In International Symposium on Combinatorial Search, volume 3, 2012. [58] Christopher Makoto Wilt and Wheeler Ruml. Effective heuristics for suboptimal best-first search. J. Artif. Intell. Res., 57:273–306, 2016. [59] Yuehua Xu, Alan Fern, and Sungwook Yoon. Learning linear ranking functions for beam search with application to planning. Journal of Machine Learning Research, 10(7), 2009. [60] Ryo Yonetani, Tatsunori Taniai, Mohammadamin Barekatain, Mai Nishimura, and Asako Kanezaki. Path planning using neural A* search. In International Conference on Machine Learning, pages 12029–12039. PMLR, 2021. [61] Ziqi Zhang and Florian Geißer. Extending graph neural networks for generalized stochastic planning. In Bridging the Gap Between AI Planning and Reinforcement Learning. 2021. [62] Tan Zhi-Xuan. PDDL. jl: An Extensible Interpreter and Compiler Interface for Fast and Flexible AI Planning. PhD thesis, Massachusetts Institute of Technology, 2022. [63] Juntang Zhuang, Tommy Tang, Yifan Ding, Sekhar C Tatikonda, Nicha Dvornek, Xenophon Papademetris, and James Duncan. Adabelief optimizer: Adapting stepsizes by the belief in observed gradients. Advances in neural information processing systems, 33:18795–18806, 2020.

Optimize Planning Heuristics to Rank, not to Estimate Cost-to-Goal: Conclusion and References

Mission-critical applications require . The goal of high availability is to provide users with consistent access to services or resources, minimizing the chances of interruption. is a specific mechanism used to achieve high availability. It involves automatically detecting the failure of a system component (like a server, network, or database) and immediately switching operations to a standby component without human intervention. This increases resiliency. high availability Automatic failover MariaDB MaxScale is a database proxy that includes features for high availability. In this article, I'll show you how you can try it out with an application implemented in Java and Svelte. online store simulator Architecture The following diagram shows the architecture of the demo application: A web application developed with JavaScript and the Svelte framework makes HTTP requests to a Java backend. The backend answers with that the frontend uses to update the user interface on the browser. server-sent events The backend is implemented with and connects to a database cluster using (reactive). The backend logic is, in short, a simulation of reads and writes to an online store database. The simulation is parameterized, and the user can adjust: Spring Boot MariaDB R2DBC How many reads to the database per minute. Product visits per minute: How many writes to the database per minute. Orders per minute: Write amplification. Products per order: How many seconds until a request to the database is considered failed. Timeout in milliseconds: The database cluster is front-ended by a called MaxScale. This proxy makes the cluster look like a single logical database to the Java backend. MaxScale also performs read/write splitting (sending writes to the primary MariaDB server and reads to replicas), as well as load-balancing of reads among replica servers using a configurable algorithm. Data is automatically replicated from the primary to the replica database servers. database proxy Building the Docker images from source I have prepared custom Docker images for every component in the simulator. You can either build the images from the source (optional) or use the already built and published images from Docker Hub. If you decide to build the images yourself, you can find the source code on GitHub: Custom images for easy deployment of replicated MariaDB topologies with MaxScale. These images are suitable only for demo applications. Use the for production deployments. : MariaDB deployments DO NOT USE THESE IN PRODUCTION! official MariaDB Docker images The backend application that connects to the database cluster. : Backend application The frontend application that makes simulation configuration requests to the backend and receives events to show the simulation result. : Frontend application Each repository has Dockerfiles that you can use to build your own Docker images. For example, to build the backend application image, run: docker build --tag alejandrodu/online-store-simulator-java-backend . Running the simulation All the services can be started using the following Docker Compose file ( ): docker-compose.yml version: "3.9"
services:
  server-1:
    container_name: server-1
    image: alejandrodu/mariadb
    ports:
      - "3306:3306"
    environment:
      - MARIADB_CREATE_DATABASE=demo
      - MARIADB_CREATE_USER=user:Password123!
      - MARIADB_CREATE_REPLICATION_USER=replication_user:ReplicationPassword123!
      - MARIADB_CREATE_MAXSCALE_USER=maxscale_user:MaxScalePassword123!

  server-2:
    container_name: server-2
    image: alejandrodu/mariadb
    ports:
      - "3307:3306"
    environment:
      - MARIADB_REPLICATE_FROM=replication_user:ReplicationPassword123!@server-1:3306

  server-3:
    container_name: server-3
    image: alejandrodu/mariadb
    ports:
      - "3308:3306"
    environment:
      - MARIADB_REPLICATE_FROM=replication_user:ReplicationPassword123!@server-1:3306

  maxscale:
    container_name: maxscale
    image: alejandrodu/mariadb-maxscale
    command: --admin_host 0.0.0.0 --admin_secure_gui false
    ports:
      - "4000:4000"
      - "8989:8989"
      - "27017:27017"
    environment:
      - MAXSCALE_USER=maxscale_user:MaxScalePassword123!
      - MARIADB_HOST_1=server-1 3306
      - MARIADB_HOST_2=server-2 3306
      - MARIADB_HOST_3=server-3 3306
    healthcheck:
      test: ["CMD", "maxctrl", "list", "servers"]
      interval: 5s
      timeout: 10s
      retries: 5

  java-backend:
    container_name: java-backend
    image: alejandrodu/online-store-simulator-java-backend
    ports:
      - "8080:8080"
    environment:
    - spring.r2dbc.url=r2dbc:mariadb://maxscale:4000/demo
    - spring.r2dbc.username=user
    - spring.r2dbc.password=Password123!
    - spring.liquibase.url=jdbc:mariadb://maxscale:4000/demo
    - spring.liquibase.user=user
    - spring.liquibase.password=Password123!
    depends_on:
      maxscale:
        condition: service_healthy

  svelte-frontend:
    container_name: svelte-fronted
    image: alejandrodu/online-store-simulator-svelte-frontend
    ports:
      - "5173:80"
    environment:
      - BACKEND_URL=http://java-backend:8080 Move to the directory in which the Docker Compose file is, and start the services in detached mode as follows: docker compose up -d Configuring MaxScale Before you start the simulation, configure MaxScale for transaction replay. Also, timeouts should be adjusted to make the simulation more interesting. Navigate to and log into the UI using: http://localhost:8989/ Username: admin Password: mariadb You'll see a dashboard with the MariaDB cluster state. There's a primary server ( ), and two replicas ( and ). Replication is already configured from (primary) to and (replicas). All servers should be up and running. server-1 server-2 server-3 server-1 server-2 server-3 Click on and then on the icon to enable parameter editing. Set the following parameters: mdb_monitor pencil ( ): This enables automatic failover. When a MariaDB server is down, MaxScale selects a replica server and reconfigures it as the new primary so that writes can continue to happen. auto_failover true ( ): This enables the automatic rejoin of recovered servers. When a failed server is up again, MaxScale detects it and configures it as an available replica server. auto_rejoin true ( ): Sets the number of monitor (a component in MaxScale that checks server status) iterations required for a server to be down in order to activate the failover process. We set a value of to make sure the failover starts immediately after failure. failcount 1 1 ( ): Connection timeout for monitor connections. We set a low value (one second) to quickly activate failover for this demo. backend_connect_timeout 1000 ( ): Read timeout for monitor connections. backend_read_timeout 1000 ( ): Write timeout for monitor connections. backend_write_timeout 1000 ( ): Primary failure timeout. master_failure_timeout 1000 ( ): How often the servers are monitored. monitor_interval 1000 ⚠️ These values are appropriate for this demo but very likely not the best for production environments! WARNING: Once the parameters are set, click on and . Done Editing Confirm You also need to enable transaction replay, which automatically re-executes failed in-flight transactions on servers that went down just after a SQL statement was routed. This is a useful feature for software developers since it prevents the need for coding failure cases and transaction retry. On the main menu, click on and then on any of the links in the list of servers. Edit the parameters as follows: Dashboard query_router_service ( ): Activates automatic retry of failed transactions. transaction_replay true ( ): Same as the previous when a deadlock occurs. transaction_replay_retry_on_deadlock true ( ): Same as the previous when a checksum mismatch occurs. transaction_replay_retry_on_mismatch true Once the parameters are set, click on and . Done Editing Confirm Starting the simulation With everything configured, you can start the simulation. Navigate to and configure the following parameters (names are, I hope, self-explanatory): http://localhost:5173/ Product visits per minute: 6000 Orders per minute: 60 Timeout in milliseconds: 8000 But before you start the simulation, you need to create the products for the online store. Click on | . Leave the default values and click on . You should see the UI updating as products are created in the database. Data Create products... Create Now, you can finally click on and see the simulation in action. Start Simulating a server failure At this point, the primary server is handling writes (orders). What happens if you stop that server? In the command line run: docker stop server-1 Depending on multiple factors, you might get some "disappointed visitors" or even a few "missed opportunities" in the simulator. Or maybe you don't get any at all! Product visits (reads) and orders (writes) continue to happen, thanks to MaxScale. Without automatic failover, you have to reconfigure everything manually, which results in more offline time, many disappointed visitors, and missed opportunities! Start the failed server: docker start server-1 Go to the MaxScale ( ) and check that is now a functioning replica. Dashboard http://localhost:8989/ server-1 You can perform a manual to make the primary server again. Click on and then mouse hover over the section. Click on the icon and select . Click and check again in the that the new primary server is . switchover server-1 mdb_monitor MASTER pencil server-1 Swap Dashboard server-1 Conclusion Automatic failover is only one of the components in highly available systems. You can use a database proxy like MaxScale to set up automatic failover, as well as other components such as load-balancing, query routing, transaction retry, topology isolation, and more. Check out the documentation at . https://mariadb.com/kb/en/maxscale/ Also published . here

Achieving High Availability with MariaDB MaxScale

In the realm of blockchain development, accessing data in a secure and efficient manner often poses a hurdle. Developers need to have seamless and dependable access to on-chain data to work efficiently But this sometimes implies navigating through different data sources, which makes extracting raw data an expensive and complex process. Stateless Solutions founder provided insights into the challenges faced by developers in accessing on-chain data. Jessica Daugherty Thanks for finding the time to join us, Jessica. Can you tell us about your background? Thanks for having me. I have navigated a dynamic path through the tech landscape, starting as a startup consultant focused on marketing, strategy, and organizational design. In 2019, I became a late-stage co-founder of a SaaS startup in the e-commerce and data space, leading to a successful exit in 2020. I then helped the acquiring company through a Series B funding and cash acquisition before diving into the blockchain world with Pocket Network. There, as head of product, I deepened my expertise in blockchain technologies and RPC. Drawing on these experiences, I founded Stateless with my co-founder, Joe Habel, in June 2023 to solve critical challenges in blockchain data access and integrity. Can you elaborate on the specific challenges developers face when trying to access blockchain data? Accessing reliable blockchain data poses significant challenges for developers, primarily due to the decentralized nature of blockchain networks. One of the key difficulties is ensuring the accuracy and trustworthiness of the data retrieved. With numerous nodes operating independently, achieving consensus on the validity of data is crucial. This raises the question of having a reliable source of truth in a decentralized system where discrepancies can arise. Another challenge is timeliness and the avoidance of stale data. In a distributed network, nodes might present outdated information due to synchronization issues, leading to the risk of making decisions based on incorrect data. Developers must navigate these hurdles to ensure the data they utilize is both current and accurate, which is vital for the integrity of decentralized applications. Do you consider the lack of standardization to be a major challenge in accessing blockchain data? The issue with standardization in Web3 is not the absence of it but rather its rigidity and misalignment with current operational needs. Originally designed for a trusted client/server architecture, these standards struggle to adapt to the untrusted, decentralized environments of blockchains. This mismatch presents developers with a dual challenge: First, the way data is accessed is too rigid, failing to accommodate the dynamic nature of decentralized technologies. This can hinder developers' ability to efficiently retrieve and utilize trustworthy data. Second, operational standards are overly flexible, leading to inconsistencies in network state management, node coordination, and understanding of deployment configurations. This variability complicates maintaining system integrity and optimizing resources. Balancing these standards is crucial for fostering innovation while ensuring reliability and security in blockchain data access. Can you tell us about the founding of Stateless Solutions? Stateless was born from the firsthand experiences and challenges faced by its founders while working at Pocket Network. As active participants in protocol design and development, as well as serving user-facing needs, we confronted these limitations and bottlenecks that pose significant threats to the security and adoption of Web3. Recognizing the urgency of these issues and the lack of viable solutions within the existing framework, we decided to establish Stateless. Our goal was to provide a mitigation layer and next-generation infrastructure that addresses these challenges head-on, paving the way for the scalable and secure development of decentralized applications and technologies. What are the challenges that you seek to address and how do you tackle them? Our core mission at Stateless is to significantly reduce trust dependencies and the risks and inefficiencies they present, enabling the full benefits of trust-minimized decentralization. This means enabling global computation at scale without the performance tradeoffs or security vulnerabilities that current services present, from data loss to asset compromises. One key challenge we're tackling is abstracting the complexity of streaming verified data from decentralized networks in a way that is accessible and straightforward for consumers. Our solution to this challenge begins at the RPC layer, where we have successfully launched a service designed to simplify access to blockchain data while maintaining high standards of security and reliability. This RPC service represents our initial beachhead, from which we aim to build additional verifiable services. These services will not only support third parties in areas such as indexing and analytics but will also embrace more advanced cryptographic and trustless techniques, including zero-knowledge (ZK) proofs, and extend beyond blockchain data to areas like deterministic AI. Utilizing our composable architecture will be pivotal for building a foundation that supports a wide array of applications and services in a trust-minimized environment with interoperability across all layers of protocols. Our collaboration with Laconic marks a significant first step towards realizing this expansive vision. By merging our robust verification layer with Laconic's advanced Merkle proofs, we are crafting a powerful solution that exemplifies the potential of our model. This integration aims to deliver data directly from within the node back to the consumer via an efficient light client, marking a groundbreaking advancement in our efforts to support different data types and cryptographic structures. Our innovative partnership paves the way for a new era of blockchain accessibility and security, reducing complexity and costs for users and developers alike. We are setting the foundation for a future where secure, decentralized applications are the norm, empowered by our trust-minimized ecosystem. Our collaboration with Laconic is just the beginning, as we continue to push the boundaries of what's possible in decentralized computation. What in your opinion is the biggest challenge in processing blockchain data? In processing blockchain data, the most significant challenge lies in ensuring the accuracy and integrity of historical data. Specifically, the inability to easily re-execute previous requests or queries to verify that the results are correct—even when dealing with the same party—poses a substantial hurdle. This challenge is twofold: First, there's the issue of verifying the correctness of data, which is crucial for maintaining the reliability and security of blockchain applications. Without the ability to re-execute or verify past transactions or queries, it becomes difficult to ensure data integrity. Second, there's the problem of auditing historical data. The current infrastructure does not support batch requests and integrity checks effectively, making it challenging to identify and rectify issues with the underlying data retroactively. How can blockchain developers overcome this challenge? At Stateless we believe a fundamental paradigm shift is needed to overcome these challenges. As emphasized by , the reliance on trusted RPC providers is a flawed model that needs reevaluation. Vitalik Buterin in December 2023 Developers should explore and adopt trust-minimized solutions, such as Stateless, that facilitate direct and secure access to blockchain data. Additionally, running personal nodes, despite its operational complexities, offers another layer of verification and control over data integrity. Can you describe your experience so far in this space? Working in the blockchain and Web3 space is both challenging and exhilarating. It offers endless opportunities for innovation, learning, and problem-solving. The pace of development and the potential for groundbreaking advancements make it, in my opinion, the most exciting field (yes, even more exciting than AI!). Despite the complexities and hurdles, the satisfaction of contributing to the evolution of decentralized technology is unparalleled. This dynamic and ever-evolving landscape keeps me motivated and eager to explore new horizons. Any parting words? Verify, don’t trust!

Demystifying Blockchain Data Access: Interview With Stateless Solutions Founder Jessica Daugherty

Stories, News & Mentions about MVPMATCH

Mvpmatch’s Stories on HackerNoon