The Shelf brings big-data and machine-learning to the world of influencer/word-of-mouth marketing

The Shelf

theshelf

Real business problems require serious custom solutions at a reasonable price. Businesses that invest in pertinent technology to make informed decisions and automate manual tasks always witness notable bottom-line growth. The market has many resources to provide the required solutions. However, each solution is designed keeping in mind the needs of a whole industry or collection of industries. The products have many features and capabilities. On the other hand, the needs of a business are usually very specific. They usually make use of a limited number of features that a provider offers. Too many features make it difficult to deliver efficiency. To quote an example, a large chain of departmental stores would require an extensive CRM software to keep track of the usage patterns its many customers. On the other hand, CRM needs a private cash-and-carry store would be limited to, say, managing contact details and email integration. In this case, buying the same software and paying the same price would be unfair and inefficient. Clearly, the best solution would be to pay for what you actually use, instead of everything that is offered by a software. Custom Solutions Some software designers offer to curate solutions to cater to the specific needs of the client. They understand the business problem at hand, restructure their product, and deliver a solution that the clients can actually use. When business interests are aligned in this manner, end users’ delight becomes possible. With the spread of cloud computing, several providers offer intuitive ready-made cloud-based pay-as-you-go-solutions. Businesses must ensure that they know how such solutions can be customized to suit their own needs. Simple solutions that actually solve problems are of much more value than fancy softwares that do not. They will meet customers’ expectations and they will have the functionalities that are actually useful. This will make work easy and infuse agility into an organization. At the end of the day, that is exactly what users want. Data Training In the artificial intelligence space, data is king. How good a solution is will depend on the kind of data it has been trained on. Its ‘ability to predict’ is directly influenced by ‘what it knows’. The nature of data which a solution is trained on determines how good the solution will be. Obviously, ready-made solutions are made using data which is available in the public domain. Such data is generic. Results from a solution which is trained on generic data are not useful for the specific needs of a business. However, there are quite a few AI based developers who train the algorithm on the client’s data. For example, if Uber would like to use AI to get insights into tweets by disgruntled users, they would want a solution that can understand feedback related phrases such as ‘app is scratchy’, ‘waited for 30 minutes’, ‘driver didn’t accept cash’ etc. A custom solutions provider will actually train the algorithms on feedback-related data specifically for the client. Support A large part of investing in a software is the process of integration. Support from the vendor is required at each and every step of the way in order to make the integration process a success. A good support service goes a long way in bringing about happy customers and, thereby, an optimistic bottom line. With custom solutions, the best part is that good customer support service is the objective of the vendor right from the beginning. The entire solution is meant to fit into the client’s needs. This makes it inevitable for the vendor to put in place a dedicated team that will ensure a seamless integration. In this case, clients will find it easy to solve their problems because help will be readily available. This feature is very particular to custom software providers and is a major boon to their clients. How ParallelDots is leading in the custom solutions space ParallelDots’ Enterprise Services team has successfully deployed AI projects across domains and industries. As leaders in the deep learning and NLP sphere, we are bringing AI right to your fingertips. Various features of ParallelDots’ Enterprise Services: EXTRAORDINARY TEAM OF DATA SCIENTISTS The ParallelDots data science team is a collection of the best lot of techies, from across the country. There is a dedicated team for data tagging as well. The purpose is to build proprietary data that can be readily used whenever a client comes aboard. DEDICATED CUSTOMER SUCCESS MANAGER The intent is to deliver solutions that actually help clients make their work easy. For this purpose, a dedicated team of customer success managers makes sure that right from integration to pilot to deployment, all processes are smoothly handled. COST-EFFECTIVE SOLUTIONS The positioning of the products is cost-effective and the pricing policy is ‘pay-as-you-go’. Users are billed only for the excess usage beyond free tier quota. The focus is on making products affordable and best-of-class. PRIMACY TO DATA PRIVACY Privacy of client data is given top priority. All of the data used to train the algorithms is proprietary and licensed. More data is constantly being generated and tagged for use by different clients. Trusted by the best Major FMCG giants including ITC are currently using , the market research product of , Inc, to automate manual tasks and get valuable insights from their data. Karna.AI ParallelDots When it comes to finding the right custom solutions, custom software development has plenty of benefits worth considering. Is it the right choice for your company? Only you will be able to decide. But, if any of the reasons we have laid out clicked in your brain as something you need, we believe your business has a great opportunity to improve and continue moving forward through the utilization of custom software development. If you want to explore or know about the custom solutions then please fill the form or simply mail at contact@paralleldots.com. We will be happy to help you! here Read more about the recently launched Google Sheets add-on for all your text analytics needs . here We hope you liked the article. Please for a free ParallelDots account to start your AI journey. You can also check demo’s of PrallelDots AI APIs . Sign Up here This article has been published after the permission of the author. You can read the original article . here

Why custom solutions work better than off-the-shelf solutions for businesses

This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license.
Authors:
(1) Yejin Bang, Centre for Artificial Intelligence Research (CAiRE), The Hong Kong University of Science and Technology;
(2) Nayeon Lee, Centre for Artificial Intelligence Research (CAiRE), The Hong Kong University of Science and Technology;
(3) Pascale Fung, Centre for Artificial Intelligence Research (CAiRE), The Hong Kong University of Science and Technology. Table of Links Abstract and Intro
Related Work
Approach
Experiments
Conclusion
Limitations, Ethics Statement and References
A. Experimental Details
B. Generation Results 4. Experiments 4.1. Setup 4.2. Models Baselines We compare with off-the-shelf multidocument summarization (MDS) models trained on Multi-news dataset (Fabbri et al., 2019) (BARTMULTI (Lewis et al., 2019) and PEGASUSMULTI (Zhang et al., 2019a)) as baselines. Those models have achieved high performance in MDS, which can also be applied in summarizing polarized articles. However, these models do not have any learning about framing bias removal or neutral writing. We also compare with the state-of-theart models (BARTNEUSFT and BARTNEUSFT-T) (Lee et al., 2022) that are fine-tuned with ALLSIDES dataset. BARTNEUSFT is fine-tuned only with articles meanwhile BARTNEUSFT-T additionally leverages titles of each article. We additionally report PEGASUSNEUSFT. Simply fine-tuning may not be effective enough to learn about framing bias. Thus, we will demonstrate how the polarity minimization loss can effectively mitigate framing bias compared to baseline and SOTA models. 4.3. Results Effective learning with extreme polarities We investigate that polarity minimization between extreme ends (left, right) is more effective than the mixture with a center media outlet. This is because left and right-wing ideologies are the opposite ends that can train models more effectively about extreme ends than center media outlets although center media is not completely free of bias. Qualitative analysis results align with the quantitative measures. For instance, as illustrated in Table 2, the polarity minimized models LR-INFO and LRC-AROUSAL both could summarize with the essential information out of polarized input articles. Especially LR-INFO, the lowest biased model, it could even use a more neutral choice of word (e.g., “protests” instead of “riots” same to target Y). 4.4. Analysis This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Yejin Bang, Centre for Artificial Intelligence Research (CAiRE), The Hong Kong University of Science and Technology; (2) Nayeon Lee, Centre for Artificial Intelligence Research (CAiRE), The Hong Kong University of Science and Technology; (3) Pascale Fung, Centre for Artificial Intelligence Research (CAiRE), The Hong Kong University of Science and Technology. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: Authors: (1) Yejin Bang, Centre for Artificial Intelligence Research (CAiRE), The Hong Kong University of Science and Technology; (2) Nayeon Lee, Centre for Artificial Intelligence Research (CAiRE), The Hong Kong University of Science and Technology; (3) Pascale Fung, Centre for Artificial Intelligence Research (CAiRE), The Hong Kong University of Science and Technology. Table of Links Abstract and Intro Related Work Approach Experiments Conclusion Limitations, Ethics Statement and References A. Experimental Details B. Generation Results Abstract and Intro Abstract and Intro Related Work Related Work Approach Approach Experiments Experiments Conclusion Conclusion Limitations, Ethics Statement and References Limitations, Ethics Statement and References A. Experimental Details A. Experimental Details B. Generation Results B. Generation Results 4. Experiments 4.1. Setup 4.2. Models Baselines We compare with off-the-shelf multidocument summarization (MDS) models trained on Multi-news dataset (Fabbri et al., 2019) (BARTMULTI (Lewis et al., 2019) and PEGASUSMULTI (Zhang et al., 2019a)) as baselines. Those models have achieved high performance in MDS, which can also be applied in summarizing polarized articles. However, these models do not have any learning about framing bias removal or neutral writing. We also compare with the state-of-theart models (BARTNEUSFT and BARTNEUSFT-T) (Lee et al., 2022) that are fine-tuned with ALLSIDES dataset. BARTNEUSFT is fine-tuned only with articles meanwhile BARTNEUSFT-T additionally leverages titles of each article. We additionally report PEGASUSNEUSFT. Simply fine-tuning may not be effective enough to learn about framing bias. Thus, we will demonstrate how the polarity minimization loss can effectively mitigate framing bias compared to baseline and SOTA models. Baselines 4.3. Results Effective learning with extreme polarities We investigate that polarity minimization between extreme ends (left, right) is more effective than the mixture with a center media outlet. This is because left and right-wing ideologies are the opposite ends that can train models more effectively about extreme ends than center media outlets although center media is not completely free of bias. Qualitative analysis results align with the quantitative measures. For instance, as illustrated in Table 2, the polarity minimized models LR-INFO and LRC-AROUSAL both could summarize with the essential information out of polarized input articles. Especially LR-INFO, the lowest biased model, it could even use a more neutral choice of word (e.g., “protests” instead of “riots” same to target Y). Effective learning with extreme polarities 4.4. Analysis

Mitigating Framing Bias with Polarity Minimization Loss: Experiments

পোলারিটি মিনিমাইজেশন লস সহ ফ্রেমিং বায়াস প্রশমিত করা: পরীক্ষা-নিরীক্ষা

Abschwächung des Framing Bias durch Polaritätsminimierungsverlust: Experimente

Mitigar el sesgo de encuadre con pérdida de minimización de polaridad: experimentos

Atténuer le biais de cadrage grâce à la perte de minimisation de polarité : expériences

ध्रुवीयता न्यूनीकरण हानि के साथ फ़्रेमिंग पूर्वाग्रह को कम करना: प्रयोग

極性最小化損失によるフレーミングバイアスの緩和: 実験

극성 최소화 손실로 프레이밍 바이어스 완화: 실험

Mitigando o viés de enquadramento com perda de minimização de polaridade: experimentos

Уменьшение смещения кадра с помощью минимизации потерь полярности: эксперименты

Polarite Minimizasyon Kaybı ile Çerçeveleme Önyargısını Azaltma: Deneyler

Giảm thiểu sai lệch khung bằng giảm thiểu tổn thất phân cực: Thử nghiệm

通过极性最小化损失减轻框架偏差：实验

When we first came up with our “big idea” back in 2017, my partner Mike Wagner and I didn’t know how much impact it would have. The idea was simple enough: We wanted to develop a virtual appliance that could discover, provision, and maintain servers from any manufacturer. By 2020, that vision became a reality with the launch of our flagship platform, which we built entirely on open-source technologies. That strategy was working well from the start. Well enough, I should say. We knew we could improve our metal-as-a-service software appliance if we could just find a way to boost performance, ensure higher system uptime, and streamline the architecture. The only way to do that, we realized, was to replace the Linux operating system and KVM virtual machine solution with something that would give us those benefits. What we did know was that we wanted to continue with our strategy of utilizing open source—and ideally tied to a very permissive license and open governance model. That would give us more flexibility as our roadmap unfolded and enable extensive use and modification without stringent (or murky) conditions. Taking the plunge After our due diligence, we transitioned to the Unix-like open source FreeBSD operating system, along with the BSD hypervisor Bhyve. We liked that the FreeBSD project had off-the-shelf capabilities within a single distribution, and had plenty of mission-critical enterprise use cases (Netflix and Sony being two of the most well-known). Together with existing technologies like the Redfish standard, the new and improved platform we’d built (called Mojo) was ready to really take off. With FreeBSD and bhyve fully in place, we were now able to add edge data center provisioning to the platform, which helps discover, configure, and provision bare metal servers and storage within a data center and at the edge. It works really well with another new technology (PhotoISP, which I’ll discuss in a moment) to manage geographically distributed bare metal assets. These open-source changes to our infrastructure proved to be the silver bullet we were looking for. More organizations began to contact us, and we started landing some really interesting use cases, including Major League Baseball. Before the 2021 season, MLB was looking for a way to move from VMware to a containerized environment across 30 data centers. In a nutshell, they needed a way to quickly process 15 different input sources to track all the player and team statistics generated during a game. By more quickly provisioning servers, that data could be quickly gathered at the edge and uploaded into Google Cloud. The Mojo platform, backed by an evolved open-source strategy, helped streamline MLB’s deployment pipelines (cutting deployment time in half). It also simplified the provisioning and management of MLB’s bare metal edge sites and made it really easy to provision Anthos, BIOS, firmware, OS patches, and upgrades. They also saw big performance gains for their GPU-enabled apps and more optimized CI/CD pipelines. Expanding a vision backed by open source Albert Einstein once said that necessity is the mother of all invention. My family lives in the Blue Ridge Mountains, and when COVID hit, word spread quickly that several teachers and students were unable to stream their classes because they had insufficient rural broadband access. Mike and I brainstormed about how we could help, and we immediately recognized how Mojo, paired with a reliable and secure CPE, could solve the problem. That led development of the Photon ISP platform, which delivers routing, firewall, and QoS through the open-source FreeBSD operating system. It also monitors IDS/IPS for wireless and fixed fiber networks and ensures uninterrupted 4G/5G and Starlink failover as a hedge to wireless ISP services. Today, the Photon ISP platform has 50 rural customers, and that number continues to grow. And its uptime is unparalleled; we have FreeBSD CPEs that have been running for more than a year without issue. Speeds have been very consistent, streaming content has been reliable, and there has been almost zero latency. We’re still pushing the limits of what we can do with our optimized open-source infrastructure. Next, we plan to finish moving the remainder of the stack to FreeBSD once a viable OCI-compliant stack becomes available. In the meantime, we still use Linux within Bhyve virtual machines for a few of our services. Simply put, the strategic shift to open-source FreeBSD and Bhyve for our platforms has catalyzed innovation and customer growth. The success with MLB—cutting deployment times and improving performance—is a great example of how the right open-source pieces can have an outsized impact while being far more flexible (not to mention cost-efficient!) than proprietary alternatives. When we first came up with our “big idea” back in 2017, my partner Mike Wagner and I didn’t know how much impact it would have. The idea was simple enough: We wanted to develop a virtual appliance that could discover, provision, and maintain servers from any manufacturer. By 2020, that vision became a reality with the launch of our flagship platform, which we built entirely on open-source technologies. That strategy was working well from the start. Well enough , I should say. enough We knew we could improve our metal-as-a-service software appliance if we could just find a way to boost performance, ensure higher system uptime, and streamline the architecture. The only way to do that, we realized, was to replace the Linux operating system and KVM virtual machine solution with something that would give us those benefits. What we did know was that we wanted to continue with our strategy of utilizing open source—and ideally tied to a very permissive license and open governance model. That would give us more flexibility as our roadmap unfolded and enable extensive use and modification without stringent (or murky) conditions. Taking the plunge After our due diligence, we transitioned to the Unix-like open source FreeBSD operating system, along with the BSD hypervisor Bhyve . We liked that the FreeBSD project had off-the-shelf capabilities within a single distribution, and had plenty of mission-critical enterprise use cases (Netflix and Sony being two of the most well-known). Together with existing technologies like the Redfish standard, the new and improved platform we’d built (called Mojo) was ready to really take off. open source FreeBSD Bhyve Redfish With FreeBSD and bhyve fully in place, we were now able to add edge data center provisioning to the platform, which helps discover, configure, and provision bare metal servers and storage within a data center and at the edge. It works really well with another new technology (PhotoISP, which I’ll discuss in a moment) to manage geographically distributed bare metal assets. These open-source changes to our infrastructure proved to be the silver bullet we were looking for. More organizations began to contact us, and we started landing some really interesting use cases, including Major League Baseball. Before the 2021 season, MLB was looking for a way to move from VMware to a containerized environment across 30 data centers. In a nutshell, they needed a way to quickly process 15 different input sources to track all the player and team statistics generated during a game. By more quickly provisioning servers, that data could be quickly gathered at the edge and uploaded into Google Cloud. The Mojo platform, backed by an evolved open-source strategy, helped streamline MLB’s deployment pipelines (cutting deployment time in half). It also simplified the provisioning and management of MLB’s bare metal edge sites and made it really easy to provision Anthos, BIOS, firmware, OS patches, and upgrades. They also saw big performance gains for their GPU-enabled apps and more optimized CI/CD pipelines. Expanding a vision backed by open source Albert Einstein once said that necessity is the mother of all invention. My family lives in the Blue Ridge Mountains, and when COVID hit, word spread quickly that several teachers and students were unable to stream their classes because they had insufficient rural broadband access. Mike and I brainstormed about how we could help, and we immediately recognized how Mojo, paired with a reliable and secure CPE, could solve the problem. That led development of the Photon ISP platform, which delivers routing, firewall, and QoS through the open-source FreeBSD operating system. It also monitors IDS/IPS for wireless and fixed fiber networks and ensures uninterrupted 4G/5G and Starlink failover as a hedge to wireless ISP services. Today, the Photon ISP platform has 50 rural customers, and that number continues to grow. And its uptime is unparalleled; we have FreeBSD CPEs that have been running for more than a year without issue. Speeds have been very consistent, streaming content has been reliable, and there has been almost zero latency. We’re still pushing the limits of what we can do with our optimized open-source infrastructure. Next, we plan to finish moving the remainder of the stack to FreeBSD once a viable OCI-compliant stack becomes available. In the meantime, we still use Linux within Bhyve virtual machines for a few of our services. Simply put, the strategic shift to open-source FreeBSD and Bhyve for our platforms has catalyzed innovation and customer growth. The success with MLB—cutting deployment times and improving performance—is a great example of how the right open-source pieces can have an outsized impact while being far more flexible (not to mention cost-efficient!) than proprietary alternatives.

How Our Open Source OS Pivot Accelerated Product Innovation

Exploring Web3's Role in Advertising and Viewer Engagement

Human Augmentation in Meme Analysis: Learnings and Challenges

Authors:
(1) Vasiliki Kougia, University of Vienna, Faculty of computer science, Vienna, Austria & UniVie Doctoral School Computer Science, Vienna, Austria;
(2) Simon Fetze, University of Vienna, Faculty of computer science, Vienna, Austria;
(3) Thomas Kirchmair, University of Vienna, Faculty of computer science, Vienna, Austria;
(4) Erion Çano, University of Vienna, Faculty of computer science, Vienna, Austria;
(5) Sina Moayed Baharlou, Boston University, Department of Electrical and Computer Engineering, Boston, MA, USA;
(6) Sahand Sharifzadeh, Ludwig Maximilians University of Munich, Faculty of Computer Science, Munich, Germany;
(7) Benjamin Roth, University of Vienna, Faculty of computer science, Vienna, Austria. Table of Links Abstract and Introduction Related Work MemeGraphs Benchmarking Analysis Conclusion, Acknowledgments, and References 3 MemeGraphs In this section, we describe our method for automatic augmentation of a memes dataset, an approach we call MemeGraphs. Our proposed method consists of three steps: 1. Scene graph construction, 2. Knowledge linking to detected entities and 3. The construction of the final MemeGraphs input. The first two steps are performed automatically by using off-the-shelf models. Hence, the result of MemeGraphs are knowledge graphs representing the meme as a whole, which can be used to classify them, e.g., for hate detection. We build on the MultiOFF dataset, which contains memes extracted from social media during the 2016 U.S. Presidential Election [28]. In what follows, we present the three individual steps of MemeGraphs in detail and how the final result is constructed. 3.1 Scene graphs For 67 memes, Schemata did not produce any output. For the remaining 676 memes, 10,426 relations and 9,666 objects were detected in total. The number of unique objects and relations was 142 and 970 respectively. And while the number of objects detected in an image ranged from 2 to 16, the number of detected relations ranged from 1 to 40. The top 15 most frequently detected object and relation types are shown in Fig. 2. 3.2 Knowledge linking 3.3 Knowledge graph input [6] This number was chosen after observation of the scene graphs resulting from the memes in order to avoid having an overcrowded scene graph with multiple objects. [7] https://spacy.io/universe/project/spacy-transformers [8] https://www.wikidata.org/wiki/Wikidata:Main_Page [9] All the texts are concatenated with a full stop. This paper is available on arxiv under CC 4.0 license. Authors: (1) Vasiliki Kougia, University of Vienna, Faculty of computer science, Vienna, Austria & UniVie Doctoral School Computer Science, Vienna, Austria; (2) Simon Fetze, University of Vienna, Faculty of computer science, Vienna, Austria; (3) Thomas Kirchmair, University of Vienna, Faculty of computer science, Vienna, Austria; (4) Erion Çano, University of Vienna, Faculty of computer science, Vienna, Austria; (5) Sina Moayed Baharlou, Boston University, Department of Electrical and Computer Engineering, Boston, MA, USA; (6) Sahand Sharifzadeh, Ludwig Maximilians University of Munich, Faculty of Computer Science, Munich, Germany; (7) Benjamin Roth, University of Vienna, Faculty of computer science, Vienna, Austria. Authors: Authors: (1) Vasiliki Kougia, University of Vienna, Faculty of computer science, Vienna, Austria & UniVie Doctoral School Computer Science, Vienna, Austria; (2) Simon Fetze, University of Vienna, Faculty of computer science, Vienna, Austria; (3) Thomas Kirchmair, University of Vienna, Faculty of computer science, Vienna, Austria; (4) Erion Çano, University of Vienna, Faculty of computer science, Vienna, Austria; (5) Sina Moayed Baharlou, Boston University, Department of Electrical and Computer Engineering, Boston, MA, USA; (6) Sahand Sharifzadeh, Ludwig Maximilians University of Munich, Faculty of Computer Science, Munich, Germany; (7) Benjamin Roth, University of Vienna, Faculty of computer science, Vienna, Austria. Table of Links Abstract and Introduction Abstract and Introduction Related Work Related Work MemeGraphs MemeGraphs Benchmarking Benchmarking Analysis Analysis Conclusion, Acknowledgments, and References Conclusion, Acknowledgments, and References 3 MemeGraphs In this section, we describe our method for automatic augmentation of a memes dataset, an approach we call MemeGraphs. Our proposed method consists of three steps: 1. Scene graph construction, 2. Knowledge linking to detected entities and 3. The construction of the final MemeGraphs input. The first two steps are performed automatically by using off-the-shelf models. Hence, the result of MemeGraphs are knowledge graphs representing the meme as a whole, which can be used to classify them, e.g., for hate detection. We build on the MultiOFF dataset, which contains memes extracted from social media during the 2016 U.S. Presidential Election [28]. In what follows, we present the three individual steps of MemeGraphs in detail and how the final result is constructed. 3.1 Scene graphs For 67 memes, Schemata did not produce any output. For the remaining 676 memes, 10,426 relations and 9,666 objects were detected in total. The number of unique objects and relations was 142 and 970 respectively. And while the number of objects detected in an image ranged from 2 to 16, the number of detected relations ranged from 1 to 40. The top 15 most frequently detected object and relation types are shown in Fig. 2. 3.2 Knowledge linking 3.3 Knowledge graph input [6] This number was chosen after observation of the scene graphs resulting from the memes in order to avoid having an overcrowded scene graph with multiple objects. [7] https://spacy.io/universe/project/spacy-transformers [8] https://www.wikidata.org/wiki/Wikidata:Main_Page [9] All the texts are concatenated with a full stop. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

MemeGraphs: Automated Augmentation for Hate Detection in Memes

Authors:
(1) Vasiliki Kougia, University of Vienna, Faculty of computer science, Vienna, Austria & UniVie Doctoral School Computer Science, Vienna, Austria;
(2) Simon Fetze, University of Vienna, Faculty of computer science, Vienna, Austria;
(3) Thomas Kirchmair, University of Vienna, Faculty of computer science, Vienna, Austria;
(4) Erion Çano, University of Vienna, Faculty of computer science, Vienna, Austria;
(5) Sina Moayed Baharlou, Boston University, Department of Electrical and Computer Engineering, Boston, MA, USA;
(6) Sahand Sharifzadeh, Ludwig Maximilians University of Munich, Faculty of Computer Science, Munich, Germany;
(7) Benjamin Roth, University of Vienna, Faculty of computer science, Vienna, Austria. Table of Links Abstract and Introduction Related Work MemeGraphs Benchmarking Analysis Conclusion, Acknowledgments, and References ABSTRACT Memes are a popular form of communicating trends and ideas in social media and on the internet in general, combining the modalities of images and text. They can express humor and sarcasm but can also have offensive content. Analyzing and classifying memes automatically is challenging since their interpretation relies on the understanding of visual elements, language, and background knowledge. Thus, it is important to meaningfully represent these sources and the interaction between them in order to classify a meme as a whole. In this work, we propose to use scene graphs, that express images in terms of objects and their visual relations, and knowledge graphs as structured representations for meme classification with a Transformer-based architecture. We compare our approach with ImgBERT, a multimodal model that uses only learned (instead of structured) representations of the meme, and observe consistent improvements. We further provide a dataset with human graph annotations that we compare to automatically generated graphs and entity linking. Analysis shows that automatic methods link more entities than human annotators and that automatically generated graphs are better suited for hatefulness classification in memes. Keywords hate speech · internet memes · knowledge graphs · multimodal representations. 1 Introduction Internet memes are items such as images, videos, or twitter posts that are widely shared on social media and typically relate to several subjects, such as politics, social, news, and current internet trends.[1] Memes are a popular form of communication and they are often used as a means to express an opinion or stance in a humorous or sarcastic manner, but they can also be hateful and promote problematic content that is likely to hurt specific groups of people and hence be harmful to society in general [2]. Thus, analyzing trending memes can provide insight into people’s reactions and opinions to important societal matters, as well as to the traits of different groups. This can help in tasks like filtering out harmful memes from internet platforms or extracting user opinions for socioeconomic studies. This work focuses on memes in the form of images with some form of superimposed text (sometimes referred to with the technical term image macros) with the goal of detecting hateful content. Lately, there has been increased interest in deep learning models for analyzing memes and classifying them [23, 10, 11], for example in the context of the Hateful Memes competition organized by Facebook [13] and the shared task in the Workshop on Online Abuse and Harms (WOAH) 2021 that was the continuation of the first competition [21]. Another shared task in Semeval 2022 aimed at detecting misogyny in memes [11]. Alongside these shared tasks, the corresponding datasets were published with human annotations capturing important properties such as hatefulness and misogyny. Recent works use multimodal representation learning based on image features from Convolutional Neural Networks (CNNs), Transformer-based language models for the text [13, 3, 14] or multimodal (vision+language) Transformer models in order to classify memes [13]. Some works additionally incorporate specifically extracted image  features such as race [33], person attributes [1, 23], and automatic image captioning output [7, 4]. However, none of these works have included the relations between objects in the form of scene graphs or background knowledge in the form of a text description for the objects depicted in the image. In this paper, we address the issue of classifying hateful memes by performing an automated augmentation in order to represent the visual information and background knowledge (Fig. 1). We built on the MultiOFF dataset, which contains memes extracted from social media during the 2016 U.S. Presidential Election [28]. An off-the-shelf scene graph generation model was employed to produce scene graphs, which contain detected visual objects and relations between them for each meme, and a NER model to detect entities that we linked to background knowledge. The scene graphs were serialized as text resulting in a unified way to represent all the modalities expressed in a meme. This allows using a (unimodal) text classifier to classify the multimodal memes. Hence, we incorporated the automatically produced augmentations to classify the memes using a text-based Transformer model and show that they can improve its performance. The explicit representation of the image as serialized tokens also provides a more interpretable intermediate representation (compared to hidden layers in multimodal models such as ImgBERT [13, 14]). Furthermore, in order to examine how the results of the automatic augmentation would deviate from human ones, we performed manual augmentation. Two human evaluators corrected the automatically produced scene graphs and manually added background knowledge. We compare our automatic MemeGraphs method with models operating on the manually augmented data (only for training or for both training and inference) and find that automatic augmentations assist in achieving better results than manual ones. Contribution Our contributions can be summarized as follows: • We propose MemeGraphs, a novel method for classifying memes utilizing scene graphs augmented with knowledge and providing insights for processing multimodal documents. • We show that adding this kind of knowledge to a text-based Transformer model can improve its classification performance. Furthermore, we show that this yields improvements compared to a simple model only using learned representations to classify the memes, such as ImgBERT. • We conduct extensive experiments with manual and automatic settings for obtaining this knowledge and show that the automatic setting of our MemeGraphs method provides more meaningful information. In the following, we discuss related work and present our MemeGraphs method. Subsequently, in Section 4, we describe the models we implemented and report their results. Finally, we provide a qualitative analysis, including a discussion of the findings of human augmentation, and compare this with the automatic MemeGraphs method.[2] [1] Disclaimer: This paper contains examples of hateful content. [2] The code and data for the MemeGraphs method are available on: https://github.com/vasilikikou/memegraphs. This paper is available on arxiv under CC 4.0 license. Authors: (1) Vasiliki Kougia, University of Vienna, Faculty of computer science, Vienna, Austria & UniVie Doctoral School Computer Science, Vienna, Austria; (2) Simon Fetze, University of Vienna, Faculty of computer science, Vienna, Austria; (3) Thomas Kirchmair, University of Vienna, Faculty of computer science, Vienna, Austria; (4) Erion Çano, University of Vienna, Faculty of computer science, Vienna, Austria; (5) Sina Moayed Baharlou, Boston University, Department of Electrical and Computer Engineering, Boston, MA, USA; (6) Sahand Sharifzadeh, Ludwig Maximilians University of Munich, Faculty of Computer Science, Munich, Germany; (7) Benjamin Roth, University of Vienna, Faculty of computer science, Vienna, Austria. Authors: Authors: (1) Vasiliki Kougia, University of Vienna, Faculty of computer science, Vienna, Austria & UniVie Doctoral School Computer Science, Vienna, Austria; (2) Simon Fetze, University of Vienna, Faculty of computer science, Vienna, Austria; (3) Thomas Kirchmair, University of Vienna, Faculty of computer science, Vienna, Austria; (4) Erion Çano, University of Vienna, Faculty of computer science, Vienna, Austria; (5) Sina Moayed Baharlou, Boston University, Department of Electrical and Computer Engineering, Boston, MA, USA; (6) Sahand Sharifzadeh, Ludwig Maximilians University of Munich, Faculty of Computer Science, Munich, Germany; (7) Benjamin Roth, University of Vienna, Faculty of computer science, Vienna, Austria. Table of Links Abstract and Introduction Abstract and Introduction Related Work Related Work MemeGraphs MemeGraphs Benchmarking Benchmarking Analysis Analysis Conclusion, Acknowledgments, and References Conclusion, Acknowledgments, and References ABSTRACT Memes are a popular form of communicating trends and ideas in social media and on the internet in general, combining the modalities of images and text. They can express humor and sarcasm but can also have offensive content. Analyzing and classifying memes automatically is challenging since their interpretation relies on the understanding of visual elements, language, and background knowledge. Thus, it is important to meaningfully represent these sources and the interaction between them in order to classify a meme as a whole. In this work, we propose to use scene graphs, that express images in terms of objects and their visual relations, and knowledge graphs as structured representations for meme classification with a Transformer-based architecture. We compare our approach with ImgBERT, a multimodal model that uses only learned (instead of structured) representations of the meme, and observe consistent improvements. We further provide a dataset with human graph annotations that we compare to automatically generated graphs and entity linking. Analysis shows that automatic methods link more entities than human annotators and that automatically generated graphs are better suited for hatefulness classification in memes. Keywords hate speech · internet memes · knowledge graphs · multimodal representations. Keywords Keywords 1 Introduction Internet memes are items such as images, videos, or twitter posts that are widely shared on social media and typically relate to several subjects, such as politics, social, news, and current internet trends.[1] Memes are a popular form of communication and they are often used as a means to express an opinion or stance in a humorous or sarcastic manner, but they can also be hateful and promote problematic content that is likely to hurt specific groups of people and hence be harmful to society in general [2]. Thus, analyzing trending memes can provide insight into people’s reactions and opinions to important societal matters, as well as to the traits of different groups. This can help in tasks like filtering out harmful memes from internet platforms or extracting user opinions for socioeconomic studies. This work focuses on memes in the form of images with some form of superimposed text (sometimes referred to with the technical term image macros) with the goal of detecting hateful content. Lately, there has been increased interest in deep learning models for analyzing memes and classifying them [23, 10, 11], for example in the context of the Hateful Memes competition organized by Facebook [13] and the shared task in the Workshop on Online Abuse and Harms (WOAH) 2021 that was the continuation of the first competition [21]. Another shared task in Semeval 2022 aimed at detecting misogyny in memes [11]. Alongside these shared tasks, the corresponding datasets were published with human annotations capturing important properties such as hatefulness and misogyny. Recent works use multimodal representation learning based on image features from Convolutional Neural Networks (CNNs), Transformer-based language models for the text [13, 3, 14] or multimodal (vision+language) Transformer models in order to classify memes [13]. Some works additionally incorporate specifically extracted image  features such as race [33], person attributes [1, 23], and automatic image captioning output [7, 4]. However, none of these works have included the relations between objects in the form of scene graphs or background knowledge in the form of a text description for the objects depicted in the image. In this paper, we address the issue of classifying hateful memes by performing an automated augmentation in order to represent the visual information and background knowledge (Fig. 1). We built on the MultiOFF dataset, which contains memes extracted from social media during the 2016 U.S. Presidential Election [28]. An off-the-shelf scene graph generation model was employed to produce scene graphs, which contain detected visual objects and relations between them for each meme, and a NER model to detect entities that we linked to background knowledge. The scene graphs were serialized as text resulting in a unified way to represent all the modalities expressed in a meme. This allows using a (unimodal) text classifier to classify the multimodal memes. Hence, we incorporated the automatically produced augmentations to classify the memes using a text-based Transformer model and show that they can improve its performance. The explicit representation of the image as serialized tokens also provides a more interpretable intermediate representation (compared to hidden layers in multimodal models such as ImgBERT [13, 14]). Furthermore, in order to examine how the results of the automatic augmentation would deviate from human ones, we performed manual augmentation. Two human evaluators corrected the automatically produced scene graphs and manually added background knowledge. We compare our automatic MemeGraphs method with models operating on the manually augmented data (only for training or for both training and inference) and find that automatic augmentations assist in achieving better results than manual ones. Contribution Our contributions can be summarized as follows: Contribution • We propose MemeGraphs, a novel method for classifying memes utilizing scene graphs augmented with knowledge and providing insights for processing multimodal documents. • We show that adding this kind of knowledge to a text-based Transformer model can improve its classification performance. Furthermore, we show that this yields improvements compared to a simple model only using learned representations to classify the memes, such as ImgBERT. • We conduct extensive experiments with manual and automatic settings for obtaining this knowledge and show that the automatic setting of our MemeGraphs method provides more meaningful information. In the following, we discuss related work and present our MemeGraphs method. Subsequently, in Section 4, we describe the models we implemented and report their results. Finally, we provide a qualitative analysis, including a discussion of the findings of human augmentation, and compare this with the automatic MemeGraphs method.[2] [1] Disclaimer: This paper contains examples of hateful content. [2] The code and data for the MemeGraphs method are available on: https://github.com/vasilikikou/memegraphs. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

MemeGraphs: Linking Memes to Knowledge Graphs

Authors:
(1) Rui Cao, Singapore Management University;
(2) Ming Shan Hee, Singapore University of Design and Technology;
(3) Adriel Kuek, DSO National Laboratories;
(4) Wen-Haw Chong, Singapore Management University;
(5) Roy Ka-Wei Lee, Singapore University of Design and Technology
(6) Jing Jiang, Singapore Management University. Table of Links Abstract and Introduction Related Work Preliminary Proposed Method Experiment Conclusion and References Appendix ABSTRACT Hateful meme detection is a challenging multimodal task that requires comprehension of both vision and language, as well as cross-modal interactions. Recent studies have tried to fine-tune pre-trained vision-language models (PVLMs) for this task. However, with increasing model sizes, it becomes important to leverage powerful PVLMs more efficiently, rather than simply fine-tuning them. Recently, researchers have attempted to convert meme images into textual captions and prompt language models for predictions. This approach has shown good performance but suffers from noninformative image captions. Considering the two factors mentioned above, we propose a probing-based captioning approach to leverage PVLMs in a zero-shot visual question answering (VQA) manner. Specifically, we prompt a frozen PVLM by asking hateful contentrelated questions and use the answers as image captions (which we call Pro-Cap), so that the captions contain information critical for hateful content detection. The good performance of models with Pro-Cap on three benchmarks validates the effectiveness and generalization of the proposed method.[1] CCS CONCEPTS • Computing methodologies → Natural language processing; Computer vision representations. KEYWORDS memes, multimodal, semantic extraction ACM Reference Format: Rui Cao, Ming Shan Hee, Adriel Kuek, Wen-Haw Chong, Roy Ka-Wei Lee, and Jing Jiang. 2023. Pro Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detection. In Proceedings of the 31st ACM International Conference on Multimedia (MM ’23), October 29-November 3, 2023, Ottawa, ON, Canada. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3581783.3612498 Disclaimer: This paper contains violence and discriminatory content that may be disturbing to some readers. 1 INTRODUCTION Memes, which combine images with short texts, are a popular form of communication in online social media. Internet memes are often intended to express humor or satire. However, they are increasingly being exploited to spread hateful content across online platforms. Hateful memes attack individuals or communities based on their identities such as race, gender, or religion [5, 8, 12, 27]. The propagation of hateful memes can lead to discord online and may potentially result in hate crimes. Therefore, it is urgent to develop accurate hateful meme detection methods. The task of hateful meme detection is challenging due to the multimodal nature of memes. Detection involves not only comprehending both the images and the texts but also understanding how these two modalities interact. Previous work [14, 28, 35, 36] learns cross-modal interactions from scratch using hateful meme detection datasets. However, it may be difficult for models to learn complicated multimodal interactions with the limited amount of data available from these datasets. With the development of Pretrained Vision-Language Models (PVLMs) such as VisualBERT [18] and ViLBERT [23], recent work leverage these powerful PVLMs to facilitate the hateful meme detection task. A common approach is to fine-tune PVLMs with task-specific data [9, 20, 26, 34, 37]. However, it is less feasible to fine-tune the larger models such as BLIP-2 [15] and Flamingo [1] on meme detection because there are billions of trainable parameters. Therefore, computationally feasible solutions other than direct fine-tuning are needed to leverage large PVLMs in facilitating hateful meme detection. Different from the approach above using PVLMs, PromptHate[2] is a recently proposed model that converts the multimodal meme detection task into a unimodal masked language modeling task. It first generates meme image captions with an off-the-shelf image caption generator, ClipCap [25]. By converting all input information into text, it can prompt a pre-trained language model along with two demonstrative examples to predict whether or not the input is hateful by leveraging the rich background knowledge in the language model. Although PromptHate achieves state-of-the-art performance, it is significantly affected by the quality of image captions, as shown in Table 1. Image captions that are merely generic descriptions of images may omit crucial details [14, 37], such as the race and gender of people, which are essential for hateful content detection. But with additional image tags, such as entities found in the images and demographic information about the people in the images, the same model can be significantly improved, as shown in Table 1. However, generating these additional image tags is laborious and costly. For instance, entity extraction is usually conducted with the Google Vision Web Entity Detection API [2], which is a paid service. Ideally, we would like to find a more affordable way to obtain entity and demographic information from the images that is critical for hateful content detection. Both above-mentioned approaches (i.e., one using PVLMs and the other converting the task to a unimodal task) have their pros and cons. In this paper, we combine the ideas from these two approaches and design a hateful meme detection method that leverages the power of a frozen PVLM to complement the unimodal approach of PromptHate. Specifically, we use a set of “probing” questions to query a PVLM (BLIP-2 [15] in our experiments) for information related to common vulnerable targets in hateful content. The answers obtained from the probing questions will be treated as image captions (denoted as Pro-Cap) and used as input to a trainable hateful meme detection model. Figure 1 illustrates the overall workflow of the method. We refer to the step of using probing questions to generate the captions as probing-based captioning. Our proposed method fills existing research gaps by: 1) Leverage a PVLM without any adaptation or fine-tuning, thereby reducing computational cost; 2) Instead of explicitly obtaining additional image tags with costly APIs, we utilize the frozen PVLM to generate captions that contain information useful for hateful meme detection. To the best of our knowledge, this is the first work that to leverage PVLMs in a zero-shot manner through question answering to assist in the hateful meme detection task. To further validate our method, we test the effect of the generated Pro-Cap on both PromptHate[2] and a BERT-based[4] hateful meme detection model. Based on the experimental results, we observe that PromptHate with Pro-Cap (denoted as Pro-CapPromptHate) significantly surpasses the original PromptHate without additional image tags (i.e., about 4, 6, and 3 percentage points of absolute performance improvement on FHM [12], MAMI [5], and HarM [28] respectively). ProCapPromptHate also achieves comparable results with PromptHate with additional image tags, indicating that probing-based captioning can be a more affordable way of obtaining image entities or demographic information. Case studies further show that Pro-Cap offers essential image details for hateful content detection, enhancing the explainability of models to some extent. Meanwhile, ProCapBERT clearly surpasses multimodal BERT-based models of similar sizes (i.e., about 7 percentage points of absolute improvement with VisualBERT on FHM [12]), proving the generalization of the probing-based captioning method. [1] Code is available at: https://github.com/Social-AI-Studio/Pro-Cap [2] https://cloud.google.com/vision/docs/detecting-web This paper is available on arxiv under CC 4.0 license. Authors: (1) Rui Cao, Singapore Management University; (2) Ming Shan Hee, Singapore University of Design and Technology; (3) Adriel Kuek, DSO National Laboratories; (4) Wen-Haw Chong, Singapore Management University; (5) Roy Ka-Wei Lee, Singapore University of Design and Technology (6) Jing Jiang, Singapore Management University. Authors: Authors: (1) Rui Cao, Singapore Management University; (2) Ming Shan Hee, Singapore University of Design and Technology; (3) Adriel Kuek, DSO National Laboratories; (4) Wen-Haw Chong, Singapore Management University; (5) Roy Ka-Wei Lee, Singapore University of Design and Technology (6) Jing Jiang, Singapore Management University. Table of Links Abstract and Introduction Abstract and Introduction Related Work Related Work Preliminary Preliminary Proposed Method Proposed Method Experiment Experiment Conclusion and References Conclusion and References Appendix Appendix ABSTRACT Hateful meme detection is a challenging multimodal task that requires comprehension of both vision and language, as well as cross-modal interactions. Recent studies have tried to fine-tune pre-trained vision-language models (PVLMs) for this task. However, with increasing model sizes, it becomes important to leverage powerful PVLMs more efficiently, rather than simply fine-tuning them. Recently, researchers have attempted to convert meme images into textual captions and prompt language models for predictions. This approach has shown good performance but suffers from noninformative image captions. Considering the two factors mentioned above, we propose a probing-based captioning approach to leverage PVLMs in a zero-shot visual question answering (VQA) manner. Specifically, we prompt a frozen PVLM by asking hateful contentrelated questions and use the answers as image captions (which we call Pro-Cap), so that the captions contain information critical for hateful content detection. The good performance of models with Pro-Cap on three benchmarks validates the effectiveness and generalization of the proposed method.[1] CCS CONCEPTS • Computing methodologies → Natural language processing; Computer vision representations. • Computing methodologies → Natural language processing; Computer vision representations. KEYWORDS memes, multimodal, semantic extraction ACM Reference Format: Rui Cao, Ming Shan Hee, Adriel Kuek, Wen-Haw Chong, Roy Ka-Wei Lee, and Jing Jiang. 2023. Pro Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detection. In Proceedings of the 31st ACM International Conference on Multimedia (MM ’23), October 29-November 3, 2023, Ottawa, ON, Canada. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3581783.3612498 Disclaimer: This paper contains violence and discriminatory content that may be disturbing to some readers. Disclaimer: This paper contains violence and discriminatory content that may be disturbing to some readers. Disclaimer: This paper contains violence and discriminatory content that may be disturbing to some readers. 1 INTRODUCTION Memes, which combine images with short texts, are a popular form of communication in online social media. Internet memes are often intended to express humor or satire. However, they are increasingly being exploited to spread hateful content across online platforms. Hateful memes attack individuals or communities based on their identities such as race, gender, or religion [5, 8, 12, 27]. The propagation of hateful memes can lead to discord online and may potentially result in hate crimes. Therefore, it is urgent to develop accurate hateful meme detection methods. The task of hateful meme detection is challenging due to the multimodal nature of memes. Detection involves not only comprehending both the images and the texts but also understanding how these two modalities interact. Previous work [14, 28, 35, 36] learns cross-modal interactions from scratch using hateful meme detection datasets. However, it may be difficult for models to learn complicated multimodal interactions with the limited amount of data available from these datasets. With the development of Pretrained Vision-Language Models (PVLMs) such as VisualBERT [18] and ViLBERT [23], recent work leverage these powerful PVLMs to facilitate the hateful meme detection task. A common approach is to fine-tune PVLMs with task-specific data [9, 20, 26, 34, 37]. However, it is less feasible to fine-tune the larger models such as BLIP-2 [15] and Flamingo [1] on meme detection because there are billions of trainable parameters. Therefore, computationally feasible solutions other than direct fine-tuning are needed to leverage large PVLMs in facilitating hateful meme detection. Different from the approach above using PVLMs, PromptHate[2] is a recently proposed model that converts the multimodal meme detection task into a unimodal masked language modeling task. It first generates meme image captions with an off-the-shelf image caption generator, ClipCap [25]. By converting all input information into text, it can prompt a pre-trained language model along with two demonstrative examples to predict whether or not the input is hateful by leveraging the rich background knowledge in the language model. Although PromptHate achieves state-of-the-art performance, it is significantly affected by the quality of image captions, as shown in Table 1. Image captions that are merely generic descriptions of images may omit crucial details [14, 37], such as the race and gender of people, which are essential for hateful content detection. But with additional image tags, such as entities found in the images and demographic information about the people in the images, the same model can be significantly improved, as shown in Table 1. However, generating these additional image tags is laborious and costly. For instance, entity extraction is usually conducted with the Google Vision Web Entity Detection API [2], which is a paid service. Ideally, we would like to find a more affordable way to obtain entity and demographic information from the images that is critical for hateful content detection. Both above-mentioned approaches (i.e., one using PVLMs and the other converting the task to a unimodal task) have their pros and cons. In this paper, we combine the ideas from these two approaches and design a hateful meme detection method that leverages the power of a frozen PVLM to complement the unimodal approach of PromptHate. Specifically, we use a set of “probing” questions to query a PVLM (BLIP-2 [15] in our experiments) for information related to common vulnerable targets in hateful content. The answers obtained from the probing questions will be treated as image captions (denoted as Pro-Cap) and used as input to a trainable hateful meme detection model. Figure 1 illustrates the overall workflow of the method. We refer to the step of using probing questions to generate the captions as probing-based captioning. Our proposed method fills existing research gaps by: 1) Leverage a PVLM without any adaptation or fine-tuning, thereby reducing computational cost; 2) Instead of explicitly obtaining additional image tags with costly APIs, we utilize the frozen PVLM to generate captions that contain information useful for hateful meme detection. To the best of our knowledge, this is the first work that to leverage PVLMs in a zero-shot manner through question answering to assist in the hateful meme detection task. To further validate our method, we test the effect of the generated Pro-Cap on both PromptHate[2] and a BERT-based[4] hateful meme detection model. Based on the experimental results, we observe that PromptHate with Pro-Cap (denoted as Pro-CapPromptHate) significantly surpasses the original PromptHate without additional image tags (i.e., about 4, 6, and 3 percentage points of absolute performance improvement on FHM [12], MAMI [5], and HarM [28] respectively). ProCapPromptHate also achieves comparable results with PromptHate with additional image tags, indicating that probing-based captioning can be a more affordable way of obtaining image entities or demographic information. Case studies further show that Pro-Cap offers essential image details for hateful content detection, enhancing the explainability of models to some extent. Meanwhile, ProCapBERT clearly surpasses multimodal BERT-based models of similar sizes (i.e., about 7 percentage points of absolute improvement with VisualBERT on FHM [12]), proving the generalization of the probing-based captioning method. [1] Code is available at: https://github.com/Social-AI-Studio/Pro-Cap [2] https://cloud.google.com/vision/docs/detecting-web This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Pro-Cap: Leveraging a Frozen Vision-Language Model for Hateful Meme Detection

প্রো-ক্যাপ: বিদ্বেষপূর্ণ মেম সনাক্তকরণের জন্য একটি হিমায়িত দৃষ্টি-ভাষা মডেল ব্যবহার করা

Pro-Cap: Nutzung eines Frozen Vision-Language-Modells zur Erkennung hasserfüllter Memes

Pro-Cap: Aprovechamiento de un modelo de visión-lenguaje congelado para la detección de memes de odio

Pro-Cap : exploiter un modèle de langage de vision figé pour la détection des mèmes haineux

प्रो-कैप: घृणास्पद मीम का पता लगाने के लिए फ्रोज़न विज़न-लैंग्वेज मॉडल का लाभ उठाना

Pro-Cap: 凍結された視覚言語モデルを活用して憎悪的なミームを検出する

Pro-Cap: 혐오스러운 밈 탐지를 위해 고정된 비전-언어 모델 활용

Pro-Cap: aproveitando um modelo de linguagem de visão congelada para detecção de memes odiosos

Pro-Cap: использование модели Frozen Vision-Language для обнаружения ненавистных мемов

Pro-Cap: Nefret Dolu Memlerin Tespiti için Donmuş Görüş-Dil Modelinden Yararlanmak

Pro-Cap: Tận dụng mô hình ngôn ngữ tầm nhìn đông lạnh để phát hiện meme đáng ghét

Pro-Cap：利用冻结视觉语言模型进行仇恨模因检测

Performance Analysis of Diverse Hateful Meme Detection Datasets

From Chat-Bots to Killer-Bots?

Governance, Risk, and Compliance (GRC) policies in digital business encompass integrated practices that ensure that your processes align with regulations, industry standards, and internal policies. Most commonly it refers to governance structures, risk management, and compliance measures. The regulatory part of the digital landscape evolves dynamically and leans towards tightening measures, at the same time, risks and error costs continue to rise. This is why creating a specific GRC strategy is essential for digital businesses. In this article, we will talk about key components and best practices for implementing a successful and suitable GRC strategy for a digital business. Key Components of an Effective GRC Strategy GRC is a complex and nuanced domain where the same strategy may work perfectly for one organization and be ruinous for another. However, there are common guidelines to follow, which allow for the creation of a successful GRC strategy that is tailored to your specific business needs. Establishing governance frameworks Governance frameworks play a vital role in defining responsibilities within your organization and empowering decision-makers at different levels. This typically involves clearly outlining roles and responsibilities, creating relevant documentation and company regulations, aligning these regulations with business goals and ensuring all employees are familiar with them. Identifying and Assessing Risks Risk management is a crucial aspect of a GRC strategy for various reasons. Conducting proper risk assessments and pondering their mitigation can save businesses significant costs by preventing potential accidents, rather than dealing with their consequences. For effectiveness, risk assessments need to focus on high-impact risks, and be conducted regularly and systematically. You can use AI tools to enhance risk analysis and assessment by identifying potential blind spots, gaps and increased human factor-related risks. This proactive approach helps in safeguarding the organization against potential threats and ensures resilience and sustainability. Implementing Compliance Programs Implementing compliance programs is about ensuring that the organization adheres to relevant laws, regulations, standards, and ethical practices. The important factor here is not to make these processes inflexible which renders them unable to accommodate frequent and ever changing legal requirements . It is recommended not only to consider the current regulations but also stay alert for forthcoming ones, preparing for them in advance and gradually, to ensure that any changes don't come as sudden disruptions for your company. Integrating GRC into Organizational Culture Integrating GRC into organizational culture is essential. Merely creating a GRC policy is insufficient if it isn’t integrated into daily routines and behaviors of employees at all levels of your organization. Only when GRC becomes a part of the organizational DNA does it become truly effective. The key point here is to understand how regulations translate into actual activities. Aligning GRC Efforts with Business Objectives A GRC strategy should never be detached from your company’s business goals, mission and values. In fact, it should actually support and add value to them. This alignment means that setting GRC priorities needs to be directly linked to key business objectives, ensuring resources are focused on areas of highest impact. 8 Steps to a Successful GRC Strategy Now, from theory to practice. The following steps aid us and our clients in developing a functional GRC strategy that doesn’t deplete the security budget, remains transparent for performance assessment and is scalable and flexible. Decide on the use of GRC Platform This is where it is absolutely crucial to do your homework. Research and compare off-the-shelf solutions that are available on the market from the point of view of whether or not they cover all your GRC needs and determine if additional complementary tools are needed. Sometimes, it is worth investing in a custom GRC solution that is tailored to fit your particular GRC demands rather than try and make do with generic solutions that may leave certain areas uncovered. Consult with compliance and GRC experts and involve your solution architects in the decision-making process to select the best option. Create a GRC Project Roadmap Set your priorities and present them in the form of defined milestones and timelines. Make sure they are realistic and are broken down into manageable phases. Then assign responsibilities for the tasks on the timeline you’ve created. Make them transparent and avoid situations where too many cooks can spoil the broth. Communicate the plan to everyone involved. Perform a Gap Analysis This stage involved identifying the existing GRC processes and evaluating them against your roadmap objectives. Then compare your current practices against industry standards, regulatory requirements, and best practices. Once you’ve identified the gaps, prioritize them based on their risk and impact on your business, and plan actions to address them. Determine and Align Stakeholder Expectations Stakeholder engagement is essential for carrying out effective GRC policies. Make sure it starts at the earliest stages. If you have to (and you probably will), plan meetings to discuss and align on the GRC strategy's goals, priorities, and outcomes. Don’t stop there, you need to keep the stakeholders in the loop of all the GRC-related developments, gather and analyze their feedback and ensure their active participation in decision-making. Define the Objectives Use the SMART objectives-setting technique. SMART stands for Specific, Measurable, Achievable, Relevant, and Time-bound and this abbreviation perfectly reflects the qualities that should define your GRC tasks. Incorporating regular reviews and feedback loops into GRC objectives is crucial for ensuring their effectiveness. Conduct a Comprehensive Organizational Risk Assessment Use a structured framework to conduct risk assessment activities. Another great tip here would be to maximize the use of existing resources such as leveraging the cross-functional expertise within your team. You may already have a compliance expert within your organization potentially working on other projects. Keep a comprehensive log or a documentation of all identified risks with set priorities and mitigation options. Standardization Standardization is key to maintaining consistency and effectiveness in your GRC policies. By standardizing procedures, training, and onboarding processes, you ensure that rules are interpreted uniformly across the organization. Develop a comprehensive set of GRC training materials and procedures making it mandatory for employees to be familiar with them.  Consider incorporating GRC clauses into employment contracts to reinforce compliance. Additionally, adopt and implement industry best practices tailored to your business structure. Conduct regular training and workshops to increase awareness of your GRC policies and keep employees updated on its developments. Management and Revision The process of refining your GRC policies is ongoing. This ensures its scalability, flexibility and alignment with industry standards. So, monitor it, conduct performance checks, gather feedback and identify improvement areas on a regular basis. Best Practices in Creating a Working GRC Policies Besides understanding the steps for creating and implementing GRC policies tailored to your business, here are some tips to ensure its effective execution. Starting small, simple, and focused No need to roll-out a huge-scale system that will complicate matters for your company. Start small, try out, build on it and expand gradually. Keep the focus on high-impact factors. Leadership commitment Cultivate initiative within your leadership by openly communicating GRC principles throughout the organization. Emphasize that these principles are fundamental to the company’s long-term success and reputation. Prioritize risk and compliance activities At an enterprise level, this is what quantifies the value achieved at each stage of your GRC policy implementation. Engage your employees This is where you can get creative. It could be interactive training sessions on ethical decision-making, workshops on identifying and mitigating risks in your workers’ operations, and forums for discussing compliance challenges. Do not forget to encourage feedback and show you value it. Introduce proper incentives You can revise your company’s performance appraisal system to include GRC-related metrics. Your employees can be rewarded not just for achieving business targets but also for demonstrating adherence to compliance standards, ethical behavior, and effective risk management in their tasks. Conclusion Well-planned and implemented GRC strategy is crucial for success in today's digital business landscape. By establishing a robust operational framework, organizations can achieve regulatory compliance, effectively manage risks, improve efficiency, safeguard their reputation, and make informed strategic decisions. Furthermore, fostering a culture of GRC awareness among stakeholders promotes a sense of collective responsibility and accountability, encouraging an environment of cohesion and resilience. Governance, Risk, and Compliance (GRC) policies in digital business encompass integrated practices that ensure that your processes align with regulations, industry standards, and internal policies. Most commonly it refers to governance structures, risk management, and compliance measures. The regulatory part of the digital landscape evolves dynamically and leans towards tightening measures, at the same time, risks and error costs continue to rise. This is why creating a specific GRC strategy is essential for digital businesses. In this article, we will talk about key components and best practices for implementing a successful and suitable GRC strategy for a digital business. Key Components of an Effective GRC Strategy GRC is a complex and nuanced domain where the same strategy may work perfectly for one organization and be ruinous for another. However, there are common guidelines to follow, which allow for the creation of a successful GRC strategy that is tailored to your specific business needs. Establishing governance frameworks Establishing governance frameworks Establishing governance frameworks Governance frameworks play a vital role in defining responsibilities within your organization and empowering decision-makers at different levels. This typically involves clearly outlining roles and responsibilities, creating relevant documentation and company regulations, aligning these regulations with business goals and ensuring all employees are familiar with them. Identifying and Assessing Risks Identifying and Assessing Risks Identifying and Assessing Risks Identifying and Assessing Risks Risk management is a crucial aspect of a GRC strategy for various reasons. Conducting proper risk assessments and pondering their mitigation can save businesses significant costs by preventing potential accidents, rather than dealing with their consequences. For effectiveness, risk assessments need to focus on high-impact risks, and be conducted regularly and systematically. You can use AI tools to enhance risk analysis and assessment by identifying potential blind spots, gaps and increased human factor-related risks. This proactive approach helps in safeguarding the organization against potential threats and ensures resilience and sustainability. Implementing Compliance Programs Implementing Compliance Programs Implementing Compliance Programs Implementing compliance programs is about ensuring that the organization adheres to relevant laws, regulations, standards, and ethical practices. The important factor here is not to make these processes inflexible which renders them unable to accommodate frequent and ever changing legal requirements . It is recommended not only to consider the current regulations but also stay alert for forthcoming ones, preparing for them in advance and gradually, to ensure that any changes don't come as sudden disruptions for your company. Integrating GRC into Organizational Culture Integrating GRC into Organizational Culture Integrating GRC into Organizational Culture Integrating GRC into organizational culture is essential. Merely creating a GRC policy is insufficient if it isn’t integrated into daily routines and behaviors of employees at all levels of your organization. Only when GRC becomes a part of the organizational DNA does it become truly effective. The key point here is to understand how regulations translate into actual activities. Aligning GRC Efforts with Business Objectives Aligning GRC Efforts with Business Objectives Aligning GRC Efforts with Business Objectives A GRC strategy should never be detached from your company’s business goals, mission and values. In fact, it should actually support and add value to them. This alignment means that setting GRC priorities needs to be directly linked to key business objectives, ensuring resources are focused on areas of highest impact. 8 Steps to a Successful GRC Strategy Now, from theory to practice. The following steps aid us and our clients in developing a functional GRC strategy that doesn’t deplete the security budget, remains transparent for performance assessment and is scalable and flexible. Decide on the use of GRC Platform Decide on the use of GRC Platform Decide on the use of GRC Platform This is where it is absolutely crucial to do your homework. Research and compare off-the-shelf solutions that are available on the market from the point of view of whether or not they cover all your GRC needs and determine if additional complementary tools are needed. Sometimes, it is worth investing in a custom GRC solution that is tailored to fit your particular GRC demands rather than try and make do with generic solutions that may leave certain areas uncovered. Consult with compliance and GRC experts and involve your solution architects in the decision-making process to select the best option. Create a GRC Project Roadmap Create a GRC Project Roadmap Create a GRC Project Roadmap Set your priorities and present them in the form of defined milestones and timelines. Make sure they are realistic and are broken down into manageable phases. Then assign responsibilities for the tasks on the timeline you’ve created. Make them transparent and avoid situations where too many cooks can spoil the broth. Communicate the plan to everyone involved. Perform a Gap Analysis Perform a Gap Analysis Perform a Gap Analysis This stage involved identifying the existing GRC processes and evaluating them against your roadmap objectives. Then compare your current practices against industry standards, regulatory requirements, and best practices. Once you’ve identified the gaps, prioritize them based on their risk and impact on your business, and plan actions to address them. Determine and Align Stakeholder Expectations Determine and Align Stakeholder Expectations Determine and Align Stakeholder Expectations Stakeholder engagement is essential for carrying out effective GRC policies. Make sure it starts at the earliest stages. If you have to (and you probably will), plan meetings to discuss and align on the GRC strategy's goals, priorities, and outcomes. Don’t stop there, you need to keep the stakeholders in the loop of all the GRC-related developments, gather and analyze their feedback and ensure their active participation in decision-making. Define the Objectives Define the Objectives Define the Objectives Use the SMART objectives-setting technique. SMART stands for Specific, Measurable, Achievable, Relevant, and Time-bound and this abbreviation perfectly reflects the qualities that should define your GRC tasks. Incorporating regular reviews and feedback loops into GRC objectives is crucial for ensuring their effectiveness. Conduct a Comprehensive Organizational Risk Assessment Conduct a Comprehensive Organizational Risk Assessment Conduct a Comprehensive Organizational Risk Assessment Use a structured framework to conduct risk assessment activities. Another great tip here would be to maximize the use of existing resources such as leveraging the cross-functional expertise within your team. You may already have a compliance expert within your organization potentially working on other projects. Keep a comprehensive log or a documentation of all identified risks with set priorities and mitigation options. Standardization Standardization Standardization Standardization is key to maintaining consistency and effectiveness in your GRC policies. By standardizing procedures, training, and onboarding processes, you ensure that rules are interpreted uniformly across the organization. Develop a comprehensive set of GRC training materials and procedures making it mandatory for employees to be familiar with them.  Consider incorporating GRC clauses into employment contracts to reinforce compliance. Additionally, adopt and implement industry best practices tailored to your business structure. Conduct regular training and workshops to increase awareness of your GRC policies and keep employees updated on its developments. Management and Revision Management and Revision Management and Revision The process of refining your GRC policies is ongoing. This ensures its scalability, flexibility and alignment with industry standards. So, monitor it, conduct performance checks, gather feedback and identify improvement areas on a regular basis. Best Practices in Creating a Working GRC Policies Besides understanding the steps for creating and implementing GRC policies tailored to your business, here are some tips to ensure its effective execution. Starting small, simple, and focused Starting small, simple, and focused Starting small, simple, and focused No need to roll-out a huge-scale system that will complicate matters for your company. Start small, try out, build on it and expand gradually. Keep the focus on high-impact factors. Leadership commitment Leadership commitment Leadership commitment Cultivate initiative within your leadership by openly communicating GRC principles throughout the organization. Emphasize that these principles are fundamental to the company’s long-term success and reputation. Prioritize risk and compliance activities Prioritize risk and compliance activities Prioritize risk and compliance activities At an enterprise level, this is what quantifies the value achieved at each stage of your GRC policy implementation. Engage your employees Engage your employees Engage your employees This is where you can get creative. It could be interactive training sessions on ethical decision-making, workshops on identifying and mitigating risks in your workers’ operations, and forums for discussing compliance challenges. Do not forget to encourage feedback and show you value it. Introduce proper incentives Introduce proper incentives Introduce proper incentives You can revise your company’s performance appraisal system to include GRC-related metrics. Your employees can be rewarded not just for achieving business targets but also for demonstrating adherence to compliance standards, ethical behavior, and effective risk management in their tasks. Conclusion Well-planned and implemented GRC strategy is crucial for success in today's digital business landscape. By establishing a robust operational framework, organizations can achieve regulatory compliance, effectively manage risks, improve efficiency, safeguard their reputation, and make informed strategic decisions. Furthermore, fostering a culture of GRC awareness among stakeholders promotes a sense of collective responsibility and accountability, encouraging an environment of cohesion and resilience.

How to Correctly Plan and Implement A GRC Strategy in Your Digital Business

In such a rapidly evolving industry as tech, where disruptions belong to everyday life, the ability to manage these occurrences is central to maintaining the continuity of operations, and the practice of doing so is widely known as crisis management. In the context of the digital domain, the thing that comes to the fore when an accident arrives is how to save sensitive information from hackers or raise servers from the dead, e.g., technical aspects of the matter. What eludes one’s attention in this rush for a quick patch is how these occurrences affect the people responsible for cleaning this mess, while the consequences run from mere distress caused by the emergency of immediate action to a nervous breakdown. Unless a CEO means to blow the entire company budget on replacing burnt-out staffers every now and then, it is paramount to take precautionary care not only of the software but also of the people who stand at the front of its protection from accidents. This type of crisis management that gives precedence to the staff is defined as human-centered, and those who develop this approach know perfectly well that better preparation of your people for troubles (both in mental and practical terms) increases the probability that their actions will actually help to override the issue, not exacerbate the situation. Leaning back on the principles of this methodology, I suggest the following steps for taking precautionary care of your subordinates to foster their robustness throughout all three stages of a crisis. Stage 1. Preparing for the Worst Though any business can find itself affected by situations that may lead to a crisis, the types of incidents vary greatly from one company to another, even within the digital realm. Consequently, the first step toward a resilient team is to identify what problems the technology you rely upon may open the door to. Spare no effort on blindspot analysis; once done, it will save you money and nerve cells. Next, when potential breaches are identified, it is time to prepare a crisis management plan—a plan of actions your team is expected to take when the disaster steps over your threshold. The more detailed your catastrophe road map is charted, the greater the chances that the issue being tackled will go smoothly and will take place in a timely manner because your company population will know exactly what to do. The major danger that awaits you at this point is to put this plan on the most remote shelf and forget about it until the need arises. Make sure that everyone on your team is acquainted with its contents and can enact them which necessitates fire drills. Simulations of the emergency will highlight weaknesses in the plan and the system in general and showcase team performance which will facilitate your understanding of the business’ preparedness for such occasions. The gaps the drill exposed should be documented and then discussed with the team to show its members how to deal with them. While malfunction simulations will help you spot weaknesses in the system, you are still running the risk of countermeasures going pear-shaped if you are unaware of breaches in the psychological immunity of those who are supposed to face them first. Thus, the next step to take if you seek to improve the robustness of your team is to understand what is capable of breaking them down. Have a talk with each of the team members and identify what stresses them out. Some people easily endure the pressure at the moment but melt down under the continuous torrent of small bugs, or they just won’t be able to deal with the storm of scolding critique from infuriated clients. I bet that one of the major stressors to name will be a fear of making a mistake. Under mentally taxing circumstances, this fear may paralyze the individual’s will to take an action that is dictated by the situation but not outlined in the plan, which may ruin the entire operation. This psychological safety is the bedrock of resilience. That is why it is critically important to show empathy to your subordinates and foster a working environment where an error is a source of knowledge, not a reason for reprimand. These talks will help you get an idea of whom you can place on the disaster recovery team—a dedicated group of people who will shoulder addressing the matter in progress—and whom you can put at the helm as a disaster recovery (DR) leader. Stage 2. In the Midst of a Disaster. When chaos unfolds, it is important to identify whether it is fully digital in character or whether it may do physical damage to your staff. The recent pandemic and a string of armed conflicts erupting across the world reminded us that death danger can transcend the boundaries of imagined probability, so the protection of your employees against such emergencies is an item to include in the plan. Naturally, physical safety is one of our basic necessities, and ensuring it at the workplace is the employer’s duty, so double-check that your people are protected from injury. Furthermore, there is nothing more destabilizing in the moment of danger than unclarity and a sense of lostness caused by a lack of communication and information in general. Thus, keeping the involved up-to-date and providing overall transparency provides them with the level of certainty that forms the solid foundation for a reassured action. What goes hand in hand with informational support is the demonstration that everything goes according to plan: stay calm and collected, and set the example of sangfroid. Panic is contagious, so nipping it in the bud will reduce the stressfulness of the situation. As a continuation of your effort to bolster the mental robustness of your team at the moment, demonstrate to your employees that they are not abandoned to their own devices. There should be a leader who coordinates their actions and points them in the right direction, unfolding a safety net of prompts for occasions when a gap in the protocol calls for a senior manager’s instructions. The right step taken at the right moment will solidify the employee’s confidence in their own abilities to handle a challenge and, as an entailment, boost their mental resistance to tech mishaps. Stage 3. Crisis Aftermath When a crisis is over, the serenity of its aftermath tempts the possibility of consigning it to oblivion. Yet, even though the storm subsided, it is not the time to sit back and relax. Just like a fire drill demands documentation of its results and progress, the real disaster requires a detailed breakdown of the actions and behaviors of the entire team. Moreover, while a simulation may prompt you on the blindspots in the crisis management plan, your employees still know that it is a simulation, so their behavior under real circumstances may digress drastically from what is expected because of the higher level of stress. Consequently, documenting your team’s moves throughout the accident will highlight a) who proved to be more resilient to the disaster and why; and b) which mitigation tactics proved to be effective when enacted by real people on the ground. Keeping track of mistakes helps to adjust the CMP and choose the people who are more fit for a place on the crisis team. A good idea is to arrange a meeting where employees could share their experiences of being in the eye of chaos, what helped them to override it, and what, all the way around, impeded faster resolution. After hitting the hot iron of recording actions during the crisis, fostering resilience in your team demands tackling the imprint the accident might have left on the minds of those who bore the brunt. A corporate psychologist will perfectly do this job for you, or you can at least subsidize a visit or two to a shrink on the side. An honest, one-on-one talk with each participant where you will praise them for apt steps and subtly mention the points for improvement also has the power of alleviating the pressure, helping them slough off the remorse for missteps, learn their lessons, and move on. Wrapping Up However, challenging the laser focus on people at a tech-driven company may be, if you seek to steadily come on top of any hurdle impeding your way, paying attention to the well-being of your employees is imperative. Nurturing resilient teams in hi-tech environments requires mitigation of issues’ effects on people's physical and mental health and calls for creating a space where they feel supported and comfortable sharing their experiences, equipping themselves with hands-on knowledge on handling crises, and adapting to changing situations. In such a rapidly evolving industry as tech, where disruptions belong to everyday life, the ability to manage these occurrences is central to maintaining the continuity of operations, and the practice of doing so is widely known as crisis management. In the context of the digital domain, the thing that comes to the fore when an accident arrives is how to save sensitive information from hackers or raise servers from the dead, e.g., technical aspects of the matter. What eludes one’s attention in this rush for a quick patch is how these occurrences affect the people responsible for cleaning this mess, while the consequences run from mere distress caused by the emergency of immediate action to a nervous breakdown. Unless a CEO means to blow the entire company budget on replacing burnt-out staffers every now and then, it is paramount to take precautionary care not only of the software but also of the people who stand at the front of its protection from accidents. This type of crisis management that gives precedence to the staff is defined as human-centered, and those who develop this approach know perfectly well that better preparation of your people for troubles (both in mental and practical terms) increases the probability that their actions will actually help to override the issue, not exacerbate the situation. Leaning back on the principles of this methodology, I suggest the following steps for taking precautionary care of your subordinates to foster their robustness throughout all three stages of a crisis. Stage 1. Preparing for the Worst Stage 1. Preparing for the Worst Though any business can find itself affected by situations that may lead to a crisis, the types of incidents vary greatly from one company to another, even within the digital realm. Consequently, the first step toward a resilient team is to identify what problems the technology you rely upon may open the door to. Spare no effort on blindspot analysis; once done, it will save you money and nerve cells. Next, when potential breaches are identified, it is time to prepare a crisis management plan—a plan of actions your team is expected to take when the disaster steps over your threshold. The more detailed your catastrophe road map is charted, the greater the chances that the issue being tackled will go smoothly and will take place in a timely manner because your company population will know exactly what to do. The major danger that awaits you at this point is to put this plan on the most remote shelf and forget about it until the need arises. Make sure that everyone on your team is acquainted with its contents and can enact them which necessitates fire drills. Simulations of the emergency will highlight weaknesses in the plan and the system in general and showcase team performance which will facilitate your understanding of the business’ preparedness for such occasions. The gaps the drill exposed should be documented and then discussed with the team to show its members how to deal with them. While malfunction simulations will help you spot weaknesses in the system, you are still running the risk of countermeasures going pear-shaped if you are unaware of breaches in the psychological immunity of those who are supposed to face them first. Thus, the next step to take if you seek to improve the robustness of your team is to understand what is capable of breaking them down. Have a talk with each of the team members and identify what stresses them out. Some people easily endure the pressure at the moment but melt down under the continuous torrent of small bugs, or they just won’t be able to deal with the storm of scolding critique from infuriated clients. I bet that one of the major stressors to name will be a fear of making a mistake. Under mentally taxing circumstances, this fear may paralyze the individual’s will to take an action that is dictated by the situation but not outlined in the plan, which may ruin the entire operation. This psychological safety is the bedrock of resilience. That is why it is critically important to show empathy to your subordinates and foster a working environment where an error is a source of knowledge, not a reason for reprimand. These talks will help you get an idea of whom you can place on the disaster recovery team—a dedicated group of people who will shoulder addressing the matter in progress—and whom you can put at the helm as a disaster recovery (DR) leader. Stage 2. In the Midst of a Disaster. Stage 2. In the Midst of a Disaster. When chaos unfolds, it is important to identify whether it is fully digital in character or whether it may do physical damage to your staff. The recent pandemic and a string of armed conflicts erupting across the world reminded us that death danger can transcend the boundaries of imagined probability, so the protection of your employees against such emergencies is an item to include in the plan. Naturally, physical safety is one of our basic necessities, and ensuring it at the workplace is the employer’s duty, so double-check that your people are protected from injury. Furthermore, there is nothing more destabilizing in the moment of danger than unclarity and a sense of lostness caused by a lack of communication and information in general. Thus, keeping the involved up-to-date and providing overall transparency provides them with the level of certainty that forms the solid foundation for a reassured action. What goes hand in hand with informational support is the demonstration that everything goes according to plan: stay calm and collected, and set the example of sangfroid. Panic is contagious, so nipping it in the bud will reduce the stressfulness of the situation. As a continuation of your effort to bolster the mental robustness of your team at the moment, demonstrate to your employees that they are not abandoned to their own devices. There should be a leader who coordinates their actions and points them in the right direction, unfolding a safety net of prompts for occasions when a gap in the protocol calls for a senior manager’s instructions. The right step taken at the right moment will solidify the employee’s confidence in their own abilities to handle a challenge and, as an entailment, boost their mental resistance to tech mishaps. Stage 3. Crisis Aftermath Stage 3. Crisis Aftermath When a crisis is over, the serenity of its aftermath tempts the possibility of consigning it to oblivion. Yet, even though the storm subsided, it is not the time to sit back and relax. Just like a fire drill demands documentation of its results and progress, the real disaster requires a detailed breakdown of the actions and behaviors of the entire team. Moreover, while a simulation may prompt you on the blindspots in the crisis management plan, your employees still know that it is a simulation, so their behavior under real circumstances may digress drastically from what is expected because of the higher level of stress. Consequently, documenting your team’s moves throughout the accident will highlight a) who proved to be more resilient to the disaster and why; and b) which mitigation tactics proved to be effective when enacted by real people on the ground. Keeping track of mistakes helps to adjust the CMP and choose the people who are more fit for a place on the crisis team. A good idea is to arrange a meeting where employees could share their experiences of being in the eye of chaos, what helped them to override it, and what, all the way around, impeded faster resolution. After hitting the hot iron of recording actions during the crisis, fostering resilience in your team demands tackling the imprint the accident might have left on the minds of those who bore the brunt. A corporate psychologist will perfectly do this job for you, or you can at least subsidize a visit or two to a shrink on the side. An honest, one-on-one talk with each participant where you will praise them for apt steps and subtly mention the points for improvement also has the power of alleviating the pressure, helping them slough off the remorse for missteps, learn their lessons, and move on. Wrapping Up Wrapping Up However, challenging the laser focus on people at a tech-driven company may be, if you seek to steadily come on top of any hurdle impeding your way, paying attention to the well-being of your employees is imperative. Nurturing resilient teams in hi-tech environments requires mitigation of issues’ effects on people's physical and mental health and calls for creating a space where they feel supported and comfortable sharing their experiences, equipping themselves with hands-on knowledge on handling crises, and adapting to changing situations.

Human-centric Crisis Management: Nurturing Resilient Teams in Hi-tech Environments

As companies embrace LLMs, what is the balance between flexibility and risk, and what does that mean for the build-or-buy debate? Although the build or buy dichotomy is a relatively ‘new’ hurdle for technology decision-makers, especially as in-house development teams upskill and build their proprietary solutions, there’s no doubt that the rise of GenAI technologies has taken that conundrum a step further. As organizations grapple with this topic, a fresh layer of decision-making comes into play. Should we be building our own GenAI solutions in-house (and can we?), or is it safer to be buying off-the-shelf products from established vendors?
As things change so rapidly, are we at risk of lock-in at the pace of tech advances, or are we investing huge amounts of resources into keeping up internally? You’ll be shocked to read that the answer, as with most strategic choices, is not clear-cut. I’m going to apply this thinking specifically to the legal sector, where the need for AI is especially prevalent, and the adoption more considered. As in any industry, though, when assessing the build vs. buy decision for LLM solutions, firms must carefully consider their specific use cases, data privacy and security requirements, and long-term ROI. Flexibility is paramount, as the chosen solution must adapt to the firm’s unique needs and integrate effectively with existing systems and workflows. The lure and limits of building in-house GenAI services For companies with more mature software development capabilities, building a custom GenAI application can seem appealing. The APIs underpinning models like GPT-3 are remarkably powerful, enabling developers to quickly spin up basic proof-of-concepts that feel nearly complete. A law firm could connect to the API and prompt it to generate case summaries or first-pass contract reviews. Voilà - a working solution in hours! However, this apparent simplicity belies the reality. Crafting a robust, enterprise-grade GenAI application involves far more than API plumbing. Sensitive data handling, security audits, intuitive UX design, ongoing maintenance as models evolve - the hidden iceberg of work is substantial. For companies lacking seasoned software teams, attempting this undertaking is risky at best. Moreover,  building in-house raises critical questions about data privacy and security. Law firms must ensure their GenAI solutions comply with strict confidentiality requirements and protect client information. Any lapses or breaches could have severe legal and reputational consequences. The convenience and constraints of buying GenAI services On the other end of the spectrum, procuring a subscription to a turnkey GenAI service like Microsoft's Copilot or OpenAI's ChatGPT Enterprise is fast and easy. These generic tools can capably handle a wide array of use cases out of the box, from ideation to writing to analysis. Simply sign up and start prompting. However, one size does not fit all. A lawyer who unwittingly relies on a ChatGPT-generated brief riddled with hallucinated case law precedents could face serious consequences in court. GenAI models are powerful but error-prone, especially when applied naively to specialized domains. Careful human oversight, ideally guided by customizable verification workflows, remains essential. Externally sourced LLMs may also have limitations in terms of flexibility and integrations. Off-the-shelf tools are rarely designed to work seamlessly with a law firm's specific document management systems, CRMs, or bespoke data sources. This lack of integration can hinder adoption and negate efficiency gains. In contrast, custom-built solutions can be tailored to a law firm's exact requirements and orchestrated to plug into existing infrastructure. However, this flexibility comes at a steep development cost and may not be feasible for firms without substantial in-house IT resources. Best of both? Increasingly, forward-looking law firms are seeking a happy medium between the rigidity of pure-play buys and the burdens of bespoke builds. The benefits here include working with partners that have already done the heavy lifting of developing secure, enterprise-hardened GenAI platforms but also providing the API and tooling extensibility to tailor the last mile more specifically. When evaluating potential partners, prioritize those that offer a modular, API-driven architecture, allowing you to leverage pre-built components for common use cases while retaining the flexibility to customize and integrate with your existing systems. This approach enables law firms to start quickly with proven solutions and then iterate and extend as their needs evolve. Increasingly, firms are now looking for partners that aren’t locked into a single LLM or vendor. The GenAI landscape is evolving rapidly, with new models and capabilities emerging all the time. That means law firms need to ensure that they always have access to the most powerful and appropriate language models for their specific applications. What’s the cost difference? In this sort of context, it’s less about the cost differential and more about cost predictability. Cost comparisons between building and buying can be complex, as hidden expenses often lurk beneath the surface. In-house development may require significant upfront investment in infrastructure, talent acquisition, and training, while ongoing maintenance and updates can add unpredictable drains on IT budgets. Conversely, SaaS subscriptions may have lower startup costs but can accumulate substantial fees over time, especially as usage scales. CXOs should carefully model the total cost of ownership for both options over a multi-year horizon, factoring in both hard costs and soft considerations like time-to-value, opportunity costs, and reputational risks. Engaging an experienced partner to guide this analysis can provide valuable perspective. Today’s middle ground has also entered consumption-based pricing into the arena, helping law firms avoid the sticker shock of upfront licenses and better align costs with realized value. The ability to start small and scale usage as adoption grows is key to maximizing ROI and minimizing risk. Bridging the gap We are all sadly familiar with the risks of vendor lock-in and grinding in-house builds, but conversely, we may well have had positive experiences of both. The point is that there’s no one-size-fits-all approach and no decision flow that points to ‘build’ in one context, ‘buy’ in another, or ‘both’ in the third. The modularity enabled by the API-driven nature of modern GenAI opens the door to composable solutions that blend the best of buy and build. With the right decisions, companies can stay agile, avoiding over-investment in bespoke tech that may rapidly become outdated in the fast-moving GenAI space, while still tailoring capabilities to their unique needs. With such rapid advances in technology, being first-movers in this profession can be risky, and organizations tend to counter the rush to act with the cost of delay. In the long run, this cost can mount far higher than first thought. We’re navigating uncertain plains at the moment, and the key is to chart a balanced course, working with partners that will elevate your capabilities and not hinder them. Flexibility, extensibility, and strategic integration are the pillars of GenAI's effectiveness today. As law firms unlock the potential of LLMs to drive efficiency and competitive advantage, they can combine the ease and speed of pre-built foundations with the flexibility and control of customizable components. The GenAI world is advancing by the second, with decisions now centered primarily on minimizing risk and maximizing ROI. As companies embrace LLMs, what is the balance between flexibility and risk, and what does that mean for the build-or-buy debate? As companies embrace LLMs, what is the balance between flexibility and risk, and what does that mean for the build-or-buy debate? Although the build or buy dichotomy is a relatively ‘new’ hurdle for technology decision-makers, especially as in-house development teams upskill and build their proprietary solutions, there’s no doubt that the rise of GenAI technologies has taken that conundrum a step further. As organizations grapple with this topic, a fresh layer of decision-making comes into play. GenAI Should we be building our own GenAI solutions in-house (and can we?), or is it safer to be buying off-the-shelf products from established vendors? As things change so rapidly, are we at risk of lock-in at the pace of tech advances, or are we investing huge amounts of resources into keeping up internally? Should we be building our own GenAI solutions in-house (and can we?), or is it safer to be buying off-the-shelf products from established vendors? As things change so rapidly, are we at risk of lock-in at the pace of tech advances, or are we investing huge amounts of resources into keeping up internally? You’ll be shocked to read that the answer, as with most strategic choices, is not clear-cut. I’m going to apply this thinking specifically to the legal sector, where the need for AI is especially prevalent, and the adoption more considered. As in any industry, though, when assessing the build vs. buy decision for LLM solutions, firms must carefully consider their specific use cases, data privacy and security requirements, and long-term ROI. Flexibility is paramount, as the chosen solution must adapt to the firm’s unique needs and integrate effectively with existing systems and workflows. The lure and limits of building in-house GenAI services The lure and limits of building in-house GenAI services For companies with more mature software development capabilities, building a custom GenAI application can seem appealing. The APIs underpinning models like GPT-3 are remarkably powerful, enabling developers to quickly spin up basic proof-of-concepts that feel nearly complete. A law firm could connect to the API and prompt it to generate case summaries or first-pass contract reviews. Voilà - a working solution in hours! software development However, this apparent simplicity belies the reality. Crafting a robust, enterprise-grade GenAI application involves far more than API plumbing. Sensitive data handling, security audits, intuitive UX design, ongoing maintenance as models evolve - the hidden iceberg of work is substantial. For companies lacking seasoned software teams, attempting this undertaking is risky at best. Moreover,  building in-house raises critical questions about data privacy and security. Law firms must ensure their GenAI solutions comply with strict confidentiality requirements and protect client information. Any lapses or breaches could have severe legal and reputational consequences. The convenience and constraints of buying GenAI services The convenience and constraints of buying GenAI services On the other end of the spectrum, procuring a subscription to a turnkey GenAI service like Microsoft's Copilot or OpenAI's ChatGPT Enterprise is fast and easy. These generic tools can capably handle a wide array of use cases out of the box, from ideation to writing to analysis. Simply sign up and start prompting. However, one size does not fit all. A lawyer who unwittingly relies on a ChatGPT-generated brief riddled with hallucinated case law precedents could face serious consequences in court. GenAI models are powerful but error-prone, especially when applied naively to specialized domains. Careful human oversight, ideally guided by customizable verification workflows, remains essential. Externally sourced LLMs may also have limitations in terms of flexibility and integrations. Off-the-shelf tools are rarely designed to work seamlessly with a law firm's specific document management systems, CRMs, or bespoke data sources. This lack of integration can hinder adoption and negate efficiency gains. In contrast, custom-built solutions can be tailored to a law firm's exact requirements and orchestrated to plug into existing infrastructure. However, this flexibility comes at a steep development cost and may not be feasible for firms without substantial in-house IT resources. Best of both? Best of both? Increasingly, forward-looking law firms are seeking a happy medium between the rigidity of pure-play buys and the burdens of bespoke builds. The benefits here include working with partners that have already done the heavy lifting of developing secure, enterprise-hardened GenAI platforms but also providing the API and tooling extensibility to tailor the last mile more specifically. When evaluating potential partners, prioritize those that offer a modular, API-driven architecture, allowing you to leverage pre-built components for common use cases while retaining the flexibility to customize and integrate with your existing systems. This approach enables law firms to start quickly with proven solutions and then iterate and extend as their needs evolve. Increasingly, firms are now looking for partners that aren’t locked into a single LLM or vendor. The GenAI landscape is evolving rapidly, with new models and capabilities emerging all the time. That means law firms need to ensure that they always have access to the most powerful and appropriate language models for their specific applications. What’s the cost difference? What’s the cost difference? In this sort of context, it’s less about the cost differential and more about cost predictability. Cost comparisons between building and buying can be complex, as hidden expenses often lurk beneath the surface. In-house development may require significant upfront investment in infrastructure, talent acquisition, and training, while ongoing maintenance and updates can add unpredictable drains on IT budgets. Conversely, SaaS subscriptions may have lower startup costs but can accumulate substantial fees over time, especially as usage scales. CXOs should carefully model the total cost of ownership for both options over a multi-year horizon, factoring in both hard costs and soft considerations like time-to-value, opportunity costs, and reputational risks. Engaging an experienced partner to guide this analysis can provide valuable perspective. Today’s middle ground has also entered consumption-based pricing into the arena, helping law firms avoid the sticker shock of upfront licenses and better align costs with realized value. The ability to start small and scale usage as adoption grows is key to maximizing ROI and minimizing risk. Bridging the gap Bridging the gap We are all sadly familiar with the risks of vendor lock-in and grinding in-house builds, but conversely, we may well have had positive experiences of both. The point is that there’s no one-size-fits-all approach and no decision flow that points to ‘build’ in one context, ‘buy’ in another, or ‘both’ in the third. The modularity enabled by the API-driven nature of modern GenAI opens the door to composable solutions that blend the best of buy and build. With the right decisions, companies can stay agile, avoiding over-investment in bespoke tech that may rapidly become outdated in the fast-moving GenAI space, while still tailoring capabilities to their unique needs. With such rapid advances in technology, being first-movers in this profession can be risky, and organizations tend to counter the rush to act with the cost of delay. In the long run, this cost can mount far higher than first thought. We’re navigating uncertain plains at the moment, and the key is to chart a balanced course, working with partners that will elevate your capabilities and not hinder them. Flexibility, extensibility, and strategic integration are the pillars of GenAI's effectiveness today. As law firms unlock the potential of LLMs to drive efficiency and competitive advantage, they can combine the ease and speed of pre-built foundations with the flexibility and control of customizable components. The GenAI world is advancing by the second, with decisions now centered primarily on minimizing risk and maximizing ROI.

Hacking GenAI Integration: Build, buy or both?

Let’s assume that you want to design a new electronic device. Electronics consist of two key elements – a printed circuit board and an enclosure. Customers will often have to order these two elements from different contract developers although some companies offer both these services. So here you face a critical problem – which of them should be developed first?
You’ve got two options. Start with PCB design One option is to develop the PCB first and only after that design the enclosure. This approach is acceptable if the appearance and ergonomics of the device do not play a vital role. In this case, electronics developers are not limited by the shape or size of the housing. They can create a PCB of simple shape (rectangle, square, or circle) and adequate size, using the most optimal layout solutions and components. When the circuit board is ready, you can move on to designing the enclosure or even use an off-the-shelf housing with minimal changes. It will save you money, but don’t expect much in terms of aesthetics. But before choosing an off-the-shelf enclosure, consult with the developers to make sure it is suitable in terms of size, material, and shape. Iterative approach Contrary to what one may expect, this option doesn’t mean that you should design the enclosure first and only then develop the electronics. PCB design is an iterative process. Consequent changes to the electronics may require changes to the enclosure and vice versa. So, this approach rather implies developing both these elements in parallel. This option is preferable when the appearance of the device and its ergonomics are as important as its functionality. The iterative approach takes more time and money compared to the first option but results in an optimal combination of functionality, ease of use, and aesthetics. Therefore, this approach is usually a better option. What can you do at each PCB development phase to make enclosure design easier? Phase 1: Requirements gathering The more thoroughly you describe the requirements for your device, the less rework the project will need. Detailed requirements also ensure there will be no misunderstandings between the customer and the development team. One should describe not only the functionality of the electronics but also the appearance of the device. Therefore, it is advisable to have a 3D model or at least a sketch of the enclosure by this phase. It will allow the team to estimate the feasibility of the project, select optimal solutions, or warn the customer if the enclosure requires redesign. How can you help? Do not make the team wait because they cannot start the development without your project requirements. The sooner you provide the engineers with all the necessary information, the sooner the work will begin. Contract developers often ask new clients to fill out a special questionnaire and list the following requirements. Functional requirements Here the customer should explain as clearly as possible what the future device should do. Describe its main functions and features. Don't worry about how they can be implemented from the technical point of view – that's the developer's job. Explain what purpose the product serves or its intended applications. Give one or two use cases. If your idea is not unique, list analogous goods available on the market and explain how yours should differ from them. Let’s imagine a customer ordering custom electronics development from a contract design house. Here is an example of how he or she could describe functional requirements for his project: The team should create a pair of devices for displaying information during sports events. The pair consists of a control panel and a scoreboard that displays the information. The system is designed for sports such as soccer, basketball, football, and baseball. The control panel should be able to transfer data to the scoreboard: team names, score, current half-time, etc. The control panel and the scoreboard must be two independent devices. The scoreboard should be able to display indicators of different types and sizes, i.e., the scoreboard should be modular. Non-functional requirements They describe the features of the device that are not directly related to its functionality but are important for some reason. Here you should indicate anything you find vital, for example: desired power source; whether the device should have a display; whether the product needs peripheral devices and which ones (for example, a camera); whether the device should be able to store data; whether the product should transmit data and which technology you prefer – Ethernet, Wi-Fi, GSM, etc. desired size and shape of the PCB. The latter is directly related to the enclosure design for your future product. Developers can implement one and the same functionality using different approaches: make a double-sided PCB or a board of six layers; use standard and cheap components or small and expensive ones, mount the components on a rigid PCB or a flexible one, etc. That’s why they would be glad to have a 3D model or a sketch of the enclosure to minimize the possibility of making the wrong choice. Here is an example of non-functional hardware requirements for the project mentioned above: 1.1. The control panel should be able to transmit data to the scoreboard wirelessly.
1.2. The scoreboard should be able to receive signals from a distance of at least 152 m.
1.3. The delay between pressing a key on the control panel and updating the information on the scoreboard should be about 50 ms.
2.1. The scoreboard should be connected to the control panel before starting work.
2.2. The control panel should have an external power source and battery backup. The battery life depends on its size, which will be determined when we get to the enclosure design.
2.3. The scoreboard should have an external power source.
2.4. The control panel should be the size of a standard full-size keyboard. The user should be able to place it on a table and use it with two hands. At the same time, it should be small and light enough so that the user can carry it in one hand and press the keys with the other hand.
3.1. The elements of the scoreboard should be easy to connect.
3.2. Incorrect connection of the components should not cause serious problems. Please note that the customer does not yet have an enclosure or even a sketch. However, he or she already has some idea about the size of the device, which was shared with the team in paragraph 2.4. Operating conditions In what environment do you plan to use your device? Is it indoors or outdoors, in the rain, in extreme cold or heat? When possible, list specific environmental conditions and their parameters: Temperature Humidity Pressure Vibration Hazardous areas: explosive environment, areas with high content of aggressive corrosive agents or other chemicals, etc. If you’re building a consumer-grade product, it is sufficient to indicate that it should operate under normal conditions: a temperature of 25±10°C, relative air humidity from 45% to 80%, and atmospheric pressure from 84 to 106.7 kPa. The customer mentioned above wants his or her control panel to be used mainly indoors, which corresponds to normal conditions. But the scoreboard has to stay in the open air. Here is how the customer can describe the requirements for this part of the system: The scoreboard electronics should be able to operate on the hottest summer days when the temperature inside the enclosure reaches about 65°C. In addition, the scoreboard should be able to operate in the rain. I would rather that the indicator elements do not require special protection from water. Alternatively, you could use a conformal coating. Certification requirements Almost any electronic device intended for commercial sale must meet quality and safety requirements. If you are not an expert, handling this matter on your own may be very difficult. Contact the notified body responsible for product certification in your country to learn about specific requirements or hire a private business that specializes in this field. Legislation in this area often requires devices to have a sufficient level of protection from certain factors: moisture, dust, fire, electromagnetic noise, electrostatic discharge events, etc. Requirements depend on the type of the device and its area of application. Make sure to share this information with both electronics developers and enclosure designers. Phase 2. Project Estimation Engineers cannot move on to the next project phase before they collect the requirements. When you provide all the information about the functions, features, and operating conditions of the product, they can estimate the project from a technical perspective. Estimating a simple project takes little time. But complex projects may require more time and effort, sometimes even an additional research phase that can help developers answer the following questions: Is it possible to create your device at all? How to design it? If there are more than one solution, which one is optimal? How much will the product cost? How much will the project cost you? How long will it take to develop your product? Is the proposed enclosure design compatible with the PCB? After that, the team will be able to tell how long the development will take. Simple projects usually take about 150 hours. Complex projects may take about 500-700 hours or even longer. How can you help? The best thing a customer can do at this phase is to give the team enough time. Additional research allows engineers to identify many less obvious problems at an early stage of the process. If you skip this step, the project may require serious modifications and alterations later, which means even more work, time, and money. Phase 3: Development This is when the actual development begins. Engineers create a PCB schematic, select components, and build a 3D model of the board. Then the PCB is manufactured and tested. If necessary, the team also creates software for the device. Many developers create a second iteration of the PCB after that. The first version of the board almost always contains faults and shortcomings. In addition, components sometimes behave in a way different from what their documentation says or begin to affect adjacent components in an unexpected way. In this case, the components have to be replaced. The team can eliminate most of these problems when creating the second iteration. The second version of the PCB often requires modifications as well, but they are usually minor fixes. How can you help? At this phase, you will most likely have to choose between appearance and functionality. You may have to give up some functions and features of the device in favor of its appearance, or vice versa, sacrifice the ease of use for the sake of functionality. Make sure you have an enclosure prototype by the time the PCB is ready for testing. Phase 4: Testing and Debugging At the final stage, engineers place the PCB inside the enclosure prototype to test how the device functions under appropriate operating conditions. If developers discover any issues, they find the cause and fix them. How can you help? Customers can and should test the product too. You should check not only the functionality of the product but its ease of use as well. After that, the enclosure may undergo further changes, mostly for the sake of ergonomics. For example, the customer may want to change the location of buttons or LEDs. This will require changes in both the enclosure and electronics design. Moreover, if the previous iteration of the device didn’t have any LEDs, and now the customer wants to add some, the developers will have to create an additional PCB for controlling the LEDs. This is a minor change, but it still requires time and effort. Nevertheless, you may find such changes absolutely necessary even if they solely improve the product’s ergonomics. Highlights If neither the PCB nor the enclosure for your future device has not been designed yet, you have two options. When the appearance of the device is not particularly important, you can first design the PCB and only then design the enclosure for it. Alternatively, you can use a cheaper off-the-shelf enclosure. If the design of the enclosure is no less important than the functionality of the device, it is recommended to prepare at least a sketch of the enclosure first. After that, both the enclosure and the electronics should be developed in parallel. Make sure to clearly communicate product requirements, including desirable PCB shape and size, to the team to avoid problems with enclosure design. It is also important to give engineers enough time to estimate the project and conduct additional research if necessary. It will highlight potential problems at the very beginning of the work. As development progresses, you may have to choose between the appearance and functionality of the device. Lastly, when the prototype of the device is complete, you should test it to check its ease of use. After that, you may want to make certain changes to both the enclosure and PCB design. This approach allows you to reach a compromise between the appearance and functionality of the product while minimizing risks.

PCB Development or Enclosure Design: Which Comes First?

MinIO makes a  powerful primary TileDB backend because both are built for performance and scale. MinIO is a single Go binary that can be launched in many different types of cloud and on-prem environments. It's very lightweight, but also feature-packed with things like and , and it provides with various applications. MinIO is the perfect companion for TileDB because of its industry-leading performance and scalability. MinIO is capable of tremendous performance – we’ve benchmarked it at 325 GiB/s (349 GB/s) on GETs and 165 GiB/s (177 GB/s) on PUTs with just 32 nodes of – and is used to build data lakes/lake houses with analytics and AI/ML workloads. replication encryption integrations off-the-shelf NVMe SSDs TileDB is used to store data in a variety of applications, such as Genomics, Geospatial, Biomedical Imaging, Finance, Machine Learning, and more. The power of TileDB stems from the fact that any data can be modeled efficiently as either a dense or a sparse multi-dimensional array, which is the format used internally by most data science tooling. By storing your data and metadata in TileDB arrays, you abstract all the data storage and management pains, while efficiently accessing the data with your favorite programming language or data science tool via our numerous APIs and integrations. Set Up TileDB Let’s dive in and create some test data using TileDB. Install the TileDB module, which should also install the dependency. pip numpy % pip3 install tiledb


Collecting tiledb

  Downloading tiledb-0.25.0-cp311-cp311-macosx_11_0_arm64.whl (10.4 MB)

 	━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 10.4/10.4 MB 2.7 MB/s eta 0:00:00

Collecting packaging

  Downloading packaging-23.2-py3-none-any.whl (53 kB)

 	━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.0/53.0 kB 643.1 kB/s eta 0:00:00

Collecting numpy>=1.23.2

  Downloading numpy-1.26.3-cp311-cp311-macosx_11_0_arm64.whl (14.0 MB)

 	━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.0/14.0 MB 2.5 MB/s eta 0:00:00

Installing collected packages: packaging, numpy, tiledb

Successfully installed numpy-1.26.3 packaging-23.2 tiledb-0.25.0 Create a test array by running the below Python script, name it . tiledb-demo.py import tiledb

import numpy as np

import os, shutil


# Local path

array_local = os.path.expanduser("./tiledb_demo")


# Create a simple 1D array

tiledb.from_numpy(array_local, np.array([1.0, 2.0, 3.0]))


# Read the array

with tiledb.open(array_local) as A:

	print(A[:]) Run the script % python3 tiledb-demo.py

[1. 2. 3.] This will create a directory called to store the actual data. tiledb_demo % ls -l tiledb_demo/

total 0

drwxr-xr-x  3 aj  staff   96 Jan 31 05:27 __commits

drwxr-xr-x  2 aj  staff   64 Jan 31 05:27 __fragment_meta

drwxr-xr-x  3 aj  staff   96 Jan 31 05:27 __fragments

drwxr-xr-x  2 aj  staff   64 Jan 31 05:27 __labels

drwxr-xr-x  2 aj  staff   64 Jan 31 05:27 __meta

drwxr-xr-x  4 aj  staff  128 Jan 31 05:27 __schema You can continue using it as is but it's no bueno if everything is local because if the local disk or node fails then you lose your entire data. Let's do something fun, like reading this same data from a MinIO bucket instead. Migrating Data to MinIO Bucket We’ll start by pulling mc in our docker ecosystem and then using play.min.io to create the bucket. Pull mc docker image % docker pull minio/mc Test with MinIO Play by listing all the buckets % docker run minio/mc ls play


[LONG TRUNCATED LIST OF BUCKETS] Create a bucket to move our local TileDB data to, name it . tiledb-demo % docker run minio/mc mb play/tiledb-demo


Bucket created successfully `play/tiledb-demo`. Copy the contents of the data directory to the MinIO bucket tiledb_demo tiledb-demo % docker run -v $(pwd)/tiledb_demo:/tiledb_demo minio/mc cp --recursive /tiledb_demo play/tiledb-demo


`/tiledb_demo/__commits/__1706696859767_1706696859767_777455531063403b811b2a2bf79d40e7_21.wrt` -> `play/tiledb-demo/tiledb_demo/__commits/__1706696859767_1706696859767_777455531063403b811b2a2bf79d40e7_21.wrt`

`/tiledb_demo/__fragments/__1706696859767_1706696859767_777455531063403b811b2a2bf79d40e7_21/a0.tdb` -> `play/tiledb-demo/tiledb_demo/__fragments/__1706696859767_1706696859767_777455531063403b811b2a2bf79d40e7_21/a0.tdb`

`/tiledb_demo/__fragments/__1706696859767_1706696859767_777455531063403b811b2a2bf79d40e7_21/__fragment_metadata.tdb` -> `play/tiledb-demo/tiledb_demo/__fragments/__1706696859767_1706696859767_777455531063403b811b2a2bf79d40e7_21/__fragment_metadata.tdb`

`/tiledb_demo/__schema/__1706696859758_1706696859758_74e7040e138a4cca93e34aca1c587108` -> `play/tiledb-demo/tiledb_demo/__schema/__1706696859758_1706696859758_74e7040e138a4cca93e34aca1c587108`


Total: 3.24 KiB, Transferred: 3.24 KiB, Speed: 1.10 KiB/s List the contents of to make sure the data has been copied. tiledb-demo % docker run minio/mc ls play/tiledb-demo/tiledb_demo

[2024-01-15 14:15:57 UTC] 	0B __commits/

[2024-01-15 14:15:57 UTC] 	0B __fragments/

[2024-01-15 14:15:57 UTC] 	0B __schema/ Note: The MinIO Client ( ), or any S3 compatible client, only copies non-empty folders. The reason for this is that in the object storage world the data is organized based on bucket prefixes, so non-empty folders are not needed. In a future blog we’ll dive deeper into how data is organized with prefixes and folders. Hence, you see only these 3 folders and not the rest that we saw in the local folder. mc Now let’s try to read the same data directly from the MinIO bucket using the Python code below, name the file . tiledb-minio-demo.py import tiledb

import numpy as np


# MinIO keys

minio_key = "minioadmin"

minio_secret = "minioadmin"


# The configuration object with MinIO keys

config = tiledb.Config()

config["vfs.s3.aws_access_key_id"] = minio_key

config["vfs.s3.aws_secret_access_key"] = minio_secret

config["vfs.s3.scheme"] = "https"

config["vfs.s3.region"] = ""

config["vfs.s3.endpoint_override"] = "play.min.io:9000"

config["vfs.s3.use_virtual_addressing"] = "false"


# Create TileDB config context

ctx = tiledb.Ctx(config)


# The MinIO bucket URI path of tiledb demo

array_minio = "s3://tiledb-demo/tiledb_demo/"


with tiledb.open(array_minio, ctx=tiledb.Ctx(config)) as A:

	print(A[:]) The output should look familiar % python3 tiledb-minio-demo.py

[1. 2. 3.] We've read from MinIO, next let's see how we can write the data directly in a MinIO bucket, instead of copying it to MinIO from an existing source. Writing Directly to the MinIO Bucket So far we’ve shown you how to read data that already exists, either in local storage or an existing bucket. But if you wanted to start fresh by writing directly to MinIO from the get-go, how would that work? Let’s take a look. The code to write data directly to the MinIO bucket is the same as above except with two line changes. The path to the MinIO bucket where TileDB data is stored must be updated to (instead of ). tiledb_minio_demo tiledb_demo We’ll use the function, as we did earlier with local storage, to create the array to store in the MinIO bucket. tiledb.from_numpy [TRUNCATED]


# The MinIO bucket URI path of tiledb demo

array_minio = "s3://tiledb-demo/tiledb_minio_demo/"


tiledb.from_numpy(array_minio, np.array([1.0, 2.0, 3.0]), ctx=tiledb.Ctx(config))


[TRUNCATED] After making these 2 changes, run the script and you should see the output below % python3 tiledb-minio-demo.py

[1. 2. 3.] If you run the script again it will fail with the below error because it will try to write again. tiledb.cc.TileDBError: [TileDB::StorageManager] Error: Cannot create array; Array 's3://tiledb-demo/tiledb_minio_demo/' already exists Just comment out the following line and you can re-run it multiple times. # tiledb.from_numpy(array_minio, np.array([1.0, 2.0, 3.0]), ctx=tiledb.Ctx(config)) % python3 tiledb-minio-demo.py

[1. 2. 3.]


% python3 tiledb-minio-demo.py

[1. 2. 3.] Check the MinIO Play bucket to make sure the data is in there as expected % docker run minio/mc ls play/tiledb-demo/tiledb_minio_demo/

[2024-01-15 16:45:04 UTC] 	0B __commits/

[2024-01-15 16:45:04 UTC] 	0B __fragments/

[2024-01-15 16:45:04 UTC] 	0B __schema/ There you go, getting data into MinIO is that simple. Did you get the same results as earlier? You should have, but if you didn't there are a few things you can check out. Common Pitfalls We’ll look at some common errors you might encounter while trying to read/write to MinIO. If your access key and secret key are incorrect, you should expect to see an error message like below tiledb.cc.TileDBError: [TileDB::S3] Error: Error while listing with prefix 's3://tiledb-demo/tiledb_minio_demo/__schema/'... The request signature we calculated does not match the signature you provided. Check your key and signing method. Next, you need to ensure the hostname and port are correct, without a proper endpoint these are the errors you would encounter. Incorrect Hostname: tiledb.cc.TileDBError: [TileDB::S3] Error: … Couldn't resolve host name Incorrect Port: tiledb.cc.TileDBError: [TileDB::S3] Error: … Couldn't connect to server Last but not least, one of the most cryptic errors I’ve seen is the following tiledb.cc.TileDBError: [TileDB::S3] Error: … [HTTP Response Code: -1] [Remote IP: 98.44.32.5] : curlCode: 56, Failure when receiving data from the peer After a ton of debugging it turns out that if you are connecting using http but the MinIO server has TLS activated then you will see the above error. Just be sure the connection scheme is set to the right configuration, in this case, config["vfs.s3.scheme"] = "https". Racks on Racks on Racks There is a rap song (you can search for it) where they rap about having stacks on stacks on stacks of * cash. But there is another rap song where they claim they have so many stacks of cash that they can’t be called “stacks” anymore, they are now “racks”. Essentially when your stacks get so big and so high you need racks on racks on racks to store your stacks of cash. cough* This is an apt comparison because your stacks of data mean as much (or more) to you as the stacks of cash they're rapping about. If only there was something like MinIO to keep all your objects – physical or virtual – safe and readily accessible. With MinIO in the mix, you can easily scale TileDB to with relative ease. You also get all the features that make MinIO great like , , , , among others right out of the box. By having all your data in MinIO, you decrease required storage complexity and therefore realize considerable savings on data storage costs, while at the same time running MinIO on commodity hardware provides the best possible performance-to-cost ratio. MinIO supercharges your TileDB engine with industry-leading performance that makes querying a joy. multiple racks across multiple datacenters Security and Access Control Tiering Object Locking and Retention Key Encryption Service (KES) We’ve added the code snippets used in this blog to a . If you have any questions on how to connect MinIO to TileDB or migrate data into MinIO be sure to reach out to us on ! git repository Slack Also appears . here

Supercharge TileDB Engine with MinIO

MinIO সহ সুপারচার্জ টাইলডিবি ইঞ্জিন

Laden Sie die TileDB-Engine mit MinIO auf

Superalimentar el motor TileDB con MinIO

Boostez le moteur TileDB avec MinIO

मिनिओ के साथ सुपरचार्ज टाइलडीबी इंजन

MinIO で TileDB エンジンをスーパーチャージする

MinIO로 TileDB 엔진 강화

Turbine o mecanismo TileDB com MinIO

Усовершенствуйте движок TileDB с помощью MinIO

MinIO ile TileDB Motorunu Güçlendirin

Công cụ TileDB siêu nạp với MinIO

使用 MinIO 增强 TileDB 引擎

Retrieval augmented generation (RAG) is by now of generative artificial intelligence (AI) applications. Supplementing your application prompt with relevant context retrieved from a vector database can dramatically increase accuracy and reduce hallucinations. This means that increasing relevance in vector search results has a direct correlation to the quality of your RAG application. a standard part There are two reasons RAG remains popular and increasingly relevant even : as large language models (LLMs) increase their context window LLM response time and price both increase linearly with context length. LLMs still struggle with both and across massive contexts. retrieval reasoning But isn’t a magic wand. In particular, the most common design, dense passage retrieval (DPR), represents both queries and passages as a single embedding vector and uses straightforward to score relevance. This means DPR relies heavily on the embeddings model having the breadth of training to recognize all the relevant search terms. RAG cosine similarity Unfortunately, off-the-shelf models struggle with unusual terms, including names, that are not commonly in their training data. DPR also tends to be hypersensitive to chunking strategy, which can cause a relevant passage to be missed if it’s surrounded by a lot of irrelevant information. All of this creates a burden on the application developer to “get it right the first time,” because a mistake usually results in the need to rebuild the index from scratch. Solving DPR’s challenges with ColBERT is a new way of scoring passage relevance using a language model that substantially solves the problems with DPR. This diagram from shows why it’s so exciting: ColBERT BERT the first ColBERT paper This compares the performance of ColBERT with other state-of-the-art solutions for the MS-MARCO dataset. (MS-MARCO is a set of Bing queries for which scored the most relevant passages by hand. It’s one of the better retrieval benchmarks.) Lower and to the right is better. Microsoft In short, ColBERT handily outperforms the field of mostly significantly more complex solutions at the cost of a small increase in latency. To test this, I created and indexed over 1,000 Wikipedia articles with both ada002 DPR and ColBERT. I found that ColBERT delivers significantly better results on unusual search terms. a demo The following screenshot shows that DPR fails to recognize the unusual name of William H. Herndon, an associate of Abraham Lincoln, while ColBERT finds the reference in the Springfield article. Also note that ColBERT’s No. 2 result is for a different William, while none of DPR’s results are relevant. ColBERT is often described in dense machine learning jargon, but it’s actually very straightforward. I’ll show how to implement ColBERT retrieval and scoring on with only a few lines of Python and (CQL). DataStax Astra DB Cassandra Query Language The big idea Instead of traditional, single-vector-based DPR that turns passages into a single “embedding” vector, ColBERT generates a contextually influenced for each token in the passages. ColBERT similarly generates vectors for each token in the query. vector (Tokenization refers to breaking up input into fractions of words before processing by an LLM. .) Andrej Karpathy, a founding member of the OpenAI team, just released an outstanding video on how this works Then, the score of each document is the sum of the maximum similarity of each query embedding to any of the document embeddings: def maxsim(qv, document_embeddings):
    return max(qv @ dv for dv in document_embeddings)

def score(query_embeddings, document_embeddings):
    return sum(maxsim(qv, document_embeddings) for qv in query_embeddings) (@ is the PyTorch operator for dot product and is the most common .) measure of vector similarity That’s it — you can implement ColBERT scoring in four lines of Python! Now you understand ColBERT better than 99% of the people posting about it on X (formerly known as Twitter). The rest of the ColBERT papers deal with: How do you fine-tune the BERT model to generate the best embeddings for a given data set? How do you limit the set of documents for which you compute the (relatively expensive) score shown here? The first question is optional and out of scope for this writeup. I’ll use the pretrained ColBERT checkpoint. But the second is straightforward to do with a vector database like DataStax Astra DB. ColBERT on Astra DB There is a popular Python all-in-one library for ColBERT called ; however, it assumes a static dataset. One of the powerful features of RAG applications is responding to dynamically . So instead, I’m going to use Astra’s vector index to narrow the set of documents I need to score down to the best candidates for each subvector. RAGatouille changing data in real time There are two steps when adding ColBERT to a RAG application: ingestion and retrieval. Ingestion Because each document chunk will have multiple embeddings associated with it, I’ll need two tables: CREATE TABLE chunks (
    title text,
    part int,
    body text,
    PRIMARY KEY (title, part)
);

CREATE TABLE colbert_embeddings (
    title text,
    part int,
    embedding_id int,
    bert_embedding vector<float, 128>,
    PRIMARY KEY (title, part, embedding_id)
);

CREATE INDEX colbert_ann ON colbert_embeddings(bert_embedding)
  WITH OPTIONS = { 'similarity_function': 'DOT_PRODUCT' }; After installing the ColBERT library ( ) and downloading , I can load documents into these tables: pip install colbert-ai the pretrained BERT checkpoint from colbert.infra.config import ColBERTConfig
from colbert.modeling.checkpoint import Checkpoint
from colbert.indexing.collection_encoder import CollectionEncoder

from cassandra.concurrent import execute_concurrent_with_args
from db import DB


def encode_and_save(title, passages):
    db = DB()
    cf = ColBERTConfig(checkpoint='checkpoints/colbertv2.0')
    cp = Checkpoint(cf.checkpoint, colbert_config=cf)
    encoder = CollectionEncoder(cf, cp)

    # encode_passages returns a flat list of embeddings and a list of how many correspond to each passage
    embeddings_flat, counts = encoder.encode_passages(passages)

    # split up embeddings_flat into a nested list
    start_indices = [0] + list(itertools.accumulate(counts[:-1]))
    embeddings_by_part = [embeddings_flat[start:start+count] for start, count in zip(start_indices, counts)]

    # insert into the database
    for part, embeddings in enumerate(embeddings_by_part):
        execute_concurrent_with_args(db.session,
                                     db.insert_colbert_stmt,
                                     [(title, part, i, e) for i, e in enumerate(embeddings)]) (I like to encapsulate my DB logic in a dedicated module; you can access the full source in my .) GitHub repository Retrieval Then retrieval looks like this: def retrieve_colbert(query):
    db = DB()
    cf = ColBERTConfig(checkpoint='checkpoints/colbertv2.0')
    cp = Checkpoint(cf.checkpoint, colbert_config=cf)
    encode = lambda q: cp.queryFromText([q])[0]

    query_encodings = encode(query)
    # find the most relevant documents for each query embedding. using a set
    # handles duplicates so we don't retrieve the same one more than once
    docparts = set()
    for qv in query_encodings:
        rows = db.session.execute(db.query_colbert_ann_stmt, [list(qv)])
        docparts.update((row.title, row.part) for row in rows)
    # retrieve these relevant documents and score each one
    scores = {}
    for title, part in docparts:
        rows = db.session.execute(db.query_colbert_parts_stmt, [title, part])
        embeddings_for_part = [tensor(row.bert_embedding) for row in rows]
        scores[(title, part)] = score(query_encodings, embeddings_for_part)
    # return the source chunk for the top 5
    return sorted(scores, key=scores.get, reverse=True)[:5] Here is the query being executed for the most-relevant-documents part ( ): db.query_colbert_ann_stmt SELECT title, part
FROM colbert_embeddings
ORDER BY bert_embedding ANN OF ?
LIMIT 5 Beyond the basics: RAGStack This article and the linked repository briefly introduce how ColBERT works. You can implement this today with your own data and see immediate results. As with everything in AI, best practices are changing daily, and new techniques are constantly emerging. To make keeping up with the state of the art easier, DataStax is rolling this and other enhancements into , our production-ready RAG library leveraging LangChain and LlamaIndex. Our goal is to provide developers with a consistent library for RAG applications that puts them in control of the step-up to new functionality. Instead of having to keep up with the myriad changes in techniques and libraries, you have a single stream, so you can focus on building your application. You can use RAGStack today to incorporate best practices for LangChain and LlamaIndex out of the box; advances like ColBERT will come to RAGstack in upcoming releases. RAGStack By Jonathan Ellis, DataStax Also appears . here

How ColBERT Helps Developers Overcome the Limits of RAG

কিভাবে ColBERT ডেভেলপারদের RAG এর সীমা অতিক্রম করতে সাহায্য করে

Wie ColBERT Entwicklern hilft, die Grenzen von RAG zu überwinden

Cómo ColBERT ayuda a los desarrolladores a superar los límites de RAG

Comment ColBERT aide les développeurs à surmonter les limites de RAG

कैसे ColBERT डेवलपर्स को RAG की सीमाओं को पार करने में मदद करता है

ColBERT が開発者による RAG の限界の克服にどのように役立つか

ColBERT가 개발자가 RAG의 한계를 극복하도록 돕는 방법

Como ColBERT ajuda os desenvolvedores a superar os limites do RAG

Как ColBERT помогает разработчикам преодолеть ограничения RAG

ColBERT, Geliştiricilerin RAG Sınırlarını Aşmasına Nasıl Yardımcı Olur?

ColBERT giúp nhà phát triển vượt qua giới hạn của RAG như thế nào

ColBERT 如何帮助开发人员克服 RAG 的限制

This paper is available on arxiv under CC 4.0 license. Authors: (1) Xiaoyu Ai, School of Electrical Engineering & Telecommunications, University of New South Wales, Sydney, NSW 2052, Australia; (2) Robert Malaney, School of Electrical Engineering & Telecommunications, University of New South Wales, Sydney, NSW 2052, Australia. Table of Links Abstract & Introduction System Overview Overcoming the limitation of key reconciliation Final Key Rate Experimental Results Discussion Conclusion & References Appendix Abstract Designing a practical Continuous Variable (CV) Quantum Key Distribution (QKD) system requires an estimation of the quantum channel characteristics and the extraction of secure key bits based on a large number of distributed quantum signals. Meeting this requirement in short timescales is difficult. On standard processors, it can take several hours to reconcile the required number of quantum signals. This problem is exacerbated in the context of Low Earth Orbit (LEO) satellite CV-QKD, in which the satellite flyover time is constrained to be less than a few minutes. A potential solution to this problem is massive parallelisation of the classical reconciliation process in which a large-code block is subdivided into many shorter blocks for individual decoding. However, the penalty of this procedure on the important final secured key rate is non-trivial to determine and hitherto has not been formally analysed. Ideally, a determination of the optimal reduced block size, maximising the final key rate, would be forthcoming in such an analysis. In this work, we fill this important knowledge gap via detailed analyses and experimental verification of a CV-QKD sliced reconciliation protocol that uses large block-length low-density parity-check decoders. Our new solution results in a significant increase in the final key rate relative to non-optimised reconciliation. In addition, it allows for the acquisition of quantum secured messages between terrestrial stations and LEO satellites within a flyover timescale even using off-the-shelf processors. Our work points the way to optimised global quantum networks secured via fundamental physics. I. INTRODUCTION Continuous Variable (CV) Quantum Key Distribution (QKD) has been intensively studied and significant breakthroughs have been achieved in both theory and experiment (see [1] for review). Compared to Discrete Variable (DV) QKD [2]–[5], CV-QKD can be implemented with well-developed technologies (e.g., homodyne detectors) in commercial fibreoptic networks [6], [7] and free-space optical communications [8], [9], providing it a potential advantage in practical deployments [10]–[14]. Considering the finite-key security of CV-QKD and DV-QKD, there are three critical parameters. These are, No, the number of original quantum signals sent by the transmitter (Alice) that are collected by the receiver (Bob); Ne, the number of quantum signals from which the protocol parameters are estimated;[1] and, ǫ, the probability that a QKD protocol fails to generate secret keys [15], [16]. To satisfy an upper limit on the failure probability of parameter estimation, Alice and Bob set Ne to a large value, which in turn implies a larger No. Despite the advantages in deployment, CV-QKD systems tend to demand a larger No to reach the same ǫ relative to DVQKD protocols. For example, to achieve a final key rate of 0.1 bits per pulse with ǫ = 10−9 , a CV-QKD protocol studied in [17] required No ≈ 109 signals. However, to achieve the same final key rate with ǫ = 10−14, the DV-QKD protocol in [18] required No ≈ 104 signals. This higher number of required signals in CV-QKD can render the classical post-processing (i.e. key reconciliation and privacy amplification[2] ) slow - possibly failing to meet target timescales for reconciliation. The end-users of a CV-QKD system expect the system to deliver two identical and secure keys under a limited time interval. For example, for satellite-based deployments, we would hope that the reconciliation is completed while maintaining a line-of-sight connection with the ground station. For a CV-QKD-enabled satellite with orbital parameters similar to Micius [20], this would mean the reconciliation should be completed in less than a few minutes. For the protocol we use in this work (see later), and for ǫ = 10−9 , this, in turn, would require the data rate of reconciliation to be at least 3.6 × 106 bits per second. For real-time reconciliation (say in subsecond timescales), two orders of magnitude increases in the reconciliation rates would be required. Demands for smaller ǫ will exacerbate the issue. Ideally, the rate of reconciliation should always be faster than the rate of quantum signalling. This all raises the question as to whether current CV-QKD reconciliation schemes are optimised for the highest possible key rates in bits per second. As we show here, this is not the case. Further optimisation is possible on all current schemes. To understand the issue better, we define reconciliation in the context of CV-QKD as a two-step scheme where the inputs to the reconciliation are non-identical N = 2No − 2Ne quadrature values[3] held by Alice and Bob (after parameter estimation), and the output is an identical bit string held by Alice and Bob [21]–[23]. Assuming a reverse reconciliation scheme, Bob first converts the quadrature values encoded by Alice in each signal to m bits. Alice, after converting each of her encoded real numbers also to m bits, then initiates some discrepancy-correction algorithms based on pre-defined errorcorrection codes to ensure her mN bits are identical to Bob’s. In this work, we will adopt Low-Density Parity-Check (LDPC) codes for the error correction. However, as alluded to above, reconciling mN bits within a limited time frame can be challenging. State-of-the-art LDPCbased reconciliation schemes for CV-QKD systems involve parallelised computation on a Graphics Processing Unit (GPU) [13], [24] or Field-Programmable Gate Arrays (FPGAs) [25], [26]. Reconciliation schemes implemented on FPGAs offer more programmable flexibility, but sometimes at the cost of reduced memory access relative to GPUs. For our purposes, both hardware architectures are useful - both offer massive parallelisation opportunities. These parallelisation solutions generally take the following two-step approach: 1) The mN bits are organised as m N-bit blocks to be reconciled. Each N-bit block is divided into multiple shorter blocks of size, say, NR. This is usually just set to a block size that can be processed within some timescale. 2) Then the m NR-bit blocks are reconciled in parallel (via independent processors) using optimally-designed LDPC decoders. However, what is missing in this approach is a proper optimisation analysis as to what the optimal value of NR is. As we show below, simply reducing NR at the cost of additional processing units is not an optimal solution. It transpires that in QKD the “penalty” cost of reducing the code rate (implicit in the use of small block lengths) significantly influences the bit per second final key rate. A more sophisticated analysis is required to determine the optimal reduced block length. Such an analysis is the key contribution of this work. Although we will adopt a specific CV-QKD protocol for our analysis, the key steps of our scheme will apply to any CV-QKD protocol. Our reconciliation scheme will deliver the highest reconciliation rate for a given processor speed - thus allowing for the optimal solution to CV-QKD reconciliation. [1] More precisely, in a CV-QKD protocol, Alice and Bob randomly select a Ne-signal subset from the No signals to estimate the parameters. [2] In this work, we focus on the key reconciliation step because it is the more time-consuming part in the post-processing steps while the privacy amplification involving only bit-wise operations can be easily implemented faster than the reconciliation [19]. [3] No and Ne are multiplied by 2 since Alice and Bob utilise both quadratures from heterodyne detection - the detection process we assume in this work.

Optimised Multithreaded CV-QKD Reconciliation for Global Quantum Networks: Abstract & Introduction

This paper is available on arxiv under CC 4.0 license. Authors: (1) Samie Mostafavi, ssmos, KTH Royal Institute of Technology; (2) Vishnu Narayanan Moothedath, vnmo, KTH Royal Institute of Technology; (3) Stefan Ronngren, steron, KTH Royal Institute of Technology; (4) Neelabhro Roy, §nroy, KTH Royal Institute of Technology; (4) Gourav Prateek Sharma, gpsharma, KTH Royal Institute of Technology; (4) Sangwon Seo, sangwona, KTH Royal Institute of Technology; (4) Manuel Olgu´ın Munoz, manual@olguinmunoz.xyz, KTH Royal Institute of Technology; (4) James Gross, jamesgr, KTH Royal Institute of Technology. Table of Links Abstract & Introduction Testbed Design and Architecture Experimental Testbed Validation Supported Experimentation Acknowledgements & References Abstract This paper presents ExPECA, an edge computing and wireless communication research testbed designed to tackle two pressing challenges: comprehensive end-to-end experimentation, and high levels of experimental reproducibility. Leveraging Openstack-based Chameleon Infrastructure (CHI) framework for its proven flexibility and ease of operation, ExPECA is located in a unique, isolated underground facility, providing a highly controlled setting for wireless experiments. The testbed is engineered to facilitate integrated studies of both communication and computation, offering a diverse array of Software-Defined Radios (SDR) and Commercial Off-The-Shelf (COTS) wireless and wired links, as well as containerized computational environments. We exemplify the experimental possibilities of the testbed using OpenRTiST, a latency-sensitive, bandwidth-intensive application, and analyze its performance. Lastly, we highlight an array of research domains and experimental setups that stand to gain from ExPECA’s features, including closed-loop applications and time-sensitive networking. Index Terms—Edge computing experimental platform, reproducibility, end-to-end experimentation, wireless testbed I. INTRODUCTION As we move onto the next decade, edge computing applications are poised for significant growth, fueled by advancements in wireless technologies such as 5G and Beyond 5G (B5G), as well as breakthroughs in compute-intensive tasks like model predictive control, multimedia processing, and machine learning algorithms. In order to achieve trustworthiness and dependability in these interdisciplinary, distributed systems, involving computer science and wireless research, it is imperative to develop sophisticated scientific tools. On one hand, research in areas like wireless networking methods necessitates close interaction with radio hardware and a high degree of adaptability, as facilitated by Software Defined Radios (SDRs). Conversely, the computational aspect requires performance workloads to be adaptable to fluctuating network conditions, necessitating the creation and efficient repeatable testing of innovative computing algorithms. End-to-end experimentation is essential for obtaining a comprehensive understanding of the system as a whole and for devising innovative approaches to overcome existing bottlenecks. Additionally, the ability to efficiently replicate these experiments is key to allowing other researchers to easily validate findings. In this realm of research, we find a diverse landscape of testbeds that have emerged in recent years, each distinguished by its size, applicability, and feature set. Notable ones include the 1) COSMOS, 2) POWDER, 3) ARA, and 4) Drexel Grid SDR testbeds. COSMOS (Cloud enhanced Open Software defined MObile wireless testbed for city-Scale deployment) is a testbed spanning over 1 square mile, equipped with an array of resources, including Software-Defined Radios (SDRs), millimeter-wave (mmWave) equipment, optical fibers, and various computational nodes for core functionality and application processing [1], [2]. COSMOS integrates with cloud infrastructure, with a primary focus on advancing cloud computing research. This testbed includes a diverse set of rooftop, intermediate, and mobile nodes, all centrally controlled. POWDER (Platform for Open Wireless Data-driven Experimental Research) serves as an even larger testbed, offering a platform for exploring wireless and mobile networks with a high level of programmability that extends down to the waveform level [3]. Covering an area of 15 km2 , POWDER comprises fixed, programmable radio nodes constructed using off-the-shelf SDR technology. The platform seamlessly integrates with cloud and edge-like compute capabilities, allowing mobile nodes to tap into the compute resources. The Drexel Grid SDR testbed features SDRs that connect over-the-air (OTA), channel emulator, or alternatively a combination of the two, to enable realistic and reproducible experimentation. Even though the testbed provides freedom in terms of experiment development, it is intended primarily for SDR-related research, and does not integrate any core, cloud or edge components. ARA (The Agricultural and ruRAl communities) represents a wireless research platform, spanning a rural expanse with a diameter of 60 km in Iowa [4]. ARA’s mission is to investigate wireless platforms and technologies within the realworld context of agricultural settings. The testbed incorporates a diverse array of wireless platforms, implemented through SDRs and programmable Commercial Off-The-Shelf (COTS) radios along with automated vehicles and various cameras and other sensors. ARA’s testbed software builds upon the Chameleon Infrastructure (CHI) software framework from the Chameleon testbed project [5]. Chameleon primarily serves as a bare-metal reconfigurable research testbed designed for edge-to-cloud experimentation, adapting the cloud paradigm for computer science research. Since its public unveiling in 2015, Chameleon has accommodated over 6,000 users across more than 800 projects, gaining valuable experience and refining methods for automating and managing research infrastructure sites. This accumulated knowledge has been encapsulated in the CHI-in-a-Box [6] testbed software distribution, which aims to simplify and optimize the deployment and operation of future associated or independent Chameleon sites. Existing wireless research-focused platforms like COSMOS, POWDER, and Drexel Grid employ their own specialized software solutions, which are not in line with prevailing cloud-native standards. These testbeds also utilize virtualization technology based on Virtual Machines (VMs), rather than opting for lighter and more edge-compatible solutions such as containers. Similar to the approach taken by the ARA testbed, we found that CHI-in-a-Box presents an excellent opportunity to develop experimental platforms at minimal cost, while offering significant flexibility in both cloud and edge-native technologies. However, when it comes to reproducibility in wireless edge-computing research, existing large-scale testbeds have limitations that we aim to address by introducing our testbed, ExPECA. By ‘reproducibility’, we mean the capacity for researchers to easily replicate experiments and validate results. Achieving this is particularly challenging for two main reasons: one, edge computing experiments involve a range of heterogeneous components like servers and radios, all interacting within a single experiment; and two, the existence of wireless links introduces variables such as interference and dynamic propagation conditions that must be controlled for consistent results. Owing to its unique design choices, ExPECA provides a highly reproducible environment for endto-end wireless and edge computing experimentation, making it one of the first platforms to effectively address this issue. In the following section, we delve into these design intricacies and their consequential impacts.

ExPECA: An Experimental Platform for Trustworthy Edge Computing Applications: Abstract & Intro

Stories, News & Mentions about THESHELF

Theshelf’s Stories on HackerNoon