WRANGLD

wrangld

This interview series celebrates all the phenomenal . Congratulations! Well-Deserved, Champion.The HackerNoon community is excited to learn more about you! Startups of the Year Winners Wrangld is Startup of the Year in Richmond VA. https://hackernoon.com/startups/north-america/north-america-richmond-va-usa What does it mean for you to win this title? Being recognized as  the SOTY in Richmond VA is awesome. It’s tangible evidence of our hardworking team and meaningful work that we could get people behind us. The real win is the users we connect with, having a cool HackerNoon award to go with it is the icing on the cake! What sets you apart from the competition? At our core we are built around User Experience - it’s not just a buzzword or department - it’s a “way of life” at WRANGLD. Innovation - building a platform that not only solves problems for our industry; but delivering tools that enable and connect our users at the same time. What do you love about your team, and why are you the ones to solve this problem? Our team unites under a common vision/mission and this is reflected in the output of everyone every single day. Most of our current team has worked together at other successful startups/companies/and wanted to be a part of building something special at WRANGLD. You can’t buy or fake that kind of culture and that’s a team if you put any challenge in front of them - they’ll conquer it. If you weren’t building your startup, what would you be doing? If money didn’t matter - I’d probably be hitting up National Parks with my wife and dog and bike-packing on the Colorado Trail. At the moment, how do you measure success? What are your core metrics? We’ve built an amazing platform quickly. Seeing it gain traction and scale is the exciting part. Right now, every new user that comes on board and realizes how it can help them change their businesses is a huge metric for us. Establishing partnerships that help our mission to modernize the trades industry is another one - partners who see the big picture and want to be a part of driving that change now is another mark for us that reaffirms we’re on the right path. Tell us how your startup is changing the world. Blue Collar Jobs get overlooked and/or looked down upon by a lot of industries, especially tech. The crazy thing here is how much these companies drive our world! Consumer expectations change and industries have to rise to meet those expectations. We help the companies of the trades industry meet and exceed those modern expectations as well as streamline their operations to help them scale and grow their companies at the same time. If you’re enjoying your heat/ac, water, etc - don’t forget to thank the men and women who make our “comforts” possible! How do you or your company intend to embrace the responsibility of this title in 2024? We’re a little biased obviously to think what we’re doing here is a big deal - but holding the title as a SOTY - we want to show our users who trust us to be the backbone of their company’s operations, our partners who empower us to provide more for our users, and anyone who joins our team’s mission - there is a reason why we’re SOTY for Richmond, VA and we’re just getting started! What goals are you looking forward to accomplishing in 2024? Making WRANGLD a household name in our industry! Adding more members to the WRANGLDPack! Changing more business owners’ lives after they embrace our platform’s capabilities! Which trend(s) are you most excited about in 2024? Share your reasons. In our industry - seeing more companies modernize their operations by embracing technology. We’re pretty stoked to show off more of how we’re utilizing AI/ML to make our tools even better for our users. Being embedded partners with our Fin-Tech portion of WRANGLD Finance - we’re excited to see more contractors be able to enable their consumers with buying options that make sense for them. 2023 had been another crazy year, especially in tech, with layoffs and the Generative AI takeover! Which trend are you most concerned about? Be as brief or as detailed as you like. There’s always a new “buzzword” topic in tech that causes waves and ripples in the industry. I think AI in general is an amazing topic and is going to change the tech industry as a whole. The “concern” I have isn’t about AI taking over and ruling over us all - I think our bigger problem comes in the form of companies touting how their “AI-Powered <Insert Tool Name Here>” product is changing the industry, etc, etc, etc - and charging crazy prices because it’s the hot topic when in reality they have a simple if/else flowchart that developers could build as a fun side project. This makes the tech industry look bad and makes it harder for companies that are building cool stuff to advance when consumers become jaded from the other products that didn’t deliver. I’ll get off my soapbox on that one now… Share your biggest success so far. Hmm… That’s a tough one… Sure it’s easy to measure success by KPIs/metrics/Sales - but I’d have to say when I look at our current team and the advancements we’ve made as a company together this fast that no one else in our industry has done yet… That’s one I tip my hat to. We would love your feedback on HackerNoon as a tech publication! How has your experience been with us? So far it’s been awesome - I spend most of my time reading articles and I enjoy learning a lot from the contributors. Any words of wisdom you’d like to share with us? There are a lot of amazing quotes/books/etc on wisdom - I’ve always enjoyed the simple ones that still ring true. One that I feel applies to the startup world is from Confucius: "When it is obvious that the goals cannot be reached, don't adjust the goals; adjust the action steps." Startups of The Year is HackerNoon’s Flagship community-driven event celebrating startups that survived and thrived in 2023. 30,000 startups across 4200+ cities and six continents participated this year to be crowned the best startup in their city. See our global winners announcement . here Companies that blog more get 67% more leads - Jumpstart your company’s blogging journey by purchasing our . Business Blogging Package The 2023 Startups Of The Year is sponsored by: . . The winners will get a free .Tech Domain, a HackerNoon NFT, and a Tech Company News Page. TECH Domains

Meet WRANGLD, Startups of the Year Winner in Richmond, VA

Startups of The Year 2023 Winners: North America 

Data wrangling, also known as data munging, is the process of cleaning, transforming, and organizing raw data for further analysis and integration. In this article, I will explore the significance and benefits of data wrangling, the process of making raw data usable for different end users, and the application of data wrangling in AI and machine learning. Let’s start by taking a closer look at data wrangling and its significance in finance. With a predicted market size of , unlocking big data’s full potential is more relevant than ever. As big data continues to evolve, companies continue to find new data types. However, as technology continues to create more data sources, data management continues to become a growing obstacle for businesses. 103B in 2027 according to Statista Likewise, Forbes reported that data scientists responsible for data preparation spend approximately for further analysis. Included in this 80% is data wrangling. 80% of their work hours preparing data The importance of data wrangling Having an in-depth understanding of data wrangling is essential for businesses and investment firms, as data wrangling is a precursor to many other data transformation and analysis processes commonly utilized by financial professionals. For instance, investors can leverage job postings data to better understand market growth and track promising start-ups, and similarly, companies are able to utilize firmographic data to better understand their competition. Both objectives require significant amounts of data to be collected and then transformed for further analysis or integration. Ultimately, both of these objectives cannot be achieved smoothly without data wrangling. The benefits of data wrangling As previously mentioned, big data has become an integral part of business and finance today. However, the full potential of said data is not always clear. Data processes, such as , are useful for recognizing your data’s potential. But to fully unleash the power of your data you will need to implement data. Here are some of the key benefits of data wrangling. data discovery Data consistency The organizational aspect of data wrangling offers a resulting dataset that is more consistent. Data consistency is crucial for business operations that involve collecting data input by consumers or other human end-users. For example, if a human end-user submits personal information incorrectly, such as making a duplicate customer account, which would consequently impact further performance analysis. Improved insights Data wrangling can provide statistical insights about metadata by transforming the metadata to be more constant. These insights are often the result of increased data consistency, as consistent metadata allows automated tools to analyze the data faster and more accurately. Particularly, if one were to build a model regarding projected market performance, data wrangling would clean the metadata in a way that would allow your model to run without any errors. Cost efficiency As previously mentioned, because data wrangling allows for more efficient data analysis and model-building processes, businesses will ultimately save money in the long run. For instance, thoroughly cleaning and organizing data before sending it off for integration will reduce errors and save developers time. Now that I’ve examined the objectives, significance, and benefits of data wrangling, let’s explore the more granular details of data wrangling as well as its relation to machine learning (ML). Data wrangling formats Depending on what type of data you are using, your final result will fall into one of four final formats: de-normalized transactions, analytical base table (ABT), time series, or document library. Let’s take a closer look at these final formats, as understanding these results will inform the first few steps of the data-wrangling process, which I will discuss later. Transactional data Transactional data refers to business operation transactions. This data type involves detailed subjective information about particular transactions, including client documentation, client interactions, receipts, and notes regarding any external transactions. Analytical Base Table (ABT) Analytical Base Table data involves data within a table with unique entries for each attribute column. ABT data is the most common business data type as it involves a variety of data types that contribute to the most common data sources. Even more notable is that ABT data is primarily used for AI and ML, which I will examine later on. Time series Time series data involves data that has been divided by a particular amount of time or data that has a relation with time, particularly sequential time. For example, tracking data regarding an application’s downloads over a year or tracking traffic data over a month would be considered time series data. Document library Lastly, document library data is information that involves a large amount of textual data, particularly text within a document. While document libraries contain rather massive amounts of data, automated data mining tools specifically designed for text mining can help extract entire texts from documents for further analysis. Data wrangling tools There are various data-wrangling methods that data analysts and other professionals use, ranging from munging data with scripts to using spreadsheets. Additionally, with some of the more recent all-in-one tools, everyone utilizing the data can access and utilize their data-wrangling tools. There also are visual data-wrangling tools for beginners that make it easier for people without programming skills. Here are some of the more common data-wrangling tools available: Excel is a simple yet powerful data-wrangling tool; Plotly (data wrangling with Python) is useful for maps and chart data; CSVKit converts data; Tabula is regarded as an all-in-one munging solution; Google DataPrep cleans, prepares, and organizes data. The data-wrangling process 1. Discovery Before you can start the wrangling process, it is critical to think about what may lie beneath your data. That is to say, it is crucial to think critically about what results you are anticipating from your data and what you will use your data for once the wrangling process is complete. Once you’ve determined your objectives, you can gather your data. 2. Organization After you’ve gathered your raw data within a particular dataset, you must structure your data. Due to the variety and complexity of data types and sources, raw data is often overwhelming to look at at first glance. 3. Cleaning When your data is organized, you can begin cleaning your data. Data cleaning involves removing outliers, formatting nulls, and eliminating duplicate data. It is important to note that cleaning data collected from web scraping methods might be a much more tedious process than cleaning data collected from a database. Essentially, web data can be highly unstructured and require much more time than structured data from a database. 4. Data enrichment This step requires that you take a step back from your data to determine if you have enough data to proceed. Finishing the wrangling process without enough data may compromise insights gathered from further analysis. For example, investors looking to analyze product review data will want a significant amount of data, accurately portraying the market, and in turn, increasing investment intelligence. 5. Validation After determining you gathered enough data, you will need to apply validation rules to your data. Validation rules, performed in repetitive sequences, confirm that your data is consistent throughout your dataset. Validation rules will also ensure data quality as well as security. This step follows similar logic utilized in data normalization, a data standardization process involving validation rules. 6. Publishing The final step of the data munging process is data publishing. Data publishing involves preparing the data for future use. This may include providing notes and documentation of your wrangling process and creating access for other users and applications. It is also important to note that, like many other data transformation processes, data wrangling is an iterative process requiring you to revisit your data regularly. To better understand the munging process, let’s take a look at data mining, a subset of data wrangling. Data mining versus data wrangling At first glance, data mining and data wrangling appear nearly identical, especially if you are not a data scientist. But as previously mentioned, data mining is actually a subset of data wrangling. While they have similar benefits, they both achieve different data processing objectives. Data mining is defined as the process of sifting and sorting through data to find patterns and hidden relationships in larger datasets. Whereas, data wrangling requires a few more steps, such as cleaning, enriching, and integration, transforming raw data for deliverable insights. Additionally, data wrangling’s ability to remove inadequate data from the dataset being mined. Ultimately, data munging enhances the data mining process, providing companies with powerful insights and hidden patterns regarding customer behavior, market trends, and product feedback. Data wrangling and machine learning Data wrangling is a crucial component of machine learning. More specifically, data munging analyzes data that feeds into the machine learning model you are building. Without data munging, the model wouldn’t have clean data to process, and therefore the model would produce false predictions, failing its objective. Likewise, as the AI software market increases, with a reported , which is still growing today, machine learning has become increasingly prevalent in the world of finance. However, because data wrangling has yet to be 100% automated, machine learning has yet to reach its fullest potential. market revenue of 10.1B in 2018 The complexity of metadata management as well as the sheer amount of processing power it takes to wrangle data properly are the main contributors to this machine-learning roadblock. Consequently, because machine learning requires extensive data wrangling, companies are eager to find the highest-quality automated data wrangling tools. Closing thoughts Ultimately, data wrangling is an excellent data preparation method for businesses looking to utilize AI, machine learning, and automated data analysis tools or data analytics processes, among other applications. And as data science continues to evolve, data-wrangling tools will look to AI for more solutions. Also published here.

The Benefits And Core Processes of Data Wrangling

Have you ever console logged the console So, you know the good ol’ console.log() Well, that’s not the only thing the console can do, it’s only the most basic. Have you ever used the console to log the console? No? You should try it. The console will show you what the console can do. Today we’ll talk about two of the functions available on the console object: and console.table() console.trace() Let’s say we have two objects: const carA = {make: 'Ford', model: 'F150', year: '2018', color: 'white'}
const carB = {make: 'Ford', model: 'F250', year: '2016', color: 'blue'} If we wanted to do some debugging and had to print these objects out to the console we could use but it won’t look too nice. console.log() Look how neat can display the two objects: console.table() You can even sort the two rows by column just by clicking on the headers! One other console function we’ll talk about today is . This will output a stack trace to the console. console.trace() And that’s all for today. I hope I’ve helped make your debugging a little less boring. Now go show off your new console skills in your next stand-up. Cheers!

Making Debugging Easy With Console Wrangling

Code monkeys are notoriously strong-willed, and can be particularly hard to break. Here, we’ll outline a few ways in which you can whip your code monkeys into shape. First things first, don’t trust them as far as you can throw them. Code monkeys will go off on a lark at the first opportunity, not returning for days on end as you wonder what’s going on. Oftentimes, they’ll come back with some godawful interface for doing some obscure thing you don’t care about, something that has nothing to do with your bottom line. Instead, wrangle those monkeys! Code monkeys crave structure, like the structure of the code they write. Provide that structure in terms of strictly and explicitly laid-out tasks, with no room for creativity or self-expression. Next, you’ll have to worry about the code monkeys talking amongst themselves, becoming dissatisfied with the lack of freedom to do as they please. They may talk about restructuring, “agile” practices, or, god forbid...unionization! You can dissuade this by making sure the code monkeys are scrambling so frantically they have no real time to self-organize. Have them firefighting so early and so often that they have no chance to even dream of something better. Now that you’ve knocked your monkeys down a notch or two, they may get the notion that they should be involved in more than code. Avoid this at all costs! Code monkeys are no good at anything but code. Their design concepts are butt-ugly, their product ideas ludicrous and far-fetched. Stamp out this behavior, and keep your teams completely isolated from one another. Don’t let design talk to the dev team! Instead, have them lob designs over the wall to the dev team. Allow zero communication. Communication facilitates unionization! Whether formal or informal, we can’t have that. Speaking of communication: let them know who’s boss. A code monkey can’t think of you as an equal! This would lead to all sorts of hippy dippy garbage like “free expression” and “equality”. Shut that down with a hard-nosed expression of authority. Demand that your pet project be done in the time you allotted! You may have code monkeys quit on you, because of your bold, assured stance. This is simply separating the wheat from the chaff. Eventually, you’ll have a cadre of chimps begging you for their next order, like a well-oiled military machine. Heed these words, O Executive, and you will be amazed at the results!

How to Wrangle Your Code Monkeys

Introduction Deep learning and Machine learning are becoming more and more important in today’s ERP (Enterprise Resource Planning). During the process of building the analytical model using Deep Learning or Machine Learning the data set is collected from various sources such as a file, database, sensors and much more. But, the collected data cannot be used directly for performing analysis process. Therefore, to solve this problem Data Preparation is done. This includes two techniques that are listed below: Data Preprocessing Data Wrangling Data Preparation is an important part of . It includes two concepts such as Data Cleaning and Feature Engineering. These two are compulsory for achieving better accuracy and performance in the and projects. Data Science Machine Learning Deep Learning Data Preprocessing Data Preprocessing is a technique that is used to convert the raw data into a clean data set. In other words, whenever the data is gathered from different sources it is collected in raw format which is not feasible for the analysis. Therefore, certain steps are executed to convert the data into a tiny clean dataset. This technique is performed before the execution of Iterative Analysis. The set of steps is known as Data Preprocessing. This includes Data Cleaning, Data Integration, Data Transformation and Data Reduction. Data Wrangling Data Wrangling is a technique that is executed at the time of making an interactive model. In other words, it is used to convert the raw data into the format that is convenient for the consumption of data. This technique is also known as . This method also follows certain steps such as after extracting the data from different data sources, sorting of data using certain algorithm is performed, decompose the data into a different structured format and finally store the data into another database. Data Munging Need of Data Preprocessing For achieving better results from the applied model in Machine Learning and Deep Learning projects the format of the data has to be in a proper manner. Some specified Machine Learning and Deep Learning model need data in a specified format, for example, Random Forest algorithm does not support null values, therefore to execute random forest algorithm null values has to be managed from the original raw data set. Another aspect is that dataset should be formatted in such a way that more than one Machine Learning and Deep Learning algorithms are executed in one dataset and best out of them is chosen. Need of Data Wrangling Data Wrangling is an important aspect for implementing the model. Therefore, data is converted to the proper feasible format before applying any model intro it. By performing filtering, grouping and selecting appropriate data accuracy and performance of the model could be increased. Another concept is that when time series data has to be handled every algorithm is executed with different aspects. Therefore Data Wrangling is used to convert the time series data into the required format of the applied model. In simple words, the complex data is converted into a usable format for performing analysis into it. Why is Data Preprocessing used? Data Preprocessing is necessary because of the presence of unformatted real world data. Mostly real world data is composed of - There are many reasons for missing data such as data is not continuously collected, a mistake in data entry, technical problems with biometrics and much more. Inaccurate data (missing data) — — The reasons for the presence of noisy data could be a technological problem of gadget that gathers data, a human mistake during data entry and much more. The presence of noisy data (erroneous data and outliers) The presence of inconsistencies is due to the reasons such that existence of duplication within data, human data entry, containing mistakes in codes or names i.e. violation of data constraints and much more. Inconsistent data — Therefore, to handle raw data, Data Preprocessing is performed. Why is Data Wrangling used? Data Wrangling is used to handle the issue of Data Leakage while implementing Machine Learning and Deep Learning. First of all, we have to understand what is Data Leakage? What is Data Leakage in Machine Learning/Deep Learning? Data Leakage is responsible for the cause of invalid Machine Learning/Deep Learning model due to the over optimization of the applied model. Data Leakage is the term used when the data from outside i.e. not part of training dataset is used for the learning process of the model. This additional learning of information by the applied model will disapprove the computed estimated performance of the model. For example when we want to use the specific feature for performing Predictive Analysis but that specific feature is not present at the time of training of dataset then data leakage will be introduced within the model. Data Leakage can be demonstrated in many ways that are given below: Leakage of data from test dataset to training dataset. Leakage of computed correct prediction to the training dataset. Leakage of future data into the past data. Usage of data outside the scope of applied algorithm In general, the leakage of data is observed from two main sources of Machine Learning/Deep Learning algorithms such as feature attributes (variables) and training dataset. How to check the presence of Data Leakage within the applied model? Data Leakage is observed at the time of usage of complex datasets. They are described below: At the time of dividing time series dataset into training and test, the dataset is a complex problem. Implementation of sampling in a graphical problem is a complex task. Storage of analog observations in the form of audios and images in separate files having a defined size and timestamp. How is Data Preprocessing performed? Data Preprocessing is performed to remove the cause of unformatted real world data which are discussed above. First of all, let’s discuss how missing data can be handled. There are three different steps that can be executed which are given below: — It is the simplest and effective method for handling the missing data. But, this method should not be performed at the time when the number of missing values are huge or when the pattern of data is related to the unrecognized basic root of the cause of statement problem. Ignoring the missing record — This is one of the best-chosen methods. But there is one limitation that when there is large dataset and missing values are large then, this method is not efficient as it becomes a time-consuming task. Filling the missing values manually — The missing values can also be filled by computing mean, mode or median of the observed given values. Another method could be the predictive values that are computed by using any Machine Learning or Deep Learning algorithm. But one drawback of this method is that it can generate bias within the data as the computed values are not accurate with respect to the observed values. Filling using computed values Let’s move further and discuss how we can deal with the noisy data. The methods that can be followed are given below: — In this method sorting of data is performed with respect to the values of the neighborhood. This method is also known as local smoothing. Binning method — In the approach, the outliers may be detected by grouping the similar data in the same group i.e. in the same cluster. Clustering method — A Machine Learning algorithm can be executed for smoothing of data. For example, regression algorithm can be used for smoothing of data using a specified linear function. Machine Learning — The noisy data can be removed manually by the human being but it is a time-consuming process so mostly this method is not given priority. Removing manually To deal with the manually the data is managed using external references and knowledge engineering tools like knowledge engineering process. inconsistent data How is Data Wrangling performed? Data Wrangling is performed to minimize the effect of Data Leakage while executing the model. In other words if one consider the complete dataset for normalization and standardization, then the cross-validation is performed for the estimation of the performance of the model leads to the beginning of data leakage. Another problem is also observed that the test model is also included for feature selection while executing each fold of cross-validation which further generates bias during performance analysis. The effect of Data Leakage could be minimized by recalculating for the required Data Preparation during the cross-validation process that includes feature selection, outliers detection, and removal, projection methods, scaling of selected features and much more. Another solution is that dividing the complete dataset into training dataset that is used to train the model and validation dataset which is used to evaluate the performance and accuracy of the applied model. But, the selection of the model is made by looking at the results of test dataset in the cross validation process. This conclusion will not always be true as the sample of test dataset could vary and the performance of different models are evaluated for the particular type of test dataset. Therefore, while selecting best model test error is overfitting. To solve this problem, the variance of the test error is determined by using different samples of test dataset. In this way, the best suitable model is chosen. Difference Between Data Preprocessing and Data Wrangling Data Preprocessing is performed before Data Wrangling. In this case, Data Preprocessing data is prepared exactly after receiving the data from the data source. In this initial transformations, data cleaning or any aggregation of data is performed. It is executed once. For example, we have a data where one attribute have three variables and we have to convert them into three attributes and delete the special characters from them. This is the concept that is performed before applying any iterative model and will be executed once in the project. On the other hand, Data Wrangling is performed during the iterative analysis and model building. This concept at the time of feature engineering. The conceptual view of the dataset changes as different models is applied to achieve good analytic model. For example, we have data containing 30 attributes where two attributes are used to compute another attribute and that computed feature is used for further analysis. In this way, the data could be changed according to the requirement of the applied model. Tasks of Data Preprocessing Different steps are involved for Data Preprocessing. These steps are described below: — This is the first step which is implemented in Data Preprocessing. In this step, the main focus is on handling missing data, noisy data, detection, and removal of outliers, minimizing duplication and computed biases within the data. Data Cleaning — This process is used when data is gathered from various data sources and data is combined together to form consistent data. This consistent data after performing data cleaning is used for analysis. Data Integration — This step is used to convert the raw data into a specified format according to the need of the model. The options used for transformation of data are given below: Data Transformation In this method, numerical data is converted into specified range i.e. between 0 and 1 so that scaling of data can be performed. Normalization — The concept can be derived from the word itself, this method is used to combine the features into one. For example combining two categories can be used to form a new category. Aggregation — — In this case, lower level attributes are converted into a higher level. Generalization — After the transformation and scaling of data duplication i.e. redundancy within the data is removed and organize the data in an efficient manner. Data Reduction Tasks of Data Wrangling The tasks of Data wrangling are described below - — Firstly, data should be understood thoroughly and examine which approach will best suit. For example: if have a weather data when we examine the data it is observed that data is from one area and so main focus is on determining patterns. Discovering As the data is gathered from different sources, the data will be present in different shapes and sizes. Therefore, there is a need of structuring the data in proper format. Structuring — Cleaning or removing of data should be performed that can degrade the performance of analysis. Cleaning — — Extract new features or data from the given dataset in order to optimize the performance of the applied model. Enrichment — This approach is used for improving the quality of data and consistency rules so that transformations that are applied to the data could be verified. Validating — After completing the steps of Data Wrangling, the steps can be documented so that similar steps can be performed for similar kind of data to save time. Publishing Continue Reading How Data Wrangling Improves Data Analytics

Data Preprocessing and Data Wrangling in Machine Learning and Deep Learning

The Generative AI industry has exploded, thanks to no small part to the release of ChatGPT in the fall of 2022, which took the world by storm. There has been a lot of hand-wrangling on the importance of model safety and data security, with the possibility of further AI regulations. As per IBM, around 70% of their survey respondents say that innovation is more important than security, which contrasts with 82% of the respondents accepting that security in AI is essential to their business. Unsurprisingly, only 24% of the projects polled in the IBM survey have components meant to secure the initiative. Security is not an add-on to thoroughly flushed-out Generative AI infrastructure. It should be weaved in and through the entire infrastructure rather than slapped on top of it to pass regulations and compliance. The immense power of Generative AI brings with it vast responsibilities. This article aims to convince readers of the importance of security in AI and provide them with the knowledge and talking points to become security champions in their organizations. Objective The objective of GenAI security should be to secure the entire infrastructure from end to end. As security champions, we must secure data ingestion infrastructure, ensuring data legitimacy, data manipulation operations, model deployment, and user interactions. For the last year, people have been complaining about the degrading quality of answers provided by ChatGPT, whereas OpenAI has refuted such claims. While it is true that the quality of responses can deteriorate over time, it is entirely possible that an internally deployed model can be modified in an unauthorized manner by bad actors. Such responses can be indistinguishable from the typical quality degradation we expect. It goes without saying that customer data, which consists of their prompt history and responses, should be considered critical data, and we need to examine any risks that might seriously compromise users' trust. While the conversations are between a human and a machine, the personal nature of such discussions should be treated just like human-to-human conversations and secured as such. Risks There is no limit to the kind of risks one has to deal with when working on GenAI infrastructure. While we are looking into the typical security best practices from traditional security paradigms, there are AI-specific steps we need to take to ensure that we achieve a reasonable security posture. Data Poisoning: We have heard the old adage of garbage-in-garbage-out. If malicious actors manage to introduce malicious data into the corpus, the responses might not be what we expect. Imagine your company analyzes data and signals intelligence for your customers. If the data was poisoned by modification, deletion, or addition, then the results are going to be far from accurate. In fact, properly crafted training data can lead to wrong responses from the AI model. Model Manipulation: While this sounds far-fetched, the National Institute of Standards and Technology has outlined how different types of cyberattacks manipulate the behavior of AI systems, focusing on evasion and privacy attacks. Model manipulation is closely tied to data poisoning; in practice, it’s hard to differentiate one from the other. Prompt Injection Attacks: This attack vector is unique to Generation AI and is not in traditional cybersecurity practices. While a lot of research has gone into understanding the reasoning pathway of the models, for all practical purposes, they do remain a black box. Attackers can bypass safety controls by specially crafting input prompts, which lead to unauthorized responses. These responses can contain confidential information or reveal some inner workings in the worst case. These attacks work because they exploit the very nature of Generative AI models, which deal with understanding natural languages. Supply Chain Attacks: Generative AI is nascent and involves integrating various software components, sometimes with little regard to security, to beat the competitors in the field of innovation. Each third-party library or component introduces another point of attack. Even malicious or modified data can fit under the umbrella of supply chain attacks. Refer to Open Worldwide Application Security Project (OWASP) LLM05: Supply Chain Vulnerabilities for more information. Approaches There isn’t a single way to handle security in the Generative AI space. Any attempts at creating a formal process run afoul of the dreaded checklist-based security compliance processes. At its core, security is not about processes and technology but a mindset and the culture that promotes it. Fundamentally, to secure Generative AI infrastructure, one needs a secure-by-design architecture. This might include, but is not limited to, access control, the least privilege model, encryption at rest and transit, and threat/anomaly detection systems. The architecture keeps evolving based on new research and changing business needs. Changes to architecture can increase the attack surface. To avoid this, there should be some form of operational review and annual security overview. One needs to have a culture where people are proactive about reviewing the changes and bold enough to discuss the possible threats. Supply chain threats are more complex to prevent, but digitally signing all components and pieces and validation before deployment can be a good start. A little more involved process may be using Software Bill of Materials tracking, which is recommended by the Cybersecurity and Infrastructure Security Agency (CISA) Conclusion The job of a CISO is becoming increasingly complex and involved. Apart from the increased DDoS attacks as reported by Cloudflare, which is 20% year-over-year, the addition of Generative AI to the broader organization infrastructure adds more moving parts to an already complex setup. While moving ahead with innovation and sidelining security might look tempting at first, it is equivalent to playing with fire. One wrong move can result in disastrous results, whether lost customer trust, personal data leak, or government-imposed penalties. An individual must be someone other than a CISO to push for a culture shift. Often, executives rely on people reporting to them to give them advice, which in turn is asked down the reporting chain. If Engineers or program Managers can make themselves security champions, they have a high chance of shining and access to opportunities for bigger responsibilities and a career boost. The future of Generative AI hinges on our ability and willingness to secure it. Organizations that recognize this fact and act proactively will be best positioned to thrive and overcome dangers and regulatory hurdles. The Generative AI industry has exploded, thanks to no small part to the release of ChatGPT in the fall of 2022, which took the world by storm. There has been a lot of hand-wrangling on the importance of model safety and data security, with the possibility of further AI regulations . further AI regulations further AI regulations As per IBM, around 70% of their survey respondents say that innovation is more important than security, which contrasts with 82% of the respondents accepting that security in AI is essential to their business. Unsurprisingly, only 24% of the projects polled in the IBM survey have components meant to secure the initiative. Security is not an add-on to thoroughly flushed-out Generative AI infrastructure. It should be weaved in and through the entire infrastructure rather than slapped on top of it to pass regulations and compliance. The immense power of Generative AI brings with it vast responsibilities. This article aims to convince readers of the importance of security in AI and provide them with the knowledge and talking points to become security champions in their organizations. Objective The objective of GenAI security should be to secure the entire infrastructure from end to end. As security champions, we must secure data ingestion infrastructure, ensuring data legitimacy, data manipulation operations, model deployment, and user interactions. For the last year, people have been complaining about the degrading quality of answers provided by ChatGPT , whereas OpenAI has refuted such claims. While it is true that the quality of responses can deteriorate over time , it is entirely possible that an internally deployed model can be modified in an unauthorized manner by bad actors. Such responses can be indistinguishable from the typical quality degradation we expect. degrading quality of answers provided by ChatGPT degrading quality of answers provided by ChatGPT quality of responses can deteriorate over time quality of responses can deteriorate over time It goes without saying that customer data, which consists of their prompt history and responses, should be considered critical data, and we need to examine any risks that might seriously compromise users' trust. While the conversations are between a human and a machine, the personal nature of such discussions should be treated just like human-to-human conversations and secured as such. Risks There is no limit to the kind of risks one has to deal with when working on GenAI infrastructure. While we are looking into the typical security best practices from traditional security paradigms, there are AI-specific steps we need to take to ensure that we achieve a reasonable security posture. Data Poisoning : We have heard the old adage of garbage-in-garbage-out. If malicious actors manage to introduce malicious data into the corpus, the responses might not be what we expect. Imagine your company analyzes data and signals intelligence for your customers. If the data was poisoned by modification, deletion, or addition, then the results are going to be far from accurate. In fact, properly crafted training data can lead to wrong responses from the AI model. Data Poisoning Model Manipulation : While this sounds far-fetched, the National Institute of Standards and Technology has outlined how different types of cyberattacks manipulate the behavior of AI systems , focusing on evasion and privacy attacks. Model manipulation is closely tied to data poisoning; in practice, it’s hard to differentiate one from the other. Model Manipulation different types of cyberattacks manipulate the behavior of AI systems different types of cyberattacks manipulate the behavior of AI systems Prompt Injection Attacks : This attack vector is unique to Generation AI and is not in traditional cybersecurity practices. While a lot of research has gone into understanding the reasoning pathway of the models, for all practical purposes, they do remain a black box. Attackers can bypass safety controls by specially crafting input prompts , which lead to unauthorized responses. Prompt Injection Attacks specially crafting input prompts specially crafting input prompts These responses can contain confidential information or reveal some inner workings in the worst case. These attacks work because they exploit the very nature of Generative AI models, which deal with understanding natural languages. Supply Chain Attacks : Generative AI is nascent and involves integrating various software components, sometimes with little regard to security, to beat the competitors in the field of innovation. Each third-party library or component introduces another point of attack. Supply Chain Attacks Even malicious or modified data can fit under the umbrella of supply chain attacks. Refer to Open Worldwide Application Security Project (OWASP) LLM05: Supply Chain Vulnerabilities for more information. Open Worldwide Application Security Project (OWASP) LLM05: Supply Chain Vulnerabilities Open Worldwide Application Security Project (OWASP) LLM05: Supply Chain Vulnerabilities Approaches There isn’t a single way to handle security in the Generative AI space. Any attempts at creating a formal process run afoul of the dreaded checklist-based security compliance processes. At its core, security is not about processes and technology but a mindset and the culture that promotes it. Fundamentally, to secure Generative AI infrastructure, one needs a secure-by-design architecture. This might include, but is not limited to, access control, the least privilege model, encryption at rest and transit, and threat/anomaly detection systems. The architecture keeps evolving based on new research and changing business needs. Changes to architecture can increase the attack surface. To avoid this, there should be some form of operational review and annual security overview. One needs to have a culture where people are proactive about reviewing the changes and bold enough to discuss the possible threats. Supply chain threats are more complex to prevent, but digitally signing all components and pieces and validation before deployment can be a good start. A little more involved process may be using Software Bill of Materials tracking, which is recommended by the Cybersecurity and Infrastructure Security Agency (CISA) recommended by the Cybersecurity and Infrastructure Security Agency (CISA) recommended by the Cybersecurity and Infrastructure Security Agency (CISA) Conclusion The job of a CISO is becoming increasingly complex and involved. Apart from the increased DDoS attacks as reported by Cloudflare , which is 20% year-over-year, the addition of Generative AI to the broader organization infrastructure adds more moving parts to an already complex setup. While moving ahead with innovation and sidelining security might look tempting at first, it is equivalent to playing with fire. One wrong move can result in disastrous results, whether lost customer trust, personal data leak, or government-imposed penalties. reported by Cloudflare reported by Cloudflare An individual must be someone other than a CISO to push for a culture shift. Often, executives rely on people reporting to them to give them advice, which in turn is asked down the reporting chain. If Engineers or program Managers can make themselves security champions, they have a high chance of shining and access to opportunities for bigger responsibilities and a career boost. The future of Generative AI hinges on our ability and willingness to secure it. Organizations that recognize this fact and act proactively will be best positioned to thrive and overcome dangers and regulatory hurdles.

Security in Generative AI Infrastructure Is of Critical Importance

These days, every company seems eager to fill a "data scientist" role, promising exciting opportunities to work with machine learning algorithms, predictive models, and deep learning frameworks. However, for many professionals who step into these positions, reality doesn’t quite match the allure. Instead of diving headfirst into AI or modeling complex data sets, they find themselves knee-deep in data extraction, cleaning, and preparation. Welcome to the world of data engineering—a domain many didn’t realize they had signed up for. This phenomenon stems from a fundamental misunderstanding by companies of what they actually need. They post job listings for "data scientists" when the bulk of their work involves cleaning data and ensuring infrastructure is in place to handle it—quintessentially data engineering tasks. The result is that professionals hired as data scientists end up doing the grunt work they didn’t expect: wrangling messy data, moving it between platforms, and preparing it for analysis. Disillusionment inevitably sets in for those who expected to spend their days building machine learning models, not writing SQL queries and setting up pipelines. For aspiring data engineers, this is a hidden opportunity. While the job market is full of companies looking for data scientists, many of these organizations need a data engineer far more than they realize. The two fields require overlapping skills, particularly in the early stages—programming, database management, and some basic statistical knowledge. However, the tasks and career paths diverge quickly. Data scientists focus on deriving insights and making predictions, whereas data engineers ensure that the data ecosystem is robust and reliable. A savvy professional can start in a data science position and pivot into a data engineering career simply by stepping up to tackle the tasks others consider beneath them. Data scientists, especially those from highly academic backgrounds, often see data cleaning and preparation as tedious. For them, this is the “boring” side of the job—the grunt work that gets in the way of more glamorous tasks like building predictive models or applying cutting-edge algorithms. Yet, without well-structured data, those algorithms are useless. Data engineers know this well and embrace the challenge of building the frameworks that data scientists rely on. From automating the extraction and transformation of data to constructing pipelines that deliver clean, well-organized datasets, these tasks are the bread and butter of data engineering. While some data scientists struggle to extract meaning from messy datasets, data engineers are busy building scalable systems that will save time and frustration down the line. Instead of wrestling with CSV files and complaining about SQL, the aspiring data engineer uses these tools to their advantage. They streamline processes, automate data preparation tasks, and implement robust pipelines that allow for real-time or scheduled data updates. They aren’t just moving data around; they’re building the backbone of the data ecosystem. By the time data scientists finish manually preparing their datasets, the data engineer has already automated the process, eliminating repetitive work and freeing up time for more strategic tasks. This disconnect between job titles and job functions can create friction within teams, with some data scientists lamenting the lack of "real" data science work in their roles. But for the data engineer, this is where they thrive. While their peers debate which machine learning framework is superior, data engineers are busy implementing production-grade solutions, moving beyond ad-hoc analyses to create systems that deliver value repeatedly. They are the unsung heroes of the data world, quietly ensuring that data flows seamlessly, insights are generated efficiently, and the organization runs smoothly. Moreover, data engineers are uniquely positioned to bridge the gap between data scientists and other business units. Once the “hard part” of data preparation is complete, they can create accessible, user-friendly applications for non-technical stakeholders. These could be dashboards, visualization tools, or web-based platforms that democratize data insights across the organization. While the data scientists are still polishing their Python scripts, the data engineer has already built something scalable, sustainable, and usable. Ultimately, this dynamic reveals a deeper truth: many companies don’t need data scientists as urgently as they think. What they really need are data engineers who can ensure their data is structured, clean, and accessible. The insights, predictions, and models that data scientists produce are only as good as the underlying data infrastructure. So while some may continue to argue over who qualifies as a “real” data scientist, data engineers know that it’s not about the title—it’s about getting the job done. If you’re an aspiring data engineer, this path could be your golden opportunity. By stepping into these misclassified data science roles, you can quietly build a career around solving the problems that others don’t want to touch. You can automate workflows, streamline processes, and ensure that the organization’s data infrastructure is solid and scalable. While your colleagues focus on tweaking their models, you’ll be building systems that bring real value to the company, and you’ll likely go unnoticed—until it becomes clear just how much the organization relies on the work you’ve done. In the end, data engineers are the ones who make data science possible. And for those willing to embrace the challenge, the rewards can be substantial—not only in terms of career growth but in the knowledge that you’re the one quietly keeping the data-driven machine running. About Me: 25+ year IT veteran combining data, AI, risk management, strategy, and education. 4x global hackathon winner and social impact from data advocate. Currently working to jumpstart the AI workforce in the Philippines. Learn more about me here: https://docligot.com These days, every company seems eager to fill a "data scientist" role, promising exciting opportunities to work with machine learning algorithms, predictive models, and deep learning frameworks. However, for many professionals who step into these positions, reality doesn’t quite match the allure. Instead of diving headfirst into AI or modeling complex data sets, they find themselves knee-deep in data extraction, cleaning, and preparation. Welcome to the world of data engineering—a domain many didn’t realize they had signed up for. This phenomenon stems from a fundamental misunderstanding by companies of what they actually need. They post job listings for "data scientists" when the bulk of their work involves cleaning data and ensuring infrastructure is in place to handle it—quintessentially data engineering tasks. The result is that professionals hired as data scientists end up doing the grunt work they didn’t expect: wrangling messy data, moving it between platforms, and preparing it for analysis. Disillusionment inevitably sets in for those who expected to spend their days building machine learning models, not writing SQL queries and setting up pipelines. For aspiring data engineers, this is a hidden opportunity. While the job market is full of companies looking for data scientists, many of these organizations need a data engineer far more than they realize. The two fields require overlapping skills, particularly in the early stages—programming, database management, and some basic statistical knowledge. However, the tasks and career paths diverge quickly. Data scientists focus on deriving insights and making predictions, whereas data engineers ensure that the data ecosystem is robust and reliable. A savvy professional can start in a data science position and pivot into a data engineering career simply by stepping up to tackle the tasks others consider beneath them. Data scientists, especially those from highly academic backgrounds, often see data cleaning and preparation as tedious . For them, this is the “boring” side of the job—the grunt work that gets in the way of more glamorous tasks like building predictive models or applying cutting-edge algorithms. Yet, without well-structured data, those algorithms are useless. Data engineers know this well and embrace the challenge of building the frameworks that data scientists rely on. From automating the extraction and transformation of data to constructing pipelines that deliver clean, well-organized datasets, these tasks are the bread and butter of data engineering. see data cleaning and preparation as tedious While some data scientists struggle to extract meaning from messy datasets, data engineers are busy building scalable systems that will save time and frustration down the line. Instead of wrestling with CSV files and complaining about SQL, the aspiring data engineer uses these tools to their advantage. They streamline processes, automate data preparation tasks, and implement robust pipelines that allow for real-time or scheduled data updates. They aren’t just moving data around; they’re building the backbone of the data ecosystem. By the time data scientists finish manually preparing their datasets, the data engineer has already automated the process, eliminating repetitive work and freeing up time for more strategic tasks. This disconnect between job titles and job functions can create friction within teams, with some data scientists lamenting the lack of "real" data science work in their roles. But for the data engineer, this is where they thrive. While their peers debate which machine learning framework is superior, data engineers are busy implementing production-grade solutions, moving beyond ad-hoc analyses to create systems that deliver value repeatedly. They are the unsung heroes of the data world, quietly ensuring that data flows seamlessly, insights are generated efficiently, and the organization runs smoothly. Moreover, data engineers are uniquely positioned to bridge the gap between data scientists and other business units. Once the “hard part” of data preparation is complete, they can create accessible, user-friendly applications for non-technical stakeholders. These could be dashboards, visualization tools, or web-based platforms that democratize data insights across the organization. While the data scientists are still polishing their Python scripts, the data engineer has already built something scalable, sustainable, and usable. Ultimately, this dynamic reveals a deeper truth: many companies don’t need data scientists as urgently as they think . What they really need are data engineers who can ensure their data is structured, clean, and accessible. The insights, predictions, and models that data scientists produce are only as good as the underlying data infrastructure. So while some may continue to argue over who qualifies as a “real” data scientist, data engineers know that it’s not about the title—it’s about getting the job done. don’t need data scientists as urgently as they think If you’re an aspiring data engineer, this path could be your golden opportunity. By stepping into these misclassified data science roles, you can quietly build a career around solving the problems that others don’t want to touch. You can automate workflows, streamline processes, and ensure that the organization’s data infrastructure is solid and scalable. While your colleagues focus on tweaking their models, you’ll be building systems that bring real value to the company, and you’ll likely go unnoticed—until it becomes clear just how much the organization relies on the work you’ve done. In the end, data engineers are the ones who make data science possible. And for those willing to embrace the challenge, the rewards can be substantial—not only in terms of career growth but in the knowledge that you’re the one quietly keeping the data-driven machine running. About Me: 25+ year IT veteran combining data, AI, risk management, strategy, and education. 4x global hackathon winner and social impact from data advocate. Currently working to jumpstart the AI workforce in the Philippines. Learn more about me here: https://docligot.com https://docligot.com

Why Many Data Science Jobs Are Actually Data Engineering

Защо много работни места в науката за данни всъщност са инженерство на данни

Γιατί πολλές θέσεις εργασίας στην επιστήμη δεδομένων είναι στην πραγματικότητα μηχανική δεδομένων

Por qué muchos trabajos de ciencia de datos son en realidad ingeniería de datos

Miksi monet tietotekniikan työpaikat ovat itse asiassa tietotekniikkaa?

多くのデータサイエンスの仕事が実はデータエンジニアリングである理由

Pse shumë punë në shkencën e të dhënave janë në të vërtetë inxhinieri e të dhënave

Зашто су многи послови науке о подацима заправо инжењеринг података

Kwanini Kazi Nyingi za Sayansi ya Data Kwa Kweli Ni Uhandisi wa Data

ஏன் பல தரவு அறிவியல் வேலைகள் உண்மையில் தரவு பொறியியல்

ስለምንታይ ብዙሓት ናይ ዳታ ሳይንስ ስራሕቲ ብሓቂ ዳታ ምህንድስና ዝኾኑ

Nima uchun ma'lumotlar fanining ko'p ishlari aslida ma'lumotlar muhandisligidir

为什么许多数据科学工作实际上是数据工程

Introduction Building generative AI requires working with a variety of developing machine learning and AI technologies. It also requires buckets of patience. These two requirements are not unrelated. Through working alongside extremely talented machine learning experts from different firms building generative AI applications, it’s clear that there are a number of ‘yikes’ aspects to developing fully realized generative AI tools. In this article, I’ll briefly summarise five pitfalls to avoid when developing a generative AI application. Don’t Assume the Outputs of LLMs Will Be Accurate This may seem obvious, but I was surprised at how unreliable the output of commercial LLMs could be. When testing major LLMs (in particular, ChatGPT, Anthropic’s Claude AI, and Amazon Bedrock) on complex financial datasets, I noted that they can generate errors or hallucinations at a rate of one per page. LLMs are guilty of often generating information that sounds plausible but is false. For example, when testing financial data, I found that LLMs produced financial ratios like ‘Non-operating income and expense’. These sound legitimate but don’t exist in the document and are completely hollow financial terms. Though effective prompt engineering can mitigate some of these hallucinations, many of them occur for initially unexplainable reasons (hence why every LLM has a recommendation for manually reviewing its output somewhere in its user interface and documentation). OCR (Optical Character Recognition)—the technology that converts images to machine-readable text—also generates a high rate of errors. It is challenging to develop a mechanism to combat OCR infidelity for each model. AI-based adaptive learning for user-flagged errors can be a helpful strategy but requires huge amounts of manual intervention. Overall, the lack of a calibrated confidence score in commercial LLMs is a huge limitation. Application developers must develop their own methods for calculating confidence and validating errors. 2. Don’t Be Complacent With Your Production Credits Any generative AI dependent on LLM production credits (such as Google’s Tensor Processing Units (TPUs)) needed to be technologically regulated to avoid spending more than accounted for - potentially bankrupting the company or developer. Bankruptcy through overenthusiastic spending of production credits might seem like an urban myth - certainly, there are no noted cases of production credits running amok - but better safe than sorry, right? There are several ways to stretch out your production credits to their fullest potential: Take advantage of short-lived, discounted, preemptible VMs from your cloud application provider (but only if your application is designed to handle interruptions and then resume from checkpoints). Leverage spot instances for non-critical workloads to significantly reduce costs while automatically shutting down idle instances. Cloud provider tools can be very helpful for this. Where appropriate, you could experiment with open-source frameworks that don’t require production credits. I’m very pro the democratic approach of open source models and their potential for collaborative innovation. And, of course, open-source models can be accessed for free. 3. Don’t Forget to Test User Experience (UE) Again, it may seem obvious, but when stuck looking at a profusion of code, it’s easy to forget that a real human being will be using the product. In a production environment, consider the pathways the user will take, even in complex and code-heavy environments. Don’t be like Google Bard, whose generative AI model couldn’t answer simple user questions about space upon release or, later, what to put on pizza (hint: it’s not glue). Many traditional UX testing tools can be adapted for generative AI products - such as a card sorting tool. Another effective way to test the technology is with a human-in-the-loop system, like beta testers or reviewers. At the moment, the company I’m with is using human beta testers to test their new generative AI technology. The testers use the tool for free - and they can harvest the usage patterns. 4. Don’t Forget the Importance of Training Data Clean, normalize, and potentially enrich your data to improve the training process. Techniques like tokenization and feature engineering might be helpful. Generally, the more data, the better (the AI algorithms I used were from a document store of 25 million documents, for example), but too much data may result in overfitting or computational bottlenecks. Slightly off-topic, but in the future, it’s possible that wrangling with training data may not be an issue. Promising advancements like AutoML automatically cleanse training data and assess the model’s performance using a number of techniques like meta-learning, Bayesian inference, and neural architecture search. For SMEs with limited coding resources, AutoML might be a promising innovation. 5. Don’t Choose the Wrong Scale Modeling Strategy There are many ways to scale a model. For example, you might split the model across multiple machines (model parallelism) or replicate the model across multiple machines (data parallelism). Complex models will likely benefit from the model’s parallelism’s device distribution; larger datasets or models with smaller architectures may benefit from increased throughput. Another consideration is whether to upgrade compute resources on a single machine, such as by increasing GPU memory (vertical scaling) or adding more resources (e.g. GPUs) - also known as horizontal scaling. Don’t forget to containerize your generative AI application to ensure consistent behaviors across different environments. Of course, rigorous testing and validation after model scaling is a must. To ensure it can handle the increased load, consider trying a variety of by-the-book testing plus load and stress testing. Summary It’s unrealistic to expect building a generative AI product to work without encountering at least one of these challenges. Ultimately, each of these challenges presents a fork in decision-making. At each juncture, the choice you make gets you one step closer to building a mature, production-ready model. Working with the latest advances in academic and industrial machine learning will help counter some of the typical frustrations, as machine learning’s competitive landscape is constantly pushing out new innovations. I organize the London Machine Learning Meetup (the largest community of AI experts in Europe), which is a free community that hosts events that unpack the latest technical advances in machine learning. Above all, accuracy and cost-effectiveness don’t have to be mutually exclusive. With controlled and strategic experimentation, building a generative AI product can be more rewarding and less frustrating than you might think. Good luck! Introduction Introduction Building generative AI requires working with a variety of developing machine learning and AI technologies. It also requires buckets of patience. These two requirements are not unrelated. Through working alongside extremely talented machine learning experts from different firms building generative AI applications, it’s clear that there are a number of ‘yikes’ aspects to developing fully realized generative AI tools. In this article, I’ll briefly summarise five pitfalls to avoid when developing a generative AI application. Don’t Assume the Outputs of LLMs Will Be Accurate This may seem obvious, but I was surprised at how unreliable the output of commercial LLMs could be. When testing major LLMs (in particular, ChatGPT, Anthropic’s Claude AI, and Amazon Bedrock) on complex financial datasets, I noted that they can generate errors or hallucinations at a rate of one per page. LLMs are guilty of often generating information that sounds plausible but is false. sounds For example, when testing financial data , I found that LLMs produced financial ratios like ‘Non-operating income and expense’. These sound legitimate but don’t exist in the document and are completely hollow financial terms. testing financial data testing financial data Though effective prompt engineering can mitigate some of these hallucinations, many of them occur for initially unexplainable reasons (hence why every LLM has a recommendation for manually reviewing its output somewhere in its user interface and documentation). effective prompt engineering effective prompt engineering OCR (Optical Character Recognition)—the technology that converts images to machine-readable text—also generates a high rate of errors. It is challenging to develop a mechanism to combat OCR infidelity for each model. AI-based adaptive learning for user-flagged errors can be a helpful strategy but requires huge amounts of manual intervention. OCR OCR Overall, the lack of a calibrated confidence score in commercial LLMs is a huge limitation. Application developers must develop their own methods for calculating confidence and validating errors. 2. Don’t Be Complacent With Your Production Credits 2. Don’t Be Complacent With Your Production Credits Any generative AI dependent on LLM production credits (such as Google’s Tensor Processing Units (TPUs)) needed to be technologically regulated to avoid spending more than accounted for - potentially bankrupting the company or developer. Bankruptcy through overenthusiastic spending of production credits might seem like an urban myth - certainly, there are no noted cases of production credits running amok - but better safe than sorry, right? There are several ways to stretch out your production credits to their fullest potential: Take advantage of short-lived, discounted, preemptible VMs from your cloud application provider (but only if your application is designed to handle interruptions and then resume from checkpoints). Take advantage of short-lived, discounted, preemptible VMs from your cloud application provider (but only if your application is designed to handle interruptions and then resume from checkpoints). Leverage spot instances for non-critical workloads to significantly reduce costs while automatically shutting down idle instances. Cloud provider tools can be very helpful for this. Leverage spot instances for non-critical workloads to significantly reduce costs while automatically shutting down idle instances. Cloud provider tools can be very helpful for this. Leverage spot instances Where appropriate, you could experiment with open-source frameworks that don’t require production credits. I’m very pro the democratic approach of open source models and their potential for collaborative innovation. And, of course, open-source models can be accessed for free. Where appropriate, you could experiment with open-source frameworks that don’t require production credits. I’m very pro the democratic approach of open source models and their potential for collaborative innovation. And, of course, open-source models can be accessed for free. open-source frameworks 3. Don’t Forget to Test User Experience (UE) Again, it may seem obvious, but when stuck looking at a profusion of code, it’s easy to forget that a real human being will be using the product. In a production environment, consider the pathways the user will take, even in complex and code-heavy environments. Don’t be like Google Bard, whose generative AI model couldn’t answer simple user questions about space upon release or, later, what to put on pizza (hint: it’s not glue ). simple user questions about space simple user questions about space it’s not glue it’s not glue Many traditional UX testing tools can be adapted for generative AI products - such as a card sorting tool. Another effective way to test the technology is with a human-in-the-loop system, like beta testers or reviewers. At the moment, the company I’m with is using human beta testers to test their new generative AI technology. The testers use the tool for free - and they can harvest the usage patterns. card sorting tool. card sorting tool. 4. Don’t Forget the Importance of Training Data Clean, normalize, and potentially enrich your data to improve the training process. Techniques like tokenization and feature engineering might be helpful. tokenization tokenization Generally, the more data, the better (the AI algorithms I used were from a document store of 25 million documents, for example), but too much data may result in overfitting or computational bottlenecks. Slightly off-topic, but in the future, it’s possible that wrangling with training data may not be an issue. Promising advancements like AutoML automatically cleanse training data and assess the model’s performance using a number of techniques like meta-learning, Bayesian inference, and neural architecture search. For SMEs with limited coding resources, AutoML might be a promising innovation. AutoML AutoML 5. Don’t Choose the Wrong Scale Modeling Strategy There are many ways to scale a model. For example, you might split the model across multiple machines (model parallelism) or replicate the model across multiple machines (data parallelism). Complex models will likely benefit from the model’s parallelism’s device distribution; larger datasets or models with smaller architectures may benefit from increased throughput. Another consideration is whether to upgrade compute resources on a single machine, such as by increasing GPU memory ( vertical scaling ) or adding more resources (e.g. GPUs) - also known as horizontal scaling. Don’t forget to containerize your generative AI application to ensure consistent behaviors across different environments. vertical scaling vertical scaling Of course, rigorous testing and validation after model scaling is a must. To ensure it can handle the increased load, consider trying a variety of by-the-book testing plus load and stress testing. Summary It’s unrealistic to expect building a generative AI product to work without encountering at least one of these challenges. Ultimately, each of these challenges presents a fork in decision-making. At each juncture, the choice you make gets you one step closer to building a mature, production-ready model. Working with the latest advances in academic and industrial machine learning will help counter some of the typical frustrations, as machine learning’s competitive landscape is constantly pushing out new innovations. I organize the London Machine Learning Meetup (the largest community of AI experts in Europe), which is a free community that hosts events that unpack the latest technical advances in machine learning. London Machine Learning Meetup London Machine Learning Meetup Above all, accuracy and cost-effectiveness don’t have to be mutually exclusive. With controlled and strategic experimentation, building a generative AI product can be more rewarding and less frustrating than you might think. Good luck!

Stories, News & Mentions about WRANGLD

Wrangld’s Stories on HackerNoon